Click here to Skip to main content
14,924,395 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
I want to reproduce the process of logging in to bestbuy.ca entirely through the python-requests module, but from all my attempts I've gotten http 4XX client-side errors (403 with the code below).

Also, when I compare the http request headers from the browser versus the one made in python, there seem to be headers absent (referer, sec-ch-ua, and others). When I try to add the referer to the session header the post request ends up hanging.

This is my code so far:

Python
import requests
loginUrl = 'https://www.bestbuy.ca/profile/signin.aspx'
otherUrl = 'https://www.bestbuy.ca/identity/login'

userInfo = {'username': 'myemail', 'password': 'mypass', 'captchaResponse': '?', 'tid': 'tid'}
s = requests.Session()
s.headers['User-Agent'] = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.82 Safari/537.36'
r = s.get(loginUrl)
s.headers['content-type'] = 'text/plain;charset=UTF-8'
s.headers['accept'] = 'application/vnd.bestbuy+json'
r = s.post(otherUrl, data=userInfo)
print(r)


What I have tried:

I tried changing the 'otherUrl' field to add garbage at the beginning/end to see if I still got the error to rule out the possibility the link was garbage - I either get a 404 or 401 error when I do this rather than the 403.

Next I tried comparing the request headers between the code and browser and found some missing headers. Adding some of these headers causes the POST request to hang.

Finally I tried entering garbage data for the payload and found I still get the 403 error request, even when the data format is completely off.

I'm completely at a loss right now and would appreciate any help, thank you!
Posted
Comments
Gerry Schmitz 17-Mar-21 18:03pm
   
Maybe ask a best buy "employee". If you can scrape other sites, obviously best buy is on to you.
Ray987 17-Mar-21 18:23pm
   
The initial get request is successful and I'm also able to login to best buy no problem on my regular web-browser so I don't think I've been flagged.

The only payload field I'm not sure of is the captchaResponse' field where I don't have a value. I would think this is what's causing the error, but I'm not entirely sure whether my header fields, formatting, or login url-link are all correct.

I'm not even sure where to find the captchaResponse field
Richard MacCutchan 18-Mar-21 5:23am
   
As you say you are not sure where to find a field. The issue relates to the page returned from the website and that is where you need to look to find the solution. It is not a programming problem per se, but more to do with interaction with the web site.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)




CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900