How to login to website using Python-request module

See more:

Python

HTTPS

webscraping

I want to reproduce the process of logging in to bestbuy.ca entirely through the python-requests module, but from all my attempts I've gotten http 4XX client-side errors (403 with the code below).

Also, when I compare the http request headers from the browser versus the one made in python, there seem to be headers absent (referer, sec-ch-ua, and others). When I try to add the referer to the session header the post request ends up hanging.

This is my code so far:

import requests
loginUrl = 'https://www.bestbuy.ca/profile/signin.aspx'
otherUrl = 'https://www.bestbuy.ca/identity/login'

userInfo = {'username': 'myemail', 'password': 'mypass', 'captchaResponse': '?', 'tid': 'tid'}
s = requests.Session()
s.headers['User-Agent'] = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.82 Safari/537.36'
r = s.get(loginUrl)
s.headers['content-type'] = 'text/plain;charset=UTF-8'
s.headers['accept'] = 'application/vnd.bestbuy+json'
r = s.post(otherUrl, data=userInfo)
print(r)

What I have tried:

I tried changing the 'otherUrl' field to add garbage at the beginning/end to see if I still got the error to rule out the possibility the link was garbage - I either get a 404 or 401 error when I do this rather than the 403.

Next I tried comparing the request headers between the code and browser and found some missing headers. Adding some of these headers causes the POST request to hang.

Finally I tried entering garbage data for the payload and found I still get the 403 error request, even when the data format is completely off.

I'm completely at a loss right now and would appreciate any help, thank you!