Click here to Skip to main content
15,891,473 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
(python 3.5)
What I am trying to do is scrape the social club site for public statistics but I can't get the login down.

I have looked for other posts with answers but nothing quite fits the bill. I don't want to use selenium or mechanize or any browser. I am trying to do this with requests and beautifulsoup. This is my first post so please also let me know if I am doing anything incorrectly/wrong.

My code compiles fine no syntax errors or anything like that. It just won't login. I have found all the post information and I get the token but I just can't get it to login. The login always fails. I know this is probably pretty basic stuff so if someone could help me out that would be great! Thanks. If i had to guess I would say it still has something to do with the authentication token in the post.

This is a test account i made so don't worry about me showing the password ect. It's fine

This is the post data when you sign in
login=smittyofwar@mail.com
password=Harambe1
__RequestVerificationToken=5xNBfVT5mZcHNd8e8Bt2pFqzPLxc36QYASAo3koVjDh-wBTJLT6rD5Yaq39mPbq_Q5vDrUcOZ-E0MzAzXrgW7qFRy-Y1

Python
import requests
from bs4 import BeautifulSoup

s = requests.session()
login_url = "http://socialclub.rockstargames.com/profile/signin"

#get the token needed to log in
tmp=s.get("http://socialclub.rockstargames.com")

findtoken = BeautifulSoup(tmp.content, "lxml")
token=findtoken.find("__RequestVerificationToken")
#__________________________________

username = 'smittyofwar@mail.com'
passwrd='Harambe1'

data = dict(login=username, password=passwrd,__RequestVerificationToken=token, next='/')

s.post(login_url, data=data,headers=dict(Referer="http://socialclub.rockstargames.com"))



r = s.get("http://socialclub.rockstargames.com")
#r = s.get("https://socialclub.rockstargames.com/member/cjg320/games/gtav/career/overview/gtaonline")

soup = BeautifulSoup(r.content, "lxml")
soup = soup.prettify()


with open('thesauce.txt', 'w', encoding='utf-8') as f:
    for line in soup:
        f.write(str(line))


What I have tried:

I have looked over countless other questions and researched through google. Also tried different code for hours.
Posted
Comments
Patrice T 27-Sep-16 12:07pm    
Never post real credential data in public forum.
Member 12762376 27-Sep-16 12:12pm    
It's just a test account. Absolutely nothing could come from it.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900