Suggestions have been incorporated. Kindly suggest, vote, comment to improve it.
Step by Step guide for scraping can be seen Here
The Facebook graph API and many other Facebook C# SDKs allow the user to do Facebook tasks from the code. But all need to have an application key of a Facebook app, requires OATH Permission and keeping record of session tokens. I thought of doing some of the basic tasks like Login, Search, Status update, from the C# code with the help of HTTP GET and POST. For this purpose, I am using the mobile version of the Facebook website. The
Response is obtained by
Response and HTML is parsed for the required data by Regex. The
Response where required is posted as HTTP
In my precious article Multi-Threaded WebScraping in CSharpDotNetTech, I explained web scraping techniques using Regex,
Response. Based on techniques explained in that article, the attached piece of code has been written. The purpose of the article is to explain
- how to HTTPWebRequest and HTTPWebReponse
- How to GET and POST data to the websrever
- How to collect Cookies
- How to Maintain Session
public static Facebook Login(string username, string
Performs Login for the given
username and password, Stores the Successful Login Cookies in
On Success returns the Facebook Object,
by which further public methods can be called
On Failure, Shows the Response
HTML in Default Browser
public bool StatusUpdate(string txt)
Updates the Text Status on User’s
public void UploadPhoto(string filepath, string caption)
Uploads the Picture from given
path to the mobile uploads folder of the current user profile with specified
public Dictionary<string, string>
SearchGroups(string keyword, int scount)
Performs Search for Groups for the
given keyword, and returns Dictionary of GroupID and Name from the specified
start count of search result
public Dictionary<string, string>
SearchPages(string keyword, int scount)
Performs Search for Pages for the
given keyword and returns Dictionary of URLs and Name of Pages from the
specified start count of the search result
Facebook fb = Facebook.Login("USERNAME", "PASSWORD");
fb.UploadPhoto(@"C:\Users\..\...\...\IMAGE.jpg", "Its the Caption");
Searching For Groups
Dictionary<string, string> groups;
int startCount = 1; do
groups = fb.SearchGroups("diet coke", startCount);
startCount += groups.Count;
string txt = "";
foreach (string k in groups.Keys)
txt += k + "\t\t" + groups[k] + Environment.NewLine;
} while (groups.Count > 0);
The same way you can Search for Pages.
How the Code Works?
Before doing login by C#, let's perform login in Mozilla and
analyze the HTTP Header by LiveHTTPHeaders. You can install the LiveHTTPHeader Adon from the Firefox site. Start LiveHTTPHeader, browse
to http://www.facebook.com, enter username and password, click Login button. The HTTP Web Request sent by the Browser will look something like this:
The First Line is the URL to which your username and password are
being sent (later in Example 3, we will see how to find this URL). Second
line tells the HTTP method and version used, which is POST and 1.1
Then all the fields are just like normal HTTP Header
as we saw in Example 1. The important stuff starts from Cookie Header,
in Example 1, once we browse to http://www.facebook.com,
there was no Cookie Header where as we received some Cookies in the
Response Header, now when we click on the Login Button, the previously
received set of Cookies is being sent in this Cookie Header.
Next Header shows Content Type, there are two major
content types used to POST data, application/x-www-form-urlencoded and
multipart/form-data. You can find more information about these here .
Next Header shows Content Length and in last line Content is being
shown. You will see your email address and password in this line.
Actually, the last line shows the data which is being sent to the server by
There are several other values also, later in the example,
we will see what these values are and from where to obtain these values!!!
Let's examine the Response Header for the above Request.
The Response Header shows a lot of Cookies, these
are the Cookies which are issued by the server on successful login,
now for any subsequect request, the browser will send these Cookies to
the server and in this way session will be maintained.
Got to Tools->Clear Recent History and delete the
Cookies, then try to browse to your Facebook profile page, and you will
see that you will be redirected to the Facebook login page.
Now let's create the same login Request header as we saw in the above screenshot and test that either we are able to successfully log in or not.
string getUrl = "https://www.facebook.com/login.php?login_attempt=1";
string postData = "lsd=AVo_jqIy&email=YourEmailAddress
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Firefox 18.104.22.168";
getRequest.CookieContainer = new CookieContainer();
getRequest.Method = WebRequestMethods.Http.Post;
getRequest.ProtocolVersion = HttpVersion.Version11;
getRequest.AllowAutoRedirect = false;
getRequest.ContentType = "application/x-www-form-urlencoded";
getRequest.Referer = "https://www.facebook.com";
getRequest.KeepAlive = true;
getUrl is assigned to the address to which data will be
postData variable is copy of the Content from above HTTP Request
Packet. Then we have created an
HTTPWebRequest Object, and set its
The Cookies which we received in Response to the Request for http://www.facebook.com
are added to the
HTTPWebRequest object, if we don't add these Cookies,
then instead of entertaining our request for login, Server will
redirect us to Login page. Next we are setting HTTP Method to Post and
Version to 1.1 (used for HTTPS).
AllowAutoRedirect property to
requests in which we try to login is very important, if this property
is set to
true, then the
HTTPWebRequest object will follow the
Redirection Responses. And
during the redirections, you may lose access to the Cookies which server sent in response to Login Request.
Now let's send the Login Info to the Server.
byte byteArray = Encoding.ASCII.GetBytes(postData);
getRequest.ContentLength = byteArray.Length;
Stream newStream = getRequest.GetRequestStream();
newStream.Write(byteArray, 0, byteArray.Length);
- Data is written to stream, now let's get the Response and see what all Cookies we receive:
We successfully logged into the system and
received 9 Cookies, the snapshot above shows very little info about the
received Cookies, you can get more info by accessing the properties of
the Cookies. Add the received Cookies to globally defined CookieCollection so that it can be used in subsequent requests. How to Check Login was Successful or Not?
Normally Cookies Count is an easy way to determine whether Login was
successfully or not, to be more sure, you can try getting HTML of Home
Page, if you are not redirected to Login Page, that means you are successfully
This way, you can inspect post data for each task and write Regex to find that data in the HTML of the page and subsequent post data request can be generated. Most of the functions above use this mechanism.
Points of Interest
Although these are very basic functions, but with understanding of these, code can be enhanced to perform more advanced functions like joining the groups, liking the page, getting stories from timeline, posting images to groups and pages, commenting on posts, sending friend requests, etc.
Note: The Facebook class uses Mozilla Firefox useragent, you shall replace it with user agent of your browser which you use to browse Facebook.Use of code for any SPAM activity may get your account blocked.