Don't try to do any client activity except sending HTTP request and getting HTTP response. The idea is: you should sent the same requests that the browser would do, not doing all the client-site activity. You can use the class
System.Net.HttpWebRequest
:
http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest%28v=vs.110%29.aspx[
^].
How to know what exactly to send? There can be different approaches. In simple cases, you can learn what the Web page does by looking at its code using, say, browser's "View Page Source" feature (or something like that, depending on your browser). You may need to download related parts of the client-side code, such as scripts.
I have a different advice. You can get any suitable HTTP spying software. In particular, there is a number of products implemented as plug-in modules to different browsers. Use one. I, for example, use HttpFox, a plug-in to Mozilla browser. Activate HTTP trace and do all manipulations in your browser as the user would do. By captured HTTP requests and responses, you can figure out how to scrape the data you need. See also:
http://en.wikipedia.org/wiki/Web_scraping[
^].
See also my past answers:
http://www.codeproject.com/Answers/285975/get-specific-data-from-web-page#answer2[
^],
How to get the data from another site[
^].
—SA