Webrowser control c#c

Question

0.00/5 (No votes)

See more:

Hi. I'm scrapping some data from some website which have a pagination. I traverse through all pages in the pagination, but when some page is load from the pagination the page is full loaded but the code is waiting a little more before my code start scraping the data.

My question is: Is there is some better implementation then this one below, when the page is completely loaded and quickly after that to start the scrapping?

C#

webBrowser1.Navigate("javascript:changeCar('"+br+"')");
while (webBrowser1.ReadyState != WebBrowserReadyState.Complete)
{
    Application.DoEvents();
}

The problem is with Application.DoEvents() which make the code to wait....

Thanks, Cheers

Posted 9-Sep-13 10:29am

Ensiferum77

Add a Solution

3 solutions

Solution 3

@Espen I will put DocumentCompleted inside my code, because sometimes the app with Application.DoEvents() broke up.

@Sergey I also use httpwebrequest sometimes, but I'm a little more comfortable when use the internet explorer wrapper, and the speed of my app here is not a problem. :)

Thanks both. New ideas here from everyone are welcome.

Posted 9-Sep-13 10:45am

Ensiferum77

Comments

Sergey Alexandrovich Kryukov 9-Sep-13 16:57pm

First of all, this is not an answer, should not be here.
IE wrapper could not speed up anything, but it will be much, much slower, and it's just pointless. Isn't it obvious? If you need to scrape, scrape it. You really need to use HttpWebRequest.
—SA

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Sergey Alexandrovich Kryukov · Accepted Answer · 2013-09-09T10:37:00

Solution 2

For Web scraping, you should rather use the class System.Net.HttpWebRequest . Well, it is actually millions times better, as a Web browser is majorly irrelevant to the problem.

Please see my past answers:
How to get the data from another site[^],
get specific data from web page[^].

—SA

Posted 9-Sep-13 10:37am

Sergey Alexandrovich Kryukov

Comments

Joezer BH 10-Sep-13 3:29am

5ed!

Sergey Alexandrovich Kryukov 10-Sep-13 10:39am

Thank you.
—SA

Espen Harlinn · Accepted Answer · 2013-09-09T10:34:00

Solution 1

You would be better off using the WebBrowser.DocumentCompleted[^] event - then you would not need to perform any loop while waiting for the page to load.

Best regards
Espen Harlinn

Posted 9-Sep-13 10:34am

Espen Harlinn

Comments

Sergey Alexandrovich Kryukov 9-Sep-13 16:55pm

As OP mentioned the purpose — Web scraping, using a Web browser is in general pointless, HttpWebRequest should be used; please see my answer.
—SA