Hi. I have very big problem please help.
I'm scrapping website and the page don't have pagination. If you like to go to the next page you need to click on form button or select from the select option menu. With webbrowser control I'm clicking on the button or the select option like this:
HtmlElement reportDropDown = webBrowser1.Document.GetElementById("ctl00_ContentPlaceHolderProductDisplay_wuc_product_display_ddlTopPage");
reportDropDown.SetAttribute("value", Convert.ToString(c));
reportDropDown.InvokeMember("onchange");
The problem is: when I scrap the first page, and the button is clicked, after clicking the button the page have some little animation with loading on the same page, while go to the next page. When the button is clicked and the loading animation is start, my code in the while loop don't wait, so start scrapping again the same results from the first page.
I've tried webbrowser1.readystate, isbusy or similar, they not working for button!!!!
I need waiting mechanism when button is clicked!!!
little demo of my while loop:
while(i <= 10) // number of pages
{
while(...)
{
//here i'm scraping data for the current page
}
// here goes button click
++i;
// here i need some mechanism for waiting
}