Click here to Skip to main content
15,887,683 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
Hi,

Thanks for help.

I am new to screen scraping and working on an asp .Net page. On the page, there is pagination. I can scrap first page but can't next ones. Pagination is based on __doPostBack function. I get the value for for __VIEWSTATE using this code:

XML
$file =file_get_contents($url);
preg_match("#<input.*?name=\"__viewstate\".*?value=\"(.*?)\".*?>.*?"
  ."<input.*?name=\"__eventvalidation\".*?value=\"(.*?)\".*?>#mis",
  $file,
  $arr_viewstate);
$viewstate = urlencode($arr_viewstate[1]);
$eventvalidation = urlencode($arr_viewstate[2]);


and Sending post request in curl using

VB
CURLOPT_POSTFIELDS =>
'__EVENTTARGET='
.urlencode('ctl00$ContentPlaceHolderBody$SearchPageNavigationTop$rptPager$ctl01')
.'&__EVENTARGUMENT='
.urlencode('')
.'&__VIEWSTATE='
.$viewstate
.'&__EVENTVALIDATION='
.$eventvalidation
.'&__LASTFOCUS='
.urlencode(''));


because
SQL
ctl00$ContentPlaceHolderBody$SearchPageNavigationTop$rptPager$ctl01


value is passed to __doPostBack function when we click on page 2.
But its not working and giving content of 1st page even i pass value for 2nd or other pages.

So please guide me.

Thanks again.
Posted
Comments
fjdiewornncalwe 17-Oct-12 16:29pm    
Why exactly do you need to scrape someone else's page?
Sergey Alexandrovich Kryukov 17-Oct-12 16:42pm    
Good question.
--SA
vivekmahajan86 18-Oct-12 11:49am    
There are only 30 products on 1st page, but in category there are more than 300 products. So i want to scrap all products.
fjdiewornncalwe 18-Oct-12 12:09pm    
That doesn't really answer my question that is why are you scraping someone else's page. Why don't you have access to just pull the data directly?
vivekmahajan86 18-Oct-12 14:21pm    
How can i pull the data directly?

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900