Click here to Skip to main content
15,399,440 members
Please Sign up or sign in to vote.
1.00/5 (1 vote)
See more:
this code is executing some some times not.... please help me

Here is my code....

C#
private void searchAndParse(string searchStr)
{
Dictionary<string, string> queryPara;
Dictionary<string, string> headerDict;
string curGoogleSearchUrl;
string curSearchRespHtml;
string curReferer;

string googleUrl = "http://www.google.co.in/";
string tmpRespHtml = crifanLib.getUrlRespHtml(googleUrl);

string firstGoogleSearchUrl = "http://www.google.co.in/search?";
queryPara = new Dictionary<string, string>();
queryPara.Add("newwindow", "1");
queryPara.Add("safe", "strict");
queryPara.Add("source", "hp");
queryPara.Add("q", searchStr);
queryPara.Add("btnK", "Google Search");

queryPara.Add("site", "");
firstGoogleSearchUrl += crifanLib.quoteParas(queryPara, false);

curReferer = googleUrl;
curGoogleSearchUrl = firstGoogleSearchUrl;

while(needContinueSearch)
{
headerDict = new Dictionary<string,string>();
headerDict.Add("referer", curReferer);
curSearchRespHtml = crifanLib.getUrlRespHtml(curGoogleSearchUrl, headerDict: headerDict);

parseSinglePageHtml(curSearchRespHtml);
if (!needContinueSearch)
{
break;
}

HtmlAgilityPack.HtmlDocument htmlDoc = crifanLib.htmlToHtmlDoc(curSearchRespHtml);
HtmlNode rootHtmlNode = htmlDoc.DocumentNode;
HtmlNode nextHtmlNode = rootHtmlNode.SelectSingleNode("//a[@id='pnnext' and @class='pn']");
if(nextHtmlNode != null)
{
curReferer = curGoogleSearchUrl;

string hrefStr = nextHtmlNode.Attributes["href"].Value;
string encodedUrl = "http://www.google.co.in" + hrefStr;

string htmlDecoded = HttpUtility.HtmlDecode(encodedUrl);

curGoogleSearchUrl = htmlDecoded;

needContinueSearch = true;
}
else
{
needContinueSearch = false;
break;
}

}
}
Posted
Comments
Peter Leow 22-Jan-14 3:05am
   
Not clear.
Tomas Takac 22-Jan-14 3:47am
   
What part of the method is not executing? Did you debug it?
rameshk14 22-Jan-14 4:05am
   
hi tomas takac

yes, i did some times its executing, but after some time (10 to 15 minutes)not working... and i am getting null value in nextHtmlNode
tgrt 22-Jan-14 16:50pm
   
That doesn't necessarily mean there's a problem with your code. The variable nextHtmlNode is setup to be null if the node "//a[@id='pnnext' and @class='pn']" isn't found.
rameshk14 23-Jan-14 3:27am
   
yes, your correct The variable nextHtmlNode is setup to be null if the node "//a[@id='pnnext' and @class='pn']" isn't found.

but

my work is getting the URLs from google search page... here i am getting google search response. some times not getting urls
Tomas Takac 22-Jan-14 17:24pm
   
I would agree with tgrt that nextHtmlNode==null is a feature not a bug. It looks for a specific link in the response and if that's not found it ends the search. You didn't really help us to understand the problem. What takes 10-15 minutes? Getting response from google search? I doubt it. Parsing the results? Not probable. Is that the whole search i.e. several iterations of the while loop? Now why do you think there is an error? What are the expected results? What are you getting instead?
rameshk14 23-Jan-14 2:35am
   
yes i am getting search response in 10-15 minutes also but i am try to get Urls and listing out into datagrid. some times datadrid is showing empty....

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

  Print Answers RSS
Top Experts
Last 24hrsThis month


CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900