Click here to Skip to main content
Rate this: bad
good
Please Sign up or sign in to vote.
See more: C++ BHO
The purpose is to eventually develop an extension for IE which is capable of detecting all search results URL's when searching for any topic on Google, and save these URL's in a text file.
 
The main problem I am facing is that all search results on Google are not visible, if you try inspecting the source code of a Google search result page, you'll find that their pages were built dynamically and therefore, there are no links available on the pages, just a bunch of dynamic java scripting.
 
How can I retrieve the true URL links using the extension?
Posted 1-Feb-13 0:07am
Daroosh657
Rate this: bad
good
Please Sign up or sign in to vote.

Solution 3

I managed to reach a solution to this problem.
 
My problem was that i thought i couldn't access the URL's on the Google search page. You won't be able to view them if you try viewing the PAGE SOURCE in IE however, the URL's are not stored in the PAGE SOURCE anyway since that's only the MAIN HTML SOURCE Page. The URL search list is actually built in dynamically and its available in the pages' DOM.
 
What you'll need to do is capture the URL's by catching them in one of the DISPID_DOM**** Events. The DOM events are fired only when the DOM is completely loaded, unlike the DISPID_DOCUMENTCOMPLETE event which is fired when the Source HTML is loaded.
 
The URL's are nested in the DOM in this way:
 
<div id="ires">
       <a class="1" href="URL">
      </a>
</div>
 
What you'll need to do is Search for the "div" Tag with an "id=ires" value using the get_id(&idStr) method. Then traverse every node within this tag till you find the "href" you are looking for.
  Permalink  
Comments
H.Brydon at 4-Feb-13 12:59pm
   
Great research. +5 for question and answer.
Rate this: bad
good
Please Sign up or sign in to vote.

Solution 1

You can use Google API to get search results. And search word could be retrieved from URI.
  Permalink  
Comments
Daroosh at 1-Feb-13 8:20am
   
Which Google API? Can you offer me some Source code to further elaborate your solution.
Kurkul at 1-Feb-13 8:29am
   
Please look at https://developers.google.com/custom-search/docs/xml_results?hl=en#wsSampleQueries to get result in xml

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

  Print Answers RSS
0 OriginalGriff 210
1 Richard MacCutchan 100
2 kbrandwijk 90
3 ProgramFOX 80
4 Mukesh Bhagat 65
0 Sergey Alexandrovich Kryukov 9,050
1 OriginalGriff 8,151
2 CPallini 2,613
3 Richard MacCutchan 2,221
4 Abhinav S 1,928


Advertise | Privacy | Mobile
Web01 | 2.8.140827.1 | Last Updated 4 Feb 2013
Copyright © CodeProject, 1999-2014
All Rights Reserved. Terms of Service
Layout: fixed | fluid

CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100