Click here to Skip to main content
15,846,038 members
Please Sign up or sign in to vote.
5.00/5 (1 vote)
See more:
The purpose is to eventually develop an extension for IE which is capable of detecting all search results URL's when searching for any topic on Google, and save these URL's in a text file.

The main problem I am facing is that all search results on Google are not visible, if you try inspecting the source code of a Google search result page, you'll find that their pages were built dynamically and therefore, there are no links available on the pages, just a bunch of dynamic java scripting.

How can I retrieve the true URL links using the extension?

I managed to reach a solution to this problem.

My problem was that i thought i couldn't access the URL's on the Google search page. You won't be able to view them if you try viewing the PAGE SOURCE in IE however, the URL's are not stored in the PAGE SOURCE anyway since that's only the MAIN HTML SOURCE Page. The URL search list is actually built in dynamically and its available in the pages' DOM.

What you'll need to do is capture the URL's by catching them in one of the DISPID_DOM**** Events. The DOM events are fired only when the DOM is completely loaded, unlike the DISPID_DOCUMENTCOMPLETE event which is fired when the Source HTML is loaded.

The URL's are nested in the DOM in this way:

<div id="ires">
       <a class="1" href="URL">

What you'll need to do is Search for the "div" Tag with an "id=ires" value using the get_id(&idStr) method. Then traverse every node within this tag till you find the "href" you are looking for.
Share this answer
H.Brydon 4-Feb-13 12:59pm    
Great research. +5 for question and answer.
You can use Google API to get search results. And search word could be retrieved from URI.
Share this answer
Daroosh 1-Feb-13 8:20am    
Which Google API? Can you offer me some Source code to further elaborate your solution.
Kurkul 1-Feb-13 8:29am    
Please look at to get result in xml

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900