Click here to Skip to main content
14,733,441 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
hi ppl !
i have a simple question :D

how search engins can search dynamic pages?
as far as i know a dynamic page is created on the fly when a users sends a request to web server.
(it doesnt exist on the earth until the request isnt made).
so search engin cant send a request to a million of web server.

thanks alot!
[no name] 30-Mar-13 13:16pm
It's called "scraping".
Sudhakar Shinde 30-Mar-13 23:03pm
Why do not you try search engine itself to find out answer? :)

1 solution

You point out the real problem and it looks like you correctly understand it.

Following the simple logic, you will see it's simply impossible to search everything that a dynamically generated page can show, no matter what search algorithm does. It's well-known that the results of some code can be even theoretically unpredictable.

In fact, search engines first perform Web scraping and collect data on the scraped site, which is called indexing:[^].

No matter how it's done, as some dynamically generated page can generate totally unique content every time it is requested, no indexing can predict relevance of any request to such content.


This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900