How search engins search dynamic pages?

Question

0.00/5 (No votes)

See more:

hi ppl !
i have a simple question :D

how search engins can search dynamic pages?
as far as i know a dynamic page is created on the fly when a users sends a request to web server.
(it doesnt exist on the earth until the request isnt made).
so search engin cant send a request to a million of web server.

thanks alot!

Posted 30-Mar-13 6:11am

RedSakura

Add a Solution

Comments

[no name] 30-Mar-13 13:16pm

It's called "scraping".

Sudhakar Shinde 30-Mar-13 23:03pm

Why do not you try search engine itself to find out answer? :)

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Sergey Alexandrovich Kryukov · Answer 1 · 2013-03-30T17:12:00

You point out the real problem and it looks like you correctly understand it.

Following the simple logic, you will see it's simply impossible to search everything that a dynamically generated page can show, no matter what search algorithm does. It's well-known that the results of some code can be even theoretically unpredictable.

In fact, search engines first perform Web scraping and collect data on the scraped site, which is called indexing:
http://en.wikipedia.org/wiki/Search_engine_indexing[^].

No matter how it's done, as some dynamically generated page can generate totally unique content every time it is requested, no indexing can predict relevance of any request to such content.

—SA