How to be notified a website is crawled by Google?

Introduction

It was always a question to me that how to be notified when a search engine robot like google's reaches to a website. On the other hand I was wondering about receiving a notification as soon as google explores my website.

I spent sometime on google and searched the internet to see if I can find a trick or technique to employ it to receive a notification once my website is hit by google itself. The question is not why it is important to know when the website is crawled by google, but the question is how to achieve this goal?!

Code Snippet

First of all, please don't expect that this article should describe an algorithm or a lengthy sample code. This is only a small but useful technique. I ran into the following PHP code snippet in PHP on a weblog somewhere on internet :

if ( strpos( $_SERVER['HTTP_USER_AGENT'], 'Googlebot' ) !== false )
{
// The email address we want to send the email to
$email_address = 'MyName@MyDomain.com';


mail($emai_address,'Google bot','The website is hit by google now.','from@MyDomain.com');


}

In the body of above 'if' statement an email is sent to MyName@MyDomain.com once the website is hit by google. Even we can send a notification to cell phones too if google reaches to a website. All we need is to substitute the code inside the 'if' block with a certain of instructions to trigger a SMS to a cell phone via web as it is described in this weblog.

You must Sign In to use this message board.

FAQ    Noise Tolerance Very HighHighMediumLowVery Low   Layout Expand Posts & RepliesThread ViewNo JavascriptNo JS + Preview   Per page 102550

Msgs 1 to 3 of 3 (Total in Forum: 3) (Refresh) FirstPrevNext

Nice Info stixoffire 20:40 15 Jun '09   Sure this works as long as the User Agent is not being impersonated. I could crawl your site with multiple user agents to determine how they look to each one.. then I would be able to classify what links are valid and what links are perhaps just dummy duplicates. A better way is that Google must have an IP Address that needs to come crawling - knowing the IP Address of the Crawler would work nicely considering it is Unique and surely can not be so easily hacked. Your thoughts? Sign In·View Thread·PermaLink how to achive in asp.net c# cb mehta 1:48 25 Aug '07   useful article can u provide me for asp.net c# thanks in advance Sign In·View Thread·PermaLink 1.00/5 (1 vote)

Last Visit: 12:05 23 Nov '09     Last Update: 12:05 23 Nov '09 1