Click here to Skip to main content
15,881,600 members
Articles / Google

Why a robots.txt file is no longer sufficient

Rate me:
Please Sign up or sign in to vote.
0.00/5 (No votes)
16 Jan 2013CPOL2 min read 8.6K   5  
Over the past year Google decided to partially ignore the robots.txt file in an attempt to increase the effectiveness of its results.

Alternatives

Members may post updates or alternatives to this current article in order to show different approaches or add new features.

No alternatives have been posted.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



Comments and Discussions