Click here to Skip to main content
12,893,107 members (58,809 online)
Click here to Skip to main content

Tagged as


5 bookmarked
Posted 16 Jan 2013

Why a robots.txt file is no longer sufficient

, 16 Jan 2013 CPOL
Over the past year Google decided to partially ignore the robots.txt file in an attempt to increase the effectiveness of its results.



Compare Revision Minor Date Status Editor
1 - publicly available No 16-Jan-13 6:35 Available Smitha Nishant
Updates in content. 11 changes had been made.


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

You may also be interested in...

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web02 | 2.8.170424.1 | Last Updated 16 Jan 2013
Article Copyright 2013 by Zac Gery
Everything else Copyright © CodeProject, 1999-2017
Layout: fixed | fluid