Headline |
Type |
Source |
Date |
Clicks |
|
Tips and Tools |
CodeClimber |
25 Apr 2017 |
1,271 |
Robots.txt meant for search engines don’t work well for web archives
A few months ago we stopped referring to robots.txt files on U.S. government and military web sites for both crawling and displaying web pages (though we respond to removal requests sent to info@archive.org). As we have moved towards broader access it has not caused problems, which we take as a good sign. We are now looking to do this more broadly.
|
Tips and Tools |
Archive.org |
25 Apr 2017 |
244 |
|
Satire |
CommitStrip |
25 Apr 2017 |
1,480 |
|
Science And Technology |
ScienceDaily |
25 Apr 2017 |
3,692 |
|
Science And Technology |
Phys.org |
25 Apr 2017 |
2,748 |
|
Science And Technology |
BBC |
25 Apr 2017 |
2,156 |
|
Industry News |
Ars Technica |
24 Apr 2017 |
4,642 |
|
Industry News |
Windows Central |
24 Apr 2017 |
3,418 |
|
Industry News |
Techcrunch |
24 Apr 2017 |
1,707 |
|
Industry News |
Engadget |
24 Apr 2017 |
3,515 |
|
Industry News |
ZDNet |
24 Apr 2017 |
4,824 |
A mind is born
I don't think I could save a file with 256 bytes, let alone get it to do anything
|
Developer News |
Linus Sakesson |
24 Apr 2017 |
3,024 |
|
Developer News |
The Verge |
24 Apr 2017 |
3,139 |
|
Developer News |
Infoworld |
24 Apr 2017 |
2,017 |
|
Developer News |
Bleeping Computer |
24 Apr 2017 |
2,160 |
|
Developer News |
Telerik |
24 Apr 2017 |
6,633 |
|
Hot Threads |
CodeProject |
24 Apr 2017 |
1,974 |
|
Hot Threads |
CodeProject |
24 Apr 2017 |
1,894 |
|
Satire |
DevOps |
24 Apr 2017 |
1,686 |
|
Science And Technology |
Ars Technica |
24 Apr 2017 |
2,346 |
|
Science And Technology |
Infoworld |
24 Apr 2017 |
1,267 |
|
Science And Technology |
Phys.org |
24 Apr 2017 |
3,022 |
|
Industry News |
Ars Technica |
21 Apr 2017 |
4,736 |
|
Industry News |
The Verge |
21 Apr 2017 |
1,356 |