6,630,586 members and growing! (16,037 online)
Email
Password
Remember me?
Lost your password?
Home
Articles
Search
Latest Articles
Top Articles
Beginner Articles
Technical Blogs
Post an Article
Post Tip/Trick
Post your Blog
Posting/Update Guidelines
Article Competition
Topic List
Quick Answers
Ask a Question
Search for an Answer
Latest Questions
Unanswered Questions
Top Questions
Help! (our FAQ)
Message Boards
ASP.NET
ATL / WTL / STL
C / C++ / MFC
Managed C++/CLI
C#
COM
Hardware & Devices
LINQ
.NET Framework
System Admin
Silverlight
General Database
Sharepoint
Visual Basic
Web Development
WPF / WCF / WF
XML / XSL
General IT Issues
Site Bugs / Suggestions
The Soapbox 2.0
All Message Boards...
Job Board
Latest
Search
Post a Job
FAQ and Pricing
Catalog
Latest
Search
Post a Catalog Item
FAQ and Pricing
Help!
What is 'The Code Project'?
General FAQ
Post a Question
Site Directory
About Us
Lounge
Soapbox
Announcements
Windows 7 Comp
Win a laptop!
Monthly Competition
Articles
Desktop Development
Button Controls
Clipboard
Combo & List Boxes
Dialogs and Windows
Desktop Gadgets
Document / View
Edit Controls
Files and Folders
Grid & Data Controls
List Controls
Menus
Miscellaneous
Printing
Progress Controls
Selection Controls
Shell and IE programming
Smart Client
Splitter Windows
Static & Panel Controls
Status Bar
Tabs & Property Pages
Toolbars & Docking windows
Tree Controls
Web Development
Ajax and Atlas
Applications & Tools
ASP
ASP.NET
ASP.NET Controls
ATL Server
Caching
Charts, Graphs and Images
Client side scripting
Custom Controls
HTML / CSS
ISAPI
Site & Server Management
Session State
Silverlight
Trace and Logs
User Controls
Validation
View State
WAP / WML
Web Security
Web Services
Enterprise Systems
Content Management Server
Microsoft BizTalk Server
Microsoft Exchange
Office Development
SharePoint Server
Multimedia
Audio and Video
DirectX
GDI
GDI+
General Graphics
OpenGL
Database
Database
SQL Reporting Services
Platforms, Frameworks & Libraries
ATL
MFC
STL
WTL
COM / COM+
.NET Framework
Win32/64 SDK & OS
Vista API
Vista Security
Cross Platform
Game Development
Mobile Development
Windows CardSpace
Windows Communication Foundation
Windows Presentation Foundation
Windows Workflow Foundation
Libraries
Windows Powershell
LINQ
Languages
C / C++ Language
C++ / CLI
C#
MSIL
VBScript
VB.NET
VB6 Interop
Other .NET Languages
XML
Java
General Programming
Algorithms & Recipes
Bugs & Workarounds
Collections
Cryptography & Security
Date and Time
DLLs & Assemblies
Exception Handling
Localisation
Macros and Add-ins
Programming Tips
String handling
Internet / Network
Threads, Processes & IPC
WinHelp / HTMLHelp
Uncategorised Quick Answers
Graphics / Design
Expression
Usability
Development Lifecycle
Debug Tips
Design and Architecture
Installation
Work Issues
Testing and QA
Code Generation
General Reading
Book Chapters
Book Reviews
Hardware Reviews
Interviews
Scrapbook
Hardware & System
Uncategorised Technical Blogs
Author Resources
Third Party Products
Product Showcase
Solution Center
Mentor Resources
Services
Product Catalog
Code-signing Certificates
Job Board
CodeProject VS2008 Addin
Feature Zones
Product Showcase
Code Signing Resources
WhitePapers / Webcasts
ASP.NET Web Hosting
News & Community
Site Map
Who's Who @ CodeProject
CodeProject MVPs
Competitions
Company Listings
Latest Articles
News
Newsletter Archive
Press Releases
Surveys
Submit an Article
Stuff from CodeProject
The Code Project
About Us
Our Awards
Media kit & Advertising info
Legal Stuff
License Info
Search
Articles / Quick Answers
Messages
Jobs
Product Catalog
Add to IE Search
Articles by David Cruwys (1 article found)
Average article rating: 4.11
Articles
Technical Blogs
Tips
Internet / Network
HTTP
C#/VB - Automated WebSpider / WebRobot
Last Updated: 15 Mar 2004 Page Views: 88,609 Rating: 4.11/5 Votes: 29 Popularity: 6.01
Bookmark Count: 135
Build a flexible WebRobot and process an entire site using a WebSpider
David Cruwys
Member
I have been programming commercially since 1990, with the last 4 years spent mainly in Java. I made the transition to .NET six months ago and have not looked back. I have written e-commerce solutions, desktop and mobile phone applications in a variety of languages (VB6, Delphi, Java, Foxpro, Clipper 87 etc...) and am currently developing a Web Application Framework in C#.
I have just launched
www.offyourbutt.com
for showcasing my products and services and this will become a test bed for my C# framework. Click here to see my full
profile
Occupation:
Web Developer
Location:
Australia
Last Updated 21 Nov 2009
Web18 | 2.2.0175 |
Advertise
|
Privacy
Copyright ©
CodeProject
, 1999-2009
All Rights Reserved.
Terms of Use
