Click here to Skip to main content
15,895,746 members
Articles / Programming Languages / Visual Basic

C#/VB - Automated WebSpider / WebRobot

Rate me:
Please Sign up or sign in to vote.
4.66/5 (35 votes)
15 Mar 20045 min read 144.4K   2.8K   170  
Build a flexible WebRobot and process an entire site using a WebSpider
Imports System
Imports System.Net

Imports Mf.Util

Namespace Mf.Service.WebSpider

   Public Class WebPageState

      Private Sub New()
      End Sub

      Public Sub New(ByVal uri As Uri)
         m_uri = uri
      End Sub

      Public Sub New(ByVal uri As String)
         MyClass.New(New Uri(uri))
      End Sub

#Region "properties"
      Private m_uri As Uri
      Private m_content As String
      Private m_processInstructions As String = ""
      Private m_processStarted As Boolean = False
      Private m_processSuccessfull As Boolean = False
      Private m_statusCode As String
      Private m_statusDescription As String

      Public ReadOnly Property Uri() As Uri
         Get
            Return m_uri
         End Get
      End Property

      Public Property ProcessStarted() As Boolean
         Get
            Return m_processStarted
         End Get
         Set(ByVal Value As Boolean)
            m_processStarted = Value
         End Set
      End Property
      Public Property ProcessSuccessfull() As Boolean
         Get
            Return m_processSuccessfull
         End Get
         Set(ByVal Value As Boolean)
            m_processSuccessfull = Value
         End Set
      End Property

      Public Property ProcessInstructions() As String
         Get
            Return m_processInstructions
         End Get
         Set(ByVal Value As String)
            m_processInstructions = Value
         End Set
      End Property

      Public Property Content() As String
         Get
            Return m_content
         End Get
         Set(ByVal Value As String)
            m_content = Value
         End Set
      End Property

      Public Property StatusCode() As String
         Get
            Return m_statusCode
         End Get
         Set(ByVal Value As String)
            m_statusCode = Value
         End Set
      End Property

      Public Property StatusDescription() As String
         Get
            Return m_statusDescription
         End Get
         Set(ByVal Value As String)
            m_statusDescription = Value
         End Set
      End Property
#End Region

   End Class

End Namespace

By viewing downloads associated with this article you agree to the Terms of Service and the article's licence.

If a file you wish to view isn't highlighted, and is a text file (not binary), please let us know and we'll add colourisation support for it.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


Written By
Web Developer
Australia Australia
I have been programming commercially since 1990, my last two major roles have been Architect/Lead Developer for an online bank and Australia's largest consumer finance comparison portal.

On the side I am a Forex Currency Trader and actively develop tools and applications for Currency Traders.

I have just launched a personal blog at www.davidcruwys.com and a website targeting Foreign Exchange traders at www.my-trading-journal.com

Comments and Discussions