Click here to Skip to main content
13,355,335 members (72,034 online)
Click here to Skip to main content
Add your own
alternative version


68 bookmarked
Posted 26 Mar 2006

A simple .NET based WebClient with JavaScript support

, 26 Mar 2006
Rate this:
Please Sign up or sign in to vote.
A simple .NET based WebClient with JavaScript support.


If you want to access web pages within your .NET code, which means having access to the HTML document object model and evaluation of embedded scripts, the only way to go is to wrap the Internet Explorer COM interface (SHDocVw). But this method has some major drawbacks:

The greatest is the inability to capture an alert() or confirm() script call. This means, if you load a web page in your .NET code with the SHDocVw library, and it has an embedded alert('Hello World'); call, your program will wait until the user presses "OK".

Another drawback is the inability to spawn two or more Internet Explorer instances within the same process having different cookie stores. All browsers instantiated in your runtime will share the same resources, therefore you are unable to, e.g., login in as two users at the same time to a cookie based web page. There is a workaround for this (launching via System.Diagnostics.Process two or more iexplore.exe, and hooking them to your COM objects in your runtime), but this method leads to unpredictable behaviour like leaving iexplore.exe instances around if your program doesn't shutdown gracefully.

For my purposes, I needed a simple web client, which allows me to access the basic HTML document object model objects and simple JavaScript code (like the evaluation of the default ASP.NET client side validators).


For building a JavaScript enabled WebClient, we need at least the following three basic components: A HTTP protocol implementation, a HTML document object model, and a JavaScript engine. Since .NET already offers a lot of built-in types for handling at least similar tasks, we will reuse them as much as possible.

A HTTP protocol implementation

The types System.Net.HttpWebRequest and System.Net.HttpWebResponse implement the HTTP protocol, and are therefore perfect for our job. They can handle GET and POST requests, allow us to read and write HTTP headers, and are capable of making HTTP as well as HTTPS connections.

A HTML document object model

Currently, the .NET framework does not support a HTML object model. But since HTML is very similar to XML, we will use System.Xml.XmlDocument and extend it to our needs.

A JavaScript engine

The .NET framework offers a JavaScript implementation called 'JScript.NET'. It is available through the Microsoft.JScript.Vsa (Visual Studio for Application) Framework, and can be easily used for our purpose.



The type Cb.Web.WebClient is very straightforward. It has an empty default constructor, a void Get(string url); method, and the HTML document object model can be accessed via myWebClient.Window.Document.

It has two events for handling dialog stuff like alert (OnAlert) and confirm (OnConfirm) JavaScript calls, and one for capturing JavaScript errors (OnError).

// Basic example
WebClient myWebClient = new WebClient();
myWebClient.OnAlert += new AlertHandler(WebClient_OnAlert);


The HTML document object model can be, as already mentioned, accessed by using the property Window.Document of a WebClient. The interface is basically the default browser DOM interface (i.e., providing methods like GetElementById, GetElementsByTagName, or Body.)

                       SetAttribute("value", email);
                       SetAttribute("value", password);

By extending System.Xml.XmlDocument we have all the features of an XML document, like for example, X-Path queries (SelectSingleNode, SelectNodes).

The type Cb.Web.Html.HtmlReader, which is used to populate a HtmlDocument, is a very basic HTML parser which is non validating. At its current state, it won't throw any parsing error, but may eat some tags or attributes. I didn't use the SgmlReader as mentioned in the article Convert HTML to XHTML and clean unnecessary tags and attributes, because I couldn't figure out an easy way to populate our DOM with it.


For implementation of our JavaScript engine, I used the Microsoft.JScript.Vsa.VsaEngine. The basic usage is explained in the great article VSA Scripting in .NET by Mark Belles.

The interesting part of this implementation is to hide the .NET signature of our DOM objects and provide the standard DOM level signature, and also to add expando features to all scripting objects. 'Expando' means the ability to attach properties to any object at any given time.

To achieve these features, all objects which are populated to our scripting engine inherit from System.Reflection.IReflect. This way, our objects are forced to provide a method called MemberInfo[] GetMember(name) which then will be used by the VsaEngine to resolve properties and methods.

class HtmlDocument : IReflect {

    // Note: this example is stripped, its main purpose is to show you
    // the idea behind expando objects and property/method resolving
    public MemberInfo[] GetMember(string name) {
        switch (name) {
            case "getElementById": return GetType().GetMember("GetElementById");
        // ... handle expando properties ...

Expando properties are implemented by creating a dynamic FieldInfo() in the above method with the given name, which stores or gets its value through an object bound Hashtable. This way, scripts like these will then work:

document.MyVar = "World";
alert("Hello " + document.MyVar);
// Will alert 'Hello World'

Final Notes

In its current state, the attached source code can handle simple web pages and evaluate simple JavaScript sources. It currently has (basically) no CSS support nor any real DOM compliance. Its performance isn't on the edge either, but it works on basic ASP.NET web pages, and can be used in multi threaded environments.

The attached demo shows most of its currently implemented features. It opens "", searches for one of my articles, and if you modify the source code and provide your username and password, will rate it with 5 points ;-). On a side note, I've removed the try/catch block, which in case of failure would rate it with 1 point...


Version 1.0 (released on 26.03.2006)

  • Initial release.



This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


About the Author

Christian Birkl
Web Developer
Germany Germany
No Biography provided

You may also be interested in...

Comments and Discussions

QuestionUpgrade framework To 4 Pin
Member 1107802116-Aug-15 17:35
memberMember 1107802116-Aug-15 17:35 
QuestionWhere can i adjust the proxy settings? Pin
sky scripter8-Dec-14 18:33
membersky scripter8-Dec-14 18:33 
QuestionException when calling 'Get' Pin
Member 1020253314-Aug-13 6:58
memberMember 1020253314-Aug-13 6:58 
QuestionException when calling 'Get' Pin
Shimmy Weitzhandler2-Jul-13 16:56
memberShimmy Weitzhandler2-Jul-13 16:56 
GeneralMy vote of 1 Pin
Shimmy Weitzhandler2-Jul-13 16:43
memberShimmy Weitzhandler2-Jul-13 16:43 
QuestionWould love to see it in NuGet. Pin
Shimmy Weitzhandler2-Jul-13 16:26
memberShimmy Weitzhandler2-Jul-13 16:26 
GeneralErrors received Pin
MichealBarlow23-Nov-09 8:21
memberMichealBarlow23-Nov-09 8:21 
GeneralFor more complex scripts Pin
Shayan Elhami4-Jun-09 9:04
memberShayan Elhami4-Jun-09 9:04 
GeneralExecute javascript [modified] Pin
Ingvarr17-Feb-09 2:45
memberIngvarr17-Feb-09 2:45 
GeneralUpdates and additions Pin
DaveEchols15-Apr-08 3:36
memberDaveEchols15-Apr-08 3:36 
GeneralRe: Updates and additions Pin
spring17-Apr-08 22:58
memberspring17-Apr-08 22:58 
GeneralSimply Amazing!!!! Pin
CaueAbreu17-Oct-07 9:38
memberCaueAbreu17-Oct-07 9:38 
QuestionRe: Simply Amazing!!!! Pin
atlaste13-Nov-08 10:16
memberatlaste13-Nov-08 10:16 
GeneralA question to ask Pin
kzfid1-Aug-07 5:28
memberkzfid1-Aug-07 5:28 
GeneralJscriptException Pin
mantunes5-May-07 13:06
membermantunes5-May-07 13:06 
GeneralAutomation of complex DHTML/AJAX Web application Pin
Alex Furman17-Mar-07 15:55
memberAlex Furman17-Mar-07 15:55 
GeneralRe: Automation of complex DHTML/AJAX Web application Pin
anoobis5-Sep-07 23:41
memberanoobis5-Sep-07 23:41 
QuestionVery interesting and useful Pin
Monarghel29-Apr-06 11:46
memberMonarghel29-Apr-06 11:46 
AnswerRe: Very interesting and useful Pin
Christian Birkl1-May-06 11:26
memberChristian Birkl1-May-06 11:26 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web04 | 2.8.180111.1 | Last Updated 26 Mar 2006
Article Copyright 2006 by Christian Birkl
Everything else Copyright © CodeProject, 1999-2018
Layout: fixed | fluid