WebClient Class with Cookies

A Technical Blog article. View original blog here.[^]

The WebClient class can be used to automate the login to a web application. If the web application requires the client to enable cookies, which is often the case when the application uses session state, the application returns an error to the client indicating that cookies must be enabled. The WebClient class can be extended to support cookies by overriding the GetWebRequest method and implementing a CookieContainer. This allows the client to login to the application and maintain the cookie state for subsequent calls to the web application.

To demonstrate this, we should load Visual Studio (2008/2010) and create a new console application. Add a new class and name it WebClientWithCookies. The class should look as follows:

class WebClientWithCookies: WebClient
{
    private CookieContainer _container = new CookieContainer();

    protected override WebRequest GetWebRequest(Uri address) 
    {
        HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest; 
        
        if (request != null) 
        { 
            request.Method = "Post";
            request.CookieContainer = _container; 
        } 

        return request;
    }
}

The override in the class provides access to the HttpWebRequest reference which facilitates the access to the CookieContainer property. A new CookieContainer object is created and associated to the Request object. This allows the client to store the cookies from the web application in the CookieContainer property of the request. We also set the request method to use POST (this is not needed if the web application allows GET requests).

Now, open the Program class of the console application and update the code to look as follows:

class Program
{
    private static string _cookies=string.Empty;
    static void Main(string[] args)
    {
        string url = "replace with url here";
        Post(url);
        Console.WriteLine("Hit return to exit");
        Console.ReadLine();
    }

    static void client_OpenReadCompleted(object sender, 
                System.Net.OpenReadCompletedEventArgs e)
    {
        if (e.Error == null)
        {
            using (Stream stream = e.Result)
            {
                using (StreamReader reader = new StreamReader(stream))
                {
                    Console.WriteLine(reader.ReadToEnd());
                }
            }
            
            WebClientWithCookies client = sender as WebClientWithCookies;
            
            if (client != null)
            {
                _cookies = client.ResponseHeaders["Set-Cookie"];
                Console.WriteLine(_cookies);
            }
        }
        else
        {
            Console.WriteLine(e.Error.Message);
        }
    }
       

    private static void Post(string url)
    {
        using (WebClientWithCookies client = new WebClientWithCookies())
        {
          client.OpenReadCompleted += 
             new OpenReadCompletedEventHandler(client_OpenReadCompleted);
          var uri = new Uri(url, UriKind.RelativeOrAbsolute);
          client.Headers.Add("Cookie",_cookies);
          client.OpenReadAsync(uri);
        }
    }
}

The Post method instantiates a web client (with cookies support) object to post the request to the web application. You should also notice that we add a request header to include any cookies that had already been persisted. The call to the web application is made asynchronous, and the handler client_OpenReadCompleted handles the result of the request. If there are no errors, the handler reads the stream and displays the response on the console. The client reference is then used to read the response headers. Here we are using Set-Cookie header name which is used by ASP.NET applications. You may need to inspect the response headers using a tool like Fiddler. The inspection can help you identify the key name for the cookie header from the system that you are using. The cookie in the response is saved in a variable. This is necessary because on any other sub-sequent call to the web application, we need to send the cookie back to the server. If the cookie is not sent back to the server, the web application will redirect back to the login page.

I hope this helps.