How to Connect to a SiteMinder Protected Resource Using an HTTP Request

• Download source code - 4.47 KB

Introduction

There are many articles describing various ways to connect to a web site using some sort of HTTP connection class, depending on the framework used. In the .NET world, this includes the HttpWebRequest and HttpWebResponse classes among others. But sometimes, the resources the application needs to connect to are protected by third party authentication providers such as SiteMinder. This article describes how to authenticate with SiteMinder and then connect to the desired URL.

Identifying Network Resources

The .NET Framework uses a Uniform Resource Identifier (URI) to identify the requested Internet resource and communication protocol. The URI consists of a scheme (or protocol), a host name, port number, path and query string. The scheme (i.e. – http://) is always required. The host name is also required, but may be specified either as an IP address, or a host name that can be resolved by Domain Name System (DNS) to an IP address. A scheme will have a default port associated with it, so a port number is not required unless the target host responds on a port other than the default. The path component locates specific information on the server and also defaults to “/”. The query string passes information to the server and is only used by certain applications.

When using the HTTP protocol, a client sends requests to the server and gets responses in return. In the sample application, the HttpWebRequest class which encapsulates my request to the server; and HttpWebResponse class which provides a container for the incoming response are used. The corresponding Java functions would be found in either the HttpUrlConnection or UrlConnection classes.

Request Flow

SiteMinder can be deployed in both proxy server and agent configurations. The agent configuration installs a software agent on the web server and is the configuration addressed by this article.

The following steps occur when a user tries to access a protected resource on a web server configured to use SiteMinder authentication:

1. The user requests a resource on the server, either through a web browser or in a program using an HTTP request.
2. The request is received by the web server and is intercepted by the SiteMinder web agent.
3. The web agent determines whether or not the resource is protected, and if so, gathers the user’s credentials and passes them to the Policy server.
4. The Policy server authenticates the user and verifies whether or not the authenticated user is authorized for the requested resource, based on rules and policies contained in the Policy store.
5. After the user is authenticated and authorized, the Policy server grants access to the protected resources.

In step 3 above, if no SiteMinder session exists, users are redirected to a login page where they are prompted to enter their credentials. Once the user is authenticated, a cookie is added to the response headers, creating a SiteMinder session. When this cookie is included on subsequent requests, the user is directed to the original URL without further prompting. More detail is presented in Figure 1 below.

Logging into SiteMinder

In order to gain access to the protected resource, the request must first be authenticated by SiteMinder. This is accomplished with the following steps:

1. Open a connection (HTTP request in this case) to the URI of the protected resource. Since the request has not yet been authenticated, the SiteMinder agent will issue a redirect to a login page. In the sample code, AllowAutoRedirect is set to false. This is important as the redirect URL will be required for the subsequent POST of login data in step 3 below. If AllowAutoRedirect were True, the response would not include a Location header and the subsequent POST would be made to the original URL, which would then redirect to the login page again. However, a POST occurs between a client and the server, any POST data carried in the payload of the request of step 3 will be lost during the redirect.

Dim request As HttpWebRequest
Dim response As HttpWebResponse
Dim url As String = PROTECTED_URL

request = WebRequest.Create(url)
request.AllowAutoRedirect = False
response = request.GetResponse

' make sure we have a valid response
If response.StatusCode <> HttpStatusCode.Found Then
    Throw New InvalidProgramException
End If

' get the login page
url = response.Headers("Location")
request = WebRequest.Create(url)
request.AllowAutoRedirect = False
response = request.GetResponse

2. When the login form is returned, it is parsed to get all the hidden and input form fields used by SiteMinder. The sample program includes a function that uses a WebBrowser and HtmlDocument object to parse the page, but this could be accomplished by any means.
3. The next step involves creating an HTTP request that POSTs all the form data, including userid and password, back to the server. The purpose of an authentication agent is to verify a user’s identity by validating their userid and password. Thus, their URLs naturally use SSL (secure sockets layer) and are encrypted for us, so we do not required further encryption in our program. However, the formatting of the POST data is interesting in as much as there are two alternatives. The sample program uses the simpler approach of setting the content type to application/x-www-form-urlencoded. Here the POST data is formatted similar to a query string and sent as part of the next request.

Dim postData As String

postData = ""
For Each inputName As String In tags.Keys
    If inputName.Substring(0, 2).ToLower = "sm" Then
        postData &= inputName & "=" & _
                    HttpUtility.UrlEncode(tags(inputName)) & "&"
    End If
Next
postData += "postpreservationdata=&"
postData += "USER=" + HttpUtility.UrlEncode(USERNAME) & "&"
postData += "PASSWORD=" + HttpUtility.UrlEncode(PASSWORD)

request = WebRequest.Create(url)
cookies = New CookieContainer
request.CookieContainer = cookies
request.ContentType = FORM_CONTENT_TYPE
request.ContentLength = postData.Length
request.Method = POST_METHOD
request.AllowAutoRedirect = False   ' Important

Dim sw As StreamWriter = New StreamWriter(request.GetRequestStream())
sw.Write(postData)
sw.Flush()
sw.Close()

response = request.GetResponse

4. If authentication is successful, the next response will contain the SiteMinder cookies. These cookies, when included in subsequent requests, will be used by SiteMinder to perform authentication without further login prompts.

Cookies

Cookies are additional pieces of information included in HTTP headers that allow information to be passed from page to page in a web application. Once the user credentials are verified, SiteMinder will create and return a cookie that establishes the user’s session.

Windows maintains a cookie database in memory. Cookies with an expiration date (in the future) are further persisted to disk. Deleting the cookie file from disk does not remove it from the cookie database in memory. As long as the user’s Windows session remains open, any new browser instances will reference any cookies in the cookie database.

To effectively logout of a SiteMinder protected application, the cookie containing the SiteMinder session value must be deleted from the cookie database. This can be accomplished in Windows either with the DeleteFromUrlCache function, or simply by setting the cookie with no expiration date or with an expiration date in the past. (Setting no expiration date will delete the cookie from persistent store, but not from the cache in memory.)

An interesting thing about the cookies from SiteMinder is found when inspecting the headers returned in the response after authentication. SiteMinder creates a "Set-Cookie" header that is compatible with Netscape’s original cookie proposal, in as much as it uses an “Expires” header as described in RFC2109, Para. 10.1.2 in place of the "Max-Age" value and is formatted as follows:

Wdy, DD-Mon-YY HH:MM:SS GMT 

Note that if you use the .NET WebHeadersCollection.GetValues method, it treats the Set-Cookie header as a comma-separated list of cookies, as defined in RFC 2965, Para. 3.2.2. The comma embedded in the expires value causes the GetValues method to split this into two cookies. This may cause a problem when working with some older servers.

On the other hand, if using the .NET Framework, the cookies can be obtained directly from the response using the Cookies property. The ParseHeaderForCookies function is included in the sample program for reference if you’re not using the .NET Framework, or if you’re so inclined to parse the headers yourself. Sometimes, inspecting the HTTP headers directly can provide insight when dealing with third-party products.

If the protected resource is a web page, then persisting the cookies can be valuable during testing. While the unexpired cookie is persisted, a separate browser window can be opened and should be able to connect to the protected resource without SiteMinder prompting for login credentials.

Public Function ParseHeadersForCookies(ByVal res As HttpWebResponse) As CookieContainer

    Dim cc As CookieContainer = New CookieContainer

    For Each header As String In res.Headers

        If header.Equals(SET_COOKIE_HEADER) Then

            Console.WriteLine("header={0}", header)
            Dim setCookies() As String = res.Headers.GetValues(header)

            For i As Integer = 0 To setCookies.Length - 1
                Dim setCookie As String = setCookies(i)

                ' handle the extra comma in the expires attribute
                Dim j As Integer = setCookie.IndexOf("expires")
                If j <> -1 Then
                    i += 1
                    setCookie &= setCookies(i)
                End If

                ' parse the cookie and update its status in the collection
                Dim equalsPos As Integer = setCookie.IndexOf("="c)
                If equalsPos <> -1 Then
                    Dim name As String = setCookie.Substring(0, equalsPos)
                    Dim value As String = setCookie.Substring(equalsPos + 1,
                        setCookie.IndexOf(";"c) - equalsPos - 1)

                    ' need to get the cookie's domain
                    Dim domain As String = ""
                    Dim domainPos As Integer = setCookie.IndexOf("domain")

                    If domainPos <> -1 Then
                        Dim endPos As Integer = setCookie.IndexOf(";"c, domainPos)
                        Dim eqPos As Integer = setCookie.IndexOf("="c, domainPos)

                        If endPos <> -1 Then
                            domain = setCookie.Substring(eqPos + 1, endPos - eqPos - 1)
                        Else
                            domain = setCookie.Substring(eqPos + 1)
                        End If

                        ' add/Update the cookie
                        cc.Add(New Cookie(name, value, "/", domain))
                    Else
                        ' add/Update the cookie
                        cc.Add(New Cookie(name, value, "/", ".company.com"))
                    End If
                End If
            Next
        End If
    Next

    Return cc

End Function

In the sample code, routines are included to persist and delete these cookies from disk, which is largely a matter of setting the expiration date. This may be useful to others, but not needed in this program since the cookies are explicitly added to the request headers.

To set a cookie:

expireDate = DateTime.Now.ToUniversalTime.AddMinutes(COOKIE_TIMEOUT_MINUTES)

For Each cookie In cookies.GetCookies(New Uri(PROTECTED_DOMAIN_URI))
    cookieString = cookie.Name & "=" & cookie.Value & "; expires = " & _
                   expireDate.ToString("ddd, dd-MMM-yyyy HH:mm:ss 'GMT'") & ";"
    rc = InternetSetCookie(PROTECTED_DOMAIN_URI, Nothing, cookieString)
Next

Note that the ToUniversalTime method is used since the format string includes "GMT".

To delete a cookie, either set it with an expiration date in the past or with no expiration date at all:

For Each cookie In cookies.GetCookies(New Uri(PROTECTED_DOMAIN_URI))
    ' pick one...
    'cookieString = cookie.Name & "=" & cookie.Value & _
                    "; expires = Sat, 01-Jan-2000 00:00:00 GMT;"
    cookieString = cookie.Name & "=" & cookie.Value & ";"

    rc = InternetSetCookie(PROTECTED_DOMAIN_URI, Nothing, cookieString)
Next

Conclusion

Working with multiple technologies can be challenging, and usually, the solution to a problem involves many issues. In this case, inspecting the HTTP headers revealed useful information. Although this application was developed under .NET, the same concepts apply using J2EE. Hopefully, some part of this experience will be valuable to others, especially when working with SiteMinder.

History

• 10-May-2010 - Original submission