Introduction
This article explains how to track ASP.NET web pages through your own web application without relying on external tracking tools. This does not eliminate the need for external tools, but it gives some satisfaction to have your own tracking that you can play with and modify to suit to your business requirements. This is all possible due to the rich API provided by the .NET framework.
I researched on the web to find out if people have developed a code to do custom tracking in .NET, and I found an excellent article by Wayne Plourde that explains how to track web pages in ASP.NET. This article derives lots of inspiration from Wayne Plourde's original article. I recommend that you read this article first as it will help you to understand page tracking concepts.
When I ported Wayne Plourde's VB.NET code to C#, I stumbled upon a few problems regarding session state and the way it is being handled in .NET framework by Microsoft architects. I have been thinking for a long time to write on this subject and share my thoughts with the CodeProject community. Now I got an excellent opportunity to share my thoughts as I am switching jobs.
When I publicized my resume for a new job, I wanted to see real time usage of web page tracking methods. I was thrilled to see that people spend an average of 2-10 seconds to see a resume, and if it does not catch their attention, it is buried forever.
Location - Location - Location
As a web application developer, I am interested to know the location of the visitors coming to a web site. This is important from business standpoint to target your audience accordingly. In a simple scenario, I am interested to know the location of people accessing my resume just for the sake of curiosity.
There are some articles on CodeProject on geographical locations of the visitors based upon an IP address. I like GeoBytes as they provide free service for non-commercial use. GeoBytes provide a simple cut-and-paste code that you can put on your site to greet your users.
I want little more on this, as I need to store geographical location of the users based upon an IP address for statistical analysis on the pattern of visitors. Currently, GeoBytes does not provide a web-service like method in which you pass an IP address from your web server and it returns the geographical location. There is a little work-around to this.
- Make a startup page say index.html.
<html>
<body>
<script language="Javascript"
src="http://gd.geobytes.com/Gd?after=-
1&variables=GeobytesCountry,GeobytesCity,GeobytesRegion">
</script>
<script language="javascript">
if(typeof(sGeobytesCountry) != "undefined" &&
typeof(sGeobytesRegion) != "undefined" &&
typeof(sGeobytesCity) != "undefined")
{
var url = 'index.aspx?IPCity='+sGeobytesCity+'&IPRegion='+
sGeobytesRegion+'&IPCountry='+sGeobytesCountry;
document.write("<META HTTP-EQUIV='Refresh' CONTENT='0; URL="+url+"'>");
} else
{
document.write("<META HTTP-EQUIV='Refresh' CONTENT='0; URL=index.aspx'>");
}
</script>
</body>
</html>
- Through the above startup HTML page, a trip is made to GeoBytes site and it returns the city, region, and country based upon an IP address in HTTP request. After this, we call our regular ASPX startup page and pass these three parameters as Request query string. Now you have the information, and use it to analyze or redirect the users to custom pages based upon their locations. For non-commercial use, this may be an acceptable solution, but for high volume sites, you need to have your own geo database.
- For demo project to work on your machine, please make sure that you define index.html as a default document type through your inetmgr MMC.
Issues with Session State
If you are developing a scalable web application in ASP.NET, you will come across a need to switch your session state from InProc to either StateServer or SQLServer in the future. In my early stages of learning ASP.NET, we always used InProc session state, but later we had to change it to StateServer from scalability point of view. There is an excellent FAQ on Session State by Patrick Y. Ng in the ASP.NET site, and it is worth spending some time to read this article to get an idea about session state in detail.
If your application uses InProc session state, Wayne Plourde's article code is sufficient enough to do the page tracking. I had to do some changes in the concepts of web page tracking to accommodate StateServer session state, due to following major reasons:
Session_End event is not supported in StateServer or SQLServer Session mode.
- Objects need to be serializable when using
StateServer or SQLServer Session mode.
This article's code works for all three session states, and you can make this code a part of your own framework and enable page tracking by having a few entries in your web.config file.
ASP.NET Page Tracking
My purpose was not to invent how ASP.NET page tracking should be done. I was just trying to make Wayne's code work for all three session states. I have explained the changes that I had to make in order for it to work for the limitations explained above.
- Simulate
Session_End event in StateServer or SQLServer Session mode by creating a cache object with a hookup that will fire exactly on session time out.
- This approach has its own caveat as the session objects will not be available when we simulate
session_end event, as ASP.NET will recycle the session at that very moment when we were trying to access it.
- We simulate the session object by creating a simple static hash table that will hold our objects until the session ends.
Basics of Wayne's Page Tracking
- Create a Session Tracker class.
- Persist this class in a ASP.NET Session.
- Use cookies to store previous values of variables that we are interested to track.
- Update the tracker class in Session whenever user visits a page.
- Publish the data when
session_start and session_end events fire.
- Show a summary of Page Visited Statistics when session expires.
Modifications to Wayne's Page Tracking for All Three Session States
- Create a modified Session Tracker Class.
- Simulate a
session_end event when session expires.
- Persist this class in a static hash table in view of
session_end event problem.
- Update the tracker class in static hash table whenever user visits a page.
- Publish the data when
session_start and session_end events fire.
- Show a summary of Page Visited Statistics when session expires.
Global.asax v/s HttpModule
The global.asax.cs provides methods that execute on specific events. Each web application has its own global.asax.cs file, but it is my own personal taste not to use this file and instead use HTTP Module due to the following reasons:
- Http Module can be used by many web applications that you develop.
- There is no need to copy code from one global.asax web application to another web application when it is going to be same for all web applications.
- Keep a common code in your own framework and use that framework all across your web applications.
Register your HttpModule
In the demo project, I have a small framework class library called EADFramework. EAD stands for Enterprise Application Development. Reference this class library in your web project. Modify web.config to register HttpModule and set parameters used by Page Tracking.
="1.0" ="utf-8"
<configuration>
<appSettings>
<add key="MailFrom" value="mailto:yourMailFrom@yourserver.com">yourMailFrom@yourserver.com" />
<add key="MailTo" value="mailto:yourMailTo@yourserver.com">yourMailTo@yourserver.com" />
<add key="SmtpServer" value="smtp.yourmailserver.com" />
<add key="SiteTracking" value="Full" />
<add key="ExcludeIPList" value="127.0.0.1,10.,192.168,172.16" />
</appSettings>
<system.web>
<httpModules>
<add name="EADHttpModule"
type="EAD.Controller.EADHttpModule, EADFramework" />
</httpModules>
<compilation defaultLanguage="c#" debug="true" />
<trace enabled="false" requestLimit="10" pageOutput="false"
traceMode="SortByTime" localOnly="true" />
<sessionState mode="StateServer"
stateConnectionString="tcpip=127.0.0.1:42424" timeout="20" />
</system.web>
</configuration>
You can turn on/off site tracking by using SiteTracking value as "Full" or "None". In my own framework, I also use "Partial" tracking to track selected pages only. This itself is a separate topic for navigation to/from pages. I use a separate class for Page Navigation using attributes defined in web.config. One of the attributes for Page Navigation is PageVisit and when set on 'true', it turns page tracking on for that page. For the purpose of simplicity, I am not putting the code for partial web page tracking. I used the PageVisit method in PageTrackerUtil class to control partial page tracking.
You can specify ExcludeIPList to filter out your local or internal IPs for tracking. You could monitor one or more specific IP by using IncludeIPList. Use IncludeUserList and ExcludeUserList to list CSV of users that need to be monitored or excluded from monitoring.
The session state is defined as a StateServer. When you run this demo, please make sure that your ASP.NET State Service is running on your web server.
Define Events for HttpModule
We need three events to register in our HttpModule for page tracking.
- Register Events:
public void Init(HttpApplication app)
{
app.BeginRequest += (new EventHandler(this.OnBeginRequest));
app.PreRequestHandlerExecute +=
(new EventHandler(this.OnPreRequestHandlerExecute));
if (app.Modules["Session"] != null)
{
SessionStateModule session = (SessionStateModule)
app.Modules["Session"];
app.AcquireRequestState +=
(new EventHandler(this.OnAcquireRequestState));
session.Start += (new EventHandler(this.OnSessionStart));
}
}
AcquireRequestState event - This event occurs when ASP.NET acquires the current session state associated with the current request. This is an ideal place to create a cache object with a callback function. This callback function timeout is set to session timeout limit. private void OnAcquireRequestState(Object source, EventArgs ea)
{
HttpApplication app = (HttpApplication)source;
HttpContext ctx = (HttpContext) app.Context;
string key = ctx.Session.SessionID;
CacheItemRemovedCallback onCacheRemove = null;
try
{
ctx.Cache.Remove(key);
onCacheRemove = new CacheItemRemovedCallback
(this.CacheRemoveCallback);
ctx.Cache.Add(key,ctx.Session.SessionID,null,
DateTime.Now.AddMinutes(ctx.Session.Timeout),TimeSpan.Zero,
CacheItemPriority.Normal,onCacheRemove);
}
catch(Exception exception)
{
string message = exception.Message;
}
}
The CacheRemoveCallBack method calls method PageTrackerUtil.TrackSessionEnd to publish statistics at the end of session. The CacheCallBackMethod is synchronized to execute at the same time when session expires.
private void CacheRemoveCallback(string key, Object source,
CacheItemRemovedReason reason)
{
if (reason == CacheItemRemovedReason.Expired)
{
PageTrackerUtil.TrackSessionEnd(key);
}
}
- Session Module events - Use
SessionStateModule to register OnSessionStart event. Use this event to signal when someone has accessed the site. private void OnSessionStart(Object source, EventArgs ea)
{
PageTrackerUtil.TrackSessionStart(HttpContext.Current);
}
PreRequestHandlerExecute - This event occurs just before ASP.NET begins executing a page. Use this event to monitor individual pages. private void OnPreRequestHandlerExecute(object source, EventArgs e)
{
PageTrackerUtil.TrackSessionPages(((HttpApplication)source).Context);
}
Page Tracking Methods
- Through above
HttpModule, we have hooked up three methods: TrackSessionStart, TrackSessionPages, and TrackSessionEnd of PageTrackerUtil to take care of the page level tracking. Since we will not have access to the Session when it is recycled by ASP.NET, we will use our own small and simple static hash table to persist objects for page level tracking.
TrackSessionStart private static void UpdateSessionConst(string key, PageTracker tracker)
{
if (SessionConst.Session[key] == null)
{
SessionConst.Session.Add(key,tracker);
}
else
{
SessionConst.Session.Remove(key);
SessionConst.Session.Add(key,tracker);
}
}public static void TrackSessionStart(HttpContext ctx)
{
string key = ctx.Session.SessionID;
string siteTracking =
ConfigurationSettings.AppSettings["SiteTracking"];
string ip = ctx.Request.UserHostAddress.ToString();
if (siteTracking == null || siteTracking.Length == 0) return;
if (siteTracking.ToLower() != "none" && ValidTracking(ctx))
{
PageTracker tracker = new PageTracker(ctx);
UpdateSessionConst(key,tracker);
EADUtility.SendMail(tracker.SessionUserHostAddress,
CreateTrackerMessageBody(tracker));
}
}
At the start of the session, we instantiate PageTracker class and persist it in our static hash table SessionConst.Session.
using System;
using System.Collections;
namespace EAD.Constant
{
public class SessionConst
{
public static Hashtable Session = new Hashtable();
public static Hashtable Page = new Hashtable();
}
}
TrackSessionPages public static void TrackSessionPages(HttpContext ctx)
{
string siteTracking =
ConfigurationSettings.AppSettings["SiteTracking"];
if (siteTracking == null || siteTracking.Length == 0 ||
siteTracking.ToLower() == "none") return;
if (ctx.Session == null) return;
string key = ctx.Session.SessionID;
siteTracking = siteTracking.ToLower();
if (siteTracking == "full" || siteTracking == "partial")
{
if (SessionConst.Session[key] != null)
{
PageTracker tracker = (PageTracker) SessionConst.Session[key];
if (tracker != null)
{
string relativeFilePath = ctx.Request.Url.AbsolutePath.Remove(0,
ctx.Request.ApplicationPath.Length).ToLower();
int pos = relativeFilePath.IndexOf("?");
if (pos > 0)
{
relativeFilePath = relativeFilePath.Substring(0,pos);
}
if (siteTracking == "full")
{
tracker.AddPage(relativeFilePath);
}
else if (PageVisit(relativeFilePath))
{
tracker.AddPage(relativeFilePath);
}
UpdateSessionConst(key,tracker);
}
}
}
At page level tracking, we retrieve PageTracker class from the static hash table and persist it in the same static hash table SessionConst.Session after adding the page level information.
TrackSessionEnd public static void TrackSessionEnd(string key)
{
string siteTracking = ConfigurationSettings.AppSettings["SiteTracking"];
if (siteTracking == null || siteTracking.Length == 0) return;
if (siteTracking.ToLower() != "none")
{
if (SessionConst.Session[key] != null)
{
PageTracker tracker = (PageTracker) SessionConst.Session[key];
SessionConst.Session.Remove(key);
SessionConst.Page.Remove(key);
if (tracker != null)
{
EADUtility.SendMail(tracker.SessionUserHostAddress+"-End",
CreateTrackerMessageBody(tracker)+
CreateTrackerPageListing(tracker));
}
}
}
}
Page Tracker Class
Until now, it was only administrative procedures for executing page level tracking. The actual page level information is stored in PageTracker class.
- The initialization of parameters is done in the constructor of
PageTracker class. public PageTracker(HttpContext ctx)
{
string KEY = ctx.Session.SessionID;
pages = new ArrayList();
expires = DateTime.Now.AddYears(1);
sessionReferrer = (ctx.Request.UrlReferrer == null) ?
string.Empty : ctx.Request.UrlReferrer.ToString();
sessionURL = (ctx.Request.Url == null) ? string.Empty :
ctx.Request.Url.ToString();
if (SessionConst.Page[KEY] == null)
{
visitCount = 1;
originalReferrer = sessionReferrer;
originalURL = sessionURL;
SessionConst.Page.Add(KEY,new PageHashData(visitCount,
sessionReferrer,sessionURL));
}
else
{
PageHashData pageHashData = (PageHashData) SessionConst.Page[KEY];
pageHashData.PageCount++;
SessionConst.Page.Remove(KEY);
SessionConst.Page.Add(KEY,pageHashData);
visitCount = ((PageHashData)SessionConst.Page[KEY]).PageCount;
originalReferrer = ((PageHashData)SessionConst.Page[KEY]).Referrer;
originalURL = ((PageHashData)SessionConst.Page[KEY]).Url;
}
userHostAddress = ctx.Request.UserHostAddress.ToString();
userAgent = ctx.Request.UserAgent.ToString();
browser = ctx.Request.Browser.Browser;
crawler = ctx.Request.Browser.Crawler.ToString();
}
- The
AddPage method is called the TrackSessionPages method. public void AddPage(string pageName)
{
PageData pti = new PageData();
pti.PageName = pageName;
pti.Time = DateTime.Now;
pages.Add(pti);
}
- The use of cookies to store information is also eliminated with the use of
PageHashData and PagaData classes which are stored in the Page hash table in SessionConst. using System;
namespace EAD.Utility
{
[Serializable]
public class PageHashData
{
private string referrer, url;
private int pageCount;
public PageHashData(int pageCount, string referrer, string url)
{
this.pageCount = pageCount;
this.referrer = referrer;
this.url = url;
}
public int PageCount { get { return pageCount;}
set{pageCount = value;} }
public string Referrer{ get { return referrer;} }
public string Url{ get { return url;} }
}
}using System;
namespace EAD.Utility
{
[Serializable]
public class PageData
{
public string PageName;
public DateTime Time;
}
}
Known Issues
I still have to figure out how much time a user spends on a particular page if they do not navigate to other pages, because page statistics are only collected when the user switches to another page.
Results
The sample result of the page tracking output is similar to the output used by Wayne's article.
Sample output when someone hits to the site:
Sample output when session expires. At this time, you get the detailed statistics on how a user has navigated your site:
| Visited Pages Count: |
7 |
| Visited Pages |
Elapsed Time |
| /index.aspx |
00:00:03 |
| /index.aspx |
00:00:02 |
| /webform2.aspx |
00:00:01 |
| /webform2.aspx |
00:00:00 |
| /webform3.aspx |
00:00:01 |
| /webform3.aspx |
00:00:00 |
| /index.aspx |
|
| Total Time: |
00:00:08 |
