Click here to Skip to main content
Click here to Skip to main content

Real-Time Website Analytics

, 3 May 2013
Rate this:
Please Sign up or sign in to vote.
Real-Time Web Stats using Node.js and R

Please note

This article is an entry in our Windows Azure Developer Challenge. Articles in this sub-section are not required to be full articles so care should be taken when voting. Create your free Azure Trial Account to Enter the Challenge.

Introduction

The Problem 

Most of the Website Analytics services available today are Descriptive. They do not provide any helpful insights whether the adoption led to a meaningful result such as retention rate. The objective here is to turn your visitors into prospective customers by employing advanced techniques such as behavioral and predictive analysis.

The Solution 

The theme of this article is to give you a comprehensive overview of how the above mentioned techniques are employed in Real-Time to achieve the retention rate. This article will be updated in 5 Phases, step by step implementing each phase involves solving a critical problem that we face today for achieving predictive and behavioral metrics.  

The application will be hosted in Windows Azure Cloud Computing Platform initially as a PaaS provider and later using it as an Iaas provider. The languages and libraries used are mentioned below:

  • Node.js(Server Side JavaScript rendered by Google V8 Engine) 
  • R Language (for statistically exploring data sets and providing valuable insights)
  • MongoDB (NoSQL Document Database)  
  • Socket.IO (JavaScript Library for Real-Time web applications)
  • R-Node (Web front-end to the statistical analysis package R created by Jamie Love)
  • Protovis (Graphing API) (or) HTML5 Canvas 
  • Rserve (Binary Server used by R-Node Component)  
Features of this Cloud hosted app are:

  • Real Time: Collected Data is converted into meaningful info in real-time with sensible graphics. 
  • Fast: Node.js , Socket.IO and MongoDB driven architecture gives the requisite performance.
  • Event Driven: Click and Multitouch gestures for modern web and mobile webapps.   
  • Heatmaps: Click, Move and Multitouch inputs generate a lot. 
  • Demographics: Shows a wealth of info about the potential users and their behavior.  
  • Infographics: Graphics generated by the R package gives the right amount of info. 
  • Chronological Events: Detailed Metrics on Session Length and Activity length. 
  • Social Media Management 
  • Mobile Dashboards (Statistics on the move) 

Background 

Actionable Metrics help us make better business decisions. It can be stressful to find out that a report you're supposed to deliver tomorrow doesn't even have the data you wanted or needed. This is where Real-Time data comes into play, helping you to setup and implement analytics. You can see how effective that marketing campaign that you just launched a minute ago. You have the power to see how users behave when they first visit you website or web app. If you've just launched a new feature, real-time data is so much valuable for gaining valuable insights for obtaining meaningful result such as retention rate.

Using the code

The minimal version of the product consists of 3 parts:

  •  The Server 

 Node.js and Socket.IO are a match made in heaven for Real-Time Messaging System.

//server.js 

var io = require('socket.io').listen(3000);

io.sockets.on('connection', function (socket) {

    socket.on('message', function (message) {
        console.log("Got message: " + message);
        io.sockets.emit('pageview', { 'url': message });
    });

});              

As you can see, the code doesn't serve any HTML pages. It just loads a page in any domain or from a file.

  • Test Webpage 

Here is an example of a demo page with a script that sends a message to the server every time the page is loaded. 

<!-- simplepage.html> 

<html>
<body>
<h1>Simple Page</h1>
<script type="text/javascript" src="socket.io.min.js"></script>
<script>
    var socket = io.connect('http://localhost:3000');
    socket.on('connect', function () {
      socket.send(window.location);
    });
</script>
</body>
</html> 

The above code is loaded from the file system. 

  • The Dashboard
Here is a bare-bones implementation of the real-time dashboard application that receives “pageview” messages from the server. 

<!-- dashboard.html>

<html>
<body>
<h1>Real Time Page Views</h1>
<ul id="pageViews"></ul>
<script src="socket.io.min.js"></script>
<script src="jquery-1.8.2.min.js"></script>
<script>
    var socket = io.connect('http://localhost:3000');
    socket.on('connect', function () {
        socket.on('pageview', function (msg) {
            $('#pageViews').append('<li>' + msg.url + '</li>');
        });
    });
</script>
</body> 
</html> 

Testing the Application

  1. Make sure you have Node.js and Socket.IO installed
  2. Start the Node.js server

    node server

  3. Double-click dashboard.html in Finder (Mac) or Explorer (Windows) to open the page in your default browser.
  4. Double-click simplepage.html in Finder or Explorer to open the page in your default browser (in a separate window).
  5. Refresh simplepage.html repeatedly: the dashboard should display each page view.    

Implementation 

Successfully registered as a Windows Azure user, and looking to test the above code in the cloud. I will be using Twitter Bootstrap framework for the fronted Web UI based on the aesthetics of Windows Azure Management Portal. The dashboard interface must be intuitive and user friendly to quickly access the insight data.

Role of Windows Azure 

The key strengths of Windows Azure as a Paas are support for Node.js out of the box and support available for MongoDB through the Addon provided by MongoLab for Azure. 

The key strengths of Windows Azure as an Iaas are the ability to run my own Virtual appliance on top of Azure for running an instance of MongoDB, R and R-Node packages inside my favourite Linux distro(Ubuntu) and the scalability on demand provided by Azure is unmatched compared to the likes of Amazon EC2, Google Compute Engine and others.

And lastly, the interface provided by the Azure Management Portal is awesome. I've never seen a dashboard as simple and sophisticated as that. I derive the design and aesthetics from it. 

Points of Interest 

The most critical part of this article project is to integrate the enormous analytics data to the R data sets. The resulting dataset obtained is used for the generation of Heatmaps, Hotspots, Click Tracking, Mouse movements, demographics and Infographics using Protovis graphical API or HTML5 Canvas.  

History

Phase 1 : Registered as a Windows Azure user and completed posting an article.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Rajiv Charan Tej K
Web Developer Cadsys India
India India
No Biography provided
Follow on   Twitter

Comments and Discussions

 
-- There are no messages in this forum --
| Advertise | Privacy | Mobile
Web03 | 2.8.140827.1 | Last Updated 3 May 2013
Article Copyright 2013 by Rajiv Charan Tej K
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid