Click here to Skip to main content
Click here to Skip to main content
Go to top

Node.Js And Stuff

, 13 Jan 2013
Rate this:
Please Sign up or sign in to vote.
Small demo app using Node.Js/Socket.IO/MongoDB/D3.Js and jQuery
This is an old version of the currently published article.

The demo app code can be download here : Demo App.zip

 


Introduction

It has been a while since I wrote an article here at codeproject, so I  apologize for that. I have been pretty busy doing um nothing I guess.

Anyway late last year I found a bit of spare time, so I looked at my to-do  / must learn list, and sitting there glaring at me was "Node.Js" in 60ft  letters. So I thought ok, why not give Node.js a try. At the time I was  doing a fair amount of ASP MVC / JQuery at work, so thought now would be a  good time to do this, as I kinda already had my head in the web zone at work,  so it would be a good time to give Node.js a go.

So this article is me playing around with Node.js a bit, and coming up  with a small demo project. The demo project is tiny as I did not want to  overload people with too much, that said the choice of demo app I have made,  ensures (at least in my mind it does) that by the time you have read this  article you will get to learn about the following elements

  • Node.Js itself and how a Node.js application is structured (well one  potential way it could be structured, that is)
  • Express which is a MVC type Node.js framework
  • NPM which is Node Package Management
  • Jade view engine
  • Stylus which is a CSS generator that works a lot like the popular  SASS/LESS CSS generators
  • Socket.IO which is a Web Socket package for Node.JS (star of the show  if you ask me)
  • D3.Js which is a data visualization JavaScript library
  • Mongo DB document database Node.js integration

Just for the record Node.js does not need to be used to create web sites. It  can be used just to create servers, it just so happens I have chosen to create a  web site with it. 

 

Brief Node.Js Overview

As I have stated I have decided to write this article using Node.js, so what  exactly is Node.js. Lets see a few descriptions of it before we carry on shall  we:

Node.js is a platform built on Chrome's JavaScript runtime for easily  building fast, scalable network applications. Node.js uses an event-driven,  non-blocking I/O model that makes it lightweight and efficient, perfect for  data-intensive real-time applications that run across distributed devices.

http://nodejs.org/

Node.js is a server side software system designed for writing scalable  Internet applications, notably web servers. Programs are written on the server  side in JavaScript, using event-driven, asynchronous I/O to minimize overhead  and maximize scalability.

Node.js is a packaged compilation of Google's  V8 JavaScript engine, the libUV platform abstraction layer, and a core library,  which is itself primarily written in JavaScript.

Node.js was created by Ryan Dahl starting in 2009, and its growth is  sponsored by Joyent, his employer. Dahl's original goal was to create the  ability to make web sites with push capabilities as seen in web applications  like Gmail. After trying solutions in several other programming languages he  chose JavaScript because of the lack of an existing I/O API. This allowed him to  define a convention of non-blocking, event-driven I/O.

http://en.wikipedia.org/wiki/Nodejs

Here is a trivial node  example, where this is a complete implementation of hello world as an HTTP  Server in Node.js:

var http = require('http');
 
http.createServer(
  function (request, response)
  {
    response.writeHead(200, {'Content-Type': 'text/plain'});
    response.end('Hello World\n');
  }
).listen(8000);
 
console.log('Server running at http://localhost:8000/');

 

JS Everywhere

One of the big things that comes by using Node.Js is that you are effectively  using JavaScript on both the client and server side code bases, so if you are  familiar with JavaScript the promise is that this single language on both sides  of the wire, could speed up your development time quite substantially.


Prerequisites

Before we start the break down of how the demo app works, if you intend to  actually run the demo code you will need to grab a few things before you start.  These are listed below:

1. Mongo DB

You will need to download and unzip Mongo DB, which is available from here (I  chose the Windows 64-bit version, but you will need to pick one that suits your  machine specifics) :
http://www.mongodb.org/downloads

2. Node.js

You will need to download Node.js, which is available from here:
http://nodejs.org/download/


Node.Js Basics

In this section we will try and examine the inner workings of the demo app.  By the end I would hope you know a bit about Node.js and some of its standard  components.

 

Required Packages

Node.Js is geared around packages, kind of like NuGet, where you request a  certain package from the package manager and it will download and place the  relevant package files in the correct location.

Package management in Node.Js is achieved using the "Node.js Command  Prompt", where we use the NPM.exe (Node Package  Manager) to install the packages.

Here is an example (in fact this is the full example of how I set up the demo  codes packages).

These Are The Steps I Followed To Install Node packages (You Will NOT  Have To Do This)

It should be noted that the following steps also create a dummy Express (more  on this later) application shell.

  1. From node command prompt
  2. mkdir WebSocketDemoApp
  3. cd WebSocketDemoApp
  4. express -c stylus
  5. npm install -d
  6. npm install socket.io 
  7. Edit package.json to include Mongo DB ("mongo DB": ">= 0.9.6-7")
  8. npm install -d

When you install a package using NPM.exe you should see that  the packages file "package.json" is updated (you can also tweak  this file a bit if you have very specific requirements, as I did where I wanted  to ensure the Mongo DB driver must be > 0.9.6-7 version)

Package.Json

Here is the resulting packages file "package.json" for the  attached demo app

{
  "name": "application-name" ,
  "version": "0.0.1" ,
  "private": true ,
  "scripts": {
    "start": "node app"
  } ,
  "dependencies": {
    "express": "3.0.0rc4" ,
    "jade": "*" ,
    "stylus": "*" ,
	"Mongo dB": ">= 0.9.6-7"
  }
}

What The Packages Typically Look Like

When Node.Js / NPM.exe have successfully downloaded and  installed packages they will be placed in a "node_modules" folder:

If we examine one of the package folders, say "Express" we can see that all  the relevant files are present

    NOTE : For the demo app I have obviously installed all the relevant packages  for you, so you do not need to do any of that.

So that is essentially how you get packages installed, we will be talking  more about packages as the demo app makes use of several packages, so don't  worry you will see more on how they are used.

 

Express (Node.js Package)

Express is a standard  Node.js package that provides standard features such as :

  • Routing
  • Routing data
  • Rendering
  • HTTP Listener

When you create a new Express application you also get a certain folder structure, and skeleton  application file. Shown below is a typical empty Express project:

We will dissect this a bit further as it will help us when we get to look at  the real demo apps code:

App skeleton

The actual "Node.js application" you get when you create a new Express application is called "pap's" which contains some  skeleton code for you, that provides the following:

  • Configuration
  • Routing
  • Http Listening/Server

Here is what that "app.js" starts out like before it is  changed, as it has been for this articles demo code:


/**
 * Module dependencies.
 */

var express = require('express')
  , routes = require('./routes')
  , user = require('./routes/user')
  , http = require('http')
  , path = require('path');

var app = express();

app.configure(function(){
  app.set('port', process.env.PORT || 3000);
  app.set('views', __dirname + '/views');
  app.set('view engine', 'jade');
  app.use(express.favicon());
  app.use(express.logger('dev'));
  app.use(express.bodyParser());
  app.use(express.methodOverride());
  app.use(app.router);
  app.use(require('stylus').middleware(__dirname + '/public'));
  app.use(express.static(path.join(__dirname, 'public')));
});

app.configure('development', function(){
  app.use(express.errorHandler());
});

app.get('/', routes.index);
app.get('/users', user.list);

http.createServer(app).listen(app.get('port'), function(){
  console.log("Express server listening on port " + app.get('port'));
});

It can be seen that this is mainly broken down into 3 parts

  1. App configuration
  2. Setting up the routing
  3. Creating a Http server and starting to listen

 

Public folder

This contains things that are um well public, such as Images/Javascript/CSS,  where you would put the files into the relevant folder 

Routes folder

This folder contains all the code which manage the routes and their views. So  you would typically have

  • GET/POST handlers
  • Rendering views
  • Building up models

Here is a small example of one of the views you get with a fresh Express application:

/*
 * GET home page.
 */

exports.index = function(req, res){
  res.render('index', { title: 'Express' });
};

Views folder

This folder holds the views for your application. Since Jade  is the default view engine you get with Express all the views you  get with a fresh Express application will be Jade  view files. As we will be covering Jade  in just a moment, we will save that discussion for the next section. 

 

Jade

Jade is one of the many  view engines you can use with Express. It is also happens to be the default one  that you get when you choose to use Node.Js  Express package.

Jade is very  sophisticated in loads of areas, and really dumb in 1. Unfortunately that 1 area  where it is dumb turned out to be a real sticking point for me.

The Pros

This is what the Jade  site list as the features

  • client-side support great readability
  • flexible indentation
  • block-expansion
  • mixins
  • static includes
  • attribute interpolation
  • code is escaped by default for security
  • contextual error reporting at compile & run time
  • executable for compiling jade templates via the command line
  • html 5 mode (the default doctype)
  • optional memory caching
  • combine dynamic and static tag classes
  • parse tree manipulation via filters
  • template inheritance
  • block append / prepend
  • supports Express JS out of the box
  • transparent iteration over objects, arrays, and even non-enumerables via  each
  • block comments
  • no tag prefix
  • filters

I guess it may be better to see an example. So let's consider one of the 2  demo app views, where we can examine the Jade  markup and the HTML it produces

Here is my "layout.jade" page (think master page if you are coming from Microsoft land  like I do)

doctype 5
html
  head
  link(rel='stylesheet', href='http://www.codeproject.com/stylesheets/style.css')
  script(src='http://www.codeproject.com/javascripts/jquery-1.8.2.min.js')
  script(src='http://www.codeproject.com/javascripts/d3.v2.js')
  link(rel='stylesheet', href='http://www.codeproject.com/jquery-ui-1.9.1.custom/css/ui-lightness/jquery-ui-1.9.1.custom.css')
  script(src='http://www.codeproject.com/jquery-ui-1.9.1.custom/js/jquery-ui-1.9.1.custom.js')
  block head
body
  div(id='mainContent')
    img(src='http://www.codeproject.com/images/Header.png')
    block content

And here is the demo apps view "home.jade" which is an extending view (that's Jade's way of dealing with Master page like functionality).

Pretty precise huh?

extends layout

block head
  link(rel='stylesheet', href='http://www.codeproject.com/stylesheets/home.css')
  script(src='http://www.codeproject.com/javascripts/home.js')

block content
  form(method='post', id='homeForm', action='http://localhost:2000/home')
    div(id='dialog', title='error', style='display:none;')
      p(id='homeWarn') You need to supply a valid email
    div(id='NewDetailsArea')
      p Enter your email address, and then click enter 
      | <input type='text' id='email' name='email' class='email'></input>
      div#homeSubmit
        input(type='submit', value='Enter', id='enterEmail')

And here is what is produced:

<!DOCTYPE html>
<html>
<head>
</head>
<link rel="stylesheet" href="http://www.codeproject.com/stylesheets/style.css">
<script src="http://www.codeproject.com/javascripts/jquery-1.8.2.min.js"></script>
<script src="http://www.codeproject.com/javascripts/d3.v2.js"></script>
<link rel="stylesheet" href="http://www.codeproject.com/jquery-ui-1.9.1.custom/css/ui-lightness/jquery-ui-1.9.1.custom.css">
<script src="http://www.codeproject.com/jquery-ui-1.9.1.custom/js/jquery-ui-1.9.1.custom.js"></script>
<link rel="stylesheet" href="http://www.codeproject.com/stylesheets/home.css">
<script src="http://www.codeproject.com/javascripts/home.js"></script>
</html>
<body>
    <div id="mainContent">
        <img src="http://www.codeproject.com/images/Header.png"><form method="post" id="homeForm" action="http://localhost:2000/home">
        <div id="dialog" title="error" style="display: none;">
            <p id="homeWarn">
                You need to supply a valid email</p>
        </div>
        <div id="NewDetailsArea">
            <p>
                Enter your email address, and then click enter
            </p>
            <input type='text' id='email' name='email' class='email'></input><div id="homeSubmit">
                <input type="submit" value="Enter" id="enterEmail"></div>
        </div>
        </form>
    </div>
</body>

I can't really go into every aspect of Jade, if you want to know  more you can read about it here : https://github.com/visionmedia/jade#readme

 

The 1 Big Con (At least in my opinion)

Jade uses semantic white  space, and it is very very fussy about mixing tabs/spaces. Even when I was a  gazillion % sure I had used all spaces, it would complain. And when it complains  all you get rendered instead of the view is a crappy Exception message.

Now it could be more you use it the better you get with it. I just found I  was forever getting these Exceptions, and was at a real loss to fix them at  times. It felt like they got fixed more by luck than anything else.

Being fair to Jade this  is the only complaint I have, it really is a very capable view engine.

 

Stylus

Now I don't profess to like CSS very much, but being a software developer am  interested in ALL aspects of software design, so I have taken note of things  like SASS/LESS for greatly improving how you work with CSS. Where these  frameworks provide  things like

  • Variables within CSS
  • Nested CSS declarations (more explicit)
  • Mixins
  • Scoping

As it turns out there is a very funky package you can use with Node.js that  you can choose to use with the  Express package, which is what  we get when we used "NPM.exe" and issued the command "npm express -c stylus"  which we talked about earlier. As you may have guessed from that command line, it is called "Stylus"and  it does most of the things listed above and more.

Here is an example of one of the demo apps "Stylus" files,  and the resultant CSS files. I am not using all the tricks, but rest assured "Stylus"  is very capable, just check  learnboost.github.com/stylus/ for more details

 

.Styl file

Here is the raw .Styl file, notice how I am using global variables there, and  that there are no semi-colons at all

foreground-color = white
background-color = black
font =  12px Verdana
general-margin = 10px

body
  padding: 50px
  font: font 
  font-weight:normal
  background-color: background-color
  width: 100%
  height: 100%
  margin: none
  padding: none

a
  color: #00B7FF

.popupText
 color : background-color

#summary
	 text-align: center

p 
 color : foreground-color

input[type="submit"], input[type="button"] 
  border: 1px solid black
  padding: 4px
  width: 120px
  margin-top: general-margin 
  margin-right: general-margin 
  height:23px
  font: font 
  foreground : foreground-color

input[type="text"] 
  height:23px
  font-weight:normal

.nodetext
  pointer-events: none
  font: font
  fill : foreground-color

.circle 
 z-index:999

.link
  stroke: foreground-color

#graph
  width: 1000px
  height: 400px
  margin-left: auto
  margin-right: auto
  border: 1px dashed foreground-color

#mainContent
  width: 1000px
  height: 600px
  margin-left: auto
  margin-right: auto

#NewDetailsArea
  margin-left: 15px
  margin-top: general-margin

#email, #NewPersonEmail, #sourcePeople, #targetPeople 
    width : 350px
    margin-left: 0px
    margin-top: general-margin
    margin-bottom: general-margin
    margin-right: general-margin

  

Resultant .css file

And here the resulting CSS file

body {
  padding: 50px;
  font: 12px Verdana;
  font-weight: normal;
  background-color: #000;
  width: 100%;
  height: 100%;
  margin: none;
  padding: none;
}
a {
  color: #00b7ff;
}
.popupText {
  color: #000;
}
#summary {
  text-align: center;
}
p {
  color: #fff;
}
input[type="submit"],
input[type="button"] {
  border: 1px solid #000;
  padding: 4px;
  width: 120px;
  margin-top: 10px;
  margin-right: 10px;
  height: 23px;
  font: 12px Verdana;
  foreground: #fff;
}
input[type="text"] {
  height: 23px;
  font-weight: normal;
}
.nodetext {
  pointer-events: none;
  font: 12px Verdana;
  fill: #fff;
}
.circle {
  z-index: 999;
}
.link {
  stroke: #fff;
}
#graph {
  width: 1000px;
  height: 400px;
  margin-left: auto;
  margin-right: auto;
  border: 1px dashed #fff;
}
#mainContent {
  width: 1000px;
  height: 600px;
  margin-left: auto;
  margin-right: auto;
}
#NewDetailsArea {
  margin-left: 15px;
  margin-top: 10px;
}
#email,
#NewPersonEmail,
#sourcePeople,
#targetPeople {
  width: 350px;
  margin-left: 0px;
  margin-top: 10px;
  margin-bottom: 10px;
  margin-right: 10px;
}

It can be seen that the global variables have been substituted where needed, and all semi-colors, braces etc etc have been rendered. What happens when you run a Node.Js/Express app that has been configured to  use "Stylus" is that the "Stylus" files *.styl are transformed into standard *.css files.



The Demo App

This section talks about what the demo app does, and also talks you through how  to get the demo app up and running on your own box.

What Does The Demo App Do?

In theory what the demo app tries to do is dead simple, and can be broken  down into the following steps:

  1. Hosts a Socket.IO WebSocket server
  2. Provide 2 routes (valid known Urls that will serve content) that a  connected client can use
    • /home :
      • Which has a GET, which allows the home page to be viewed. The  1st time this route is called some initial seed data will be placed  into the application MongoDB database (Providing there is not  already some data present)
      • Which also has a POST which will accept the entered Email  address for a new user, and will save that in the application  MongoDB database, and the redirect the user to the /d3Demo route
    • /d3Demo
      • Which has a GET which simply grabs all the previously stored  Person and Link objects from the application MongoDB database and  shows this in a D3.js Force type (think spring connected) graph  layout
      • Has a button to add a new Person, which will launch a jQuery UI  dialog to allow the new details to be entered. If entered correctly  shall call the Socket.IO "addNewPersonMessage" message  to ask the node application server to add a new Person to the  application MongoDB database
      • Has a button to add a new Link, which will launch a jQuery UI  dialog to allow the new details to be entered. If entered correctly  shall call the Socket.IO "addNewLinkMessage" message to  ask the node application server to add a new Person to the  application MongoDB database
      • Initialises a Socket.IO WebSocket client connection back to the  applications Socket.IO WebSocket server, which allows the following  messages and callbacks to be sent and receieved
        • addNewPersonMessage : Message from client to  server to add new Person object to application MongoDB database
        • addNewLinkMessage : Message from client to  server to add new Link object to application MongoDB database
        • newPersonCallback : Message from server to ALL  clients, which allows client(s) to respond to the new Person  that has been added. In this demo this simply means adding the  new Person to the D3.js Force type (think spring connected)  graph layout
        • newLinkCallback : Message from server to ALL  clients, which allows client(s) to respond to the new Link that  has been added. In this demo this simply means adding the new  Link to the D3.js Force type (think spring connected) graph  layout

If that is all to hard to understand this diagram may help to illustrate this further (click the image for  larger version) :

 

Running The Demo App

  1. Make sure you have started "Mongod.exe" process from  wherever you unzipped the MongoDB download to (for me "mongodb-win32-x86_64-2.2.1\bin\mongod.exe"  is the location for the EXE). If you want to start with a clean database to  see how things work the MongoDB files are available at the "C:\data\db"  directory, where the demo app will create a database (providing you don't  change anything) called "node-mongo-socketDemo.XXXX", so  just delete all those file with that file name
  2. Using the "Node.Js command prompt" from where Node.Js  installed for you (usually gives you a menu item for this)  navigate to  the folder where you downloaded
    this articles code to, and issue the  following command to the Node.js command line : "node app"

If you have successfully started the demo app you should see something like  this:

 

Demo App Structure

The demo app is made up of several folders/parts which can be seen in the  screen shot below:

I will endevour to go through these parts within the rest of this article

 

The Demo App Itself

As I have stated the attached demo app makes use of Express which we went  through in the Node.Js sections above, it should hopefully be a little bit  familiar.

So here is the entire demo app ("app.js") code. Which I will  describe in a bit of detail:

var express = require('express');
var home = require('./routes/home');
var d3demo = require('./routes/d3demo');
var PersonProvider = require('./public/javascripts/personProvider').PersonProvider;
var personProvider = new PersonProvider('localhost', 27017);
var LinkProvider = require('./public/javascripts/linkProvider').LinkProvider;
var linkProvider = new LinkProvider('localhost', 27017);
var CommonHelper = require('./public/javascripts/commonHelper').CommonHelper;
var commonHelper = new CommonHelper();

var http = require('http');
var path = require('path');
var app = express();

//=============================================================================
//    EXPRESS SETUP
//=============================================================================
app.configure(function(){
  app.set('port', process.env.PORT || 2000);
  app.set('views', __dirname + '/views');
  app.set('view engine', 'jade');
  app.use(express.favicon());
  app.use(express.logger('dev'));
  app.use(express.bodyParser());
  app.use(express.methodOverride());
  app.use(app.router);
  app.use(require('stylus').middleware(__dirname + '/public'));
  app.use(express.static(path.join(__dirname, 'public')));
});

app.configure('development', function () {
    app.use(express.errorHandler({ dumpExceptions: true, showStack: true }));
});

app.configure('production', function () {
    app.use(express.errorHandler());
});


//=============================================================================
//    ROUTING
//=============================================================================
app.get('/home', function (req, res) {
    home.homeGet(req, res, commonHelper, personProvider, linkProvider);
});

app.post('/home', function (req, res) {
    home.homePost(req, res, personProvider);
});

app.get('/d3demo', function (req, res) {
    d3demo.d3demoGet(req, res, commonHelper, personProvider, linkProvider);
});


var server = http.createServer(app);
var io = require('socket.io').listen(server);

server.listen(app.get('port'), function(){
    console.log("Express server listening on port " + app.get('port'));
});


//=============================================================================
//    SOCKETS CREATION
//=============================================================================
io.sockets.on('connection', function (socket) {

    socket.on('addNewPersonMessage', function (data) {
        console.log("addNewPersonMessage recieved person (email:'" + data.email + "')");

        personProvider.save({
            email: data.email
        }, function (error, docs) {
            if(error == null) {
                console.log('Sucessfully saved new person');
                console.log(docs[0]);
                io.sockets.emit('newPersonCallback', { _id: docs[0]._id, email: docs[0].email });
            } 
        });
    });


    socket.on('addNewLinkMessage', function (data) {
        console.log("addNewLinkMessage recieved link (source:'" + data.source + "', target:'" + data.target + "')");
        linkProvider.save({
            source: data.source,
            target: data.target
         }, function (error, docs) {
            if(error == null) {
                console.log('Sucessfully saved new link');
                console.log(docs[0]);
                io.sockets.emit('newLinkCallback', { _id: docs[0]._id, source: docs[0].source, target: docs[0].target });
            } 
        });
    });
});

It can be seen that the code above pretty much achieves the following  functions:

  • Creates a couple of MongoDBProviders (which will see more on later)
  • Configures Express
  • Creates the Http server
  • Creates a Socket.IO  WebSocket server, and sets up message handlers and callbacks to the clients

I think most of this is self explanatory, but I we will be covering some of  it, such as the routes and how the Socket.IO WebSockets stuff works in more detail

 

Demo App Screens And How They Work

Assuming you have managed to get the demo app up and running you should now  be able to browse using the following Url  :

Home Route

http://localhost:2000/home : Which should show you something like this:

From here you can fill in a email address (which will be validated as being  correctly formatted) and then click enter to proceed to the next screen. I use  JQuery UI to show validation errors.

Recall from earlier that this is what this route does

  • It has a GET, which allows the home page to be viewed. The  1st time this route is called some initial seed data will be placed  into the application MongoDB database (Providing there is not  already some data present)
  • It also has a POST which will accept the entered Email  address for a new user, and will save that in the application  MongoDB database, and the redirect the user to the /d3Demo route

The Home route is handled by the following code

/*
 * GET home page : Seed the MongoDB data on 1st GET request
 */

exports.homeGet = function(req, res, commonHelper, personProvider, linkProvider){
        commonHelper.seedData(personProvider, linkProvider, function() {
        res.render('home');
    });
};


/*
 * POSTED home page : Saved its the state in MongodDB
 * and redirect to D3Demo route
 */

exports.homePost = function (req, res, personProvider) {
    var newUserEmail = req.body.email;
    console.log("/Home posted Email :" + newUserEmail);
    personProvider.save({
        email: newUserEmail,
    }, function (error, docs) {
        if(error == null) {
            res.redirect('/d3demo');
        } else {
            res.render('home');
        }
    });
};

I think most of this explained in the comments above, apart from how the data  is seeded which is done as follows:

CommonHelper = function () {
};


CommonHelper.prototype.getPersistedDataCount = function (provider, callback) {
    provider.findAll(function (error, docs) {
        var data = 0;
        if (error == null) {
            callback(docs.length);
        }
        else {
            callback(data);
        }
    });
}


CommonHelper.prototype.seedData = function (personProvider, linkProvider, callback) {
    personProvider.findAll(function (error, docs) {
        var data = 0;
        if (error == null) {
            if (docs.length == 0) {
                seedPeople(personProvider, function () {
                    seedLinks(linkProvider, callback);
                });
            } else {
                callback();
            }
        }
        else {
            callback();
        }
    });
}

function seedPeople(personProvider, callback) {
    console.log('No existing users were found so seeding people data');
    personProvider.save(
        [
            { "email": "andy.monks@gmail.com" },
            { "email": "ryanW@gmail.com" },
            { "email": "sarah2008@gmail.com" },
            { "email": "sarahb@hotmail.com" },
            { "email": "sachabarber@gmail.co.uk" }
        ]
        , function (error, docs) {
            if (error != null) {
                console.log("There was an error seeding the data, bugger all can be done....ooops");
            }
            else {
                console.log('There are now ' + docs.length + ' users');
            }
            callback();
        });
}

function seedLinks(linkProvider, callback) {
    console.log('No existing links were found so seeding link data');
    linkProvider.save(
        [
            { "source": "andy.monks@gmail.com", "target": "ryanW@gmail.com" },
            { "source": "andy.monks@gmail.com", "target": "sarah2008@gmail.com" },
            { "source": "andy.monks@gmail.com", "target": "sachabarber@gmail.co.uk" }
        ]
        , function (error, docs) {
            if (error != null) {
                console.log("There was an error seeding the data, bugger all can be done....ooops");
            }
            else {
                console.log('There are now ' + docs.length + ' links');
            }
            callback();
        });
}

exports.CommonHelper = CommonHelper;

The only other thing that really needs further explanation is  how the MongoDB database providers work.

This is shown below for the PersonProvider where this code is based on the code found  in this article : http://howtonode.org/express-mongodb

var Db = require('mongodb').Db;
var Connection = require('mongodb').Connection;
var Server = require('mongodb').Server;
var BSON = require('mongodb').BSON;
var ObjectID = require('mongodb').ObjectID;

PersonProvider = function (host, port) {
    this.db = new Db('node-mongo-socketDemo', new Server(host, port, { safe:true, auto_reconnect: true }, {}));
    this.db.open(function () { });
};

PersonProvider.prototype.getCollectionSafe = function (callback) {
    this.db.collection('people', { safe: true }, function (error, person_collection) {
        if (error) callback(error);
        else callback(null, person_collection);
    });
};

PersonProvider.prototype.getCollection = function (callback) {
    this.db.collection('people', function (error, person_collection) {
        if (error) callback(error);
        else callback(null, person_collection);
    });
};

PersonProvider.prototype.findAll = function (callback) {
    this.getCollection(function (error, person_collection) {
        if (error) callback(error)
        else {
            person_collection.find().toArray(function (error, results) {
                if (error) callback(error)
                else callback(null, results)
            });
        }
    });
};

PersonProvider.prototype.findById = function (id, callback) {
    this.getCollection(function (error, person_collection) {
        if (error) callback(error)
        else {
            person_collection.findOne({ _id: person_collection.db.bson_serializer.ObjectID.createFromHexString(id) }, 
                function (error, result) {
                    if (error) callback(error)
                    else callback(null, result)
            });
        }
    });
};

PersonProvider.prototype.save = function (people, callback) {
    this.getCollection(function (error, person_collection) {
        if (error) callback(error)
        else {
            if (typeof (people.length) == "undefined")
                people = [people];
            person_collection.insert(people, function () {
                callback(null, people);
            });
        }
    });
};




exports.PersonProvider = PersonProvider;

The LinkProvider is exactly the same code but it deals with a  different collection.

 

 

D3Demo Route

This is the route that will you will be directed to when you come from the  home route. It should be noted that the 1st time you go through the home route  some seed data is placed in the Mongo DB database, such that the demo has some  nice data to see the 1st time you run it.

You are however completely fine to navigate to this url at any time you like,  and it should show you the uber cut down social network graph something like  that shown below,  which you can add to using the buttons provided

http://localhost:2000/d3demo : Which should show you something like this  (click the image for larger version) :

Recall from earlier that this is what this route does

  • Has a GET which simply grabs all the previously stored  Person and Link objects from the application MongoDB database and  shows this in a D3.js Force type (think spring connected) graph  layout
  • Has a button to add a new Person, which will launch a jQuery UI  dialog to allow the new details to be entered. If entered correctly  shall call the Socket.IO "addNewPersonMessage" message  to ask the node application server to add a new Person to the  application MongoDB database
  • Has a button to add a new Link, which will launch a jQuery UI  dialog to allow the new details to be entered. If entered correctly  shall call the Socket.IO "addNewLinkMessage" message to  ask the node application server to add a new Person to the  application MongoDB database
  • Initialises a Socket.IO WebSocket client connection back to the  applications Socket.IO WebSocket server, which allows the following  messages and callbacks to be sent and receieved
    • addNewPersonMessage : Message from client to  server to add new Person object to application MongoDB database
    • addNewLinkMessage : Message from client to  server to add new Link object to application MongoDB database
    • newPersonCallback : Message from server to ALL  clients, which allows client(s) to respond to the new Person  that has been added. In this demo this simply means adding the  new Person to the D3.js Force type (think spring connected)  graph layout
    • newLinkCallback : Message from server to ALL  clients, which allows client(s) to respond to the new Link that  has been added. In this demo this simply means adding the new  Link to the D3.js Force type (think spring connected) graph  layout

The basic D3Demo route is a simple GET which returns a model which is made up  of an array of Person and their count, and Links and their count

/*
 * GET D3 gets a model which has an array of Person and their count, and Links and their count
 */

exports.d3demoGet = function (req, res, commonHelper, personProvider, linkProvider) {
    personProvider.findAll(function (errorPeople, people) {
        if (errorPeople == null) {
            var peopleCount=people.length;
            var peopleJson = JSON.stringify(people);
            linkProvider.findAll(function (errorLinks, links) {
                if (errorLinks == null) {
                    var linksCount=links.length;
                    var linksJson = JSON.stringify(links);
                    res.render('d3demo', 
                    { 
                        peopleCount: peopleCount, 
                        people: peopleJson,  
                        linksCount: linksCount, 
                        links: linksJson  
                    });
                }
                else {
                    res.render('d3demo', 
                        { 
                            peopleCount: peopleCount, 
                            people: peopleJson,  
                            linksCount: 0, 
                            links: JSON.stringify([]),  
                        });
                }
            });
        }
        else {
            res.render('d3demo', 
                { 
                    peopleCount: 0, 
                    people: JSON.stringify([]),  
                    linksCount: 0, 
                    links: JSON.stringify([]),  
                });
        }
    });
};

 

When the page is rendered it will create a D3.JS force directed graph.  Here is the code that creates the graph of the model data.

function createGraph() {

    graph = new myGraph("#graph");

    var people = $.parseJSON($('#people').val());
    peopleCount = people.length;
    for (var i = 0; i < people.length; i++) {
        graph.addNode(people[i]._id, people[i].email);
    }

    var links = $.parseJSON($('#links').val());
    linksCount = links.length;
    for (var i = 0; i < links.length; i++) {
        graph.addLink(links[i].source, links[i].target);
    }
}

function myGraph(el) {

    // Add and remove elements on the graph object
    this.addNode = function (id, email) {
        nodes.push({ "id": id, "email": email });
        update();
    }

    this.removeNode = function (email) {
        var i = 0;
        var n = findNode(email);
        while (i < links.length) {
            if ((links[i]['source'] == n) || (links[i]['target'] == n)) links.splice(i, 1);
            else i++;
        }
        nodes.splice(findNodeIndex(email), 1);
        update();
    }

    this.addLink = function (source, target) {
        links.push({ "source": findNode(source), "target": findNode(target) });
        update();
    }

    var findNode = function (email) {
        for (var i in nodes) { if (nodes[i]["email"] === email) return nodes[i] };
    }

    var findNodeIndex = function (email) {
        for (var i in nodes) { if (nodes[i]["email"] === email) return i };
    }

    // set up the D3 visualisation in the specified element
    var w = $(el).innerWidth(),
            h = $(el).innerHeight();

    var vis = this.vis = d3.select(el).append("svg:svg")
                .attr("width", w)
                .attr("height", h);

    var force = d3.layout.force()
                .gravity(.05)
                .distance(100)
                .charge(-100)
                .size([w, h]);

    var nodes = force.nodes(),
                links = force.links();

    var update = function () {
        var link = vis.selectAll("line.link")
                .data(links, function (d) { return d.source.id + "-" + d.target.id; });
        link.enter().insert("line")
                .attr("class", "link");
        link.exit().remove();
        var node = vis.selectAll("g.node")
                .data(nodes, function (d) { return d.email; });

        var nodeEnter = node.enter().append("g")
                .attr("class", "node")
                .call(force.drag);

        nodeEnter.append("image")
                .attr("class", "circle")
                .attr("xlink:href", "../Images/Friend.png")
                .attr("x", "-8px")
                .attr("y", "-8px")
                .attr("width", "40px")
                .attr("height", "40px");

        nodeEnter.append("text")
                .attr("class", "nodetext")
                .attr("dx", 12)
                .attr("dy", ".35em")
                .text(function (d) { return d.email });
        node.exit().remove();

        force.on("tick", function () {
            link.attr("x1", function (d) { return d.source.x; })
                    .attr("y1", function (d) { return d.source.y; })
                    .attr("x2", function (d) { return d.target.x; })
                    .attr("y2", function (d) { return d.target.y; });

            node.attr("transform", function (d) { return "translate(" + d.x + "," + d.y + ")"; });
        });

        // Restart the force layout.
        force.start();
    }

    // Make it all go
    update();
}

I can't really go into a blow by blow account of how to use D3.Js but it has very good demos  and good documentation at http://d3js.org/. If you have not  come across D3.Js you should  really check it out, as it realyll it quite a mad library, which has the following  features to name a few:

  • jQuery like selectors
  • SVG Based graphics
  • Dynamic layouts

There is also a very good D3.Js tutorial here by Espen Harlinn : http://www.codeproject.com/Articles/523444/D3-crash-course which I recommend you read 

From the d3Demo page you are also able to manipulate your own session, and  submit edits back to the server via a WebSocket which will be broadcast to other  clients connected to the appliations server socket.

Lets continue to have a look at how you can use the WebSockets

Client Side WebSocket Comms

We have already seen the server side portion of the  Socket.IO WebSocket setup  which was done as follows:

io.sockets.on('connection', function (socket) {

    socket.on('addNewPersonMessage', function (data) {
        console.log("addNewPersonMessage recieved person (email:'" + data.email + "')");

        personProvider.save({
            email: data.email
        }, function (error, docs) {
            if(error == null) {
                console.log('Sucessfully saved new person');
                console.log(docs[0]);
                io.sockets.emit('newPersonCallback', { _id: docs[0]._id, email: docs[0].email });
            } 
        });
    });




    socket.on('addNewLinkMessage', function (data) {
        console.log("addNewLinkMessage recieved link (source:'" + data.source + "', target:'" + data.target + "')");
        linkProvider.save({
            source: data.source,
            target: data.target
         }, function (error, docs) {
            if(error == null) {
                console.log('Sucessfully saved new link');
                console.log(docs[0]);
                io.sockets.emit('newLinkCallback', { _id: docs[0]._id, source: docs[0].source, target: docs[0].target });
            } 
        });
    });
});

So how does this server side socket code interact with the client side  Javascript code. Well that is all handled within the "socketComms.js" client  side code file, where the socket is setup as follows:

$(document).ready(function () {

    var socket = io.connect('http://localhost');
    socket.on('newPersonCallback', onNewPersonCallback);
    socket.on('newLinkCallback', onNewLinkCallback);
}

 

Add Person

Which will show a jQuery UI dialog which allows a new Person to be addeed,  which if sucessfully added to the Mongo DB database will be broadcast to everyone using the Node.js  Socket.IO package,  which provides WebSocket communications between server and client.

It can be seen from this dialog that a new person (for the uber simple social  network) requires a email. Essentially assuming that the the user picks the  correct data the following code is called which calls the server side socket  code, which results in a new Person being persisted to the database. This new  Person is then broadcast to ALL connected server socket clients, where they ALL  handle the callback, and add the new Person to their own graphs.

The important line below is the "Socket.Emit(..)" which is the client writing  data to the server socket

$("#new-people-dialog").dialog({
    autoOpen: false,
    height: 250,
    width: 500,
    title: 'Enter a email for the new person',
    modal: true,
    buttons: {
        "Add email": function () {
            var isValid = true;
            var email = $('#NewPersonEmail');
            allFields = $([]).add(email);
            allFields.removeClass("ui-state-error");
            isValid = isValid && checkLength(email, 6, 80);
            // From jquery.validate.js (by joern), contributed by Scott Gonzalez: http://projects.scottsplayground.com/email_address_validation/
            isValid = isValid && checkRegexp(email, ......);
            if (isValid) {
                var emailValue = email.val();
                socket.emit('addNewPersonMessage', { email: emailValue });
                $(this).dialog("close");
            }
        },
        Cancel: function () {
            $(this).dialog("close");
        }
    },
    close: function () {
        allFields.val("").removeClass("ui-state-error");
    }
});

function onNewPersonCallback(data) {
    peopleCount = peopleCount + 1;
    createSummary();
    graph.addNode(data._id, data.email);
}

And here is the relevant server side socket code

socket.on('addNewPersonMessage', function (data) {
    console.log("addNewPersonMessage recieved person (email:'" + data.email + "')");

    personProvider.save({
        email: data.email
    }, function (error, docs) {
        if(error == null) {
            console.log('Sucessfully saved new person');
            console.log(docs[0]);
            io.sockets.emit('newPersonCallback', { _id: docs[0]._id, email: docs[0].email });
        } 
    });
});

 

Add Link

Which will show a jQuery UI dialog which allows a new Linke between 2  existing people to be addeed, which is broadcast to everyone using the Node.js  Socket.IO package

It can be seen from this dialog that a new link (for the uber simple social  network) requires 2 existing people (from existing people). Essentially assuming  that the the user picks the correct data the following code is called which  calls the server side socket code, which results in a new Link being persisted  to the database. This new Link is then broadcast to ALL connected server socket  clients, where they ALL handle the callback, and add the new Link to their own  graphs.

The important line below is the "Socket.Emit(..)" which is the client writing  data to the server socket

$("#new-links-dialog").dialog({
    autoOpen: false,
    height: 300,
    width: 500,
    title: 'Enter a link between existing people',
    modal: true,
    buttons: {
        "Add link": function () {
            var isValid = true;
            var sourcePeople = $('#sourcePeople');
            var targetPeople = $('#targetPeople');

            allFields = $([]).add(sourcePeople, targetPeople);
            allFields.removeClass("ui-state-error");
            isValid = isValid && checkHasSelection('#sourcePeople option:selected');
            isValid = isValid && checkHasSelection('#targetPeople option:selected');
            if (isValid) {
                var sourcePerson = $('#sourcePeople option:selected').text();
                var targetPerson = $('#targetPeople option:selected').text();
                socket.emit('addNewLinkMessage', { source: sourcePerson, target: targetPerson });
                $(this).dialog("close");
            }
        },
        Cancel: function () {
            $(this).dialog("close");
        }
    },
    close: function () {
        allFields.val("").removeClass("ui-state-error");
    }
});


function onNewLinkCallback(data) {
    console.log('onNewLinkCallback');
    console.log(data);
    linksCount = linksCount + 1;
    createSummary();
    graph.addLink(data.source, data.target);
}

And here is the relevant server side socket code

socket.on('addNewLinkMessage', function (data) {
    console.log("addNewLinkMessage recieved link (source:'" + data.source + "', target:'" + data.target + "')");
    linkProvider.save({
        source: data.source,
        target: data.target
        }, function (error, docs) {
        if(error == null) {
            console.log('Sucessfully saved new link');
            console.log(docs[0]);
            io.sockets.emit('newLinkCallback', { _id: docs[0]._id, source: docs[0].source, target: docs[0].target });
        } 
    });
});

My Final Thoughts

The Good

I have to say I really enjoyed creating this article/small demo application,  and I found Node.js to very well thought out, and it does have a great community  behind it, and there are many many great packages out there. Generally I would  get stuck (yes that happened a few times), but the answer would generally be  pretty obvious, and if I really got stuck which I did once, I would find the  answer on StackOverflow.com pretty easily. One of the things I really liked  about Node.js is just how light weight it is.

So that's the good what about the bad. Sadly there was one bad apple, at  least for me there was.

The Bad

One area where I really struggled was using the Jade view engine, which as I  have stated uses semantic white space. I would find that I would be using a text  editor and putting in what I thought to be quite reasonable spaces, and I would  fire the app up, and I would just end up with an Exception being shown in place  of my desired HTML contents. When the Exception would state "You can not mix  spaces and tabs", to my mind I had not used any tabs. This was really really  tricky to work with, which is a shame as Jade itself its mad and very feature  rich, and in many ways is light years ahead of other view engines such as  Microsoft Razor which is used in ASP MVC. I just have to question the sanity of  anything that uses semantic white space.

I guess this could be fixed by using the correct editor, and being more familiar  with Jade I guess. Problem was I really could not see what the Exception was  talking about

The Outstanding Questions I Still Have

I still have a few unanswered questions about Node.js, which are out lined  below. Feel free to answer these in this articles forum if you feel you have  some opinions on them

  1. How do you manage all the related files for a fully Node.Js application,  as there is not really any idea that brings it all together in a single  project. Sure I could use a nice text editor such as Sublime text editor,  but that is still opening 1 file at a time. I am looking for an integrated   IDE that manages a Node.js project. I resorted to creating a dummy VS2010  solution and dragging all my Node.js apps files into it, such that I could  manage it all by opening one single file (the VS2010 solution file).
  2. What text editor do people use wih Jade files. As stated above this was  a definite sticking point for me
  3. I am still not clear what constitutes a good/useful/appropriate Node.Js  appllication. Sure Node,js talks about fast, scalable network applications,  but when you out HTML views and the like on top of that, it seems to be just  like any other HTML solution that supports some sort of web sockets /  SignalR (where SignalR is actually using Socket.IO anyway) comms. I guess  what I am asking is what is a real god example of an application that would  be a good fit for Node.js

As I say if some of you reading this feel you have answers to these questions,  please make them known in this articles forum

 

That's It

I hope you have enjoyed this article and have got something out of it. I have  quite enjoyed writing this one, and think it turned out ok. If you did enjoy it,  a vote or a comment are most welcome. 

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Sacha Barber
Software Developer (Senior)
United Kingdom United Kingdom
I currently hold the following qualifications (amongst others, I also studied Music Technology and Electronics, for my sins)
 
- MSc (Passed with distinctions), in Information Technology for E-Commerce
- BSc Hons (1st class) in Computer Science & Artificial Intelligence
 
Both of these at Sussex University UK.
 
Award(s)

I am lucky enough to have won a few awards for Zany Crazy code articles over the years

  • Microsoft C# MVP 2014
  • Codeproject MVP 2014
  • Microsoft C# MVP 2013
  • Codeproject MVP 2013
  • Microsoft C# MVP 2012
  • Codeproject MVP 2012
  • Microsoft C# MVP 2011
  • Codeproject MVP 2011
  • Microsoft C# MVP 2010
  • Codeproject MVP 2010
  • Microsoft C# MVP 2009
  • Codeproject MVP 2009
  • Microsoft C# MVP 2008
  • Codeproject MVP 2008
  • And numerous codeproject awards which you can see over at my blog

Comments and Discussions


Discussions posted for the Published version of this article. Posting a message here will take you to the publicly available article in order to continue your conversation in public.
 
QuestionBroken link Pinprotectorthatraja22-Jul-14 1:01 
GeneralJade Whitespace complaints PinprotectorDaveAuld29-Jan-14 11:15 
GeneralRe: Jade Whitespace complaints PinmvpSacha Barber29-Jan-14 11:35 
GeneralRe: Jade Whitespace complaints PinprotectorDaveAuld30-Jan-14 1:27 
GeneralRe: Jade Whitespace complaints PinmvpSacha Barber30-Jan-14 1:56 
QuestionIDE's PinmemberHansHammel2-May-13 22:10 
QuestionCannot GET / [modified] Pinmembersatuday18-Apr-13 7:10 
AnswerRe: Cannot GET / PinmvpSacha Barber18-Apr-13 7:15 
AnswerRe: Cannot GET / Pinmemberartapo23-Apr-13 11:50 
BugRe: Cannot GET / PinmemberPie At The Raven14-May-13 1:29 
AnswerRe: Cannot GET / Pinmemberkanary18-Oct-13 12:01 
GeneralRe: Cannot GET / PinmvpSacha Barber18-Oct-13 18:50 
QuestionTry knockout also ! Pinmembergisvlasta14-Mar-13 18:32 
AnswerRe: Try knockout also ! PinmvpSacha Barber14-Mar-13 20:45 
QuestionRegarding sublime Pinmembergisvlasta14-Mar-13 18:23 
AnswerRe: Regarding sublime PinmvpSacha Barber14-Mar-13 20:39 
GeneralMy vote of 5 PinmemberL Hills11-Mar-13 6:56 
GeneralRe: My vote of 5 PinmvpSacha Barber11-Mar-13 7:30 
Questionvery good paper Pinmembermarco lugano4-Mar-13 8:55 
AnswerRe: very good paper PinmvpSacha Barber4-Mar-13 10:03 
GeneralMy vote of 5 PinmemberDr.Luiji4-Mar-13 2:25 
GeneralRe: My vote of 5 PinmvpSacha Barber4-Mar-13 3:41 
GeneralMy vote of 5 PinmemberMihai MOGA16-Feb-13 19:25 
GeneralRe: My vote of 5 PinmvpSacha Barber16-Feb-13 19:59 
QuestionMy 5 PinmemberDee_Bee15-Feb-13 19:02 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web01 | 2.8.140916.1 | Last Updated 13 Jan 2013
Article Copyright 2013 by Sacha Barber
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid