Click here to Skip to main content
65,938 articles
CodeProject is changing. Read more.
Articles / web / HTML

Making Dashboards with Dc.js - Part 1: Using Crossfilter.js

4.84/5 (36 votes)
21 Jan 2014CPOL5 min read 218K  
Using crossfilter.js to manipulate Javascript arrays

Introduction

Dc.js is a JavaScript library used to make interactive dashboards in JavaScript. By clicking and selecting different events in graphs, you can filter the entire dashboard to drill into a particular event.

Image 1

This is the first of a 4 part series. You can find my other articles here:

Background

In order to do this, DC.js relies on two other JavaScript plugins/libraries: D3.js and Crossfilter js.

D3.js is the evolution of Protovis. Up until a few years ago, most browser-based graphs were static (Protovis) or based on a non-JavaScript plugin like Flash or Java, which hides the underlying code making it impossible to alter with new functionality or have it work without the plug-in. D3.js, on the other hand, is done in the browser's native JavaScript and SVG, so you can see how it is manipulating objects inside the DOM. With D3.js, it is now much easier to make graphs interactive.

While D3.js allows you to make really cool graphs, it isn't a graphing library. D3.js will build and manipulate co-ordinate systems, axises and shapes; but it doesn't know what a bar chart or a pie chart is. This is where Dc.js comes in. Dc.js defines line graphs, bar and pie charts and uses D3.js's objects to build them. This makes it much easier to focus on what you want to display instead of generating the display itself.

Crossfilter.js is a JavaScript plug-in used to slice and dice JavaScript arrays. This allows Dc.js to easily manipulate the datatable that the graphs use, so they can refresh with the filtered data. The example on the Crossfilter website was the actual inspiration for the dc.js library itself.

Using the Code

Now that we have the background, we can start to write some code. With this first art, we're going to start off with how Crossfilter works because most of the actual dashboard code is manipulating the data for the charts. Once you have a good understanding of how that works, the actual graphing is pretty simple. I'm going to try to cover several different scenarios, so you can hopefully avoid some of the pitfalls as you start to use it.

Download the Crossfilter.js file from GitHub and include it in your HTML page. For these examples, I'm going to use the raw GitHub source for a reference.

JavaScript
<script type="text/javascript" src="https://rawgithub.com/NickQiZhu/dc.js/master/web/js/crossfilter.js"></script>

We'll first need some data. This data was pulled from the Crossfilter API documentation:

JavaScript
var data = [
  {date: "2011-11-14T16:17:54Z", quantity: 2, total: 190, tip: 100, type: "tab"},
  {date: "2011-11-14T16:20:19Z", quantity: 2, total: 190, tip: 100, type: "tab"},
  {date: "2011-11-14T16:28:54Z", quantity: 1, total: 300, tip: 200, type: "visa"},
  {date: "2011-11-14T16:30:43Z", quantity: 2, total: 90, tip: 0, type: "tab"},
  {date: "2011-11-14T16:48:46Z", quantity: 2, total: 90, tip: 0, type: "tab"},
  {date: "2011-11-14T16:53:41Z", quantity: 2, total: 90, tip: 0, type: "tab"},
  {date: "2011-11-14T16:54:06Z", quantity: 1, total: 100, tip: 0, type: "cash"},
  {date: "2011-11-14T16:58:03Z", quantity: 2, total: 90, tip: 0, type: "tab"},
  {date: "2011-11-14T17:07:21Z", quantity: 2, total: 90, tip: 0, type: "tab"},
  {date: "2011-11-14T17:22:59Z", quantity: 2, total: 90, tip: 0, type: "tab"},
  {date: "2011-11-14T17:25:45Z", quantity: 2, total: 200, tip: 0, type: "cash"},
  {date: "2011-11-14T17:29:52Z", quantity: 1, total: 200, tip: 100, type: "visa"}
]; 

We'll make our Crossfilter instance.

JavaScript
var ndx = crossfilter(data);
For our first example, we'll setup a filter using one of the integer columns. Say we want to get all the transactions with a total equal to 90. To do this, we need to setup a dimension.
JavaScript
var totalDim = ndx.dimension(function(d) { return d.total; });   

Now we can start to filter it. If we wanted to find all the totals equal to 90, we can do the following:

JavaScript
var total_90 = totalDim.filter(90); 

To see the result, we can print out the total_90 variable to the console.

JavaScript
print_filter("total_90"); 

This prints out the following. I'll mark the webconsole data in black with white writing:

"total_90(6) = [
	{"date":"2011-11-14T17:22:59Z","quantity":2,"total":90,"tip":0,"type":"tab"},
	{"date":"2011-11-14T17:07:21Z","quantity":2,"total":90,"tip":0,"type":"tab"},
	{"date":"2011-11-14T16:58:03Z","quantity":2,"total":90,"tip":0,"type":"tab"},
	{"date":"2011-11-14T16:53:41Z","quantity":2,"total":90,"tip":0,"type":"tab"},
	{"date":"2011-11-14T16:48:46Z","quantity":2,"total":90,"tip":0,"type":"tab"},
	{"date":"2011-11-14T16:30:43Z","quantity":2,"total":90,"tip":0,"type":"tab"}
]"  

### Tip ###

Since we want to be able to see if our filters are working correctly, I've created a small function to printout the data to the webconsole ("Tools" > "Web Developer" > "Web Console" in Firefox).

JavaScript
function print_filter(filter){
	var f=eval(filter);
	if (typeof(f.length) != "undefined") {}else{}
	if (typeof(f.top) != "undefined") {f=f.top(Infinity);}else{}
	if (typeof(f.dimension) != "undefined") {f=f.dimension(function(d) { return "";}).top(Infinity);}else{}
	console.log(filter+"("+f.length+") = "+JSON.stringify(f).replace("[","[\n\t").replace(/}\,/g,"},\n\t").replace("]","\n]"));
} 

### End Tip ###

Each of these short hand filters has a long hand equivalent. For filter(90), it is the same as using filterExact(90):

JavaScript
var total_90 = totalDim.filterExact(90); 
JavaScript
print_filter("total_90"); 
"total_90(6) = [
	{"date":"2011-11-14T17:22:59Z","quantity":2,"total":90,"tip":0,"type":"tab"},
	{"date":"2011-11-14T17:07:21Z","quantity":2,"total":90,"tip":0,"type":"tab"},
	{"date":"2011-11-14T16:58:03Z","quantity":2,"total":90,"tip":0,"type":"tab"},
	{"date":"2011-11-14T16:53:41Z","quantity":2,"total":90,"tip":0,"type":"tab"},
	{"date":"2011-11-14T16:48:46Z","quantity":2,"total":90,"tip":0,"type":"tab"},
	{"date":"2011-11-14T16:30:43Z","quantity":2,"total":90,"tip":0,"type":"tab"}
]"   

If we want to filter a range from 90 to 100 inclusive, we'd put the parameter in brackets. Since we want to include 100 in our filter, we'll need to have it go to 101. This is the same as total Dim.filterRange([90,101]);

JavaScript
var total_90_101= totalDim.filter([90,101]); 
JavaScript
print_filter("total_90_101");  
"total_90_101(7) = [
	{"date":"2011-11-14T16:54:06Z","quantity":1,"total":100,"tip":0,"type":"cash"},	
	{"date":"2011-11-14T17:22:59Z","quantity":2,"total":90,"tip":0,"type":"tab"},	
	{"date":"2011-11-14T17:07:21Z","quantity":2,"total":90,"tip":0,"type":"tab"},	
	{"date":"2011-11-14T16:58:03Z","quantity":2,"total":90,"tip":0,"type":"tab"},	
	{"date":"2011-11-14T16:53:41Z","quantity":2,"total":90,"tip":0,"type":"tab"},	
	{"date":"2011-11-14T16:48:46Z","quantity":2,"total":90,"tip":0,"type":"tab"},	
	{"date":"2011-11-14T16:30:43Z","quantity":2,"total":90,"tip":0,"type":"tab"}
]" 

You can even get a little more fancy like only grabbing items divisible by 3. This is the same as totalDim.filterFunction(function(d) { if (d%3===0)return d; } );

JavaScript
var total_3= totalDim.filter(function(d) { if (d%3===0) {return d;} } );  
JavaScript
print_filter("total_3"); 
"total_3(7) = [	
	{"date":"2011-11-14T16:28:54Z","quantity":1,"total":300,"tip":200,"type":"visa"},
	{"date":"2011-11-14T17:22:59Z","quantity":2,"total":90,"tip":0,"type":"tab"},	
	{"date":"2011-11-14T17:07:21Z","quantity":2,"total":90,"tip":0,"type":"tab"},	
	{"date":"2011-11-14T16:58:03Z","quantity":2,"total":90,"tip":0,"type":"tab"},	
	{"date":"2011-11-14T16:53:41Z","quantity":2,"total":90,"tip":0,"type":"tab"},	
	{"date":"2011-11-14T16:48:46Z","quantity":2,"total":90,"tip":0,"type":"tab"},	
	{"date":"2011-11-14T16:30:43Z","quantity":2,"total":90,"tip":0,"type":"tab"}
]" 

That was pretty easy. Filtering on numbers is pretty straightforward unless you have an entry which is NaN. This will completely confuse crossfilter, so make sure there are no NaNs in your dataset.

Moving onto strings, if we wanted to find all the entries where people used visa, again, we need to first create a dimension, this time on the type, then we can do our filter.

JavaScript
var typeDim  = ndx.dimension(function(d) {return d.type;});
JavaScript
var visa_filter = typeDim.filter("visa"); 
JavaScript
print_filter("visa_filter"); 
"visa_filter(2) = [	
	{"date":"2011-11-14T17:29:52Z","quantity":1,"total":200,"tip":100,"type":"visa"},
	{"date":"2011-11-14T16:28:54Z","quantity":1,"total":300,"tip":200,"type":"visa"}
]"

If we wanted to do the same thing with cash, again, pretty easy.

JavaScript
var cash_filter = typeDim.filter("cash");  
JavaScript
print_filter("cash_filter");
"cash_filter(2) = [	
	{"date":"2011-11-14T17:25:45Z","quantity":2,"total":200,"tip":0,"type":"cash"},	
	{"date":"2011-11-14T16:54:06Z","quantity":1,"total":100,"tip":0,"type":"cash"}
]"  

We can even sum up our total column for just the cash entries using the ReduceSum function. Ok, this is where it gets a little tricky. Previously, we were filtering on the dimension. You would think that you'd use the reduceSum function on the filtered data. That is not the case. If we do a ReduceSum on the filtered data through the group function, it won't observe the current filter and will give you back the totals per type in key value format. This kindof makes sense and is pretty handy for dc.js, but not when you want to try to access the data for cash.

JavaScript
var total = typeDim.group().reduceSum(function(d) {return d.total;});
JavaScript
print_filter("total");
"total(3) = [
	{"key":"tab","value":920},
	{"key":"visa","value":500},
	{"key":"cash","value":300}
]"

Instead to get the total for cash, you have to do a Groupall on the crossfilter object itself, which observes all filters; so that when we do a ReduceSum, we get the sum of the total column for the current filter.

JavaScript
var cash_total = ndx.groupAll().reduceSum(function(d) {return d.total;}).value() 
JavaScript
console.log("cash_total="+cash_total);  
"cash_total=300" 

So if the crossfilter object observes all filters, how come it didn't observer the visa filter when we decided to filter on cash? Well ... it's just kind of quirky. If you try to do a filter for cash and visa, it still has the cash filter applied.:

JavaScript
var cash_and_visa_filter = typeDim.filter(function(d) { if (d ==="visa" || d==="cash") {return d;} });  
JavaScript
print_filter("cash_and_visa_filter");
"cash_and_visa_filter(2) = [
	{"date":"2011-11-14T17:25:45Z","quantity":2,"total":200,"tip":0,"type":"cash"},
	{"date":"2011-11-14T16:54:06Z","quantity":1,"total":100,"tip":0,"type":"cash"}
]" 

We need to first clear all filters and then it will work. It's good practise to always clear filters before starting another one.

JavaScript
typeDim.filterAll()
JavaScript
var cash_and_visa_filter = typeDim.filter(function(d) { if (d ==="visa" || d==="cash") {return d;} });
JavaScript
print_filter("cash_and_visa_filter");
"cash_and_visa_filter(4) = [
	{"date":"2011-11-14T17:29:52Z","quantity":1,"total":200,"tip":100,"type":"visa"},
	{"date":"2011-11-14T16:28:54Z","quantity":1,"total":300,"tip":200,"type":"visa"},
	{"date":"2011-11-14T17:25:45Z","quantity":2,"total":200,"tip":0,"type":"cash"},
	{"date":"2011-11-14T16:54:06Z","quantity":1,"total":100,"tip":0,"type":"cash"}
]" 

You've now seen the basics of what crossfilter can do. The next article will showcase on using it with dc.js.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)