Click here to Skip to main content
Click here to Skip to main content

D3.js crash course

By , 15 Jan 2013
Rate this:
Please Sign up or sign in to vote.

Introduction

Data-Driven Documents (d3js.org[^]) is a javascript library that offers an interesting approach to visualization for the web. D3 enables direct inspection and manipulation of the standard document object model (DOM). With D3 we can selectively bind input data to document elements and define dynamic transforms that both generate and modify the content of the DOM.

D3’s primary authors Michael Bostock, Vadim Ogievetsky and Jeffrey Heer have created a library that uses representational transparency to improve expressiveness and provide natural integration with developer tools. It’s quite different from other visualization libraries since it focuses on direct inspection and manipulation of the document object model.

The functionality of D3 can broadly be separated into four categories:

  1. Loading data
  2. Binding data to elements
  3. Transforming elements by inspecting the bound data, and updating the bound properties of the elements
  4. Transitioning elements in response to user interaction

The transformation of elements based on bound data is the most important category and D3 provides the structure for performing these transformations, while we define how we want those transformations carried out – leaving us in full control of the final output.

D3 is not a traditional visualization framework. Instead of introducing a new graphical grammar, D3’s authors choose to solve a different, smaller problem: the manipulation of documents based on data.

Basically D3 can be thought of as a visualization “kernel” rather than a framework, somewhat resembling other document transformers such as jQuery, CSS and XSLT. These libraries share a common concept of a selection: A set of elements is selected matching the given criteria before applying a set of operations that transforms the selected elements. JavaScript-based selections provide flexibility on top of CSS, since styles can be altered dynamically in response to user interaction and changes to the data.

The designers of D3 choose to adopt the W3C Selectors API for selection; a mini-language consisting of predicates that filter elements by tag (“tag”), class (“.class”), unique identifier (“#id”), attribute (“[name=value]”), containment (“parent child”), adjacency (“before ~ after”), and other facets. Since predicates can be intersected (“.a.b”) and unioned (“.a, .b”) we have a rich and concise method for selection.

D3’s operates on set of elements queried from the current document. Data joins bind data to elements, enabling functional operations that depend on data, and produce enter and exit sub-selections that enables creation and destruction of elements in correspondence with data. Operations are, by default, applied instantaneously, while animated transitions interpolate attributes and styles smoothly over time. The library facilitates event handlers that respond to user input and enable interaction. D3 also provides a number of helper modules that simplify common visualization tasks, such as layout and scales.

A tiny example

Let’s start by creating a small interactive visualization based on a simple particle system:

<head runat="server">
<title>An introduction to d3.js</title>
<style type="text/css">
body
{
  background-color:#222;
}

rect 
{
  fill: steelblue;
  stroke-width: 2.5px;
}
</style>
<script src="http://d3js.org/d3.v3.min.js"></script>
</head>

The script tag loads the minified version 3 of D3 from its canonical location: http://d3js.org/d3.v3.min.js

The body of our page consists of a single script tag. The code creates an svg element and attaches the particle function to the mousemove event of the element.

<body>
    <script type="text/javascript">
        var w = 600,
            h = 400,
            z = d3.scale.category20b(),
            i = 0;

        var svg = d3.select("body").append("svg:svg")
            .attr("width", w)
            .attr("height", h)
            .on("mousemove", particle);

        function particle()
        {
            var m = d3.mouse(this);

            svg.append("svg:rect")
                .attr("x", m[0] - 10)
                .attr("y", m[1] - 10)
                .attr("height", 20)
                .attr("width", 20)
                .attr("rx", 10)
                .attr("ry", 10)
                .style("stroke", z(++i))
                .style("stroke-opacity", 1)
                .style("opacity", 0.7)
              .transition()
                .duration(5000)
                .ease(Math.sqrt)
                .attr("x", m[0] - 100)
                .attr("y", m[1] - 100)
                .attr("height", 200)
                .attr("width", 200)
                .style("stroke-opacity", 1e-6)
                .style("opacity", 1e-6)
                .remove();
        }

    </script>
</body>

d3.mouse returns the x and y coordinates of the current d3.event, relative to the specified container – the coordinates are returned as a two-element array [x, y].

Next we use D3 to append a rect element to the svg, centered at the current mouse position – attaching a transition that lasts for 5 seconds, removing the rect element on completion.

The end result is a rather pleasing visualization that looks like a cross between northern lights and a kaleidoscope.

Basics

Well that was the teaser and now it’s time to take closer look at D3 basics – first we’ll transform this beautiful svg:

<svg id="visual" width="220" height="220">
 <rect width="100" height="100" rx="15" ry="15" x="40" y="40" />
 <rect width="100" height="100" rx="15" ry="15" x="60" y="60" />
 <rect width="100" height="100" rx="15" ry="15" x="80" y="40" />
 <rect width="100" height="100" rx="15" ry="15" x="100" y="25" />
 <rect width="100" height="100" rx="15" ry="15" x="120" y="50" />
</svg>

into:

Which is performed by the following piece of code:

<script type="text/javascript">
 var visual = d3.select("#visual");
 var w = visual.attr("width");
 var h = visual.attr("height");
 var rectangles = visual.selectAll("rect");
 rectangles.style("fill", "steelblue")
  .attr("x", function ()
   {
    return Math.random() * w;
   })
  .attr("y", function ()
   {
    return Math.random() * h;
   });
  </script>

D3 implements two top-level methods for selecting elements: select and selectAll. The methods accept selector strings; select selects only the first matching element, while selectAll selects all matching elements in document traversal order. Here we use the select method to create a selection that contains one element: the svg element with id="visual".

D3 selections are arrays of arrays of elements and D3 binds additional methods to the array so that we can perform operations on the selected elements, like setting an attribute on all the selected elements. A peculiarity of D3 is that selections are grouped and hence arrays of arrays of elements. This is done to preserve the hierarchical structure of subselections – which we will get back to later. Usually we can ignore this detail, but this is why a single-element selection looks like [0][0] and not [0].

A Simple Line Chart

It’s time to do something a bit more interesting, like drawing a line chart using data retrieved from a CSV file that resides on the server. That’s right; there is no law that says we have to use json or some verbose xml format just because we’re doing javascript.

The data for following the line chart was retrieved from the Norwegian Petroleum Directorate’s FactPages.

The above chart says a bit about why oil trades at $111 a barrel – while there is still a significant amount of oil left, it’s no longer as accessible as it used to be; and retrieving it has become more costly and time consuming. It’s not an infinite resource. Ah, well, back to the code:

<script>
    var margin = { top: 20, right: 20, bottom: 30, left: 50 },
        width = 600 - margin.left - margin.right,
        height = 400 - margin.top - margin.bottom;

First we define a margin object with properties for the four sides of our chart, then we set the width and height that we later will use as the dimensions for our chart.

    var parseDate = d3.time.format("%Y").parse;

d3.time.format creates a time formatter using the given specifier %Y, which says that this formatter will handle four digit years. We then assign the parse function for the formatter to parseDate. We will later use this parse function to extract the year from the csv data retrieved from the server.

Next we need to set up the scales for our chart. Scales are functions that map values from an input domain to values in an output range.

    var x = d3.time.scale()
        .range([0, width]);

d3.time.scale implements a scale that converts values from the input domain to dates. The time scale also provides suitable ticks based on time intervals, greatly simplifying the work required to generate axes for nearly any time-based domain.

    var y = d3.scale.linear()
        .range([height,0]);

d3.scale.linear is the most common scale which maps a continuous input domain to a continuous output domain. The mapping is linear since the output range value y can be expressed by the linear function of the input values x: y = mx + b.

It’s now quite easy to set up the axis for our chart:

    var xAxis = d3.svg.axis()
        .scale(x)
        .orient("bottom");

    var yAxis = d3.svg.axis()
        .scale(y)
        .orient("left");

d3.svg.axis creates an axis component which we configure to use the scales we just created.

d3.svg.line creates a line generator with the default x- and y-accessor functions that generates SVG path data for an open piecewise linear curve, which is what we want for a line chart.

    var line = d3.svg.line()
        .x(function (d) { return x(d.year); })
        .y(function (d) { return y(d.oil); });

The accessor functions is invoked for each element in the data array passed to the line generator, which allows us to map year and oil to x and y respectively.

Now we need to create the SVG element for the chart and add an SVG G (group) element to the SVG which will contain the final chart.

    var svg = d3.select("body").append("svg")
        .attr("width", width + margin.left + margin.right)
        .attr("height", height + margin.top + margin.bottom)
        .append("g")
        .attr("transform", "translate(" + margin.left + "," + margin.top + ")");

It’s now time to load our data from the server, and D3 provides built-in support for loading and parsing comma-separated values (CSV). CSV is more space-efficient than JSON, and this can improve loading times for large datasets.

    d3.csv("YearlyOilProduction.csv", function (error, data)
    {

d3.csv issues an HTTP GET request for the comma-separated values (CSV) file at the specified url. The request is processed asynchronously, and when the CSV data is available, the specified callback will be invoked with the parsed rows as the argument.

        data.forEach(function (d) {
            d.year = parseDate(d.year);
            d.oil = +d.oil;
        });

Once we have our data we need to ensure that it’s in the expected format by using the parser we created earlier to coerce the year into a Date value, and d.oil = +d.oil to coerce oil into a number.

With our data in the expected format, we are ready to set the input range for the scales, and d3.extent will return the minimum and maximum value of the passed data.

        x.domain(d3.extent(data, function (d) { return d.year; }));
        y.domain(d3.extent(data, function (d) { return d.oil; }));

Now we are ready to actually render the chart: We append a g element for each axis, and specify a translate transform for the x-axis to move it to the bottom of our chart – remember that for an SVG coordinates are relative to the upper left corner, with positive y-axis going downwards.

        svg.append("g")
            .attr("class", "x axis")
            .attr("transform", "translate(0," + height + ")")
            .call(xAxis);

        svg.append("g")
            .attr("class", "y axis")
            .call(yAxis)
            .append("text")
            .attr("transform", "rotate(-90)")
            .attr("y", 6)
            .attr("dy", ".71em")
            .style("text-anchor", "end")
            .text("million Sm3");

.call(xAxis) causes the passed axis object to render itself into the new g element.

Finally we create an SVG path element that uses the line generator we created earlier to render the line segments for the data.

        svg.append("path")
            .datum(data)
            .attr("class", "line")
            .attr("d", line);
    });

</script>

Serving data efficiently using a synchronous HTTP handler

Visualizations often require a vast amount of data, and providing the data in an efficient manner is usually important.

One way to do this really efficiently is to implement an Asp.Net HTTP handler. Given how easy this really is, it’s surprising that it’s not done more often:

public class DataRequestHandler : IHttpHandler
{
    public bool IsReusable
    {
        get 
        { 
            return true; 
        }
    }

    public void ProcessRequest(HttpContext context)
    {
        var response = context.Response;
        response.ContentType = "text/csv";

        response.Output.WriteLine(ProductionData.CsvHeader);

        List<ProductionData> productionDataList = Global.Data;
        foreach (ProductionData element in productionDataList)
        {
            response.Output.WriteLine(element.ToString());
        }

        response.End();
    }
}

That’s it – with the above code we’ve implemented a complete Asp.Net HTTP handler which serves up CSV formatted data.

IsReusable returns true because the handler is stateless, which means that it doesn’t contain fields that may change between invocations of ProcessRequest – and that allows Asp.Net to reuse an instance of the handler.

The implementation of ProductionData.CsvHeader is simple:

public static string CsvHeader
{
    get
    {
        return "timeStamp,oil,gas,ngl,condensate,oe,water";
    }
}

The same can be said about ProductionData.ToString():

public override string ToString()
{
    string result = timeStamp.Year.ToString() + "-" + timeStamp.Month.ToString() + "," +
        oil.ToString(CultureInfo.InvariantCulture) + "," +
        gas.ToString(CultureInfo.InvariantCulture) + "," +
        ngl.ToString(CultureInfo.InvariantCulture) + "," +
        condensate.ToString(CultureInfo.InvariantCulture) + "," +
        oe.ToString(CultureInfo.InvariantCulture) + "," +
        water.ToString(CultureInfo.InvariantCulture);
    return result;
}

Real world industrial database servers often need to serve up hundreds of thousands of records, and doing so using XML or JSON really hurts the performance of the server. While binary would be even better, CSV is a good compromise between performance and flexibility – and it’s a reasonable way to provide data to HTML5 applications.

History

  • 10th of January 2013 - Initial posting
  • 16th of January 2013 - Minor changes - Jeremy David Thomson[^] noted that D3 is a registered trademark of TigerLogic Corporation[^], 25A Technology Dr. Suite 100, Irvine, CA 92618, USA

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Espen Harlinn
Architect Goodtech Projects & Services AS
Norway Norway
Principal Architect - Goodtech Projects & Services AS.
 
Specializing in integrated operations and high performance computing solutions.
 
I’ve been fooling around with computers since the early eighties, I’ve even done work on CP/M and MP/M.
 
Wrote my first “real” program on a BBC micro model B based on a series in a magazine at that time. It was fun and I got hooked on this thing called programming ...
 
A few Highlights:
  • High performance application server development
  • Model Driven Architecture and Code generators
  • Real-Time Distributed Solutions
  • C, C++, C#, Java, TSQL, PL/SQL, Delphi, ActionScript, Perl, Rexx
  • Microsoft SQL Server, Oracle RDBMS, IBM DB2, PostGreSQL
  • AMQP, Apache qpid, RabbitMQ, Microsoft Message Queuing, IBM WebSphereMQ, Oracle TuxidoMQ
  • Oracle WebLogic, IBM WebSphere
  • Corba, COM, DCE, WCF
  • AspenTech InfoPlus.21(IP21), OsiSoft PI
 
More information about what I do for a living can be found at: harlinn.com or LinkedIn
 
You can contact me at espen.harlinn@goodtech.no

Comments and Discussions

 
Questionusing data from angularjs resource PinmemberLopezd3-Jan-14 1:03 
AnswerRe: using data from angularjs resource PinmentorEspen Harlinn3-Jan-14 1:16 
GeneralRe: using data from angularjs resource PinmemberLopezd3-Jan-14 3:19 
GeneralRe: using data from angularjs resource PinmentorEspen Harlinn3-Jan-14 12:12 
QuestionIs course about D3 usage with asp? PinmemberMember 1026568810-Sep-13 19:29 
AnswerRe: Is course about D3 usage with asp? PinmentorEspen Harlinn3-Jan-14 1:10 
GeneralMy vote of 5 PinmvpSergey Alexandrovich Kryukov11-Jun-13 13:56 
GeneralRe: My vote of 5 PinmvpEspen Harlinn12-Jun-13 10:00 
GeneralRe: My vote of 5 PinmvpSergey Alexandrovich Kryukov12-Jun-13 10:33 
QuestionProblem with loading the project Pinmemberamirmn713-Mar-13 5:37 
AnswerRe: Problem with loading the project PinmvpEspen Harlinn13-Mar-13 6:03 
GeneralRe: Problem with loading the project Pinmemberamirmn713-Mar-13 9:51 
GeneralRe: Problem with loading the project PinmvpEspen Harlinn13-Mar-13 9:58 
GeneralMy vote of 5 PinmemberOlivier Giulieri5-Mar-13 16:40 
GeneralRe: My vote of 5 PinmvpEspen Harlinn5-Mar-13 22:50 
QuestionThis has nothing to do with Tiger Logic's D3 PinmemberJeremy David Thomson14-Jan-13 14:58 
AnswerRe: This has nothing to do with Tiger Logic's D3 PinmemberSean Holm14-Jan-13 23:46 
GeneralRe: This has nothing to do with Tiger Logic's D3 PinmvpEspen Harlinn15-Jan-13 21:58 
AnswerRe: This has nothing to do with Tiger Logic's D3 PinmvpEspen Harlinn15-Jan-13 21:52 
QuestionImpressive PinmemberMuigai Mwaura11-Jan-13 5:32 
AnswerRe: Impressive PinmvpEspen Harlinn11-Jan-13 6:01 
GeneralMy vote of 5 PinmemberAbinash Bishoyi11-Jan-13 3:04 
GeneralRe: My vote of 5 PinmvpEspen Harlinn11-Jan-13 3:29 
GeneralRe: My vote of 5 PinmemberS.M. Zamshed Farhan16-Jan-13 20:41 
QuestionVery timely, I just used some D3.Js in my latest one PinmvpSacha Barber10-Jan-13 4:12 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web03 | 2.8.140415.2 | Last Updated 16 Jan 2013
Article Copyright 2013 by Espen Harlinn
Everything else Copyright © CodeProject, 1999-2014
Terms of Use
Layout: fixed | fluid