Click here to Skip to main content
13,252,065 members (56,262 online)
Click here to Skip to main content
Add your own
alternative version

Stats

29K views
452 downloads
14 bookmarked
Posted 8 Sep 2014

HTTP 206 Partial Content In Node.js

Rate this:
Please Sign up or sign in to vote.
A step-by-step workthrough of HTTP 206 implementation in Node.js.

Table of content

Introduction

In this article, I would like to explain the basic concept of HTTP status 206 Partial Content and a step-by-step implementation walkthrough with Node.js. Also, we will test the code with an example based on the most common scenario of its usage: an HTML5 page which is able to play video file starting at any second. 

Prerequisites

  • Basic HTTP Knowledge
  • Intermediate Node.js Skill
  • Basic HTML5 Skill
  • Basic JavaScript Skill

A brief of partial content

The HTTP 206 Partial Content status code and its related headers provide a mechanism which allows browser and other user agents to receive partial content instead of entire one from server. This mechanism is widely used in streaming a video file and supported by most of browsers and players such as Windows Media Player and VLC Player.

The basic workflow could be explained by these following steps:

  1. Browser requests the content.
  2. Server tells browser that the content can be requested partially with Accept-Ranges header.
  3. Browser resends the request, tells server the expecting range with Range header.
  4. Server responses browser in one of following siturations: 
    • If range is available, server returns the partial content with status 206 Partial Content. Range of current content will be indicated in Content-Range header.
    • If range is unavailable (for example, greater than total bytes of content), server returns status 416 Requested Range Not Satisfiable. The available range will be indicated in Content-Range header too.

Let's take a look at each key header of these steps.

Accept-Ranges: bytes

This is the header which is sent by server, represents the content that can be partially returned to browser. The value indicates the acceptable unit of each range request, usually is bytes in most of situations. 

Range: bytes=(start)-(end)

This is the header for browser telling server the expecting range of content. Note that start and end positions are both inclusive and zero-based. This header could be sent without one of them in following meanings: 

  • If end position is omitted, server returns the content from indicated start position to the position of last available byte.
  • If start position is omitted, the end position will be described as how many bytes shall server returns counting from the last available byte.
Content-Range: bytes (start)-(end)/(total)

This is the header which shall appear following HTTP status 206. Values start and end represent the range of current content. Like Range header, both values are inclusive and zero-based. Value total indicates the total avaliable bytes.

Content-Range: */(total)

This is same header but in another format and will only be sent following HTTP status 416. Value total also indicates the total avaliable bytes of content.

Here is couple examples of a file with 2048 bytes long. Note the different meaning of end when start is omitted.

Request first 1024 bytes

What browser sends:

GET /dota2/techies.mp4 HTTP/1.1
Host: localhost:8000
Range: bytes=0-1023

What server returns:

HTTP/1.1 206 Partial Content
Date: Mon, 15 Sep 2014 22:19:34 GMT
Content-Type: video/mp4
Content-Range: bytes 0-1023/2048
Content-Length: 1024

(Content...)

Request without end position

What browser sends:

GET /dota2/techies.mp4 HTTP/1.1
Host: localhost:8000
Range: bytes=1024-

What server returns:

HTTP/1.1 206 Partial Content
Date: Mon, 15 Sep 2014 22:19:34 GMT
Content-Type: video/mp4
Content-Range: bytes 1024-2047/2048
Content-Length: 1024

(Content...)

Note that server does not have to return all remaining bytes in single response especially when content is too long or there are other performance considerations. So following two examples are also acceptable in this case:

Content-Range: bytes 1024-1535/2048
Content-Length: 512

Server only returns half of remaining content. The range of next request will start at 1536th byte.

Content-Range: bytes 1024-1279/2048
Content-Length: 256

Server only returns 256 bytes of remaining content. The range of next request will start at 1280th byte.

Request last 512 bytes

What browser sends:

GET /dota2/techies.mp4 HTTP/1.1
Host: localhost:8000
Range: bytes=-512

What server returns:

HTTP/1.1 206 Partial Content
Date: Mon, 15 Sep 2014 22:19:34 GMT
Content-Type: video/mp4
Content-Range: bytes 1536-2047/2048
Content-Length: 512

(Content...)

Request with unavailable range 

What browser sends:

GET /dota2/techies.mp4 HTTP/1.1
Host: localhost:8000
Range: bytes=1024-4096

What server returns:

HTTP/1.1 416 Requested Range Not Satisfiable
Date: Mon, 15 Sep 2014 22:19:34 GMT
Content-Range: bytes */2048

 

With understanding of workflow and headers above, now we are able to implement the mechanism in Node.js.

Get started to implement in Node.js

Step 1 - Create a simple HTTP server

We will start from a basic HTTP server as following example shows. This is pretty enough to handle most of requests from browsers. At first, we initialize each object we need, and indicate where are the files located at with initFolder. We also list couple of filename extensions and their corresponding MIME names to construct a dictionary, which is for generating Content-Type header. In the callback function httpListener(), we limit GET as the only allowed HTTP method. Before start to fulfill the request, server will return status 405 Method Not Allowed if other methods appear and return status 404 Not Found if file does not exist in initFolder.  

// Initialize all required objects.
var http = require("http");
var fs = require("fs");
var path = require("path");
var url = require('url');

// Give the initial folder. Change the location to whatever you want.
var initFolder = 'C:\\Users\\User\\Videos';

// List filename extensions and MIME names we need as a dictionary. 
var mimeNames = {
    '.css': 'text/css',
    '.html': 'text/html',
    '.js': 'application/javascript',
    '.mp3': 'audio/mpeg',
    '.mp4': 'video/mp4',
    '.ogg': 'application/ogg', 
    '.ogv': 'video/ogg', 
    '.oga': 'audio/ogg',
    '.txt': 'text/plain',
    '.wav': 'audio/x-wav',
    '.webm': 'video/webm'
};

http.createServer(httpListener).listen(8000);

function httpListener (request, response) {
    // We will only accept 'GET' method. Otherwise will return 405 'Method Not Allowed'.
    if (request.method != 'GET') { 
        sendResponse(response, 405, {'Allow' : 'GET'}, null);
        return null;
    }

    var filename = 
        initFolder + url.parse(request.url, true, true).pathname.split('/').join(path.sep);

    var responseHeaders = {};
    var stat = fs.statSync(filename);

    // Check if file exists. If not, will return the 404 'Not Found'. 
    if (!fs.existsSync(filename)) {
        sendResponse(response, 404, null, null);
        return null;
    }
    responseHeaders['Content-Type'] = getMimeNameFromExt(path.extname(filename));
    responseHeaders['Content-Length'] = stat.size; // File size.
        
    sendResponse(response, 200, responseHeaders, fs.createReadStream(filename));
}

function sendResponse(response, responseStatus, responseHeaders, readable) {
    response.writeHead(responseStatus, responseHeaders);

    if (readable == null)
        response.end();
    else
        readable.on('open', function () {
            readable.pipe(response);
        });

    return null;
}

function getMimeNameFromExt(ext) {
    var result = mimeNames[ext.toLowerCase()];
    
    // It's better to give a default value.
    if (result == null)
        result = 'application/octet-stream';
    
    return result;
}

 

Step 2 - Capture the Range header by using regular expression

With the basic HTTP server, now we can handle the Range header as following code shows. We split the header with regular expression to capture start and end strings. Then use parseInt() method to parse them to integers. If returned value is NaN (not a number), the string does not exist in header. The parameter totalLength represents total bytes of current file. We will use it to calculate start and end positions. 

function readRangeHeader(range, totalLength) {
        /*
         * Example of the method 'split' with regular expression.
         * 
         * Input: bytes=100-200
         * Output: [null, 100, 200, null]
         * 
         * Input: bytes=-200
         * Output: [null, null, 200, null]
         */

    if (range == null || range.length == 0)
        return null;

    var array = range.split(/bytes=([0-9]*)-([0-9]*)/);
    var start = parseInt(array[1]);
    var end = parseInt(array[2]);
    var result = {
        Start: isNaN(start) ? 0 : start,
        End: isNaN(end) ? (totalLength - 1) : end
    };
    
    if (!isNaN(start) && isNaN(end)) {
        result.Start = start;
        result.End = totalLength - 1;
    }

    if (isNaN(start) && !isNaN(end)) {
        result.Start = totalLength - end;
        result.End = totalLength - 1;
    }

    return result;
}

 

Step 3 - Check if range can be satisfied

Back to the function httpListener(), now we check if the range is available after the HTTP method gets approved. If browser does not send Range header, the request will be directly treated as normal request. Server returns entire file and HTTP status is 200 OK. Otherwise we will see if start or end position is greater or equal to file length. If one of them is, the range can not be fulfilled. The status will be 416 Requested Range Not Satisfiable and the Content-Range will be sent. 

    var responseHeaders = {};
    var stat = fs.statSync(filename);
    var rangeRequest = readRangeHeader(request.headers['range'], stat.size);
   
    // If 'Range' header exists, we will parse it with Regular Expression.
    if (rangeRequest == null) {
        responseHeaders['Content-Type'] = getMimeNameFromExt(path.extname(filename));
        responseHeaders['Content-Length'] = stat.size;  // File size.
        responseHeaders['Accept-Ranges'] = 'bytes';
        
        //  If not, will return file directly.
        sendResponse(response, 200, responseHeaders, fs.createReadStream(filename));
        return null;
    }

    var start = rangeRequest.Start;
    var end = rangeRequest.End;

    // If the range can't be fulfilled. 
    if (start >= stat.size || end >= stat.size) {
        // Indicate the acceptable range.
        responseHeaders['Content-Range'] = 'bytes */' + stat.size; // File size.

        // Return the 416 'Requested Range Not Satisfiable'.
        sendResponse(response, 416, responseHeaders, null);
        return null;
    }

 

Step 4 - Fulfill the request

Finally the last puzzle piece comes. For the status 206 Partial Content, we have another format of Content-Range header including start, end and total bytes of current file. We also have Content-Length header and the value is exaclty equal to the difference between start and end. In the last statement, we call createReadStream() and assign start and end values to the object of second parameter options, which means the returned stream will be only readable from/to the positions.

// Indicate the current range.
responseHeaders['Content-Range'] = 'bytes ' + start + '-' + end + '/' + stat.size;
responseHeaders['Content-Length'] = start == end ? 0 : (end - start + 1);
responseHeaders['Content-Type'] = getMimeNameFromExt(path.extname(filename));
responseHeaders['Accept-Ranges'] = 'bytes';
responseHeaders['Cache-Control'] = 'no-cache';

// Return the 206 'Partial Content'.
sendResponse(response, 206,
    responseHeaders, fs.createReadStream(filename, { start: start, end: end }));

 

Here is the complete httpListener() callback function.

function httpListener(request, response) {
    // We will only accept 'GET' method. Otherwise will return 405 'Method Not Allowed'.
    if (request.method != 'GET') {
        sendResponse(response, 405, { 'Allow': 'GET' }, null);
        return null;
    }

    var filename =
        initFolder + url.parse(request.url, true, true).pathname.split('/').join(path.sep);

    // Check if file exists. If not, will return the 404 'Not Found'. 
    if (!fs.existsSync(filename)) {
        sendResponse(response, 404, null, null);
        return null;
    }

    var responseHeaders = {};
    var stat = fs.statSync(filename);
    var rangeRequest = readRangeHeader(request.headers['range'], stat.size);

    // If 'Range' header exists, we will parse it with Regular Expression.
    if (rangeRequest == null) {
        responseHeaders['Content-Type'] = getMimeNameFromExt(path.extname(filename));
        responseHeaders['Content-Length'] = stat.size;  // File size.
        responseHeaders['Accept-Ranges'] = 'bytes';

        //  If not, will return file directly.
        sendResponse(response, 200, responseHeaders, fs.createReadStream(filename));
        return null;
    }

    var start = rangeRequest.Start;
    var end = rangeRequest.End;

    // If the range can't be fulfilled. 
    if (start >= stat.size || end >= stat.size) {
        // Indicate the acceptable range.
        responseHeaders['Content-Range'] = 'bytes */' + stat.size; // File size.

        // Return the 416 'Requested Range Not Satisfiable'.
        sendResponse(response, 416, responseHeaders, null);
        return null;
    }

    // Indicate the current range. 
    responseHeaders['Content-Range'] = 'bytes ' + start + '-' + end + '/' + stat.size;
    responseHeaders['Content-Length'] = start == end ? 0 : (end - start + 1);
    responseHeaders['Content-Type'] = getMimeNameFromExt(path.extname(filename));
    responseHeaders['Accept-Ranges'] = 'bytes';
    responseHeaders['Cache-Control'] = 'no-cache';

    // Return the 206 'Partial Content'.
    sendResponse(response, 206, 
        responseHeaders, fs.createReadStream(filename, { start: start, end: end }));
}

 

Also the entire workflow can be summarized with following chart.

Test the implementation

So how do we test our work? As we just mentioned in Introduction, the most common scenario of partial content is streaming and playing videos. So we create an HTML5 page which includes a <video/> with ID mainPlayer and a <source/> tag. Function onLoad() will be triggered when mainPlayer has preloaded the metadata of current video. It is used for checking if there is any numeric parameter existing in URL. If yes, mainPlayer will skip to indicated second.

<!DOCTYPE html>
<html>
    <head>
        <script type="text/javascript">

            function onLoad() {
                var sec = parseInt(document.location.search.substr(1));
                
                if (!isNaN(sec))
                    mainPlayer.currentTime = sec;
            }
        
        </script>
        <title>Partial Content Demonstration</title>
    </head>
    <body>
        <h3>Partial Content Demonstration</h3>
        <hr />
        <video id="mainPlayer" width="640" height="360" 

            autoplay="autoplay" controls="controls" onloadedmetadata="onLoad()">
            <source src="dota2/techies.mp4" />
        </video>
    </body>
</html>

 

Now we save our page as "player.html" under initFolder along with video file "dota2/techies.mp4". Activate Node.js, execute the script then open the URL in browser:

http://localhost:8000/player.html

This is how it looks like in Chrome:

Because there are no parameters in the URL, the file will be played starting at 0th second.

Next is the fun part. Let's try to open this one and see what happens:

http://localhost:8000/player.html?60 

If you press F12 to open Chrome Developer Tools, switch to Network tab and click the detail of latest log. You will notice that the Range header string sent by your browser is something like this:  

Range:bytes=225084502-

Pretty interesting, right? When function onLoad() changes the currentTime property, browser computes the curresponding byte postion of 60th second in this video. Because mainPlayer has preloaded the metadata, including format, bit rate and other fundamental information, the start position comes out almost immediately. After that, the browser is able to download and play the video without requesting first 60 seconds. Same thing happens if you click the timeline before mainPlayer reaches the position you just clicked.  It works!!

Conclusion

We have implemented an HTTP server in Node.js which supports partial content. We also tested it with an HTML5 page. But this is just a beginning. If you have understood the whole thing about these headers and workflow, you can try to implement it with other frameworks such like ASP.NET MVC and Web WCF Service. But don't forget to enable Task Manager to see CPU and memory usage. Like we discussed in A breif of partial content, server does not have to return all remaining bytes in single response. To find out a balance point of performance will be an important mission.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Robert Vandenberg Huang
Software Developer
Taiwan Taiwan
Software developer, video game enthusiast, drummer and also a huge Jazz music fan. Have been working on software development since junior high school. Love sharing knowledge with people, learning new things and having interaction with developer community.

My Homepage at GitHub

You may also be interested in...

Pro

Comments and Discussions

 
QuestionThanks Pin
daveywc25-Jun-17 21:51
memberdaveywc25-Jun-17 21:51 
QuestionHow can I ask two ranges with parameters?, for example http://localhost:8000/player.html?60-90 Pin
Member 1205649119-Oct-15 11:52
memberMember 1205649119-Oct-15 11:52 
AnswerRe: How can I ask two ranges with parameters?, for example http://localhost:8000/player.html?60-90 Pin
Robert Vandenberg Huang24-Oct-15 3:24
professionalRobert Vandenberg Huang24-Oct-15 3:24 
GeneralRe: How can I ask two ranges with parameters?, for example http://localhost:8000/player.html?60-90 Pin
Member 1205649126-Oct-15 9:05
memberMember 1205649126-Oct-15 9:05 
QuestionGood Job Pin
M@dHatter8-Sep-14 10:38
memberM@dHatter8-Sep-14 10:38 
GeneralRe: Good Job Pin
Robert Vandenberg Huang8-Sep-14 14:09
memberRobert Vandenberg Huang8-Sep-14 14:09 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web04 | 2.8.171114.1 | Last Updated 8 Sep 2014
Article Copyright 2014 by Robert Vandenberg Huang
Everything else Copyright © CodeProject, 1999-2017
Layout: fixed | fluid