Binary Formats in JavaScript: Base64, Deflate, and UTF8

• Download source code - 17.5 KB

Introduction

This article demonstrates the using of binary formats in JavaScript code. JavaScript, by its nature, cannot operate with binary data represented as a fragment of memory – as a byte array. That makes it difficult to use community developed algorithms and encodings. A good example is the DEFLATE compressed format. This raises more problems if the JavaScript code has to be run on a web browser: data has to be delivered over HTTP.

In the proposed implementation, a byte array is emulated by a regular JavaScript array of objects. Also, the given implementation tries to solve the problem of binary data transfer to a client-side script. Let’s assume we have DEFLATE compressed data (.NET’s System.IO.Compression namespace, Java’s java.util.zip.*, PHP’s http_deflate) and there is a way to transfer it to the client in BASE64 format.

Using the Code

The deflate.js contains the functions and classes that implement the decompression part of the DEFLATE algorithm (RFC 1951). To use this algorithm, its input has to be presented as a stream of bytes.

// create BASE64 byte stream reader
var reader = new Base64Reader(base64string);

The class exposes the readByte() method that returns the next byte, or -1 if it’s the end of the stream.

// create inflator
var inflator = new Inflator(reader);

The Inflator class, as in the previous class, exposes the readByte() method that returns the next byte from the decompressed byte stream. The binary stream can be consumed at that point.

If regular text is compressed, and it needs to be re-encoded from UTF-8 bytes to characters, we use the Utf8Translator class to retrieve the characters instead of the bytes.

// create translator
var translator = new Utf8Translator(inflator);

The class exposes the readChar() method that returns a one-character string with the next available character, or null to indicate the end of the stream. The deflate.js file also contains UnicodeTranslator and DefaultTranslator.

For convenience, there is the TextReader class that exposes not only the readChar() method, but also the readToEnd() and readLine() methods.

Those functions/classes can be used not only within the web browser's context, but in OS scripting or legacy ASP programming.

The SamplePage.htm, included in the package, displays the RFC 1951 memo content.

Points of Interest

The deflate.js functions will help to perform selective compression of data for AJAX requests. Most of the data transmitted in AJAX operations is text or a textual presentation of the binary data.

Since not all web browsers can retrieve remote data as an array of bytes (as responseBody in IE’s XmlHTTPRequest), BASE64 encoded data has to be transmitted to the client from the server. Even if BASE64 data grows 133% for its original, compression of textual data by 75% will still reduce the amount of data to be stored/transferred.

Emulation of byte array as an array of objects in JavaScript reduces the performance of the solution, e.g., to extract 50K takes 1-2 sec(s) in a web browser context.

RFC 1951, 2779, 2781, and 4648 were used to implement the underlying algorithms. There are well written memos. There are lots of formats based on the open DEFLATE compressed format (e.g., GZIP, PNG, SVGZ, SWF); implementing it in JavaScript gives one more way to access/reuse data.