Click here to Skip to main content
13,557,929 members
Click here to Skip to main content
Add your own
alternative version


99 bookmarked
Posted 1 Jul 2003

zipstream, bzip2stream: iostream wrappers for the zlib and bzip2 libraries

, 2 Oct 2003
Rate this:
Please Sign up or sign in to vote.
STL compliant, stream-to-stream, zlib and bzip2 wrapper with wide char support.


This article presents two zipped STL iostream implementation based on the library zlib (see download link above) and bzip2 (see download link above). This means that you can easily manipulate zipped streams like any other STL ostream/istream.

To give you an idea, consider following snippet that prints "Hello World":

ostringstream output_buffer;
// writing data
output_buffer<<"Hello world"<<endl 

Now, the same snippet but with zipped output using zlib:

// zip_ostream uses output_buffer as output buffer :)
zip_ostream zipper( output_buffer );

// writing data as usual
zipper<<"Hello world"<<endl 

or using bzip2:

// zip_ostream uses output_buffer as output buffer :)
bzip2_ostream bzipper( output_buffer );

// writing data as usual
bzipper<<"Hello world"<<endl 

As you can see adding zipped buffers into your existing applications is quite straightforward. To summarize, let's see some quick facts about zipstream and bzip2stream:

  • STL compliant,
  • any-stream-to-any-stream support,
  • char, wchar_t support,
  • fining tuning of compression properties,
  • support custom allocators (New!)

Why another wrapper? Why not use gzstream?

Writing wrappers around the zlib library is popular on CodeProject. If you search for 'zip' you will find at least 14 articles on the topic. Moreover, if you crawl on the web and especially on zlib home page, you can find dozens of other wrappers.

So why another wrapper? Well, none of the wrappers are fully STL compliant. Ok, this is not true since gzstream (see download link above) implements fstream-like STL streams. However, gzstream has three drawbacks:

  1. It does not allow buffer to buffer compression since it is based on gzip i/o methods: only file to buffer or buffer to file are supported,
  2. it is licensed under LGPL which makes it difficult to use in commercial apps,
  3. it does not support wchar_t

Last reason to write this wrapper: it is a good exercise to understand and implement iostreams.

Wrapper architecture

The three drawbacks of gzstream pushed me to re-implement an STL wrapper for zlib (and later on, do some cut and paste to get bzip2 working).

This wrapper takes a user defined i/ostream to write or read compressed data. This approach is quite flexible since the user can give any stream (istringstream, ifstream, or custom stream) to store or load the compressed data.

Internally zip_stream acts as a triple buffer: the streambuf object in itself, zlib library and the user-defined stream. For example, during the compression process, the buffers are used:

  • first buffer: the data to compress is buffered into a streambuf object,
  • second buffer: when overflow is called, the first buffer data is sent to zlib which also buffers the data internally. If zlib outputs data, it is sent to the user-defined stream,
  • third buffer: the user-defined stream is buffered.

Some care must be taken when flushing: you must use the method zflush that will first flush the streambuf, then flush the zlib buffer, then flush the user-defined stream. Note that you should avoid flushing as it degrades compression.

Implementing iostreams

Since I'm not an STL expert, I will very briefly discuss this part. There's room for a tutorial on this topic...

To implement custom iostream, you need to take the following steps:

  • implement a custom my_streambuf, inherited from streambuf. You need to override the virtual methods sync, underflow and overflow. sync and overflow are used in output streams and underflow is used in input streams.
  • implement a custom my_ostream, inherited from ostream. It will use my_streambuf
  • implement a custom my_istream, inherited from istream. It will use my_streambuf as stream buffer.

Class quick reference

All the zlib classes are in the zlib_stream namespace and all the bzip2 classes are in the bzip2_stream namespace.

The two main classes of the zlib wrapper are basic_zip_ostream and basic_zip_istream which implement respectively compression and decompression and behave like classic basic_ostream and basic_istream.

Classical typedef are also provided for these classes:

  • zip_ostream, zip_istream for char streams
  • wzip_ostream, wzip_istream for wchar_t streams.

The bzip2 classes have similar names, just replace zlib by bzip2: basic_zip_streambuf becomes basic_bzip2_streambuf.


This class inherits from basic_ostream:

<PRE lang=c++>template< typename Elem, typename Tr = char_traits<Elem;>, typename ElemA = std::allocator<Elem>, typename ByteT = unsigned char, typename ByteAllocatorT = std::allocator<ByteT> > basic_zip_ostream : public basic_ostream<Elem, Tr>


  • Elem,Tr are the classical basic_ostream template parameters,
  • ElemA is the allocator for a Elem buffer used internally,
  • ByteT is the byte type used internally (you should not change that),
  • ByteAT is a custom allocator for a ByteT buffer used internally


<PRE lang=c++>basic_zip_ostream( ostream_reference ostream_, bool is_gzip_ = false, size_t level_ = Z_DEFAULT_COMPRESSION, EStrategy strategy_ = DefaultStrategy, size_t window_size_ = 15, size_t memory_level_ = 8, size_t buffer_size_ = 4096 );
  • ostream_ is a user defined output stream,
  • is_gzip_, true if you want to add the gzip header and footer,
  • level_, compression level 0, bad and faster to 9 max and slower,
  • strategy_, compression strategy, see EStrategy enum,
  • window_size_, memory_level_ are advanced zlib settings, check zlib manual,
  • buffer_size_, read buffer size

Note that if you choose the gzip option, a header will be automatically added in the constructor and the gzip footer (CRC + data size) will be added in the destructor.

Other methods

  • Flush all buffers (zlib and ostream): <PRE lang=c++>basic_zip_ostream& zflush()

    This method must be called before using the compressed data! Since zlib does it's own buffering and ostream::flush is not virtual there is no way to avoid this problem.

  • Return the CRC of the uncompressed data: <PRE lang=c++>long get_crc();
  • Return the uncompressed data size: <PRE lang=c++>long get_in_size();
  • Return the compressed data size: <PRE lang=c++>long get_out_size();

Predefined typedefs

<PRE lang=c++>typedef basic_zip_ostream<char> zip_ostream; typedef basic_zip_ostream<wchar_t> zip_wostream;


This class inherits from basic_istream:

<PRE lang=c++>template< typename Elem, typename Tr = char_traits<Elem;>, typename ElemA = std::allocator<Elem>, typename ByteT = unsigned char, typename ByteAT = std::allocator<ByteT> > basic_zip_istream : public basic_istream<Elem, Tr>


<PRE lang=c++>basic_zip_istream( istream_reference istream_, size_t window_size_ = 15, size_t read_buffer_size_ = 1024 * 10, size_t input_buffer_size_ = 1024 * 5 )
  • istream_, input stream containing the compressed data,
  • window_size_, should be compatible with compression window size,
  • read_buffer_size_, size of the streambuf buffer size,
  • input_buffer_size_, size of the zlib input buffer size

Other methods

  • Tells if it is a gzip file: <PRE lang=c++>bool is_gzip() const
  • Checks CRC (must be a gzip file) <PRE lang=c++>bool check_crc() const
  • Return the CRC of the uncompressed data: <PRE lang=c++>long get_crc() const;
  • Return the uncompressed data size: <PRE lang=c++>long get_out_size() const;
  • Return the compressed data size: <PRE lang=c++>long get_in_size() const;

Predefined typedefs

typedef basic_zip_istream<char> zip_istream;
typedef basic_zip_istream<wchar_t> zip_wistream;

How to ...

All the following examples are valid for both zlib and bzip2 wrappers.

Compress to a buffer

<PRE lang=c++>ostringstream buffer; zip_ostream zipper(buffer); // writing stuff zipper<<... //flushing VERY IMPORTANT! zipper.zflush(); // buffer.str() is ready

Compress to a file

<PRE lang=c++>ofstream file("",ios::out | ios::binary); { zip_ostream zipper(file, true /* gzip file*/); // writing stuff zipper<<... } // the stream is flushed, the destructor is called and gzip header appended // the file is ready

Decompress from a buffer

<PRE lang=c++>istringstream buffer; zip_istream unzipper(buffer); // reading stuff unzipper>>...

Decompress from a file

<PRE lang=c++>ifstream file("", ios::in | ios::binary); zip_istream unzipper(file); // reading stuff unzipper>>... // if the file was gzip, we can check the crc if (unzipper.is_gzip()) std::cout<<"crc check: "<<( unzipper.check_crc() ? "ok" : "failed");

Using it in your project

Zlib wrapper

  • Read the license terms,
  • Copy zipstream.hpp and zipstream.ipp in your include directory,
  • Make sure zlib is available,
  • Add #include "zlibstream.hpp" to include the headers,

bzip2 wrapper

  • Read the license terms,
  • Copy bzip2stream.hpp and bzip2stream.ipp in your include directory,
  • Make sure zlib is available,
  • Add #include "bzip2stream.hpp" to include the headers,


  • 30-09-2003, 1.7, added custom allocators (suggestion of <forgot, send me your e-mail>), fixed warning in ostream constructors
  • 21-09-2003, fixed bugs with CRC and size, writing thanks to Jeroen Dirks and gigimenegolo
  • 08-08-2003, 1.5
    • Fixed gzip footer problem: CRC is read
    • data is put back in the buffer when zip file has finished
  • 07-18-2003, 1.4 Fixed gzip header problem.
  • 07-3-2003, 1.3, add bzip2 wrapper
  • 07-2-2003, 1.2, wchar_t working
  • 07-2-2003, 1.1, fixed bugs in gzip header and zip_to_stream
  • 07-01-2003, 1.0, initial release



These zlib and bzip2 wrappers are licensed under the zlib/libpng license.


This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


About the Author

Jonathan de Halleux
United States United States
Jonathan de Halleux is Civil Engineer in Applied Mathematics. He finished his PhD in 2004 in the rainy country of Belgium. After 2 years in the Common Language Runtime (i.e. .net), he is now working at Microsoft Research on Pex (

You may also be interested in...


Comments and Discussions

GeneralRe: STLPort bug ! Pin
Anonymous20-Oct-03 10:13
sussAnonymous20-Oct-03 10:13 
GeneralMissing header file Pin
mridey30-Sep-03 18:58
membermridey30-Sep-03 18:58 
GeneralRe: Missing header file Pin
mridey30-Sep-03 19:00
membermridey30-Sep-03 19:00 
GeneralRe: Missing header file Pin
Jonathan de Halleux1-Oct-03 1:06
memberJonathan de Halleux1-Oct-03 1:06 
Generalsomething nebulous Pin
wac30-Sep-03 2:46
memberwac30-Sep-03 2:46 
GeneralRe: something nebulous Pin
Jonathan de Halleux1-Oct-03 1:07
memberJonathan de Halleux1-Oct-03 1:07 
GeneralRe: something nebulous Pin
Jonathan de Halleux2-Oct-03 23:53
memberJonathan de Halleux2-Oct-03 23:53 
GeneralSome fixes Pin
Jeroen Dirks23-Sep-03 10:57
memberJeroen Dirks23-Sep-03 10:57 
GeneralRe: Some fixes Pin
Jonathan de Halleux23-Sep-03 11:08
memberJonathan de Halleux23-Sep-03 11:08 
GeneralRe: Some fixes Pin
Jonathan de Halleux25-Sep-03 1:16
memberJonathan de Halleux25-Sep-03 1:16 
GeneralA bug in basic_unzip_streambuf Pin
gigimenegolo16-Sep-03 6:32
membergigimenegolo16-Sep-03 6:32 
GeneralRe: A bug in basic_unzip_streambuf Pin
Anonymous16-Sep-03 21:51
sussAnonymous16-Sep-03 21:51 
GeneralRe: A bug in basic_unzip_streambuf Pin
Jonathan de Halleux25-Sep-03 1:11
memberJonathan de Halleux25-Sep-03 1:11 
QuestionHow does this thing work? Pin
Thomas J. Clancy19-Aug-03 4:11
memberThomas J. Clancy19-Aug-03 4:11 
AnswerRe: How does this thing work? Pin
Jonathan de Halleux24-Aug-03 0:29
memberJonathan de Halleux24-Aug-03 0:29 
GeneralRe: How does this thing work? Pin
Thomas J. Clancy24-Aug-03 10:06
memberThomas J. Clancy24-Aug-03 10:06 
GeneralRe: How does this thing work? Pin
Anonymous24-Aug-03 20:49
sussAnonymous24-Aug-03 20:49 
Questionzip_ostream writing the wrong crc? Pin
vcken18-Aug-03 16:57
membervcken18-Aug-03 16:57 
AnswerRe: zip_ostream writing the wrong crc? Pin
Jonathan de Halleux24-Aug-03 0:25
memberJonathan de Halleux24-Aug-03 0:25 
Generalfile-to-file, stream-to-stream, but... Pin
Kochise12-Aug-03 2:45
memberKochise12-Aug-03 2:45 
Generalit does buffer to buffer ! Pin
Jonathan de Halleux12-Aug-03 2:58
memberJonathan de Halleux12-Aug-03 2:58 
Generala question Pin
seaskyzhang11-Aug-03 4:07
memberseaskyzhang11-Aug-03 4:07 
GeneralRe: a question Pin
Jonathan de Halleux11-Aug-03 7:40
memberJonathan de Halleux11-Aug-03 7:40 
Generalmultiple files from gzip Pin
Tim Enderling1-Aug-03 3:53
memberTim Enderling1-Aug-03 3:53 
GeneralRe: multiple files from gzip Pin
Jonathan de Halleux8-Aug-03 15:19
memberJonathan de Halleux8-Aug-03 15:19 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web04-2016 | 2.8.180515.1 | Last Updated 3 Oct 2003
Article Copyright 2003 by Jonathan de Halleux
Everything else Copyright © CodeProject, 1999-2018
Layout: fixed | fluid