Click here to Skip to main content
11,496,146 members (529 online)
Click here to Skip to main content

Zip bytes in memory and unzip file into bytes buffer

, 7 Sep 2004 55.5K 1.6K 46
An article on how to zip bytes in memory
The site is currently in read-only mode for maintenance. Posting of new items will be available again shortly.


Currently, I am coding a web server running under MS windows, for performance, I need to zip htm files in memory and return gzipped content to client. After googling for a while, I couldn't find what I needed, maybe I missed it. If you don't want to code Huffman and LZ77 arithmetic by yourself from scratch, this is a shortcut for you. What I have done is just extracting all deflate codes into just one header file 'deflate.h', omit all things that can be, inline all functions. Thanks to god, there is no cyclic references problem in the end.

The second job was done in last year, which can be used to unzip file. I also extracted all source relate to 'inflate' into one header file 'inflate.h'. I wish you don't call me "c++ scripter" after using them Smile | :)

By using this, there will be nothing needed such as linking with gzip lib in your project. gzip is a large lib, for easy reading, I omitted the copyright statement, but I wish you do that!

Using deflate.h to zip bytes buffer

You should use the compress function as following:

  FILE* out;
  Byte*    outbuf = (Byte*)ALLOC(OUTBUFLEN);
  int      outlen = 0;

  Byte*    inbuf = (Byte*)ALLOC(INBUFLEN);
  int      len;
  int      a;

  FILE* in = fopen("aaa.htm","rb");
  len = fread(inbuf,sizeof(Byte),INBUFLEN,in);

  //for(a=0;a<1000;a++) // for test purpose.
    outlen = compress(inbuf,len, outbuf);

  out = fopen("bbb.gz","wb");
  len = fwrite(outbuf,sizeof(Byte),outlen,out);

As you know, you can use other calc'ed result byte buffer as your input. What I added into gzip is the "compress" function in the end of 'deflate.h' head file, which is like the following:

int compress(Byte* inbuf,int len, Byte* outbuf)
  int err    = Z_OK;
  z_stream  strm;
  uLong crc  = 0;     /* crc32 of uncompressed data */
  int done  = 0;
  int  outlen  = 0;

  Byte*    nextout; /* output buffer */
  strm.zalloc  = (alloc_func)0;
  strm.zfree  = (free_func)0;
  strm.opaque  = (Byte*)0;
  strm.next_in  = Z_NULL;
  strm.next_out = Z_NULL;
  strm.avail_in = strm.avail_out = 0;
  err = deflateInit2(&strm, Z_BEST_COMPRESSION, Z_DEFLATED, 
  /* windowBits is passed < 0 to suppress zlib header */
  strm.next_out = (Byte*)ALLOC(Z_BUFSIZE);
  nextout = strm.next_out;

  strm.avail_out = Z_BUFSIZE;

  // write gzip header
  outbuf[outlen++] = 0x1f;
  outbuf[outlen++] = 0x8b;
  outbuf[outlen++] = Z_DEFLATED;
    outbuf[outlen++] = 0;
  outbuf[outlen++] = 0xff;

  strm.next_in  = (Byte*)inbuf;
  strm.avail_in = len;

  while (strm.avail_in != 0)
      err = deflate(&strm, Z_NO_FLUSH);
        if (err != Z_OK) break;

  crc = crc32(crc, (const Byte *)inbuf, len);
  if (strm.avail_in != 0) 
    return -1;  
  /* should be zero already anyway */
  for (;;) {
    len = Z_BUFSIZE - strm.avail_out;

    if (len != 0) {
      zmemcpy(outbuf+outlen, nextout, len);
      outlen += len;

      strm.next_out = nextout;
      strm.avail_out = Z_BUFSIZE;
    if (done) 
    err = deflate(&strm, Z_FINISH);
    done = (strm.avail_out != 0 || err == Z_STREAM_END);

  // write gzip tailer
  outbuf[outlen++] = (Byte)crc&0xff;
  outbuf[outlen++] = (Byte)crc>>8&0xff;
  outbuf[outlen++] = (Byte)crc>>16&0xff;
  outbuf[outlen++] = (Byte)crc>>24&0xff;
  outbuf[outlen++] = (Byte)strm.total_in;
  outbuf[outlen++] = (Byte)strm.total_in>>8;
  outbuf[outlen++] = (Byte)strm.total_in>>16;
  outbuf[outlen++] = (Byte)strm.total_in>>24;

  if (strm.state != Z_NULL) 
    err = deflateEnd(&strm);

  return outlen;

if you dislike those typedef's such as Byte in your project, you can undefine them in the end of the header file.

Using inflate.h to unzip file into bytes buffer.

As to unzip file, you include 'inflate.h' in your project, and use it like the following:

  LUFILE*    zf = NULL;
  DWORD e;    // for debug use
  zf = lufopen(bstrTheme,0,2,&e);
  if (zf==NULL) 
    return FALSE;
  long nlen;
  BYTE* buf = NULL;
  ATLTRY(buf = new BYTE[MAX_LEN]);
  char szCurFileName[UNZ_MAXFILENAMEINZIP+1];

  unzFile uf = unzOpenInternal(zf); // there is a gotofirstfile in the end
    goto LError;

  int err = unzGoToFirstFile(uf);
  while (err == UNZ_OK)
    int n = lstrlenA(szCurFileName)-1;

    bool haderr=false;
    nlen = 0;
    for (;;)
      int res = unzReadCurrentFile(uf,buf,MAX_LEN);
      nlen += res;
      if (res<0) 
      if (res==0) 
    err = unzGoToNextFile(uf);


Note, this header was extracted from zlib version 1.1.3, and the 'deflate.h' was extracted from gzip 1.2.4.

Points of Interest

if you want more compression ratio, you can take this as your start, and change those configuration value, maybe other stuff :-P If you successfully did that at last, please let me know. Anyway, don't ask me what's LZ77, I know nothing about it, and I also don't know who Huffman is, and so on, sorry for my ignorance. As I think, he must be a boring man Wink | ;-)


All these 2 files had been tested by using Winzip 9.0

Comments are appreciated, thanks for your time.


This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


About the Author

Software Developer (Senior)
China China
No Biography provided

Comments and Discussions

Questioninflate.h - Thanks Pin
Castaa14-Aug-11 18:12
memberCastaa14-Aug-11 18:12 
QuestionThis one not work propertly Pin
EugeneWechsler29-Jul-11 2:55
memberEugeneWechsler29-Jul-11 2:55 
I tried to use this method but it definitely doesn't work with not small files (output greater than buffer). Then I studied ZLib sources - the interesting one is their "Compress" method. It can't be used for GZIP because adds ZLIB headers to output instead GZIP headers. But its code can be easily
modified with use of this topic's code. Just replace DeflateInit with DeflateInit2 pass windowBits < 0 , add GZIP headers and trailer and you get normal GZIP on output for one iteration (single DEFLATE call) without using additional buffers.

I tested results - decompressed them later with .NET GZipStream.

I am not posting sources because they are on Clarion. Anyone interested in them anyway?
QuestionCan you give me an example in uncompress !! Pin
Minh Quang Nguyen24-Oct-07 17:58
memberMinh Quang Nguyen24-Oct-07 17:58 
QuestionHow to convert bytes array to char array Pin
Balkrishna Talele23-Mar-06 19:38
memberBalkrishna Talele23-Mar-06 19:38 
Generalerror in uncompress Pin
Shlomo12-Jul-05 12:22
memberShlomo12-Jul-05 12:22 
QuestionHow can I uncompress Pin
Klaus Kurt24-Jun-05 5:47
memberKlaus Kurt24-Jun-05 5:47 
AnswerRe: How can I uncompress Pin
Ajay Reddy6-Mar-13 14:44
memberAjay Reddy6-Mar-13 14:44 
GeneralDoes not work for large file Pin
jaypt22-Feb-05 8:26
memberjaypt22-Feb-05 8:26 
QuestionRe: Does not work for large file Pin
ksyroczynski8-Sep-06 12:28
memberksyroczynski8-Sep-06 12:28 
AnswerRe: Does not work for large file Pin
ldehai6-Apr-07 5:03
memberldehai6-Apr-07 5:03 
GeneralThank you! Pin
rainbowsally25-Sep-04 1:16
memberrainbowsally25-Sep-04 1:16 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.150520.1 | Last Updated 8 Sep 2004
Article Copyright 2004 by ChauJohnthan
Everything else Copyright © CodeProject, 1999-2015
Layout: fixed | fluid