Click here to Skip to main content
Click here to Skip to main content

QuickLZ Pure C# Port

, 23 Dec 2006
Rate this:
Please Sign up or sign in to vote.
This article describes a C# port of the new QuickLZ compression algorithm


It seems there is no end to the porting of good C/C++ algorithms to .NET, and so here comes another one.  This algorithm comes courtesy of Lasse Mikkel Reinhold, and is called QuickLZ.

You can read more about QuickLZ at the website, however I will outline some of the aspects of QuickLZ as I go.

What IS QuickLZ?

It comes as no surprise if you haven't heard of QuickLZ, it is a new compression algorithm particularily well suited for psuedo streaming (packet compression).  It was originally released early November 2006, and updated to 1.10 later that month.  The porting is based on 1.10, which as of the writing of this article boasts an incredible 20-30% improvement over other well known algorithms such as LZO1X and LZF.  Complete benchmark information on the C implementation can be found at the above linked website, however here is an overall assessment:

QuickLZ 1.10 LZO 1X 2.02 LZP-1 LZF 1.6 ZLIB-1 1.2.3
Compressed Size 61.0% 60.2% 59.9% 60.7% 46.4%
Compression MB/Sec 148 81.8 60.4 60.9 7.45
Decompression MB/Sec 380 307 89.3 198 120

If you are reading this article, there is a chance you also read my article including a port of MiniLZO.  Based on the numbers here, it's easy to see why LZO was my previous choice.  However, QuickLZ is now 19.2% faster on average than LZO on decompression, and an astonishing 44.7% faster on compression.  And approximately 0.8% less efficient on final compression.  This large improvement by QuickLZ made it a prime candidate for porting to .NET and while it doesn't totally replace my minilzo port, it does give a better alternative for those who don't need the LZO format.

Things To Remember

While I was porting the code, which was incredibly clear and easy to port, I noticed a number of areas for improvement.  This specific iteration of the port is a first revision and is based on the optimizations for better ratio working with data > 100kb in size.  A more refined and "Minimalistic" version of the algorithm is possible, and may come as a complimentary article specifically regarding packet compression.  Meanwhile, this class serves as a general use class for both files and packets that should outperform other managed options.  I have not profiled this port to C# whatsoever, and I know that for simplicity of the public interface there is some additional overhead.  If you must make the Unsafe versions public and use them, make sure you understand the code before you cry to me about it not working Smile | :)

A comment made during the port of MiniLZO was how the original uncompressed size was not easy to attain from the compressed data, so prepending the size was requested but would ultimately break compatibility with the original LZO1X implementation.  Based on the standard implementation of QuickLZ, I have provided a public method to find the original uncompressed size quickly and painlessly.

Without Further Ado...

So let's look at how to use the safe(?) interface to the implementation.  All source examples are psuedo, and assume knowledge of basic C# operations.

byte[] data = File.ReadAllBytes("...");
byte[] compressed = QuickLZ.Compress(data, 0, (UInt32)data.Length);

This little snippet shows the basic idea of obtaining some data, in this case loading it from a file, and passing it into the compression method, which takes a Byte[] for the buffer to read from, a starting offset within the buffer to start at, and the length from the starting point to include in compression.  This is useful if you need to work with a packet, where packet headers may not be included in the compression/decompression routines.  The return value is an exact size Byte[] stripped of the extra scratch space allocated during compression.  It should be noted that QuickLZ imposes a requirement that during compression the destination buffer is required to be the length of the original data plus an additional 36000 bytes for temporary workspace.  This implementation copies the temporary destination buffer into an exact size buffer for final return.

UInt32 size = QuickLZ.GetDecompressedSize(data, 0);

This little snippet here shows how you can manually obtain the size of the compressed data out of the Byte[] directly.  The second argument, again, reflects where the actual compressed data headers begin.  It should be noted that this method is never really needed, but is provided for completion.

byte[] decompressed = QuickLZ.Decompress(compressed, 0);

This last snippet is fairly self explanitory by this point. It takes a Byte[] for the compressed data, and a starting point once again.  It returns an exact size buffer like Compress, however does not have the added overhead of copying from a temporary destination, as the exact uncompressed size is known ahead of time, which is why the GetDecompressedSize method is not required.


QuickLZ as a new alternative to the compression scene and seems to currently be the fastest within this category of compression.  I hope you will find this port useful, and as always if you find any bugs or bottlenecks that degrade performance please let me know.


This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


About the Author

Web Developer
Canada Canada
Short and simple, I'm a self contracted programmer, my strongest programming skills are in C/C++ and C#/.NET. I have a nack for porting C algorithms to C#.

Comments and Discussions

GeneralWell done Pinmemberronnotel16-Jan-07 4:47 
GeneralThis should be a stream implementation to follow .NET semantics. PinmemberDale Thompson8-Jan-07 6:23 
GeneralRe: This should be a stream implementation to follow .NET semantics. PinmemberBehind The Scene13-Jan-07 19:16 
Generalversion without unsafe code PinmemberColby Dillion27-Dec-06 21:42 
GeneralRe: version without unsafe code PinmemberAstaelan28-Dec-06 9:38 
GeneralRe: version without unsafe code PinmemberColby Dillion28-Dec-06 11:48 
GeneralRe: version without unsafe code PinmemberAstaelan28-Dec-06 13:13 
GeneralRe: version without unsafe code PinmemberColby Dillion28-Dec-06 16:53 
I'm going to reply to your private message soon but I thought I would share this for everyone else.
I modified the Compress method of your code to use the same Array.Resize method as mine. Compressing a 7.5Mb buffer in a loop 100 times it shaved about 0.8 seconds off of a 23 second run. I don't think that is the major hold up. I think the main thing is that the JIT is able to compile the array access code down to something remarkably similar to the unsafe pointer code. Also, I believe that because it is using the array access methods of the language the JIT is able to perform better optimizations whereas your pointer code is mostly limited to what you are able to do with your algorithm and pointer math smarts.
End result is that I believe our code turns into something similar which is the reason there isn't a huge speed gap. Aside from the memcpy_up method, I don't believe there are any optimizations you could make in your code that I couldn't make in mine.
I work in the medical imaging field and have been able to write code using C# to perform real time operations on large images (2000x2000 pixels) without using pointer math. Much of our software is deployed using ClickOnce and is run in an IE app domain and so we don't use unsafe assemblies. The .Net runtime provides many features to help you write very efficient code in a short amount of time, if you know where to find them. Also, because C# is compiled to IL, it isn't always as intuitive as you would think to find hot spots in your code and so a good profiler such as red gate or dottrace is invaluable.
It comes down to asking yourself if you want to fight the machine or use it to your advantage. Arrays are much nicer to work with. Wink | ;)
GeneralRe: version without unsafe code PinmemberAstaelan1-Jan-07 2:04 
GeneralRe: version without unsafe code PinmemberColby Dillion1-Jan-07 10:05 
GeneralRe: version without unsafe code PinmemberAstaelan1-Jan-07 10:23 
QuestionIs there a way to detect Network Printers' IP addresses? PinmemberJan Palmer25-Dec-06 22:03 
NewsCompression ratio for small data sources Pinmemberrlasse25-Dec-06 9:48 
GeneralRe: Compression ratio for small data sources PinmemberAstaelan25-Dec-06 18:15 
GeneralInteresting PinmemberStanislav Panasik24-Dec-06 23:21 
GeneralRe: Interesting PinmemberPHS24125-Dec-06 0:43 
GeneralRe: Interesting PinmemberStanislav Panasik25-Dec-06 6:32 
GeneralRe: Interesting PinmemberAstaelan25-Dec-06 7:42 
AnswerRe: Interesting PinmemberStanislav Panasik25-Dec-06 8:06 
GeneralRe: Interesting PinmemberAstaelan25-Dec-06 8:19 
GeneralRe: Interesting PinmemberStanislav Panasik26-Dec-06 0:22 
GeneralRe: Interesting PinmemberBehind The Scene13-Jan-07 19:20 
GeneralRe: Interesting Pinmemberrlasse25-Dec-06 10:36 
GeneralRe: Interesting PinmemberAstaelan25-Dec-06 18:21 
GeneralRe: Interesting PinmemberAstaelan25-Dec-06 7:39 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web03 | 2.8.150302.1 | Last Updated 23 Dec 2006
Article Copyright 2006 by Astaelan
Everything else Copyright © CodeProject, 1999-2015
Layout: fixed | fluid