Click here to Skip to main content
Click here to Skip to main content

Encryption and compression, native and managed

By , 29 Jun 2012
Rate this:
Please Sign up or sign in to vote.

Imagen

Imagen

Imagen

Introduction

Every application usually stores data, and basic encryption is needed to prevent access by unauthorized subjects. If you are using C++, Crypto++ is a very powerful library to encrypt and compress. The problem is that I had many implementations of the same code. I had some using CString, some using ATL. Some were using GZip, some ZLib. For the web, I used RSA (because .NET supports it); for pure native implementations, I used DEM encryption. Besides, compiling the Crypto++ sources every time a project takes too long.

Usually, given a message, 3 operations are used to convert data:

  • Compression:To reduce the size of the info
  • Encryption:To prevent unauthorized access
  • Text conversion:Binary information can't be used directly in XML and other formats. So, conversion to b64 and others is useful
For each operation there are different algorithms supported by this library:
  • Compression:ZLib, GZip
  • Encryption:AES, RSA, DEM (only unmanaged), Blowfish
  • Text conversion:Binary, base 2, 8, 16 (hex), 64
All encryption algorithms use CBC and PKCS7 padding.

The solution contains several projects:
  • CryptoLib:Library that contains Crypto++ and unmanaged algorithms


  • CryptTest:MFC application that uses CryptoLib


  • NetCryptLib:.NET C++/CLI library to use CryptoLib (Crypto++) from .NET


  • NETPCryptLib:Pure .NET encryption library using .NET libraries only (no unmanaged code)


  • ZLib.NET:.NET managed zlib compression for NETPCryptLib


  • NetCryptTest:.NET WinForms application that uses NetCryptLib and NETPCryptLib


  • NetWebTest:.NET Web application that consumes modifies and returns converted info from NetCryptTest


  • RSAKeyGenerator:.NET RSA Key generator


Basic structure

The basic idea is to keep all the conversion stuff away from the user. So, all the processing is done in an "Internal" class that the user has no access to.

The user should only be able to access the encryption keys, the main buffer (to write and get data), and the conversion options.

This is the basic structure of the conversion libraries (CryptoLib, NetCryptLib, NetPCryptLib):

ImgProc

For the .NET library, a "NET" prefix is added to classes and variables. For pure .NET, the "NETP" prefix is used.

Libraries Usage

  • 1. Assign encryption parameters (keys, CBC block, etc). This is optional because the library generate random parameters on startup
  • 2. Set conversion options.
  • 3. Fill buffer with data.
  • 4. Process.

Samples

This sample can be found in the "Test" button in the CryptTest MFC application.

//Key and CBC initial block
const int maxoutputlen = 1000000;
const unsigned char BFKEY[16] = { 0xa1, 0xb2, 0xc3, 0xd4, 0xe5, 0xf6, 0xe1, 0xd2, 0xc3, 0xb4, 0xa5, 0xf3, 0x9f, 0x38, 0x2f, 0x01 }; 
const unsigned char CBCIV[8] = {0xaa, 0xbb, 0xcc, 0xdd,0xee, 0xff, 0x11, 0x22};

//Create new object
CInfoFormat* infotest = new CInfoFormat();

//Set conversion options
infotest->optbuf->optencrypt = TEncryptBlowFish;
infotest->optbuf->optzip = TZipZLib;
infotest->optbuf->optoutput = TOutputBase64;

//Set mode (encode/decode)
infotest->optbuf->optoper = TOperEncode;

//Set Keys
memcpy(infotest->keys->bfkey,BFKEY,16);
memcpy(infotest->keys->bfiv,CBCIV,8);
infotest->keys->bfkeylen = 16;

//Reinitialize encryption objects with the new keys
infotest->InitEngine();

//Assign the buffer a test string
strncpy_s((char*)(infotest->optbuf->buffer), maxoutputlen, "Test", 4);
infotest->optbuf->bufferlen = 4;

//Perform the conversion
infotest->OpClose();

//Assign the result to a CString
CString CadRes = CString((char*)infotest->optbuf->buffer, (int)infotest->optbuf->bufferlen);
//Show result
AfxMessageBox(CadRes);

//Set mode to decode
infotest->optbuf->optoper = TOperDecode;
//Decode the buffer which is already encrypted in the object
infotest->OpClose();

//Assign the result to a CString
CString CadDes = CString((char*)infotest->optbuf->buffer, (int)infotest->optbuf->bufferlen);
//Show result
AfxMessageBox(CadDes);

//Delete object
delete infotest;

This sample can be found in the "Test Crypto++/CLI" button in the NETCryptTest .NET application.

//Key and CBC initial block
byte[] BFKEY = { 0xa1, 0xb2, 0xc3, 0xd4, 0xe5, 0xf6, 0xe1, 0xd2, 0xc3, 0xb4, 0xa5, 0xf3, 0x9f, 0x38, 0x2f, 0x01 }; 
byte[] CBCIV = {0xaa, 0xbb, 0xcc, 0xdd,0xee, 0xff, 0x11, 0x22};

//Create new object
CNETInfoFormat infotest = new CNETInfoFormat();

//Set conversion options
infotest.NETbuf.optNetencrypt = TNETEnumEncrypt.TNETEncryptBlowFish;
infotest.NETbuf.optNetzip = TNETEnumZip.TNETZipZLib;
infotest.NETbuf.optNetoutput = TNETEnumOutput.TNETOutputBase64;

//Set mode (encode/decode)
infotest.NETbuf.optNetoper = TNETEnumOperation.TNETOperEncode;

//Set Keys
BFKEY.CopyTo(infotest.NETkeys.NETbfkey, 0);
infotest.NETkeys.NETbfkeylen = BFKEY.Length;
CBCIV.CopyTo(infotest.NETkeys.NETbfiv, 0);

//Reinitialize encryption z with the new keys
infotest.Initialize();

//Assign the buffer a test string
byte[] bres = Encoding.ASCII.GetBytes("Prueba");
bres.CopyTo(infotest.NETbuf.NETbuffer, 0);
infotest.NETbuf.NETbufferlen = bres.Length;

//Assign unmanaged params from managed ones
infotest.NETbuf.AssignParams();

//Perform the conversion
infotest.OpClose();

//Assign the result to string
string strenc = Encoding.ASCII.GetString(infotest.NETbuf.NETbuffer, 0, infotest.NETbuf.NETbufferlen);
//Show result
MessageBox.Show(strenc);

//Set mode to decode
infotest.NETbuf.optNetoper = TNETEnumOperation.TNETOperDecode;
//Assign unmanaged params from managed ones
infotest.NETbuf.AssignParams();
//Decode the buffer which is already encrypted in the object
infotest.OpClose();
            
//Assign the result to string
string strdec = Encoding.ASCII.GetString(infotest.NETbuf.NETbuffer, 0, infotest.NETbuf.NETbufferlen);
//Show result
MessageBox.Show(strdec);

This sample can be found in the "Test Pure .NET" button in the NETCryptTest .NET application.

//Key and CBC initial block
byte[] BFKEY = { 0xa1, 0xb2, 0xc3, 0xd4, 0xe5, 0xf6, 0xe1, 0xd2, 0xc3, 0xb4, 0xa5, 0xf3, 0x9f, 0x38, 0x2f, 0x01 };
byte[] CBCIV = { 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff, 0x11, 0x22 };

//Create new object
CNETPInfoFormat infotest = new CNETPInfoFormat();

//Initialize object
infotest.Initialize();

//Set conversion options
infotest.NETPBuf.optPNetencrypt = TNETPEnumEncrypt.TNETPEncryptBlowFish;
infotest.NETPBuf.optPNetzip = TNETPEnumZip.TNETPZipZLib;
infotest.NETPBuf.optPNetoutput = TNETPEnumOutput.TNETPOutputBase64;

//Set mode (encode/decode)
infotest.NETPBuf.optPNetoper = TNETPEnumOperation.TNETPOperEncode;

//Set Keys
BFKEY.CopyTo(infotest.NETPKeys.NETPbfkey, 0);
infotest.NETPKeys.NETPbfkeylen = BFKEY.Length;
CBCIV.CopyTo(infotest.NETPKeys.NETPbfiv, 0);

//Assign encryption objects the new keys
infotest.InitKeysCrypt();

//Assign the buffer a test string
byte[] bres = Encoding.ASCII.GetBytes("Prueba");
bres.CopyTo(infotest.NETPBuf.NETPbuffer, 0);
infotest.NETPBuf.NETPbufferlen = bres.Length;

//Perform the conversion
infotest.OpClose();

//Assign the result to string
string strenc = Encoding.ASCII.GetString(infotest.NETPBuf.NETPbuffer, 0, infotest.NETPBuf.NETPbufferlen);
//Show result
MessageBox.Show(strenc);

//Set mode to decode
infotest.NETPBuf.optPNetoper = TNETPEnumOperation.TNETPOperDecode;
//Decode the buffer which is already encrypted in the object
infotest.OpClose();

//Assign the result to string
string strdec = Encoding.ASCII.GetString(infotest.NETPBuf.NETPbuffer, 0, infotest.NETPBuf.NETPbufferlen);
//Show result
MessageBox.Show(strdec);

Files

To process large files or messages, the libraries use OpOpen() to keep adding data and OpClose() when finished.

Web

For the web sample, the WinForms application sends a POST message and adds the converted text in the post.
The web application (ASP.NET) decodes the text, modifies it and encodes it back.
The Winforms receives the result encoded, decodes it and shows it.

The ASP.NET uses the pure .NET library because some servers don't admit C++/CLI libraries.

Compatibility between libraries (pure .NET and Crypto++/CLI)

A message can be encoded with Crypto++/CLI library and decoded with the pure .NET library (and vice versa). To ensure that it can be done, both libraries must have the same encryption keys and CBC initial blocks.

So there are two methods in the .NET Test application:
  • AssignRSAToCryptoPlus
  • SetManPWDFromCryptoPlus
AssignRSAToCryptoPlus assigns the random generated .NET keys to the Crypto++ library. For some reason, the Crypto++ random generated RSA numbers don't work with .NET libraries
SetManPWDFromCryptoPlus assign all the random parameters from the Crypto++ library to the .NET one.

For the web application, the pure managed library contains a class "CGlobalKeys" so that both libraries use the same fixed keys.

Complications faced

I tried to use the class MemoryStream when possible in the .NET but in the encryption algoritms failed to work correctly.

One of the most difficult things was to keep track of the "remanent" bytes when adding data with OpOpen() in the .NET pure library
That's because the size of the buffer used is not a multiple of each transformation size. And I wanted to keep the buffer size arbitrary.
For that a class CNETBufTemp is responsible to add the last bytes of the buffer (that can't be processed) to the beginning of the next one.

I also had to add padding because encryption algorithms don't provide it. For the pure .NET zlib I used ComponentAce ones.

Some stats

Could't resist making some performance stats between libraries

Type Compression Encryption Output MFC Crypto++/CLI Pure .NET
Encoding None None B64 438 420 649
Decoding None None B64 282 310 1,370
Encoding None None Hex 953 959 59,818
Decoding None None Hex 593 647 1,533
Encoding None RSA None 4,093 5,787 7,601
Decoding None RSA None 93,859 100,654 151,542
Encoding None Blowfish None 109 117 1,305
Decoding None Blowfish None 109 114 1,377
Encoding None AES None 125 116 407
Decoding None AES None 156 149 507
Encoding Gzip None None 625 591 470
Decoding GZip None None 250 254 110
Encoding ZLib None None 640 615 3,763
Decoding ZLib None None 250 246 2,706

Type Compression Encryption Output MFC Crypto++/CLI Pure .NET
Encoding Gzip RSA B64 4,844 6,451 9,685
Decoding Gzip RSA B64 87,140 90,469 155,662
Encoding Gzip Blowfish B64 1,047 1,011 2,403
Decoding Gzip Blowfish B64 531 556 2,784
Encoding Gzip AES B64 1,047 995 1,478
Decoding Gzip AES B64 547 582 1,947
Encoding ZLib RSA B64 4,813 6,341 12,197
Decoding ZLib RSA B64 87,187 90,317 142,540
Encoding ZLib Blowfish B64 1,047 995 5,588
Decoding ZLib Blowfish B64 531 553 6,268
Encoding ZLib AES B64 1,032 985 4,695
Decoding ZLib AES B64 562 587 5,383

For all this tests, a 4,62Mb file was used. In both the MFC App and the .NET app, the if{"DoEvents"} part was disabled.

There are some simple conclusions that I extract from this tests:

  • Pure managed encryption is very slow. ZLib and Blowfish are from 6 to 11 times slower that native counterparts.
    If I were writing a managed compression, encryption or any other type of "bit management algorithm", I wouldn't use .NET.
    But there are times when you have no option but to use "pure" .NET and then this libraries are very useful.


  • .NET RijndaelManaged is 4 times slower that native but I had to do some buffer management to make it work (could be improved).


  • .NET GZipStream was faster that the Crypto++ counterpart. Probably using faster algorithm inside.


  • In many cases, the C++/CLI (with interop included) was faster that the MFC native. Probably the cause being CFile slower that BinaryReader/BinaryWriter.


  • I had to put A LOT of effort to make the pure .NET work. I could't find an octal conversion that works OK, and the hexadecimal pure .NET is very, very slow.
    Crypto++ might have a bit of a learning curve (specially the compiling part).
    But when you are done, it's all there. And the design (templates, etc) help a lot.


  • RSA is too slow. It should be used with AES or Blowfish and a random key (RSA encrypts the random key used in AES y Blowfish).


  • .NET C++/CLI interop didn't affect the results much.


  • There was a lot of dispersion in the pure .NET results when running the same benchmark several times.

My conclusion: .NET is very, very fast as long as you can use the built in classes in the framework.
Once you start doing buffer management, bytes and bits processing, custom conversions, lots of layers of processing, etc, performance suffers......a lot.

Fortunately, the .NET framework is really vast and C++/CLI can provide fast libraries for .NET to fill the gaps when performance matters.

Source code and test apps

The source project is VS2010 and the binaries are compiled using VS2010.
I have to see if I compile it in older versions of VS (to do).

To run the Web test, set the NetWebTest project as Startup Project and run it. Then run the NETCryptTest binary.
This way, you can even debug the web app.

Remember that random keys are loaded each time a test app start, so if you encrypt a file, close the app and try to decrypt it, an error will occur.

The sourcecode does't do any error management (to do).
There are a lot of improvements to do to the sourcecode.

History

  • 2012-06-29:
    • Added .NET library and .NET pure library
    • Added .NET test apps
    • Added support for blowfish, bits, octal and hex
    • Added stats
    • Added contiguous file processing, buffer size independent
    • Memory improvements and error corrections
  • 2009-08-26. First version.

References

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

ErnestoNet
Software Developer
Argentina Argentina
System developer from Argentina.
 
Programmed in VB 5,6,.NET, C#, Java, PL-SQL, Transac-SQL, C, C++ and even some "calculator" language.
 
Love to build small, useful applications.
Usually building big and complicated apps based on solid, reliable components.
 
Hobbies: reading, photography, chess, paddle, running.

Comments and Discussions

 
GeneralMy vote of 5 Pinmembergabegabe15-Sep-13 5:53 
BugThe download zip files are broken PinmemberSimon Hughes7-Jul-12 6:20 
GeneralRe: The download zip files are broken PinmemberErnestoNet7-Jul-12 14:45 
GeneralRe: The download zip files are broken PinmemberSimon Hughes9-Jul-12 2:04 
General非常实用 Pinmemberpophelix10-Dec-09 22:52 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web03 | 2.8.140415.2 | Last Updated 29 Jun 2012
Article Copyright 2009 by ErnestoNet
Everything else Copyright © CodeProject, 1999-2014
Terms of Use
Layout: fixed | fluid