Encrypt/Self Decrypt Files in Native C++ and .NET

Requirements

WTL/ATL7 projects are built as UNICODE. You can make the decryptor project as non-Unicode if you want to. Projects require Visual Studio 2003 or higher. WTL projects require WTL, which can be downloaded from the Microsoft Website. OpenSSL projects require OpenSSL Crypto Lib.

Introduction

This is a simple encrypt and self-decrypt project. It encrypts based on the chosen algorithm and password. It writes out a self-decrypting executable which asks for a password to decrypt the data. For a "native" version, the way it creates the self-extracting executable is by embedding the resources, using the NT-only UpdateResource function. For a .NET version, it builds the executable from embedded C# source and then writes out a resource file, passing it on to the compiler to be embedded.

Background

A few things (among many) to remember:

Quote: "When using CBC mode with block ciphers, you need to worry about CBC rollover. [...] On average, a cipher with an X-bit blocksize will have two ciphertext values collide every 2^X/2 blocks. Thus, it's important not to encrypt more than this amount of data with a given key" - SSL and TLS Designing and Building Secure Systems
Clear any sensitive data with such helpers as SecureZeroMemory (it's not Win2k3 only) and not with memset. In .NET, at least use Array.Clear. Refer to Scrubbing Secrets in Memory (MSDN).
As soon as you're done with an object , always call .Clear methods of the SymmetricAlgorithm/HashAlgorithm-derived classes and CryptoStream.
.NET comes with password-deriving class called PasswordDeriveBytes in cases when you need to derive a key from a user's input.
OpenSSL comes with a password-deriving function, also.

Code Snippets

ATL7 MS CryptoAPI Classes (without Error Checking)

//these are ATL7 helper classes for dealing with MS Crypto API
CCryptProv prov;
//initialize the MS crypto provider
HRESULT hr = prov.Initialize([PROV_RSA_FULL|PROV_RSA_AES], NULL, 
                             [MS_ENHANCED_PROV|MS_ENH_RSA_AES_PROV]); 
//this check is needed in cases when crypto api was not used before
//on the machine then it will return NTE_BAD_KEYSET and we just need
//to initialize again and specify CRYPT_NEWKEYSET param
if(hr == NTE_BAD_KEYSET) 
  hr = prov.Initialize(dwProviderType, NULL, szProvider, 
                      CRYPT_NEWKEYSET); 

//we are deriving encryption key from user input
CCryptDerivedKey derKey; 
//SHA hash will be used to derive the key
CCryptSHAHash sha;
sha.Initialize(prov); 

//get password from user
TCHAR pPass1[MAX_PASS_LEN], pPass2[MAX_PASS_LEN];  
int nPass1 = ::GetWindowText(hWndPwd1, pPass1, MAX_PASS_LEN);  
int nPass2 = ::GetWindowText(hWndPwd2, pPass2, MAX_PASS_LEN); 

//add user's input to our hash
hr = sha.AddData((BYTE*)pPass1, nPass1*sizeof(TCHAR)); 

//clear the password buffers
SecureZeroMemory(pPass1, MAX_PASS_LEN*sizeof(TCHAR)); 
SecureZeroMemory(pPass2,MAX_PASS_LEN*sizeof(TCHAR)); 

//SHA is 20 bytes, so we can specify 20 right away
BYTE tmpHash[20];
DWORD dw= sizeof(tmpHash); 
//let's hash the password 101 times
//dictionary based attacks against the user password become harder
for(int i=0;i<101; ++i)
{ 
  hr = sha.GetValue(tmpHash, &dw);
  sha.Destroy();
  hr = sha.Initialize(prov);
  hr = sha.AddData(tmpHash, dw);
}
//clear the temp hash buffer
SecureZeroMemory(tmpHash, dw);

//initialize the algorithm which derives the key from our SHA hash
//and specify the algorithm to be used for encryption
hr = derKey.Initialize(prov,sha,[CALG_AES_256|CALG_3DES]); 

//largest is 16 for AES, 8 for 3DES
BYTE iv[16];
//generate cryptographically strong random bytes
hr = prov.GenRandom(16, iv);
//set above bytes as IV
hr = derKey.SetIV(iv);
//set mode to CBC
hr = derKey.SetMode(CRYPT_MODE_CBC);
//set padding
hr = derKey.SetPadding(PKCS5_PADDING);

//find out "dummy" embeded decryptor executable
HRSRC res =FindResource(NULL, MAKEINTRESOURCE(IDR_DUMMY), _T("dummy")); 
HGLOBAL hGlobal = LoadResource(NULL, res); 
int resLen = SizeofResource(NULL, res); 
void* buf = LockResource(hGlobal); 
//write out the decryptor executable
FILE* file = _tfopen((LPCTSTR)sFileName, _T("wb")); 
fwrite(buf, 1, resLen, file);
fclose(file);

//start the resources update for decryptor executable
BOOL bRet = FALSE;
HANDLE hUpdateRes = BeginUpdateResource(sFileName, FALSE);
//encrypt user's chosen file's data to be written to the decryptor resources
derKey.Encrypt(TRUE, (BYTE*)data.m_pData, &dwFileSize, dwFileSize + blocksize);

//get rid of the hash
sha.Destroy();

//write out algid, so decryptor will know with which alg to decrypt
UpdateResource(hUpdateRes, RT_RCDATA, _T("1"), 
               MAKELANGID(LANG_NEUTRAL, SUBLANG_NEUTRAL), 
               (void*)(LPCTSTR)sAlgID, sAlgID.GetLength()*sizeof(TCHAR));
//write out filename, so decryptor will know what to name the file
//this can also be encrypted, but i didn't bother
UpdateResource(hUpdateRes, RT_RCDATA, _T("2"), 
               MAKELANGID(LANG_NEUTRAL, SUBLANG_NEUTRAL), 
               dlgOpen.m_szFileTitle, 
               _tcslen(dlgOpen.m_szFileTitle)*sizeof(TCHAR));
//write out the encrypted data
UpdateResource(hUpdateRes, RT_RCDATA, _T("3"), 
             MAKELANGID(LANG_NEUTRAL, SUBLANG_NEUTRAL), 
              data.m_pData, dwFileSize));
//write out random IV
UpdateResource(hUpdateRes, RT_RCDATA, _T("4"), 
             MAKELANGID(LANG_NEUTRAL, SUBLANG_NEUTRAL),
             iv, sizeof(iv))
//finish resource update
EndUpdateResource(hUpdateRes, FALSE);

//Similar steps for decryption, just in reverse
//WTL app that uses OpenSSL crypto lib is very similar to this

.NET App (without Error Checks)

//Use .NET's helper class to derive encryption Key and IV from user's input
//iterate hash 100 times
PasswordDeriveBytes pdb = new PasswordDeriveBytes(txtPass1.Text, 
                          UnicodeEncoding.Unicode.GetBytes("somesalt"), 
                          "SHA512", 100);
SymmetricAlgorithm symalg=null;
//pick the algorithm to be used for encryption from user's choice
//and set Key and IV
//algid number will be written to the resources so that decryptor will
//know which alg to use for decryption
if(cmbAlg.Text == "Rijndael")
{
  symalg = new OpenCrypto.RijndaelOpenProvider();
  algid = "1";
  //just standard sizes
  Key = pdb.GetBytes(32);
  IV = pdb.GetBytes(16);
}
else if(cmbAlg.Text == "TripleDES")
{
  symalg = new OpenCrypto.TripleDESOpenProvider();
  algid = "2";
  //just standard sizes
  Key = pdb.GetBytes(24);
  IV = pdb.GetBytes(8);
}
else if(cmbAlg.Text == "Blowfish")
{
  symalg = new OpenCrypto.BlowfishOpenProvider();
  algid = "3";
  //just standard sizes
  Key = pdb.GetBytes(32);
  IV = pdb.GetBytes(8);
}
//etc ...

//open the user's chosen file
Stream inStream = dlgEncryptFile.OpenFile();
//create encryptor
ICryptoTransform encryptor = symalg.CreateEncryptor(Key, IV);
//clear the Key, encryptor now has it
Array.Clear(Key, 0, Key.Length);
//clear the IV, encryptor now has it
Array.Clear(IV, 0, IV.Length);
//use CryptoStream helper to write out the data for encryption
CryptoStream cryptStream = new CryptoStream(inStream, encryptor, 
                                            CryptoStreamMode.Read);

//preallocate the whole buffer + OutputBlockSize
//whole buffer is preallocated because AddResource later on takes only buffer
byte[] encrData = new byte[inStream.Length + encryptor.OutputBlockSize];
int readBlockSize = encryptor.OutputBlockSize * 1000;
int totalEncrBytes=0;
//loop through file and encrypt
for(int BytesRead=0; totalEncrBytes < encrData.Length; 
    totalEncrBytes += BytesRead)
{
  if(totalEncrBytes + readBlockSize > encrData.Length)
    readBlockSize = encrData.Length - totalEncrBytes;
  BytesRead = cryptStream.Read(encrData, totalEncrBytes, readBlockSize);
  if(BytesRead == 0)
    break;
}

//clear any sensitive resources
encryptor.Dispose();
cryptStream.Clear();
cryptStream.Close();
symalg.Clear();

//write out the resources into standard .net .resources file
ResourceWriter resWriter = new ResourceWriter(applicationPath + 
                                                "\\encrypted.resources");
resWriter.AddResource("1", algid);
//filename
resWriter.AddResource("2", fileTitle);
//total bytes encrypted
resWriter.AddResource("3", totalEncrBytes);
//encrypted file's data
resWriter.AddResource("4", encrData);
encrData = null;
inStream.Close();

//generate the resources
resWriter.Generate();
resWriter.Dispose();

//build the decryptor assembly dynamically using CSharpCodeProvider
BuildDecryptorAssembly();

//this is in finally{} block
//to ensure that .resources file will be deleted even if exception was thrown
if(File.Exists(applicationPath + "\\encrypted.resources"))
  File.Delete(applicationPath + "\\encrypted.resources");
    
private void BuildDecryptorAssembly()
{
//standard stuff to compile and build the assembly dynamically
  CSharpCodeProvider csprov = new CSharpCodeProvider();
  ICodeCompiler compiler = csprov.CreateCompiler();

//specify standard references, as well as the opencrypto.dll assembly
  CompilerParameters compilerparams = new CompilerParameters(new string[] 
                {"System.dll", "System.Windows.Forms.dll", "System.Drawing.dll", 
                  "opencrypto.dll"},
                  txtOutFile.Text, false);

  compilerparams.GenerateExecutable = true;
  
//specify our encrypted resource to be put into the new decryptor assembly
  compilerparams.CompilerOptions = "/target:winexe /resource:\""
                                   + applicationPath + 
                                   "\\encrypted.resources\",encrypted";
                                   
//compile assembly from source
  CompilerResults cr = compiler.CompileAssemblyFromSource(compilerparams, ...);
  
//this is just for debugging to see if there are any errors
  CompilerErrorCollection errs = cr.Errors;
  if(!errs.HasErrors)
    MessageBox.Show("Operation Successful");
  else
  {
    foreach(CompilerError err in errs)
      MessageBox.Show(err.ToString());
  }
}

//The pure .NET App's code is almost exactly the same

Two Projects Depend on OpenSSL

Two of the samples in this article build upon the OpenSSL crypto library, which builds without any problems with a Visual C++ 7 compiler. For building instructions, refer to the INSTALL.W32 file that comes with the OpenSSL library. You can build a Release/Debug of static/DLL library versions. Some of the algorithms implemented by the OpenSSL library are patented. Please refer to the README file of the OpenSSL library, especially to the Patents section. Here is the quote from the README file:

"Various companies hold various patents for various algorithms in various locations around the world. You are responsible for ensuring that your use of any algorithms is legal by checking if there are any patents in your country. The file contains some of the patents that we know about or are rumored to exist. This is not a definitive list.

RSA Security holds software patents on the RC5 algorithm. If you intend to use this cipher, you must contact RSA Security for licensing conditions. Their web page is http://www.rsasecurity.com/. RC4 is a trademark of RSA Security, so use of this label should perhaps only be used with RSA Security's permission.

The IDEA algorithm is patented by Ascom in Austria, France, Germany, Italy, Japan, the Netherlands, Spain, Sweden, Switzerland, UK and the USA. They should be contacted if that algorithm is to be used; their web page is http://www.ascom.ch/"

To build a static library without patented algorithms -- for example, mdc2, rc5, idea, etc. -- define: no-mdc2 no-rc5 no-idea in do_ms.bat for Visual C++ WIN32. About the legal issues, if any, with RC* algorithms, you can read here.