Click here to Skip to main content
Rate this: bad
good
Please Sign up or sign in to vote.
See more: C++ C cryptography
I have been playing around with the Microsoft Crypto API. Using CALG_RC4 I am trying to encrypt a plaintext where the key is derived from an md5 hash of the plaintext.
 
I have verified that the "password" md5, ciphertext and lengths of both buffers are the same when I pass them to the decrypt function as they were after being encrypted.
 
CryptDecrypt however does not successfully decrypt the ciphertext back to plaintext.
 
Does anyone know why? I'm obviously doing something wrong.
 
An additional note: the context (HCRYPTPROV) is different between the two functions (they are in separate binaries). Not sure if that matters, I know it does for certain KeyBlob stuff.
 
Forgive the poor formatting of my copy paste:
 
BOOL _Encrypt( MessagePair * _write_data )
{
 
static  HCRYPTPROV			hCryptContext		= NULL;
	HCRYPTHASH			hHash				= 0;
	HCRYPTKEY			hKey				= 0;
	DWORD				data_len			= 4;			
	DWORD				crypto_len;		
	BOOL				retval				= false;
	DWORD error;
 
if( !hCryptContext )
{
	if( !CryptAcquireContextA( &hCryptContext,
                                         NULL, 
                                         MS_ENHANCED_PROV_A,
                                         PROV_RSA_FULL,  0 ) )
	{
			return false;
	}
}
 
_write_data->hash_key = (PBYTE) Get_MD5_Generic( _write_data->data,
                                                  _write_data->data_len,
                                                   hCryptContext );
 
_write_data->hash_len = HASHLEN;
 
if( !_write_data->hash_key ) return false;
 
//begin to hash data block
if( CryptCreateHash( hCryptContext, CALG_MD5, 0, 0, &hHash ) )
{
	if( CryptHashData( hHash, 
                           (BYTE *) _write_data->hash_key, 
                            _write_data->data_len, 
                             NULL ) )
	{
 
        if( CryptDeriveKey( hCryptContext, ENCRYPT_ALG, hHash, KEYLEN, &hKey ) )
	{
	      if( CryptEncrypt( hKey, 
                                NULL, 
                                true, 
                                NULL,
                               (PBYTE) _write_data->data,
                                (DWORD*) &_write_data->data_len, BUFFERSIZE ) )
		{
			retval = true;
		}
		else
		{
			error = GetLastError();
		}
		CryptDestroyKey( hKey );	
	}
		
}
CryptDestroyHash( hHash );
}
 
if( _write_data->hash_key && !retval )
{
	VirtualFree( _write_data->hash_key, 0, MEM_RELEASE );	
	_write_data->hash_key = NULL;
}
 
return retval;
 
}
 
bool _Decrypt( MessagePair * message )
{
static  HCRYPTPROV		hCryptContext;
	HCRYPTHASH		hHash;
	HCRYPTKEY		hKey;
 
	BOOL			retval = false;
 
	DWORD			decrypt_len = LOGBUFFERSIZE;
	DWORD			err = 4;
	DWORD			hashsize = 0x10;
 
	if( !hCryptContext )
	{
		if( !CryptAcquireContextA( &hCryptContext, NULL, MS_ENHANCED_PROV_A, PROV_RSA_FULL,  0 ) )
		{
			return NULL;
		}
	}
 
	if( CryptCreateHash( hCryptContext, CALG_MD5, 0, 0, &hHash ) )
	{
 
		if( CryptHashData( hHash, (BYTE *) message->hash_key, message->hash_len, 0 )  )
			{				
 
			if( CryptDeriveKey( hCryptContext, ENCRYPT_ALG, hHash, KEYLEN, &hKey ) )
			{
				if( CryptDecrypt( hKey, 
                                                   NULL, 
                                                   true, 
                                                   NULL, 
                                                   (BYTE *)message->data, 
                                                   (DWORD*)&message->data_len ) )
				{
					retval = true;
				}
				CryptDestroyKey( hKey );							
			}		
			else
			{
				err = GetLastError();
			}
		}		
	CryptDestroyHash( hHash );
}
 
return retval;
}
Posted 6-Feb-13 17:54pm
Edited 11-Feb-13 6:35am
v3
Comments
Andrew Cherednik at 7-Feb-13 0:33am
   
I think CryptEncrypt depending on your BUFFERSIZE needs to be called in sequence until all chunks are encrypted
POP_POP_R3T at 7-Feb-13 2:04am
   
Yeah, but if im not mistaken... I think that is only for block ciphers. I was under the impression that RC4 was a stream cipher. In which case I think I can set Final for both Encrypt and Decrypt on the first and only pass. Atleast I think thats how it works.
Marius Bancila at 7-Feb-13 2:44am
   
The obvious question is if CryptDecrypt fails, what is the value returned by GetLastError()?
POP_POP_R3T at 7-Feb-13 2:49am
   
The call to CryptDecrypt isn't failing in the code above, so ERROR_SUCCESS. The problem is that the ciphertext is not decrypting to the original plaintext.

1 solution

Rate this: bad
good
Please Sign up or sign in to vote.

Solution 1

Ok I figured it out.
 
Turns out I had my blinders on...in two parts
 
First:
even though my first comment to Andrew's point mentions setting "final" on my call to encrypt I still managed to overlook the flag (arg3) and never finalize encryption. Whoops!
 
Second:
When calling CryptHashData i was accidently using _write_data->data_len instead of hash_len
 
Simple mistake, simple fix!
 
http://en.wikipedia.org/wiki/Rubber_duck_debugging[^]
  Permalink  
v2

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



Advertise | Privacy | Mobile
Web02 | 2.8.140709.1 | Last Updated 11 Feb 2013
Copyright © CodeProject, 1999-2014
All Rights Reserved. Terms of Service
Layout: fixed | fluid