Using CAPICOM in .NET for Digital Signatures with ASCII/UTF8 Content

• Download source files - 137 Kb

Overview

In .NET 1.0-1.1, we do not have more options to play with PKCS7 Signed data other than using CAPICOM COM Library included in Microsoft Platform SDK. CAPICOM provides SignedData object to generate/validate attached and detached digital signatures as well as many other useful objects.

One of the common problem I faced when I worked with digital signatures was to generate PKCS7 formatted digital signatures in .NET such that Java-based systems can validate. The same problem arises when you happen to validate digital signatures generated by Java-based systems in .NET platform. The issue was that CAPICOM accepts only Unicode content to generate/validate signatures whereas Java-based systems usually use UTF8 or ASCII content for digital signatures which actually cause some problems when you use CAPICOM in .NET based code. This article will show you how you can solve the problems mentioned below.

The Problem

Assume the following scenario: You have a .NET based application (BizTalk/ASP.NET Application/Web Services etc.) and would like to integrate with a Java-based system using XML Messages and PKCS7 formatted digital signatures. Let's say your system needs to communicate with this external system two-way by sending and receiving signed XML messages. To send the message, you need to sign the XML message such that the Java-system can validate. And, while receiving, you need to validate the signatures generated by the Java-system before any more processing. If you decide to use CAPICOM from your .NET code, it is highly likely that you will face the following problems in this scenario.

Validating Attached Signatures

Attached form of digital signatures embeds the clear text inside the PKCS7 envelope. You need to first validate the signature and retrieve the content afterwards. You probably need to code something like this to do this operation:

public string VerifyAttachedSignature(string base64SignedContent)
{
    CAPICOM.SignedData signedData = new CAPICOM.SignedData();
    signedData.Verify(base64SignedContent, false,
      CAPICOM.CAPICOM_SIGNED_DATA_VERIFY_FLAG.CAPICOM_VERIFY_SIGNATURE_ONLY);
    string clearText = signedData.Content;
    return clearText; 
}

Even if the code validates the signature perfectly, you will face problems while trying to retrieve the signedData.Content field to extract the clear text from the signature if the Java-based system has used UTF8/ASCII encoding during signature generation. The problem was because the clearText returned is expected to be Unicode whereas it's actually a Binary String. One solution would be using Utilities object in CAPICOM to get UTF8 (or ASCII) byte array and convert back to .NET string using System.Text.Encoding class. Then the code becomes:

public string VerifyAttachedSignature(string base64SignedContent)
{
    CAPICOM.SignedData signedData = new CAPICOM.SignedData();
    signedData.Verify(base64SignedContent, false,
      CAPICOM.CAPICOM_SIGNED_DATA_VERIFY_FLAG.CAPICOM_VERIFY_SIGNATURE_ONLY);
    //assuming the content is ASCII or UTF8

    string binaryString = signedData.Content;
    CAPICOM.Utilities utility = new CAPICOM.Utilities();
    string clearText =System.Text.Encoding.UTF8.GetString(
            (byte[])utility.BinaryStringToByteArray(binaryString));
    return clearText; 
}

This seems to be solving the issue, but actually does not! This code will work if the number of characters in the clear text content is even! That's because COM Interop while getting binaryString from signedData.Content will truncate the last character while converting to UNICODE if the real content has odd number of characters.

Validating Detached Signatures:

The issue in Detached form of signature is similar but more dramatic. Take a look at the following code which is used to validate signatures with Detached form using CAPICOM:

public void VerifyDetachedSignature(string clearTextMessage,
                                    string base64SignedContent)
{
    CAPICOM.SignedData signedData = new CAPICOM.SignedData();
    CAPICOM.Utilities utility = new CAPICOM.Utilities();
    //assuming the content is ASCII or UTF8

    signedData.Content=utility.ByteArrayToBinaryString(
       System.Text.Encoding.UTF8.GetBytes(clearTextMessage));
    signedData.Verify(base64SignedContent, true, 
       CAPICOM.CAPICOM_SIGNED_DATA_VERIFY_FLAG.CAPICOM_VERIFY_SIGNATURE_ONLY); 
}

As you might have noticed, I used the same trick to convert .NET string to Binary String using ByteArrayToBinaryString method of CAPICOM.Utilities object. This will enable us to pass ASCII/UTF8 content as if it is UNICODE to CAPICOM to validate. However, the same problem happens here and the consequence would be impossible to handle. If the clearTextMessage has odd number of characters, we will never be able to pass real content to the CAPICOM, so validation will always fail!

Generating Attached Signatures:

Java-code expects us to generate signatures with ASCII/UTF8 encoded strings. A typical code to do so would be as follows:

public string GenerateAttachedSignature(string clearTextMessage, 
             CAPICOM.Certificate myClientCertificate)
{
    CAPICOM.SignedData signedData = new CAPICOM.SignedDataClass();
    CAPICOM.Utilities utility = new CAPICOM.UtilitiesClass();
    //Content has to be UTF8 as our Java friend expects in this format 

    signedData.Content = utility.ByteArrayToBinaryString(
         System.Text.Encoding.UTF8.GetBytes(clearTextMessage));
    CAPICOM.Signer signer = new CAPICOM.Signer();
    signer.Certificate = myClientCertificate;
    string signedMessage = signedData.Sign(signer, false,
        CAPICOM.CAPICOM_ENCODING_TYPE.CAPICOM_ENCODE_BASE64);
    return signedMessage; 
}

This code addresses the issue of attaching UTF8 content, and generates a signature that can be verified by our Java Friend. However, because of the truncation issue I mentioned earlier, they will extract data with last character truncated if the number of characters is odd. There is a solution, which may or may not be applicable to all scenarios; adding one extra space to the end of the clearTextMessage before signing would do the trick as below:

...
signedData.Content = utility.ByteArrayToBinaryString(
    System.Text.Encoding.UTF8.GetBytes(clearTextMessage + " "));
...

Generating Detached Signatures:

Detached side of the problem is again difficult to handle:

public string GenerateDetachedSignature(string clearTextMessageWithEvenCharacters, 
          CAPICOM.Certificate myClientCertificate)
{
    CAPICOM.SignedData signedData = new CAPICOM.SignedDataClass();
    CAPICOM.Utilities utility = new CAPICOM.UtilitiesClass();
    //Content has to be UTF8 as our Java friend expects in this format 

    signedData.Content = utility.ByteArrayToBinaryString(
           System.Text.Encoding.UTF8.GetBytes(
                       clearTextMessageWithEvenCharacters));
    CAPICOM.Signer signer = new CAPICOM.Signer();
    signer.Certificate = myClientCertificate;
    string signedMessage = signedData.Sign(signer, true, 
        CAPICOM.CAPICOM_ENCODING_TYPE.CAPICOM_ENCODE_BASE64);
    return signedMessage; 
}

The code provided above requires the caller to send clear text with even number of characters in order for it to work properly otherwise it will generate a signature for the content last character truncated which may or may not be handled by the Java-based system.

The Solution

As you see, root of the problem is because CAPICOM manipulates only Unicode strings while validating and generating digital signatures. CAPICOM is actually designed by Microsoft to be used in languages like VB, so the method parameters in CAPICOM Type library is defined as BSTR which is actually VB version of Unicode. BSTR is actually a 32-bit pointer to a Unicode character array, preceded by 4-byte long and terminated by a 2-byte null character (ANSI 0). Based on my experiments in VB 6, I found out that none of these problems happen there, i.e., methods of the Utilities class does not truncate the last character, and everything works just fine. This actually brought my attention to the Runtime Callable Wrapper (RCW) that needs to be generated to be able to use CAPICOM from .NET world. It looks like when I tried to pass or retrieve ANSI BSTR (Binary String) from .NET to RCW, it was the guy who actually causes truncation.

Based on all these, I decided to modify the generated RCW so that no marshalling occurs between managed world and unmanaged COM world. I will still use the Utilities class to generate and retrieve Binary Strings, but I will be preventing any BSTR to/from .NET string conversion. The best candidate would be using .NET IntPtr instead of BSTR marshalling as BSTR is actually a pointer.

As a first step, I used ILDASM.EXE to decompile .NET RCW CAPICOM.dll into IL code, using the command below:

C:\CapicomWork>ILDASM CAPICOM.dll /OUT:CAPICOM.IL

ILDASM generated two files CAPICOM.IL which contains IL code for CAPICOM RCW, and CAPICOM.res resource file. I opened the CAPICOM.IL and located the places for Content field declarations of SignedDataClass as below:

.method public hidebysig newslot specialname virtual 
instance void set_Content([in] string marshal( bstr) pVal) 
                                                runtime managed internalcall
{
    .custom instance void 
       [mscorlib]System.Runtime.InteropServices.DispIdAttribute::.ctor(int32) 
                                                = ( 01 00 00 00 00 00 00 00 ) 
    .override CAPICOM.ISignedData::set_Content
} // end of method SignedDataClass::set_Content


.method public hidebysig newslot specialname virtual 
instance string 
marshal( bstr) 
get_Content() runtime managed internalcall
{
    .custom instance void 
       [mscorlib]System.Runtime.InteropServices.DispIdAttribute::.ctor(int32) 
                                                = ( 01 00 00 00 00 00 00 00 ) 
    .override CAPICOM.ISignedData::get_Content
} // end of method SignedDataClass::get_Content

Since Content is a property, IL code includes two methods for get and set operations; get_Content, and set_Content. From the IL code above, Content field is actually interpreted as string and marshaled as BSTR. I changed the IL code above with the following:

.method public hidebysig newslot specialname virtual 
instance void set_Content([in] native int pVal) runtime managed internalcall
{
    .custom instance void 
      [mscorlib]System.Runtime.InteropServices.DispIdAttribute::.ctor(int32) 
                                               = ( 01 00 00 00 00 00 00 00 ) 
    .override CAPICOM.ISignedData::set_Content
} // end of method SignedDataClass::set_Content


.method public hidebysig newslot specialname virtual 
instance native int 
get_Content() runtime managed internalcall
{
    .custom instance void 
      [mscorlib]System.Runtime.InteropServices.DispIdAttribute::.ctor(int32) 
                                               = ( 01 00 00 00 00 00 00 00 ) 
    .override CAPICOM.ISignedData::get_Content
} // end of method SignedDataClass::get_Content

As you have noticed, I replaced the string marshal (bstr) with native int which is actually IntPtr in C#. I had to do the same changes on ISignedData and SignedData interfaces not to cause any conflict.

However, this change alone will not be enough as it is difficult to generate a proper BSTR IntPtr in .NET side. The other problem was to retrieve the correct string in case of Attached signature content extraction. I should be able to get the content in byte array to overcome potential data truncation.

Utilities class was the best candidate to help me in these challenges as I can pass byte array and get BSTR pointer with no extra work using ByteArrayToBinaryString method. Similarly, using BinaryStringToByteArray method would let me handle the second issue of retrieving proper message attached in the message. I changed the UtilitiesClass (as well as the IUtilities interface) in IL as below:

method public hidebysig newslot virtual 
instance object 
marshal( struct) 
BinaryStringToByteArray([in] native int
                               BinaryString) runtime managed internalcall
{
    .custom instance void 
      [mscorlib]System.Runtime.InteropServices.DispIdAttribute::.ctor(int32)
                                              = ( 01 00 06 00 00 00 00 00 ) 
    .override CAPICOM.IUtilities::BinaryStringToByteArray
} // end of method UtilitiesClass::BinaryStringToByteArray


.method public hidebysig newslot virtual 
instance native int 
ByteArrayToBinaryString([in] object marshal( struct) varByteArray)
                                     runtime managed internalcall
{
    .custom instance void 
      [mscorlib]System.Runtime.InteropServices.DispIdAttribute::.ctor(int32) 
                                               = ( 01 00 07 00 00 00 00 00 ) 
    .override CAPICOM.IUtilities::ByteArrayToBinaryString
} // end of method UtilitiesClass::ByteArrayToBinaryString

Then save the CAPICOM.IL and run the following command from the command prompt to re-build the RCW CAPICOM.dll using .NET's ILASM tool:

C:\CapicomWork>ilasm /dll CAPICOM.IL /
                      resource=CAPICOM.res /KEY=mykey.snk /output=Capicom.dll

The final code

After the successful build with ILASM, I rewrote the Digital signature routines as below:

public string SignFromText(string plaintextMessage, 
                     bool bDetached, Encoding encodingType) 
{
    CAPICOM.SignedData signedData = 
                      new CAPICOM.SignedDataClass();
    CAPICOM.Utilities u = new CAPICOM.UtilitiesClass();
    signedData.set_Content(u.ByteArrayToBinaryString(
                     encodingType.GetBytes(plaintextMessage)));
    CAPICOM.Signer signer = new CAPICOM.Signer();
    signer.Certificate = ClientCert;
    this._signedContent = signedData.Sign(signer, bDetached, 
            CAPICOM.CAPICOM_ENCODING_TYPE.CAPICOM_ENCODE_BASE64);
    return _signedContent; 
}

public bool VerifyDetachedSignature(string plaintextMessage, 
                   string signedContent, Encoding encodingType) 
{
    try
    {
        this._clearText = plaintextMessage;
        this._signedContent = signedContent;
        CAPICOM.SignedData signedData = 
                         new CAPICOM.SignedDataClass();
        CAPICOM.Utilities u = new CAPICOM.UtilitiesClass();
        signedData.set_Content(u.ByteArrayToBinaryString(
                   encodingType.GetBytes(plaintextMessage)));
        signedData.Verify(_signedContent,true, 
           CAPICOM.CAPICOM_SIGNED_DATA_VERIFY_FLAG
                    .CAPICOM_VERIFY_SIGNATURE_ONLY);
        SignerCert=null;
        CAPICOM.Signer s = (CAPICOM.Signer) signedData.Signers[1];
        SignerCert = (CAPICOM.Certificate)s.Certificate; 
        return true; 
    }
    catch(COMException e)
    {
        return false;
    }
}

public bool VerifyAttachedSignature(string signedContent,
                                           Encoding encodingType) 
{
    try
    {
        this._signedContent = signedContent;
        CAPICOM.Utilities u = new CAPICOM.Utilities();
        CAPICOM.SignedData signedData = new CAPICOM.SignedData();
        signedData.Verify(_signedContent,false, 
          CAPICOM.CAPICOM_SIGNED_DATA_VERIFY_FLAG.CAPICOM_VERIFY_SIGNATURE_ONLY);
        SignerCert=null;
        CAPICOM.Signer s = (CAPICOM.Signer) signedData.Signers[1];
        SignerCert = (CAPICOM.Certificate)s.Certificate; 
        this._clearText = encodingType.GetString(
            (byte[])u.BinaryStringToByteArray(signedData.get_Content()));
        return true; 
   }
   catch(COMException e)
   {
        return false;
   }
}

The bold lines above actually show the differences in the code after applying changes made in CAPICOM RCW. We can not use SignedData.Content property anymore as C# does not support properties with IntPtr return type so we have to explicitly specify the get_Content or set_Content methods whenever needed. encodingType parameter enables us to pass or retrieve the content whichever encoding we like. The SignFromText accepts bDetached parameter for distinguishing detached/attached signing requests, and should be set to true if you want to generate detached signature.

Using The Sample Source Files

In order to run the sample properly, you need the following installed:

• .NET 1.1 (you can download from Microsoft website).
• CAPICOM library v2.1.0.0 (you can download from Microsoft website).
• A Certificate with a Private key installed in Personal Store (Current User or Machine Store).