Generic BinaryReader and BinaryWriter Extensions

• Download source files (C++/CLI) - 5.68 KB
• Download the compiled assembly (binary) - 9.58 KB

Introduction

A while ago, I was working on an application that needed to convert between structs and byte[]. I had many different kinds of structs that I needed to work with, and at first, I tried to maintain a large number of methods for each struct type. You could imagine how bloated the code was.

Eventually, I decided I would give a crack at making all of that conversion code generic, and this miniature project was born. The project provides a static class (like BitConverter) and extension methods for both BinaryReader and BinaryWriter to enable conversion between structs (of any kind) and a series of bytes as an Array or Stream.

Getting Started: The Impossible Way

Since I was somewhat familiar with C# already, naturally I would start the project in C# as well. To start it off, I wrote some code for one of the two conversion methods (shown below) and hit compile:

public static unsafe byte[] GetBytes<T>(T value)
{
    // Check to see if the input is null.
    if (value == null)
    {
        throw new ArgumentNullException();
    }
 
    // Check to see if the input has value type semantics.
    if (!(value is ValueType))
    {
        throw new ArgumentException();
    }
 
    // Create an array of bytes the same size as the input.
    byte[] bytes = new byte[sizeof(T)];
 
    // Pin the byte array down so the garbage collector can't move it.
    fixed (byte* p1 = &bytes[0])
    {
        // Cast the pointer type from byte to T.
        T* p2 = (T*)p1;
 
        // Assign the value to the byte array.
        *p2 = value;
    }
 
    // Return the byte array.
    return bytes;
}

Instead of compiling successfully, Visual Studio gave me an error: “Cannot take the address of, get the size of, or declare a pointer to a managed type ('T').” This was logical. After all, T was not guaranteed to have value type semantics; you can't take the size of something that doesn't have a definite size, and a class can have any size during runtime unless it happens to derive from System.ValueType (e.g. Byte, Int32, UInt64, Single, Decimal, DateTime…). The next logical step would be to force the input to be a ValueType. I removed the error checking code and constrained the type parameter to ValueType:

public static unsafe byte[] GetBytes<T>(T value) where T : ValueType
{
    // Create an array of bytes the same size as the input.
    byte[] bytes = new byte[sizeof(T)];
 
    // Pin the byte array down so the garbage collector can't move it.
    fixed (byte* p1 = &bytes[0])
    {
        // Cast the pointer type from byte to T.
        T* p2 = (T*)p1;
 
        // Assign the value to the byte array.
        *p2 = value;
    }
 
    // Return the byte array.
    return bytes;
}

After hitting the compile key, the error that Visual Studio gave me this time baffled me: “Constraint cannot be special class 'System.ValueType'.” Upon further research, I came upon an MSDN article that suggested constraining the type parameter to a struct. The constraint was accepted by the compiler, but again, it complained that taking the address, pointer, or size of a generic type was not allowed. (Smart compiler, eh?) This, to me, seemed like a dead end. How else was I supposed to write this method?

The C++/CLI Way

I also happened to be playing around with C++/CLI in Visual Studio at around the same time, when an idea hit me. If C++/CLI could do so many things that C# could not (like mixing managed code with unmanaged code), could it let me get around both restrictions in C#? There was only one way to find out, and after an hour of reading C++/CLI tutorials, I came up with this:

generic <typename T>
where T : System::ValueType
array<unsigned char> ^GenericBitConverter::GetBytes(T value)
{
       array<unsigned char> ^bytes = gcnew array<unsigned char>(sizeof(T));
       *reinterpret_cast<interior_ptr<T>>(&bytes[0]) = value;
       return bytes;
}

The method actually worked, and in just 3 lines of code!

Here is the expanded, more readable version (equivalent to the above, but split up to make the steps more obvious):

generic <typename T>
where T : System::ValueType
array<unsigned char> ^Iku::GenericBitConverter::GetBytes(T value)
{
       // Create an array of bytes the same size as the input.
       array<unsigned char> ^bytes = gcnew array<unsigned char>(sizeof(T));
 
       // Create a pointer to the byte array.
       interior_ptr<unsigned char> ptr1 = &bytes[0];
 
       // Cast the pointer to the same type as the input.
       interior_ptr<T> ptr2 = reinterpret_cast<interior_ptr<T>>(ptr1);
 
       // Assign the value to the byte array.
       *ptr2 = value;
 
       // Return the byte array.
       return bytes;
}

… and that is the GenericBitConverter. It works with any type that is derived from ValueType (a struct), and thus will work with all the basic .NET data types like double and decimal (except string, which is really an object) as well as enumerations and other complex structures.

Making an Extension Method in C++/CLI

Now that the GenericBitConverter was in place, it would be nice if the same functionality were available to the BinaryReader and BinaryWriter classes, so I decided to extend those classes with the same functionality as well. This was not any easier than getting the GenericBitConverter to work properly. Writing the method was easy at first:

generic <typename T>
where T : System::ValueType
T Iku::IO::BinaryReaderExtension::ReadValue(BinaryReader ^binaryReader)
{
       return GenericBitConverter::ToValue<T>
		(binaryReader->ReadBytes(sizeof(T)), 0);
}

It just would not show up as an extension method in the C# code editor.

It was time to investigate why, so I wrote one in C# and I disassembled the assembly. While inspecting the output, I found the ExtensionAttribute attribute applied to three places: the assembly, the static class containing the extension method, and the extension method itself. Applying the attributes to the proper place in the C++/CLI project, I was able to get the extension method to show up:

… and there are the generic BinaryReader and BinaryWriter extensions.

Now you can do stuff like this:

DateTime timeStamp = binaryReader.ReadValue<DateTime>();

static void Main(string[] args)
{
    SampleEnumeration sampEnum = SampleEnumeration.All ^ SampleEnumeration.Four;

    byte[] enumBytes = GenericBitConverter.GetBytes<SampleEnumeration>(sampEnum);

    // Writes "f0 ff ff ff ff ff ff ff"
    foreach (byte bite in enumBytes)
    {
        Console.Write("{0:x2} ", bite);
    }

    Console.WriteLine();
}

enum SampleEnumeration : long
{
    None = 0x0000000000000000,
    All = -1L,
    One = 0x0000000000000001,
    Four = 0x000000000000000f
}

Neat? Feel free to use this within your own C# or Visual Basic projects (by referencing the output of this project). It works perfectly!

Points of Interest

• C++/CLI also lets you constrain generic parameters to the ValueType type; it does not complain. C# just demands struct in place of ValueType.
• The C++/CLI unsigned char type is actually a synonym for the Byte type and not the Char type. The C++/CLI Char type is actually wchar_t.
• Interior pointers (interior_ptr) point to their objects, but do not fix them in place. This is markedly different from C#’s fixed behavior, which fixes the object in memory and prevents the garbage collector from moving it (as long as the fixed statement is in scope).
• Although C++/CLI does not offer any syntactic sugar for creating extension methods, it can be done by placing an ExtensionAttribute attribute on the method, its containing class, and its containing assembly. (ExtensionAttribute can be found under the namespace System::Runtime::CompilerServices) This nature of extension methods means it can be made from any language that supports attributes, static classes, and static methods.
• The C# compiler will assume a generic type can be any regular object even if you constrain it to be a struct (or a ValueType). I have not found a way around it in C# (and thus this project was born).

Updates

• Feb. 27, 2009: Added a download for the compiled assembly; not everyone has the ability to compile a C++/CLI project (users of the Express Editions come to mind). Now everyone can use it!