Click here to Skip to main content
Click here to Skip to main content
Go to top

Unmanaged Arrays in C# - No Problem

, 3 Jan 2009
Rate this:
Please Sign up or sign in to vote.
Using unmanaged arrays is simple and easy in C#! Includes useful code examples.

Introduction

Using unmanaged arrays in C# is really easy! Wasier than you might have thought..

Why Use Unmanaged Arrays in C#?

  • Because you're using P/Invoke or Interop (calling native DLLs / COM objects) and need to pass pointers to unmanaged memory.
  • Because you want to directly update buffers passed to or from unmanaged code without copying back and forth to managed memory.
  • Because you're developing a real-time application that should not be interrupted by the garbage collector (not even for a couple of milliseconds).

But Why Not Use C++/CLI Instead?

  • Because you're already developing in C# and are more familiar with its syntax.
  • Because C# is cool.

Background

I wanted to develop some real-time audio app in C#, but was pretty surprised to learn about the behavior of its generational mark-and-sweep garbage collector. (A behavior exhibited with many other non real-time GCs as well, e.g., the default Java GC.)

The CLR GC pauses the execution of all threads in the process to perform its sweep phase, i.e., when managed objects are compacted in memory and inaccessible objects are freed. This is done pretty much unexpectedly (though only initiated by a managed heap allocation), and for an undetermined amount of time (actually there are three types of collections for the three generations, Gen0 are the shortest and Gen2 are the longest but least frequent). You can read much more about the CLR GC here and here.

Searching the Internet for solutions, I could not find an easy explanation/code showing how to use unmanaged (non garbage collected) memory in C#, to bypass the managed heap and its drawbacks completely. I researched a bit, and found a plausible solution that actually performs extremely well (as fast as unsafe managed code does), all in native C#.

Usage Example

The following example allocates a 25 element uint array from the unmanaged heap, zeros the newly allocated memory segment, sets and reads its 25th element and then frees the memory.

unsafe
{
    uint* unmanagedArray = (uint*) Unmanaged.NewAndInit<uint>(25);

    unmanagedArray[24] = 23984723;
    uint testValue = unmanagedArray[24];

    Unmanaged.Free(unmanagedArray);
}

The Code: 1st Attempt

I've found an easy way to replicate C type heap arrays associated with the malloc, calloc, free, and realloc methods, but in native C#. This technique uses methods from the Marshal class in System.Runtime.InteropServices, which is destined mainly for interop purposes.

The following class can allocate and free memory from the process' unmanaged heap. It requires an unsafe context to execute:

static unsafe class Unmanaged
{
    public static void* New<T>(int elementCount) 
        where T : struct
    {
        return Marshal.AllocHGlobal(Marshal.SizeOf(typeof(T)) * 
					elementCount).ToPointer();
    }

    public static void* NewAndInit<T>(int elementCount)
        where T : struct
    {
        int newSizeInBytes = Marshal.SizeOf(typeof(T)) * elementCount;
        byte* newArrayPointer = 
		(byte*) Marshal.AllocHGlobal(newSizeInBytes).ToPointer();

        for (int i = 0; i < newSizeInBytes; i++)
            *(newArrayPointer + i) = 0;
			
        return (void*) newArrayPointer;
    }

    public static void Free(void* pointerToUnmanagedMemory)
    {
        Marshal.FreeHGlobal(new IntPtr(pointerToUnmanagedMemory));
    }

    public static void* Resize<T>(void* oldPointer, int newElementCount)
        where T : struct
    {
        return (Marshal.ReAllocHGlobal(new IntPtr(oldPointer),
            new IntPtr(Marshal.SizeOf(typeof(T)) * newElementCount))).ToPointer();
    }
}

But There's a Problem with the Above Approach

A commenter alerted me that Marshal.SizeOf(typeof(T)) is not correct, and sizeof(T) should be used instead. The reason is that Marshal.SizeOf(typeof(T)) returns the size of a type after it will be marshaled (that is, converted by the marshaler according to MarshalAs attributes and default marshalling behavior). For example:

sizeof(System.Char) == 2
Marshal.SizeOf(System.Char) == 1

sizeof(System.Boolean) == 1
Marshal.SizeOf(System.Boolean) == 4 //(!)

See also:

2nd Attempt in C++/CLI

Since C# does not support querying the size of a generic variable (i.e. sizeof(T)) I went on to write it in C++/CLI:

public ref class Unmanaged abstract sealed
{
public:
	generic <typename T> where T : value class
	static void* New(int elementCount)
	{
		return Marshal::AllocHGlobal(sizeof(T) * elementCount).ToPointer();
	}

	generic <typename T> where T : value class
	static void* NewAndInit(int elementCount)
	{
		int sizeInBytes = sizeof(T) * elementCount;
		void* newArrayPtr = Marshal::AllocHGlobal(sizeInBytes).ToPointer();
		memset(newArrayPtr, 0 , sizeInBytes);
		return newArrayPtr;
	}

	static void Free(void* unmanagedPointer)
	{
		Marshal::FreeHGlobal(IntPtr(unmanagedPointer));
	}

	generic <typename T> where T : value class
	static void* Resize(void* oldPointer, int newElementCount)
	{
		return Marshal::ReAllocHGlobal(IntPtr(oldPointer), 
			IntPtr((int) sizeof(T) * newElementCount)).ToPointer();
	}
};

Notes

The above will only work with unmanaged types, i.e., primitive variables, structs containing primitive variables, or structs containing other such structs. The C# compiler does not allow taking pointers to other types.

Detailed Notes

The New() method returns a generic pointer (void*) to the memory allocated from the unmanaged heap. It accepts a type T and an element count to calculate the memory requirement of the array. Note that if you are planning to do marshalization, as when using it with a struct containing unblittable fields, you'll need to use Marshal.Sizeof() instead of sizeof() as it takes MarshalAs attributes into account. Then use Marshal.StructureToPtr() and Marshal.PtrToStructure() instead of direct assignments to memory.

The NewAndInit() method will allocate and then zero the memory region. Since C# and C++/CLI do not support parameterless constructors for structs (or "value classes" as they're called in C++/CLI), zeroing the memory should be sufficient.

The Free() method will work only with memory allocated with New() or NewAndInit(), and is pretty much undefined for arbitrary pointers.

The Resize() method will allocate a new memory segment according to the new element count, then copy the elements to the new segment and return a pointer to it.

As far as I'm aware of, the above class (and the code depending strictly on it) does not make any usage of the managed heap, therefore should not leave any work for the garbage collector.

Performance

Using this method, writing and reading to unmanaged memory is as fast as unsafe reads/writes to managed memory (this was shown by a rudimentary benchmark). Since there's no bound checking, it may even perform faster.

I would love to hear your comments and suggestions!

History

  • 30th December, 2008: Version 1.0
  • 1st January, 2009: Version 1.1 - Corrected C++/CLI code and fixed many inaccurate statements

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

AnonX

Antarctica Antarctica
No Biography provided

Comments and Discussions

 
GeneralMy vote of 2 Pinprofessional53V3N30-Jul-14 10:10 
GeneralMy vote of 4 PinmemberEdo Tzumer1-May-12 4:12 
QuestionSo is it a problem after all? PinmemberAl_S9-Jul-09 2:33 
GeneralRe: So is it a problem after all? Pinmemberthree1111-Aug-09 23:10 
GeneralRe: So is it a problem after all? Pinmemberben.Kloosterman28-Jan-10 21:19 
GeneralA couple questions PinmemberAndrew Voelkel22-Feb-09 17:05 
GeneralRe: A couple questions PinmemberAnonX27-Feb-09 10:25 
GeneralRe: A couple questions PinmemberAndrew Voelkel1-Mar-09 7:01 
GeneralRe: A couple questions PinmemberAnonX4-Mar-09 9:53 
GeneralRe: A couple questions PinmemberObiwan Jacobi5-Mar-09 18:43 
GeneralCorrected version in C++/CLI (draft) [modified] PinmemberAnonX31-Dec-08 1:58 
GeneralAn experimental idea: Autofinalized unmanaged arrays (some conceptual C++/CLI code) [modified] PinmemberAnonX31-Dec-08 12:31 
GeneralAnother useful tool: a generic, auto-marshalling wrapper class for an unmanaged struct (in C#) [modified] PinmemberAnonX3-Jan-09 9:38 
GeneralThanks for the early comments, the article is still a work in progress.. PinmemberAnonX30-Dec-08 11:11 
GeneralRe: Thanks for the early comments, the article is still a work in progress.. PinmemberRoberto Collina30-Dec-08 22:33 
Generalsizeof(T) vs Marshal.SizeOf(typeof(T)) PinmemberDaniel Grunwald30-Dec-08 8:25 
GeneralRe: sizeof(T) vs Marshal.SizeOf(typeof(T)) PinmemberJörgen Sigvardsson30-Dec-08 9:49 
GeneralRe: sizeof(T) vs Marshal.SizeOf(typeof(T)) PinmemberAnonX30-Dec-08 11:17 
What you point out is very interesting and important (not to say surprising), I will take a look at that tomorrow.. (Wikipedia says that Char (and bool!) are not blittable types, that's also surprising)
 
I also thought that a "where T : blittable" would be very good idea. Being able to dereference generic variables can open the doors to very creative and useful uses.
GeneralRe: sizeof(T) vs Marshal.SizeOf(typeof(T)) - You are correct, there's a significant error in the article. &lt;-- [modified] PinmemberAnonX30-Dec-08 23:22 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web03 | 2.8.140916.1 | Last Updated 3 Jan 2009
Article Copyright 2008 by AnonX
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid