Click here to Skip to main content
15,867,141 members
Articles / Programming Languages / C#

Unmanaged Arrays in C# - No Problem

Rate me:
Please Sign up or sign in to vote.
4.48/5 (16 votes)
3 Jan 2009CPOL4 min read 149.4K   47   21
Using unmanaged arrays is simple and easy in C#! Includes useful code examples.

Introduction

Using unmanaged arrays in C# is really easy! Wasier than you might have thought..

Why Use Unmanaged Arrays in C#?

  • Because you're using P/Invoke or Interop (calling native DLLs / COM objects) and need to pass pointers to unmanaged memory.
  • Because you want to directly update buffers passed to or from unmanaged code without copying back and forth to managed memory.
  • Because you're developing a real-time application that should not be interrupted by the garbage collector (not even for a couple of milliseconds).

But Why Not Use C++/CLI Instead?

  • Because you're already developing in C# and are more familiar with its syntax.
  • Because C# is cool.

Background

I wanted to develop some real-time audio app in C#, but was pretty surprised to learn about the behavior of its generational mark-and-sweep garbage collector. (A behavior exhibited with many other non real-time GCs as well, e.g., the default Java GC.)

The CLR GC pauses the execution of all threads in the process to perform its sweep phase, i.e., when managed objects are compacted in memory and inaccessible objects are freed. This is done pretty much unexpectedly (though only initiated by a managed heap allocation), and for an undetermined amount of time (actually there are three types of collections for the three generations, Gen0 are the shortest and Gen2 are the longest but least frequent). You can read much more about the CLR GC here and here.

Searching the Internet for solutions, I could not find an easy explanation/code showing how to use unmanaged (non garbage collected) memory in C#, to bypass the managed heap and its drawbacks completely. I researched a bit, and found a plausible solution that actually performs extremely well (as fast as unsafe managed code does), all in native C#.

Usage Example

The following example allocates a 25 element uint array from the unmanaged heap, zeros the newly allocated memory segment, sets and reads its 25th element and then frees the memory.

C#
unsafe
{
    uint* unmanagedArray = (uint*) Unmanaged.NewAndInit<uint>(25);

    unmanagedArray[24] = 23984723;
    uint testValue = unmanagedArray[24];

    Unmanaged.Free(unmanagedArray);
}

The Code: 1st Attempt

I've found an easy way to replicate C type heap arrays associated with the malloc, calloc, free, and realloc methods, but in native C#. This technique uses methods from the Marshal class in System.Runtime.InteropServices, which is destined mainly for interop purposes.

The following class can allocate and free memory from the process' unmanaged heap. It requires an unsafe context to execute:

C#
static unsafe class Unmanaged
{
    public static void* New<T>(int elementCount) 
        where T : struct
    {
        return Marshal.AllocHGlobal(Marshal.SizeOf(typeof(T)) * 
					elementCount).ToPointer();
    }

    public static void* NewAndInit<T>(int elementCount)
        where T : struct
    {
        int newSizeInBytes = Marshal.SizeOf(typeof(T)) * elementCount;
        byte* newArrayPointer = 
		(byte*) Marshal.AllocHGlobal(newSizeInBytes).ToPointer();

        for (int i = 0; i < newSizeInBytes; i++)
            *(newArrayPointer + i) = 0;
			
        return (void*) newArrayPointer;
    }

    public static void Free(void* pointerToUnmanagedMemory)
    {
        Marshal.FreeHGlobal(new IntPtr(pointerToUnmanagedMemory));
    }

    public static void* Resize<T>(void* oldPointer, int newElementCount)
        where T : struct
    {
        return (Marshal.ReAllocHGlobal(new IntPtr(oldPointer),
            new IntPtr(Marshal.SizeOf(typeof(T)) * newElementCount))).ToPointer();
    }
}

But There's a Problem with the Above Approach

A commenter alerted me that Marshal.SizeOf(typeof(T)) is not correct, and sizeof(T) should be used instead. The reason is that Marshal.SizeOf(typeof(T)) returns the size of a type after it will be marshaled (that is, converted by the marshaler according to MarshalAs attributes and default marshalling behavior). For example:

C#
sizeof(System.Char) == 2
Marshal.SizeOf(System.Char) == 1

sizeof(System.Boolean) == 1
Marshal.SizeOf(System.Boolean) == 4 //(!)

See also:

2nd Attempt in C++/CLI

Since C# does not support querying the size of a generic variable (i.e. sizeof(T)) I went on to write it in C++/CLI:

MC++
public ref class Unmanaged abstract sealed
{
public:
	generic <typename T> where T : value class
	static void* New(int elementCount)
	{
		return Marshal::AllocHGlobal(sizeof(T) * elementCount).ToPointer();
	}

	generic <typename T> where T : value class
	static void* NewAndInit(int elementCount)
	{
		int sizeInBytes = sizeof(T) * elementCount;
		void* newArrayPtr = Marshal::AllocHGlobal(sizeInBytes).ToPointer();
		memset(newArrayPtr, 0 , sizeInBytes);
		return newArrayPtr;
	}

	static void Free(void* unmanagedPointer)
	{
		Marshal::FreeHGlobal(IntPtr(unmanagedPointer));
	}

	generic <typename T> where T : value class
	static void* Resize(void* oldPointer, int newElementCount)
	{
		return Marshal::ReAllocHGlobal(IntPtr(oldPointer), 
			IntPtr((int) sizeof(T) * newElementCount)).ToPointer();
	}
};

Notes

The above will only work with unmanaged types, i.e., primitive variables, structs containing primitive variables, or structs containing other such structs. The C# compiler does not allow taking pointers to other types.

Detailed Notes

The New() method returns a generic pointer (void*) to the memory allocated from the unmanaged heap. It accepts a type T and an element count to calculate the memory requirement of the array. Note that if you are planning to do marshalization, as when using it with a struct containing unblittable fields, you'll need to use Marshal.Sizeof() instead of sizeof() as it takes MarshalAs attributes into account. Then use Marshal.StructureToPtr() and Marshal.PtrToStructure() instead of direct assignments to memory.

The NewAndInit() method will allocate and then zero the memory region. Since C# and C++/CLI do not support parameterless constructors for structs (or "value classes" as they're called in C++/CLI), zeroing the memory should be sufficient.

The Free() method will work only with memory allocated with New() or NewAndInit(), and is pretty much undefined for arbitrary pointers.

The Resize() method will allocate a new memory segment according to the new element count, then copy the elements to the new segment and return a pointer to it.

As far as I'm aware of, the above class (and the code depending strictly on it) does not make any usage of the managed heap, therefore should not leave any work for the garbage collector.

Performance

Using this method, writing and reading to unmanaged memory is as fast as unsafe reads/writes to managed memory (this was shown by a rudimentary benchmark). Since there's no bound checking, it may even perform faster.

I would love to hear your comments and suggestions!

History

  • 30th December, 2008: Version 1.0
  • 1st January, 2009: Version 1.1 - Corrected C++/CLI code and fixed many inaccurate statements

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Antarctica Antarctica
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
QuestionThe free method not work Pin
Alexandre Bencz15-Jul-16 3:29
Alexandre Bencz15-Jul-16 3:29 
QuestionUnmanaged Function returns *** how to access it in managed code Pin
Ravi Sreeraman25-Sep-14 19:56
Ravi Sreeraman25-Sep-14 19:56 
GeneralMy vote of 2 Pin
Zachary Patten30-Jul-14 10:10
Zachary Patten30-Jul-14 10:10 
GeneralMy vote of 4 Pin
Joezer BH1-May-12 4:12
professionalJoezer BH1-May-12 4:12 
QuestionSo is it a problem after all? Pin
alleyes9-Jul-09 2:33
professionalalleyes9-Jul-09 2:33 
GeneralRe: So is it a problem after all? Pin
three1111-Aug-09 23:10
three1111-Aug-09 23:10 
GeneralRe: So is it a problem after all? Pin
ben.Kloosterman28-Jan-10 21:19
ben.Kloosterman28-Jan-10 21:19 
GeneralA couple questions Pin
Andrew Voelkel22-Feb-09 17:05
Andrew Voelkel22-Feb-09 17:05 
You seem to be pursuing exactly the problem I am pursuing - getting reliable low latency audio performance while using getting as close to standard C# programming. I actually found a pointer to your article in comments you made on a discussion about a C# ASIO interface. I have been trying to decide whether it was worth the investment getting my feet wet with managed C++. It seems that it is.

I've got a few questions. Are you sure that if you are using unmanaged arrays in this way in code that is called from a high priority unmanaged thread such as the ASIO callback thread, but in an assembly that also contains managed code, that the thread will not be locked out or suspended during garbage collection of the managed code/threads? If so, what determines whether a thread will be suspended? Any thread containing any managed code will be suspended? I am curious. What have your investigations told you? Have you incorporated your ideas with ASIO code and tried it out? If so, would you be willing to make any of that code available?

I'm also curious how one might attach a managed UI running in another thread to such a concoction. Have you any architectural thoughts on where and how to make the division between the safe UI code and the unsafe realtime code in such a way that the real time code remains lockout free and yet the overall code structure remains as clean as possible?

I'd be interested in any feedback you have. I'd also be interested in any relevant links or references to forums where this sort of thing might be discussed.

- Andy
GeneralRe: A couple questions Pin
AnonX27-Feb-09 10:25
AnonX27-Feb-09 10:25 
GeneralRe: A couple questions Pin
Andrew Voelkel1-Mar-09 7:01
Andrew Voelkel1-Mar-09 7:01 
GeneralRe: A couple questions Pin
AnonX4-Mar-09 9:53
AnonX4-Mar-09 9:53 
GeneralRe: A couple questions Pin
Obiwan Jacobi5-Mar-09 18:43
Obiwan Jacobi5-Mar-09 18:43 
GeneralCorrected version in C++/CLI (draft) [modified] Pin
AnonX31-Dec-08 1:58
AnonX31-Dec-08 1:58 
GeneralAn experimental idea: Autofinalized unmanaged arrays (some conceptual C++/CLI code) [modified] Pin
AnonX31-Dec-08 12:31
AnonX31-Dec-08 12:31 
GeneralAnother useful tool: a generic, auto-marshalling wrapper class for an unmanaged struct (in C#) [modified] Pin
AnonX3-Jan-09 9:38
AnonX3-Jan-09 9:38 
GeneralThanks for the early comments, the article is still a work in progress.. Pin
AnonX30-Dec-08 11:11
AnonX30-Dec-08 11:11 
GeneralRe: Thanks for the early comments, the article is still a work in progress.. Pin
rcollina30-Dec-08 22:33
rcollina30-Dec-08 22:33 
Generalsizeof(T) vs Marshal.SizeOf(typeof(T)) Pin
Daniel Grunwald30-Dec-08 8:25
Daniel Grunwald30-Dec-08 8:25 
GeneralRe: sizeof(T) vs Marshal.SizeOf(typeof(T)) Pin
Jörgen Sigvardsson30-Dec-08 9:49
Jörgen Sigvardsson30-Dec-08 9:49 
GeneralRe: sizeof(T) vs Marshal.SizeOf(typeof(T)) Pin
AnonX30-Dec-08 11:17
AnonX30-Dec-08 11:17 
GeneralRe: sizeof(T) vs Marshal.SizeOf(typeof(T)) - You are correct, there's a significant error in the article. &lt;-- [modified] Pin
AnonX30-Dec-08 23:22
AnonX30-Dec-08 23:22 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.