Click here to Skip to main content
15,861,168 members
Articles / Programming Languages / C#

CLR Hosting - Customizing the CLR - Part 2

Rate me:
Please Sign up or sign in to vote.
4.86/5 (15 votes)
11 Jul 2012Ms-PL9 min read 51.1K   1.2K   48   16
.NET apps are run by the .NET runtime. There exists an unmanaged API which allows you to run apps under a customized runtime. This API lets you strengthen security, provide a different deployment strategy, add framework logging, provide your own memory management implementation, and sandbox an appli

Image 1

Introduction

This is a follow-up article to CLR-Hosting-Customizing-the-CLR. In this article we will create some sample customizations where we modify assembly-loading, strengthen security, log exceptions, and replace the default memory handler.

Background

When I started experimenting with these interfaces the main goal was to find ways to help me in testing and debugging managed applications.

Sometimes I insert instrumenting code directly in the source code, aka the old printf-method, but nowadays I usually use Trace calls. The disadvantage is that I have to remove them when I am done. What can be more troublesome is that some assemblies come without source code. Even in-house assemblies can be a problem. In my project we include precompiled assemblies made by another department, the downside is that I lack read permissions for their source tree. 

Another approach I use is looking at the program in a debugger, preferably Windbg with some managed extension, such as Sos or Sosex. The disadvantage of this is that it is time-consuming, and sometimes hard to know when to break execution and where to look.

What attracted me with the customization of the CLR approach was that, the Application Under Test (AUT) would be unaware of that it was under supervision. It doesn't need code modifications, and it would work for all applications out of the box.

Customizations

In this article we will see how to build a Sandbox, log exceptions, change assembly loading, and replace the memory manager. There are many customization interfaces left to explore, but those are beyond the scope of this article.

AppSandboxer.exe

A Sandbox is about two related concepts.

Isolation

Isolation can be interesting in order to limit side-effects from applications. Everything that happens inside a Sandbox is local, it doesn't affect the rest of the system. It can for instance be used to retry trial software over and over again Smile | <img src= " /> , or rerun tests in a fresh environment, just by deleting the old and creating a new one. This is similar to undo disks, when working with virtual machines.

Limiting runtime permissions

Instead of creating a complete copy of the system inside a sandbox to achieve isolation, it is possible to remove permissions, so side-effects are limited. If you have downloaded a program, and the origin is unknown, it might not be totally safe to run it. If you have a firewall, you should be able to remove the internet access for the application. If we downloaded an app that only is supposed to show the time, we might be interested to remove IO permissions, but let it have access to the internet for time synchronization. The more permissions we remove, the less side-effects are propagated to the rest of the system.

Security Manager and PermissionSet

In an early version of the CLR host interface there was IHostSecurityManager, which contains method such as ResolvePolicy, ProvideAssemblyEvidence, ProvideAppDomainEvidence, and DetermineApplicationTrust. I made a small customization, but this interface was of limited use. The Policy related to AppDomains had been deprecated, adding Zone information to the Evidence to strengthen security had no effect. It has been superseded by the use of PemissionSet.

The easiest way to strengthen security is to do it in the AppDomainManager itself. When the method CreateAppDomain is called, you simply create a PermissionSet object, to which you add or remove permissions. Then you pass this permission set as a parameter when you create the AppDomain.

Implementation

Our Sandbox will allow Execution, IO read permission (current dir), and interaction with the console. The code builds on the previous article, where we showed the boiler plate code to get started with AppDomainManagers. Below I just list the essential code. Full source can be found in the source attachment.

C#
public sealed class AppDomainManagerSandboxer : AppDomainManager, IAppDomainManager
{
  public void Run(string assemblyFilename, string friendlyName)
  {
    var appDomainInfo = new AppDomainSetup();
    appDomainInfo.ApplicationBase = new System.IO.FileInfo(assemblyFilename).DirectoryName;
    
    // Start with no permissions
    PermissionSet permSet = new PermissionSet(PermissionState.None);
    
    // Allow it to run
    permSet.AddPermission(new SecurityPermission(SecurityPermissionFlag.Execution));
    
    // Allow interaction with console
    permSet.AddPermission(new UIPermission(PermissionState.Unrestricted));
    
    // Allow reading the current dir
    permSet.AddPermission(new FileIOPermission(FileIOPermissionAccess.PathDiscovery,
                          AccessControlActions.View, appDomainInfo.ApplicationBase));
    permSet.AddPermission(new FileIOPermission(FileIOPermissionAccess.Read, 
                          AccessControlActions.View, appDomainInfo.ApplicationBase));
                          
    // Strong name lists.
    // Contains assemblies that are considered safe, that run with full trust
    var strongNames = new StrongName[0];
    ad = AppDomain.CreateDomain(friendlyName, AppDomain.CurrentDomain.Evidence,
                                appDomainInfo, permSet, strongNames);

    int exitCode = ad.ExecuteAssembly(assemblyFilename);
                                
    AppDomain.Unload(ad);
  }
}

Running the Sample

First SampleApp6.exe is executed normally, then within our Sandbox, with IO permissions removed. The whole exception is traced in full detail to the OutputDebug. I recommend using a program such as DebugView.exe to see those types of logs.

Image 3

AppRedirector.exe

Ever wondered why web applications load assemblies from the bin directory and not from the root? We too can emulate this behavior by manipulating PrivateBinPathProbe and PrivateBinPath.

Implementation

C#
public sealed class AppDomainManagerRedirector : AppDomainManager, IAppDomainManager
{
  public void Run(string assemblyFilename, string friendlyName)
  {
     var appDomainInfo = new AppDomainSetup();

     // Prevent loading from current dir
     appDomainInfo.PrivateBinPathProbe = "*";
     
     // Set base dir. Mandatory if PrivateBinPath is used
     var baseDir = System.IO.Path.GetDirectoryName(assemblyFilename);
     appDomainInfo.ApplicationBase = baseDir;
          
     // Load assemblies from plugins and bin subfolders.
     var appDir = System.IO.Path.Combine(baseDir, "bin");
     var pluginDir = System.IO.Path.Combine(baseDir, "plugins");     

     appDomainInfo.PrivateBinPath = pluginDir + ";" + appDir;

     AppDomain ad = AppDomain.CreateDomain(friendlyName, null, appDomainInfo);
     AppDomainManager appDomainManager = ad.DomainManager;

     try
     {
        int exitCode = ad.ExecuteAssembly(assemblyFilename);
        System.Diagnostics.Trace.WriteLine(string.Format("ExitCode={0}", exitCode));
        System.Diagnostics.Trace.WriteLine("Executed Run");
     }
     catch (System.Exception)
     {
        string message = string.Format("Unhandled Exception in {0}",
                                       System.IO.Path.GetFileNameWithoutExtension(assemblyFilename));
        System.Diagnostics.Trace.WriteLine(message);
     }
     finally
     {
        AppDomain.Unload(ad);
     }
  }
}

Running the Sample

In this particular implementation, we tell the App to use the subfolders bin and plugins, and completely ignore the root folder. We use SampleApp5.exe for demo purposes, which just prints "--- abc ---". I have made a modified version of TestLib.dll in the bin folder, which instead prints "*** abc ***" to get a visual difference of the output.

Image 4

AppSupervisor.exe

For testing, logging, and diagnostics, it can be useful to log runtime data. Far too often, I have seen misusage of try/catch. Sometimes exceptions are silently ignored, leaving the app in a corrupt state, then the app crashes at a later time when some other code runs. We will make a small Exception logger that will log all managed exceptions that occur, regardless whether they are handled or not. This way we can go back and see if indeed these exceptions occurred for the right reason. We will achieve this by registering a handler for FirstChanceExceptions in all AppDomains.

Implementation

C#
public sealed class AppDomainManagerSupervisor : AppDomainManager, IAppDomainManager
{
  public override AppDomain CreateDomain(string friendlyName, 
                  Evidence securityInfo, AppDomainSetup appDomainInfo)
  {
     Trace("CreateDomain");
     System.Diagnostics.Trace.WriteLine(string.Format("AppDomain::CreateDomain({0})", friendlyName));
     var appDomain = base.CreateDomain(friendlyName, securityInfo, appDomainInfo);
     appDomain.FirstChanceException += AppDomainFirstChanceException;
     return appDomain;
  }

  public static void AppDomainFirstChanceException(object sender,
                     System.Runtime.ExceptionServices.FirstChanceExceptionEventArgs e)
  {
     System.Console.Error.WriteLine("AppDomainFirstChanceException - First Chance Exception");
     var sb = new StringBuilder();
     sb.AppendLine("AppDomainFirstChanceException - First Chance Exception");
     sb.AppendLine(e.Exception.StackTrace);
     System.Diagnostics.Trace.WriteLine(sb.ToString());
  }
}

Running the Sample

SampleApp1.exe throws a NotImplementedException, the exception however is caught inside the program so we will never know about it. Running it through AppSupervisor.exe, it will be detected as a first chance exception and logged to DebugOutput.

Image 5

AppExperimental.exe

A natural add on to a supervisor would also be to add an UnhandledExceptionHandler. Normally, if an exception is not caught, it will eventually lead to a crash of the application. First it will unwind the stack and continue looking for exception handlers, until it runs out of stacks to unwind. What is possible to do is, adding a global Exception handler on the Application level or on the AppDomain level.

Implementation

C#
public sealed class AppDomainManagerExperimental : AppDomainManager, IAppDomainManager
{
  public void Run(string assemblyFilename, string friendlyName)
  {
    // Add Exception Handler on Appliction level
    Application.ThreadException += ApplicationThreadException;
    
    // Add Exception Handler on Current Domain
    AppDomain.CurrentDomain.UnhandledException += AppDomainUnhandledException;
    AppDomain ad = AppDomain.CreateDomain(friendlyName);
    
    // Add Exception Handler on newly created domain
    ad.UnhandledException += AppDomainUnhandledException;
    AppDomainManager appDomainManager = ad.DomainManager;

    try
    {
      int exitCode = ad.ExecuteAssembly(assemblyFilename);
      System.Diagnostics.Trace.WriteLine(string.Format("ExitCode={0}", exitCode));
    }
    catch (System.Exception)
    {
      string message = string.Format("Unhandled Exception in {0}",
                                     System.IO.Path.GetFileNameWithoutExtension(assemblyFilename));
      Trace(message);
      System.Console.Error.WriteLine(message);
    }
    finally
    {
      AppDomain.Unload(ad);
    }
  }
}

Running the Sample

For the purpose of demonstration, I will use SampleApp2.exe. It throws a NotImplementedException, which is not caught by any handler.

Image 6

Funny... it didn't end up in either of our exception handlers.

SampleApp3.exe works exactly like SampleApp2.exe, but it actually installs the Unhandled Exception Handler itself.

Look closely. The handler stops working.

Image 7

The unhandled exception handler installed by SampleApp3.exe stops working. It is a known bug in the CLR. It was reported for the 2.0 runtime. You can read more about it here: CLR hosting exception handling in a non-CLR-created thread. The explanation is: "The behavior is indeed a bug caused by the CLR execution engine and the CRT competing for the UnhandledExceptionFilter". We are executing via COM, which automatically inserts a try/catch. The unhandled exception filter can only be set once. COM sets it first. Then the CLR tries to do it. According to this page, it was supposed to be fixed for the release of v4.0. But I didn't get it to work. If anyone does, please tell.

Replacing the Memory Manager

IHostControl::GetHostManager is an interface where you can plug in your own implementation of a memory handler.

We shall see how it can be done.

Implementation

First we will need a modified IHostControl implementation, and supply a custom memory manager called MyHostMemoryManager.

C#
HRESULT STDMETHODCALLTYPE MyHostControlMemoryManager::GetHostManager(REFIID riid,void **ppv)
{
   if (riid == IID_IHostMemoryManager)
   {
      IHostMemoryManager *pMemoryManager = new MyHostMemoryManager();
      *ppv = pMemoryManager;
      return S_OK;
   }
   // IID_IHostTaskManager
   // IID_IHostThreadpoolManager
   // IID_IHostSyncManager
   // IID_IHostAssemblyManager
   // IID_IHostGCManager
   // IID_IHostPolicyManager
   // IHostSecurityManager
   else
   {
      *ppv = NULL;
      return E_NOINTERFACE; 
   }
}

Now we are finished with our IHostControl implementation. The next step is to implement the IHostMemoryManager.

MC++
class MyHostMemoryManager : public IHostMemoryManager
{
public:

   virtual HRESULT STDMETHODCALLTYPE CreateMalloc( 
      /* [in] */ DWORD dwMallocType,
      /* [out] */ IHostMalloc **ppMalloc)
   {
      *ppMalloc = new MyHostMalloc();
      return S_OK;
   }

   virtual HRESULT STDMETHODCALLTYPE VirtualAlloc( 
      /* [in] */ void *pAddress,
      /* [in] */ SIZE_T dwSize,
      /* [in] */ DWORD flAllocationType,
      /* [in] */ DWORD flProtect,
      /* [in] */ EMemoryCriticalLevel eCriticalLevel,
      /* [out] */ void **ppMem) 
   {
      *ppMem = ::VirtualAlloc(pAddress, dwSize, flAllocationType, flProtect);
      return S_OK;
   }

   virtual HRESULT STDMETHODCALLTYPE VirtualFree( 
      /* [in] */ LPVOID lpAddress,
      /* [in] */ SIZE_T dwSize,
      /* [in] */ DWORD dwFreeType)
   {
      ::VirtualFree(lpAddress, dwSize, dwFreeType);
      return S_OK;
   }
   
   virtual HRESULT STDMETHODCALLTYPE VirtualQuery( 
      /* [in] */ void *lpAddress,
      /* [out] */ void *lpBuffer,
      /* [in] */ SIZE_T dwLength,
      /* [out] */ SIZE_T *pResult) 
   {
      *pResult = ::VirtualQuery(lpAddress, (PMEMORY_BASIC_INFORMATION) lpBuffer, dwLength);
      return S_OK;
   }

   virtual HRESULT STDMETHODCALLTYPE VirtualProtect( 
      /* [in] */ void *lpAddress,
      /* [in] */ SIZE_T dwSize,
      /* [in] */ DWORD flNewProtect,
      /* [out] */ DWORD *pflOldProtect) 
   {
      ::VirtualProtect(lpAddress, dwSize, flNewProtect, pflOldProtect);
      return S_OK;
   }

   virtual HRESULT STDMETHODCALLTYPE GetMemoryLoad( 
      /* [out] */ DWORD *pMemoryLoad,
      /* [out] */ SIZE_T *pAvailableBytes)
   {
      *pMemoryLoad = 30;  // percent
      *pAvailableBytes = 100 * 1024*1024;
      return S_OK;
   }
   virtual HRESULT STDMETHODCALLTYPE RegisterMemoryNotificationCallback( 
      /* [in] */ ICLRMemoryNotificationCallback *pCallback)
   {
      return S_OK;
   }

   virtual HRESULT STDMETHODCALLTYPE NeedsVirtualAddressSpace( 
      /* [in] */ LPVOID startAddress,
      /* [in] */ SIZE_T size)
   {
      return S_OK;
   }

   virtual HRESULT STDMETHODCALLTYPE AcquiredVirtualAddressSpace( 
      /* [in] */ LPVOID startAddress,
      /* [in] */ SIZE_T size)
   {
      return S_OK;
   }

   virtual HRESULT STDMETHODCALLTYPE ReleasedVirtualAddressSpace( 
      /* [in] */ LPVOID startAddress)
   {
      return S_OK;
   }
};

Oh my god! That is a dirty implementation. No error handling. Even skipping the implementation in a few methods. Don't ever use this code in a real project. The important method here is really the MyHostMemoryManager::CreateMalloc where we implement the IHostMalloc interface.

MC++
class MyHostMalloc : public IHostMalloc
{
public:
  HRESULT STDMETHODCALLTYPE Alloc(SIZE_T cbSize,
          EMemoryCriticalLevel eCriticalLevel,
          void** ppMem)
  {
    void* memory = new char[cbSize];
    *ppMem = memory;
    g_noAllocs++;
    return S_OK;
  }

  HRESULT STDMETHODCALLTYPE DebugAlloc(SIZE_T cbSize,
          EMemoryCriticalLevel       eCriticalLevel,
          char*       pszFileName, 
          int         iLineNo,
          void**     ppMem)
  {
    void* memory = new char[cbSize];
    ZeroMemory(memory, cbSize);
    *ppMem = memory;
    return S_OK;      
  }

  HRESULT STDMETHODCALLTYPE Free(void* pMem)
  {
    g_noFrees++;
    delete [] pMem;
    return S_OK;
  }
};

The implementation of IHostMalloc is straightforward. It is just a dummy implementation, forwarding calls to new and delete. What is possible to do is adding magic numbers before and after a memory region, or add your own counters. I added two counters, one for the number of Allocs and one for the number of Frees.

Just for the record, new and delete are very slow. Adding more CPUs won't help, calls to new and delete never run in parallel. The hardware must guarantee that a memory region is only allocated once. A good memory manager would therefore allocate a big block of memory once and give out portions of it, so the stall can be avoided. Hey, just like the default one in the CLR runtime does. Smile | <img src= " />

Running the Sample

The memory manager is present in all AppLaunchers, just add "-mem" as the second parameter.

Image 9

The last two lines contain counters that are implemented by our custom implementation.

Troubleshooting

The .net app may fail to start for several reasons. One reason is that the assemblies are not found, or the .config is not found. FUSLOGVW.exe is an excellent tool for troubleshooting assembly load errors.

WPF apps may not start. I had to mark the main thread of the AppLauncher as STA, Single Thread Apartment. In wpf, only an STA thread may allocate objects.

The .net app runs in the process space of AppLauncher. It will not read the correct config file. Please copy the MyNetApp.exe.config to AppLauncher.exe.config.

The probing path and searchpath of DLLs may be wrong. Assemblies are only loaded from current directory or subfolders. It doesnt seem to work starting the AppLauncher from one location and loading DLLs from another location. There is a permission error.

All these errors are revealed by FUSLOGVW.exe

Conclusion

It was unfortunate that the UnhandledException handler didn't work. Personally, it would have been nice to get a customization that gives me extra logging when I need to test or diagnose bugs in an application. I see some potential in using the CLR hosting interfaces. If you write something cool and useful with these interfaces, please tell.

Points of Interest

This article is a follow up article to CLR-Hosting-Customizing-the-CLR, it is mandatory reading. The main source of documentation is of course MSDN itself, .NET Framework 2.0 Hosting Interfaces.

A great book regarding the CLR Hosting API, is Customizing the Microsoft® .NET Framework Common Language Runtime. It is a bit old, but the best (and perhaps the only one in existence).

History

  • 11th July, 2012, Initial post
  • 24th May, 2019, updated source code to support WPF apps, and added Troubleshooting section.

License

This article, along with any associated source code and files, is licensed under The Microsoft Public License (Ms-PL)


Written By
Architect Visma Software AB
Sweden Sweden
Mattias works at Visma, a leading Nordic ERP solution provider. He has good knowledge in C++/.Net development, test tool development, and debugging. His great passion is memory dump analysis. He likes giving talks and courses.

Comments and Discussions

 
Generalwhy do I get no IO security exception? Pin
Southmountain1-Jun-19 17:04
Southmountain1-Jun-19 17:04 
Questionx64 Build Pin
CatchExAs22-Apr-14 5:26
professionalCatchExAs22-Apr-14 5:26 
AnswerRe: x64 Build Pin
CatchExAs22-Apr-14 6:13
professionalCatchExAs22-Apr-14 6:13 
GeneralRe: x64 Build Pin
Mattias Högström22-Apr-14 11:20
Mattias Högström22-Apr-14 11:20 
GeneralRe: x64 Build Pin
CatchExAs23-Apr-14 4:26
professionalCatchExAs23-Apr-14 4:26 
GeneralRe: x64 Build Pin
Mattias Högström23-Apr-14 22:56
Mattias Högström23-Apr-14 22:56 
QuestionIHostSyncManager implementation Pin
Ruurd Keizer15-Aug-13 2:59
Ruurd Keizer15-Aug-13 2:59 
AnswerRe: IHostSyncManager implementation Pin
Mattias Högström18-Aug-13 10:23
Mattias Högström18-Aug-13 10:23 
GeneralRe: IHostSyncManager implementation Pin
Ruurd Keizer18-Aug-13 20:52
Ruurd Keizer18-Aug-13 20:52 
QuestionPassing native data into appdomain Pin
David Terk2-May-13 4:36
David Terk2-May-13 4:36 
AnswerRe: Passing native data into appdomain Pin
Mattias Högström2-May-13 20:40
Mattias Högström2-May-13 20:40 
When the first version of the hosting API was released, a large part was in native code. In many ways it was easier to get started with. The problem was that sometimes it resulted in unintended marshaling of the whole appdomain. The marshaling, created in memory copies, which could lead to bugs and slow performance.
The recommended way to communicate with an AppDomain is through managed code. Meaning that you will also need purely managed or (C++/CLI) code to access it.

Is it only a native API that you want to access? In order to access a native API from managed code, I would create a C++/CLI wrapper. Even if it is the other way around (inject native objects into managed code), I would use a C++/CLI wrapper. Much easier to write and debug.

Hypothetically. If you really do want to use hosting, you can write an AppDomainManager. In the CreateAppDomain method you can store the reference to a new AppDomain. When you receive data or events, in your hosting App, you can call methods on objects in the AppDomain, or even shuffle data between App Domains. You can expose a callable object (maybe through a singleton), or use reflection to find suitable classes. In some way or another, I think you are forced to use the C++/CLI. The native objects needs to be marshaled/serialized into managed objects. C++ class A, must have a managed version class A too, in order to inject it and use it. This is one of the reasons for using C++/CLI.

I would avoid using the hosting API if possible.
Please clarify if I have misunderstood your question.
QuestionGood Pin
BillW336-Aug-12 5:12
professionalBillW336-Aug-12 5:12 
GeneralMy vote of 5 Pin
Madhan Mohan Reddy P11-Jul-12 18:37
professionalMadhan Mohan Reddy P11-Jul-12 18:37 
Questionexcellent! Pin
Southmountain9-Jul-12 8:47
Southmountain9-Jul-12 8:47 
QuestionCool Pin
Sacha Barber9-Jul-12 5:00
Sacha Barber9-Jul-12 5:00 
AnswerRe: Cool Pin
Mattias Högström10-Jul-12 6:32
Mattias Högström10-Jul-12 6:32 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.