Click here to Skip to main content
Click here to Skip to main content

Runtime Compilation (A .NET eval statement)

By , 13 Jan 2006
 

Introduction

There are several languages out there (primarily interpreted languages) which provide a convenient eval statement or function. These usually accept simple fragments of code and execute them on the fly, often used for mathematical expressions. .NET leaves this useful feature out of its BCL (Base Class Library), and buries access to its compiler under incomplete documentation. I was working on a project of my own when I realized that I needed some form of runtime compilation, and went off on a tangent to build an easy-to-use library (also adding full XML documentation).

Using the code

The library is centered around the Eval class, which contains the static methods that access the compiler.

The methods provided are as follows:

public static AssemblyResults CreateAssembly(ICodeCompiler compiler,
                                          string assemblySource,
                                          CompilerParameters options,
                                          Language language);
public static TypeResults CreateType(ICodeCompiler compiler,
                                     string typeSource,
                                     string typeName,
                                     CompilerParameters options,
                                     Language language);
public static MethodResults CreateMethod(ICodeCompiler compiler,
                                         string methodSource,
                                         string methodName,
                                         CompilerParameters options,
                                         Language language);

public static AssemblyResults CreateVirtualAssembly(ICodeCompiler compiler,
                                         string assemblySource,
                                         bool debug,
                                         Language language,
                                         params string[] references);
public static TypeResults CreateVirtualType(ICodeCompiler compiler,
                                         string typeSource,
                                         string typeName,
                                         Language language,
                                         bool debug,
                                         params string[] references);
public static MethodResults CreateVirtualMethod(ICodeCompiler compiler,
                                         string methodSource,
                                         string methodName,
                                         Language language,
                                         bool debug,
                                         params string[] references);

Each method compiles the source provided, loads the result into memory, and returns a reference to an object wrapping the final result (along with any warnings generated by the compiler). If any errors are generated during the compilation, a CompilationException is thrown, which contains the compiler's errors. The "Virtual" group of methods stores the results of compilation directly to memory, using a pre-generated set of CompilerParameters.

The Language class and its subclasses provide information about an individual language to the Eval class, exposing methods that generate various language-specific segments of the code which are necessary in the CreateType and CreateMethod methods of the Eval class.

The result classes each expose the actual result through a property named by the type of the result (respectively through the Assembly, Type, and Method properties), as well as the collection of warnings generated during compilation (through the Warnings property). In addition, each class provides indirect access to some of the more commonly used abilities of its underlying result (each of which acts as described in its XML documentation).

The AssemblyResult class provides the following method:

public TypeResults GetType(string typeName, bool throwOnError);

The TypeResults class provides the following methods:

public MethodResults GetMethod(string name);
public MethodResults GetMethod(string name, 
                            params Type[] paramTypes);

public object Instantiate(params object[] parameters);

Finally, the MethodResults class provides the following method:

public object Invoke(params object[] parameters);

This method automatically creates an instance of the declaring class on which to invoke the wrapped method, allowing non-static methods to be invoked as if they were static.

Points of interest

Microsoft included nearly everything that I needed to write this library in their standard .NET Framework, except the final wrapper. If you find that you need something beyond the capabilities of my library, the System.CodeDom.Compiler and System.Reflection namespaces should provide most of what you need.

To do

More than a few things, most likely... Currently only one update comes to mind. If there is any demand for it, I will make the results of my runtime compilation interface unloadable from memory by using AppDomains (inspired by and probably borrowing from the DevX article Dynamically Executing Code in .NET).

This is my first article posted, so I'd appreciate any feedback. E-mail feedback is accepted, but I'd prefer feedback through the message board provided by the CodeProject.

History

  • 12th January, 2006
    • Updated code (again) to fix a bug with referenced libraries and an issue with in-memory compilation. Both features should now work as intended.
  • 6th December, 2003
    • Refactored code (added Language class and subclasses, updated Eval class) in order to make the Eval class work fully with languages other than C#, added VB.NET example, and updated the article to reflect these changes.
  • 14th January, 2003
    • Updated code to fix minor bugs with referenced libraries.
  • 11th January, 2003
    • Zip files finally corrected (extensive paths removed).
  • 2nd January, 2003
    • Example project, screenshot, and To do added, a few bugs fixed.
  • 31st December, 2002:
    • Article first posted.
    • ~5:31 PM EST: <pre> blocks fixed (Sorry!).

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

About the Author

Eric Astor
Web Developer
United States United States
Member
No Biography provided

Sign Up to vote   Poor Excellent
Add a reason or comment to your vote: x
Votes of 3 or less require a comment

Comments and Discussions

 
You must Sign In to use this message board.
Search this forum  
    Spacing  Noise  Layout  Per page   
GeneralMy vote of 5memberMember 17933029 Apr '13 - 4:12 
Good stuff. Very good. A winform/wpf based UI would perhaps been easier to code and also easy for users like me to consume. Hardly matters.
GeneralGood job!memberShane Story3 Sep '10 - 2:37 
I haven't had to use this feature yet, but I appreciate the groundwork you've done in case I ever do.
Shane
Jesus loves you! John 3:16 · Rom 3:23 · Rom 6:23 · Rom 10:9

GeneralMy vote of 5memberPrasenjit Purohit6 Jul '10 - 19:28 
A very good and complete article.
Generalnice articlememberniuzj30 Nov '09 - 20:30 
My vote is 5
GeneralByRef ArgumentsmemberMember 28930215 Sep '08 - 3:33 
It might be stupid, but I'm able to pass arguments by reference to a virtual method I created. I can pass the argument, but they work as if they were by value.
I'm I missing something stupid? Please tell me how to do this.
Thanks.
GeneralPassing an array to the MethodmemberMisterT9917 Oct '07 - 6:12 
Hi Eric,
 
Thanks for the article. This has really helped alot.
 
I would like to pass an array to the method, so I tried changing the source text to:
 
Public Function Method1(ByVal s() As String) As String
 
This gave a parameter count mismatch exception.
 
Any idea how to do this?
 
Thanks,
Tom
Generalcompile error on MC++ VS003memberclee005 Jul '06 - 12:25 
hi
 
could someone point what im missing, i cp this code from C# version,
C# compile ok, but not in MC++,
 
compile error 'Evaluator::Eval::CreateVirtualMethod' : function does not take 5 arguments
 

CSharpCodeProvider* aaa = new CSharpCodeProvider();
 
stringChanger = Eval::CreateVirtualMethod(
aaa->CreateCompiler(),
source->ToString(),
S"StringChanger",
new CSharpLanguage(),
false);
 
what am i missing?
 
thx
 

NewsDynamic CodememberPrimadi6 Jun '06 - 0:03 
Better Code i think :
http://www.west-wind.com//presentations/dynamicCode/DynamicCode.htm
QuestionReference COM-Object?memberdEUs[CPP]5 May '06 - 0:08 
Hi,
 
is it possible to somehow create the interop.dll needed for COM-Objects on the fly, too? So the user just tells me "I need want to use the COM-Object with this and this GUID" and I internally create the interop for him and add it to the compiler-references.
 
Regards,
Daniel
QuestionInclude current namespace?memberMellmountain24 Mar '06 - 0:21 
Hi Eric,
 
thanks for the lib! I've a problem when using the CreateVirtualMethod. I want to compile a method with classes from my own namespace. How do i make the compiler accept my classes?
 
Thanks!

AnswerRe: Include current namespace?memberMellmountain29 Mar '06 - 2:07 
Well now I've solved the problem, the sollution is so simple (doh!)
 
I had no idea that you could send a .exe file to CompilerOptions. All I did was to send my .exe file to the compiler options like this:
 
CompilerOptions cpOptions = new CompilerOptions();
cpOptions.ReferencedAssemblies.Add("myexe.exe");
 
Happy coding ppl!
/Mellmountain
QuestionCompile multiple language source at same time?memberFocusedWolf4 Mar '06 - 6:07 
Hi, i'm not sure really how to approach this situation.
 
I want the option to do stuff where one script can call another, and it doesn't matter which .net language they are writtin in cause i'm calling them with reflection anyway.
 
So maybe theirs so compiler option flag? hmm.
 
Also how can multiple source files be handled?...zzz confussing to do anything but compile an assembly and call it.
QuestionUsing runtime compilation in a multi-threaded applicationmemberx-cubed22 Feb '06 - 9:32 
Thanks for the library Eric!
 
I have an application that instantiates a number of classes called FGI, which each have a Calculation property that holds a simple mathematically equation. I am using CreateVirtualMethod to create an in-memory method called Calculate, to run this equation and return the result. The calculations are all loaded from a database when the program starts, and are (probably) not going to be modified while the program is running.
 
Because this Evaluation library creates the DLL, even for "in-memory" functions, I end up with threading issues where multiple threads are presumably trying to write to the DLL at one time, causing an occassional FileNotFoundException.
 
- Is there anyway to avoid using the DLL completely?
- Is there anyway to create the method in an existing instance of a class?
(ie: directly within each instance of FGI, bearing in mind that each instance of the class will use a different calculation)
- Or do I really need to lock all the other objects in the application?
 

 
Thanks
Carey Bishop
 
-- modified at 15:47 Wednesday 22nd February, 2006
AnswerRe: Using runtime compilation in a multi-threaded applicationmemberEric Astor22 Feb '06 - 9:47 
Carey,
 
I'm glad the library's helping you! Out of curiosity, are you using the most up-to-date version of the library, as posted above? I cleared up a few issues with the in-memory compilation in this revision.
 
I'm afraid there's no way to completely avoid using the DLL - the compiler appears to need to be invoked with a file targeted, even when invoked programmatically. As to creating the method within an existing instance, this is certainly impossible with this library. I know locking on an external object may remove quite a bit of the benefit - but I don't think I can be very helpful.
 
However, it sounds like the issue is that the compiler is automatically targeting the same DLL somehow, despite the supposed guarantee of uniquely chosen names from the "in-memory" compilation (not my code, part of the .NET Framework)... it's possible that the compiler simply isn't equipped to be run this way programmatically.
 
Could you possibly refactor this part of your code to output to different DLLs for each thread, and then simply delete them? Their deletion should be possible as soon as you've loaded the Assembly into your AppDomain - which, at least theoretically, has already happened once you get a Results object back.
 
I'm somewhat short on time at the moment... I'd be glad to hear about any further problems you have or progress you make, though, and I'll certainly look into integrating fixes/improvements for them into this library once I have the time.
 
Thanks,
Eric Astor
AnswerRe: Using runtime compilation in a multi-threaded applicationmemberx-cubed22 Feb '06 - 11:11 
Thanks for your help Eric.
 
Yes, I am using the most recent version of the library.
 
I was considering the different DLLs solution, but I've found a better and simpler way. My previous VB6 experience led me to believe that having to lock objects would be complex and buggy, but it seems that using the Enter() and Exit() methods of System.Threading.Monitor on an arbitary global variable works fine. I just lock the global variable, create the method, then unlock it. Works like a charm!
 

 

 
Thanks again,
Carey
GeneralYaaay...memberRoss Presser13 Jan '06 - 3:36 
Finally ... looks like a decent replacement for the MSScript control.
 
I have a few projects that were coded around the MSScript control originally, and I've tried to move them ... one tried out using JScript.NET's eval method; another continues to use MSScript control, and crashes pretty often; another gave up on full eval functionality and became an XML-based text substitutor, with XML attributes controlling some additional abilities (like toupper() and string.format() calls). A real eval function looks like it will definitely fit the bill.
 
Now I only need time to migrate...
GeneralReferencing external assembliesmemberEric Astor12 Jan '06 - 4:05 
Well, for everyone who's asked me how to do this: My apologies! I've finally gotten around to investigating this, and it looks like I actually introduced an error in my Eval library at some point. For those who are interested, the actual error is using the Assembly's FullName in the "using" statements, rather than digging a couple of levels deeper and getting the name I should actually be referencing.
 
Regardless, I'll put the edited version into the CodeProject updating process in the near-future, and if you want the fixed library before it goes up, just send me an e-mail!
GeneralRe: Referencing external assembliesmemberEric Astor12 Jan '06 - 4:22 
Please note that with this fix, the references part of the compiler options should now function as intended. For a DLL in the same directory that your application is running in, simply add a reference to "YOURDLLNAMEHERE.dll"! For a library located elsewhere, just make sure that the string specifies a path to the library, relative to the application directory (an absolute path works also, but probably isn't a good idea).
Generalinclude dllmember[ac]Sebastian12 Apr '05 - 5:13 
Hi,
i hope anyone can help me.
I would like include a extern Namespace in the Runtimer Compiler, it is a dll from my own class.
 

///
public static TypeResults CreateType(ICodeCompiler compiler, string typeSource, string typeName, CompilerParameters options, Language language)
{
StringBuilder sourceBuilder = new StringBuilder();
options.ReferencedAssemblies.Add("MyUSB.dll"); //<--- ERROR
foreach(string referenced in options.ReferencedAssemblies)
{
sourceBuilder.Append(language.useStatement(Assembly.LoadFrom(referenced).FullName));
}
 

how can i include the Dll with her functions in the compiler?
on compile i become the error mesage BC30203 Identifer expected
 
I would like access with the functions in this dll the usb port. In my other projects i can include this with using MyUsb;.
 
Sebastian

GeneralFeature suggestion: namespacesmembersteve willer6 Aug '04 - 4:37 
Your library is very useful, thanks. If you wanted to further develop it, my feature suggestion would be support for a separate set of "using" statements, since you can't just stick it in the body of the code you're eval'ing.
 
Without support for 'using', it makes it hard to use external packages.
GeneralSpecify output directorymemberadlionelC2 Jun '04 - 6:55 
Hi,
 
I've compiled and ran with success a project using this class. I'm able to compile inline C# from an asp .net webapp. But now I've discovered a problem on an other computer with the same webapp :
 
While options set GenerateInMemory to true, a file is written (Evaluated.dll) in the current directory. On the second computer the current dirctory is C:\WinNNt\System32. Because it is file-write protected the script fails with an access denied error (CS0016).
 
So how can I set the output directory to an authorized one? I've tried to set options.CompilerOptions to "/out:blahblah", it writes the dll in the good directory but in this case I'm said that C:\winnt\System32\evaluated.dll cannot be found!
 
Thanks for your answers,
 
Lionel
GeneralRe: Specify output directorymemberadlionelC2 Jun '04 - 22:12 
OK, problem solved :
 
There is a little mistake in the code. If you define an output assembly in the compiler options, file output will be on even if you set GenerateInMemory to True. So because file output is true the generated dll is written in the current directory (c:\winnt\system32 if you are running a webapp).
 
When GenerateInMemory is working, a temporary dll (whose name is a unique and autogenerated one) is put in your temp directory, which is file-writeable.
 
Nota : in my case my custom type couldn't be found in the temporary assembly. To solve this problem I've removed this part of code from the createType method :
 
options.OutputAssembly + "." +
GeneralRe: Specify output directorymemberEric Astor3 Jun '04 - 0:53 
Thanks very much for pinning down the problem... it seems we were working in parallel on this since you posted the original request, but I had no luck locating the line giving the difficulty. I'll make this update available with a new version of Eval soon.
Questioncan this be used as a server side componentsussAnonymous22 May '04 - 19:00 
can anyone help, I need
 

--example of command line to compile into assembly and;
-- example of using this from an .aspx file
GeneralReference External Structuremembergscloete@yahoo.com21 Jan '04 - 8:11 
Hi,
 
this is a awesome piece of code 12 out of 10!! Smile | :) I works really good, but I am a vb person and wanted to do the following, and is just not sure where to modify the code to achieve it:
 
The virtual function that I built up it the string builder, I want to pass a structure to the function, but the structure is declare in another project of mine, which is in the solution and I have referenced it in the Evaluator project, but during compile time, it still does not reconigce that structure ie it states the type is not defined.
 
Can you perhaps help, guide me in the right direction please
Thanks
Gert
GeneralRe: Reference External StructurememberEric Astor21 Jan '04 - 10:24 
Sure I can help you with this... Just see my reply and the responses to it on one of the earlier threads on this message board, Using methods from external class[^]
GeneralRe: Reference External Structuremembergscloete@yahoo.com21 Jan '04 - 10:27 
thanks alot Smile | :)
i worked it more and more and got it right... yea!!
now this code is even better
 

GeneralRe: Reference External StructurememberLoneRanger4 Jul '05 - 11:03 
could you please post the solution to your question?
the referenced message seems to be removed.
 
greetings

GeneralRe: Reference External Structurememberpaulhj29 Dec '05 - 9:30 
Could you please repost the "Using methods from external class" post. It is no longer on the message board. Thanks.
QuestionSecurity?memberKeith Farmer10 Dec '03 - 9:10 
This would make a great way to have script/macro functionality within a program, but to be safe we'd need to be able to restrict the calls to specific namespaces. For example, we could provide a namespace that exposes disk access functionality to a specific directory, blocking root, and expose certain application objects (ie, preferences) while blocking others (a database connection). Giving unfettered access to System.* and Microsoft.* could be dangerous in some cases.
 
I haven't had a chance to check, but is this supported or even feasible under this model?
AnswerRe: Security?memberEric Astor10 Dec '03 - 11:37 
Thanks for your comments...
 
You are certainly correct as to the usefulness of runtime compilation of .NET languages for script/macro functionality. This is both supported and completely feasible under this library, as there are actually no required referenced assemblies when compiling under .NET. It's therefore relatively simple to provide such security by making sure that the only references you create when compiling the String are to assemblies that you want the code to be able to access.
 
Giving access to only a part of an assembly (without creating some sort of proxy type and assembly) may be more difficult, but I suspect it would be possible to do this type of selective screening by parsing the String of code supplied and resolving which namespaces/classes are referenced. This isn't necessarily a simple task, but I believe it's certainly feasible.
GeneralRe: Security?memberKeith Farmer11 Dec '03 - 8:41 
Eric Astor wrote:
It's therefore relatively simple to provide such security by making sure that the only references you create when compiling the String are to assemblies that you want the code to be able to access.
 
So the assembly references are the ones passed to the compiler; the code being compiled has no actual say in the matter?
 
I admit, the IDE has spoiled me in this matter Wink | ;)
 
Thanks -- I'd been looking for something like this, but they always compiled to file rather than memory. I was wondering if in-memory compilation was available.
GeneralRe: Security?memberEric Astor11 Dec '03 - 10:00 
You're right... the code being compiled has no say in its references, only which namespaces it would consider as local (i.e. use/Imports).
 
About in-memory compilation: it's not quite in memory, although it is for most intents and purposes. Behind the scenes of the compiler interfaces, they actually compile to a temporary file (taking care of any complications, thankfully) then load the assembly into memory from that file. Anyway, it's about as close to in-memory compilation as you're likely to be able to get in .NET without using a RAM-disk.
GeneralRe: Security?memberbtodoroff15 Dec '03 - 9:20 
Looking at the example I'm a bit confused. I don't see where the reference to the System.Text namespace is specifically allowed, so I'm having trouble understanding how I would limit access to certain assemblies while allowing references to others.
 
Thanks,
Brian
 
P.S. Great article, has me thinking of all sorts of possibilities.
GeneralRe: Security?memberEric Astor15 Dec '03 - 11:39 
Well, there is a difference between namespace and assembly. System.Text is not an independent assembly... it's a subassembly of System, although it's a separate namespace. Therefore, a reference to System would force the availability of all the subassemblies of System, including Text and IO. If you wanted to block individual subnamespaces, it would be a bit more complicated... It would take something that could parse for the use of forbidden namespaces and reject the code. Essentially, what you'd need is a moderately capable filter layer - not too difficult, at least in concept. I may code up a way to filter specific namespaces, but it depends on the time I have available (which is running short right now).
 
Edit: see next reply
GeneralRe: Security?memberEric Astor15 Dec '03 - 11:43 
Whoops, my mistake... I hadn't realized, but it looks like the System assembly reference actually is included by default. Getting a capable namespace filter in place is now my top priority for this project as soon as I can get some time free to spend.
 
Edit: See next reply
GeneralRe: Security?memberEric Astor30 Dec '03 - 1:59 
Ok, although I haven't coded up that security layer I mentioned, a new article has been posted by someone else dealing with .NET Code Access Security. This is exactly what we're looking for. Essentially, all you'll have to do is add properties to any methods that will call your dynamically-compiled code, restricting what they're allowed to access. As security checks use a stack walk, they'll encounter these methods and refuse the permissions you specify to the dynamically-compiled code.
 
So here's the article I'm talking about:
http://www.codeproject.com/dotnet/UB_CAS_NET.asp[^]
AnswerRe: Security?memberChristopher Lord10 Dec '03 - 17:17 
Look into "Code Access Security" as it relates to .NET.
QuestionExample ?memberstef138 Dec '03 - 21:22 
Eric, I just see your article today and i'm very interressed in, but I can't find your example code. The 3 archives provided are the same. Am I silly (drunk ?) or I missed something ?
AnswerRe: Example ?memberEric Astor9 Dec '03 - 0:55 
Well, it's certainly not your imagination... it appears that there was a mistake with my archives. I'll do my best to get this updated ASAP.
AnswerRe: Example ?memberEric Astor10 Dec '03 - 0:56 
Well, the example code is fixed and in place now... my apologies for the mix-up.
GeneralRe: Example ?memberstef1310 Dec '03 - 4:11 
Thanks a lot for your reactivity Smile | :)
QuestionAssembly unload?membercristianb8 Dec '03 - 10:10 
Does the generated Assembly run in the same AppDomain as caller AppDomain?

I think after the compilation a new Assembly will be loaded.

As far as I know an assembly cannot be unloaded till the .NET AppDomain is not unloaded. Did you thought at this subtle problem?


AnswerRe: Assembly unload?memberEric Astor8 Dec '03 - 11:53 
I understand the problem, and thank you for pointing it out... however, I am already aware of the issue and have a design for the creation of proxies and dynamic proxy classes that should allow the Assembly to be loaded into a new AppDomain and to work across AppDomain boundaries through Remoting. If this works out, with sufficient interest and enough of my own free time, then it may be possible to make the Assemblies optionally unloadable, although using this option would be likely to cause a performance penalty.
AnswerRe: Assembly unload?membercristianb8 Dec '03 - 12:17 

I just read the "TO DO" section. Frown | :(
Generalvb.netmemberDrStein994 Nov '03 - 12:22 
Eric,
 
Great article! Thanks for providing such a powerful piece of code.
 
Unfortunately, I lack c# knowledge, and do not have a c# compiler. I conduct my programming in vb.net
 
I see there is a fully compiled .dll that I can use. Hopefully I can muscle around figuring out the translation.
 
Was wondering if you have any vb.net examples, or any information regarding how to operate this? Thanks.
GeneralRe: vb.netmemberEric Astor4 Nov '03 - 14:53 
The VB.NET usage for the library should be very similar to the C# usage, actually... and I would hope that the documentation (which mirrors the .NET base library's style) would be helpful. But if you'd prefer a sample in VB.NET and capable of working with VB.NET code, I can certainly do that without too much trouble. Give me a few days, and I should have it up.
GeneralRe: vb.netmemberDrStein995 Nov '03 - 6:25 
Eric,
 
Thanks for the prompt response. I spent yesturday trying to translate some code, and I made some progress - but still having trouble.
 
I dont want to put you out of your way. I am very interested in making this work for me. I looked around and didn't seem to find any documents, am I missing something? The XML help is working on my object browser, and I read the entire article on this page.
GeneralRe: vb.netmemberEric Astor5 Nov '03 - 11:08 
What I meant was the XML help, actually... And it won't be a problem to get a VB.NET sample up. I should be able to get it onto the page sometime around this weekend.
GeneralRe: vb.netmemberEric Astor7 Dec '03 - 3:00 
Sorry for the time it took to get the VB.NET sample up and running... Not only was I much busier than I had expected, but it turned out that I actually had left the multiple-language feature out, so I had to integrate a new piece to the design. Anyway, the update should be going through the pipeline at Code Project now, and should be up relatively soon.

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Permalink | Advertise | Privacy | Mobile
Web03 | 2.6.130523.1 | Last Updated 13 Jan 2006
Article Copyright 2002 by Eric Astor
Everything else Copyright © CodeProject, 1999-2013
Terms of Use
Layout: fixed | fluid