More than 5000... - C# Discussion Boards

2-Jul-07 9:25

I let this run for an hour to get to 5000 before I gave up. Someone with more CPU and physical RAM than I have should run it and see where it ends...

using System;
using System.Collections.Generic;
using System.Text;
using System.CodeDom.Compiler;
   
namespace MethodCountLimitFinder {
   class Program {
      static void Main(string[] args) {
         Int32 methodCount = 1;
         Microsoft.CSharp.CSharpCodeProvider cscp = new Microsoft.CSharp.CSharpCodeProvider();
         ICodeCompiler icc = cscp.CreateCompiler();
   
         CompilerParameters cp = new CompilerParameters();
         cp.GenerateExecutable = false;
         cp.GenerateInMemory = true;
   
         CompilerResults cr = null;
         string pre = "using System;" + Environment.NewLine +
                  Environment.NewLine +
                  "namespace Tester {" + Environment.NewLine +
                  "   class Test {" + Environment.NewLine;
         string post = "   }" + Environment.NewLine +
                  "}";
         string inner = string.Empty;
   
         while (true) {
            inner += "      public Int32 Method" + methodCount.ToString() + "() {" + Environment.NewLine +
                     "         return 42;" + Environment.NewLine +
                     "      }" + Environment.NewLine;
   
            cr = icc.CompileAssemblyFromSource(cp, pre + inner + post);
   
            if (cr.Errors.Count > 0)
               break;
   
            methodCount++;
   
            if (methodCount % 10 == 0)
               System.Console.WriteLine(methodCount.ToString());
         }
   
         System.Console.WriteLine(methodCount);
      }
   }
}

Share and enjoy

Sean

DavidNohejl2-Jul-07 9:36

DavidNohejl

2-Jul-07 9:36

Sean Michael Murphy wrote:
while (true) {
inner += " public Int32 Method" + methodCount.ToString() + "() {" + Environment.NewLine +
" return 42;" + Environment.NewLine +
" }" + Environment.NewLine;

cr = icc.CompileAssemblyFromSource(cp, pre + inner + post);

if (cr.Errors.Count > 0)
break;

methodCount++;

if (methodCount % 10 == 0)
System.Console.WriteLine(methodCount.ToString());
}

Sean Michael Murphy wrote:
Someone with more CPU and physical RAM than I have should run it and see where it ends...

No wonder, always use StringBuilder for string concatenation in a loop.

"Throughout human history, we have been dependent on machines to survive. Fate, it seems, is not without a sense of irony. " - Morpheus
"Real men use mspaint for writing code and notepad for designing graphics." - Anna-Jayne Metcalfe

Martin#2-Jul-07 10:03

Martin#

2-Jul-07 10:03

Cannot been said to often!

Good answere!

PIEBALDconsult2-Jul-07 11:56

2-Jul-07 11:56

Well it can be said too often, but it's appropriate here.

Sean Michael Murphy2-Jul-07 15:28

2-Jul-07 15:28

dnh wrote:
No wonder, always use StringBuilder for string concatenation in a loop.

Hmmm. Interesting. When I originally undertook to code this snippet to try to figure an answer to this guys question, optimization was pretty far from my mind. I mean, I cranked the original bit of code out in 15 minutes (or so) and had originally coded it so the methods would be recreated every time. I took another 5 minutes and optimized it so that only 1 method (the new one) would have to be concatenated to the "guts", which was then stuck in between the fixed "header" and "footer" of the class.

It ran slowly, but I assumed that most of the overhead was in the actual code compilation (compiling classes of 15000 lines), and not a little bit of string concatenation. So I've re-written it using StringBuilder and timed both versions for 500 iterations. The original code did 500 iterations on my PC in 161.5222 seconds. This version:

StringBuilder inner = new StringBuilder();
 
DateTime startTime = DateTime.Now;
   
for (Int32 i = 0; i < 500; i++) {
   inner.Append("      public Int32 Method" + methodCount.ToString() + "() {" + Environment.NewLine +
                "         return 42;" + Environment.NewLine +
                "      }" + Environment.NewLine);
 
   StringBuilder code = new StringBuilder(pre);
   code.Append(inner);
   code.Append(post);
   cr = icc.CompileAssemblyFromSource(cp, code.ToString());
 
   if (cr.Errors.Count > 0)
      break;
 
   methodCount++;
 
   if (methodCount % 10 == 0)
      System.Console.WriteLine(methodCount.ToString());
}
 
TimeSpan ts = DateTime.Now - startTime;
 
System.Console.WriteLine(ts.TotalSeconds);

did it in 160.111. Much less that 1% slower.

Not a string concatenation to be found, except for the line joins.

Anything to add?

Thanks.
Sean

DavidNohejl2-Jul-07 21:54

DavidNohejl

2-Jul-07 21:54

Sean Michael Murphy wrote:
Anything to add?

I'd agree that most time takes compilation, but the thing about string concatenation with + is that it's -unlike compilation - completely unnecessary. And I don't think that using StringBuilder for concatenating strings in big loops is optimalization - I think it's something you should do without thinking.

btw you're still allocating 7 or so strings in

inner.Append("      public Int32 Method" + methodCount.ToString() + "() {" + Environment.NewLine +
                "         return 42;" + Environment.NewLine +
                "      }" + Environment.NewLine);

every cycle, that's 3500 unnecessary allocations Smile | :)

Anyway, cool way to check for number of methods limit indeed.

PIEBALDconsult3-Jul-07 7:14

3-Jul-07 7:14

dnh wrote:
I think it's something you should do without thinking.

Never do anything without thinking.

DavidNohejl3-Jul-07 7:41

DavidNohejl

3-Jul-07 7:41

I'll repeat: *Always* use string builder for concatenating strings in big loops. And I stay behind my claim.

That being said, if that loop had about 5 iterations in 99,99% and much more in 0,01%, then you have to thinkg about it - IIRC StringBuilder would be slower. But if that task is something that must end in some very limited time or under very limited memory, you can't afford that 0,01% and even if performing worse in average, StringBuilder would be better choice.

Re: More than 100000...

vytheese2-Jul-07 23:16

vytheese

2-Jul-07 23:16

Hi,

<br />
            while( true)<br />
            {<br />
                do<br />
                {<br />
                    inner.Append("      public Int32 Method" + methodCount.ToString() + "() {" + Environment.NewLine + "         return 42;" + Environment.NewLine + "      }" + Environment.NewLine);<br />
                    methodCount++;<br />
                } while ((methodCount % 1000) != 0);    <br />
                <br />
                cr = icc.CompileAssemblyFromSource(cp, pre + inner.ToString() + post); <br />
                <br />
                if (cr.Errors.Count > 0)   <br />
                    break; <br />
                <br />
               System.Console.WriteLine(methodCount.ToString() + " Compiled successfuly ==> so not succed");<br />
            } <br />
            System.Console.WriteLine(methodCount + " may be approximately to -1000 of method count");<br />
<br />

I modified slightly your code as the above and executed, Its going on till 100000 ( above 1 lakh ), My machine got down, So I planned to run today night.

Now I feeling, I shouldn't ask this question first of all Wink | ;)

Thanks,
Vythees

-- modified at 5:24 Tuesday 3rd July, 2007

Thanks,
Vythees

Re: More than 100000...

Sean Michael Murphy3-Jul-07 5:01

3-Jul-07 5:01

vytheeswaran wrote:
Now I feeling, I shouldn't ask this question first of all

Don't be crazy. I enjoyed thinking about it.

Sean

PIEBALDconsult3-Jul-07 7:37

More than 1000000 [modified]

3-Jul-07 7:37

A) I don't think there's any need for including the NewLines.

B) Why step by one? Why not double methodCount after each successful compile?

PIEBALDconsult3-Jul-07 11:50

3-Jul-07 11:50

In reference to A: error CS1034: Compiler limit exceeded: Line cannot exceed 16777214 characters

The following succeeds at 1000000, and then I killed it while it tried 2000000.

namespace MethodCountLimitFinder
{
    class Program
    {
        [System.STAThreadAttribute]
        static void Main ( string [] args )
        {
            Microsoft.CSharp.CSharpCodeProvider provider = 
                new Microsoft.CSharp.CSharpCodeProvider() ;

            System.CodeDom.Compiler.CompilerParameters cp = 
                new System.CodeDom.Compiler.CompilerParameters() ;
            cp.GenerateExecutable = false ;
            cp.GenerateInMemory = true ;

            System.CodeDom.Compiler.CompilerResults cr = null ;

            System.Text.StringBuilder inner = 
                new System.Text.StringBuilder ( "namespace Tester { class Test {" ) ;

            int methodCount = 1000000 ;

            while ( true )
            {
                System.Console.WriteLine ( methodCount ) ;
                
                for ( int i = methodCount ; i > 0 ; i-- )
                {
                    inner.AppendFormat ( "void M{0}(){{}}\n" , methodCount++ ) ;
                }
                
                inner.Append ( "}}" ) ;
                
                cr = provider.CompileAssemblyFromSource ( cp , inner.ToString() ) ;
                
                if ( cr.Errors.Count > 0 )
                {
                    break ;
                }
                
                inner.Remove ( inner.Length - 2 , 2 ) ;
            }

            foreach (  System.CodeDom.Compiler.CompilerError ce in cr.Errors )
            {
                System.Console.WriteLine ( ce.ToString() ) ;
            }
        }
    }
}

-- modified at 21:11 Tuesday 3rd July, 2007

2000000 and counting...

C:\>maxi
1000000
2000000
4000000
error CS0001: Internal compiler error (0x80004005)
error CS0001: Internal compiler error (0xc0000017)
error CS0583: Internal Compiler Error (0xc0000005 at address 5A16E208): likely culprit is 'PARSE'.
error CS0586: Internal Compiler Error: stage 'PARSE'
error CS0587: Internal Compiler Error: stage 'PARSE'
error CS0587: Internal Compiler Error: stage 'BEGIN'

C:\>

-- modified at 1:56 Wednesday 4th July, 2007

After 3000000 I started hitting resource limits and timeouts. So now I simply have a program write a file with the code and compile it at the command line, that got me to 4000000 so far...

Sean Michael Murphy4-Jul-07 18:36