Click here to Skip to main content
15,798,200 members
Please Sign up or sign in to vote.
1.00/5 (1 vote)
See more:
I wanted to know why i get a OutOfMemory Exception in the following:
basicly I have a textbox/string with ~ 90.000.000 characters (or more)
And what I want do is writing them to a file..

BinaryWriter writer = new BinaryWriter(new FileStream(AppDomain.CurrentDomain.BaseDirectory + "test.txt", FileMode.Create, FileAccess.Write));

using (StreamWriter outfile = new StreamWriter(AppDomain.CurrentDomain.BaseDirectory + "test.txt"))

So on this way I get the memory exceptions.

If I try to reduce the size by splitting the main string into
subs ..

if (textBox1.TextLength >= 28000000)
    int step_size = 28000000;
    int step_count = textBox1.TextLength / step_size;

    int cur_begin = 0;
    int cur_end = step_size;

    textBox4.AppendText("TextBoxCharacterCount: " + textBox1.TextLength.ToString() + Environment.NewLine);
    for (int i = 0; i < step_count; i++)

        string sub = textBox1.Text.Substring(cur_begin, cur_end);
        if (i == 0)

            BinaryWriter writer = new BinaryWriter(new FileStream(AppDomain.CurrentDomain.BaseDirectory + textBox2.Text + ".h", FileMode.Create, FileAccess.Write));
            BinaryWriter writer = new BinaryWriter(new FileStream(AppDomain.CurrentDomain.BaseDirectory + textBox2.Text + ".h", FileMode.Open, FileAccess.Write));

        cur_begin = cur_end + 1;
        cur_end = cur_end + step_size - 1;

    string end = textBox1.Text.Substring(cur_begin, textBox1.TextLength);
    BinaryWriter end_writer = new BinaryWriter(new FileStream(AppDomain.CurrentDomain.BaseDirectory + textBox2.Text + ".h", FileMode.Open, FileAccess.Write));
    BinaryWriter writer = new BinaryWriter(new FileStream(AppDomain.CurrentDomain.BaseDirectory + textBox2.Text + ".h", FileMode.Create, FileAccess.Write));

I still get the memory exception at sub this time.
But why ? a string can contain 2.147.483.647 characters in c#
and the current size would be far below, means ~ 28.000.000

And the most important fact is, that if i want write a string with
~ 29.000.000 with the very first method, it works , but if i try to split
90.000.000 into 28.000.000 parts it doesnt work at all ...


What I have tried:

destribes above in the related question :)
Sergey Alexandrovich Kryukov 15-Feb-16 17:48pm    
So what? Eventually, you face some data size which is not suitable for available memory. Isn't it a 32-bit application?
You may think of not reading the whole file at once, using a database, and so on...
GAFO 15-Feb-16 18:01pm    
well, i need to read a whole file since i need to convert a .xm file into a byte array -
works fine until the size of the array goes above the 29.000.000 limit - and i am not as much into c# , since im a c++ coder
Sascha Lefèvre 15-Feb-16 18:34pm    
Any file already is a byte array. Either you have a misconception here or you didn't express yourself clearly.
GAFO 15-Feb-16 18:39pm    
BinaryReader reader = new BinaryReader(new FileStream(openFileDialog1.FileName, FileMode.Open, FileAccess.Read));
byte[] ImageData = reader.ReadBytes((int)reader.BaseStream.Length);

string hex = BitConverter.ToString(ImageData).Replace("-", ", 0x");
int count = hex.Count(f => f == 'x');
count = count + 1;

To make myself clear, this is reading all the bytes of an file into an byte array, then it get moved into an string and - replaced with the hex style of e.g. 0xA3 , this string is way too long for writing as it seems, even if c# should manage the memory on its own compared to c++ , Hope its not clear, the lenght of this string would be around 90.000.000 characters within the string. (or much more) At the end this string has to be written into a file, thats basicly all which should be done

First, why are you using BinaryWriter to write a text file?

It's a lot less code to just do this and end up treating the files as text files:
File.WriteAllText("someFilePath.h", TextBox1.Text);
Share this answer
GAFO 15-Feb-16 17:59pm    
I can tell you why -> its too big as it seems
Dave Kreskowiak 15-Feb-16 18:01pm    
Using BinaryWriter is not going to solve this at all.

On which line does the exception throw?
GAFO 15-Feb-16 18:03pm    
the exception is directly at the line of File.WriteAllText :)
isnt there a way to write large (realy huge) strings into a file without oom ?
Sascha Lefèvre 15-Feb-16 18:22pm    
Did you compile your project for x86? If yes, give "Any CPU" a try (which will result in x64 if you have a 64bit CPU).
GAFO 15-Feb-16 18:26pm    
it is at "Any CPU" and I thought that
c# is supposed to manage its memory automaticly, thatswhy Im using it in this case , normaly there shouldnt be any outofmemory error O.o
After clarifying the ultimate purpose of this as per the comments above:

There's no need to read the whole file at once. You can read and convert it in chunks.

This should do what you want; I tested it with a 100MB file:
string inputFile = @"...";
string outputFile = inputFile + ".txt";
int bufferSize = 50000;

using (var reader = new BinaryReader(new FileStream(inputFile, FileMode.Open, FileAccess.Read)))
    if (reader.BaseStream.Length > 0)
        using (var writer = new StreamWriter(outputFile, append: false, encoding: Encoding.ASCII))

            byte[] buffer = new byte[bufferSize];

            while (true)
                int remainingBytes = (int)(reader.BaseStream.Length - reader.BaseStream.Position);
                int bytesToRead = Math.Min(bufferSize, remainingBytes);

                if (bytesToRead == 0)

                if (bytesToRead == bufferSize)
                    reader.Read(buffer, 0, bytesToRead);
                    buffer = reader.ReadBytes(bytesToRead);

                string hex = BitConverter.ToString(buffer).Replace("-", ", 0x");
                writer.Write(", 0x");
Share this answer
GAFO 15-Feb-16 20:23pm    
oooh thank you very much :)
worked very fine and so fast in comparison to before. Rated full stars
Sascha Lefèvre 16-Feb-16 6:37am    
You're welcome! :)
You should loop with much smaller block than that. Something between 8KB and 64 KB might be adequate.

Also, you should not stream and writer in each iteration. This is very basic code optimization. This is not premature optimization but just a good habit to write efficient code.

Also, you should use using statement when creating an object that is disposable. Your code is not exception safe!

Exception safety in C# – more than just trying and catching[^]

Then you have off-by one errors. In some case, your code won't write last partial block.

Also also code duplication. Have you never heard of: Don't repeat yourself - Wikipedia, the free encyclopedia[^]

So you should not repeat code that write data (3 places) and also not repeat code that compute target file name. This code is poorly reusable. Cut and paste programming is a very bad habit.

Finally, you are not given proper argument to string.Substring(Int32, Int32). If would be a good idea to read document or at least Intellisense information while writing code.

String.Substring Method (Int32, Int32) (System)[^]

Solution 2 code would be a good starting point for the structure although I'm not sure that he write data as you expect it.
Share this answer
GAFO 16-Feb-16 14:06pm    
thank you for providing this suggestion, and Solution 2 was working very nice :)
and about the reuseability of the code - its a quick tool for a simple purpose, means
the structuring etc is not important in this case. Ofc I could have made classes etc to manage every single part, but that would be too much for this case ;) anyways the information in case of error handling was good to know for bigger projects in the future in case of c# and .Net

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900