string memory allocation (urgent help needed)

Question

1.00/5 (2 votes)

See more:

.NET2.0

Hi,

I am reading a large file upto 10-20 MB in a string in 1 go in below line

string[] lines = System.IO.File.ReadAllLines(file_name);

I save each line to database , and dispose the connection.

Now when i again do this operation , the windows app crashes with out of memory exception.

I see during first read operation memory increases and is not freed when operation completes.

How can i free the memory allocated to this string.
Thanks,
Shreyas

Posted 23-Sep-13 3:47am

srastogi85

Add a Solution

2 solutions

Solution 1

Keep your connection open and just change your command object for inserting, if you have at least 2Gb memory on your system then a 10-20Mb file read in full is nothing.

Posted 23-Sep-13 3:59am

Mehdi Gholam

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Sergey Alexandrovich Kryukov · Accepted Answer · 2013-09-23T04:00:00

Solution 2

No, you cannot. You don't have control over the memory deallocation. It is done by the Garbage Collector at some moment of time after some object become unreachable, and you don't control when exactly it happens. Please see:
http://en.wikipedia.org/wiki/Garbage_collection_%28computer_science%29[^].

Perhaps the file you read with ReadAllLines is just too big. Instead, you can use the class System.IO.StreamReader and read the data line by line using System.IO.StreamReader.ReadLine:
http://msdn.microsoft.com/en-us/library/system.io.streamreader.aspx[^],
http://msdn.microsoft.com/en-us/library/system.io.streamreader.readline.aspx[^].

It would help you to divide a file by smaller chunks and do something with those chunks one at a time.

—SA

Posted 23-Sep-13 4:00am

Sergey Alexandrovich Kryukov

Comments

srastogi85 23-Sep-13 10:14am

Thanks Sergey,
I am planning to modify the code based on below lines , by having using clause and streamreader
, i think this might solve the issue for big files. Please suggest you views for this ?

using (StreamReader sr = new StreamReader("TestFile.txt"))
{
string line;
// Read and display lines from the file until the end of
// the file is reached.
while ((line = sr.ReadLine()) != null)
{
Console.WriteLine(line);
}
}
}

Sergey Alexandrovich Kryukov 23-Sep-13 10:18am

That is the right usage. What will you do with the data read is your business.
Will you accept my answer formally now?
—SA

srastogi85 23-Sep-13 10:20am

yes sure sir :) thanks a lot

Sergey Alexandrovich Kryukov 23-Sep-13 11:06am

You are very welcome.
—SA

srastogi85 23-Sep-13 10:26am

This still didn't solved the problem

Sergey Alexandrovich Kryukov 23-Sep-13 11:07am

Then you need to explain what have you done and what happened.
—SA