Download 900MB File from FTP 'System.OutOfMemoryException' C#

Question

0.00/5 (No votes)

See more:

I am trying to download a 900MB file via ftp. The following code seems to work fine for smaller files but I am recieving an out of memory error when working with this large file. Any help would be appreciated.

WebClient request = new WebClient();
request.Credentials = new NetworkCredential("username", "password");

byte[] fileData = request.DownloadData("ftp://testftp01/downloadfile.exe");

FileStream file = File.Create(@"c:\temp\downloadfile.exe");                
file.Write(fileData, 0, fileData.Length);
file.Close();

Thanks in advance for any help.

Posted 30-Jun-10 7:11am

astroudjr

Updated 30-Jun-10 7:18am

Shog⁹

v3

Add a Solution

2 solutions

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

William Winner · Answer 1 · 2010-06-30T07:30:00

When you try to create a new array with 943,718,400 elements (900 MB), you get that. The array has too many elements.

WebClient ~~(which is what I assume you're using)~~(since your code clearly says that's what you're using!) is for simple situations.

You'll have to switch to WebRequest and use a ResponseStream. Then, you can read it byte by byte.

You could also try the DownloadFile method of WebClient. Since it downloads directly to a file, it might be able to handle larger files.

Sandeep Mewara · Answer 2 · 2010-06-30T07:33:00

Solution 2

By default request size is around 1-2MB. In order to download/transfer request having size more than that needs configuration setting.

Try setting size of request in config file.
Look here: http://msdn.microsoft.com/en-us/library/e1f13641.aspx[^]
http://msdn.microsoft.com/en-us/library/debx8sh9.aspx[^]

Set appropriate MaxRequestLength attribute and try.

Posted 30-Jun-10 7:33am

Sandeep Mewara

Comments

William Winner 30-Jun-10 13:38pm

Reason for my vote of 2
well, first, the second article is about posting data, which is the opposite of what he's trying. Secondly, if it was a problem with MaxRequestLength, the error would have been a ConfigurationErrorsException not an out of memory exception.

Sandeep Mewara 30-Jun-10 13:55pm

I don't agree to your second point. MaxRequestLength issue is not thrown as a ConfigurationErrorsException. Application fails to occupy the space needed in memory as it is not specified properly in Config file. If you see, it's not like the property is missing entirely. It's just that the value is not appropriate and thus no ConfigError exception! System cannot automatically know what config error it is, as it's a logical error and not syntactical error.

Sandeep Mewara 30-Jun-10 13:58pm

Further, alternative could be downloading it in chunks instead of 900MB as a whole that would keep things in control.

William Winner 30-Jun-10 14:08pm

From the MSDN that you pointed to:
maxRequestLength

Optional Int32 attribute.

Specifies the limit for the input stream buffering threshold, in KB. This limit can be used to prevent denial of service attacks that are caused, for example, by users posting large files to the server.

The default is 4096 KB. If the threshold is exceeded, a ConfigurationErrorsException exception is thrown.

Sandeep Mewara 30-Jun-10 14:12pm

Well agreed. But would like to share that we have an upload-download module in our application (a web though), and we dont get any config error when it exeeds. Setting the proper boundary value makes it work fine.

William Winner 30-Jun-10 14:13pm

ok...I was just going off of the documentation.

astroudjr 1-Jul-10 9:45am

So how would I get it in chunks?