Click here to Skip to main content
Click here to Skip to main content

Large attachments/files in ASP WebApi.Problem resolved

, 28 Dec 2012
Rate this:
Please Sign up or sign in to vote.
Article explains how to deal with large files in ASP.NET Web API

Introduction

Recently I have been involved in development of lightweight http service for uploading clinical reports and some additional patient observation data.

Major requirement to this service was to communicate by JSON and allow to transfer a large amount of data (filles can be large than 400 MB). Another one lies in hosting this service on IIS.  

Background 

Basically there are few issues while dealing with large files in ASP.NET WEB Api and IIS. 

First of all it's a limitation imposed on the service by ASP framework by default, and second one related  with having to deal with out of memory issue when allowing third-party clients to upload lagre buffered files.

So let's resolve this issues.  

IIS issue (configuration) 

If you are using Web Hosting within IIS context, you have to deal with maxAllowedContentLength and/or maxRequestLength – depending on the IIS version you are using. 

Unless you explicitly modify web.config , the default IIS 7 upload size will remains the same - about 28 Mb, and as a result if you try to upload file, which is larger than 28 mb, the response will be 403.3 error.

So let's explicitly change the size up to 1 GB:

<system.webServer>
  <security>
    <requestFiltering>
      <requestLimits maxAllowedContentLength="1073741824" />
    </requestFiltering>
  </security>
</system.webServer>   

ASP.NET limitation 

As concerns ASP.NET, it has it own limitation barrier, by default it's set to 4Mb, so lets change it on our flavor: 

<system.web>
  <httpRuntime maxRequestLength="2097152" />
</system.web>   

Out of memory issue 

So many of you might thought that it will be end very soon, but unfortunately not yet, the last thing that we must change is dealing with beffering while transfering large files in WebApi.

The main resoun in hidden behind the curtain, by default ASP.NET WebApi buffers entire request  stream in memory, so as result. Lets pretend  that we load entirely 1 GB of data in Memory.. this is not quite good...

Fortunately there is an easy way to force WEB API into streamed mode.

The major part which is responsible for that in WebApi is IHostBufferPolicySelector (see below) , we need to replace it with our own implementation. fortunately WebApi has a very flexible extensibility part.

public interface IHostBufferPolicySelector
{
   bool UseBufferedInputStream(object hostContext);
   bool UseBufferedOutputStream(HttpResponseMessage response);
}  

And one more, there is a service in WebApi  framework that is responsible for decision of  whether request will be buffered or not- on per basis (System.Web.Http.WebHost.WebHostBufferPolicySelector).

Let me depict some example: 

public class PolicySelectorWithoutBuffer : WebHostBufferPolicySelector
{
   public override bool UseBufferedInputStream(object hostContext)
   {
      var context = hostContext as HttpContextBase;
      if (context != null)
      {
         if (string.Compare(context.Request.RequestContext.RouteData.Values["controller"].ToString(), "uploading", StringComparison.InvariantCultureIgnoreCase)==0)
            return false;
      }
      return true;
   }
   public override bool UseBufferedOutputStream(HttpResponseMessage response)
   {
      return base.UseBufferedOutputStream(response);
   }   

In the above example we check if the request is routed to UploadingController, if so we will not buffer the request.

The last one, we need to register  our service in GlobalConfiguartion , so lets do it Wink | <img src=  

GlobalConfiguration.Configuration.Services.Replace(typeof(IHostBufferPolicySelector), new PolicySelectorWithoutBuffer());   

So what happens under the hood? Well, in the Web API pipeline HttpControllerHandler makes the decision whether to use buffering of the request input stream or not by inspecting the lazily loaded instance of policy selector (a.k.a. our IHostBufferPolicySelector). If the request should not be buffered (like ours), it uses the good old HttpRequest.GetBufferlessInputStream  from the System.Web namespace. As a result the underlying Stream object is returned immediately and we can begin processing the request body before its complete contents have been received. 

Points of Interest 

I this article I'm not go into details about how to upload files to Web API but in my next article I will go into few aspects of it. 

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Oleksandr Kulchytskyi
Software Developer (Senior) Frog (Aricent group)
Ukraine Ukraine
No Biography provided
Follow on   Twitter

Comments and Discussions

 
GeneralMy vote of 5 PinmemberSavalia Manoj M28-Dec-12 17:05 
Good Work...!!
GeneralRe: My vote of 5 PinmemberOleksandr Kulchytskyi28-Dec-12 22:21 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.140721.1 | Last Updated 28 Dec 2012
Article Copyright 2012 by Oleksandr Kulchytskyi
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid