WCF First Call Slow

WCF Slow to Start

A week or so ago we got reports that our production web application was slow. With further testing, we realized that application's performance slowdown was caused by one particular WCF call taking 15 - 30 seconds. The slowness was not consistent either! Sometimes it was less than a second, and the next call it would hit 30 seconds! We think we have the problem fixed - it was a combination of IIS host service startup costs, and attempting to hit a local proxy on first call.

Researching the issue lead me to realize this type of issue is widespread, so here are some tips for others dealing with the same issue:


This first thing you can do is setup the service trace viewer on your web server by modifying the web.config or app.config of your service host. Add full tracing, test your service, and review the resulting log files to see where the slowdown is occuring. Too many times I have been burned by assuming I knew where the peformance bottleneck is and fixing non-issues. So measure twice before you start trying to fix a performance problem.

Service trace viewer is included with the MS Windows SDK. Once you have it, you can easily copy the executable wherever you need it.
http://msdn.microsoft.com/en-us/library/ms732023.aspx[^]

Tip - sort the WCF calls in Service Trace Viewer by call time. Then you can easily spot the bottlenecks.

IIS Startup Time

1) If you are hosting your WCF service in IIS, there is a high cost for the first call. On the first call to your WCF service, IIS must compile your service. Then IIS must construct and start the service host.

Furthermore, IIS automatically stops services that are idle for a certain period of time. The default idle timeout is 20 minutes. Keep in mind if you have a load balancer, then each server will timeout independently, which can cause strange behavior as you jump between servers.

FIXES:

1) I have seen a couple of references to this warmup template which basically hits your server to cause IIS to compile and start your service. Cannot say that I have tried it: http://blogs.msdn.com/b/joelo/archive/2006/08/13/697044.aspx[^]
2) IIS has an idle timeout. So once your service has been hit once, and is running, if it doesn't have activity for some period of time, IIS will automatically stop it. You can alter this timeout, to make your service less likely to idle, or turn off the idle timeout feature entirely:
http://technet.microsoft.com/en-us/library/cc771956(WS.10).aspx[^]

also
http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/83b35271-c93c-49f4-b923-7fdca6fae1cf.mspx?mfr=true[^]

Do not use default HTTP proxy

Set useDefaultProxy="false" in your client http binding to tell WCF not to use the default proxy on your client computer:
http://msdn.microsoft.com/en-us/library/system.servicemodel.basichttpbinding.usedefaultwebproxy.aspx[^]
When you have useDefaultProxy="true" you are telling WCF to pick the proxy settings from IE.

Use XML Serializer

You can improve the startup times of a WCF Client by generating XmlSerializer serialization code:
http://msdn.microsoft.com/en-us/library/aa751883.aspx[^]

Netbios on TCP

There are repeated references to high costs for initializing app domains where the host server has been configured to use netbios over TCP. E.g. http://www.netframeworkdev.com/windows-communication-foundation/calling-wcf-service-hosted-in-iis-6-is-slow-66240.shtml[^] The resolution is to disable Netbios over tcp:
http://www.petri.co.il/disable_netbios_in_w2k_xp_2003.htm[^]

Publisher Evidence - Network Call

When IIS initializes an application pool, it may verify that an assembly is legitimate using publisher evidence. According to the docs, this is no longer enabled by default in .net 4.
http://msdn.microsoft.com/en-us/library/bb629393.aspx[^]

WCF Security

Make sure your WCF client security settings are consistent with your service. Do not enable security on your client that is not used by your service.