Slow .Net performance in IIS when memory hungry app on multi core processor (>=8)

Case:

.Net actions uses lot of memory (50+mb) cause to run very slow in IIS

Symptoms:

.Net application standalone is fast, in IIS running 10 times slower on a multi core processor (>=8)

Garbage collector is running all the time 40% (Performance monitor .Net CLR Memory -> %Time in GC)

From How GC works: http://msdn.microsoft.com/en-us/library/ms973838.aspx

Choosing Which Garbage Collector to Use

The CLR has two different GCs: Workstation (mscorwks.dll) and Server (mscorsvr.dll). When running in Workstation mode, latency is more of a concern than space or efficiency. A server with multiple processors and clients connected over a network can afford some latency, but throughput is now a top priority. Rather than shoehorn both of these scenarios into a single GC scheme, Microsoft has included two garbage collectors that are tailored to each situation.

Server GC:

• Multiprocessor (MP) Scalable, Parallel
• One GC thread per CPU
• Program paused during marking

Workstation GC:

• Minimizes pauses by running concurrently during full collections

The server GC is designed for maximum throughput, and scales with very high performance. Memory fragmentation on servers is a much more severe problem than on workstations, making garbage collection an attractive proposition. In a uniprocessor scenario, both collectors work the same way: workstation mode, without concurrent collection. On an MP machine, the Workstation GC uses the second processor to run the collection concurrently, minimizing delays while diminishing throughput. The Server GC uses multiple heaps and collection threads to maximize throughput and scale better.

http://msdn.microsoft.com/en-us/library/ee851764.aspx#troubleshooting_guidelines

Workstation or Server Garbage Collection

Determine if you are using the correct type of garbage collection. If your application uses multiple threads and object instances, use server garbage collection instead of workstation garbage collection. Server garbage collection operates on multiple threads, whereas workstation garbage collection requires multiple instances of an application to run their own garbage collection threads and compete for CPU time.

An application that has a low load and that performs tasks infrequently in the background, such as a service, could use workstation garbage collection with concurrent garbage collection disabled.

http://msdn.microsoft.com/en-us/library/ee787088.aspx

how server and workstation gc work
Workstation garbage collection can be concurrent or non-concurrent. Concurrent garbage collection enables managed threads to continue operations during a garbage collection.
If you specify server garbage collection, the CLR uses workstation garbage collection with concurrency disabled.

• Server garbage collection can be resource-intensive. For example, if you have 12 processes running on a computer that has 4 processors, there will be 48 dedicated garbage collection threads if they are all using server garbage collection. In a high memory load situation, if all the processes start doing garbage collection, the garbage collector will have 48 threads to schedule.

If you are running hundreds of instances of an application, consider using workstation garbage collection with concurrent garbage collection disabled. This will result in less context switching, which can improve performance.

Workstation Concurrent garbage collection enables interactive applications to be more responsive by minimizing pauses for a collection. Managed threads can continue to run most of the time while the concurrent garbage collection thread is running. This results in shorter pauses while a garbage collection is occurring.

To improve performance when several processes are running, disable concurrent garbage collection.

Solution:

http://support.microsoft.com/kb/911716

Aspnet.config

<configuration>

<runtime>

<gcServer enabled="false"/><!-- false is workstation Gc-->

</runtime>

</configuration>

Considerations: gcServer is configured for throughput, what is the effect on processes that do not use a lot of memory. Is there a performance difference in concurrent call?