Having IIS performance problems? Do you have w3wp.exe high CPU usage? In this article, we will discuss some tips to identify what is causing high CPU usage for your ASP.NET web applications. There are a lot of reasons that your IIS worker process (w3wp.exe) could be using a lot of CPU. We will cover some of the top reasons and how to troubleshoot IIS performance problems.
To start, you should look at which web requests are currently executing with IIS to see if that helps you identify the issue.
How to View Running Web Requests in IIS
One of the first things you should do is see which web requests are currently executing. This may help you identify a specific URL that is causing the problem. There is a chance that you could recognize one of the URLs that is known to take a very long time or cause high CPU issues.
Although, it may also show a bunch of web requests that are queued up and not directly lead you to the root cause.
Via the IIS User Interface
Via the IIS management console, you can view the running worker processes. You can view which IIS application pool is causing high CPU and view the currently running web requests.
After selecting “Worker Processes” from the main IIS menu, you can see the currently running IIS worker processes.
If you double click on a worker process, you can see all of the currently executing requests. Here is an example from one of our servers. You can see that each request is in different parts of the ASP.NET pipeline and currently executing different HTTP modules.
Via Command Line
The appcmd.exe utility can be useful for a lot of things, including the currently running web requests.
C:\Windows\System32\inetsrv>appcmd list requests REQUEST "f20000048000021c" (url:GET /nopcommerce, time:6312 msec, client:localhost, stage:BeginRequest, module:IIS Web Core)
Understanding The Results & Things to Look For
You can view the currently executing web requests via the IIS user interface or command line. Either way, it returns the same information.
- URL – the complete URL that was being executed
- Time – the total amount of time, in milliseconds, the web request has been executing
- Client – the address of the user that initiated the request
- Stage – the stage of the IIS pipeline that the request is currently in
- Module – the ASP.NET module that is currently executing
There are a few things you want to pay attention to when looking at the requests.
- Are all of the requests for the same URL? Perhaps that URL is the source of the problem
- Is a high number of requests coming from the same client? Perhaps a specific user is slamming your web server with traffic.
- Are all the requests stuck in the same stage or module? There could be a problem with requests getting hung in that specific step of the IIS pipeline.
6 Common Causes of IIS Worker Process High CPU
There are a lot of reasons that you can be seeing w3wp.exe high CPU usage. I have selected five common causes to cover in this article.
- High error rates within your ASP.NET web application
- Increase in web traffic causing high CPU
- Problems with application dependencies
- Garbage collection
- Requests getting blocked or hung somewhere in the ASP.NET pipeline
- Inefficient .NET code that needs to be optimized
1. High Error Rates Within Your ASP.NET Web Application
Application errors could be happening in your application, and you don’t even know it. Some errors will cause your users to receive an error message of some form. Other errors may happen, and nobody knows about it.
There are a few places you can look for application error rates and actual errors. Including Windows Event Viewer, IIS Logs, and more. Read this article to learn more: IIS Logs, Error Logs and More – 6 Ways to Find Failed ASP.NET Requests
Windows Performance Counters for Error Rates
There are two specific performance counters I would recommend reviewing for high error rates. You can check these by opening Performance Monitor within Windows and adding the counters to the chart view.
.NET CLR Exceptions -> # of Exceps Thrown / sec – Check this to see if a lot of exceptions are being thrown by your application. It is possible for your application to have a lot of hidden errors that can cause big performance problems. Any exceptions at all are bad, but some are unavoidable.
W3SVC_W3WP -> % 500 HTTP Response Sent – Any requests with a 500 status code are internal server errors. Make sure this percentage is very low. It should be 0-1%.
2. Increase in Web Traffic Causing High CPU
One of the simplest explanations for w3wp high CPU usage is an increase in web traffic. However, if you don’t have any baseline for what your normal volume of traffic is, it can be had to know if traffic has increased.
If you are using some application monitoring tool that tracks the traffic, be sure to check it and see if traffic levels have changed.
If you don’t have any way to know if traffic levels have changed, you could use your IIS log files to try and find out. You can use VisualLogParser or Log Parser Lizard to query them.
There are also windows performance counters for Requests / Sec and Requests Current to see the current traffic rates in real time.
Possible Reasons for Higher Web Traffic
If traffic levels are up, you should evaluate if it should be or not. Here some things to think about in regards to increased traffic levels:
- Client or user – Is there a huge spike in traffic from a specific client, user, or source? Perhaps something that accesses your site is not working correctly. You may need to block a specific IP address.
- Bots – Similar to a specific user causing a lot of traffic, it could be a bot causing it. Look at the user agents being used to access your site in your IIS logs.
- Oprah effect – Did Oprah or somebody mention your products? Did you just go viral? Getting a lot of attention is awesome, but you may need to scale up to handle it.
If your site is receiving a lot more traffic, you may need to get a bigger server (scale up) or more servers (scale out). If your site doesn’t get very many requests per second, traffic may not be the problem.
Many ASP.NET applications have 10-30 requests per second. However, I have also seen lightweight web requests on busy apps doing over 100 requests per second. The volume of traffic from one web app to another and the related CPU usage varies wildly. It is all about your specific application.
How to Identify Expensive Web Requests
If you are using an APM solution, like Retrace, it can help you track performance over time of your entire application. It can also tell you which specific web requests take the longest.
APM tools use .NET profilers and other data collection techniques to understand the performance of your application down to the code level.
By identifying which web requests are taking the longest, you may be able to identify your high CPU issue or identify parts of your application that could use some performance review to improve.
3. Problems With Application Dependencies
Today’s web applications leverage a lot of different types of external services and dependencies. Including SQL, NoSQL, queuing, caching, and many external HTTP web services.
Slowdowns from any application dependency can cause performance problems with your application.The most common problems are slow SQL queries or problems with external HTTP web services.
Tracking the performance of your application to this level requires an APM tool, like Retrace. Retrace tracks the performance of your application down to the code level. It can help you quickly identify slow SQL queries, web service calls, and much more.
4. Garbage Collection
Microsoft .NET utilizes garbage collection to manage the allocation and release of memory. Depending on what your application does, the allocation and cleanup of application memory can cause a lot of garbage collection activity. For example, the usage of lots of large string variables on the large object heap causes garbage collection problems.
To measure if garbage collection could be causing problems for your application, check this Windows Performance Counter:
.NET CLR Memory -> % Time in GC – This counter reflects what % of the time your application spends doing garbage collection. If this number spikes a lot above 5-10%, you want to investigate memory usage further.
Garbage collection also has two modes. You may need to enable server mode, which is not the default.
5. Requests Are Getting Blocked and Hang in the ASP.NET Pipeline
There are many steps in the lifecycle of an ASP.NET request. This includes basic steps like the start of the request, authentication, authorization, evaluating the correct HTTP handler and finishing the request. As part of this, there are many standard and custom HTTP modules that may be in use.
The first thing I showed you was how to view the currently executing web requests. If you are having performance problems, I would suggest looking to see if all the requests seem to hang on a specific HTTP module.
HTTP modules can be native code or managed .NET code. They can be standard .NET modules like FormsAuthenticaion or SessionStateModule. Many applications also use 3rd party or custom HTTP modules.
Are You Using Sessions?
Most people don’t realize how much of a performance overhead ASP.NET sessions cause. On every page load, they cause your IIS worker process to retrieve your session object, lock it, and then release it at the end of the request.
Sessions cause locking and can cause a bottleneck within your application. If you are using a session provider like Redis or SQL, any performance slowdowns will impact the performance of your application.
6. Inefficient .NET Code That Needs to Be Optimized
If none of the other common causes was a problem, you might need to dig into the code to make improvements. You will need to use .NET code profilers to capture what methods are being called in your code and how long they take. Profilers can help you identify specific methods in your code that are very slow.
Warning: Profilers do add overhead to your application. So if your application already has very high CPU usage (90+%), profiling will be very slow, and the performance of your application will get worse. This could make your application unusable even depending on the detail level of profiling you are doing and the CPU usage before you started profiling.
My favorite .NET profiler is RedGate’s ANTS Performance Profiler. They do offer a free trial that you can take advantage of.
Check out these articles to learn more about ANTS and how to profile ASP.NET applications.
Good Luck With Your IIS Performance Problems!
Hopefully, you are reading this because you are trying to figure out how to improve performance. I hope your server isn’t at 100% and you are scrambling!
As a provider of APM solutions, we spend a lot of time working on and thinking about application performance problems. We hope that this guide has been helpful.
- 6 Reasons Cloud Monitoring Is Different Than Server Monitoring - May 25, 2017
- How DevOps Increases Security, Not Hurts It - May 18, 2017
- What Is Function-as-a-Service? Serverless Architectures Are Here! - May 15, 2017
- Microsoft Build 2017 Review of Day Two – Windows News & Updates - May 11, 2017
- Microsoft Build 2017 Review of Day One – Azure News & Updates - May 10, 2017