Throttling. Why is application not using all available processing power.

Question

5.00/5 (1 vote)

See more:

I have A LOT of calculations to perform so I wrote a quick program to do them for me.

For some reason when I can't seem to use all of my CPU, it seems like the system is throttling it. I wrote a simple application to demonstrate this behavior. See the below code:

C#

Parallel.For(0, 10000, x =>
{
    int i = 0;
    while (true)
    {
        i++;
    }
});

Running the above code in a console app, it maxes out all of my processors and my CPU utilization is at 100% (as I would expect).

In my calculation, I am creating several Lists and arrays so I added that to the above code and that is when I saw it behave just as my actual program was (where it would use less than 100% of the CPU). See example code below:

C#

Parallel.For(0, 10000, x =>
{
    int i = 0;
    while (true)
    {
        i++;
        List<int> vs = new List<int>();
    }
});

On my computer, which has 12 logical processors, the CPU utilization only reaches 70-80%. When running on a higher-performance PC I have, which has 32 logical processors, the behavior is even more dramatic - the highest the CPU utilization goes is 50%. And my real application that is performing the calculations only uses 20% of the total CPUs.

And as a follow-up, in Task Manager I see that it is using all logical cores more or less evenly - it isn't maxing out some and others sitting idle.

I am curious why this is happening, and more importantly, how can I make my program use all of the processor?

What I have tried:

I tried monitoring garbage collection but that didn't offer any clues (and also wouldn't make sense because I would expect garbage collection to show up as CPU usage).

I tried increasing the process priority from normal to "High" and even "Real-Time" but that didn't change anything.

Posted 12-Nov-21 6:53am

MrGlass3

Updated 12-Nov-21 10:36am

Add a Solution

Comments

[no name] 12-Nov-21 13:10pm

Maybe it's a problem with the "performance monitor"; how is anyone supposed to know? Or care?

MrGlass3 12-Nov-21 13:15pm

When I run in with Visual Studio debugger attached the CPU utilization matches what is shown in the Task Manager.
And I care because I would much prefer to use near 100% of my CPU instead of 20%. It is literally the difference between the calculations taking 5 days vs over a month to complete.

PIEBALDconsult 12-Nov-21 13:13pm

Allocating RAM is not CPU-intensive maybe?
Avoid the allocations if possible.
Use allocate-and-clear instead.

MrGlass3 12-Nov-21 13:21pm

Something relating to memory allocation is all I could come up with (which seems wrong...I have always thought that was ignorable fast?). But I would've thought that would be a system as a whole bottleneck if that were the case. But I can launch multiple instances of this application and it eventually will begin to use 100% of the CPU. If allocation were a bottleneck I would expect multiple instances would slow down the already running processes because they would have to be waiting for their turn to allocate memory?

PIEBALDconsult 12-Nov-21 13:32pm

It depends on several factors of course.
Allocation is definitely not "ignorable fast".
And the larger the space required, the more effort is required to do it. And if you do it, then throw it all away, just to do it again, is very wasteful.
What size Lists are we talking about? They begin (I think) at sixteen slots, then double their allocation as required. Just knowing what capacity you need up front can save a lot of time/effort. But definitely try no to throw away a List (or similar, including StringBuilder) just to make another. Empty it and reuse it.

MrGlass3 12-Nov-21 13:39pm

During the allocation, isn't it the CPU that is performing that allocation? I understand declaring new lists constantly slowing down how fast calculations can be performed but isn't it the processor that is allocating the memory? Would that show up as CPU usage?

In my actual application, the list lengths are less than 100. In the above code, they are obviously empty so just the default size, which is dropping my CPU to half.

PIEBALDconsult 12-Nov-21 14:35pm

If you have one hundred items...
If you construct a "default" List, it should allocate capacity for sixteen items (or references to the items).
When you add a seventeenth item, it will allocate room for thirty-two items -- and copy the existing sixteen items over.
The same at thirty-three and again at sixty-five.
This is a waste of time/effort if you already know you need a hundred.
So using new List<int>( 100 ) can save a bit.

MrGlass3 12-Nov-21 14:44pm

I appreciate the help in making the code be more efficient - and I'm sure there are several other bits of code that could be written in a more efficient way so that it can be run faster. But the bigger concern is how to fully utilize the processing power that is available - even if there is inefficient code in there that is decreasing the speed of the calculations by 50%, the fact that only 20% of the processing power is being used means there is a 400% increase in processing capacity left on the table because the application is somehow being throttled?

PIEBALDconsult 12-Nov-21 15:00pm

I disagree with your conclusion.
Is the code doing any IO?

MrGlass3 12-Nov-21 15:16pm

No IO of significance. Calculations are performed, the results saved to a list then every 10 minutes a new task is created saving the results to a csv.

If we back things up and focus on the example code where a new list is created every loop and the CPU utilization drops to 50%. I'm with you in that the only thing that I can think of is memory allocation, but I'm confused about what is happening during that. Isn't the CPU working to allocate that? And if not, then whatever is responsible for allocating the memory clearly can't keep up with the processor's demands. But then how can I launch a second instance of the application and the first instance is able to loop through just as fast - clearly if there is something else that is allocating the memory it can do it at least twice as much as it was.

PIEBALDconsult 12-Nov-21 15:15pm

You now have me wondering...
In .net, memory allocation/deallocation is performed by the Garbage Collector. Does that CPU usage get charged to a separate GC process? Or the process which requested the memory?

MrGlass3 12-Nov-21 15:21pm

hmmm. At first I was thinking that at least could make sense (which as much as I want a solution, I will settle for something that makes sense at this point lol). But even if the GC were being attributed to a different process, that process would still show up in the CPU utilization in the Task Manager and there should be a process utilization that is going through the roof...and launching a second instance of the application should slow the first one down.

ughh I was excited you could be on to something!

Dave Kreskowiak 12-Nov-21 16:07pm

It's not "ignorable fast" because the memory, under .NET, has to come from the managed heap. If there isn't enough available, it has to request a block of memory from Windows, add that block to the heap, then do your allocation. There's there's garbage collection overhead in some cases, and memory management by Windows, heap fragmentation, ...

I think what you're not seeing is the CPU use while execution is in the kernel or in .NET CLR.

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Dave Kreskowiak · Answer 1 · 2021-11-12T10:36:00

Solution 1

OK, so you want to see your CPU beg for mercy using a Parallel.For?

Here you go:

C#

public void TestMethod()
{
    ParallelLoopResult result = Parallel.For(0, 50, x =>
    {
        Random random = new Random();
        double result = 1.0;

        for (int i = 1; i < 1000000000; i++)
        {
            result += random.NextDouble();
        }

        Console.WriteLine($"{result}");
    });
}

Posted 12-Nov-21 10:36am

Dave Kreskowiak

Comments

MrGlass3 12-Nov-21 16:45pm

I don't want to use 100% of the processor just to see it scream. I have millions of calculations to perform that and want them to be run as fast as possible. Apparently, it is sitting idle 80% of the time and I want to make it run at or close to 100% so it finishes in days instead of weeks.

The example code I wrote was just for a very simplistic demonstration of what I am observing.

Dave Kreskowiak 12-Nov-21 16:49pm

OK, so what you are seeing is the CPU time of YOUR CODE, not where execution of your code has wandered into .NET CLR or Windows kernel. If you're doing a lot of memory alloc/free, that CPU time isn't going to show up in the CPU usage of your process.

MrGlass3 12-Nov-21 16:52pm

Whether it is my code or some underlying windows process related to memory allocation, wouldn't it show up in Task Manager CPU usage? I'm not just looking at the CPU usage of my process, I'm looking at the CPU usage of the whole system.

And the other thing that I can't get to make sense, is I can launch a second instance of the application and they both run just as fast as if there was only one running so the second instance isn't slowing the first one down at all.

Dave Kreskowiak 12-Nov-21 16:59pm

No. At the bottom of the Performance tab in Task Manager window, click the Open Resource Monitor link.

Click on the CPU line in the window that pops up. On the right at the tip, you'll see the CPU usage graph with 2 lines in it. The blue graph is user mode CPU usage, while the green one is kernel.

Your code will run as fast as Windows will allow it to. Windows is a shared system, so your code will not be allowed to hog the entire CPU. Windows has thousands of threads running, and your code will only account for a few of them.

MrGlass3 12-Nov-21 17:53pm

Interesting. Learning something new here!

still confused though, when I do that the blue graph (Maximum frequency) is at 100% (it's actually at 150%) when I'm not running that program. Then when I launch it the Maximum Frequency drops to 135% and CPU usage goes to 20%. Then I launch a second instance and the frequency drops to 115% and the CPU usage goes to 40%. Does that behavior make sense to you?

And clearly it is capable of doing more things than it is doing (since I can launch a second instance without it slowing down). So is there a way I can configure or have my application run so that windows allows it more CPU time/resource?

Dave Kreskowiak 12-Nov-21 18:02pm

Process.PriorityClass Property (System.Diagnostics) | Microsoft Docs[^]

Don't expect huge performance gains. Windows already does a good job of scheduling.

MrGlass3 12-Nov-21 18:09pm

Unfortunately I tried that already. Both setting it in the application, and applying it from within task manager - didn't make any difference, like you said.

I guess I'm confused yet though. If windows has enough resources available to run 2-3 without any impact to the applications, why can't it run one faster?

Right now I'm looking at dividing up the work into 5 different applications and launching them all simultaneously so it gets done 5 times faster. That seems like hacking together a work around that shouldn't be necessary.

Dave Kreskowiak 12-Nov-21 19:10pm

A solution to a poorly understood problem is a bad solution.

This is going to come down to what you're doing in your algorithm and how you've implemented it. I have no idea what you're doing nor with what data, but I would start with minimizing or pre-allocating memory and objects as much as possible, and reusing them as much as possible.

Your example code spent a HUGE amount of time allocating List<t> objects, which come with a memory cost probably larger than you're expecting. I would probably look at replacing using any of the collection classes and go for straight up arrays instead.

MrGlass3 15-Nov-21 10:57am

I'm really just trying to understand what is going on here. So if I'm understanding what you are saying correctly, you are saying that .NET or Windows is more or less seeing that the code that is being executed is poorly written and is limiting the speed/resources available to that process. I find that a bit odd that it is more or less judging the code that was written, but that makes sense from the behavior I am seeing. And I could see that being a bit of a preventative measure Windows has in place to prevent viruses or bad actor programs from locking up a computer; but then if that is what is happening, I would expect windows to offer an override for someone to do something like I am wanting for a real purpose.

If that is not what you are saying, then could you explain how two processes can be launched at the same time without them running slower than if only one was running?

Really just looking for a system-level explanation so I can understand this moving forward for future programs, such as when I am referencing an external library that I am not free to modify but still underusing the CPU.

Dave Kreskowiak 15-Nov-21 11:23am

Windows is not examining your code to see how good or poorly it is written. It's a pre-emptive multitasking system that, intrinsically, your code, and every other user executable running on the system cannot hog all cores of the processor indefinitely.

If your code is written to allocate thousands of objects per second, you're going to be spending a sh*t-ton of time in the .NET CLR and kernel with memory management to get you the memory you need to allocate those objects. Your code isn't going to be running during that time.

So, you have to rethink how you approach the algorithm problem you're trying to solve to be more efficient with those resources.

MrGlass3 15-Nov-21 12:02pm

Ok, so it isn't a hardware limitation it is an artificial limitation implemented by the .NET CLR that prevents any single process from utilizing a given amount of memory allocation?

I say artificial limitation because the computer is able to run multiple instances at full speed.

Dave Kreskowiak 15-Nov-21 12:05pm

No, it's not an artificial limitation.

AGAIN, IT IS HOW YOU APPROACH SOLVING YOUR ALGORITHM! You're thinking in terms of the DOS days, when your code had sole access to all system resources, including the processor.

In a shared system (Windows), you do not have that at all. ALL resources are shared, including memory and CPU. You have to switch how to think about you write code to accommodate that.

MrGlass3 15-Nov-21 12:25pm

First I just want to thank you for being patient and continuing to have this discussion, I know I've been kind of discussing in circles and I'm aware it must be frustrating. So I appreciate the time you've taken, over multiple days to discuss this.

Having said that, I am ignoring the comment about rewriting the algorithm because I am focusing on understanding what is the limitation or bottleneck. If I need a justification, let's say this is a future situation where I have a complex calculation that is being performed in an external dll which I have no ability to modify so it is what it is and all I can do is try to maximize how many calculations are performed given the current situation.

So here is my low level missing link in my understanding: it cannot be a limitation of the hardware on how quick or often memory can be allocated because it can literally do more than twice as much as it is currently doing; if a second instance of the application is launched.

So before we go further, can we agree on that statement - that it can't be a hardware limitation if a second instance is able to perform exactly the same as one running alone?

Linke I said, I very much appreciate you time.

Dave Kreskowiak 15-Nov-21 12:27pm

Sorry, I can't explain this in a manner you'll understand. I've done the best I can.

Frankly, I think you're coming to the wrong conclusions because of misunderstandings on what you're seeing in terms of performance based on the tools you're using to evaluate that. For example, you're looking at a CPU% value for a process without understanding exactly how that value is determined and what it really means.

MrGlass3 15-Nov-21 12:37pm

Can't fault you on not wanting to continue discussing.

Without fully understanding, I can still accept CPU% is the wrong metric. But whatever is limiting it, the hardware is capable of doing twice as much without slowing anything down, yet.

MrGlass3 16-Nov-21 12:44pm

Sorry one last question that might help clear things up (in my mind at least). Like you had mentioned, CPU% is probably the wrong thing to be looking at; what attribute would be a good thing to look at in the Resource Monitor to help figure out what is bottlenecking this application?