|
I did recently try out the Jet Brains profiler on that block of code that did the dictionary tuple lookup just for fun to see if profilers have improved since I last used them. It showed that my bottleneck was in GetHashCode(). That was identified as my #1 application hot-spot. The method that actually did the dictionary lookup was ranked almost dead last. That's a major profiler failure in my book. It didn't point me anywhere close to the right place. I messed around with the profiler for about half an hour trying various options to try to get it to show me the call tree that was at fault, but it didn't seem possible. Maybe it is and I just didn't set it up correctly. Like I said, I only spent about half an hour on it.
However, to be fair, I guess GetHashCode() *was* the ultimate method at fault, but none of the views in Jet Brains indicated that it was from the dictionary lookup and because I wasn't seeing call trees, if I didn't know what was going on, I probably would have never connected the two.
Probably the top 10 methods identified were internal .NET methods like GetHashCode() that really had nothing to do with figuring out the performance issue. Since it isn't really GetHashCode() that's the problem, but using a tuple in a dictionary is. The profiler also ranked TryGetValue as pretty low, so there was no indication that that method was calling GetHashCode() so much.
|
|
|
|
|
Your point is much clearer, and more powerfully expressed ... to me ... with this excellently written personal example.
thanks, Bill
“I have diligently numbered the days of pure and genuine happiness which have fallen to my lot: They amount to 14.” Abd-Ar Rahman III, Caliph of Cordoba, circa 950CE.
|
|
|
|
|
Yes, but before I do, your original email opens up a bigger issue, I think. To demonstrate, let me ask:
Does anybody really use a debugger? You see, I think that any real software engineer should be able to write bug-free code and if looking at someone else's code, should be able to identify the bugs in a fairly straight forward process.
Pretty silly, right?
Of course this is tonge-in-cheek, but the point is that a profiler is a tool that has a place in the engineer's toolbox. By asking the question in a way that belittles the use of the tool, it seems the results of the answers will be biased, or at least the interpretation of the results will be.
(Full disclosure: Maybe there are a few who are so good they can optimize their code without a profiler, but I have learned the hard way that I am not one of those - I also used to believe profilers are for the weak.)
Engineers use tools. Engineers use data from experiments to test their products and, maybe, improve them. Ignoring a tool doesn't make sense to me.
I agree that profilers can be hard to use and produce a lot of noise, but using a lathe is hard, too, until you understand how to get good results from it. A profiler is not the primary tool, to be sure, but there are cases where it can be Indispensable (and has been to me). Here are where I think it makes a lot of sense:
1) Education: How does one get experienced in performance analysis and tuning with C#. And how to we know we are right? Sure, moving the invarient out of the loop make it run faster, but are there other things we could/should have done, as well? Profilers are a great way to learn where the bottlenecks are, especially when working with a new language or in a new environment. One line of code taking 85% of my runtime? How do I know it's that one? (Linq is an easy target, but what about your enum example? How do you know the first time?) I've had experiences similar to yours where I see something obviously bad and let the designer know only to be told "no way it's my problem"; When I show them profiling output proving they are the problem, they start to listen. Data are powerful things!
2) Real-time systems: I think much of the discussion on this thread has been about business-logic type applications. Large databases, searching, etc. But what if you are building a real-time processing system with streaming data in one thread, real-time analysis/processing in another and real-time display in a third (and it's actually much more complicated that that in real life as those threads are really collections of threads, possibly). I live in this world and I can tell you that I use a profiler before my final build (often many times, but certainly once!) to see how much slack I have. That is, where do I have performance issues that are close to being a problem. For those are the ones that may bite me in real-world use. My stopwatch tells me I'm OK, but I'm not. (Concrete example, I need to process 100 frames of data per second. I profile the code and find my main processing loop takes 0.009 ms. So I'm good, right? Well what happens if something else is running on the same machine, taking cycles away from me? I want to look at that function to squeeze out more performance if I can. And each change I make can be profiled to see if what I did really was an improvement or not and by how much. Some performance improvements require tricky code or code that's harder to maintain but provide very little real-world improvement; I stay away unless I really need every microsecond. I want to know, however, not just optimize until I can't any longer... and, yes, I've gone so far as to write in-line assembly code where I needed to! But I don't want to go there in my first pass of optimization.
3) Proving to a component manufacturer that their component is the problem. I've been here, also. I have a DLL from a hardware manufacturer that is causing a problem and I can prove it by getting accurate timings of how long some of their calls take. Yes, simple timer calls can be used for this, as well, but when I'm calling several dozen such functions, running the profiler can give me all the timings, quickly, and I am not modifying my code to do so.
4) Finding bugs. Yes! Similar to #2, actually. Imagine you have a program that meets the spec in terms of performance. So you ship. But you have one section of code that calls a function multiple times for no good reason. This is poor programming, but because it's done in multiple threads or in various methods that are not obviously related so it's hard to spot. The profiler can show that CheckStatus (for example) is being called a lot. You may look at to why this is and could fix what is currently (or soon to be) a bug in logic. If you are running on a system where battery life is important, then I consider this a bug.
I know you may not be convinced, and that's OK. I do 100% agree that understanding what you are doing, thinking and analyzing code FIRST is very important, but I don't think it's enough for most people.
I think this is a great discussion for opening up these potential issues.
Thanks!
|
|
|
|
|
However, no one has mentioned the the core reason that profilers exist:
We write code in high-level languages which get converted to machine language (CPU Operation Codes).
A profiler can tell you that your code which is only 1 line (of high-level language) actually takes 5 CPU operations to complete.
That part isn't actually obvious, because we write code at one layer and most people who haven't written assembly don't really understand this.
Mostly you get lucky that the high-level languages and built in compilers optimize even very badly written high-level language into a smaller number of CPU operations.
They do so much work these days, your code may not even be represented the way you think it is by the time it gets to the CPU.
So, unless you are saying you can turn your high-level language into CPU operations in your head, you can still write code you think will perform well which actually somewhat inefficient when turned into OP Codes tha the CPU actually runs.
That's where a profiler can come in handy. You can see what your code really does to the poor little processor.
One of the best and easiest to use profilers is in ICSHarpCode's SharpDev Studio. Try out the IDE (Integrated Dev Environment) and I think you'll be amazed.SharpDev Studio - Open Source & Free
[^]
Hopefully I've provided some food for thought.
|
|
|
|
|
Nope. I can't convert C# to ASM in my head  .
A lot of the performance issues I've resolved over the years, are just bad designs or code that's doing redundant work. One thing that another responder pointed out is that in the "real world" (non gaming ), nobody is going to appreciate you shaving off a 100ms or 250ms or even 1000ms or even a second or two. However, in gaming where people get off to frame rate numbers, I can see how squeezing every little ms out of your code would be useful.
I used to work on a project where the end goal was to process 4 BILLION records. One day I was optimizing some code and my boss yelled at me for wasting my time. I simply brought up my calculator and calculated 4,000,000,000 / 3600000 = 1111.11 hrs / 24 = 46 days for him and pointed out that cutting out a miniscule 1ms per record would save 46 CPU days on the project. He said "Oh" and walked away. Lol. I ended up shaving off a few MINUTES per record fairly easily.
|
|
|
|
|
You are right though. There is so much sloppy code out there in the business world that your solid list you mentioned in your original post will make amazing differences in the code.
That's why no one uses profilers any more.  Half the coders are too busy out there blithely creating sloppy code and the other half are too busy out there applying solid principles cleaning it up.
|
|
|
|
|
Yes a profiler can't always get you the stats you need. Like others said, they are useful for finding hotspots in large distributed applications. For example I worked on a large web based CRM system that had randomly had extremely slow page render times. The whole system used DataSets as core data objects that were associated with classes defined in C# code that was stored in the database and dynamically compiled.
Now, I could have gone in and manually instrumented several hundreds of methods and objects, but using a profiler I was able to find the slow spots in several of the subsystems including the process of compiling the data objects. It took a little experimentation to find out which of several approaches was fast enough. The worst offender ended up being really inefficient code in the site navigation that involved a bad caching implementation.
I'm skipping around to your last example. A memory profiler will find that issue pretty quickly, in addition you will have a significant amount of time in garbage collection when you look at performance counters. I also like to instrument my code with custom performance counters so I can monitor it in production easily.
Ok, the highest performance C# app I worked on was a click scoring web service that took web server log lines and had to return a score in 100ms or less. It peaked out at about 6,000 scores per second averaging less than 1ms average response time. The entire first stage involved mapping strings to ids using Dictionary<string, int=""> objects. Then there were a number of shared statistics that were maintained based on those integer lookups. Several hundred rules were run on all that computed data. That was combined with a naive bayes classifer, and the statistics were updated with the results of the scoring, and the score was returned to the client.
The first versions loaded data as needed, but eventually everything was hugely sped up by loading the entire database in memory as the first step and then generating all the ids in the client. Also writing data out to the database was done with sql batch update objects which out perform any kind of stored procedures by wide margins. But even then, stored procedures haven't been faster than inline sql for years.
Micro-optimization like you are talking about can get you a long ways, but at some point secondary effects like garbage collection, memory paging, working set management, cache lines, etc. are going to become issues especially when you are working with something like a 16 Gigabyte process size with .net. At that point you have to look at more global issues.
|
|
|
|
|
SledgeHammer01 wrote: Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow.
Unfortunately, while there are millions of programmers, only a few have the analytic skills that Jon Skeet ... and, presumably, you ? ... have: [^].
To the rarer programmer who has "got down" in the plumbing, and studied the internal behavior of every operator in their language, and the structure of its framework, carried out timing experiments, etc. : I salute you !
cheers, Bill
“I have diligently numbered the days of pure and genuine happiness which have fallen to my lot: They amount to 14.” Abd-Ar Rahman III, Caliph of Cordoba, circa 950CE.
|
|
|
|
|
SledgeHammer01 wrote: look at a block of code for a few minutes and instantly identify why it's slow
I hate to destroy your self confidence, but some code is work just perfectly with input A and perform terribly with input B (see compression algorithms), so looking at the code and even understanding it means nothing without the real-time environment...
SledgeHammer01 wrote: maybe it's just a talent
The word you looking for is neglection me think...
SledgeHammer01 wrote: from my limited experience with profilers
Better learn profilesr (how to use and interpret its output) and after that we will hear you again...
I'm not questioning your powers of observation; I'm merely remarking upon the paradox of asking a masked man who he is. (V)
|
|
|
|
|
|
SledgeHammer01 wrote: You couldn't use your profiler without the proper input either
That's profiler IS. Check your code in real-time environment. And if you believe that you can solve the issue by seeing the input, you never saw real input and never understood the issue!
I'm not questioning your powers of observation; I'm merely remarking upon the paradox of asking a masked man who he is. (V)
|
|
|
|
|
Many, many times. Both static and dynamic profiling is an important tool in enterprise development. Yes, your basic steps are useful in a single application, but when you move into a large distributed, multi user system simple code viewing doesn't cut the mustard any more. The time it takes to look through hundreds of methods and calculate which ones will be a problem under which conditions is waisted if you have a decent set of diagnostic tools at your hand.
Real example. I had a junior put together a piece of work. Simple service for validating the entry from a form. Now applying your logic above the code would check out fine. Performance was good on the test system and there were no obvious conflicts, data was being cached, look ups avoided, etc, etc, etc. The memory footprint was a pig, absolutely horrendous. And for each user the problem increased. We needed to pool resources, reduce cached data and in some places slowed down response times.
|
|
|
|
|
I also find this curious. "You didn't use a profile, did you" is a pretty common piece of bullshit that people sling. It's almost always perfectly obviously what the problem is, or that something will be a problem, without even running the code let along profiling it.
Basically what that comes down to is, if you do something stupid, it's going to suck. And preemptively not doing stupid sh*t is not "premature optimization".
Of course it's not always obvious. For example,
xor ecx, ecx
_benchloop:
lzcnt eax, edx
add ecx, 1
jnz _benchloop Why does this measure the latency of lzcnt, instead of its throughput? It doesn't look like it should do that, so in the original code that contained the "problem lzcnt", that problem gave me quite the chase. Initially I didn't even notice something was wrong.
No actual profiler was involved though.
|
|
|
|
|
They always just say "male, aged 18-35, who used to kick cats", so it's a waste of time even consulting them.
I wanna be a eunuchs developer! Pass me a bread knife!
|
|
|
|
|
Good luck finding the most important bottle necks in applications of several millions lines of code without a profiler!  Hell, even in smaller applications I wouldn't even bother finding bottlenecks without a profiler. Why would you deny yourself a helpful tool?
Wout
|
|
|
|
|
wout de zeeuw wrote: Why would you deny yourself a helpful tool?
Because he thinks the tool sucks
M.D.V. 
If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about?
Help me to understand what I'm saying, and I'll explain it better to you
Rating helpful answers is nice, but saying thanks can be even nicer.
|
|
|
|
|
They do[^].
(sorry; it was just too easy)
Software Zen: delete this;
|
|
|
|
|
I find the idea of not being able to use every tool available to you is incredibly naive. Granted that you should be able to solve simple performance problems just by looking at them, but some problems are just too big and interwoven to solve like this. Also, it's not always just your code that you're profiling - don't forget that you can often identify problem areas in code that you're using that there might be alternatives to. So yes, I use profilers regularly. I use them because I'm aware enough to know I don't know everything and that there's always something new that I can learn.
|
|
|
|
|
>>> Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow.
Are you serious!? Maybe for a single method or simple application, but for a complex application there's no way you can "just see it". Here's a simple real world example, recently I used a profiler to see where the bottleneck was in a web service which sits on top of 3 or 4 other layers (and I won't elaborate cause I'm keeping this simple). It turns out the bottleneck was in a piece of Microsoft code used within the ORM we use, so that was refactored to use a different method to do the same thing. No way you would have just seen that by looking, not least cause the code wasn't there to see.
I'd happily admit I also wouldn't be able to just see an issue in my own code some of the time. In the above example we're talking 1000s of lines spread over multiple assemblies, why would I bother trawling through all that when I can just use a profiler?
I was lucky enough to be able to test this for a single thread, good luck if you come across an issue that only occurs when you've got multiple threads involved.... 
The profiler is your friend.
PS. does this mean you never use SHOWPLAN/EXPLAIN either?
|
|
|
|
|
Absolutely I use a profiler. You're right that looking at code can often provide the answer, but when you're code base is several thousand lines of code split across multiple components using a profiler can at the very least help identify where the problem is.
It's also useful to spot things which might not be a problem with your code. Maybe a library you're using is doing something stupid, perhaps it's a little known feature of the .NET framework which is causing you problems. We recently had an issue with an older piece of software which was running into that, looking at the code would not have helped in the slightest but a profiler quickly sorted out where the problem was.
A profiler should be a part of any developers toolbox, fair enough that it shouldn't be the only solution to every problem, but it should be a tool that you are comfortable using and comfortable to know when to use.
Eagles may soar, but weasels don't get sucked into jet engines
|
|
|
|
|
Inline SQL works fine so long as your queries are parameterised.
|
|
|
|
|
Any reason why parametrized SQL code would be slower than a stored procedure? I mean, unless your statement is so big that the network overhead of sending it would be significant, a parametrized statement would have its execution plan cached by the server, same as the stored procedure.
Of course it's a lot more interesting to get all the results you need in one statement instead of looping over every row and issuing a SQL statement for every row. Same goes for LDAP.
|
|
|
|
|
Additionally, using the stored procedure doesn't eliminate the parameterized SQL code -- so now you have two things that need to be profiled, by two different profilers.
|
|
|
|
|
I rarely use profilers, but there are a few cases when even good programmers may have trouble seeing where code is slow. Sometimes there may be a library call which you think will be fast, but turns out to be slow. This could be due to system call(s) that take longer than the programmer realizes or due to the library function's algorithm working very differently than the programmer realized. We all make assumptions about things that sometimes aren't true. One very big issue is using something like using strlen(some_string) as a constant in a C program. Many programmers use that as though it is a constant in the compiler's mind, when it may not be (GCC has an __attribute__ that lets the compiler see that it is, but it's not standard). That's the kind of thing that could end up in a loop, possibly as part of the loop check, without many programmers realizing that it's a problem. While you might see that code, there are similar circumstances which are less obvious. These are often the results of invisible code introduced by compilers for languages higher than C. Overloaded operators, copy constructors, and destructors can all have much more impact on code than programmers realize. Knowing what kinds of things your language may hide is a good way to avoid these, but some things still slip through.
Other things that are easy to miss is how contentious locks, semaphores, mutexes, and other blocking operations are. These are difficult to figure out sometimes.
There are also times when you need to demonstrate to someone else where the bottle necks in code are. If your cocky junior programmer weren't your subordinate but rather your boss then the output of a profiler would be something that you could use to combat the cockiness. This is particularly useful when you get "Your code is slow! Fix it!" and you can come back with "This external code/hardware/whatever, which isn't under my control, is what is slow."
Another time when using a profiler would be very useful would be if there were disagreement or uncertainty over which of multiple parts of a program were responsible for the different amounts of the slowness. Maybe you need to prioritize the order in which these are addressed. For you or I it may be obvious that a section of code is slower than it has to be, but when comparing two suboptimal sections of code it is very often that eyeballs alone are not enough.
There are also times when a profiler can be used to find bugs that would take a lot of stepping in a debugger to locate.
|
|
|
|
|
I've done both, and yes, using your noggin to choose a better algorithm wins over using a profiler every time. The profiler can only tell you how to make your bubble sort run faster, it can't tell you to choose to implement a quicksort instead.
We can program with only 1's, but if all you've got are zeros, you've got nothing.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
