|
|
Comments and Discussions
|
|
 |

|
Hi Shiv Sir,
This is one of the best videos from you. Super explanation.
I have one question:
Suppose I have to execute one select from my .Net application then my view is definitely I will not go ahead and wrap that statement around some Procedure. I will keep it inline only. Any views on this?
|
|
|
|

|
hello sir
many thanks for your post !
i am one of biggest fan of yours.. Good post i learn a lot from post as well comments.
please update with post.
offcourse you are respectable by reading your posts and books many juniors creaked interview.
plese keep posting.
Your reply for my commant can make my day
|
|
|
|

|
hello Shiv sir
many thanks for your post !
i am one of biggest fan of yours.. Good post i learn a lot from post as well comments.
please update with post.
offcourse you are respectable by reading your posts and books many juniors creaked interview.
plese keep posting.
Your reply for my commant can make my day
|
|
|
|

|
I like the way he explained about it.
|
|
|
|
|

|
Gives a solid clarification on inline queries execution and Stored Procedures
|
|
|
|

|
Long-long time ago they were invented to create some kind of ugly apps on server side (when "clients" were just a simple programs and there was only two layers). Novadays SP is not needed at all, since Application Server solves everything - from abstraction of tables from classes till security. In other words, everything you counted as "benefits of SP". Moreover: throwing away SP you REALLY can have everything in central place: changing in business logic will touch only your C#/Java/Ruby code, no necessary to login to database and jump on procedures/triggers. Moreover: knowing and using one general purpose language you don't need to study sh*t like PL/SQL, T-SQL, etc. - just SQL as an ugly brother to make queries.
Anyway my score is 3 since you gave good hint related perfomance: SP are the same fast as queries, who need all those SP?...
|
|
|
|

|
Relax buddy. Try to do mass data (big data) operations in C# or Ruby - then you know what SPs or SQL are good for. "It's the economy big data, stupid"
|
|
|
|

|
Dear You wrote a good article but in this way I made an experiment which increase performance and reduce load 80 percent mean which procedure was taking 10 sec that return record in 2 sec.
Example of my experiment
Create procedure sp_abc
@v1 varchar(50) ,
@v2 varchar(50)
as
Begin
Decclare @vuse1 varchar(50)
Decalare @vuse2 varchar(50)
set @vuse1=@v1
set @vuse2=@v2
select * from tblname where colunmname1=@v1
end
|
|
|
|

|
How can you write a query like your first example:
"Select * from Users where UserName='" + UserName + "' and Password='" + Password + "'"
without mentioning SQL Injection?!
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|

|
I've been a SQL Server DBA for over 10 years, and a .NET developer for nearly as long. Stored procs do have many performance improvements, especially as you start using newer technologies like LINQ and Entity Framework. Other commenters have already laid out very good reasons and proofs of why sprocs are "usually" faster, so I won't repeat them, here. Suffice it to say, in 90% of real world scenarios, sprocs are faster.
|
|
|
|

|
http://blog.sqlauthority.com/2010/02/16/sql-server-stored-procedure-optimization-tips-best-practices/
|
|
|
|

|
Yes very true.
We should not use sp_ for stored procedures instead we should rely on using usp_
Regards
Varun Sareen
Varun Sareen (Dot Net Developer)
|
|
|
|

|
When I first read the title I thought it, surely this depends on the RDBMS (and it does). For example SQL Server does things completely different to the way Oracle does things. However since this only refers to SQL Server I can understand some of your reasoning.
However you haven't actually discussed the reason why I believe stored procedures are quicker.
You didn't look at the parse time taken, or the time it took to send that query to the server. Although the cache is hit, you still have the time it takes to parse the SQL query by the SQL server and sending it to the SQL server.
For example say you have a query that is 100 lines long (which is where your going to actually notice the difference), there is a considerable amount of time that SQL server has to take and verify that there is no syntax errors and compile it to send it to the query optimizer. In a Stored Proc this is only done once where as for inline SQL its done every time you run that statement. Now mr SQL Server may after looking at the SQL say this is the same as a previous SQL statement, and grab the query plan out of the cache. However it still needs to verify that the syntax of the statement is correct.
Some RDBMS systems, like MySQL do a simple text comparison to whats in the cache (for example "SELECT * FROM Customer" is a different query to "Select * From Customer"), so the parser can ignore this step. As this text comparison is generally O(n), there is some a performance effect however we are talking about cycles here not seconds.
Another problem with your logic, is the time it takes to send the SQL statement to the SQL server if it is physically hosted somewhere else. In fact as the Ping time increases you would see a noticeable difference in performance.
|
|
|
|

|
This is a great inspiring article. I am pretty much pleased with your good work. You put really very helpful information. Keep it up once again.
|
|
|
|

|
You've cleared a misconception that has been irritating me for years.
Good job
5 from me
|
|
|
|

|
If you have one exe, and hundreds of databases at your customers, than maintainig stored procedures is an other story
Useful article, thank you!
|
|
|
|

|
In my opinion in IT the right answer is always: "like medicine, it depends on circumstances and scope"
Ing. Rinaldi Marco
|
|
|
|

|
Their tests were useful to show that there is in all occasions that the stored procedure is faster, However it showed the cases it is better to use stored procedure, for example (Sub select).
See in:
http://www.blackwasp.co.uk/SpeedTestSqlSproc_2.aspx
You can not reach a conclusion with partial tests. :|
|
|
|
|
|

|
i read your article in quetpond and here, i love it?
bcoz of questpond i cracked my interview ,thank you
|
|
|
|

|
Nice position. But I think you could write much about
|
|
|
|

|
I'd suggest you see what Jasmine2501 wrote, in every other case this article has no diagnostic value.
Also, you might consider showing off on more complex query.
|
|
|
|

|
Good article. It's very strange that people does not know the basis and their comments give evidence to that fact.
|
|
|
|

|
Please do this test again without using the * in your query. The * causes behavioral differences from properly written queries and is known to affect performance. Please give more emphasis to the fact that stored procedures have many other advantages and are still generally preferred over ad-hoc SQL statements. People will use this article to argue with managers, and they will be wrong. So, please make a bigger point out of the benefits of stored procedures (security, DRY, etc).
Oh also, another thing I'd like to see you do is to reinstall the database between tests. Since your SQL statements are identical, there's a possibility that one is faster because you ran the other one first. Over time, with repeated queries against the same table, SQL Server gets faster as it generates statistics and creates indexes to improve query plans. If you're hammering on the Users table looking up users all day, SQL Server will be able to optimize ANY attempt to do that, regardless of the source of the query. So, if you ran your inline SQL after running the proc 1000 times, I would expect it to be faster because SQL Server has been able to optimize the table for that specific type of lookup.
These are my suggestions for a more valid test:
1. Remove the * from your queries
2. Get a different row each time (you're always grabbing the same row)
3. Run the tests against different physical databases (so that performing one test doesn't affect the outcome of the other)
4. Return some useful data (such as User ID that was found) and consume it on the client side.
5. Try the other two kinds of operations, updating and deleting.
6. Run two load tests at the same time - one using procs and one using SQL - see which users "get through" more quickly when the server is stressed.
(Your conclusion is probably correct, but I don't think the test actually proves it).
modified 16 Jul '12 - 17:45.
|
|
|
|

|
I agree, this would be a test worth taking time to analyze.
|
|
|
|

|
I will update the article with feedbacks given with proper assumptions and more complex experiments.
Give me some time i will incorporate the things you have said.
|
|
|
|

|
I applaud you for taking this topic on. It is definitely one that is full of controversy (as you have seen in the comments). I would recommend qualifying your assertion a bit. The point you are trying to make (from what I have seen) is that the newer versions of SQL (2005 and greater?) are more intelligent about how they compile inline queries and so they reduce or, in certain circumstances, eliminate the performance benefits of a stored procedure. I would have to agree with this as long as the "in certain circumstances" remains. That, I believe, is where your problem lies. Your article seemed to overstep that line a couple times into absolutes (like in your title). I'm not convinced that we can state absolutes yet though. For instance, how do value changes in the parameters of the inline query change the caching? How about using dynamic SQL (built at runtime)? What if the inline SQL were to be called from a different client? I'm not asking for answers to these questions, but I'm instead pointing out that I believe there are probably edge cases where stored procedures will still be more performant.
In the end, however, I think you did a good job on this article. You put a lot of work into it and you weren't afraid to tackle a hard issue. You got us thinking and that is a great thing. I appreciate that and I look forward to more articles from you. Thanks.
|
|
|
|

|
Tim Corey wrote: I applaud you for taking this topic on. It is definitely one that is full of controversy (as you have seen in the comments).
But some one had to take this topic as well i wanted to educate myself.
Tim Corey wrote: "in certain circumstances" remains.
I agree this is not coming up in my article and thats why this confusion.
Tim Corey wrote: absolutes
I agree on this part my absolutes are something i need to work.
Thanks for making me better.
|
|
|
|

|
Good one.
Thanks for pointing out this myth.
modified 16 Jul '12 - 6:30.
|
|
|
|

|
Your treatise is based on a FALSE premise.
You state the "standard" reply will be: “Stored procedures are precompiled and cached so the performance is much better.”
This is incorrect and any half-decent SQL admin will, in fact, tell you that "The execution plan is evaluated and saved WITH the stored procedure".
They will also tell you that AS LONG AS the data mix remains consistent, saving the execution plan gives better performance than re-evaluating it every time.
In my experience, SQL DBAs are consistent in their approach: stored procedures provide an advantage over "in line" SQL where (a) the SQL is executed frequently (b) the data mix is consistent.
The single biggest advantage of stored that you appear to have missed is parameters and the simple way they make SQL Injection attacks much more difficult to execute.
|
|
|
|

|
fpnzmarksh wrote: "The execution plan is evaluated and saved WITH the stored procedure".
the evaluation means precompilation so must be my english.
fpnzmarksh wrote: The single biggest advantage of stored that you appear to have missed is parameters and the simple way they make SQL Injection attacks much more difficult to execute.
Can't i specify parameter by inline code...??? as said in the article.
I agree to your data mix part but that applies to inline as well. I have also put a video demonstration for the same to break this myth. But can not provide more facts than that.
|
|
|
|

|
I've had this "discussion" so many times, esp when it concerns any kind of ORM. Managers and even other devs will say "absolutely not, we can't use an ORM, we don't want the performance hit of dynamic SQL". Even after I've described the SQL caching mechanism, most people still balk.
Quite the shame.
SPs certainly have their place. I've written plenty of insanely complex SQL for reporting and such that absolutely had no business coming from code dynamically. But that's only because it's just so much easier to test and work with as a sproc.
For CRUD stuff, when I see mountains of generated Insert, update, delete Sp's for that, I can hear an angel gag.
But this is an excellent article. Kudos!
vbfengshui
|
|
|
|

|
The ORM thing is a excellent point you have put forward. I will add that to article also. Many people shy away from ORM because the SQL's are dynamic and can lead to performance issues ( Even i was party to it some months back). By the way the SQL generated by ORM are parametrized so SP and ORM SQL will have no difference.
|
|
|
|

|
Exactly, Any good ORM is going to generate parameterized dynamic SQL, so there's virtually no diff between using the ORM's dynamic SQL and SP's.
And it saves SO much work, in generating and then maintaining all those SP's.
vbfengshui
|
|
|
|

|
The article is correct, but is misleading, due to the simple examples chosen.
A dynamic query will be "compiled" and stored in the query cache, as well as a stored procedure. The plan is stored with the dynamic sql statement as key. This means if the exact same statement is run again, then there is no problem. If the dynamic sql does not use bind variables, and the "variables changes", each permutation of the dynamic sql will be compiled, and the query plan will be saved for reuse (until it is dropped by the server). There is a finite number of queries available to the query-cache. If these cached statements are wasted on permutations of the same statements, (e.g. statements that require a user id on systems with 1000 , 10000 or 100000 users), dynamic sql will lead to miserable performance, due to statement cache trashing. When a statement is executed that is not found in cache, a query plan must be created, and that can be expensive.
Dynamic sql can also lead to unnecessary network bandwidth usage. When using a stored procedure it is possible to use T-SQL, and do some extractions server side, that will process multiple tables and temporary result, resulting in a small result set. When doing dynamic sql, you usually have simple statements, processing is client side, and a lot of unnecessary data is transferred to the client side.
Dynamic sql can be faster when than using a precompiled stored procedure when your data is very unevenly distributed, and you require a different query plan depending on bind variables. Here you can either let the stored procedure be recompiled on every call, or is you can anticipate the pattern, use one stored procedure to branch to two or more different stored procedures (with the same implementation, but different names and different execution plans).
Stored procedures are usually a layer between the database and your client, therefore I think your example is too simple. You have a statement, that accesses one table, that requires a "no brainer" execution plan (if there is an index, use the index, else use full table scan). You should try setting up a sample, where you generate statistics for a report, saying which users have seen what table entries on an hourly basis, or something similar. Something that requires joining and selection, that cannot be accomplished using a single statement, and a statement that will only return a fragment of the data it processes. I think you will see a big difference at some point.
|
|
|
|

|
JesperMadsen123 wrote: If the dynamic sql does not use bind variables, and the "variables changes", each permutation of the dynamic sql will be compiled, and the query plan will be saved for reuse (until it is dropped by the server).
Thats why i mentioned in my article if you use parameters how is SP performance different than writing your parametrized dynamic SQL.
JesperMadsen123 wrote: If these cached statements are wasted on permutations of the same statements,
Will parametrized SQL have the same combination , i think NO that what is stated in the article.
|
|
|
|

|
I think the name of the article should be "Stored procs can be shown to be slower than inline SQL if you're really careful how you do it"
But yeah, I agree, this example is too simple. I think you should:
1. Remove the * from your queries
2. Get a different row each time (you're always grabbing the same row)
3. Run the tests against different physical databases (so that performing one test doesn't affect the outcome of the other)
4. Return some useful data (such as User ID that was found) and consume it on the client side.
5. Try the other two kinds of operations, updating and deleting.
6. Run two load tests at the same time - one using procs and one using SQL - see which users "get through" more quickly when the server is stressed.
|
|
|
|

|
Jasmine2501 wrote: "Stored procs can be shown to be slower than inline SQL if you're really careful how you do it"
I have least intention of doing it.
Jasmine2501 wrote: But yeah, I agree, this example is too simple. I think you should:
Even if i use complex examples it will not matter. SQL server does not distinguish between queries coming from code and stored proc. Same plan is created. The big trump card was caching which differentiated performance between inline SQL and stored proc.
I will update this article with more proper assumptions. But the complexity of the SQL should not matter.
|
|
|
|

|
Yes, but the whole reason you're seeing the caching strategy you're seeing is because your test is exceedingly trivial. Therefore it means nothing in the real world. I applaud the effort, but the technique is invalid.
|
|
|
|

|
Dear Sir,
You are one of the personalities I respect most in code project. Using procedures has its own advantages (not only performance).
This articles caused one of my junior programmers rebel against me when I asked him write queries in stored procedure. This can be misleading in ways you can't possibly imagine.
modified 14 Jul '12 - 2:49.
|
|
|
|

|
Thanks for your respect and i hope i live to that.I am really sorry but my intentions where to clear some misunderstandings. Hope you guys have united over a BEER again.
In the article i have already stated the advantages and i personally prefer stored procedure. I just wanted to break the MYTH that stored procedure increase performance.
You can tell your junior developer to CHILL and stick to stored proc. But performance can not be a criteria for choosing SP's. Many people migrate there inline SQL thinking that stored procs will increase performance but thats no more true.
Sorry again for all misunderstanding.
modified 14 Jul '12 - 7:17.
|
|
|
|

|
Reading through your article, I see what you are getting at, but I remain unconvinced (meaning, I am not highly certain what you say is correct). Here are some further points you may want to expand upon or consider:
- Dynamic queries. You talk about "dynamic" queries, but what you are really showing is ad-hoc queries. Dynamic queries are strings that can be built in, say, a stored procedure and then those strings can be executed. You don't need to use .Net in order to make use of dynamic queries. One time I have found dynamic queries to be useful is when searching multiple fields. Take a hypothetical Facebook search as an example. If somebody wants to find a friend, they can search on multiple parameters. Maybe they search on first name and high school, but not on last name. Based on that information, you can build a dynamic query that will not take into account the unused field(s). This dynamically generated query will use a different query plan, and so may use different indexes and perform faster. Similar things can be done with ad-hoc queries in most but not all cases (e.g., when a dynamic query must be created based on the result of another query, an ad-hoc query wouldn't be able to do this without an extra trip between client/server).
- "Dynamical". You mention "dynamical" queries. Maybe you are trying to be clever (to convey that some queries have characteristics in common with dynamic queries, but are not dynamic themselves), but it comes across as a typo.
- Auto-parameterization. You haven't mentioned how auto-parameterization works for variable length strings and strings which vary between unicode and non-unicode characters. For example, does it help to specify the parameter length when creating an ad-hoc query?
- Parsing. Somebody mentioned that there is a parsing stage when passing in an ad-hoc query. This makes sense to me, and is something you may want to analyze further.
- ORMs. You may also want to talk about ORMs, such as Entity Framework. All queries from EF are ad-hoc (unless you explicitly add a stored procedure to the model), so that could be worth mentioning for those who think ad-hoc queries are old-fashioned.
- Version control. You may want to delve deeper into version control, although that doesn't have much to do with performance, so up to you. I only mention it because you do talk about things which have nothing to do with performance (e.g., security concerns). Keep in mind that there are two sides to this coin. Red Gate has some version control software for SQL Server, but historically it has been difficult to ensure SQL stored procedures get put into version control.
- Editing. Another non-performance concern is ease of editing queries. If they are in stored procedures, they my be easier to edit in SSMS. If they are in strings, deploying them may be easier. There are a bunch of pros/cons either way, so you may want to mention those.
- Performance optimizations. I have seen ad-hoc queries perform better than stored procedures, and I think the reason was because they were used differently, and so had different statistics associated with them. The solution I usually ended up going with was either to ad index hints or to split a stored procedure into multiple stored procedures so they can use different query plans. Advanced scenarios like this may be worth mentioning.
- Multiple statements. If you have multiple statements, it may be possible to use ad-hoc queries, but you may have to transfer data between your application/web server and your SQL Server. A stored procedure would allow for all those statements to exist in a specific scope and for all the data to be processed entirely on the SQL Server. Worth a mention.
- Deficiencies. Ad-hoc queries and stored procedures may not always be interchangeable. For example, stored procedures may have a unique scope in which a certain scope statement may operate (e.g., SCOPE_IDENTITY). Also, recursive SQL calls are another thing which stored procedures handle better than ad-hoc queries (especially in versions without CTEs). This is partially a performance concern, as ad-hoc queries may require multiple trips between the app/server.
- Cloud / client. In some environments, such as on Azure, there may be significant latency or network lag between the web server (or "compute instance" is it is called in Azure) and the SQL Server. In these scenarios, things such as the size of the query sent across the network and the number of queries sent across the network become more important. There are also client applications that may have hundreds of milliseconds worth of lag, and bandwidth limitations would be very apparent. Worth a mention.
- Unexplained links. Rather than point the reader to other articles, explain the important part and then send them to that article if they wish to verify your statements or research further (e.g., when you say that auto-parameterization will help, you should go into more detail to describe how and when it will help, and when it may not).
- Images. I'm not a fan of the cutesy images you have added, I suspect without permission from the copyright holders (e.g., Spiderman). Artists can be very protective of their property. If you want some free images, maybe look at the ones in Office or some free galleries online.
- Linked titles. You have linked the section titles. Linking to them from the table of contents is good, but making them links is not useful unless those links go somewhere.For an example of how to properly handle this, view the source of my article here.
- Versions. You don't make much mention of versions of SQL. For example, you don't say which version you are testing with. And you haven't mentioned SQL Server 2008 or SQL Server 2012. Also, the last company I worked for is still using SQL Server 2000, so I wouldn't discount that just because there are new versions available. And I am really just assuming you are using SQL Server. What about other vendors, such as MySQL? If you aren't covering those, it would be good to make mention of it.
You've made some good points, but I think people could really benefit from a more rigorous and in-depth article. Otherwise, you are just providing the same incompletely researched position of people who think stored procedures are superior.
|
|
|
|

|
I would like to answer in detail to your suggestions and as well incorporate some of your comments in my article. You have put some wonderful comments and do not want to answer in one liner. Give me a day. Thanks for your inputs.
|
|
|
|

|
thank you very much!
its very useful for me!
|
|
|
|

|
Thanks for breaking myth.
|
|
|
|

|
You may use fast forwardonly cursors. All you need is to declare them accordingly. They are as fast as SQL query are.
crc of abc
|
|
|
|

|
To be sure that your database connection is handled by right driver, not by a generic one, how do you connect to database? Do you specify the driver, or does ODBC connection specify a driver? If not, then you have to use the correct driver.
crc of abc
|
|
|
|
|

|
There is more to performance than just "is it put in the cache". And more to SP's than just performance.
1. Cache Reuse: Sp's will exist in the cache once. (or a small number of times if you recomple). TSQL will cache the exact Execution string. Unless SQL can Auto-Parameterise the query it will fill the cache with 1,000's of slightly different parameter values & will not get a cache hit. Thus you burn RAM filling the cache with nothing useful.
2. SP Optimised Parsing: SP's should be called with CommandType.StoredProcedure as this bypasses the query parse phase, something you can't do with TSQL which must be called as CommandType.Text
3. Security: Passing parameters to SP's can prevent SQL injection attacks (unless you build Dynamic SQL inside your SP).
Denying permission to Tables & Views & only permitting Execute permission on SP's also goes a long way to preventing SQL injection & other security attacks.
4. Network Traffic. In your example TSQL was 8.7 times the bytes compared to the SP'. Developers who use nHibernate, LinqToSQL & similar TSQL generation dev tools will find will find they are extremely verbose & need to be used very carefully in order to craft good quality output. Recently I tuned a commercial eCommerce site which was dieing. Inbound TSQL queries from the web servers were responsible for 40% of the total net traffic. Even with Quad NIC's the Network was the bottleneck. The Query cache was massive, CPU was high just dealing with it all. The nHibernate queries had unneccessarily long table alises. Also extra CPU was needed to handle the network stack & Interupts.
5. Testing: It is much easier to generate a load &/or to unit test via SP's. Much harder to stress test the DB layer to breaking point by injecting load via the app layer. Rarely do companies have the budget for that much H/W.
Stored Procs are an abstraction layer, they make a huge difference in multiple ways. Yes they prevent the use of some nice Dev Tool features. I'd mention to your developers that 4GL tools like LightSpeed & VS can write their code for them. So why don't they use them? They will probably answer because they can write more efficient, smaller etc code. Same for the DB layer.
Hope it helps
Dave
|
|
|
|
 |
|
|
General News Suggestion Question Bug Answer Joke Rant Admin
|
Many developers think stored procedures are precompiled so they are faster. I have a different story to tell.
| Type | Article |
| Licence | CPOL |
| First Posted | 2 Jul 2012 |
| Views | 63,362 |
| Bookmarked | 61 times |
|
|