|
As much as I hate jumping into another man's argument, I feel compelled to answer this one.
Quote: When you have as much experience as me in the kernel I might listen to you
The problem with saying things like this is that somebody comes along with more experience. I'll see your 14 years and raise you 31 more. I've been doing this since 1967 on various machine types, mostly on kernel / monitor / core / OS, whatever you wish to call it. And yes, some of my kernel code has been into space.
Allocating from something called heap / common storage / pool / whatever is always slower than just adding / subtracting a size from the stack pointer. Plus, the pool is usually one shared resource so for multithreaded or multiprocessor kernels some sort of locking mechanism is needed to prevent corruption of the basic allocation / deallocation structures. Add in the possibility of doing allocations at interrupt level too and you complicate the locking mechanism further.
I've also dome a fair amount of kernel performance analysis over the years and guess what it reveals as a common bottleneck in kernel programming, heap allocation / deallocation. These are places where one would concentrate the efforts to find smarter and faster allocation methods, split poole, etc. And even though better algorithms have developed over the years, carving small arrays / structures / blocks of code on the stack just flys over calling pool allocation.
Now it is true that you need kernel pool for longer life objects and maybe for things too large to be of practical use on the stack but please don't argue that all things must always go on one place over another, that's just too many absolutes. Sounds more like a religious argument then a logical one.
|
|
|
|
|
I dont know if you followed the rest of this thread but what I recomended was using the heap so that verifier can see where your buffers are getting overrun, then go back to using a stack buffer when the code is good.
Yes, heap allocaiton is slow, thats why preallocaiton is a good idea for frequently used memory blocks of a known size. Look aside lists for example. I mentioned this too. This obviattes any performance issues.
What you say about locking mecnahisms isnt the case though. The memory manager deals with that, not the code, so it doesnt add complication. Shared data between threads will though, but this wont be the case here since the guy is only using the buffer in one function.
Re kernel performance, look aside lists. Plus IO is the real bottle neck.
But here is my central point, whic is always valid: Stability is more imporant than speed. Always.
==============================
Nothing to say.
|
|
|
|
|
Erudite__Eric wrote: then go back to using a stack buffer when the code is good
Did you read your own posts!? ...when did you say that!? 
|
|
|
|
|
said the same thing in my reply which I was typing as you entered yours.
|
|
|
|
|
|
I think this guy may be delusional... 
|
|
|
|
|
|
This is why I don't like jumping into someelse's argument, it becomes mine 
Yes, I've followed the posts and just to be sure, I went back and re-read all of yours specifically (you should do the same to refresh your memory). You never mention going back to a stack buffer "after the code is good". In fact, quite the opposite, you were adament about buffers always being from the heap, always, always. You repeated that quite often. So the resposes where to those statements.
Erudite__Eric wrote: What you say about locking mecnahisms isnt the case though. The memory manager
deals with that, not the code, so it doesnt add complication.
Well that's a convenient hand wave, blaming the underlying function rather than the caller who invokes it. The point is that memory allocation at that level is costly and that even if the "memory manager" has to deal with the locking, etc, you still are resposible for chosing a methodology that invokes that call over stack allocation. So, the introduction of the overhead is your choice, the kernel code is just giving you what you asked for. Don't blame it.
Erudite__Eric wrote: Plus IO is the real bottle neck
IO is a "wait state" event and not chewing up cpu cycles, which was what this discussion was all about. The kernel / application is free to do other things while IO is going on using any number of asynchronous IO techniques. If you wish to now have a discussion on all the things that affect application / kernel performance, we can do that too.
Erudite__Eric wrote: Stability is more imporant than speed. Always.
Only an idiot would argue in favor of "instability". Of course stability is important. In fact, if you're getting paid to do code, your client / employer will assume stability and will find someone else to deliver it if you fail. So most don't even bother listing stability as a priority, it's assumed you will deliver it. On the other hand, many will list speed as the priority, depending on the application. Imagine trying to defend a radar application that is too slow to catch all the incoming phase radar data by saying "but it's stable!!".
|
|
|
|
|
Chuck O'Toole wrote: You never mention going back to a stack buffer "after the code is good".
I did, here: http://www.codeproject.com/Messages/4071887/Re-making-a-unsigned-char-array-gives-a-buffer-ove.aspx[^]
Chuck O'Toole wrote: IO is a "wait state" event and not chewing up cpu cycles,
I am talking user mode IO. Ring 3 Ring 0 transitions are really heavy. Surprisingly so in fact.
HW IO is generally very quick.
Chuck O'Toole wrote: it's assumed
And surprisingly often, ignored. I can not tell you how many times I have had verifier barf on third party drivers, yet it is a tool MS introduced expresly to improve quality.
You would also not believe how often I have seen memory allocation used unchecked. Guaranteed BSOD as soon as resources get a bit tight.
==============================
Nothing to say.
|
|
|
|
|
Besides the fact you know nothing about me, you should also recognize people always disagree on implementations and learn to accept that.
|
|
|
|
|
Oh come on, can't you handle a bit of friendly disagreement? 
==============================
Nothing to say.
|
|
|
|
|
Erudite__Eric wrote: Rule one. All arrays go on the heap. Period.
A better rule might be to implement all arrays with std::vector
|
|
|
|
|
Same thing. std::vector allocates heap, but yes, stl is always useful.
==============================
Nothing to say.
|
|
|
|
|
Erudite__Eric wrote: Same thing. std::vector allocates heap, but yes, stl is always useful.
Yep, vector gives a lot of advantages over an array and could even be made to use the stack with a different allocator
|
|
|
|
|
Vectors use the heap effectively because they allocate memory ahead of time instead of doing a new every time. They effectively get around the slowness at the expense of having some extra memory allocated ahead of time. Think he refuses to hear other people's arguments at all.
|
|
|
|
|
Oh, and by the way, if you regularly need a set size of heap buffer you can preallocate and manage yourself. In the kernel you can use look aside lists to do the same. Very good they are too. I work a lot in the kernel. Believe me, stability is more important.
==============================
Nothing to say.
|
|
|
|
|
Stability? ..How do you equate stability with one's own errors in coding?
|
|
|
|
|
Because it is very easy to get sh*t data in code. With sh*t data it is very easy to overun a buffer. A stack cprruption is very hard to debug. A verifier barf on a heap overrun is very easy to debug.
It is that simple. Make your life simple.
Make the code 100%. Dont worry about speed. Modern languages have already killed performabnce to such an extent and memory is so cheap that for a commercial app on a PC it is wrong to think of speed and resources,
On a dedicated computer on a satellite with expensive limited HW it is a different thing, But think of this. You develope your code using heap based buffers, and when it is 100% solid, you then use stack based buffers.
See where I am going? It is called clever engineering. Develope the product. Make it perfect. Use any technique you can to achieve that, and the verifier tools from microsoft are very very good.
==============================
Nothing to say.
|
|
|
|
|
I think some of the others have addressed your question directly already (as to the error), but another interesting follow-up for you comes up. Why do you need buffers that large? Maybe you need to think of a different way of achieving your end-solution.
|
|
|
|
|
You said your app is dialog based, this also means that you have multiple threads, even though you may not be creating them manually. You can in fact allocate arrays of arbitrary size on the stack, but this is true only for the main thread. If you try to allocate on the stack of a thread, then this stack will be limited by the amount defined upon creation of that thread! If you do not create that thread yourself, or do not pass a stack size, then a default value will be assumed, and this value is probably smaller than what you wish to allocate.
The solution is to either
1. Allocate the array on the main thread and pass a pointer to it to the other thread, or
2. Allocate on the heap, rather than on the stack, or
3. Find out how to manually override the stack size of the thread in question (if that is even possible)
Having said that, the few answers I did bother to skim over seem to be mostly mistaken in their analysis and reasoning, even though some of the suggestions (such as allocating on the heap) might work.
Hope this helps.
P.S.: While I stated the stack of the main thread can be arbitrarily big, you still may have to tell the compiler this, via options, as Chuck already pointed out. I just tested on VS 2003 and VS 2010, and both define a default stacksize of 1 MB. I've found that, on a 12GB system I could increase the stack to 10GB and define an array of 1000 million doubles on the stack (in the main thread). (in VS, you can find that option under Linker/System/Stack Reserve Size)
modified 10-Nov-11 9:21am.
|
|
|
|
|
I have a CWinApp based DLL that worked fine in VS6. Since converting it to VS2010, I can't get it to access the InitInstance function (or even the constructor for that matter).
I can see using Process Explorer that it loads at the correct time, but it doesn't initialise properly. I can access its functions using GetProcAddress, so I can probably work around it, but I'd like to know what the problem is!
(It is difficult to debug because I don't load it directly - I tell a third party DLL about it and (presumably) that DLL uses LoadLibrary and GetProcAddress to load my DLL).
Anyone else experience this?
|
|
|
|
|
What version of Studio was the old DLL built with? Under certain conditions, old DLLs that were built with different studio versions will fail to load (simple DLLs that use plain C/C++ usually don't have this issue but more complex DLLs do).
I don't recall exactly the issue that leads to this problem, may be the fact that you're using different MFC versions and there was some sort of non-reverse compatible change along the way or something similar, but the easiest solution if you have the source, is to upgrade the source to compile in the same studio version as your main executable.
Good luck!
|
|
|
|
|
I used to build everything with VS6 and now everything with VS2010 - I'm not sure what the third-party stuff is built with.
Interestingly, the VS6 DLL still works with the VS2010 build...
|
|
|
|
|
The build environment isn't the problem, it's really some of the configuration... like I said, can't remember exactly what the problem was.
|
|
|
|
|
Look in your redist directory for the MFC dll's that ship with VS2010. You'll see that in addition to the main dll, there are two other types of dll's that must be in the same directory.
One is under a folder with the MFCLOC suffix, and the other suffix is OPENMP.
Copy the appropriate versions of each family of these dll's to your runtime folder.
Your redist directory should be under \Program Files\Visual Studio 10\VC\redist
The difficult we do right away...
...the impossible takes slightly longer.
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
