Click here to Skip to main content
15,885,546 members
Please Sign up or sign in to vote.
2.00/5 (1 vote)
See more:
Help - I have a relatively large mixed c/C++ project (15 files .c and .cpp) which I recently modified. The modifications caused the program to gradually consume all available memory. I have temporarily removed the changes from the most suspect files and the problem still persists. The program uses a lot of ODBC MySQL queries, but is very clean about freeing Statement Handles, closing cursors and Connections. There are some mallocs in process initialization but they are only one time and there is a free for each. I have checked for memory leaks with a variety of tools including Virtual Leak Detector, Deleaker and crtdbg and they have all indicated no leaks. So I am looking at some other kind of problem like unintended recursion or an unintended entry into adjacent routines. The program doesn't abort even when it has consumed all available memory - things just become very slow.I am looking for a windows tool (preferably free) which will analyze the programs private working set and identify what the various memory blocks are from. Would really appreciate any help.
Posted
Comments
Jochen Arndt 16-Jun-14 3:43am    
Leak detectors have built-in lists for functions that allocate memory. But these lists can't cover all possible functions. So check your code for system functions that might allocate memory that must be freed by the caller.

Things always become slow when you run out of resources and takes a long time to exit from the program that is nothing to investigate.

There is one obvious question are you sure it's your program that is growing and consuming all the resources? By that I mean the SQL server isn't on the same machine is it and it is bloating it's cache up to consume all memory.
 
Share this answer
 
Comments
Member 10492119 15-Jun-14 23:31pm    
No...the database is on a separate server so my program is actually consuming all memory.
The Private Working Set has now grown to 1.6 GB and climbing at a rate of 12 MB / Minute. It has been running for about about 2 hours now so it's growth is fairly consistent. I just can't stare at the code any longer. It has been nearly 4 days.
Member 10492119 16-Jun-14 7:33am    
Have started using Microsoft Debug Diagnostic tool and am seeing a lot of evidence that there is a memory leak in myodbc5a - there were nearly 500,000 allocations which were not 'free'ed
you need to identify the problems first.

For that it is useful to find some areas which are central code fields and comment ALL out and run the code. If that does solve the problem, activate step after step more details and you find the code which is the problem.

Check the Logs of the SQL-Server for finding some bugs in your database-connection code.

Have you lists or array with cached data somewhere?
 
Share this answer
 
For static code analysis, here's a list of tools: http://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis[^]

However, it may be that there is nothing wrong with your memory management per se - maybe it is just that your program isn't well designed and really uses that much memory. Since you obviously have a database application, the prime reason should be obvious: holding data from your database in memory! You should check all functions that read data and make sure they don't hold onto that data for longer than necessary. Maybe you can also find ways to reduce the required amount of memory.

Another potential problem is SQL joins: If you don't pay attention on how you read data from joined tables, then the amount of memory required to process a join of multiple tables can easily explode! Example: "Select * from persons, addresses, cars, insurances, bankaccounts where ..." even if the filter reduces the set of records to just a few, if each of the five involved has only 100 records, then the initial select joining 5 tables creates a virtual table with 10 billion records! You should strive to avoid that by using sub select statements that join tables one on one. Of course, that is highly hypothetical: a good database engine may recognize that and optimize accordingly - but depending on how exactly your sql statements look, you may need to optimize them manually.
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900