How Can I Improve The Speed Of Clist.Removeall()

Question

0.00/5 (No votes)

See more:

I'm using templet CList to treat large data,but I find that the speed of CList.RemoveAll() is very slow when the data is large(about 100 millom),so please help me to improve the speed.
thank you!

Posted 29-Mar-15 22:32pm

hanxingde

Add a Solution

Comments

Kornfeld Eliyahu Peter 30-Mar-15 4:46am

If you remove all items, you may drop the whole object and create a new - empty - instance...

hanxingde 30-Mar-15 5:20am

then,how i can drop the whole object? if i do not use RemoveAll(),how can i free the the memory it take?

Jochen Arndt 30-Mar-15 7:07am

This would not help because the CList destructor calls RemoveAll().

Richard MacCutchan 30-Mar-15 5:03am

You cannot change the laws of physics. !00,000,000 items in a list will take 100,000,000 times a small finite time, which results in a larger time, to be removed. You may need to rethink your design.

hanxingde 30-Mar-15 5:27am

thank you for replying me,but i have no idea on treating large data,if i don't use CList,what can i instead of it?CArray? I think they all have no quick speed to delete so many data.can you help me in designing?

Richard MacCutchan 30-Mar-15 5:30am

The issue is not which control you use but how you manage your data. Holding 100,000,000 items in memory at any one time will consume large amounts of system resources and slow things down. You have to find a way of breaking your data into smaller sets.

hanxingde 30-Mar-15 5:44am

very good idea!
if i do not load so many data into the memory at the same time,how can i to treat them continously, for example, filtering,drawing curve quickly. when the user click "next page",i must read the corresponding data into memory from a large file,then treat according to some algorithm,then draw on the screen,i think it will take unacceptable time.i want to know how you process similar thing? thank you！

Richard MacCutchan 30-Mar-15 6:02am

I am sorry, but it is not possible to answer such a question in a Quick Answers forum. You will need to spend some time studying how your program operates, and where you can make changes to speed things up. It may be useful to have a separate thread to load the data while the main thread is doing other work. But I know nothing about your program so I would not know which is the best option.

hanxingde 30-Mar-15 6:41am

thank you for your reply!it give me some idea for this problem

Member 11547038 30-Mar-15 6:02am

I donot know what is your program for. But if it has lot's of page to load. Can you cut the data of page into pieces and load some of them at one time. For example: when you read page 7, maybe you can load page 6,7,8,9. Then you can go to next/forward page quickly. And when you change page, load a new page and delete a old page at the same time.

Hope this may give you some idea.

hanxingde 30-Mar-15 6:42am

thank you! your reply really give me some idea.

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Jochen Arndt · Answer 1 · 2015-03-30T01:20:00

Having a look at the implementation shows that CList nodes are stored in CPlex blocks. These blocks are allocated using the block size parameter passed to the CList constructor[^].

Using a larger block size value will result in less memory allocations and de-allocations and will therefore reduce the execution time. If the maximum number of items is known in advance, that can be passed to have only one big memory allocation.

However, the RemoveAll implementation will still iterate through all nodes which took some time.