How to allocate a large continuous memory ?

Question

1.00/5 (1 vote)

See more:

Hi,
I have to allocate a large continuous memory, for instance, 200M, using "new char[1024*1024*200]". The available memory is enough, but may be divided into tiny patches. That results the invoke of new throws a exception. Is there a solution can I adopt ?
Thanks for your attention.

Posted 23-Dec-14 16:40pm

cxxxxf

Add a Solution

Comments

Sergey Alexandrovich Kryukov 23-Dec-14 22:57pm

Why do you think ig should be "divided into tiny patches" (fragmented). If you allocate it all at once, it depends on what you do with it. It is either successful or not. In this way, the question does not seem to make a lot of sense.
—SA

chandanadhikari 24-Dec-14 1:33am

can you please show some code and what exception 'new' is throwing .
maybe this can help :: http://www.codeproject.com/Articles/27487/Why-to-use-memory-pool-and-how-to-implement-it

CPallini 24-Dec-14 12:18pm

What system are you using? As far as I know, on modern OSs memory is allocated in pages. A contiguous block in process space isn't necessarily mapped to a contiguous block of physical memory.

2 solutions

Solution 1

Things you can try if you're sure that something's fragmenting the heap (i.e. you're allocating loads of objects and only deleting half of them):

- check that something about the OS isn't knackering the allocation (e.g. 32 bit address space just isn't big enough; Windows habit of just in time expanding the swap file and the allocation timing out) and fix that if possible

- don't dynamically allocate your array; make the data item a static or global

- do large allocations like this before you do anything else in main()

- try the code on another compiler/OS combination (a different address space layout might pay off or give you a hint as to why it's not working on the target)

- use something like VirtualAlloc on Windows

- create and use a memory mapped file backed by the swap file on windows

If that lot run out come back and ask again!

Posted 23-Dec-14 22:01pm

Aescleal

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Philippe Mori · Accepted Answer · 2014-12-24T05:30:00

The first question is do you really have to have such continuous big array. Why not using 200 array of 1 MB for example?

200M is a big array for 32 bit application on computer that does not have much RAM etc.

As far as I know large allocation are forwarded to the system and thus does not dépends much on the compiler but the OS. Fragmentation occurs when memory is allocated and unallocated repetively.

If you are using STL, you can use deque<> if you only want the memory to appears consecutive in code but does not the data itself be consecutive. Otherwise as mentionned in Solution 1, memory mapped file might be another solution.

By the way, your question does not give us enough details to give the best possible answer in your case.