C / C++ / MFC

24-Apr-04 20:11

<br />
typedef struct{<br />
  char buff[20];<br />
} my_struct;<br />
<br />
my_struct my_arr[1000][30][10][10][3];<br />
<br />
int main(){    <br />
  for ( int a = 0; a < 1000; ++a )        <br />
  for ( int b = 0; b < 30; ++b )            <br />
  for ( int c = 0; c < 10; ++c )                <br />
  for ( int d = 0; d < 10; ++d )                    <br />
  for ( int e = 0; e < 3; ++e )                    <br />
  {                        <br />
    my_arr[a][b][c][d][e].buff[0]='0';<br />
    my_arr[a][b][c][d][e].buff[19]='9';<br />
  }<br />
  return 0;<br />
}<br />

In previous thread, I have asked about the problem when using large data. Thanks so much for all of your replying.

I based on yours comment to check my code again and I found that I had a mistake. Yes, the program run normally.

Here I had changed code to another type:

<br />
typedef struct{<br />
  CByteArray buff;<br />
} my_struct;<br />
<br />
typedef struct{<br />
  my_struct dat[30][10][10][3];<br />
} data;<br />
<br />
CArray<data, data&> my_arr;<br />

In CWinApp::InitInstance():

<br />
  int n=1000<br />
  my_arr.SetSize(n);<br />
  for ( int a = 0; a < n; ++a )        <br />
  for ( int b = 0; b < 30; ++b )            <br />
  for ( int c = 0; c < 10; ++c )                <br />
  for ( int d = 0; d < 10; ++d )                    <br />
  for ( int e = 0; e < 3; ++e )                    <br />
  {                        <br />
    my_arr[a].dat[b][c][d][e].buff.SetSize(1);<br />
  }<br />
}<br />

I tested n from 100 to 1000 and CArray seem to be not effective. When n>=500, the program run more slowly. And when n=1000, the system alerted that out of virtual memory.
Please note that the buff size is only 1 (beside 20 in previous version)

In my project, buff size is varying.
n is about 300-800
All remaining size of array are const

Here are my questions:

In this case, which is my best solution (Using CArray or not)?

If anyone has experience in using CArray and processing large data, please give me a hint.

Thank you very much.

Mike Nordell25-Apr-04 6:19

Mike Nordell

25-Apr-04 6:19

nguyenvhn wrote:

typedef struct{
CByteArray buff;
} my_struct;

typedef struct{
my_struct dat[30][10][10][3];
} data;

CArray<data, data&> my_arr;

From a cursory glance at MFC code, and unless I made a severe mistake:

You are aware that every element in my_arr is of type data, which is holding nine thousand CByteArray's? CByteArray's which each are of size 20 bytes and requires construction, including construction of parent class object CObject? So just one single instance of data at minimum requires 180.000 bytes.

For 'n' in your code you are:

1. Creating m = n * 9000 CByteArray's, for a total of at least m * 20 bytes.

2. Iterating over those m ByteArrays, calling SetSize for them, which allocates at least 16 (or is it 32, I really don't remember the allocation granularity of the SBH) bytes each, for a size of m*16 or m*32 bytes.

This becomes m*(20+16) or m*(20+32) bytes, minimum.

If n=1, you allocate at least 324.000 bytes.

If n=1000, you allocate at least 324.000.000 bytes.

Should SBH allocation granularity be 32, then you would get a memory load of about 446MB just to hold those nine million bytes. Should you use the debug heap, I think you can safely add another m*16 or m*32 bytes (for a memory load between 446 and 720MB respectively).

Just to create the nine million *holders* of the data, not the data itself, for the n=1000 case, you use 180 million bytes.

In my project, buff size is varying.

I suggest you at minimum should stop using MFC for collections (CByteArray even have a vfptr!), and start using a custom allocator.

In this case, which is my best solution (Using CArray or not)?

Not.

To be completely honest I'd say using any MFC collection class is almost always the wrong thing, but that's a completely different discussion.

nguyenvhn25-Apr-04 21:23

25-Apr-04 21:23

Thank you very much.

I have no knowledge about custom allocator. Could you tell me more about it?

In addition, Could you give me an article to make more clearly that "using any MFC collection class is almost always the wrong thing"? And so that, what is the true replacement? Please, I am using Visual C++ and MFC as the main programming framework.

I hope to get lot of useful information from you.
Thanks again.

Mike Nordell26-Apr-04 4:14

Mike Nordell

26-Apr-04 4:14

nguyenvhn wrote:
Could you give me an article to make more clearly that "using any MFC collection class is almost always the wrong thing"?

Once you gain more experience in C++, It'll come to you.

But to give a starting place, you might want to Google some news.

And so that, what is the true replacement?

There is no silver bullet.

Paul Ranson25-Apr-04 6:45

Paul Ranson

25-Apr-04 6:45

I don't remember anything about the MFC collection classes, and I plan for it to remain that way.

Can you tell us something about your application?

Paul

nguyenvhn25-Apr-04 21:38

25-Apr-04 21:38

Thanks for your replying.

In before, my application has done well. But now the size of data is increase very high. My application must deal with its but I dont want to write new application from scratch. I want to update from old source code.

In old version, I used the following declare:

my_struct my_arr[C][W][D][P][S] with my_struct's size is 20 bytes as mention in previous thread.

C, W, D, P, S are const and C=120.

Now C can be 1000. I think my data structure may be too large.

Could you give me any idea or example to solver this kind of problem?

Thank you.

Paul Ranson26-Apr-04 1:16

Paul Ranson

26-Apr-04 1:16

If your 'my_struct' can remain 20 bytes then the following works for variable 'C', I don't recommend any type of container since the overhead of managing it and implementing 'operator[]' is probably going to be significant.

The total allocation is 180MB, this is much less than commonly fitted as main RAM to modern PCs so the quantitiy per se isn't too much. You do need to consider how you access it in order to work with the CPU data caches rather than against them, but without knowing what you do to the data I can't help there.

struct my_struct
{
    char buff [ 20 ] ;
} ;

typedef my_struct my_struct_array [ 30 ][ 10 ][ 10 ][ 3 ] ;
const unsigned my_arr_size = 1000 ;

int main()
{
    my_struct_array * my_arr = new my_struct_array [ my_arr_size ] ;

    for ( unsigned a = 0; a < my_arr_size; ++a )
        for ( unsigned b = 0; b < 30; ++b )
            for ( unsigned c = 0; c < 10; ++c )
                for ( unsigned d = 0; d < 10; ++d )
                    for ( unsigned e = 0; e < 3; ++e )
                    {
                        my_arr [ a ][ b ][ c ][ d ][ e ].buff [ 0 ] = '0' ;
                        my_arr [ a ][ b ][ c ][ d ][ e ].buff [ 19 ] = '9' ;
                    }
    delete [] my_arr ;

    return 0;
}

Paul

nguyenvhn26-Apr-04 15:16

26-Apr-04 15:16

So good. I found an idea to solver my problem based on your code.
Thank you very much.
I will post more thread in this forum. If you see, help me.

Hash Table

Member 104413524-Apr-04 20:04

Member 1044135

24-Apr-04 20:04

hay all

can any one tell me What is hash table how can i use it in a clint server aplication

Garth J Lancaster24-Apr-04 21:38

Garth J Lancaster

24-Apr-04 21:38

I suggest :-

1) learn how to spell/type

2) following this link http://www.brpreiss.com/books/opus4/html/page203.html[^]to learn about hash tables in general

3) doing some google searches to get the basic questions out of the way.

The answer to Princess Anastasia wrote:
how can i use it in a clint server aplication is 'boundless' - since you provide no useful information that would allow someone here to help you.

There are also client-server examples here on CP, try looking for them and see if they meet the criteria you're looking for.

4) most people won't respond to your question because it sounds like a 'homework'/'assignment' question - if you do have such, and want an answer, then try posting some code that proves you've at least attempted something for yourself.

Sorry to be harsh

Garth

Prakash Nadar24-Apr-04 22:39

Prakash Nadar

24-Apr-04 22:39

Good Reply. 5!

God is Real, unless declared Integer.

DougW4824-Apr-04 21:50

DougW48

24-Apr-04 21:50

A Hash Table is a data structure where you input elements by a key value. Even though the other response you got was horribly rude and unwarranted, he was right that you should probably look it up to determine how you might want to write this table. They can be as simple as an array, and as complex as multi-dimensional arrays with lists, or whatever. If you plan on putting a lot of elements into your table, you should probably read up on MSDN about efficiency in design. If you use dynamic memory in your hash table (which you probably will), then you will most likely be causing the operating system to page fault as it passes between links. If this is the case, then while a hash table is supposed to be fast, it will actually be no better than a linked list.

After you get a hash structure, you can use it to store socket connections. You've probably got a lot of reading to do to be ready to design this application.

And ignore it when you get somebody responding like you've done something wrong by asking a question. Some people forget that these forums are set up for asking questions and getting advice.

Garth J Lancaster25-Apr-04 1:08

Garth J Lancaster

25-Apr-04 1:08

I'm sorry if you think my reply was rude and unwarranted - I'm not an a**ehole, despite appearances, and, I definately havnt forgotten that these forums are set up for asking questions and getting advice (else how myself would I have learnt anything, except by the generosity of the people here)

rude - no, I was more polite than a lot of people can be - you should see the typical reply to some such questions - at least I gave a pointer to some info on hash tables - good work from you to point out how they may be used in a client-server situation.

unwarranted - depends on how many times you've done someones homework for them I guess - Im quite prepared to help people, when someone has tried to do some pre-reading, and info on hash tables per se is available all over the net ..

I'd prefer to try to teach a person how to fish, rather than spoon feed them.

I'll accept your criticism - maybe next time my response will be different.

may your god go with you.
Garth