C / C++ / MFC

Neville Franks26-Dec-02 13:13

26-Dec-02 13:13

I'm a bit surprised you are seeing a big difference. Are you opening and closing the file each time? If so that would have a big hit. What about doing a big dummy read to get the entire file cached.

If you *could* rewrite the code then I'd suggest trying memory mapped files.

Neville Franks, Author of ED for Windows. www.getsoft.com
Make money with our new Affilate program

C^hris L^osinger26-Dec-02 13:30

26-Dec-02 13:30

Neville Franks wrote:
I'm a bit surprised you are seeing a big difference

in some cases, the difference is over 12x. CreateFile etc is essentially unusable, but it has the feature that you can pass a HANDLE out of a DLL, while you can't pass a FILE* out.

Neville Franks wrote:
Are you opening and closing the file each time?

no. it just the fact that i'm lots of little reads/writes (parsing images, etc).

-c

I'm not the droid you're looking for.

Neville Franks26-Dec-02 13:46

26-Dec-02 13:46

Chris Losinger wrote:
while you can't pass a FILE* out.

why not?

So the DLL code isn't yours and thus can't be changed or it is yours and you don't want to change it because ...

Neville Franks, Author of ED for Windows. www.getsoft.com
Make money with our new Affilate program

C^hris L^osinger26-Dec-02 13:50

26-Dec-02 13:50

Neville Franks wrote:
why not?

because the DLL and the EXE will be using different copies of the CRT.

-c

I'm not the droid you're looking for.

Neville Franks26-Dec-02 13:57

26-Dec-02 13:57

Chris Losinger wrote:
because the DLL and the EXE will be using different copies of the CRT.

So? I assume the DLL opens the file with fopen, the returned FILE* points to memory in the .EXEs heap, in which case it can happilly use it. Or am I missing something fundamental here.

Neville Franks, Author of ED for Windows. www.getsoft.com
Make money with our new Affilate program

C^hris L^osinger26-Dec-02 14:16

26-Dec-02 14:16

Neville Franks wrote:
Or am I missing something fundamental here

each CRT copy has its own table of file handles, these are not shared between modules. so if you fopen a file in a DLL and get a FILE * of "53", that will probably not correspond to anything meaningful in your EXEs file handle table.

the HANDLE that comes from a CreateFile, however, is maintained by the OS, not the CRT, so you can share that between modules without problems.

-c

I'm not the droid you're looking for.

Neville Franks26-Dec-02 14:39

26-Dec-02 14:39

I've looked in the VC++ fopen docs and can't see anything about each CRT instance getting it's own table. Looking at the CRT code it uses _malloc_crt() to allocate a stream object.

I don't understand why you would be using the file handle anyway. Surely you just pass the FILE* back to some DLL function to read/write. And if the CRT is in a DLL why is their another "handle table" in the .EXE.

Have you actually proved there is a problem with fopen et.all.?

Neville Franks, Author of ED for Windows. www.getsoft.com
Make money with our new Affilate program

C^hris L^osinger26-Dec-02 15:13

26-Dec-02 15:13

Neville Franks wrote:
Have you actually proved there is a problem with fopen et.all.?

yes, a 2 minute DLL project can prove this:

in a DLL:

typedef struct strct_t
{
   FILE *fp;
} strct;

void __stdcall OpenAFile(strct * theStruct)
{
   theStruct->fp = fopen("c:\\autoexec.bat", "r");
}

in the EXE:

strct s;
OpenAFile(&s);
fclose(s.fp);

it crashes in the fclose.

Neville Franks wrote:
Looking at the CRT code it uses _malloc_crt() to allocate a stream object.

and then it associates one of the entries in the __piob array to the stream. this array is global to the CRT instance (look in crt\src\_file.c).

a quick skip through Google will show dozens of instances of this problem. and the answer is always "use CreateFile" or "link to a shared CRT". unfortunately, the second is not an option for me, either.

the problem is really: how to make ReadFile/WriteFile behave in a manner worthy of being the low-level file operation on a modern OS?

-c

I'm not the droid you're looking for.

Neville Franks26-Dec-02 15:50

26-Dec-02 15:50

Chris Losinger wrote:
"link to a shared CRT". unfortunately, the second is not an option for me, either.

I missed the shared CRT bit. Guess I lead you down the wrong track. Sorry about that.

Neville Franks, Author of ED for Windows. www.getsoft.com
Make money with our new Affilate program

Matt Gullett26-Dec-02 15:49

Matt Gullett

26-Dec-02 15:49

Have you tried using the FILE_FLAG_SEQUENTIAL_SCAN flag?

fileHandle = CreateFile(pFileName, GENERIC_READ,
 FILE_SHARE_READ, NULL, OPEN_EXISTING,
 FILE_ATTRIBUTE_NORMAL | FILE_FLAG_SEQUENTIAL_SCAN, NULL);

C^hris L^osinger26-Dec-02 19:00

26-Dec-02 19:00

i tried it. but there was no real difference (i stopped it at the 10x point, instead of letting it get to the 12x mark).

-c

I'm not the droid you're looking for.

Michael Dunn26-Dec-02 18:52

Michael Dunn

26-Dec-02 18:52

I took a one-minute glance through fwrite() and it does its own buffering, thus explaining the speed difference. If you can pre-calculate the amount of data you'll be writing, and that amount isn't huge, put it all into a memory-mapped file. That way you can use the same APIs but everything will be in memory.

--Mike--
If it doesn't move and it should: WD-40. If it moves and it shouldn't: duct tape.

1ClickPicGrabber - Grab & organize pictures from your favorite web pages, with 1 click!
My really out-of-date homepage
Sonork-100.19012 Acid_Helm

C^hris L^osinger26-Dec-02 18:59

26-Dec-02 18:59

but i would also have to duplicate the buffering for seek, tell, read, eof, etc.. it'd be far far far too much work (which means more chances for bugs).

too bad.

Frown | :(

-c

I'm not the droid you're looking for.

Mike Nordell27-Dec-02 1:22

Mike Nordell

27-Dec-02 1:22

Chris Losinger wrote:
fwrite, fread and friends are easily many times faster than WriteFile, ReadFile and friends, when doing lots of small reads and writes.

Expected. The Win32 functions need to (among other things) do user->kernel->user mode transitions, which is horribly slow in IA32. The stdio functions perform buffering all in user-mode (no context switch until the buffer is full), and you can even increase the buffer with setvbuf (disregard any Micros~1 documentation lying to you that you can only use 32KB buffers - the buffer can be all the memory you can give it. Probably they haven't bothered to update the documentation from the ~1990 MS-DOS CRT limits).

If you are only given a file HANDLE (from e.g. a DLL) and want faster stdio functions, you should have a look at _open_osfhandle. That should put you on the right track.

C^hris L^osinger27-Dec-02 6:27

27-Dec-02 6:27

Mike Nordell wrote:
_open_osfhandle

thanks for the tip.

open_osfhandle gives me a low-level, non-buffered, I/O handle. so, that doesn't buy me anything. i can use fdopen on that handle to get a stream I/O handle, but i haven't figured out how to detach the stream handle from the I/O handle when i'm done.

it's a nightmare Smile | :)

-c

I'm not the droid you're looking for.

Controls on an MDI App

Steven M Hunt26-Dec-02 11:51

Steven M Hunt

26-Dec-02 11:51

I created a NON doc/view MDI application using the MFC ClassWizard (or whatever it's called in .NET) and I don't know where to place the code for some edit fields I want to put in the MDI Child Windows. I put it in the OnCreate() function but it didn't work. Help!

-- Steve

LIB from DLL

Hockey26-Dec-02 11:30

Hockey

26-Dec-02 11:30

Is it possible to generate a lib file from a DLL? I downloaded a DLL and it only comes with a def file. I would like to use the DLL the easy by including the lib for the DLL at compile time?

I could recompile the DLL source, but would prefer to use some tool to generate a lib file from the existing DLL

Thanks! Smile | :)

"An expert is someone who has made all the mistakes in his or her field" - Niels Bohr

Neville Franks26-Dec-02 13:06

26-Dec-02 13:06

If you've got a DSP for it then add this to your project and make it a dependancy of the project. That's the easiest way. VC++ will do the rest.

I think there is a utility to create a lib, but I can't think what it is just now.

Neville Franks, Author of ED for Windows. www.getsoft.com
Make money with our new Affilate program

Hockey26-Dec-02 17:35

Hockey

26-Dec-02 17:35

Neville Franks wrote:
If you've got a DSP

Nope Frown | :(

"An expert is someone who has made all the mistakes in his or her field" - Niels Bohr

Neville Franks26-Dec-02 18:51

26-Dec-02 18:51

Hockey wrote:
Neville Franks wrote:
If you've got a DSP

Nope

Should be easy enough to just create a nice new one and whack all the files into it.

Neville Franks, Author of ED for Windows. www.getsoft.com
Make money with our new Affilate program

C^hris L^osinger26-Dec-02 15:15

26-Dec-02 15:15

IMPLIB used to do it for Win16.

maybe this can help:

http://www.geocities.com/SiliconValley/5806/download.htm#IMPLIB

-c

I'm not the droid you're looking for.

Hockey26-Dec-02 17:37

Hockey

26-Dec-02 17:37

Cool...i'll check it out

Thanks! Smile | :)

"An expert is someone who has made all the mistakes in his or her field" - Niels Bohr