Click here to Skip to main content
14,241,320 members

C / C++ / MFC

 
GeneralRe: _findfirst and fopen very slow Pin
David Crow31-Jan-19 6:14
memberDavid Crow31-Jan-19 6:14 
QuestionRe: _findfirst and fopen very slow Pin
jeron131-Jan-19 4:58
memberjeron131-Jan-19 4:58 
AnswerRe: _findfirst and fopen very slow Pin
cristiapi31-Jan-19 5:20
membercristiapi31-Jan-19 5:20 
QuestionRe: _findfirst and fopen very slow Pin
David Crow31-Jan-19 6:04
memberDavid Crow31-Jan-19 6:04 
AnswerRe: _findfirst and fopen very slow Pin
cristiapi31-Jan-19 6:10
membercristiapi31-Jan-19 6:10 
GeneralRe: _findfirst and fopen very slow Pin
jeron131-Jan-19 8:28
memberjeron131-Jan-19 8:28 
GeneralRe: _findfirst and fopen very slow Pin
cristiapi1-Feb-19 5:57
membercristiapi1-Feb-19 5:57 
AnswerRe: _findfirst and fopen very slow Pin
Randor 31-Jan-19 14:23
professional Randor 31-Jan-19 14:23 
Hi,

Some comments:

1.) Reading files may be slightly faster if you read using a multiple of the drive sector size or read it all at once.

2.) If you've ever wondered why file-backed cache implementations save files into a hierarchical folder structure it's because enumerating 10,000 files in a single folder may cause a small performance hit. If you plan on storing many thousands of files you may want to design a folder structure. Maybe something simple such as alphabetical A-C, D-F ... or something based on timestamp. This is not much of an issue on a modern SSD but old spindle drives take a performance hit.

3.) The code you have shown above is reading the file contents into a local buffer. You would get a huge performance boost by using the MapViewOfFile function to map the file directly into your process space.

Have a look at the Creating a View Within a File sample. This sample is demonstrating how to take a large file and map only 1kb at a time into your process. Don't do that.

You stated that your files are around ~40 kb so I'd recommend mapping the entire file into your process address space. I'd also recommend using two file mappings. While FileA is being processed you can have the operating system map FileB into your process. This would mitigate any latency caused by the i/o subsystem.

The majority of your latency is between opening files. I highly recommend the second file mapping.

Best Wishes,
-David Delaune
GeneralRe: _findfirst and fopen very slow Pin
cristiapi1-Feb-19 0:29
membercristiapi1-Feb-19 0:29 
AnswerRe: _findfirst and fopen very slow Pin
leon de boer31-Jan-19 14:54
memberleon de boer31-Jan-19 14:54 
GeneralRe: _findfirst and fopen very slow Pin
Stefan_Lang31-Jan-19 21:36
memberStefan_Lang31-Jan-19 21:36 
GeneralRe: _findfirst and fopen very slow Pin
leon de boer1-Feb-19 4:42
memberleon de boer1-Feb-19 4:42 
GeneralRe: _findfirst and fopen very slow Pin
Stefan_Lang1-Feb-19 4:59
memberStefan_Lang1-Feb-19 4:59 
GeneralRe: _findfirst and fopen very slow Pin
cristiapi1-Feb-19 5:44
membercristiapi1-Feb-19 5:44 
GeneralRe: _findfirst and fopen very slow Pin
cristiapi1-Feb-19 6:24
membercristiapi1-Feb-19 6:24 
GeneralRe: _findfirst and fopen very slow Pin
leon de boer1-Feb-19 21:40
memberleon de boer1-Feb-19 21:40 
GeneralRe: _findfirst and fopen very slow Pin
cristiapi1-Feb-19 23:35
membercristiapi1-Feb-19 23:35 
GeneralRe: _findfirst and fopen very slow Pin
leon de boer2-Feb-19 0:33
memberleon de boer2-Feb-19 0:33 
AnswerRe: _findfirst and fopen very slow Pin
Stefan_Lang31-Jan-19 21:27
memberStefan_Lang31-Jan-19 21:27 
AnswerRe: _findfirst and fopen very slow Pin
Joe Woodbury1-Feb-19 5:36
professionalJoe Woodbury1-Feb-19 5:36 
AnswerRe: _findfirst and fopen very slow Pin
cristiapi2-Feb-19 1:13
membercristiapi2-Feb-19 1:13 
Questionwrite in .xlsx extension excel file using c++ Pin
Member 1413460229-Jan-19 23:49
memberMember 1413460229-Jan-19 23:49 
AnswerRe: write in .xlsx extension excel file using c++ Pin
Richard MacCutchan29-Jan-19 23:52
protectorRichard MacCutchan29-Jan-19 23:52 
AnswerRe: write in .xlsx extension excel file using c++ Pin
leon de boer30-Jan-19 2:09
memberleon de boer30-Jan-19 2:09 
RantC++ Typedefs Pin
nitrous_00729-Jan-19 4:45
membernitrous_00729-Jan-19 4:45 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.