Problem with writing files on disk

Question

2.50/5 (4 votes)

See more:

I should write 1024 files on disk. The content of files are a simple counter. When I write for example 64 files, every thing is OK but when the number of file increases, the content of file are corrupted. The procedure is as follow:
A thread reads buffers with 32KB length from a FIFO. FIFO is implemented using Link List. In this thread, the read data is written on disk. The write procedure is as follow:

char path[MAX_PATH+1];
char ch_itoa[8];
strcpy(path, "d:\\counter\\");
itoa(class_pointer->m_startWriteLL->index, ch_itoa, 10);
strcat(path, ch_itoa);
strcat(path, ".bin");

HANDLE fHandle;
fHandle = CreateFile(CA2W(path), GENERIC_READ | GENERIC_WRITE, 0, NULL, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
DWORD dwPos = SetFilePointer(fHandle, 0, NULL, FILE_END);
DWORD dwBytes = class_pointer->m_startWriteLL->size;
bool bRet = LockFile(fHandle, dwPos, 0, dwBytes, 0);
WriteFile(fHandle, class_pointer->m_startWriteLL->data, dwBytes, &dwBytesWritten, NULL);
bRet = UnlockFile(fHandle, dwPos, 0, dwBytes, 0);
CloseHandle(fHandle);

At first I open appropriate file and then seek to the end of file, then I write data (32 KB) to the file and finally I close file. At next time, for writing new data to the file, this procedure is repeated. All of 1024 files are written as above. As I mentioned, when the number of files is little, there is no problem but when I should write data to 1024 files, files are corrupted. I should mention that at each time, just one file is opened for writing. Can any body say me why there is corruption? What is the solution?

[edit]Code block added - OriginalGriff[/edit]

Posted 17-Apr-11 20:28pm

eh.azimi

Updated 25-Apr-11 21:06pm

OriginalGriff

v4

Add a Solution

Comments

OriginalGriff 18-Apr-11 2:34am

Without seeing your code? No.
Edit your question (using the "improve Question" widget) and post the relevant code fragment.

Sergey Alexandrovich Kryukov 18-Apr-11 2:57am

Not unless you show the offending code.--SA

[no name] 18-Apr-11 4:50am

Use std::list<cstring> to store file names.
It seems to me, that you are using limited buffer to store names or content of files.

3 solutions

Add a Solution

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

NuttingCDEF · Answer 1 · 2011-04-25T21:58:00

Solution 2

A few thoughts / questions that might help resolve this.
1. What platform / operating system is this running on? What development environment are you using?
2. What sort of device is your D-drive (fixed hard disc, memory stick, . . .)
3. What file system is it formatted with (NTFS, FAT, HFS, . . .)
4. When you say 'corrupted', can you describe what you expect / what you get / how they differ?
5. Have you explored exactly how many files you can write before corruption occurs? Is it always exactly the same number? If you exceed that number is it only files after that point that are corrupted or is it then all files written?
6. Do all 1,024 files you want to write already exist or is it a problem that only occurs when new files are created (or only when existing files are appended to)?
7. Are you sure that the problem is the file writing and not in the data you're trying to write?

Posted 25-Apr-11 21:58pm

NuttingCDEF

Comments

Niklas L 26-Apr-11 4:08am

5 for your remote debugging abilities :)

eh.azimi 26-Apr-11 4:43am

Thanks for your message.
1. OS is Win server 2003 32 bits. I use ATL technology on .Net 2008.
2. Fixed hard disk.
3. NTFS
4. Corruption means gap between data. It is almost periodic in each 32KB. Because the nature of data, The content should parse from 0 to 255 in a periodic way. Corruption causes jump from a value (for example 125) to an unexpected value (for example 200 instead of 126).
5. It is between 128 to 256. I don't know exact number.
6. Sorry, I don't understand your mean.
7. yes. I'm sure.

NuttingCDEF 26-Apr-11 5:34am

6. You are writing to files something like 0.bin, 1.bin, . . . 1023.bin. Before you start, do all these files exist? Or only some of them? Or none? Presumably any that don't already exist are created by your code. Which ones are corrupted when you write to them? The ones that already existed? The ones that are newly created? Some mixture of existing / new? All the files?

4. Are you saying that when writing each single file, the data in that one file should just be an increasing sequence of values - but sometimes it jumps unexpectedly. Is that within a single cycle of writing to each file? Or are you cycling through the set of files multiple times with the corruption happening at the boundary between cycles?

5. Can you get a bit more info on when you get corruption, which files it affects (e.g. 0.bin to 99.bin OK, but 100.bin to 1023.bin corrupted)? Which file does the corruption start at? Or looking at the files in sequence are the corrupted ones mixed up with the good ones?

Debojyoti Majumder · Answer 2 · 2011-04-25T21:26:00

Well, if the code running in parallel there are so many possibilities of this kind of error.

You can try for these things.

1.WaitForSigleObject(fHandle) after WriteFile()
2.Check the return status of the API's
3.Log dwBytes and check they are equal to 32 o not.

Hope these things might help you.

Niklas L · Answer 3 · 2011-04-25T22:06:00

Solution 3

From the posted code, the most probable cause is your linked list being corrupted. If I were you, I would use the debugger to check that the data in the list is correct, on write or read, which ever is the easiest.

Posted 25-Apr-11 22:06pm

Niklas L

Comments

eh.azimi 26-Apr-11 4:44am

Link list is OK. I checked it.

Problem with writing files on disk

3 solutions

Solution 2

Solution 1

Solution 3

Add your solution here

Preview 0

Problem with writing files on disk

3 solutions

Solution 2

Solution 1

Solution 3

Add your solution here

Preview 0

Existing Members

...or Join us