|
I am writing a small article about file enumeration, and in one of my CppUnit test I *discovered* that FindFirstFile/FindNextFile won't correctly report file attributes on FAT32 and FAT16. Specifically, FILE_ATTRIBUTE_TEMPORARY attribute is the problem here:
Here is what my code does:
SetFileAttributes(sTestFile.c_str(), FILE_ATTRIBUTE_TEMPORARY);
DWORD attr_1 = GetFileAttributes(sTestFile.c_str());
FindFirstFile(sTestFile.c_str(), &fd);
It took me nearly a day to realize that problem is due to File system and not my code. And now after spending hours going through on-line documentation I decided to ask here:
1. Is this documented behavior or a bug?
2. Am I doing something wrong in my code. Maybe it is something about temporary files that I failed to take into account.
Any help would be highly appreciated. I have tested this on two computers, both running XP, and I get same results on both.
Finally, here is a simple console project that demonstrates the problem:
#include "tchar.h"
#include "Windows.h"
#include <string>
#include <iostream>
using namespace std;
int _tmain(int argc, _TCHAR* argv[])
{
basic_string<tchar> sPath = _T("");
basic_string<tchar> sTestFile = _T("\\test_file.txt");
TCHAR root[256] = {0};
DWORD size = 256;
WIN32_FIND_DATA fd;
if (::GetCurrentDirectory(size, root) != 0)
sPath = root;
else return 1;
sPath += sTestFile;
CreateFile(
sPath.c_str(),
GENERIC_READ | GENERIC_WRITE,
FILE_SHARE_READ,
NULL,
CREATE_ALWAYS,
FILE_ATTRIBUTE_TEMPORARY,
NULL);
_tprintf(_T("\nA file %s is created...\n"), sPath.c_str());
_tprintf(_T("\nFile attributes (as reported by GetFileAttributes() ) are now: %x \n"), GetFileAttributes(sPath.c_str()));
_tprintf(_T("\nExplicitly set FILE_ATTRIBUTE_TEMPORARY attribute...\n"));
SetFileAttributes(sPath.c_str(), FILE_ATTRIBUTE_TEMPORARY);
_tprintf(_T("\nFile attributes (as reported by GetFileAttributes() ) are now: %x \n"), GetFileAttributes(sTestFile.c_str()));
FindFirstFile(sTestFile.c_str(), &fd);
_tprintf(_T("\nAccessing file with FindFirstFile()...\n"));
_tprintf(_T("\nFile attributes (as reported by FindFirstFile() ) are now: %x\n"), fd.dwFileAttributes);
DeleteFile(sPath.c_str());
_tprintf(_T("\n\n// copied from winnt.h\n"));
_tprintf(_T("//#define FILE_ATTRIBUTE_READONLY 0x00000001 \n"));
_tprintf(_T("//#define FILE_ATTRIBUTE_HIDDEN 0x00000002 \n"));
_tprintf(_T("//#define FILE_ATTRIBUTE_SYSTEM 0x00000004 \n"));
_tprintf(_T("//#define FILE_ATTRIBUTE_DIRECTORY 0x00000010 \n"));
_tprintf(_T("//#define FILE_ATTRIBUTE_ARCHIVE 0x00000020 \n"));
_tprintf(_T("//#define FILE_ATTRIBUTE_DEVICE 0x00000040 \n"));
_tprintf(_T("//#define FILE_ATTRIBUTE_NORMAL 0x00000080 \n"));
_tprintf(_T("//#define FILE_ATTRIBUTE_TEMPORARY 0x00000100 \n"));
_tprintf(_T("//#define FILE_ATTRIBUTE_SPARSE_FILE 0x00000200 \n"));
_tprintf(_T("//#define FILE_ATTRIBUTE_REPARSE_POINT 0x00000400 \n"));
_tprintf(_T("//#define FILE_ATTRIBUTE_COMPRESSED 0x00000800 \n"));
_tprintf(_T("//#define FILE_ATTRIBUTE_OFFLINE 0x00001000 \n"));
_tprintf(_T("//#define FILE_ATTRIBUTE_NOT_CONTENT_INDEXED 0x00002000 \n"));
_tprintf(_T("//#define FILE_ATTRIBUTE_ENCRYPTED 0x00004000 \n"));
return 0;
}
</tchar></tchar></iostream></string>
modified on Saturday, January 10, 2009 3:38 PM
|
|
|
|
|
Hi,
it has been a long time I used it, but here is what I remember:
1. CreateFile creates a file, opens it and returns a handle to it;
2. when you are done, you must call CloseHandle;
3. file data and metadata (such as last write time, and the flags) is only guaranteed to be committed
to disk when the file got closed.
Issue 3 is what allows different file systems to behave slightly differently.
Also I don't expect FindFile to keep track exactly of what is happening with open files.
Hence I suggest you set the files the way you want them, make sure they are all closed, and only then
observe in any way you see fit, including enumeration through FindFile.
|
|
|
|
|
Thank you for your answer, I guess I was to hasty to post my question (and the fact that I went to bed at 4:30 AM last night certainly didn't help). I forgot to say that I removed all error checking and file handles because it did not affect my problem, and I wanted a simplest/shortest possible test.
The same thing happens even if file handle is closed (I even tried calling ::Sleep(5000) right after CloseHandle, just to be sure (I was also desperate ).
Basically, my Unit test goes through a loop of all possible file attributes, it creates a new file, sets one Attribute and checks if my enumeration class finds the file. If it does, test is successful, and loops goes to test another Attribute.
And all is fine until FILE_ATTRIBUTE_TEMPORARY is tested, and only if I start tests on my USB drive (which is FAT32 based), on my hard drive test works fine.
You had a very good idea, but unfortunately that is not it 
Best regards,
loreia
|
|
|
|
|
Fair enough.
This page on CreateFile[^] holfd the following sentence:
"Specifying the FILE_ATTRIBUTE_TEMPORARY attribute causes file systems to avoid writing data back to mass storage if sufficient cache memory is available, because an application deletes a temporary file after a handle is closed. In that case, the system can entirely avoid writing the data. Although it doesn't directly control data caching in the same way as the previously mentioned flags, the FILE_ATTRIBUTE_TEMPORARY attribute does tell the system to hold as much as possible in the system cache without writing and therefore may be of concern for certain applications."
which seems to hint temporary files may be treated in a very lazy way; maybe you shouldn't really
worry too much if the only condition under which your tests fail is with the TEMP bit set.
|
|
|
|
|
Thanks again 
I've read that sentence like 300 times in last few days, and I must admit that I don't fully understand it. OK, English is my second language (third in fact) and I may be missing something, but this is how I see it:
When application sets FILE_ATTRIBUTE_TEMPORARY attribute this information *should* be written somewhere in file system, and GetFileAttributes and FindFirstFile *should* read the same thing from file system.
I don't understand why these two functions report different Attributes on FAT.
Now, I can implement my class in such a way that when searching for FILE_ATTRIBUTE_TEMPORARY attribute, my class would additionally call GetFileAttributes within FindNextFile block.
But I find this workaround a bit clumsy, and I was hoping to avoid it.
Best regards
loreia
|
|
|
|
|
Well, one way one could interpret the sentence is: a temporary file is private, it belongs to the process and will not survive the process, so once you decide to make a file temporary, there is no need for further file operations as long as it keeps holding the data for the owning process (so the only thing
it may have to write to disk is data that exceeds the cache capacity). Once the file gets closed, it
should be deleted (not sure by whom: the OS or the app, probably the OS, maybe only when the app
exits, but then your tests have finished too).
(I know you can disagree on all that, the file system should always tell the truth, etc)
You currently create the file as a regular file (despite your comment in the source), and later set the
TEMP flag; so the file gets created on disk as a regular file, and once you turn it into a TEMP file,
it may or may not reflect that on disk.
This is a little experiment you could do: create file, write data, make temp, show current time,
wait 10 seconds, write more data, close file, now use Explorer to see modified datetime: did it
update after you turned it into temp? if not, it supports the "TEMP means no valid metadata" hypothesis.
And maybe, just maybe, if you create it the start as a TEMP file right away, it will show that
everywhere, and later on you could clear the TEMP flag and all would be good...
|
|
|
|
|
Thanks, you last suggestion helped me better understand how TEMP files work.
When I run test program on my hard drive, "test_file.txt" is not created at all, Explorer simply doesn't show it. I can see file only when I run program on my USB drive. After realizing that, I re-read this sentence from MSDN:
"Specifying the FILE_ATTRIBUTE_TEMPORARY attribute causes file systems to avoid writing data back to mass storage if sufficient cache memory is available, because an application deletes a temporary file after a handle is closed. In that case, the system can entirely avoid writing the data."
So, TEMP file is held in memory as long enough cache memory is available. And after seeing that file is NOT deleted on my USB stick, I can verify that application is the one responsible for deleting file (unless FILE_FLAG_DELETE_ON_CLOSE is set in CreateFile)
Now, at least I fully understand what MSDN documentation says about FILE_ATTRIBUTE_TEMPORARY .
Off course, my problem still remains. GetFileAttributes and FindFirstFile *should* see the same set of file attributes, and should report the same DWORD value.
And BTW, you really have a "hawk eye", I meant to call CreateFile with FILE_ATTRIBUTE_TEMPORARY parameter but I made a mistake when copied source code here (in original that parameter is "*iter").
But it doesn't change much, no matter how I create file, once I set TEMP attribute, FindFirstFile chokes on it(on FAT32).
|
|
|
|
|
To complicate matters, I think removable devices such as USB sticks, work or may work slightly differently from regular disks; you (sometimes?) can set a device attribute that makes it safe
to remove them at all times, which tells me they can only cache on read, not on write; if you don't,
they may operate faster (on writes that is) but you officially need to "safely remove hardware"
telling Windows it should flush the cache to the device.
I don't always find how to control this feature though, it should be on one of the Device Property tabs but sometimes seems missing; I use both XP and Vista, and it may depend on the specific USB stick (some tend to install and use their own driver).
As a last remark: USB sticks really contain a microcontroller and a flash EEPROM, so there is software
involved there too; they get to execute FAT commands, so your TEMP problem could even be stick-specific!
If you ever discover the finer details on the TEMP matter, feel free to write and publish an article
on the subject!
|
|
|
|
|
This post brought smile to my face, I too am an electronics engineer and I program microcontrollers in my spare time (Microchip's PIC family) and my next project (if I ever find time to do it, that is) is to implement procedure to save data (collected from various sensors) on USB stick.
Back to topic...
I think Windows simply flushes data on USB stick because they are removable, and that is the only reason I can see the file with TEMP attribute. So it is probably not specific to my UBS stick.
But I was thinking along the same lines, so I tested on three different USB stick and two computers, and then I created a small FAT32 (and FAT) partition on my hard drive and tested it there too.
I got the same results everywhere, FindFirstFile identifies TEMP file only on NTFS partitions
|
|
|
|
|
loreia wrote: FindFirstFile identifies TEMP file only on NTFS partitions
FindFirstFile is returning the FAT attributes from reading the disk because it was designed to also work with 16-bit programs. FILE_ATTRIBUTE_TEMPORARY is an attribute which does not exist in the FAT specification[^] but does in NTFS. Would you expect to see FILE_ATTRIBUTE_ENCRYPTED on a FAT partition? No because only NTFS supports encryption. GetFileAttributes is different and checks internel objects such as the cache manager. There are *many* attributes in the GetFileAttributes Function[^] which are not present on a FAT based disk.
Best Wishes,
-David Delaune
|
|
|
|
|
David, thank you a lot for your answer.
I know from experience that FAT32 doesn't support encryption, so answer to your question is: no. LOL
So, TEMP attribute is NTFS specific, and it is not supported on FAT32. Thanks for clarifying that, but what I don't understand is why SetFileAttributes returns TRUE when setting TEMP attribute to a file on FAT32.
It SetFileAttributes function returned FALSE I wouldn't be in this mess in the first place.
My understanding is: if FAT32 doesn't support TEMP files, than SetFileAttributes should fail to set TEMP attribute.
I was wrong in presuming that "error" (for the lack of better description) was in FindFirstFile. But in fact FindFirstFile correctly reports that TEMP attribute is not set on FAT32 (as it cannot be set on FAT32).
And the real problem is in SetFileAttributes/GetFileAttributes functions that *claim* that file has a TEMP attribute set even on FAT32 (and this is in direct contradiction to FAT32 File System Specification that you posted a link for).
I you know why are SetFileAttributes/GetFileAttributes acting in this way, your explanation would help me a lot.
Best regards
loreia
|
|
|
|
|
loreia wrote: My understanding is: if FAT32 doesn't support TEMP files, than SetFileAttributes should fail to set TEMP attribute.
You need to seperate FAT from the operating system within your mind. When you call SetFileAttributes and set the file as FILE_ATTRIBUTE_TEMPORARY what happens internally is that the NT kernel calls CcInitializeCacheMap[^] and CcSetAdditionalCacheAttributes[^] which disables write-behind and essentially enables a "lazy write" which means that the file may or may not be written to disk depending on an internal algorithm based on critera such as i/o priority, filesize, media type ect...
The difference is that FAT32 does not have anywhere to store this extra information as it is an older technology. NTFS on the other hand was designed to store additional file attributes for both now and in the future as it can be extended.
FindFirstFile[^] has many other problematic surprises[^]. Some of which stem from supporting 16 bit applications.
GetFileAttributes is not lying to you when it says the file is FILE_ATTRIBUTE_TEMPORARY.
Best Wishes,
-David Delaune
|
|
|
|
|
What a great explanation, Now I fully understand what happened in my program.
Thanks a lot, I owe you one
Best regards,
loreia
|
|
|
|
|
OK, that seems to explain it all. Thanks.
|
|
|
|
|
GetFileAttributes() is used to retrieve a set of FAT-style attributes. See if GetFileAttributesEx() behaves any differently.
"Love people and use things, not love things and use people." - Unknown
"The brick walls are there for a reason...to stop the people who don't want it badly enough." - Randy Pausch
|
|
|
|
|
Hi David, thanks for your suggestion.
GetFileAttributesEx() won't help me. I had problems understanding why ::FindFirstFile and GetFileAttributes() report different set of file attributes on FAT32.
And now I get it, TEMP files are not flushed to HDD, and therefore limitations of FAT can be see with ::FindFirstFile and not with GetFileAttributes().
I made a small test in which I created a file on USB stick (with FAT32 file system), then added a TEMP attribute to it and:
a) read file attributes right after creating file
b) then I safely removed USB drive (forcing windows to flush file to FAT)
c) re-read file attributes, now both ::FindFirstFile and GetFileAttributes() reported correct values
Now I can safely continue developing my enumeration class.
One more time, a BIG thank you to all three of you for your help, suggestions and explanations.
Best regards
loreia
|
|
|
|
|
the same way as with a button? There is a static control in my dialog, and if it is clicked (single click) an action must be performed.
There is sufficient light for those who desire to see, and there is sufficient darkness for those of a contrary disposition.
Blaise Pascal
modified on Saturday, January 10, 2009 1:32 PM
|
|
|
|
|
The CStatic should have the BS_NOTIFY SS_NOTIFY style. If you are using MS Visual Studio then you can set this property in the dialog resource editor by setting the 'Notify' property to TRUE.
Some documentation:
STN_CLICKED Notification[^]
Best Wishes,
-David Delaune
[Edit]
Updated erroneous style.
|
|
|
|
|
Hi,
I tried what they said in MSDN:
The STN_CLICKED notification message is sent when the user clicks a static control that has the SS_NOTIFY style. The parent window of the control receives this notification message through the WM_COMMAND message.
I put SS_NOTIFY and BS_NOTIFY style on the static, but no WM_COMMAND was received when I clicked on it
There is sufficient light for those who desire to see, and there is sufficient darkness for those of a contrary disposition.
Blaise Pascal
|
|
|
|
|
BS_NOTIFY is for buttons, not statics, don't use it on a static, use only SS_NOTIFY. And you won't receive a command message you can handle using ON_COMMAND, you need to use ON_NOTIFY, specifying the ID (make sure this is not the default IDC_STATIC but something else, at least to avoid confusement) of your static and STN_CLICKED, or i think there should be an ON_STN_CLICKED macro somewhere too.
> The problem with computers is that they do what you tell them to do and not what you want them to do. <
|
|
|
|
|
sashoalm wrote: I put SS_NOTIFY and BS_NOTIFY style on the static
Sorry about that, I meant to say SS_NOTIFY. You also should change the ID of the CStatic to something other than the default. In your message map add:
ON_STN_CLICKED(IDC_STATIC_YOURNAME, &CYourClass::OnStnClickedStatic)
And in your header:
afx_msg void OnStnClickedStatic();
Best Wishes,
-David Delaune
|
|
|
|
|
Could there be something wrong with my static control? It's not working, and I checked that the id is not the default IDC_STATIC.
Here's how the static is defined in the *.rc2 file
LTEXT "Static",IDC_MY_STATIC_ID,81,198,109,8,
SS_CENTERIMAGE, SS_NOTIFY
I turned on the message loop tracing with MFC Tracer, but the only WM_LBUTTONUP and WM_LBUTTONDOWN messages were appearing when I was clicking on the static:
WndProc: hwnd=0x5B0BBA, msg = WM_LBUTTONDOWN (0x0001, 0x015700D0)
WndProc: hwnd=0x5B0BBA, msg = 0x036A (0x0000, 0x00000000)
WndProc: hwnd=0x5B0BBA, msg = 0x036A (0x0000, 0x00000001)
WndProc: hwnd=0x5B0BBA, msg = 0x036A (0x0000, 0x00000002)
WndProc: hwnd=0x5B0BBA, msg = 0x036A (0x0000, 0x00000003)
WndProc: hwnd=0x5B0BBA, msg = WM_LBUTTONUP (0x0000, 0x015700D0)
WndProc: hwnd=0x5B0BBA, msg = 0x036A (0x0000, 0x00000000)
WndProc: hwnd=0x5B0BBA, msg = 0x036A (0x0000, 0x00000001)
WndProc: hwnd=0x5B0BBA, msg = 0x036A (0x0000, 0x00000002)
WndProc: hwnd=0x5B0BBA, msg = 0x036A (0x0000, 0x00000003)
[Added]
I checked again and the hwnd is actually that of the dialog! The static control didn't receive any event in the first place.
There is sufficient light for those who desire to see, and there is sufficient darkness for those of a contrary disposition.
Blaise Pascal
modified on Saturday, January 10, 2009 1:28 PM
|
|
|
|
|
I found where the problem is, I had incorrectly written the flags. The dialog was in a *.rc2 file so it had to be edited manually.
Thanks, your solution works now
There is sufficient light for those who desire to see, and there is sufficient darkness for those of a contrary disposition.
Blaise Pascal
|
|
|
|
|
Glad you have located the problem, I was beginning to scratch my beard with absolute dismay.
Best Wishes,
-David Delaune
|
|
|
|
|
Be careful that you have to change the default id of static - IDC_STATIC to something else like IDC_STATIC_MY_LABEL. Or else the static will not notify any messages.
Regards,
Jijo.
_____________________________________________________
http://weseetips.com[ ^] Visual C++ tips and tricks. Updated daily.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
