![]() |
Desktop Development »
Files and Folders »
General
Intermediate
License: The Code Project Open License (CPOL)
XFile - Extending the Win32 File API for Server ApplicationsBy Hans DietrichXFile extends the Win32 file functions with a non-MFC class that includes functions to implement file rollover, file shrinking, file compare, buffered writes, mapped file reads, zipping, and automatic file size limits. |
VC6Win2K, WinXP, MFC, Dev
|
|
Advanced Search Add to IE Search |
|
|
|
||||||||||||||||
CXFile represents a collection of file-handling code snippets I have put together over a few years working on NT client/server systems. My main reason for writing these functions was to make it easier to implement such systems - I wanted consistent interfaces, plenty of diagnostics about what was going on, and extensions that go beyond anything in the Win32 API. I also wanted to make it independent of MFC, because some older systems I work on use only Win32 SDK.
When I started pulling together different pieces of code for CXFile, one of the first things I realized was that I would have to do something about all the TRACE statements that were scattered throughout the code. I am a big fan of using TRACE statements as a way to monitor actual system operation, but I was afraid I would have to yank all TRACE statements out because I did not want to be tied to MFC. I was very reluctant to do this, because alternative is to put debug output statements in application code, which becomes very messy.
Luckily I found an article by Paul Mclachlan, "Getting around the need for a vararg #define just to automatically use __FILE__ and __LINE__ in a TRACE macro". Paul's TRACE replacement is not only free of MFC dependency, it also provides file and line number output, is thread-safe, and is completely encapsulated in one header file. I have made some minor changes to Paul's class (such as adding thread ID to the output), and so I renamed it to XTrace.h to prevent any conflicts.
With the big TRACE problem out of the way, I could now decide how to use TRACE in a consistent manner. I decided I wanted diagnostics of two types: a basic informational-type debug message, that reports values and code flow; and an error diagnostic for API failures and other serious problems. So I use the standard TRACE macro for debug messages, and a new TRACEERROR macro for API failures. It is easy to disable one or both of these macros, by replacing them with usual
#define TRACE ((void)0)definition, and I have included this capability in XFile.cpp source.
Now for CXFile features. In addition to basic file operations such as opening/creating, reading, writing, copying, deleting, and renaming files, CXFile includes extended functions:
//////////////////////////////////////////////////////////////////////////// // // Compare() // // Purpose: Binary compare two files // // Parameters: lpszFile1 - file 1 name // lpszFile2 - file 2 name // pbResult - pointer to BOOL that receives result of compare; // TRUE = files are identical. pbResult is valid // only if Compare() returns TRUE. // // Returns: BOOL - TRUE = no problems encountered during compare, see // pbResult for result of compare. If FALSE returned, // pbResult is meaningless. //
///////////////////////////////////////////////////////////////////////////// // // Rollover() // // Purpose: Rollover file to new file and optionally zip, continue // with empty file // // Parameters: lpszRolloverFileName - name to use for new rollover file // lpszFileNameBuffer - buffer for rollover file name; if // not NULL, the rollover file name // will be copied into this buffer // dwFileNameBufferSize - size of lpszFileNameBuffer in TCHARs; // if 0, rollover file name will not be // copied into lpszFileNameBuffer // bFailIfExists - operation if file exists: // TRUE = if new file already exists, // the function fails // FALSE = if new file already exists, // the function overwrites the existing // file and succeeds // bZip - operation if Rollover() succeeds: // TRUE = zip rollover file; name of // archive will be same as rollover file, // with .zip extension; then delete // rollover file // FALSE - do not zip // bGenerateRolloverLogFile - create entry in rollover log file: // TRUE = create entry // FALSE = do not create entry // // Returns: BOOL - TRUE = success //
Shrink function will optionally look for the start of next record/line, and further reduce size to eliminate partial record. /////////////////////////////////////////////////////////////////////////////// // // Shrink() // // Purpose: Shrink an open file when it exceeds a size limit // // Parameters: dwMaxFileSize - maximum file size in bytes // dwShrinkToSize - size in bytes to shrink file to // lpDelimiter - byte array containing one or more // characters that comprise line/record ending - // e.g., "\r\n"; use NULL if partial record // elimination is not desired // dwDelimiterSize - size in bytes of delimiter array // // Returns: DWORD - Number of bytes file was shrunk. 0 = file was not // shrunk, (DWORD)-1 = error occurred. // // Notes: Shrink may be called anytime after a file has been opened. // Shrink will have no effect on files opened as readonly. If // the file size exceeds dwMaxFileSize, it will be shrunk to // the size specified by dwShrinkToSize. Regardless of the type // of file (text or binary), the first part of the file will be // discarded, leaving only the last dwShrinkToSize bytes. // // Then, if lpDelimiter is non-NULL, this byte array will be // searched for in the first 64K bytes of the new file. If found, // the new file will begin at the first byte past the lpDelimiter // array that was found. This ensures that text files and other // record-oriented files will begin at the beginning of a line // or record. Note that this may cause the new file to be smaller // than the specified dwShrinkToSize. // // lpDelimiter is treated as a byte array by Shrink, whether the // app is compiled for Unicode or not. For example, if you pass // _T("\r\n") as the delimiter array. and 2*sizeof(TCHAR) as the // delimiter size, Shrink will look for the sequential bytes // 0x0D 0x00 0x0A 0x00 // in the file if your app is compiled for Unicode. If it is // compiled for ANSI, Shrink will look for the sequential bytes // 0x0D 0x0A. // So it is up to you to use a delimiter string that matches what // is actually in the file, regardless of how your app is compiled. // Shrink does no conversion on the data it reads from the file. // // Shrink will be useful to keep frequently appended-to files from // growing too large. You should only use Shrink on files where // the most recent data is appended to the end of the file - for // example, log files that are written by server processes. // // Shrink() will always attempt to position the file at EOF before // returning. //
Zip, this issue goes away. The zip engine is implemented in single .cpp/.h pair of files - no lib or DLL is necessary. Currently CXFile::Zip() is limited to creating an archive containing only one entry, although zip engine is capable of producing multi-file archives. The zip archive that is produced can be read by other compression programs such as WinZip. If you do not need zip capability, you can define DO_NOT_INCLUDE_XZIP at top of XFile.cpp, and remove XZip.cpp and XZip.h from your project. /////////////////////////////////////////////////////////////////////////// // // Zip() // // Purpose: Zip a file // // Parameters: lpszZipArchive - name of zip archive to create // lpszSrcFile - name of file to zip; the file name and // extension from lpszSrcFile will be used as // the entry name in the zip file // bFailIfExists - operation if zip archive exists: // TRUE = if zip archive already exists, the // function fails // FALSE = if zip archive already exists, the // function overwrites the existing zip archive // and succeeds // bDeleteSrcFile - operation if zip creation succeeds: // TRUE = delete lpszSrcFile // FALSE = do not delete lpszSrcFile // // Returns: BOOL - TRUE = success // // Notes: Zip() compresses one file and stores it in a new zip archive. // It does not handle more than one file per archive. // // Zip() supports Unicode in the following way: the names of // the zip archive and zip source file may be passed as Unicode // strings (if built with _UNICODE defined). However, the source // file name is stored internally in the zip archive as ANSI. // This is a basic limitation of the zip engine. //
/////////////////////////////////////////////////////////////////////////// // // Printf() // // Purpose: Write printf-formatted output to an open file // // Parameters: lpszFmt - format specification (see printf) // ... - variable number of args // // Returns: int - number of characters actually written, or negative // value if error occurs // // Notes: A maximum of MAX_PRINTF_CHARACTERS characters can be // outputted. //
Shrink to discard from the beginning, or by discarding data from end of the file. /////////////////////////////////////////////////////////////////////////////// // // SetSizeLimit() // // Purpose: Set parameters to limit file size // // Parameters: dwSizeLimitBytes - maximum file size in bytes; if the limit // type is SHRINK, when dwSizeLimitBytes is // exceeded, the "shrink to" size will be set at // dwSizeLimitBytes / 2. // eType - limit type: // TRUNCATE - file size will be limited by // discarding writes when the file has reached // dwSizeLimitBytes // SHRINK - file size will be limited by // discarding first half of file, and then // using lpDelimiter string to discard partial // record. See Shrink() for more details. // lpDelimiter - byte array containing one or more // characters that comprise line/record ending - // e.g., "\r\n" // dwDelimiterSize - size in bytes of delimiter array // // Returns: BOOL - TRUE = success // // Notes: Setting a file size limit should only be done for files that // are being written sequentially - i.e., new data is written to // the end of the file. // // The file size limit will be enforced when Flush() is called, // which typically happens when the internal CXFile write buffer // is full. The first time Flush() is called, the data will always // be written to the file, since the file pointer is at the // beginning of the file and the max size hasn't been exceeded yet. // Therefore, the effect of the max size limit will only be seen // when it is larger than the internal buffer size, so that // multiple Flush()'s will occur, and the max size limit will be // triggered. //
CXFile::OpenMapped(), you use Read() API for reading a mapped file - this is same one used to read a disk file. There is no need to use MapViewOfFile or UnmapViewOfFile. One important thing to note: with mapped reads, the file offset (and therefore the read buffer size) must be a multiple of the system's virtual memory allocation granularity, or the next call to MapViewOfFile() will fail. Use CXFile::GetAllocationGranularity() to retrieve this value and allocate a buffer. /////////////////////////////////////////////////////////////////////////////// // // OpenMapped() // // Purpose: Open file for mapped reads // // Parameters: lpszFile - file name // // Returns: BOOL - TRUE = success //
FF FE (hex values). For big-endian systems, the BOM is FE FF. Where a BOM is useful is with files that are typed as text, but not known to be in either big- or little-endian format. In that situation, BOM serves both as a hint that the text is Unicode, and also what type of byte-ordering was used to create the file.
You have option to specify Unicode header in Open():
/////////////////////////////////////////////////////////////////////////////// // // Open() // // Purpose: Open file // // Parameters: lpszFile - file name // bReadOnly - TRUE = file will be opened for read only // FALSE = open file for reading and writing // bTruncate - TRUE = open existing and truncate, or create new // FALSE = open existing (do not truncate) // or create new // bUnicode - TRUE = file is Unicode text file, write Unicode // header (Byte Order Mark, or BOM) // FALSE = do not write Unicode header // // Returns: BOOL - TRUE = success // // Notes: Some text editors (e.g., TextPad) and other applications write // a Unicode header of two bytes (0xFF followed by 0xFE). However, // TextPad seems to have no problem figuring out that the file // contains UTF-16 even if the header is not present. You should // test this with your application to see if it makes any // difference. In particular, you should verify that the presence // of a BOM does not interfere with your app's read operations. //
/////////////////////////////////////////////////////////////////////////////// // // GetRecord() // // Purpose: Read byte data from an open file. The file may have been // opened either with Open() or OpenMapped(). Up to dwBytes of // data are read into buffer specified by lpszBuffer, or until // the record delimiter is found. See SetRecordDelimiter(). // Each successive GetRecord() will return the next record, // until 0 is returned at the end of file. // // Parameters: lpszBuffer - pointer to byte buffer that byte data will be // read into // dwBytes - size of buffer in bytes // // Returns: DWORD - number of bytes read, or (DWORD)-1 if error; EOF found // if (DWORD)-2 returned. // // Notes: GetRecord() starts reading from current file position. Use // of Seek() will interfere with the sequential reading of data. // // GetRecord() may be used on both text and binary files. The // record delimiter (specified by SetRecordDelimiter()) will be // searched for as a sequence of bytes, regardless of whether // the file is ANSI text, Unicode, or binary. //
/////////////////////////////////////////////////////////////////////////////// // // Prepend() // // Purpose: Write byte data to the beginning of an open file. Data is // written to beginning of file, before the data that was in the // file. The data that was in the file is preserved and begins // immediately after the new data. // // Parameters: lpszBuffer - pointer to byte buffer containing data // dwBytes - number of bytes in buffer // // Returns: DWORD - number of bytes written, or (DWORD)-1 if error //
/////////////////////////////////////////////////////////////////////////////// // // Search() // // Purpose: Search for a sequence of bytes // // Parameters: lpSearch - byte array containing one or more values // that comprise the search string; any value // permitted, including nul. // dwSearchSize - size in bytes of search string // bCaseInsensitive - TRUE = search will be done ignoring case // (of those characters that fall in the ANSI // range a-z and A-Z); // FALSE - characters in the search string and // characters in the file will be compared // "as is", with no case conversion. // // Returns: DWORD - >= 0 if lpSearch string was found; return value is the // file position of the string found. If the search failed, // (DWORD)-1 is returned. // // Notes: After a successful search, the file position will be set to the // first character position of the string found. After an // unsuccessful search, the file position will be unchanged. // // The search always starts at (or one byte past) the current file // position; Seek() is not called prior to starting the search. // If the starting file position is past the beginning of the file // (i.e., not 0), then the search will start at one past the current // file position. Otherwise, it will start at the current file // position (the beginning of the file). This allows successive // searches to be done without having to move the file pointer past // a previous successful search. // // Searches do not wrap. //
CXFile buffers file writes to its own buffer, and writes buffer to file when it becomes full. This reduces file API calls, especially for short data records. The buffer size can be set programmatically. GetLastError() code is converted to string format and dumped. All errors are reported by TRACEERROR(), while other useful debug information is reported by TRACE(). With Paul Mclachlan's excellent TRACE class, both types of output include source module and line number. One or both types of output can also be easily disabled within XFile.cpp. CXFile will only write whatever data is in byte buffer that is passed to it, with one exception: if CXFile object has been constructed with the Unicode flag set to TRUE, or has been opened with the Unicode flag set to TRUE, then CXFile will write Unicode header (see above) when writing data at beginning of file. This flag should be used only for plain text files.
CXFile will read whatever data is in file, and return it in buffer passed to it. No CR/LF or any other type of conversion is done.
CXFile does not care what type of data is in files that you zip. However, the names of the files being zipped are handled as ANSI internally by zip engine. If you look at XZip.cpp, the ZipAdd function is prototyped as ZRESULT ZipAdd(HZIP hz, const TCHAR *dstzn, void *src, unsigned int len, DWORD flags)Internally,
dstzn parameter is converted from Unicode to ANSI, and this ANSI string is used as the entry name within the zip archive. The src parameter is name of source file (that will be compressed by ZipAdd), and should also be passed as a Unicode string if _UNICODE is defined.
To integrate CXFile class into your app, you first need to add following files to your project:
#include "XTrace.h" line from XFile.cpp, and uncomment #define TRACEERROR ((void)0) line. The XZip.cpp and XZip.h files are also optional, if you do not need Zip() function. Uncomment line #define DO_NOT_INCLUDE_XZIP at top of XFile.cpp.
If you include CXFile in project that uses precompiled headers, you must change C/C++ Precompiled Headers settings to Not using precompiled headers for XFile.cpp and XZip.cpp.
Next, include the header file XFile.h in appropriate project files (stdafx.h usually works well). Now you are ready to start using CXFile. There are many notes concerning usage of various functions in XFile.cpp. Please read all function header for each function you wish to use.
There are two ways to construct CXFile object. Typical way is to specify file name in ctor:
CXfile file("myfile.txt"); file.Printf("This is test %d", nTest);If you do not specify file name in
ctor, you must call Open(): CXfile file;
file.Open("myfile.txt");
file.Printf("This is test %d", nTest);
// Known limitations: // 1. File size must be smaller than a DWORD value. // 2. Maximum number of Printf() output characters (TCHARs) must be less // than MAX_PRINTF_CHARACTERS. // 3. When using the mapped file functions, the read buffer MUST be a // multiple of system's virtual memory allocation granularity, or // the next call to ReadMapped will fail because the file offset // will be illegal. (The first call will succeed because the file // offset will be 0). // 4. XFile has only been tested with files.
CXFile. Here is output from Test 1, that shows a binary file created and then read. Its contents are then checked.
Zip(), Shrink(), and Rollover(), the main reason is to have better error reporting. Instead of having to check the return code, and then do error reporting yourself after each API call, all you have to do is check the return code - CXFile does the error reporting for you. 
XFile.cpp(2611) : fatal error C1010: unexpected end of file while looking for precompiled header directive. How can I fix this?
LINK : fatal error LNK1104: cannot open file "mfc42u.lib" Error executing link.exe. How can I fix this?
You can configure the Visual Studio toolbars to include the Select Active Configuration combobox. This lets you see at a glance what configuration you are working with.
MapViewOfFile() will fail. Here is what MSDN says about this:
| The combination of the high and low offsets must specify an offset within the file that matches the system's memory allocation granularity, or [MapViewOfFile] fails. That is, the offset must be a multiple of the allocation granularity. Use the GetSystemInfo function, which fills in the members of a SYSTEM_INFO structure, to obtain the system's memory allocation granularity. |
You can use CXFile::GetAllocationGranularity() to retrieve the system's memory allocation granularity and allocate a buffer. Search for "granularity" in XFile.cpp for more information.
//#define DO_NOT_INCLUDE_XZIP
XTrace code is based on an article by Paul Mclachlan, "Getting around the need for a vararg #define just to automatically use __FILE__ and __LINE__ in a TRACE macro". MoveFileEx.
Prepend function.
GetRecord function.
GetTotalRecordsRead function.
Search function
ClearBuffer function.
SetSizeLimit and Shrink parameters to accept byte array rather than string, and also added parameter for byte array size. This allows sequence of non-text values to be used for delimiter.
FILE_FLAG_SEQUENTIAL_SCAN flag to CreateFile, suggested by mattb79.
#define XFILE_ERROR ((DWORD)-1)
#define XFILE_EOF ((DWORD)-2) This software is released into the public domain. You are free to use it in any way you like. If you modify it or extend it, please to consider posting new code here for everyone to share. This software is provided "as is" with no expressed or implied warranty. I accept no liability for any damage or loss of business that this software may cause.
General
News
Question
Answer
Joke
Rant
Admin
|
PermaLink |
Privacy |
Terms of Use
Last Updated: 19 May 2003 Editor: Sean Ewington |
Copyright 2003 by Hans Dietrich Everything else Copyright © CodeProject, 1999-2009 Web17 | Advertise on the Code Project |