Click here to Skip to main content
Click here to Skip to main content

XReverse - Reverse a text file using memory-mapped files

By , 14 May 2003
Rate this:
Please Sign up or sign in to vote.

Introduction

Viewing log files is an important part of monitoring the health of servers and other enterprise-related systems. While not a usual requirement, reversing a log file is becoming more common as interest in monitoring server applications grows. What "reversing a log file" means is that file is reversed, back-to-front, so that last line becomes first line, etc.

When I first started investigating use of file reversal for some server applications that I was working on, I immediately found references to tac, a GNU utility whose name is meant to imply what it does: tac is cat in reverse. On examination of source code, I found that tac uses standard C run-time to read and write files. I began to think of how to improve on that approach, since server-based software is one place where performance does matter - and also where log files of a million lines is not unheard of.

This led me to consider using Win32's memory-mapped file API's to read file. A quick test proved the feasibility, and so CXReverse was born.

CXReverse Features

  • Very fast - use of memory-mapped files gives good performance even on very large files.

  • Handles any delimited text file - you can specify standard ANSI text files, with lines delimited by \r\n, or record-oriented files with custom record delimiters. CXReverse also works on Unicode files.

  • Limit number of lines reversed - to minimize load on the server, you can specify the number of lines that are to be reversed. I have found that it is rarely necessary to go back more than 100-300 lines on most servers. This also helps because the default buffer size for reads is set to the normal system memory allocation granularity, which is usually 64K bytes. So, if the entire number of lines will fit within one buffer, it saves a lot of processing time.

    Uses only Win32 API's - CXReverse has no dependency on MFC.

CXReverse API

/////////////////////////////////////////////////////////////////
//
// Reverse()
//
// Purpose:     Reverse existing file
//
// Parameters:  lpszExistingFileName - name of existing file
//              lpszNewFileName      - name of new file
//              bFailIfExists        - operation if file exists:
//                                     TRUE = if new file already
//                                     exists, the function fails
//                                     FALSE = if new file already
//                                     exists, the function overwrites
//                                     the existing file and succeeds
//              nMaxLines            - maximum number of lines to
//                                     output;  0 = no limit
//              lpszDelimiter        - line/record delimiter;  for
//                                     normal ANSI text files, this
//                                     will be "\r\n".  The delimiter
//                                     may comprise any sequence of
//                                     characters, and does not need
//                                     to include \r or \n.
//
// Returns:     BOOL - TRUE = success
/

How To Use

To integrate XReverse into your app, you first need to add following files to your project:

  • XReverse.cpp
  • XReverse.h

If you include XReverse in project that uses precompiled headers, you must change C/C++ Precompiled Headers settings to Not using precompiled headers for XReverse.cpp.

Next, include the header files XReverse.h in appropriate project files. Now you are ready to start using XReverse. Please see XReverseTestDlg.cpp for examples of how to use CXReverse::Reverse().

Known Limitations

XReverse works only with text files that have fixed delimiters - i.e., same delimiter is used for every line/record.

Demo App

The XReverse.exe demo tests CXReverse::Reverse(). Here is some of the output:

XReverse screenshot

The demo app tests CXReverse::Reverse() on a text file with \r\n delimiters, and also a text file using a delimiter of ***~|||. The digital counters show number of times the test has been run, and elapsed time (elapsed time includes time to create text files). You have option of running test once (which will test both files), running same test 100 times, or running a test 100 times that will only output 200 lines.

Frequently Asked Questions

  1. Can I use XReverse in non-MFC apps?
    Yes. It has been implemented to compile with any Win32 program.

  2. Can I use XReverse to reverse a server log that has more than 1,000,000 lines?
    Yes, you can do this, but I do not recommend it. It is very unlikely that anyone is going to read that many lines. Reversing only the last 1000 - or even the last 100 - lines is usually more than enough, and will take only a fraction of the time and a fraction of the disk space.

  3. When I try to include XReverse.cpp in my MFC project, I get the compiler error XReverse.cpp(713) : fatal error C1010: unexpected end of file while looking for precompiled header directive. How can I fix this?
    When using XReverse in project that uses precompiled headers, you must change C/C++ Precompiled Headers settings to Not using precompiled headers for XReverse.cpp. Be sure to do this for All Configurations.

    XReverse screenshot

  4. When I try to build the demo app, I get the linker error LINK : fatal error LNK1104: cannot open file "mfc42u.lib" Error executing link.exe. How can I fix this?
    The default installation options of Visual C++ v6.0 don't install the Unicode libraries of MFC, so you might get an error that mfc42u.lib or mfc42ud.lib cannot be found. You can fix this either by installing the Unicode libs from the VC++ install CD, or by going to Build | Set Active Configuration and selecting one of the non-Unicode configurations.

    XReverse screenshot

    You can configure the Visual Studio toolbars to include the Select Active Configuration combobox. This lets you see at a glance what configuration you are working with.

  5. Can we use XReverse in our (shareware/commercial) app?
    Yes, you can use XReverse without charge or license fee. It would be nice to acknowledge my Copyright in your About box or splash screen, but this is up to you.

Acknowledgments

Revision History

Version 1.1 - 2003 May 8

  • Initial public release

Usage

This software is released into the public domain. You are free to use it in any way you like. If you modify it or extend it, please to consider posting new code here for everyone to share. This software is provided "as is" with no expressed or implied warranty. I accept no liability for any damage or loss of business that this software may cause.



License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Hans Dietrich
Software Developer (Senior) Hans Dietrich Software
United States United States
I attended St. Michael's College of the University of Toronto, with the intention of becoming a priest. A friend in the University's Computer Science Department got me interested in programming, and I have been hooked ever since.
 
Recently, I have moved to Los Angeles where I am doing consulting and development work.
 
For consulting and custom software development, please see www.hdsoft.org.






Comments and Discussions

 
GeneralBinary Frmat Pinmembercoolnaveen123420-Apr-06 0:10 
Generaldoesn't work for unicode Pinmemberdan o30-Jan-04 4:41 
GeneralRe: doesn't work for unicode PinmemberHans Dietrich30-Jan-04 6:39 
Please email text files, I will take a look.
 
Hans
hdietrich2@hotmail.com

GeneralRe: doesn't work for unicode PinmemberHans Dietrich2-Feb-04 9:39 
Questionare memory mapped files more efficient? Pinmemberumeca7419-May-03 22:46 
AnswerRe: are memory mapped files more efficient? Pinmemberdjsox23-May-03 6:20 
GeneralPrecompiled Headers PinmemberMattamoo16-May-03 7:53 
GeneralRe: Precompiled Headers PinmemberNeville Franks16-May-03 8:25 
GeneralGreat to see MMF being used PinmemberNeville Franks15-May-03 22:05 
GeneralRe: Great to see MMF being used PinmemberJohn Johansson17-May-03 11:31 
GeneralRe: Great to see MMF being used PinmemberNeville Franks17-May-03 20:06 
GeneralRe: Great to see MMF being used PinmemberJohn Johansson17-May-03 21:24 
GeneralRe: Great to see MMF being used PinmemberNeville Franks17-May-03 22:20 
GeneralRe: Great to see MMF being used PinmemberJohn Johansson17-May-03 22:57 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.140415.2 | Last Updated 14 May 2003
Article Copyright 2003 by Hans Dietrich
Everything else Copyright © CodeProject, 1999-2014
Terms of Use
Layout: fixed | fluid