Click here to Skip to main content
Click here to Skip to main content

Tagged as

HexEdit - Window Binary File Editor

, 17 Oct 2012 MIT
Rate this:
Please Sign up or sign in to vote.
Open-source hex editor with powerful binary templates

Introduction

After being in beta since the last year I have finally released the official version of HexEdit 4.0.  The beta version worked fine so I have not been in too much of a hurry to release it.  There were a few minor bugs fixes though.

HexEdit is written in C++ and this version requires the new MFC provided with VS2008 (with feature pack), VS2010, or later. It should be easy enough to build but see the "Building HexEdit 4.0" section below if you have any problems.  There is now also a project file for VS2012 (Visual Studio 11).

The first version of HexEdit (1999) was open source but later versions were shareware (though I have always maintained a free version).  There was little point in making HexEdit 2.0 open source since it used the commercial BCG library. (The BCG library is an excellent MFC extension library so I do not regret using it.) Luckily, a few years ago Microsoft bought the BCG code and incorporated it into MFC.  So now HexEdit 4.0 is open source once more (see http://www.hexedit.com); there is also a shareware version available for those who want to pay for it (see http://www.hexeditpro.com) which has a few minor additions.

To build HexEdit 4.0 you only need Visual Studio, plus some open source code and libraries (most of which came from right here on CodeProject).  You can use the program or any of the source code for whatever purpose you see fit as long as you abide by the relatively unrestrictive license requirements (see below) or any specific requirements for any of the included 3rd party code.

Development History

I wrote HexEdit in 1997-1998 as it was something I needed in my work and the hex editors available at the time were poor. HexEdit was always designed to be simple to use and familiar to users of Windows software such as MS Word and Visual Studio. In fact it borrowed many ideas from these and other common Windows programs.

HexEdit 1.0

In January 1999, I released HexEdit 1.0 with source code to the public and got a lot of positive feedback. Some people even emailed asking how to pay for it! Some of the acclaimed features of the first version were unlimited undo (everything not just file changes), editable file size not limited to memory size, EBCDIC support, simple (but fast) searches and file compares, keystroke macros.  The most useful was the floating "Properties" window which could show all sorts of interesting stuff about the byte(s) at the current cursor position in the active file.

At that time I had a huge list of things that I wanted to add - some were user suggestions but mainly they were my own ideas. I decided to make a shareware version, so that I would have some encouragement to continue adding features and so that I could justify the large amount of time I spent on it.

Some of the things on my TODO list at that time were:

  1. color schemes (including colors for different byte ranges)
  2. highlighter (like in MS Word)
  3. printing (similar to MS Word)
  4. bookmarks (like in Visual Studio but better)
  5. background searches (automatically find all occurrences of the current search string in a background thread)
  6. advanced templates (including expressions, optional parts, loops, etc)
  7. C/C++ parser that could create a basic template from a C struct (or C++ class)
  8. aerial view (using abovementioned color schemes)
  9. improved compare including insertions/deletions
  10. better search including number search, alignment restrictions etc
  11. disk editor and memory editor
  12. ruler (with draggable handles like in Word)
  13. keyboard and toolbar customization

The only ones remaining from that list are a better file compare utility and a memory editor. I have actually implemented both of these in the past but they were not of sufficiently high quality to be added to the released version.

HexEdit 2.0

In 2001 I released HexEdit 2.0 (shareware) which used the very cool BCG library. I mainly wanted BCG for the toolbar/keyboard customization but it had several other very useful things which allowed me to add features similar to MS Office and Visual Studio. Unfortunately, using this commercial library meant people could no longer easily use the source code for the free version of HexEdit. I have always made earlier versions (about 2-3 years behind the shareware version) available for free with source code (on request).

In the intervening years I have added most of the the abovementioned features and many more (but my TODO list is longer than ever!). I won't bore you with the history -- the next section covers the major features of the current version but for details you can go to http://www.hexedit.com/hex-edit-features.htm.

HexEdit 2.5

This was a major release as it introduced the template system including full template editing.

HexEdit 3.0

The display engine was rewritten in this version which allowed new features like change tracking, "stacked" display mode, sector display, etc.  This also improved display speed, especially for long lines.  The max line length was increased to 32,767.

HexEdit 4.0

In this version of HexEdit I converted the code from using BCG to MFC9. I started this early in 2009 and it took longer than expected. Part of the reason is that I have converted the modeless dialogs to use the new docking system - the fancy one like in Visual Studio where you can dock several windows on top of each other and tabs are automatically added. The dockable windows are the Properties dialog, Find dialog, Calculator, Bookmarks dialog and Explorer window.

Features

HexEdit has very many useful features which I can only just touch on here. For details see http://www.hexedit.com/hex-edit-features.htm or the Overview of Features in the help. Here I will just explain the general philosophies of the design of HexEdit and introduce a few features as examples.

One thing to remember is that if you need a feature it might already be there. I often get emails asking for a specific feature which is already present.  Try to find it in the help.

Another thing to remember is that if you often find yourself spending a lot of time doing a tedious or repetitive task then there is probably a feature in HexEdit to save you time. Another type of email I get is from users saying they found a useful feature that they did not even know they needed but has since saved them a lot of time.

Simple

An overriding principle has been to keep the user interface as simple and uncluttered as possible. It is a common failing of other hex editors that they assume an advanced user who uses the program every day with bulging menus full of rarely used features.

That's why the menus are short (apart from the Operations menu which has many similar items), and each Options page does not have more than 10 options.

The idea is that the user interface is not overwhelming. An infrequent user (as users of hex editors often are) can use the program without constantly referring to the help. Of course, if you need it there is lots of help available.

Mark

The mark is a simple facility that allows you to mark a position in a file and return to it later. It is also useful in many other areas such as keystroke macros.  I think I got this idea from a text editor I used (pEdit, Epsilon or Emacs).

Help

One problem with keeping the user interface simple is that some of the more powerful features may not be obvious. This is where HexEdit provides extensive help in the provided help (quickstart, overview, reference, tutorials, etc) as well as tooltips, tip of the day and even video tutorials (see http://www.hexedit.com/videos.htm).

Familiar

Whenever appropriate I have tried to emulate the common behavior of other Windows programs. For example, many hex editors use the Tab key to switch between the hex and character areas.

There is also a de facto standard for the behavior of cursor keys, mouse, etc as seen in many editors. Also, you can select multiple bytes by holding down the Shift key and using the cursor keys - for example, Shift+Ctrl+End selects all bytes to end of file.

HexEdit often has similar behavior to common Microsoft programs like Word and Visual Studio. This is because Microsoft obviously do a lot of research to ensure their software is easy to use - so it's probably a good behavior (though I am yet to meet anyone who likes Ribbon Bars). Also, many users are already familiar with these programs so they should feel right at home. Here are some examples:

Highlighter

The highlighter works just like the highlighter in MS Word.

Printing

You can set up margins, headers/footers etc much like in MS Word.

Bookmarks

Bookmarks are based on the bookmarks of Developer Studio (the precursor to Visual Studio), but with improvements. Interestingly Visual Studio later duplicated some of the features that first appeared in HexEdit, like the floating/dockable bookmarks window.

Useful

Part of the simplicity is useful features that are there without any further effort.

Background Searches

Whenever you perform a search HexEdit automatically does a background search throughout the rest of the file (and all other open files) in a background thread. This will not interfere with your editing, since the background search runs in a low-priority thread. When the background search is finished you can immediately see all occurrences highlighted in a special color.

The background search will normally be completed before you even notice unless you are working with very large files - but while it is running the progress is displayed in the background search occurrences pane. When finished a count of the occurrences is shown in the status bar pane.

Huge Files

HexEdit has always easily edited files of any size without any extra effort on the part of the user. Before HexEdit most hex editors could not edit files bigger than they could load into memory. Since then many have copied how HexEdit works.

HexEdit was the first binary editor to allow editing of files more than 4 GBytes (2^32 bytes) in size. There have been several improvements to HexEdit since the first version so that you can now insert huge files into other files, etc.  All operations on the selection (including compression, encryption, checksums, CRCs, etc) work with a selection length of any size. Progress is shown in the status bar and you can, of course, abort the operation by pressing the Spacebar or Escape key.

Customizable

The ability to configure HexEdit has always been important. The user should be able to decide how they want to work.

Toolbars

In HexEdit 2.0 I incorporated the BCG library which (among many other things) allowed for complete customization of the toolbars, keyboard, menus (including many context menus), etc.

Info Tips

Many programs show a small (usually yellow) tip window when you hold the mouse over an item of interest. HexEdit takes this further with Info Tips.

In a binary editor what you find interesting about some of the bytes of a file may not be what someone else does, and can change for different circumstances. Info Tips allow you to create one or more tips based on several predefined values (addresses, offsets, file data etc) and using C-like expressions and with many formatting options.

You can perform calculations on addresses. For example, is the offset within the current sector divisible by three?

You can examine the data at the mouse position using dozens of different formats (floating point, date etc). For example, display "YES" or "NO" depending on whether the 6th bit is on or off, or check whether the current IBM floating point value is negative.

File Filter List

When you open or save a file most programs present a list of "filters" in the "Files of Type" drop down list in the standard file open/save dialog for the types of files the program knows about. Since HexEdit allows you to work with any type of file you can customize this filter list any way you like.

You can add or delete filters in the Options dialog on the "File Type Filters" page. Since you may work with different types of files at different times you can turn the filters on and off without deleting them using the checkboxes.

Moreover the same filter list is available in the "HexEdit Explorer" window. This allows you to filter the list of files that you see. This is another example of integration between the features (see below)

Compatible

Not only is the user-interface compatible with what you are used to in other programs, HexEdit also attempts to work well with other software.

Clipboard

When using the clipboard HexEdit supports text data and binary data. The binary data format is exactly the same as the format used by the Visual Studio hex editor.

The Copy As C Source dialog allows copying to the clipboard in a large number of different text formats suitable for initializing strings and arrays in C, C++, C# and other languages. In fact the integer and floating point options are compatible with the CSV (comma-separated values) formats used by many programs such as spreadsheets.

Import/Export

You can import/export from/to Motorola-S and Intel-hex formats. These formats are used by many device such a EPROM programmers. HexEdit includes many nice features such as support for noncontiguous records.

You can also import and export using hex text, where each byte is stored as 2 hex digits in ASCII. For CSV format use the Copy As C Source dialog (see above).

Flexible

One thing about HexEdit that many users have commented on is the simple tool set that provides the flexibility to easily perform a wide variety of tasks.

Jump Tools

On the Edit toolbar are two simple drop-down lists that provide a large number of facilities. These are the hex and decimal jump tools. There primary purpose is to allow you to type in an address to immediately jump to it. The drop-down lists provide a history of the last addresses jumped to.

But they also do a lot more. First as you move the cursor through the file they are updated with the current address. To remember an address (by adding it to the history) it is simply a matter of clicking in the tool and pressing Enter.

A further feature is that as you type a value into one tool, the corresponding value is shown in the other tool. This allows quick and easy conversion between hex and decimal.

Finally, you can evaluate any expression in these tools including assigning to variables that are stored between sessions. (See below for more on expressions in HexEdit.) As you type an expression into a jump tool the result is displayed in other visible jump tools. (You can have any number of jump tools visible by customizing the toolbars.)

Keystroke Macros

Sometimes I get a request for a feature where I just have to say "no", because it is just too special to be of general use. Even then, most of the time I can simply point out how the feature can be accomplished by using existing an existing tool or a combination of tools.

One of the most flexible capabilities is provided by keystroke macros. Of particular use are the calculator buttons which are all recorded and stored in keystroke macros. Many users do not realize that macros can work between files so you can do things like copy a bit of a file and paste it into a temporary file for further manipulation.

Macros can also call other macros. This is demonstrated in the "AllChars" macros distributed with HexEdit which creates a new file and fills it with all 256 different byte values.

Integrated

Part of the flexibility of HexEdit is that there are many simple tools that have been designed to work together. An example, mentioned above, is the way the calculator buttons can be used in keystroke macros. Here are some more examples:

Color Schemes

Color schemes are useful for setting up the colors used for other features such as bookmarks, search occurrences, change tracking etc.

One very useful feature of color schemes is the ability to display colors for ranges of byte values. For example, in the default ASCII color scheme normal text is blue while control characters are red, except for the control characters that are normally found in text files (tab, CR, etc) which are green.

The use of color schemes is also crucial in combination with the aerial view feature. If you have particular byte value(s) that you are interested in you can create a color scheme to show up the value or range of values in the aerial view.

If you have a file format which you know nothing about you can use the "rainbow" color scheme to get an idea of the different byte values in the file - this often immediately shows up areas of interest.

Searches

Searches are another area where the various tools work well together. For example, the history list of the search tool (edit toolbar) is integrated with the search histories in the different pages of the Find dialog.

Another example is that the Find dialog can set bookmarks at all found occurrences.

The Find dialog also supports aligned searches which are a very useful and powerful feature. The alignment can be relative to the mark (see above).

Powerful

Properties Dialog

The Properties dialog is a small floating/dockable window that has several pages of information about the current file and the byte(s) at the current location in the file. First, as you move the cursor (caret) in the file the display is updated to show information about the current byte(s). There are several pages so you can see information about integers, floating point numbers, dates, etc.

Templates

Since its introduction the template system has easily been the feature that has attracted the most interest. I have been told that people have spent weeks and even months to write a program to edit a particular file format. When they discover HexEdit's templates they can often accomplish close to the same thing with just a few hours work.

The secret of HexEdit's templates is the expression evaluator which takes C-like expressions, but can handle more data types such as dates. With expressions you can analyze just about any type of field from a file and use that to control the evaluation of the template.

Expressions can be used to control branching (IF and SWITCH) and loops (FOR with counter or with terminating condition). In addition, JUMPs allows out-of-order processing such a "header" at end of file. STRUCTs can be self-referential so you can model recursive data structures.

An example of the power of templates is that they can be used to look into a filesystem (when combined with the disk editor) to display information and the data from say, a deleted file. As an example, I created a template for the FAT12 filesystem (as used on MSDOS/Windows floppy disks) which could show the files in the root directory and even drill down into sub-directories.

One of the most powerful parts of the template system is the C/C++ parser. With it you can paste a C struct (or just about any C or C++ code) and have HexEdit add the data fields to your template. The parser supports all parts of the C standard (preprocessor, unions, bit-fields, etc) and even some extra things like #pragma pack.

Programmer focused

Binary editors are the domain of programmers so many of the features are aimed squarely at programmers. For example, expressions (as used in templates, calculator, jump tools etc) are based on expressions from the C-language. Similarly formatting (as in templates, Info Tips etc) are based on printf format specifiers (or strftime formats for dates).

zlib

Zlib is the de facto standard for data compression. It provides excellent compression, there is an open source library (in C) and it is unencumbered by patents or any others restrictions. Moreover, many compilers and run-time environments provide it by default (e.g. Delphi, .Net, ...). It is used in many file formats and communication protocols - for example, it is used in .zip files where it is called "deflate". (Zip files also support other algorithms but nowadays files are generally compressed with "deflate" as it is the best.)

Over the years I have worked on several projects that used zlib. To assist testing and debuging I added zlib compression with support for every one of its options. Previously, I had to create a small test program to test such things as the best strategy to use for my data or how often to write full-flush points. Now I can just load my data into HexEdit to test the different zlib options.

Building HexEdit 4.0

To build HexEdit you need to download one or two of the above zip files. First you need the project zip file.  Alternatively you can get the latest source from the SVN repository at SourceForge (https://hexedit1.svn.sourceforge.net/svnroot/hexedit1/trunk/HexEdit) (but you may still need the files in the extras zip file).

You will probably need the extras zip file, which contains the FreeImage and boost files required to build the project.  (If you have FreeImage and Boost installed you may not need these.)  It also contains some image "source" files (PDN format), help files, and the Installer source files.

Finally, you will need Microsoft's Visual Studio 2008 or later. There are separate project files for different versions of VS. When installing VS you need to have installed the C++ language and MFC DLLs.  I have been told that you cannot use the free Visual Studio Express editions as they do not include MFC.

Depending on what you want to do you may need some other (free) tools.  You will need Wix if you want to build the HexEdit installer hexed4_0.msi (see below). You will need Paint.Net if you want to edit the graphics (see http://www.getpaint.net/) since the "source" graphics are in .PDN format. Sorry, but you can't currently change and rebuild the help system but the HexEdit.CHM file is provided.

HexEdit.exe

Here I describe how to build HexEdit.

Hopefully (after downloading and unzipping the project and extras files), all you need to do is open the project for the version of Visual Studio you are using, tell the compiler where to find the Boost and FreeImage headers and library, and build HexEdit. If all goes well this will create HexEdit.exe in the appropriate sub-directory which you can run and use.  You may get an error at the end of the build of the release version if you don't have ReplaceVistaIcon.exe in your path - but you can safely ignore this error.

You can run HexEdit.exe without using the installer. It works fine by itself but to get the best effect you need to copy a few files into the same place as HexEdit.exe is run from. Namely three files from the Graphics directory (Backgrnd.bmp, Splash.bmp, About.jpg) and other files for various facilities: HexEdit.tip (tip of the day), ebcdic.tab (custom EBCDIC translation table), DefaultToolbarImages.bmp (for toolbar icon editing), BinaryFileFormat.dtd and Default.xml (for templates creation), other XML files (templates), some XML and TXT files (for template C/C++ parser - see help), etc.

Visual Studio 2008

First you need to have the MFC Feature Pack installed since this includes the version of MFC where MS added the BCG code. (It was not quite ready for the release of VS2008 so was put in a separate feature pack.) If you have VS2008 but not the feature pack you can download it here.

To set up VS2008 to find the FreeImage and Boost files you need to add to the Include files and Library files locations. To do this open the Options dialog (Tools/Options) then go to the Projects and Solutions/VC++ Directories then select Include files or Library files from the "Show directories for" drop down list.

Then just open HexEditVS2008.sln and build it.

Visual Studio 2010 and VS2012

These come with the required version of MFC. You do not need to install the MFC Feature Pack.

The Include files and Library files locations (for FreeImage and Boost) should already be set up in the correct sub-directory. But if you are using your own copy of Boost and FreeImage then you must set these yourself.

Just open HexEditVS2010.sln in VS2010 (or HexEditVS2012.sln in VS2012) and build it.

MSXML.DLL 3.0

When I started work on the template system I was keen to make the format for template files as open as possible, and also use a text (not binary) format (in the hope that others might also support the format). I had just been reading about XML and this seemed the ideal format for template files. Moreover, using an appropriate DTD would save me a lot of work in parsing any text file to ensure it conforms to the correct format. At about this time Microsoft had released IE5 (Internet Explorer 5.0) which included the XML SDK (MSXML.DLL), so it was a simple matter to use that.

Nowadays the original MSXML.DLL is no longer supported or distributed with Windows or IE. The oldest version of the XML SDK that MS still supports is MSXML3.DLL so I have just converted HexEdit to use that (uses MSXML2 namespace). I believe MSXML3.DLL is distributed with Windows 7, Vista and XP. If you are using W2K (Windows 2000) you will need to have SP4 installed or IE6.

FreeImage 3.10

There are three files in the FreeImage sub-directory, needed for HexEdit's compilation (FreeImage.h), linking (FreeImage.lib) and running (FreeImage.DLL). You can copy these files to an existing place where they will be found by the compiler or add the sub-directory to the compiler's location for include files and library files.

I normally just copy FreeImage.DLL into the Windows system directory (typically C:\Windows\system32) so it is always found when you run HexEdit. Of course, you should be careful not to overwrite a new version of the DLL with the version supplied (3.10). If you have a newer version of FreeImage installed then HexEdit should be compatible but please contact me if there are any problems.

If you want to rebuild FreeImage from the source, or download the documentation, or link with the very latest release, etc, please see the website at http://freeimage.sourceforge.net/.

Boost 1.44.0

Boost is huge so I have not included all of it, but I have included all the headers you need to build HexEdit in the boost subdirectory. However, as the Boost headers files use <> to #include other Boost headers you need to add the HexEdit source directory to the list of include directories. Don't add the "Boost" sub-directory as the Boost header files always use include Boost as part of the file name (ie #include <Boost/...>).

Alternatively you can download and install the full Boost library (recommended) and set up an include location to point to it. For example, I use D:\include\boost_1_44_0 which has the header files in the boost subdirectory.

For more information, or to download the full library please go to http://www.boost.org/.

Help

Currently the HexEdit help file is built using the commercial RoboHelp product, so I do not include the source files for the help system, but instead just include the HTML help file (HexEdit.chm). I also create a web version of the help system which is normally available at http://www.hexedit.com/help/Hexedit.htm, though this is not integrated with HexEdit so does not include some things like context-sensitive help.

ReplaceVistaIcon

Another optional step is to have replace the default HexEdit icon with one that looks better on Vista and Windows 7. In the res sub-directory there is a files called hexedit2.ico that is Windows icon files with sizes up to 48x48 with 8-bit and 32-bit resolution. As Visual Studio does not properly handle icons like these I used a 3rd party commercial icon editor (see Real World Icon Editor) to create this file.

To add this icon to HexEdit.exe it is necessary to use a program called ReplaceVistaIcon.exe. (See http://www.codeproject.com/KB/GDI/using_vista_icons.aspx) This is done in a post-build event in the batch file HexEditPostBuild.bat.

You need to download the exe from CodeProject (http://www.codeproject.com/KB/GDI/using_vista_icons.aspx), but this is optional as the .exe will work fine with the lower quaility icon. Once you have ReplaceVistaIcon.exe you need to place it in your PATH or somewhere that the batch file can find it.

Installer

This version of HexEdit uses the open source WIX installer. I chose WIX as it was the only free installer that produced a .MSI file (MicroSoft Installer file) and had the features I wanted. Though very flexible it is difficult to use unless you are very familiar with how the Windows Installer works. See http://wix.sourceforge.net/.

To build the installation you need to download and install WIX (I used version 2.0). In the HexEdit Install directory you will need to edit make.bat and wixui\makelib.bat to change all occurrences of D:\Program Files\wix to the directory where you installed WIX.

Of course, you have to build the release configuration of HexEdit. You also need merge modules for the CRT (C run-time) and MFC DLLs. If you did not install the merge modules when you installed Visual Studio you will need to do that now. (Alternatively you can remove the merge module lines from HexEdit.wxs but then HexEdit will not run on systems that do not have those DLLs already installed.)

The main file that controls the installation is HexEdit.wxs (an XML file). You may need to change the batch files and HexEdit.wxs if you have installed Visual Studio or WIX etc in non-standard locations. If you are using the HexEdit.exe built with VS2008 or VS2012 (instead of VS2010) then you will also need to edit HexEdit.wxs to change where HexEdit.exe is found (change "ReleaseVS2010" to "ReleaseVS2008" or "ReleaseVS2012") and the names of the MS merge modules (change "100" to "90" in the merge module names).

When everything is set up correctly just run wixui\makelib.bat and then make.bat to create the Microsoft Install file (HexEd4_0.msi).

Design and Code

Some areas of the design of the software are poor (for various reasons) but other parts are excellent. The overall design conforms to MFC design patterns (including the classic MFC Document-View model which is similar to MVC, but combining the view and controller). The classes CHexEditView and CHexEditDoc provide the core of HexEdit, by implementing the view and document of the Document-View model.

A core part is the ability to handle huge files keeping track of changes (in memory and temporary disk files). This is all handled in the CHexEditDoc class (see DocData.cpp). Another important part is the display of the file (and editing) which is mainly handled by CScrView which is derived from MFC's CView and is the base class for CHexEditView. It handles a display of up to 2^60 pixels in X-direction and 2^32 in the Y-direction, including rulers, scrolling, scrollbars etc.

There are many comments in the code including detailed design explanations at the start of some source files. Apart from that, the best way I can think to give an overview of the code is to explain the contents of each of the files.

I will just list the header (.h) files as the .cpp files generally have the same name as their corresponding .h file. For example the class CHexEditDoc is implemented in HexEditDoc.cpp plus several other files which were split from it. Generally each header file contains one class and perhaps related structs or classes. There are exceptions as noted.

HexEdit.h - CHexEditApp class for the one and only application object
Stdafx.h - used to build pre-compiled headers
resource.h - resource IDs used in HexEdit.rc and the C++ code

MainFrm.h - MFC derived class that handles the main app window
ChildFrm.h - MFC derived class that handles each files "window"
HexEditDoc.h - CHexEditDoc class (see HexEditDoc.cpp, DocData.cpp, Template.cpp, BGSearch.cpp, BGAerial.cpp).
HexEditView.h - CHexEditView class (see HexEditView.cpp and Printer.cpp)
ScrView.h - CScrView provides a scrollable view (base class of CHexEditView)
DataFormatView.h - CDataFormatView handles the template display (see also Template.cpp)
AerialView.h - CAerialView class - displays aerial view (also see BGAerial.cpp)

Prop.h - contains classes for Properties dialog (property sheet, property pages and controls)
BookMarkDlg.h - CBookMarkDlg dialog class for viewing/adding bookmarks (see also bookmark.h)
FindDlg.h - classes for Find dialog (property sheet and pages)
Explorer.h - HexEdit Explorer dialog (dialog and control classes)
CalcDlg.h - CCalcDlg class for calculator (see also CalcEdit.h below)

Options.h - property sheet and pages for the Options dialog
DFFD*.h - dialogs used for template editing
TParseDlg.h - C/C++ parse dialog used in template editing (see TParse.h)
NewFile.h - dialog used for "New File" and "Insert Block" commands
OpenSpecialDlg.h - open special dialog for disk editing etc (also see SpecialList.h)
RecentFileDlg.h - displays the recently used files dialog (also see HexFileList.h)
CopyCSrc.h - CCopyCSrc dialog used for "Copy As C Source"
Algorithm.h - CAlgorithm class for encryption algorithm dialog
Password.h - CPassword class for encryption password dialog
CompressDlg.h - dialog for zlib compression
EmailDlg.h - email support dialog
Tipdlg.h - shows tip of the day
Dialog.h - dialogs used in macros (GetStr CMacroMessage CSaveMessage CMultiplay
- also CFileDialog derived classes including CHexFileDialog (see below)
HexPrintDialog.h - custom printing dialog (override of the MFC CPrintDialog
SaveDffd.h - dialog presented when a template has not been saved (save/save as/cancel)
NewScheme.h - small dialog used when creating a new color scheme
BookmarkFind.h - used by Find dialog when bookmarking found occurrences
DirDialog.h - directory selection dialog (see below)

Splasher.h - CSplashWnd for splash screen (see below)
TipWnd.h - CWnd derived class to show a small "tip" window (see below)
TransparentListBox.h, TransparentStatic2.h - transparent controls (see below)
CCalcEdit.h - CCalcEdit class which handles calculator edit box
control.h - various text and combo controls used in dialogs and on toolbars

GenDockablePane.h - dockable window used to makes Calculator, Find dialog etc dockable
ResizeCtrl.h - CResizeCtrl (see below)
BCGMisc.h - a few classes derived from BCG (now MFC) classes
UserTool.h - CHexEditUserTool is derived from CUserToolto allow command line substition for user tools
SimpleSplitter.h - CSimpleSplitter is used in Explorer dialog for the split between folder/file sections
HexEditSplitter.h - CHexEditSplitter allows showing aerial/template views in a split window
TabView.h - CHexTabView (derived from CTabView allows showing aerial/template views in tabs
CoordAp.h - CPointAp CSizeAp CRectAp (see below) for 64-bit (vertically) coordinate system

TParser.h - C/C++ code parser used by TParseDlg (see TParseDlg.h above)
HexEditMacro.h - handles recording and playback of Keystroke macros
HexFileList.h - stores global list of all recent files
SpecialList.h - stores info about that volumes and raw disks in the system
Bookmark.h - CBookmarkList class for global storage for bookmarks
Scheme.h - CScheme stores all color schemes
NavManager.h - global storage of navigation points
SystemSound.h - CSystemSound contains static method for getting, setting, playing sounds (see below)
Timer.h - timer class can time events (see below)

UpdateChecker.h - checks for the latest version of HexEdit via the Internet (see below)
Xmltree.h - wrapper of the MS XML SDK (see below)
BigInter.h - BigInteger class for handling numbers > 64-bits
CFile64.h - CFile64 class for 64-bit file access (see below)
crypto.h - CCrypto class wraps some of Crypto API (see below)
Boyer.h - boyer class for searching (see below)
range_set.h - template class for set with ranges (see below)
Expr.h - expr_eval class handles C-like expressions (used in templates etc)
IntelHex.h - CReadIntelHexand CWriteIntelHex (see below)
SRecord.h - CReadSRecordand CWriteSRecord (see below)

misc.h - miscellaneous global functions (see below)
md5.h - routines for generating an MD5 checksum
ntapi.h - declarations for direct access to NT API (bypassing Windows) for disk editor
w2k_def.h - more NT (not 9X) info - included by ntapi.h
optypes.h - defines unary and binary operations (for Calculator and Operations menu)
SVNRevisionTemplate.h - used by SVN uitlity to create SVNRevision.hSVNRevision.h - just stores HexEdit version information

Code of Interest

There are several classes and functions I have created for HexEdit that could easily be used elsewhere. In fact many people have emailed and asked for permission to use various parts of the code. (There are also components from other people that I have incorporated into the code - these are listed in the later Third Party Code section.)

Classes

boyer - Implements Boyer-Moore algorithm for searching a memory block. This is the reason HexEdit's searches are so fast. Over the years it has been enhanced to support options like case-insensitive searches, alignment, etc.

timer - Utility class used for timing tests. Many people have used or copied this.

range_set class - I originally wrote this as a template class which emulates the STL std::set class but much more efficiently handles ranges of values, as well as adding facilities to input/output ranges as a string. An article on it appeared in the June 1999 C/C++ Users Journal. Since then it has found numerous uses in HexEdit (as well as being used by many other people) such as in the color range selection (Colors page of Options dialog) or the matching range of values for a CASE of a template SWITCH statement.

CSystemSound - Contains very simple but effective system for defining and playing Windows sounds as seen in the Sound Control Panel applet. HexEdit uses this for many sounds such as for searches.

UpdateChecker - Simple class to read a version number from a text file on the Internet. HexEdit uses this for its monthly check for updates.

CReadSRecord and CWriteSRecord - Classes used in HexEdit to import to and export from Motorola-S format files. Similar classes exist for Intel-hex files.

CXmlTree - Class that wraps the Microsoft XML SDK (DOM based) to make it easily to process XML template files. It's incomplete and could be easier to use.

CCrypto - Class that wraps some of the Microsoft Crypto API. This is used to implement the encryption commands. Currently it only stores information about all the single key encryption algorithms (not public key). It also stores the current password.

CPointAp, CRectAp, CSizeAp - Originally I used the MFC CPoint, CRect and CSize classes but these had limitations on coordinate sizes (originally were 16-bit and are still limited to 32-bit coordinates). The "Ap" classes support 64-bit coordinates on the X-axis, and 32-bits on the Y-axis. These classes were necessary for the HexEdit views to properly display huge files.

CDirDialog - CFileDialog derived class that allows the user to select a directory (folder) by "subclassing" the standard Windows file open dialog. Unlike the Windows recommended SHBrowseForFolder (horrible) it has many useful features such as being able to see the files in a directory, pasting a full path name, etc, etc.

CTipWnd - CWnd derived class for displaying a small tip window. Use in various places in HexEdit such as Info Tip windows. Has facilities for changing text and background color and having the window fade in/out.

CFileNC - File handling class (derived from CFile64 - see below) for non-cached sector-based access. This is necessary according to MSWindows documentation for raw disk and volume reading used in disk editor. Note that other disk editors do not used non-cached file access, which contravenes Microsoft guidelines (but seems to work and in fact makes them faster).

CHexFileDialog - This is derived from the MFC CFileDialog class and is the base class for all the standard file dialogs (open, save, ect). It is used to remember and restore each file dialog's window position.

CScrView - This was the first class I ever wrote for HexEdit. It is derived from the MFC CView class and is the base class for the main view class (CHexEditView). I wrote it to be similar to CScrollView which I found had limitations. It is probably not that well designed but other people have used it in their software and found it useful.

CHexEdit, CDecEdit - Text controls that override the MFC CEdit class (text box) to allow editing of hex and decimal numbers, including space or comma separators.

CHexEditSplitter - Overrides the MFC CSplitterWnd to allows a dynamic (fixed maximum) number of split windows in each file window. This is used to allow the template and aerial views to be displayed in split windows next to the normal (hex) view.

Functions

There are also some useful global functions defined in misc.cpp.

Color processing functions include: BRIGHTNESS (macro), get_hls, get_rgb, tone_down, same_hue, etc. These are used in the display of many user-interface elements.

Number formatting functions are used in in various places in the user interface. AddSpaces/AddCommas are used for hex/decimal numbers (eg wherever addresses are displayed). NumScale is used for abbreviated display of large numbers (eg KByte, MByte etc).

Number conversions functions are used in the Floating Point Numbers Properties page for conversion of C# Decimals, IBM FP numbers and Turbo/Delphi real48. These are quite complex and took me a lot of work.

Date conversion functions are used in the Date Properties page.

Multiple monitor routines are used to make HexEdit behave well in a variety of situations that confuse other programs. Functions are: OutsideMonitor, MonitorMouse, MonitorRect.

Third Party Code

There are many pieces of code that I have incorporated into HexEdit to save writing them myself. These are all open source and generally, I have included the source code. CodeProject was an invaluable resource as most of the code listed below is MFC extensions that I found right here at CodeProject (all except zlib, FreeImage, Boost and CFile64).

I have not included the source for FreeImage as it is 10MBytes but you can download it from SourceForge. I have not included all the source for Boost, only the parts that are used by HexEdit as it is huge - you can download the full source code, documentation etc from their web site.

The installer that I use (WIX) is also open source but I have not included the WIX source only the files needed to build the installation. I also use a commercial help compiler so I have not included the source files for the help system. Only the final HexEdit.chm file is included.

As far as I am aware all of the code that I have incorporated into HexEdit is open source and free for reuse. Please email me if I have violated any licensing terms and I will rectify the situation.

Boost C++ library - mainly a huge number of template classes provided by several authors. Currently I only use the CRC and PRNG facilities. See http://www.boost.org/.

ZLIB Compression library by Jean-loup Gailly and Mark Adler - This is a free general purpose compression library (with C source available) which is used extensively in the industry. (I would consider it to be a de facto standard). I added this in HexEdit 3.2 partly for my own convenience as I have used the ZLIB library in several other projects. The Compression Options dialog offers complete control of all ZLIB options. See http://www.zlib.net/.

FreeImage DLL by Hervé Drolon et al - A brilliant open source library for dealing with image files and manipulating images in memory. It made possible the HexEdit Aerial View bit is also used in other places in HexEdit. It has been ported to several platforms but we only use the Win32 DLL that they provide. See http://freeimage.sourceforge.net/.

MFC Grid Control by Chris Maunder - An excellent grid control, which is used in several places in HexEdit - for example, in the Filters page and the Info Tips page of the Options dialog, the Recent File dialog and also (with Tree Extensions - see below) the template tree view. See http://www.codeproject.com/kb/miscctrl/gridctrl.aspx.

MFC Grid Tree Extensions by Ken Bertelson - An extension to the MFC grid control that allows one of the columns to act as a "tree control". This allows the effect of a multi-column tree control which is invaluable for the HexEdit template (tree) view. See http://www.codeproject.com/kb/miscctrl/gridtreectrl.aspx.

CFile64 Class by Sam Blackburn - When I released the source code for HexEdit 1.0 in 1999, Sam enhanced it to use "64-bit" file access, thereby allowing HexEdit to easily work with files greater than 4 GBytes in size. (Windows supports 64 bit file positioning but the MFC CFile class only allowed 32-bit file addresses). Sam's CFile64 class (incorporated, with modifications, into HexEdit) is just one of numerous useful (non-UI) classes provided by his WFC library. (Website no longer available.)

Window Splitter class by Robert A. T. Káldy - A simple, functional class that provides a nice-looking splitter. Unlike MFC splitters I found it very easy to use and extend for my needs. See http://www.codeproject.com/KB/splitter/kaldysimplesplitter.aspx.

Dialog Resize class by Herbert Menke - CResizeCtrl is a simple to use but very effective class that allows automatic move/resize of controls in a dialog when it is resized. This is used for all resizable dialogs in HexEdit such as the Recent File dialog, Bookmarks dialog etc. See http://www.codeproject.com/KB/dialog/resizectrl.aspx.

Big Integer class by Cap'n Code - In order to be able to display C# Decimal values (see Real Values Page of Properties dialog), I needed to be able to handle 96-bit integers. The Microsoft compiler only supports integers up to 64-bits but I found this class at CodeProject. Despite some caveats on its use by the author I cannot fault it after extensive testing. See CppIntegerClass.aspx.

Transparent Static Controls by Ali Rafiee - An MFC class that provides "see through" static controls. It is used in the HexEdit About box. See http://www.codeproject.com/KB/static/TransparentStaticCtrl.aspx.

CSplashWnd by P. J. Naughter - used for HexEdit's splash screen. (Can't find the page but I'm sure it was from CodeProject.)

MD5 algorithm by Ron Rivest - This algorithm is a cryptographic checksum (sometimes also called a one-way hash function or message digest). It generates a 128-bit value that is often used to check the veracity of files transferred over an insecure link in order to detect tampering. The MD5 command was added to HexEdit 3.2 as an MD5 value is often provided for files downloaded via the Internet.

WIX Installer Toolkit by Rob Mensching et al - A free MSI based installer which produces nice enough installations and is very flexible. However, it is somewhat difficult to understand and use. See http://wix.sourceforge.net/.

Subversion source control by Jim Blandy et al - Not part of the source code but crucial to the development, SVN (sub-version) provides an excellent free version control system. I have used many version control systems and this is easily the best. See http://subversion.tigris.org/.

TortoiseSVN SVN client - Stefan Küng et al - TSVN (TortoiseSVN) provides a very nice interface to SVN, integrating with Windows Explorer. (SVN itself only provides command line tools.) In the past I have found version control a pain but TSVN actually makes it enjoyable. See http://tortoisesvn.net/.  

Some Code - Double-Buffering using CMemDC  

The editors asked me to include some example code in the article. There are many pieces of code in HexEdit that are interesting, but I couldn't really decide what to explain now so I just chose the last thing I was working on. This was some code changes to improve the "marching ants" display in the Aerial View using double-buffering. It uses the little-known CMemDC class that comes with MFC.

If you don't know what "marching ants" and the Aerial View are then I will give a quick explanation. Aerial View is an alternative view of a file where each byte is shown as a single pixel the color of which is determined by the current color scheme. It is suprisingly useful as a high-level overview of a file. 

The Aerial View is also useful for getting an idea of where in the file different things are such as search occurrences, bookmarks, the selection etc. This can be shown in the left border of the Aerial View and within the view as "marching ants". Marching ants are like a slowly moving trail of ants. This makes them easier to see as the human eye is very good at detecting movement.

The problem with marching ants (in HexEdit 3.5) was that they are only drawn one pixel wide and when zoomed in on the Aerial View they were not as obvious as they could be. It was an easy matter to drawn the ants wider when zoomed in but this caused a flicker which was not obvious when the ants were thinner. Which brings us to double-buffering.

The flickering is a classic example of why "double-buffering" was invented. It is caused by the background (the aerial view bitmap) being drawn over the top of the old ants before the new ants are drawn at a slight offset. Double-buffering is used for animation by drawing each new image in an offscreen buffer before showing it. In this way artifacts of the drawing process are not visible to the user. 

It was a simple matter to convert the code to use double-buffering using the CMemDC class. First, here is what the old code (in AerialView.cpp) looked like.

BOOL CAerialView::OnEraseBkgnd(CDC* pDC)
{
    // Work out dimensions of window client area
    ...
    // Get background color and create new brush
    ...
    // Draw background of borders using device context (pDC)
    ...
    return TRUE;
}

void CAerialView::OnDraw(CDC* pDC)
{
    // Work out colors (selection, bookmarks, etc)
    ...
    // If not printing and not finished aerial view bitmap draw progress indicator
    ...
    // Draw margin indicators in left border using pDC
    ...
    draw_bitmap(pDC);   // draw the actual aerial view pixels

    // Draw marching ants on top of the bitmap using pDC
    ...
}

If you don't know how Windows handles window drawing I should explain that it is handled in two stages by the Windows messages WM_ERASEBKGND and WM_PAINT, which are roughly correspond to the above MFC CView methods OnEraseBkgnd and OnDraw. Having separate erase and draw stages is useful for a few reasons (eg to avoid flickering by drawing using the noErase flag) but since we are using double-buffering the need for the separate erase stage is obviated - instead we just erase at the start of drawing.

I have added double-buffering to MFC code in the past using the very useful Keith Rule's CMemDC class found right here on CodeProject (see http://www.codeproject.com/KB/GDI/flickerfree.aspx). Using it is a simple matter of creating a CMemDC instance (which creates the offscreen buffer) and then drawing to it's device context instead of the view's device context. When the CMemDC instance is destructed (at the end of OnDraw) it automatically BLTs the offscreen buffer to the display.

However, when I tried to compile the CMemDC code I got a name conflict. It seems that at some stage MFC added a CMemDC class of its own. So I used the MFC CMemDC class instead. Luckilly it works exactly the same way as Keith Rule's CMemDC (probably no coincidence).

The only code changes required were: 

  • Move all OnEraseBkgnd code to start of OnDraw. 
  • Create a CMemDC instance (bufDC) at the start of OnDraw.
  • Change uses of pDC to bufDC.GetDC().
BOOL CAerialView::OnEraseBkgnd(CDC* pDC)
{
    return TRUE;
}

void CAerialView::OnDraw(CDC* pDC)
{
    CMemDC bufDC(*pDC, this);

    // Work out dimensions of window client area
    ...
    // Get background color and create new brush
    ...
    // Draw background using buffered device context (bufDC.GetDC())
    ...
    // Work out colors (selection, bookmarks, etc)
    ...
    // If aerial view bitmap is still being generated then draw progress indicator
    ...
    // Draw indicators (selection, bookmarks, etc) in left border using bufDC.GetDC()
    ...
    draw_bitmap(&(bufDC.GetDC()));

    // Draw marching ants on top of the bitmap using bufDC.GetDC()
    ...
    // bufDC destructor gets called here (end of scope) which renders the buffered
    // bitmap onto the display (pDC)
}

How Can You Help? 

Bugs 

I do not believe there are many bugs in HexEdit, though there are a few design flaws. If you find a bug please report it in the HexEdit Bugs forum. Even better would be if you could track down the cause and send a fix.

Enhancements

I am more than willing to incorporate enhancements into HexEdit if you would like to submit code changes. You should probably check with me before you start to avoid duplication of effort.

Templates 

Several users of HexEdit have submitted templates which are included with the distribution.

If you have an idea for a template for a specific file format please contact us. If you have any problems creating or debugging your template please email me as I am very willing to assist.

Conclusion 

HexEdit 4.0 has several improvements over earlier versions. The main change is moving to the new MFC and using the dockable windows and other new GUI features.  But the most popular feature is still the template system.

If nothing else you may find HexEdit useful simply for viewing or editing binary data. You might be able to use of some of the code or discover how to use facilities of the new MFC. If you are really keen you can tinker with it and add features that you require.

History

Jan 2011 First article to coincide with release of HexEdit 4.0 beta.

July 2012 Minor update for official release of HexEdit 4.0.  Created three separate downloads: binary, project, extra build files.  The code has numerous minor bug fixes and a few improvements over the last beat version.  There is also a project files for VS2012.

License

This article, along with any associated source code and files, is licensed under The MIT License

Share

About the Author

Andrew Phillips

Australia Australia
Andrew has a BSc (1983) from Sydney University in Computer Science and Mathematics. Andrew began programming professionally in C in 1984 and has since used many languages but mainly C and C++.
 
Andrew has a particular interest in STL, user interface design and .Net. He has written articles on STL for technical journals such as the C/C++ User's Journal.
 
In 1997 Andrew began using MFC and released the source code for a Windows binary file editor called HexEdit, which was downloaded more than 1 million times. Since then he released a shareware version which is updated regularly (see http://www.hexedit.com). A new open source version using the wonderful new MFC is in the works.

Comments and Discussions

 
GeneralMy vote of 5 PinmemberEarleH22-Oct-12 10:14 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web02 | 2.8.1411019.1 | Last Updated 17 Oct 2012
Article Copyright 2011 by Andrew Phillips
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid