Effective Handling of Multiple Images Using Atalasoft DotImage

This is a showcase review for our sponsors at CodeProject. These reviews are intended to provide you with information on products and services that we consider useful and of value to developers.

• Download a trial version

Abstract

Atalasoft DotImage includes many tools for manipulating or analyzing images. Included in the suite is a class called ImageSource which is the basis for being able to work effectively with an arbitrary number of images without worrying about the details of where they come from and how they’re managed. This document will describe the ImageSource class and its descendents in detail.

In batch image processing or in image manipulation applications, it is often necessary to work with large quantities of images. While it is possible to write code to manage these for specific cases, it is often more convenient to be able to solve this problem in the more general case and then use that as a basis for more specific cases.

In DotImage, the abstract class ImageSource does just this. The way to think of ImageSource is to think of it as half of a source/sink pair. An ImageSource is a place from which images come. The sink is an application or an image consumer. Images are managed through an acquire/release model. The ImageSource object performs the following services:

• Allows images to be acquired in order.
• Allows images to be released in any order.
• Tracks memory used by images that were made available.
• Automatically frees released images using either lazy or aggressive mechanisms.
• Allows limited reacquisition of released images.
• Allows for a reloading mechanism to allow images to be cached.

In this model, an image can be thought of as a resource. Rather than simply being read and used, an image is acquired from the ImageSource and then released when finished. Any number of consumers can acquire any given image, and it will only be released when each Acquire has been balanced with a Release.

In this way, an ImageSource can be used as follows:

public void ProcessImages(ImageSource source)
{
   while (source.HasMoreImages()) {
      AtalaImage image = source.AcquireNext();
      ProcessImage(image);
      source.Release(image);
   }
}

An image that has been acquired and not yet released can be acquired any number of times. In the example above, all the images within the ImageSource are processed serially. It is simple to make this a parallel process, by creating worker threads to perform the processing and allowing them to acquire and release the images as well. Structuring the code as follows makes that possible:

public void ProcessImages(ImageSource source)
{
   while (source.HasMoreImages()) {
      AtalaImage image = source.AcquireNext();
      CreateImageWorker(source, image, ProcessImage);
      source.Release(image);
   }
}

private void ProcessImage(ImageSource source, AtalaImage image)
{
   // do processing here

   source.Release(image);
}

public delegate void ProcessImageProc(ImageSource source, 
                                      AtalaImage image);

public void CreateImageWorker(ImageSource source,
            AtalaImage image, ProcessImageProc proc)
{
   source.Acquire(image); // Acquire here

   Thread t = CreateImageWorkerThread(source, image, proc);
   t.Start();
}

private Thread CreateImageWorkerThread(ImageSource source,
               AtalaImage image, ProcessImageProc proc)
{
   // threading details left out

}

In this code, the main loop acquires each image, passes it to CreateImageWorker, then releases it. CreateImageWorker calls Acquire (now the second time), then creates a worker thread to do the processing, starts it, and returns. The worker thread calls ProcessImage which does the work before calling Release. In this way, the images are processed in parallel instead.

ImageSource categorizes images into three groups, Acquired, Released, and Culled. An image that is Acquired is in memory and is available for use. An image that is Released is in memory, but should not be used until it has been reacquired. An image that is Culled is no longer in memory, but may have the facility to be reloaded.

For example, this code will always work:

TryOne(ImageSource source)
{
   source.Reset();
   AtalaImage image = source.AcquireNext();
   AtalaImage image1 = source.Acquire(0); // reacquire the 0th image

}

If image is non-null, image1 will always be non-null and identical to image.

This code will work in most cases:

TryTwo(ImageSource source)
{
   source.Reset();
   AtalaImage image = source.AcquireNext();
   source.Release(image);
   AtalaImage image1 = source.Acquire(0); // reacquire the 0th image

}

ImageSource will mark image as Released, and unless there are severe memory restrictions, the image can be reacquired. The resulting image should be checked for null, however.

This code will only reliably work if the particular ImageSource implements re-loadable images:

TryThree(ImageSource source)
{
   source.Reset();
   while (source.HasMoreImages()) {
      AtalaImage image = source.AcquireNext();
      source.Release(image);
   }
   // reacquire the 0th image

   AtalaImage image1 = source.Acquire(0);
}

The ability to reload an image is not defined within ImageSource, but is instead left to a class that descends from ImageSource. ImageSource on its own is geared perfectly for situations where an image can be accessed once and only once, such as a video source or a scanner with an automatic feeder.

Since not every ImageSource falls into this category, there is an abstract descendant of ImageSource called RandomAccessImageSource. For a RandomAccessImageSource, any image can be reliably acquired at any time and in any order. Again, images may be Acquired, Released, and Culled, but in this case, Acquire should always succeed.

RandomAccessImageSource adds the array operator to the object and the Count property. In this way, it is possible to access the image source in the following way:

public void ProcessImages(RandomAccessImageSource source)
{
   for (int i=0; i < source.Count; i++) {
      AtalaImage image = source[i]; // this does the acquire

      ProcessImage(image);
      source.Release(image);
   }
}

From here, it is a short step to get to the main concrete ImageSource class, FileSystemImageSource. FileSystemImageSource allows a client to iterate over a set of image files as well as multiple frames within image files that support that. Since it is clearly a variety of ImageSource that can trivially reload images, it descends from RandomAccessImageSource. As designed, FileSystemImageSource can iterate over all image files within a folder, all files matching a pattern within a folder, or through a list of files. Optionally, FileSystemImageSource will also iterate across all frames.

For better or for worse, the pattern matching is limited to that provided by .NET for files, which is not full regular expression matching. On one hand, it is consistent with the general Windows User Interface, but on the other hand, it is somewhat limited.

To avoid that inherent limitation, yet maintain compatibility, FileSystemImageSource includes a file filter hook to allow a client to perform all filtration of image files. By setting the FileFilterDelegate property to a method of the form:

bool MyFilter(string path, int frameIndex, int frameCount)
{
}

a client is able to allow or disallow any file based on its own criteria. By returning true from FileFilterDelegate, a file or frame within a file will be included. Returning false will ignore the file or frame.

To implement a custom ImageSource, create a class that inherits from either ImageSource or RandomAccessImageSource. A class that inherits from ImageSource asserts that it can provide a sequence of images in order. To do so, a class must implement the following abstract methods:

protected abstract ImageSourceNode LowLevelAcquireNextImage();

LowLevelAcquireNextImage gets the next available image in the sequence, and returns it packaged in an ImageSourceNode. An ImageSourceNode is used to manage an image while it is in memory. The main constructor for ImageSourceNode takes an AtalaImage as an argument and an object that implements the IImageReloader interface. An IImageReloader is a class that makes it possible to reload an image into memory. For a typical class inheriting from ImageSource, the LowLevelAcquireNextImage() will simply return a new ImageSourceNode with a valid image, but a null IImageReloader. This indicates that the image can’t be reloaded once it has been culled from memory. If it is not possible to acquire the next image, LowLevelAcquireNextImage should return null.

protected abstract bool LowLevelHasMoreImages();

LowLevelHasMoreImages returns a boolean indicating whether or not there are more images to be loaded.

protected abstract void LowLevelReset();

LowLevelReset is used to return an ImageSource to its starting state, if possible. For some ImageSources, this is not always possible. If it is not possible to Reset, this method should do nothing.

protected abstract void LowLevelSkipNextImage();

LowLevelSkipNextImage is called when an image that had previously been loaded is still available. For example, if ImageSource needs to load an image, it will call LowLevelAcquireNext, but if it determines that it is not necessary to load an image, it will not call LowLevelAcquireNext. In this case, it is necessary to allow your class to maintain its bookkeeping.

protected abstract void LowLevelDispose();

LowLevelDispose is called to allow a class to dispose of any non-reclaimable resources when the class is garbage collected. This might include closing files, releasing devices, closing network connections, etc.

protected abstract bool LowLevelFlushOnReset();

LowLevelFlushOnReset indicates whether or not ImageSource should dump all cached images upon Reset. For ImageSource varieties that will not return the same sequence of images every single time, this method should return true. Typically, most classes will return false to take full advantage of the cache.

protected abstract bool LowLevelTotalImagesKnown();

LowLevelTotalImagesKnown returns true if this ImageSource can know a priori how many images are available, false otherwise.

protected abstract int LowLevelTotalImages();

LowLevelTotalImages returns the total number of available images. If LowLevelTotalImagesKnown returns false, this will never be called.

A RandomAccessImageSource adds a new method to implement:

protected abstract ImageSourceNode LowLevelAcquire(int index);

LowLevelAcquire acts just like LowLevelAcquireNext except that it passes in an index. With this method, it’s convenient to implement LowLevelAcquireNext in terms of LowLevelAcquire.

It is important to note that a class that inherits from RandomAccessImageSource must provide an IImageReloader when it is asked to load an image. Without this, it is impossible to guarantee a robust operation of the ImageSource.

In addition, RandomAccessImageSource implements LowLevelTotalImagesKnown, returning true.

The real power in ImageSource is the ability to create new sources that can be used generically. What follows is a complete example of an ImageSource that can access Windows AVI files.

In this class, we want to be able to load every frame of an AVI file. Since AVI files can be read at any point, this is a good candidate for a RandomAccessImageSource as the base class, although a plain ImageSource would work.

This class contains a number of PInvoke definitions that link directly to the Win32 AVI calls. A discussion of the operation of these methods is beyond the scope of this document.

Most of the work is in opening the AVI file and loading a frame. All the rest of the abstract members of RandomAccessImageSource end up being one line methods. This is a very good thing as it leads to highly robust software.

using System;
using System.Runtime.InteropServices;
using Atalasoft.Imaging;

namespace AviSource
{
   public class AviImageSource : RandomAccessImageSource
   {
       string _fileName;
       IntPtr _aviFileHandle = IntPtr.Zero;
       int _currentFrame = 0;
       int _firstFramePosition;
       int _totalFrames = 0;
       IntPtr _aviStream = IntPtr.Zero;
       AVISTREAMINFO _streamInfo = new AVISTREAMINFO();

       static AviImageSource()
       {
          AVIFileInit();
       }

       public AviImageSource(string fileName)
       {
          _fileName = fileName;
          // LowLevelReset will force the file to be loaded

          // and will fetch all the relevant information

          LowLevelReset();
       }
   
       protected override void LowLevelReset()
       {
          // attempt to load the file if we haven't

          if (_aviFileHandle == IntPtr.Zero) 
          {
              OpenAvi();
              LoadAviInfo();
          }
          // reset the frame counter

          _currentFrame = 0;
       }

       private void CloseAvi()
       {
          // clear everything out

          _currentFrame = 0;
          _totalFrames = 0;

          // if the file handle is non-null, there may be a stream to close

          if (_aviFileHandle != IntPtr.Zero) 
          {
              // if the stream handle is non-null, close it

              if (_aviStream != IntPtr.Zero) 
              {
                 AVIStreamRelease(_aviStream);
                 _aviStream = IntPtr.Zero;
              }
              AVIFileRelease(_aviFileHandle);
              _aviFileHandle = IntPtr.Zero;
          }
       }

       private void OpenAvi()
       {
          // open the file and get a stream interface

          int result = AVIFileOpen(out _aviFileHandle, _fileName,
                       32 /*OF_SHARE_DENY_WRITE*/, 0);
          if (result != 0) 
              throw new Exception("Unable to open avi file " + 
                        _fileName + " (" + result + ")");

          result = AVIFileGetStream(_aviFileHandle, out _aviStream, 
                   0x73646976 /* 'vids' -> four char code */, 0);
          if (result != 0) 
              throw new Exception("Unable to get video stream (" + 
                                                    result + ")");
       }

       private void LoadAviInfo()
       {
          if (_aviStream == IntPtr.Zero)
              throw new Exception("LoadAviInfo(): Bad stream handle.");

          // get first frame

          _firstFramePosition = AVIStreamStart(_aviStream);
          if (_firstFramePosition < 0) 
              throw new Exception("LoadAviInfo():" + 
                    " Unable to get stream start position.");

          // get total frame count

          _totalFrames = AVIStreamLength(_aviStream);
          if (_totalFrames < 0) 
              throw new Exception("LoadAviInfo(): " + 
                     "Unable to get stream length.");

          // pull in general information

          int result = AVIStreamInfo(_aviStream, ref _streamInfo,
                           Marshal.SizeOf(_streamInfo));
          if (result != 0) 
              throw new Exception("LoadAviInfo(): unable " + 
                        "to get stream info (" + result + ")");
       }

       // this method retrieves a frame from the file.

       // the class is internal because it will be used by

       // the AviImageReloader class.

       internal AtalaImage GetAviFrame(int frame)
       {
          // set up a bitmap info header to make a frame request

          BITMAPINFOHEADER bih = new BITMAPINFOHEADER();
          bih.biBitCount = 24;
          bih.biCompression = 0; //BI_RGB;

          bih.biHeight = _streamInfo.frameBottom;
          bih.biWidth = _streamInfo.frameRight;
          bih.biPlanes = 1;
          bih.biSize = (uint)Marshal.SizeOf(bih);

          // the getFrameObject is an accessor for retrieving a frame

          // from an AVI file. We could make exactly one when the stream

          // is opened, but this works just fine.


          IntPtr frameAccessor = AVIStreamGetFrameOpen(_aviStream, ref bih);
          if (frameAccessor == IntPtr.Zero) 
              throw new Exception("Unable to get frame decompressor.");

          IntPtr theFrame = AVIStreamGetFrame(frameAccessor, 
                               frame + _firstFramePosition);

          if (theFrame == IntPtr.Zero) 
          {
              AVIStreamGetFrameClose(frameAccessor);
              throw new Exception("Unable to get frame #" + frame);
          }

          // make a copy of this image

          AtalaImage image = AtalaImage.FromDib(theFrame, true);
          // closing the frame accessor drops

          // the memory used by the frame as well

          AVIStreamGetFrameClose(frameAccessor);

          return image;
       }

       protected override ImageSourceNode LowLevelAcquireNextImage()
       {
          if (_currentFrame >= _totalFrames)
              return null;
          AtalaImage image = GetAviFrame(_currentFrame);
          if (image != null) 
          {
              ImageSourceNode node = new ImageSourceNode(image, null);
              _currentFrame++;
              return node;
          }
          return null;
       }

       protected override ImageSourceNode LowLevelAcquire(int index)
       {
          if (index < 0 || index >= _totalFrames)
              return null;
          AtalaImage image = GetAviFrame(index);
          if (image != null) 
          {
              ImageSourceNode node = new ImageSourceNode(image, 
                            new AviImageReloader(this, index));
              _currentFrame++;
              return node;
          }
          return null;
       }

       protected override bool LowLevelTotalImagesKnown()
       {
          return true;
       }

       protected override int LowLevelTotalImages()
       {
          return _totalFrames;
       }

       protected override bool LowLevelHasMoreImages()
       {
          return _currentFrame < _totalFrames;
       }

       protected override void LowLevelSkipNextImage()
       {
          _currentFrame++;
       }

       protected override bool LowLevelFlushOnReset()
       {
          return true;
       }

       protected override void LowLevelDispose()
       {
          CloseAvi();
       }

   
       #region AviHooks
       [DllImport("avifil32.dll")]
       private static extern void AVIFileInit();

       [DllImport("avifil32.dll", PreserveSig=true)]
       private static extern int AVIFileOpen(
          out IntPtr ppfile,
          String szFile,
          int uMode,
          int pclsidHandler);

       [DllImport("avifil32.dll")]
       private static extern int AVIFileGetStream(
          IntPtr pfile,
          out IntPtr ppavi,  
          int fccType,       
          int lParam);

       [DllImport("avifil32.dll")]
       private static extern int AVIStreamRelease(IntPtr aviStream);

       [DllImport("avifil32.dll")]
       private static extern int AVIFileRelease(IntPtr pfile);

       [DllImport("avifil32.dll")]
       private static extern void AVIFileExit();

       [DllImport("avifil32.dll", PreserveSig=true)]
       private static extern int AVIStreamStart(IntPtr pAVIStream);

       [DllImport("avifil32.dll", PreserveSig=true)]
       private static extern int AVIStreamLength(IntPtr pAVIStream);

       [DllImport("avifil32.dll")]
       private static extern int AVIStreamInfo(
          IntPtr pAVIStream,
          ref AVISTREAMINFO psi,
          int lSize);

       [DllImport("avifil32.dll")]
       private static extern IntPtr AVIStreamGetFrameOpen(
          IntPtr pAVIStream,
          ref BITMAPINFOHEADER bih);

       [DllImport("avifil32.dll")]
       private static extern IntPtr AVIStreamGetFrame(
          IntPtr pGetFrameObj,
          int lPos);

       [DllImport("avifil32.dll")]
       private static extern int AVIStreamGetFrameClose(IntPtr pGetFrameObj);     
       #endregion
   }
}

In addition to this class, it is necessary to have a class that implements IImageReloader. For this, we provide an AviReloader class which encapsulates enough information to reload a frame from a file. In this case, it is the frame index and the AviImageSource from which it came. AviImageSource has one internal method which extracts a frame and converts it to an AtalaImage. Rather than keep any more information than is needed, we can just use this method. This assumes that the AVI file and the associated stream will still be open when the image is reloaded, but since this is kept across the life of the AviImageSource object, this is a safe assumption to make.

using System;
using Atalasoft.Imaging;

namespace AviSource
{
   public class AviImageReloader : IImageReloader
   {
       private int _frame;
       private AviImageSource _source;
       public AviImageReloader(AviImageSource source, int frame)
       {
          _source = source;
          _frame = frame;
       }

       #region IImageReloader Members

       public AtalaImage Reload()
       {
          return _source.GetAviFrame(_frame);
       }

       #endregion

       #region IDisposable Members

       public void Dispose()
       {
       }

       #endregion
   }
}