Image Resizing - outperform GDI+

• Download demo project - 10.0 KB
• Download source - 17.5 KB

Introduction

Even the GDI+ methods aren't the most comprehensive. A commonly used image resizing methods are e.g. Nearest Neighbor, Bilinear or Bicubic. These are part of some "standard", but experts and freaks know that these filters doesn't provide as nice images as other filters can. In some software products (especially specialized software for astronomical and medical imaging), we can see filters with strange names like Hermite, Mitchell or Lanczos. These filters works with reference to image's frequency spectra or edges. One of the most unpleasant artifacts is the Moiré effect which can appear while scaling detailed image down. In order to prevent these artifacts, enhance edges and reduce pixelation - more complex filters were designed.

In spite of GDI+, filters have big problems with Moiré patterns, they make some artifacts on image borders (it's not an error and can be eliminated, as will be discussed later). This article presents class for image resampling with more precise filters.

Background

The first mentoined artifact was the Moiré pattern which appears due to aliasing. This artifact can be seen when one shrinks images without blurring image before zooming, so the high frequencies in the image create a pattern. Samples taken from old images create new frequencies in the new image. This effect is most visible when using a filter with low radius (this filter includes too few adjacent pixels around sample from old image). These examples shows the aliasing artifact produced by a so called Nearest Neighbor filter (first image from left).

Much better quality can be obtained by using the GDI+ Low filter (second image - filter has radius 1). Even better quality is provided by the High Quality Bicubic filter (radius 2). Finally, there is the image output from filtering via Lanczos8 filter (radius 8) from the ResamplingService class. This filter has three times larger radius, so it provides highest quality image from these filters (roof tiles are more visible and sharper):

The original roof image can be found in Wikipedia in the "Moiré pattern" article. It's hard to see, but that last image is sharper than the third one, but there is other more visible defect in the GDI+ resampled version:

This problem can be easily solved using the ImageAttributes object in DrawImage method override (don't forget to dispose it):

System.Drawing.Imaging.ImageAttributes attr = 
    new System.Drawing.Imaging.ImageAttributes();

attr.SetWrapMode(System.Drawing.Drawing2D.WrapMode.TileFlipXY);
//...

attr.Dispose();

An alternative solution is to set the PixelOffsetMode property of the Graphics object to an appropriate value:

Graphics grfx = Graphics.FromImage(imgInput);

grfx.PixelOffsetMode = PixelOffsetMode.HighQuality;

Image Resampling with GDI+ Filters

Image resizing (or resampling) is one of the most common functions of every raster image processing tool. If you've decided for GDI+ to resize your images, you can choose from variety of filters. So called Nearest Neighbor (System.Drawing.Drawing2D.InterpolationMode.NearestNeighbor) filter is the fastest one, but it produces blocky artifacts on image stretching because this filter just copies samples from old image to a new image (no interpolation between them). On the contrary, when downsampling by factor 2, every second pixel is lost:

Much better results can be achieved using some interpolation spline. When the image is scaled up, a space between pixels must be filled with some new pixel values. Value of the new pixel can be computed as weighted average of nearest pixel and its neighbors (weights are given by the filter). One of the simplest interpolation/resampling filter is Linear filter (InterpolationMode.Low). The weight of neighboring pixels is a linear function. If you want achieve the "best" quality, then use High Quality Bicubic (InterpolationMode.HighQualityBicubic). This interpolation includes more neighbors in every new pixel value and weights are distributed more effectively (the function is a cubic spline):

The interpolation of the scaled image is done by image filtering, or convolution between image (intensity function) and convolution kernel. If you want closer explanation of how convolution works, see one of the excellent articles from Christian Grauss at this site. He also explains bilinear interpolation.

Image Resampling with Custom Filters

Now point our view to ResamplingService.cs. This code file contains few classes for precise resampling. The most important class is ResamplingService, whose method provides all the functionality.

There is one class representing common resampling filter:

public abstract class ResamplingFilter

When you run the resampling, a new instance of ResamplingFilter is created as well. The filter depends on the Filter property of the ResamplingService object.

A few people mentioned the fact that older version of this library could not resample images with alpha channel. So I've made it much more versatile after this feedback, primarily by using the ushort array (except GDI+ Bitmap). Benefits of this approach are listed below. To resample a GDI+ Bitmap, one must add some functionality to provide conversion between Bitmap and ushort[][,] types (the conversion routines are made for 32 bits per pixel with alpha, but can be easily extended to support arbitrary color formats).

Resampling using ResamplingService has a disadvantage in speed (it's not real-time), but you'll obtain fine results:

In the extreme case, we can compare results on upsampling four times. The results are obtained using GDI+ HQ Bicubic filter (left), Paint Shop PRO 8 Smart Size (middle) and current implementation of Lanczos8 filter:

About the Filters

I've implemented a collection of different resampling filters. Some of them provide even better results than filters in professional imaging applications. ResamplingService supports these filters (you can add your own filter by inheriting the ResamplingFilter class and adding new filter creation in the Create method of the base class). Here is the brief description of ResamplingFilters enumeration members:

Box	equivalent to Nearest Neighbor on upsampling, averages pixels on downsampling
Triangle	equivalent to Low; the function can be called Tent function for its shape
Hermite	use of the cubic spline from Hermite interpolation
Bell	attempt to compromise between reducing block artifacts and blurring image
CubicBSpline	most blurry filter (cubic Bezier spline) - known samples are just "magnets" for this curve
Lanczos3	windowed Sinc function (sin(x)/x) - promising quality, but ringing artifacts can appear
Mitchell	another compromise, but excellent for upsampling
Cosine	an attemp to replace curve of high order polynomial by cosine function (which is even)
CatmullRom	Catmull-Rom curve, used in first image warping algorithm (did you see Terminator II ?)
Quadratic	performance optimized filter - results are like with B-Splines, but using quadratic function only
QuadraticBSpline	quadratic Bezier spline modification
CubicConvolution	filter used in one example, its weight distribution enhances image edges
Lanczos8	also Sinc function, but with larger window, this function includes largest neighborhood

Using the Code

The project demo solution is extremely simple to understand. All you have to do is to use ResamplingService in your own project by copying ResamplingService.cs into your project directory and start using this with a few lines of code:

ResamplingService resamplingService = new ResamplingService();

resamplingService.Filter = ResamplingFilters.CubicBSpline;

ushort[][,] input = ConvertBitmapToArray((Bitmap)imgInput);

ushort[][,] output = resamplingService.Resample(input, nSize.Width, 
    nSize.Height);

Image imgCustom = (Image)ConvertArrayToBitmap(output);

This code will create a ResamplingService instance, sets its Filter property such that it will use the CubicBSpline filter, converts the input image to array, resamples the array bitmap into a new array, and finally converts the output array to image. Note that conversion methods in the sample code works only with bitmaps whose pixel format is 32bpp with alpha (4 planes).

Beyond Conventional Methods

None of the presented filters (GDI+ or other) can preserve edges, add artifcial detail or do something more on image. Somebody can ask: "And what about that expensive zooming software, that uses edge-enhancing diffusion, adaptive splines or fractals to achieve good visual quality?"

The truth is, that these technologies are licensed, but not unknown. Everyone can design some adaptive filter (actual kernel depends on current pixel neighborhood). But these filters are too slow for real-time applications (TV, DVD, games, ...). Yes, of course, it's very practical for photographers. They preserve quality before performance.

I'm not familiar with spline approaches, but I've done some investigation on fractals, because it can be used to add good-looking detail into natural images, not only keep image edges. This samples were made by using nontrivial fractal resolution enhancement technique:

Many students on my faculty are interested in digital photography, so the fractal library will be freely available for academic purposes. This is a better choice than very expensive professional zooming software on the market. The only only difference between academic version and the professional is the ratio between quality and speed. PRO software is quick with compromise results, the scientific approach needs to sacrifice tens of minutes to zoom an average-sized image, but the quality gain is sensible (here in 400% zoom)...

Original crop of the wood in fall image:

Photoshop's Bicubic filter:

Genuine Fractals PrintPro - top zooming software available:

Our implementation of fractal-based zooming:

Conclusion

I hope you've got a brief look onto the resampling topic. We discussed a way how to work with your own filters. To get more information, try to look for some technical articles. You can find several other implementations of resampling methods, but be careful about its properties. I tested my class for many grotesque image sizes (like 1x2397 from 191x2 pixels) and eliminated all artifacts on edges, repairing wrong weight distributions I saw in other sources.

One programmer showed me his GUI specialized on resampling. I've discovered and removed a terrible problem, which was causing black and white stripes over all images when reducing its size by an odd factor - of course due to the wrong weight distribution. This was a week before his resampling app became commercial component. I think the presented class is OK, but if you find anything or if you have any ideas how to optimize this common filtering algorithm, I'll be happy to discuss it.

Changes in the Last Version

The code was modified and made more readable (more private access modifiers, global variables reduced, more blank lines, indentation and XML comments). The library works with ushort[][,] except Bitmap. Why ushort[][,] and why not something else like byte[,]? I've chosen this type because it can contain a bitmap with almost any thinkable color format (from black&white image to 16-bit per channel image plus alpha) and all the color formats can be handled the same way. The jagged array of ushort (ushort range is four bytes or 0-65535) can contain any number of planes. The reason for jagged array except only ushort[,] is that the resampling procedure precomputes pixel weights and then filters every plane with same weights, so this solution gives better performance on color images. The code is now 100% managed (no unsafe code blocks). The way the library is used was also changed and the filters were enumerated to be more transparent.