Image Warping Using Linear Interpolation and Double Buffer Panel

• Download source - 766 KB

Introduction

Image warping is the process of digitally manipulating an image such that any shapes portrayed in the image have been significantly distorted. While image data could be transformed in various ways, a pure image distortion means that points are mapped to points without changing the colors. This can be based mathematically on any function from (part of) the plane to the plane. If the function is an injection, the original can be reconstructed. If the function is a bijection, any image can be inversely transformed. [Wikipedia]

Image warping is a very common technique used by people who work on image caricaturization or cartoonization. However, it is very hard to find the idea implemented on the Internet. Using some code from Christian Graus' great set of articles called "Image Processing For Dummies", I have developed a very primitive image warper, just to give the idea of the operation.

Image Processing in C#

C/C++ is the language of image processing because it is native code and fast. However, because we would like to incorporate the great .NET features with image processing, we would want to do image or graphics processing in C#. One way is to create C++ .NET wrappers, but if you don't want to bother with that, you can always implement them in C#, and most of the time, the algorithms end up performing much faster than you imagine. This is because by using unsafe code, we can get more native (even though not fully! .NET framework is still there, but less overhead) and have lower level access. This property of C# makes it more popular for graphics applications and algorithm implementations, day by day.

Bilinear Interpolation

In Mathematics, bilinear interpolation is an extension of linear interpolation for interpolating functions of two variables on a regular grid. The key idea is to perform linear interpolation first in one direction, and then in the other direction. This way, the problem of finding a suitable value to place in the warped image is solved. In most of the cases, it is enough for a good approximation. The diagram below just explains how the whole thing looks like. [Wikipedia]

Here is what Wiki says on bilinear interpolation:

or, equivalently,

where f is the image function, and x and y are the coordinates of the current position.

In my use of bilinear interpolation, I determine the new pixel value for the destination image. The image is warped according to the mouse input. So, the pixel values around that mouse position are interpolated. This reduces the calculations.

Offset Filter

This filter can also be named as a displacement filter, and is totally taken from Christian Graus' code. Its basic job is to calculate the translation of pixels from the original position to the destination position. Please refer to his code for further details.

Double Buffering

Because user interaction is significantly high in this application, I use a double buffered panel. Double buffering, as the name implies, creates a buffer of image data before sending it to the screen handle. Because there is always a buffer, the screen will not flicker when the user interacts.

I have used the Panel control to achieve this. The new Panel class looks like this:

public class DoubleBufferPanel : Panel
{
    public DoubleBufferPanel()
    {
        // Set the value of the double-buffering style bits to true.



        this.SetStyle(ControlStyles.DoubleBuffer | ControlStyles.UserPaint |
                      ControlStyles.AllPaintingInWmPaint, true);

        this.UpdateStyles();
    }
}
// and we do everything on the paint event
// ....

// This is just the part that calculates and paints the new warped image. 
// The grid is also drawn, and the new values are calculated.


private void panel1_Paint(object sender, PaintEventArgs e)
{
    if (checkBox2.CheckState == CheckState.Checked)
    {
        Image temp = (Image)img.Clone();
        CalcOffsets();
        OffsetFilter((Bitmap)temp, GImg);
        if (img != null)
            e.Graphics.DrawImage(temp, new Rectangle
                (0, 0, panel1.Width, panel1.Height));
    }
    else
    {
        if (img != null)
            e.Graphics.DrawImage(img, new Rectangle
                (0, 0, panel1.Width, panel1.Height));
    }
    if (checkBox1.CheckState == CheckState.Checked)
    {
        Pen P = new Pen(Color.Red);
        for (int i = 1; i < 9; i++)
            for (int j = 1; j < 9; j++)
            {
                e.Graphics.DrawLine(P, new Point(i * 48 + G[i, j].X, j * 64 + 
                    G[i, j].Y), new Point(i * 48 + G[i, j + 1].X, 
                    (j + 1) * 64 + G[i, j + 1].Y));
                e.Graphics.DrawLine(P, new Point(i * 48 + G[i, j].X, 
                    j * 64 + G[i, j].Y), new Point((i + 1) * 48 + 
                    G[i + 1, j].X, j * 64 + G[i + 1, j].Y));
            }
    }
}

Understanding the Code

OK, here is the core: the interpolation. The grid cell size is 64*48 (I know it is pretty big), and interpolation is calculated in this grid (which is also not desired in some cases). G is the grid itself, and GImg is the image which is warped according to the deformation on the grid.

for (int m = 0; m < 48; m++)
{
    for (int n = 0; n < 64; n++)
    {
        double xfrac = (double)1 / 48.0;
        double yfrac = (double)1 / 64.0;
        double s = (G[i + 1, j].X - G[i, j].X) * m * xfrac + G[i, j].X;
        double t = (G[i + 1, j].Y - G[i, j].Y) * m * xfrac + G[i, j].Y;
        double u = (G[i + 1, j + 1].X - G[i, j + 1].X) * 
                m * xfrac + G[i, j+1].X;
        double v = (G[i + 1, j + 1].Y - G[i, j + 1].Y) * 
                m * xfrac + G[i, j+1].Y;
        GImg[i * 48 + m, j * 64 + n].X = -(int)( s + (u - s) * n * yfrac);
        GImg[i * 48 + m, j * 64 + n].Y = -(int)( t + (v - t) * n * yfrac);
    }
}

Using the Image Warp Tool

Just run the code, and check the "Show Original Image" and "Show Grids" checkboxes. Now, double click a point on the grid and translate that point. You will see the image is warped. When you double click again, you stop warping that point.

Further Improvements

• The grid points should be closer so that the warping can be done from almost any location.
• Some distortion caused by extending a certain cell in the image should be properly handled and corrected.
• Bicubic interpolation can be implemented.
• And some more, that I cannot remember right now.

Conclusion

This was just a simple attempt for me to develop a warping application, I then came up with more sophisticated ways. I believe this can help someone who is crazy about searching a simple warp idea, like me.