Click here to Skip to main content
Click here to Skip to main content

Image Warping Using Linear Interpolation and Double Buffer Panel

, 10 Dec 2007 CPOL
Rate this:
Please Sign up or sign in to vote.
An image warping tool in C# which uses simple displacement filters.

Introduction

Image warping is the process of digitally manipulating an image such that any shapes portrayed in the image have been significantly distorted. While image data could be transformed in various ways, a pure image distortion means that points are mapped to points without changing the colors. This can be based mathematically on any function from (part of) the plane to the plane. If the function is an injection, the original can be reconstructed. If the function is a bijection, any image can be inversely transformed. [Wikipedia]

Image warping is a very common technique used by people who work on image caricaturization or cartoonization. However, it is very hard to find the idea implemented on the Internet. Using some code from Christian Graus' great set of articles called "Image Processing For Dummies", I have developed a very primitive image warper, just to give the idea of the operation.

Image Processing in C#

C/C++ is the language of image processing because it is native code and fast. However, because we would like to incorporate the great .NET features with image processing, we would want to do image or graphics processing in C#. One way is to create C++ .NET wrappers, but if you don't want to bother with that, you can always implement them in C#, and most of the time, the algorithms end up performing much faster than you imagine. This is because by using unsafe code, we can get more native (even though not fully! .NET framework is still there, but less overhead) and have lower level access. This property of C# makes it more popular for graphics applications and algorithm implementations, day by day.

Bilinear Interpolation

In Mathematics, bilinear interpolation is an extension of linear interpolation for interpolating functions of two variables on a regular grid. The key idea is to perform linear interpolation first in one direction, and then in the other direction. This way, the problem of finding a suitable value to place in the warped image is solved. In most of the cases, it is enough for a good approximation. The diagram below just explains how the whole thing looks like. [Wikipedia]

Screenshot

Here is what Wiki says on bilinear interpolation:

 f(x,y) \approx f(0,0) \, (1-x)(1-y) + f(1,0) \, x(1-y) + f(0,1) \, (1-x)y + f(1,1) xy.

or, equivalently,

 f(x,y) \approx \begin{bmatrix}1-x & x \end{bmatrix} \begin{bmatrix}f(0,0) & f(0,1) \\f(1,0)

where f is the image function, and x and y are the coordinates of the current position.

In my use of bilinear interpolation, I determine the new pixel value for the destination image. The image is warped according to the mouse input. So, the pixel values around that mouse position are interpolated. This reduces the calculations.

Offset Filter

This filter can also be named as a displacement filter, and is totally taken from Christian Graus' code. Its basic job is to calculate the translation of pixels from the original position to the destination position. Please refer to his code for further details.

Double Buffering

Because user interaction is significantly high in this application, I use a double buffered panel. Double buffering, as the name implies, creates a buffer of image data before sending it to the screen handle. Because there is always a buffer, the screen will not flicker when the user interacts.

I have used the Panel control to achieve this. The new Panel class looks like this:

public class DoubleBufferPanel : Panel
{
    public DoubleBufferPanel()
    {
        // Set the value of the double-buffering style bits to true.



        this.SetStyle(ControlStyles.DoubleBuffer | ControlStyles.UserPaint |
                      ControlStyles.AllPaintingInWmPaint, true);

        this.UpdateStyles();
    }
}
// and we do everything on the paint event
// ....

// This is just the part that calculates and paints the new warped image. 
// The grid is also drawn, and the new values are calculated.


private void panel1_Paint(object sender, PaintEventArgs e)
{
    if (checkBox2.CheckState == CheckState.Checked)
    {
        Image temp = (Image)img.Clone();
        CalcOffsets();
        OffsetFilter((Bitmap)temp, GImg);
        if (img != null)
            e.Graphics.DrawImage(temp, new Rectangle
                (0, 0, panel1.Width, panel1.Height));
    }
    else
    {
        if (img != null)
            e.Graphics.DrawImage(img, new Rectangle
                (0, 0, panel1.Width, panel1.Height));
    }
    if (checkBox1.CheckState == CheckState.Checked)
    {
        Pen P = new Pen(Color.Red);
        for (int i = 1; i < 9; i++)
            for (int j = 1; j < 9; j++)
            {
                e.Graphics.DrawLine(P, new Point(i * 48 + G[i, j].X, j * 64 + 
                    G[i, j].Y), new Point(i * 48 + G[i, j + 1].X, 
                    (j + 1) * 64 + G[i, j + 1].Y));
                e.Graphics.DrawLine(P, new Point(i * 48 + G[i, j].X, 
                    j * 64 + G[i, j].Y), new Point((i + 1) * 48 + 
                    G[i + 1, j].X, j * 64 + G[i + 1, j].Y));
            }
    }
}

Understanding the Code

OK, here is the core: the interpolation. The grid cell size is 64*48 (I know it is pretty big), and interpolation is calculated in this grid (which is also not desired in some cases). G is the grid itself, and GImg is the image which is warped according to the deformation on the grid.

for (int m = 0; m < 48; m++)
{
    for (int n = 0; n < 64; n++)
    {
        double xfrac = (double)1 / 48.0;
        double yfrac = (double)1 / 64.0;
        double s = (G[i + 1, j].X - G[i, j].X) * m * xfrac + G[i, j].X;
        double t = (G[i + 1, j].Y - G[i, j].Y) * m * xfrac + G[i, j].Y;
        double u = (G[i + 1, j + 1].X - G[i, j + 1].X) * 
                m * xfrac + G[i, j+1].X;
        double v = (G[i + 1, j + 1].Y - G[i, j + 1].Y) * 
                m * xfrac + G[i, j+1].Y;
        GImg[i * 48 + m, j * 64 + n].X = -(int)( s + (u - s) * n * yfrac);
        GImg[i * 48 + m, j * 64 + n].Y = -(int)( t + (v - t) * n * yfrac);
    }
}

Using the Image Warp Tool

Just run the code, and check the "Show Original Image" and "Show Grids" checkboxes. Now, double click a point on the grid and translate that point. You will see the image is warped. When you double click again, you stop warping that point.

Further Improvements

  • The grid points should be closer so that the warping can be done from almost any location.
  • Some distortion caused by extending a certain cell in the image should be properly handled and corrected.
  • Bicubic interpolation can be implemented.
  • And some more, that I cannot remember right now.

Conclusion

This was just a simple attempt for me to develop a warping application, I then came up with more sophisticated ways. I believe this can help someone who is crazy about searching a simple warp idea, like me.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Tolga Birdal
CEO Gravi Information Technologies and Consultancy Ltd
Turkey Turkey
Currently, also an MSc. student in Technical University of Munich, I develop practical application in computer vision for more than 5 years. I design real-time solutions to industrial and practical vision problems, both 3D and 2D. Very interested in developing algorithms in C relating math and vision.
 
Please visit Gravi's web page (http://www.gravi.com.tr) and my page (http://www.tbirdal.me) to learn more about what we develop.
 
I admire performance in algorithms.
 
"Great minds never die, they just tend to infinity..."
Follow on   LinkedIn

Comments and Discussions

 
QuestionQuestion about calculations PinmemberMember 109719514-Aug-14 18:44 
AnswerRe: Question about calculations PinmemberMember 109719514-Aug-14 18:52 
GeneralRe: Question about calculations PinmemberTolga Birdal28-Aug-14 22:54 
QuestionAbsolutely excellent PinmemberMember 109719514-Aug-14 18:25 
QuestionIs this code work for any photo in bmp format? PinmemberOMER_KHAN18-Oct-11 5:57 
GeneralGood piece of code Pinmembernico4nicolas14-Apr-09 23:23 
GeneralRe: Good piece of code PinmemberTolga Birdal16-Apr-09 11:29 
GeneralThanks very mush. Pinmemberfutou31-Dec-08 5:23 
GeneralRe: Thanks very mush. PinmemberTolga Birdal1-Jan-09 6:45 
GeneralNice effect, code could use some refactoring... PinmemberJoan Charmant21-Mar-08 4:52 
GeneralRe: Nice effect, code could use some refactoring... PinmemberTolga Birdal21-Mar-08 5:04 
Hi Joan,
 
Thanks for your feedback. All those things you say are correct and are already in my mind.
However, I have written the code only for testing purposes and then moved to antother implementation. That's why I couldn't spend much time on it. I hope people use it, improve it and post to code project so that the code can improve.
 
Regards,
Tolga
QuestionDoesn't seem to do anything? Pinmemberstano10-Dec-07 3:11 
AnswerRe: Doesn't seem to do anything? PinmemberTolga Birdal10-Dec-07 5:56 
GeneralRe: Doesn't seem to do anything? PinmemberSupermanDT22-Aug-12 12:48 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.141223.1 | Last Updated 10 Dec 2007
Article Copyright 2007 by Tolga Birdal
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid