12,512,439 members (62,760 online)
alternative version

71.7K views
49 bookmarked
Posted

# Image Warping Using Linear Interpolation and Double Buffer Panel

, 10 Dec 2007 CPOL
 Rate this:
An image warping tool in C# which uses simple displacement filters.

## Introduction

Image warping is the process of digitally manipulating an image such that any shapes portrayed in the image have been significantly distorted. While image data could be transformed in various ways, a pure image distortion means that points are mapped to points without changing the colors. This can be based mathematically on any function from (part of) the plane to the plane. If the function is an injection, the original can be reconstructed. If the function is a bijection, any image can be inversely transformed. [Wikipedia]

Image warping is a very common technique used by people who work on image caricaturization or cartoonization. However, it is very hard to find the idea implemented on the Internet. Using some code from Christian Graus' great set of articles called "Image Processing For Dummies", I have developed a very primitive image warper, just to give the idea of the operation.

## Image Processing in C#

C/C++ is the language of image processing because it is native code and fast. However, because we would like to incorporate the great .NET features with image processing, we would want to do image or graphics processing in C#. One way is to create C++ .NET wrappers, but if you don't want to bother with that, you can always implement them in C#, and most of the time, the algorithms end up performing much faster than you imagine. This is because by using unsafe code, we can get more native (even though not fully! .NET framework is still there, but less overhead) and have lower level access. This property of C# makes it more popular for graphics applications and algorithm implementations, day by day.

## Bilinear Interpolation

In Mathematics, bilinear interpolation is an extension of linear interpolation for interpolating functions of two variables on a regular grid. The key idea is to perform linear interpolation first in one direction, and then in the other direction. This way, the problem of finding a suitable value to place in the warped image is solved. In most of the cases, it is enough for a good approximation. The diagram below just explains how the whole thing looks like. [Wikipedia]

Here is what Wiki says on bilinear interpolation:

$f(x,y) \approx f(0,0) \, (1-x)(1-y) + f(1,0) \, x(1-y) + f(0,1) \, (1-x)y + f(1,1) xy.$

or, equivalently,

$f(x,y) \approx \begin{bmatrix}1-x & x \end{bmatrix} \begin{bmatrix}f(0,0) & f(0,1) \\f(1,0)$

where f is the image function, and x and y are the coordinates of the current position.

In my use of bilinear interpolation, I determine the new pixel value for the destination image. The image is warped according to the mouse input. So, the pixel values around that mouse position are interpolated. This reduces the calculations.

## Offset Filter

This filter can also be named as a displacement filter, and is totally taken from Christian Graus' code. Its basic job is to calculate the translation of pixels from the original position to the destination position. Please refer to his code for further details.

## Double Buffering

Because user interaction is significantly high in this application, I use a double buffered panel. Double buffering, as the name implies, creates a buffer of image data before sending it to the screen handle. Because there is always a buffer, the screen will not flicker when the user interacts.

I have used the Panel control to achieve this. The new Panel class looks like this:

public class DoubleBufferPanel : Panel
{
public DoubleBufferPanel()
{
// Set the value of the double-buffering style bits to true.

this.SetStyle(ControlStyles.DoubleBuffer | ControlStyles.UserPaint |
ControlStyles.AllPaintingInWmPaint, true);

}
}
// and we do everything on the paint event
// ....

// This is just the part that calculates and paints the new warped image.
// The grid is also drawn, and the new values are calculated.

private void panel1_Paint(object sender, PaintEventArgs e)
{
if (checkBox2.CheckState == CheckState.Checked)
{
Image temp = (Image)img.Clone();
CalcOffsets();
OffsetFilter((Bitmap)temp, GImg);
if (img != null)
e.Graphics.DrawImage(temp, new Rectangle
(0, 0, panel1.Width, panel1.Height));
}
else
{
if (img != null)
e.Graphics.DrawImage(img, new Rectangle
(0, 0, panel1.Width, panel1.Height));
}
if (checkBox1.CheckState == CheckState.Checked)
{
Pen P = new Pen(Color.Red);
for (int i = 1; i < 9; i++)
for (int j = 1; j < 9; j++)
{
e.Graphics.DrawLine(P, new Point(i * 48 + G[i, j].X, j * 64 +
G[i, j].Y), new Point(i * 48 + G[i, j + 1].X,
(j + 1) * 64 + G[i, j + 1].Y));
e.Graphics.DrawLine(P, new Point(i * 48 + G[i, j].X,
j * 64 + G[i, j].Y), new Point((i + 1) * 48 +
G[i + 1, j].X, j * 64 + G[i + 1, j].Y));
}
}
}

## Understanding the Code

OK, here is the core: the interpolation. The grid cell size is 64*48 (I know it is pretty big), and interpolation is calculated in this grid (which is also not desired in some cases). G is the grid itself, and GImg is the image which is warped according to the deformation on the grid.

for (int m = 0; m < 48; m++)
{
for (int n = 0; n < 64; n++)
{
double xfrac = (double)1 / 48.0;
double yfrac = (double)1 / 64.0;
double s = (G[i + 1, j].X - G[i, j].X) * m * xfrac + G[i, j].X;
double t = (G[i + 1, j].Y - G[i, j].Y) * m * xfrac + G[i, j].Y;
double u = (G[i + 1, j + 1].X - G[i, j + 1].X) *
m * xfrac + G[i, j+1].X;
double v = (G[i + 1, j + 1].Y - G[i, j + 1].Y) *
m * xfrac + G[i, j+1].Y;
GImg[i * 48 + m, j * 64 + n].X = -(int)( s + (u - s) * n * yfrac);
GImg[i * 48 + m, j * 64 + n].Y = -(int)( t + (v - t) * n * yfrac);
}
}

## Using the Image Warp Tool

Just run the code, and check the "Show Original Image" and "Show Grids" checkboxes. Now, double click a point on the grid and translate that point. You will see the image is warped. When you double click again, you stop warping that point.

## Further Improvements

• The grid points should be closer so that the warping can be done from almost any location.
• Some distortion caused by extending a certain cell in the image should be properly handled and corrected.
• Bicubic interpolation can be implemented.
• And some more, that I cannot remember right now.

## Conclusion

This was just a simple attempt for me to develop a warping application, I then came up with more sophisticated ways. I believe this can help someone who is crazy about searching a simple warp idea, like me.

## Share

 CEO Gravi Information Technologies and Consultancy Ltd Turkey
Currently, also an MSc. student in Technical University of Munich, I develop practical application in computer vision for more than 5 years. I design real-time solutions to industrial and practical vision problems, both 3D and 2D. Very interested in developing algorithms in C relating math and vision.

"Great minds never die, they just tend to infinity..."

## You may also be interested in...

 Pro Pro

 First Prev Next
 Question about calculations Member 109719514-Aug-14 17:44 Member 10971951 4-Aug-14 17:44
 Re: Question about calculations Member 109719514-Aug-14 17:52 Member 10971951 4-Aug-14 17:52
 Re: Question about calculations Tolga Birdal28-Aug-14 21:54 Tolga Birdal 28-Aug-14 21:54
 Absolutely excellent Member 109719514-Aug-14 17:25 Member 10971951 4-Aug-14 17:25
 Is this code work for any photo in bmp format? OMER_KHAN18-Oct-11 4:57 OMER_KHAN 18-Oct-11 4:57
 Good piece of code nico4nicolas14-Apr-09 22:23 nico4nicolas 14-Apr-09 22:23
 Thank you for sharing it, I was looking for something like that. I have one question and one remark. First, where did you find the picture to illustrate the bilinear interpolation? I think it's a a very good one and I would like to use it as well, is there some rights on it? My remark is very simple, your code is good despite the fact that there is no comment but you should have done something more flexible, it would have been easy to have an adjustable grid and the size of the picture shouldn't be written, it should be a variable. There are some easy modifications to do so I can do that myself, you did the hardest part, thank you again for that.
 Re: Good piece of code Tolga Birdal16-Apr-09 10:29 Tolga Birdal 16-Apr-09 10:29
 Thanks very mush. futou31-Dec-08 4:23 futou 31-Dec-08 4:23
 Re: Thanks very mush. Tolga Birdal1-Jan-09 5:45 Tolga Birdal 1-Jan-09 5:45
 Nice effect, code could use some refactoring... Joan Charmant21-Mar-08 3:52 Joan Charmant 21-Mar-08 3:52
 Re: Nice effect, code could use some refactoring... Tolga Birdal21-Mar-08 4:04 Tolga Birdal 21-Mar-08 4:04
 Doesn't seem to do anything? stano10-Dec-07 2:11 stano 10-Dec-07 2:11
 Re: Doesn't seem to do anything? Tolga Birdal10-Dec-07 4:56 Tolga Birdal 10-Dec-07 4:56
 Re: Doesn't seem to do anything? SupermanDT22-Aug-12 11:48 SupermanDT 22-Aug-12 11:48
 Last Visit: 31-Dec-99 18:00     Last Update: 1-Oct-16 5:10 Refresh 1