Click here to Skip to main content
Click here to Skip to main content
Articles » Multimedia » OpenGL » General » Revisions
 

Zoom an image with different interpolation types.

By , 21 Aug 2011
Rate this:
Please Sign up or sign in to vote.
This is an old version of the currently published article.

Table Of Contents 

Introduction 

When a small region of input image is displayed to large area, interpolation is a required factor. OpenGL provides two interpolation methods to create a large(zoomed) image display of a small region of a texture.  If we provide better interpolation, the output image quality will be high. Interpolation is a "Process used to estimate an unknown value between two known values by utilizing a common mathematical relation". ZoomInterpolatoin creates zoomed image with different interpolation types. OpenGL provides a Bi-Linear interpolation at its maximum. When zooming a very small texture area to large area, Bi-Linear provides a less quality of zoomed image. 

Interpolation is a method of creating intermediate pixels, from the nearest valid pixels by applying some equations. Bi-Linear interpolation creates an intermediate pixel with the help of nearest 4 pixels. Bi-Cubic interpolation, a high quality version, creates an intermediate pixel with the help of nearest 16 pixels. 

ScreenShotNew2.jpg

Screenshot of ZoomInterpolation Application. Small area of flower is displayed without blocky edges. Bi-Cubic BSpline method is used to prepare this image. 

Background 

In OpenGL, we can specify an interpolation type(Filter type) to create a resized image of texture. When a texture or region of a texture is mapped to a higher or lower screen region, OpenGL uses this interpolation type to create texture image in different size.

To create a large image from a small texture area, OpenGL provides two interpolation options, GL_NEAREST, and GL_LINEAR. OpenGL texture mapping provides a maximum of 2*2 Bi-Linear interpolation. In addition to OpenGL interpolation types, here different versions of Bi-Cubic interpolation is implemented in pixel shaders.    

OpenGL GL_NEAREST Interpolation.  

GL_NEAREST just copies data available in nearest pixel and it create a blocky effect when a very small region is zoomed to a high screen region. In nearest interpolation, intermediate pixels are created with nearest valid pixel. This kind of interpolated images are blocky since same data is copied to intermediate pixels.

Below image is created through OpenGL GL_NEAREST interpolation and its quality is very bad. Edge of the flower seems to be a "stair case". We can remove this stair case through better interpolation methods.  

 2_Nearest.jpg

Interpolation with GL_NEAREST interpolation type.  

We can use the following code to achieve nearest interpolation in OpenGL fixed function pipeline. 

glTexParameteri( GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER, GL_NEAREST); 
glTexParameteri( GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER, GL_NEAREST);   

   

OpenGL GL_LINEAR Interpolation.  

OpenGL provides a good interpolation method, GL_LINEAR. It interpolates nearest 4 pixels and create a smooth image comparing to the GL_NEAREST method. GL_LINEAR method interpolate both in X and Y direction. Therefore it called Bi-Linear interpolation.   

Following image is created with GL_LINEAR interpolation and its edges are smooth when comparing to the image created through GL_NEAREST interpolation. Here "stair case" effect is not visible. But we can see a shade of yellow and green through the edge of flower, and a small "stair case" visible at some edges of the flower.     

 2_Linear.jpg

Interpolation with GL_LINEAR interpolation type.    

The following code can be used to achieve Bi-Linear interpolation in OpenGL fixed function pipeline.    

glTexParameteri( GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER, GL_LINEAR ); 
glTexParameteri( GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER, GL_LINEAR );    

GLSL Bi-Linear 

OpenGL provides only two interpolations methods to create zoomed display of an image(texture). When zooming a very small region of a texture, we can see some blocky edges in the output image. 

The first step to create a better interpolation method is to create Bi-Linear interpolation( Which is same as that of GL_LINEAR interpolation. But we prepare a pixel shader with Bi-Linear interpolation). 

A pixel shader is a program which will execute for each pixels of a rendering. More details of Pixel Shaders are avaialable at http://en.wikipedia.org/wiki/Pixel_shader or http://nehe.gamedev.net/article/glsl_an_introduction/25007/   

Bi-Linear Shader Implementation 

In Bi-Linear interpolation nearest four pixels are considered to create an intermediate pixel. 

BiLinearLogic.png

In the above figure an intermediate pixel F(p’,q’) is created by interpolating nearest four texels F(p,q), F(p,q+1), F(p+1,q), F(p+1,q+1).

Texel is the term used to indicate an element in a texture( similar to a pixel in screen)[A texel, or texture element (also texture pixel) is the fundamental unit of texture space].   

Initially two texels in top row and bottom row are interpolated with linear interpolation[horizontal]. Then the interpolated pixels of the top row and bottom row are also linearly interpolated[vertical]. Explanation with code will help to understand the above description.   

An interpolation factor a, is used in X direction to interpolate F(p,q) and F(p,q+1) as follows.

F(p’’) = a * F(p,q) + (1.0 – a ) F(p, q+1) // Linear interpolation in X direction[Top].
F(p+1’’) = a * F(p+1,q) + (1.0 – a ) F(p+1, q+1) // Linear interpolation in X direction[Bottom].    

An interpolation factor b, is used in Y direction to interpolate F(p’’) and F(p+1’’) as follows.

 F(p’,q’) = b * F(p’’) + (1.0 - b) *  F(p+1’’)    

Corresponding GLSL shader code.    

// Function to get a texel data from a texture with GL_NEAREST property.
// Bi-Linear interpolation is implemented in this function with the 
// help of nearest four data.
vec4 tex2DBiLinear( sampler2D textureSampler_i, vec2 texCoord_i )
{
    vec4 p0q0 = texture2D(textureSampler_i, texCoord_i);
    vec4 p1q0 = texture2D(textureSampler_i, texCoord_i + vec2(texelSizeX, 0));

    vec4 p0q1 = texture2D(textureSampler_i, texCoord_i + vec2(0, texelSizeY));
    vec4 p1q1 = texture2D(textureSampler_i, texCoord_i + vec2(texelSizeX , texelSizeY));

    float a = fract( texCoord_i.x * fWidth ); // Get Interpolation factor for X direction.
											 // Fraction near to valid data.

    vec4 pInterp_q0 = mix( p0q0, p1q0, a ); // Interpolates top row in X direction.
    vec4 pInterp_q1 = mix( p0q1, p1q1, a ); // Interpolates bottom row in X direction.

    float b = fract( texCoord_i.y * fHeight ); // Get Interpolation factor for Y direction.
    return mix( pInterp_q0, pInterp_q1, b ); // Interpolate in Y direction.
}       

In previous method nearest four pixels are used to create an intermediate pixel. In this method nearest 16 pixels are used to create an intermediate pixel F(p’q’). Therefore the output image quality will increase. In the below figure an intermediate pixel F(p'q')[Near to F(p,q)] is created by interpolating nearest 4*4 pixels from F(p-1, q-1) to F(p+2,q+2).  Details of this interpolation is explained with the help of code and necessary equations.   

BiCubicLogic.jpg

Following equation [from Digital Image Processing: PIKS Inside, Third Edition] is used to interpolate nearest 16 pixles. First two sigma(Σ)  are appeared as 2 for loops in the shader code.   

BiCubicEquation.png

Where F(p+m,q+n) indicate texel data at location (p+m, q+n). Rc() denotes a Bi-Cubic interpolation function such as a BSpline, Traingular, Bell cubic interpolation function. In this sample I just use a Triangular, Bell, and B-Spline, and CatMull-Rom interpolation function.   

Following is a GLSL shader code for Bi-Cubic Interpolation.  

vec4 BiCubic( sampler2D textureSampler, vec2 TexCoord )
{
    float texelSizeX = 1.0 / fWidth; //size of one texel 
    float texelSizeY = 1.0 / fHeight; //size of one texel 
    vec4 nSum = vec4( 0.0, 0.0, 0.0, 0.0 );
    vec4 nDenom = vec4( 0.0, 0.0, 0.0, 0.0 );
    float a = fract( TexCoord.x * fWidth ); // get the decimal part
    float b = fract( TexCoord.y * fHeight ); // get the decimal part
    for( int m = -1; m <=2; m++ )
    {
        for( int n =-1; n<= 2; n++)
        {
			vec4 vecData = texture2D(textureSampler, 
                               TexCoord + vec2(texelSizeX * float( m ), texelSizeY * float( n )));
			float f  = Triangular( float( m ) - a );
			vec4 vecCooef1 = vec4( f,f,f,f );
			float f1 = Triangular ( -( float( n ) - b ) );
			vec4 vecCoeef2 = vec4( f1, f1, f1, f1 );
            nSum = nSum + ( vecData * vecCoeef2 * vecCooef1  );
            nDenom = nDenom + (( vecCoeef2 * vecCooef1 ));
        }
    }
    return nSum / nDenom;
}     

BiCubic() function get a texture coordinate (x,y) and returns the interpolated value of nearest 16 texels.   

Nearest 16 texels are iterated through 2 for loops from [x-1,y-1] to [x+2, y+2]. 

    for( int m = -1; m <=2; m++ )
    {
        for( int n =-1; n<= 2; n++)  

For each nearest element an interpolation factor is calculated with the corresponding Bi-Cubic interpolation function. In the above BiCubic() function Triangular() is used to get an interpolation factor. Different versions of Bi-Cubic interpolation[BSpline, Traingular, Bell] can be created by changing Triangular() and its logic. When plotting values from Triangular(), we will get the below image in a triangle form.   

 TriangularCurve.png

From plotting of triangular function, left most value of X axis is -2 and right most value of X axis is +2. Triangular(x) is plotted in Y direction. It indicates Triangular function returns lower value for a high input and high value for a low input. The return value of Triangular()  is multiplied with current data. In effect an intermediate pixel is created by interpolating nearest 16 data. The weightage will be higher for nearest pixels. The output image of Triangular Bi-Cubic is smoother than Bi-Linear.  

Implementation of Bi-Cubic Interpolation[ Triangular ] 

Triangular function is a simple Bi-Cubic function, as defined in the following equation.  

TriangularEquation.png

Here return value of R(x) is x+1 when x is between -1 and 0, and return value is x+1 when x is between 0 and 1. Following pseudo code gives more idea of triangular function. This function provides a low value for high input and a high value for a low input.  

if( -1 < x && x <= 0 )
{ 
    return x + 1
}
else if( 0 < x && x <= 1 )
{
    return x - 1
}   

TriangularCurve.png

Above diagram is the output of Triangular() function. Triangular(x) is plotted in Y direction. 

The following code is used in GLSL shader for Triangular Bi-Cubic implementation.   

 float Triangular( float f )
{
	f = f / 2.0;
	if( f < 0.0 )
	{
		return ( f + 1.0 );
	}
	else
	{
		return ( 1.0 - f );
	}
	return 0.0;
}

 2_Triangular.jpg

Interpolation with GLSL Bi-Cubic Triangular interpolation type.  

Implementation of Bi-Cubic Interpolation[ Bell ]   

When comparing the Bi-Cubic Triangular image quality is increased. But blurred effect can be seen. The reason of this blurred edge is weight returned from Triangular() for near and far texel. If weight of near texel is high and far texel is very low, then Bi-Cubic interpolation can achieve smooth edges. Another Bi-Cubic interpolation funciton which creates a bell shaped( nearest values[Center] are high) and far values are low[left and right ends]. 

BellEquation.png

Above equation has three conditions to create the following curve. Corresponding code is implemented in BellFunc(). If we plot the input values from -2 to +2 of BellFunc()in a graph we will get the following curve. Where X direction contain x provided to BellFunc and Y direction is corresponding return value[BellFunc(x)].  

BellCurve.png

The above diagram indicates when x is high, BellFunc(x) is very low therefore far data of an intermediate texel get very low weight. For nearest texel BellFunc(x) is high and get a high weight.  

The following code is used in GLSL shader for Bell shaped Bi-Cubic implementation.  

float BellFunc( float x )
{
	float f = ( x / 2.0 ) * 1.5; // Converting -2 to +2 to -1.5 to +1.5
	if( f > -1.5 && f < -0.5 )
	{
		return( 0.5 * pow(f + 1.5, 2.0));
	}
	else if( f > -0.5 && f < 0.5 )
	{
		return 3.0 / 4.0 - ( f * f );
	}
	else if( ( f > 0.5 && f < 1.5 ) )
	{
		return( 0.5 * pow(f - 1.5, 2.0));
	}
	return 0.0;
} 

 2_Bell.jpg

Interpolation with Bi-Cubic Bell interpolation type.   

Implementation of Bi-Cubic Interpolation[ B-Spline ]  

When comparing the Bi-Cubic Bell shaped, B spline is smoother and edges are more clear. Below equation is used to create BSpline() function.   

BSplineEquation.png

The code of this equation is implemented in BSpline() function.  

BSplineCurve.png

Above diagram is the output of BSpline() function. BSpline(x) is plotted in Y direction. x starts from -2.0 and ends at +2.0. It indicate when x is high, BSpline(x) is very low and therefore far data of an intermediate texel get very lower weight. When comparing to Bell shaped wave, values in far range ( near -2 and +2 ) are very low. Therefore final output image is also smoother than Bell Bi-Cubic interpolated image. The following code is used in GLSL shader for BSpline implementation.    

float BSpline( float x )
{
	float f = x;
	if( f < 0.0 )
	{
		f = -f;
	}

	if( f >= 0.0 && f <= 1.0 )
	{
		return ( 2.0 / 3.0 ) + ( 0.5 ) * ( f* f * f ) - (f*f);
	}
	else if( f > 1.0 && f <= 2.0 )
	{
		return 1.0 / 6.0 * pow( ( 2.0 - f  ), 3.0 );
	}
	return 1.0;
}  

 2_BSpline.jpg

Interpolation with Bi-Cubic BSpline interpolation type.    

Implementation of Bi-Cubic Interpolation[CatMull-Rom]  

All above Bi-Cubic methods creates a blurred( or smooth ) effect. After applying above interpolation methods, the edges of image become smooth. This method(CatMull-Rom method) preserves edges of image. Following equation is used to create CatMullRom() function, which is used in GLSL shader for CatMullRom interpolation.

CatMullRom.jpg 

This equation is available in Reconstruction Filters in Computer Graphics [Equation 8]. If we replace B with 0.0 and C with 0.5 this equation provides CatMul-Rom interpolation values.The following code is used in GLSL shader for CatMul-Rom implementation.     

float CatMullRom( float x )
{
    const float B = 0.0;
    const float C = 0.5;
    float f = x;
    if( f < 0.0 )
    {
        f = -f;
    }
    if( f < 1.0 )
    {
        return ( ( 12 - 9 * B - 6 * C ) * ( f * f * f ) +
            ( -18 + 12 * B + 6 *C ) * ( f * f ) +
            ( 6 - 2 * B ) ) / 6.0;
    }
    else if( f >= 1.0 && f < 2.0 )
    {
        return ( ( -B - 6 * C ) * ( f * f * f )
            + ( 6 * B + 30 * C ) * ( f *f ) +
            ( - ( 12 * B ) - 48 * C  ) * f +
            8 * B + 24 * C)/ 6.0;
    }
    else
    {
        return 0.0;
    }
} 

2_CAtMull.jpg

The interpolation curve of CatMull Rom is little different with other Bi-Cubic interpolation curves. Here very near data get high weight to multiply, but far data get negative value. Therefore the effect of far data will be less in the intermediate pixel. 

About ZoomInterpolation Application: 

ZoomInterpolation is an application created to demonstrate different interpolation methods. On startup it creates a texture with a bitmap [Flower.bmp] available in resource of this application.  

    BMPLoader BMPLoadObj; // To load an RGB of a bmp file 
    BYTE* pbyData = 0;
    BMPLoadObj.LoadBMP( IDB_BITMAP_FLOWER, m_nImageWidth, m_nImageHeight, pbyData );
    m_glTexture.Create( m_nImageWidth, m_nImageHeight, pbyData );// Texture creating with bmp  

This application displays two images, zoomed image is in left, and actual image is at right bottom area. Two viewports are used for displaying actual image and zoomed image. A red rectangle indicates the zoomed image. Small area of image is selected for texture mapping and displayed to screen.    

Following code is used to draw the actual image with a RED rectangle indicating zoomed image area.  

/*
 This function draws miniature of actual image with a Red region
 indicating the zoomed area.
 */
void CZoomInterpolationDlg::DrawActualImage()
{
    // Set Rendering area of Actual image.
    glViewport( 805, 10, 200, 150 );

    // Image is attached.
    m_glTexture.Enable();

    // Entire image is mapped to screen.
    m_glVertexBuffer.DrawVertexBuffer( GL_QUADS );
    m_glTexture.Disable();

    // Set Red color for Zoom area indication.
    glColor3f( 1.0, 0.0, 0.0 );

    float fXStart = m_fXOffset * 2;
    float fYStart = m_fYOffset * 2;
    float fWidth = m_fZoomWidth * 2;
    float fHeight = m_fZoomHeight * 2;

    // Draw a rectangle indicate zoom area.
    glBegin( GL_LINE_LOOP );
    glVertex2d( -1.0 + fXStart         ,  -1.0 + fYStart );
    glVertex2d( -1.0 + fXStart + fWidth,  -1.0 + fYStart );
    glVertex2d( -1.0 + fXStart + fWidth,  -1.0 + fYStart + fHeight );
    glVertex2d( -1.0 + fXStart         ,  -1.0 + fYStart + fHeight );
    glVertex2d( -1.0 + fXStart         ,  -1.0 + fYStart );
    glColor3f( 1.0, 1.0, 1.0 );
    glEnd();
}

Following code is used to create the zoomed image with current interpolation. 

/*
 Creating zoomed image.
 If shader enabled, this function set parameters to shader and Render.
 If shader is not requried, normal texture mapping is used.
 */
void CZoomInterpolationDlg::DrawZoomedImage()
{
    // Set viewport for Zoomed image display.
    glViewport( 0, 0, 800, 600 );
    if( 0 != m_pShader )
    {
        // If GLSL shader is selected use the same for interpolated image display.
        m_pShader->EnableShader();
        if( !m_pShader->SetTexture( "ImageTexture", m_glTexture.GetTextureID() ))
        {
            TRACE( L"SetTexture failed" );
        }

        if( !m_pShader->SetParam( "fWidth", m_nImageWidth ))
        {
            TRACE( L"SetTexture failed" );
        }

        if( !m_pShader->SetParam( "fHeight", m_nImageHeight ))
        {
            TRACE( L"SetTexture failed" );
        }
        m_glVertexBufferZoom.DrawVertexBuffer( GL_QUADS );
        m_pShader->DisableShader();
    }
    else
    {
        // OpenGL Interpolation.
        m_glTexture.Enable();
        m_glVertexBufferZoom.DrawVertexBuffer( GL_QUADS );
        m_glTexture.Disable();
    }
}

In ZoomInterpolation a rendering area is prepared, then set two viewports to display two types images. First one displays zoomed image, and second display is a miniature of actual image.  glViewport() calls  at the first step of CZoomInterpolationDlg::DrawActualImage() and CZoomInterpolationDlg::DrawZoomedImage() prepares different image display area, zoomed image and miniature of actual image. 

Pan operation:     

When mouse clicks on zoomed image and pans, the texture mapping region changes with respect to mouse move and it creates a pan effect.  

Zoom and UnZoom: 

When Mouse scroll, the zoom/unzoom is implemented by increasing or decreasing the texture mapping area. CZoomInterpolationDlg::HandleZoom() handles zoom and unzoom. Two buttons “Zoom+” and “Zoom-“ is also added to increase zoom or decrease zoom.

Loading new Image: 

A button “Load Image” is available to load an image file from your machine. The image area is created in an aspect ratio of 4:3[800X600 pixels]. Therefore input image is expected in an aspect ratio of 4:3 for better quality. Otherwise stretched/skewed image will be displayed. BMPLoader class is used to retrieve RGB data from bitmap with the help of Gdi+ library. This class supports different image extensions such as .bmp, .jpg, .png, .tga etc. 

Plotting of Interpolation Curve:    

A simple class is created to plot the curve indicating weights applied in Bi-Cubic interpolation function. Triangular, Bell, and BSpline are plotted with the same code of GLSL shader code. This plotting shows minimum and maximum values in X, and Y direction. This graphical representation helps to identify the weight applied to nearest pixels and far pixels. In X direction the distance is shown. In Y direction the weight for a distance is shown. In all Bi-Cubic functions higher weight is applied to very nearest pixel( When distance is 0). The weight decreases on increasing the distance.  

Main Classes Used in ZoomInterpolation Application:  

BMPLoader: This class is used for getting RGB information of an image file. 
                  This class supports bmp, jpg, png, and tga format files. GDI+ classes are used to load different image file formats.  
GLExtension: This class holds all opengl extensions required for pixel shader. 
GLSetup: This class is used for creating a rendering context and other opengl initialisations. 
GLVertexBuffer: This class holds vertex data required for rendering a Quad image with texture mapping.  
PlotInterpCurve: This class draws current interpolation curve with the help of GDI. 
ZoomInterpolation: This class handles main operations of ZoomInterpolation Application.
                   All mouse move and button handling are implemented in this class.
                   All other classes in this application are used in this class.  

Limitations:   

1) The input image is expected in an aspect ratio of 4:3 for better output image. Otherwise stretched/skewed image will be displayed.   
2) For GLSL shader implementation, following OpenGL extensions are required in your machine.     
a. GL_ARB_fragment_shader  
b. GL_ARB_vertex_shader  
If your machine do-not have a supporting graphics device, this application will display interpolation with openGL fixed function pipeline(Nearest and Linear). 

3) The shaders for different Bi-Cubic and Bi-Linear interpolation is prepared with the help some equations, I expect some issues in this code. Smile | :)  

References: 

History: 

  1. 07-Aug-2011: Initial Version.  
  2. 08-Aug-2011: Added details of ZoomInterpolation application. 
  3. 21-Aug-2011: Added CatMull-Rom interpolation, Interpolation curve with Range values. 

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Santhosh G_

India India
No Biography provided

Comments and Discussions


Discussions posted for the Published version of this article. Posting a message here will take you to the publicly available article in order to continue your conversation in public.
 
GeneralMy vote of 1 PinmemberMember 103334182-Nov-13 22:44 
Questionthanls Pinmemberarefqq17-Aug-13 5:30 
SuggestionFew remarks PinmemberDanny Ruijters28-Jul-13 22:44 
QuestionWindow Level and Window Width PinmemberHay Hoang Van1-Jun-13 5:42 
QuestionHi Can't Make it work with Texture Rectangle Pinmemberabhishek_bansal14-Apr-13 0:00 
GeneralMy vote of 5 PinmemberBartlomiej Filipek22-Mar-13 3:19 
GeneralRe: My vote of 5 PinmemberSanthosh G_22-Mar-13 4:00 
GeneralMy vote of 5 PinmemberFrans Vander Meiren7-Nov-12 22:57 
GeneralRe: My vote of 5 PinmemberSanthosh G_8-Nov-12 12:54 
QuestionCould this project have Windows Forms Csharp version? urgent PinmemberMichael Lev31-Oct-12 15:34 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web03 | 2.8.140415.2 | Last Updated 21 Aug 2011
Article Copyright 2011 by Santhosh G_
Everything else Copyright © CodeProject, 1999-2014
Terms of Use
Layout: fixed | fluid