Click here to Skip to main content
Click here to Skip to main content
Go to top

Image processing as fast as possible in C#

, 19 Nov 2013
Rate this:
Please Sign up or sign in to vote.
Extensible, hardware accelerated image processing library written in HLSL and C#.

Introduction

Most image processing applications use hardware acceleration for heavy image filters. So why not this bring to C#? That's what this library does.

Effects in this library:

  • Blur
  • Directional Blur
  • Zoom Blur
  • Invert
  • Tone Level
  • Gray scale
  • Old Image
  • MonoChrome
  • Pixelate
  • Ripple
  • Wave Warper
  • Swirl

Custom effects are supported to extend the library.

Using the Effects:

EffectHelper class is the actual worker so you must load it first.

EH = new EffectHelper();
Size s = EffectHelper.GetMaxSize();
EH.Load(s.Width, s.Height); 

EffectHelper class has two methods (ApplyEffect, ApplyMultipleEffects) that you will use to apply an effect. They:

  • take the input (imagedata(Frame), effectdata)
  • copy imagedata to video memory
  • process (one effect or multiple effects)
  • Copy imagedata to system memory
  • return the output data.

The difference between them is that ApplyMultipleEffects doesn't return the processed image and take it as input again. it process all given effects and then return.

To apply an effect to an image:

BitmapData bd;
Bitmap b = (Bitmap)CurrentBitmap.Clone();
Frame frm = Frame.OpenBitmap(b, out bd);

frm = EH.ApplyEffect(frm, data);

Frame.CloseBitmap(b, bd); 

To apply multiple effects:

BitmapData bd;
Bitmap b = (Bitmap)CurrentBitmap.Clone();
Frame frm = Frame.OpenBitmap(b, out bd);
EffectData[] dataseries = new EffectData[3];
dataseries[0] = SwirlEffect.GetData(20f, 1f, new PointF(.5f, .5f));
dataseries[1] = ZoomBlurEffect.GetData(0.5f, new PointF(0.5f, 0.5f));
dataseries[2] = GrayScaleEffect.GetData();
frm = EH.ApplyMultipleEffects(frm, dataseries);
Frame.CloseBitmap(b, bd);    

To register a new effect:

scaleEff = new ScaleExtended(EH.Device);
EH.Manager.Register("ScaleExtended", scaleEff); 

Effects

Before you read the following code you must know that:

  1. You must have a background about how DirectX shader effects work to well understand how these effects work.
  2. All effects are rendered perpixel and written by HLSL shader language (it is similar in syntax to C#).
  3. So these effects are not written by c# code but it uses directX to comile the effect code (found in resources folder) to be run on video processor. this takes the advantage of performance but has some limits.

  4. You can create your own effects easly by Shazzam tool, but when you use it here you must add this part of hlsl code
technique Tech
{
    pass Pss
    {
        pixelshader = compile ps_2_0 main();
    }
}

The code you write on shazzam tool is simply a small program running on graphics card and the above lines tells the directx hlsl compiler that you will use the method 'main()' as the start of this small program.

Let's start.

1. Blur Effect

float Amount : register(C1);
sampler2D  Texture1Sampler : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
    float4 c = 0;
    float r360 =  0.0174533f;
    float rad ;
    float xOffset;
    float yOffset;

    c += tex2D(Texture1Sampler, uv);
    float2 v; 
 
    for(int i=0; i<8; i++)
    {
        rad= i*45 * r360;
        xOffset = cos(rad);
        yOffset = sin(rad);
        v.x = uv.x -( Amount * xOffset);
        v.y = uv.y -( Amount * yOffset);
        c += tex2D(Texture1Sampler, v);
    }
   
    c /= 9;
    
    return c;
} 

2. Directional Blur

 float Angle : register(C0);
float Amount : register(C1);
sampler2D  Texture1Sampler : register(S0);
float4 main(float2 uv : TEXCOORD) : COLOR
{
    float4 c = 0;
    float rad = Angle * 0.0174533f;
    float xOffset = cos(rad);
    float yOffset = sin(rad);
    for(int i=0; i<16; i++)
    {
        uv.x = uv.x - Amount/16 * xOffset;
        uv.y = uv.y - Amount/16 * yOffset;
        c += tex2D(Texture1Sampler, uv);
    }
    c /= 16;
    
    return c;
} 

3. Zoom Blur

sampler2D  input : register(S0);

float progress  : register(c0);
float2 center : register(c1);

float4 main(float2 uv : TEXCOORD) : COLOR
{

float BlurAmount =- progress * .4;
 float4 c = 0;    
 uv -= center ;

 for (int i = 0; i < 14; i++)  {
  float scale = 1.0 + BlurAmount * (i / 13.0);
  c += tex2D(input , uv * scale + center );
 }
   
 c /= 14;
  
return c;
} 

4. Grayscale

sampler2D input : register(s0);

float4 main(float2 uv : TEXCOORD) : COLOR 
{ 
        
    float4 color; 
    color= tex2D( input , uv.xy); 
        
    color.r = (color.r +color.g + color.b)/3.0f;
    color.g = color.r;
    color.b = color.r;
    return color; 
}

5. Invert

sampler2D Texture1Sampler : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
   float4 color = tex2D( Texture1Sampler, uv );
   float4 invertedColor = float4(color.a - color.rgb, color.a);
   return invertedColor;
} 

6. Monochrome

float4 FilterColor : register(C0);
sampler2D  Texture1Sampler : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
   float4 srcColor = tex2D(Texture1Sampler, uv);
   float3 rgb = srcColor.rgb;
   float3 luminance = dot(rgb, float3(0.30, 0.59, 0.11));
   return float4(luminance * FilterColor.rgb, srcColor.a);
} 

7. Old

sampler2D TexSampler : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
    float4 color = tex2D(TexSampler, uv);
    
    float gray = dot(color, float4(0.3, 0.59, 0.11, 0)); 
    color = float4(gray * float3(0.9, 0.8, 0.4) , color.a); 
   
    return color;
}  

8. Ripple

float2 Center : register(C0);
float Amplitude : register(C1);
float Frequency: register(C2);
float Phase: register(C3);
float AspectRatio : register(C4);
sampler2D Texture1Sampler : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
 
    float2 dir = uv - Center; // vector from center to pixel
    dir.y /= AspectRatio;
    float dist = length(dir);
    dir /= dist;
    dir.y *= AspectRatio;

    float2 wave;
    sincos(Frequency * dist + Phase, wave.x, wave.y);
        
    float falloff = saturate(1 - dist);
    falloff *= falloff;
        
    dist += Amplitude * wave.x * falloff;
    float2 samplePoint = Center + dist * dir;
    float4 color = tex2D(Texture1Sampler, samplePoint);

    float lighting = 1 - Amplitude * 0.2 * (1 - saturate(wave.y * falloff));
    color.rgb *= lighting;
    return color;
}  

9. Swirl

float2 Center : register(C0);
float SpiralStrength : register(C1);
float AspectRatio : register(C2);
sampler2D input : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
    float2 dir = uv - Center;
    dir.y /= AspectRatio;
    float dist = length(dir);
    float angle = atan2(dir.y, dir.x);

    float newAngle = angle + SpiralStrength * dist;
    float2 newDir;
    sincos(newAngle, newDir.y, newDir.x);
    newDir.y *= AspectRatio;
    
    float2 samplePoint = Center + newDir * dist;
    bool isValid = all(samplePoint >= 0 && samplePoint <= 1);
    return isValid ? tex2D(input, samplePoint) : float4(0, 0, 0, 0);
} 

10. Wave wrapper

sampler2D input : register(s0);
float Time : register(C0);
float WaveSize: register(C1);

float dist(float a, float b, float c, float d){
    return sqrt((a - c) * (a - c) + (b - d) * (b - d));
}

float4 main(float2 uv : TEXCOORD) : COLOR 
{ 
    float4 Color = 0;
    float f = sin(dist(uv.x + Time, uv.y, 0.128, 0.128)*WaveSize)
                  + sin(dist(uv.x, uv.y, 0.64, 0.64)*WaveSize)
                  + sin(dist(uv.x, uv.y + Time / 7, 0.192, 0.64)*WaveSize);
    uv.xy = uv.xy+((f/WaveSize));
    Color= tex2D( input , uv.xy);
    return Color; 
} 

11. Pixelate

float2 PixelCounts : register(C0);
float BrickOffset : register(C1);
sampler2D Texture1Sampler : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
   float2 brickSize = 1.0 / PixelCounts;

   // Offset every other row of bricks
   float2 offsetuv = uv;
   bool oddRow = floor(offsetuv.y / brickSize.y) % 2.0 >= 1.0;
   if (oddRow)
   {
       offsetuv.x += BrickOffset * brickSize.x / 2.0;
   }
   
   float2 brickNum = floor(offsetuv / brickSize);
   float2 centerOfBrick = brickNum * brickSize + brickSize / 2;
   float4 color = tex2D(Texture1Sampler, centerOfBrick);

   return color;
} 

12. Tone level

float Levels : register(C0);
sampler2D Texture1Sampler : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
    float4 color = tex2D( Texture1Sampler, uv );
    color.rgb /= color.a;

    int levels = floor(Levels);
    color.rgb *= levels;
    color.rgb = floor(color.rgb);
    color.rgb /= levels;
    color.rgb *= color.a;
    return color;
}  

ScaleExtended (Example of effect extension)

sampler2D input : register(s0);

float scalex : register(C0);
float scaley : register(c1);

float centerx : register(C2);
float centery : register(c3);
float4 main(float2 uv : TEXCOORD) : COLOR 
{ 
    float4 color; 
    
    uv.x -= centerx;
    uv.y -= centery;
    
    uv.x /= scalex;
    uv.y /= scaley;
    
    uv.x += centerx;
    uv.y += centery;
    
    if (uv.x < 0 || uv.y < 0 || uv.x > 1 || uv.y > 1 )
    color = 0 ;
    else
    color= tex2D( input , uv.xy); 


    return color; 
}

Multiple effects in one shot

Tthis is an example on applying multiple effects at a time

BitmapData bd;
Bitmap b = (Bitmap)CurrentBitmap.Clone();

Frame frm = Frame.OpenBitmap(b, out bd);

EffectData[] dataseries = new EffectData[3];

dataseries[0] = SwirlEffect.GetData(20f, 1f, new PointF(.5f, .5f));
dataseries[1] = ZoomBlurEffect.GetData(0.5f, new PointF(0.5f, 0.5f));
dataseries[2] = GrayScaleEffect.GetData();
frm = EH.ApplyMultipleEffects(frm, dataseries);

Frame.CloseBitmap(b, bd); 

This code will do the following

SwirlEffect then >> ZoomBlurEffect then >> GrayScaleEffect.

There is an option in the test application "Set As Original Image" to put the result as input:

Advantages

  • Take the maximum hardware acceleration of your GPU.
  • extend your custom effects.
  • can draw a full scene (3D objects, particles, and blend modes) with lights and 3d shadows in your custom effect (full DirectX features).
  • every input is handled separately by the ShaderEffect class hence you can pass texture, color, matrix, point, vector, or any other type as input.

Limits

  • Number of effects is small but you can use Shazzam Shader Editor to get more and more effects and extend the library .
  • Max size of image is hardware dependent. But most Graphics cards support 2048 X 2048 pixels.
  • Library uses software vertex processing so it renders only the ps_2_0 effects but you can change the device initialization at EffectHelper to hardware vertex processing to allow ps_3_0 effects( this will run only on Graphics card that support ps_3_0 and if not it will throw exception).
  • Effects are running on GPU so any running game with fullscreen mode will pause the effect till returning to window mode or closing the game.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

sameh obada

Egypt Egypt
DotNet developer
 
Graphics researches and GPU Acceleration Developer
 
undergraduate medical student

Comments and Discussions

 
GeneralMy vote of 1 PinmemberMember 107301792-Jun-14 20:54 
QuestionMessage Automatically Removed Pinmemberpittypan20-May-14 23:50 
GeneralMy vote of 2 PinmemberUday P.Singh16-Dec-13 19:17 
QuestionAlready have the pixel shader effects PinmemberUday P.Singh16-Dec-13 19:15 
Suggestiondirectx references PinmemberFlorian Rosmann7-Dec-13 22:33 
GeneralRe: directx references Pinmembersameh obada7-Dec-13 23:36 
Bugno valid win32 app PinmemberFlorian Rosmann7-Dec-13 22:32 
GeneralRe: no valid win32 app Pinmembersameh obada7-Dec-13 23:32 
BugOutofvideomemory exception PinmemberDan Roma26-Nov-13 7:07 
GeneralRe: Outofvideomemory exception Pinmembersameh obada26-Nov-13 9:35 
GeneralRe: Outofvideomemory exception PinmemberDan Roma27-Nov-13 8:06 
GeneralRe: Outofvideomemory exception Pinmembersameh obada27-Nov-13 10:17 
GeneralRe: Outofvideomemory exception PinmemberDan Roma2-Dec-13 5:12 
GeneralRe: Outofvideomemory exception Pinmembersameh obada2-Dec-13 11:18 
GeneralRe: Outofvideomemory exception PinmemberDan Roma3-Dec-13 3:03 
GeneralRe: Outofvideomemory exception Pinmembersameh obada4-Dec-13 4:37 
QuestionCompiling Error? Pinmemberdescartes24-Nov-13 23:39 
AnswerRe: Compiling Error? Pinmembersameh obada25-Nov-13 5:14 
AnswerRe: Compiling Error? Pinmemberdescartes25-Nov-13 6:33 
QuestionCool article PinmemberMember 1035257720-Nov-13 21:58 
AnswerRe: Cool article Pinmembersameh obada21-Nov-13 2:23 
GeneralRe: Cool article PinmemberMember 1035257721-Nov-13 3:31 
GeneralDelicious! PinprofessionalRavi Bhavnani20-Nov-13 6:11 
QuestionSir, this is an absolute 10 PinmemberRO2520-Nov-13 3:41 
QuestionAccess problem PinmemberMohammed Hameed19-Nov-13 18:22 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web01 | 2.8.140916.1 | Last Updated 19 Nov 2013
Article Copyright 2013 by sameh obada
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid