Click here to Skip to main content
15,075,525 members
Articles / Multimedia / OpenGL
Posted 7 Oct 2004


51 bookmarked

Generating Outlines in OpenGL

Rate me:
Please Sign up or sign in to vote.
4.78/5 (12 votes)
7 Oct 20045 min read
Using multi-pass techniques to generate outlines in OpenGL.


I recently had to write some code to outline 3D objects using OpenGL. I hunted around the Internet for methods to do this, and eventually came across two main techniques. The first involves the use of Polygon Offsets, and the second involves the use of the Stencil Buffer. This article explains how these two methods work, and explains in a bit more detail what the stencil buffer is.


There is often a requirement in 3D graphics to outline objects on the screen. This can be used for a variety of different reasons. For example, a CAD system might outline an object in red to show that it is currently selected. Also, densely triangulated objects do not often draw well in wireframe, and an outline mode can be a way of portraying the shape of an object to a user without overcomplicated wireframe rendering. A final application could be within engineering drawing, where an outline of a 3D model could be used to automatically generate particular black and white projections of a model.

Outlining a 3D object

The generation of outlines uses multiple passes through the scene geometry to achieve its goal. Multi-pass rendering techniques are becoming more and more common, and are often used in games software to generate effects such as shadows. Shadows cannot be created with conventional z-buffer techniques, but can with multi-pass techniques. I'll briefly explain how this would be done at the end of the article; although I won't show any code for shadows.

The first technique I found is quick and dirty. It involves drawing the object in wireframe with the GL line style set to use very thick lines, then over-drawing the image using the background color. This leaves the edges of the thick wireframe lines that were protruding the object visible, and gives you your required outline. One of the rendering passes must be offset. I chose to pull the surfaces forward out of the screen, hence ensuring that the polygons render over the top.

The second technique uses a feature of the graphics card called the Stencil Buffer. The Stencil Buffer is like the Z-Buffer, but allows you to do per pixel tests during rendering passes. In this instance, I draw the object in the background color, writing a value into the stencil buffer every time I successfully write a pixel to the screen. Then the object is redrawn in wireframe, using thick lines, but only writing when the stencil buffer is empty.

In order to quickly get some OpenGL code into existence, I have used a library and demo application from another Code Project article. OGLTools by Jonathan de Halleux provides an object oriented framework for creating GL render contexts and handling all the Windows specific OpenGL stuff. Note that in order to use the stencil buffer, you have to make sure that you get a stencil buffer in your pixel format. I did that by hacking Jonathan's projects slightly.

Using the code

The demo application is based on Jonathan de Halleux's application. I have edited the draw loop so that it can draw in one of three ways. The first using Polygon Offsets, the second using the Stencil Buffer, and the third using the Stencil Buffer along with display lists. In the third algorithm, I have drawn the objects in a dark blue rather than the background color so that you can see the effect from that. You can select the mode from a standard menu in the application.

Outlining a 3D object - rendering in blue

The first block of code shows how to draw the object using polygon offsets:

// Push the GL attribute bits so that we don't wreck any settings
glPushAttrib( GL_ALL_ATTRIB_BITS );
// Enable polygon offsets, and offset filled polygons forward by 2.5
glPolygonOffset( -2.5f, -2.5f );
// Set the render mode to be line rendering with a thick line width
glPolygonMode( GL_FRONT_AND_BACK, GL_LINE );
glLineWidth( 3.0f );
// Set the colour to be white
glColor3f( 1.0f, 1.0f, 1.0f );
// Render the object
// Set the polygon mode to be filled triangles 
glPolygonMode( GL_FRONT_AND_BACK, GL_FILL );
glEnable( GL_LIGHTING );
// Set the colour to the background
glColor3f( 0.0f, 0.0f, 0.0f );
// Render the object
// Pop the state changes off the attribute stack
// to set things back how they were

The second block of code demonstrates the use of the stencil buffer:

// Push the GL attribute bits so that we don't wreck any settings
glPushAttrib( GL_ALL_ATTRIB_BITS );
glEnable( GL_LIGHTING );
// Set the clear value for the stencil buffer, then clear it
glEnable( GL_STENCIL_TEST );
// Set the stencil buffer to write a 1 in every time
// a pixel is written to the screen
glStencilFunc( GL_ALWAYS, 1, 0xFFFF );
// Render the object in black
glPolygonMode( GL_FRONT_AND_BACK, GL_FILL );
glColor3f( 0.0f, 0.0f, 0.0f );
glDisable( GL_LIGHTING );
// Set the stencil buffer to only allow writing
// to the screen when the value of the
// stencil buffer is not 1
glStencilFunc( GL_NOTEQUAL, 1, 0xFFFF );
// Draw the object with thick lines
glLineWidth( 3.0f );
glPolygonMode( GL_FRONT_AND_BACK, GL_LINE );
glColor3f( 1.0f, 1.0f, 1.0f );
// Pop the state changes off the attribute stack
// to set things back how they were

Explaining the key GL calls in a little more detail: glClearStencil is used to set the clear value for a call to glClear( GL_STENCIL_BUFFER_BIT ). This allows you to initialize the stencil buffer to any value you want. glEnable( GL_STENCIL_TEST ) is pretty self explanatory, turning on the use of the stencil buffer. The key functions are glStencilFunc and glStencilOp. glStencilFunc allows you to set a test function, specify a reference value, and a mask that will be applied to both the stencil buffer and the reference value before any test is done. The test function can consist of a variety of comparison operators, or the generic GL_NEVER and GL_ALWAYS. glStencilOp explains what to do in the event of the stencil buffer test failing, the stencil buffer test passing but the z-buffer test failing, and both the stencil buffer test passing and the z-buffer test passing. Your options here include keeping the existing value, zeroing the buffer or replacing, inverting, incrementing or decrementing the value.

Points of Interest

Finally, I would like to explain one of the many methods for using the stencil buffer and multi-pass rendering to produce shadows. The following process will produce a shadow on a ground plane from a simple object:

  1. Load the view transform into the modelview matrix. This sets the system to be looking from the eye point.
  2. Draw the object and ground plane, with lighting enabled.
  3. Push the modelview matrix with glPushMatrix().
  4. Construct a light view matrix and multiply it into the modelview matrix. The light view matrix is generated in a specific way using the equation of the ground plane, and the light position.
    void shadowMatrix(Glfloat m[4][4],
    GLfloat plane[4],
    GLfloat light[4])
    GLfloat dot = plane[0]*light[0] + plane[1]*light[1] +
    plane[2]*light[2] + plane[3]*light[3];
    m[0][0] = dot - light[0]*plane[0];
    m[1][0] = - light[0]*plane[1];
    m[2][0] = - light[0]*plane[2];
    m[3][0] = - light[0]*plane[3];
    m[0][1] = - light[1]*plane[0];
    m[1][1] = dot - light[1]*plane[1];
    m[2][1] = - light[1]*plane[2];
    m[3][1] = - light[1]*plane[3];
    m[0][2] = - light[2]*plane[0];
    m[1][2] = - light[2]*plane[1];
    m[2][2] = dot - light[2]*plane[2];
    m[3][2] = - light[2]*plane[3];
    m[0][3] = - light[3]*plane[0];
    m[1][3] = - light[3]*plane[1];
    m[2][3] = - light[3]*plane[2];
    m[3][3] = dot - light[3]*plane[3];
  5. Set up a polygon offset so that the rendered shadow polygons do not interleave with the ground plane.
  6. Render the ground plane to the stencil buffer - setting a value every time the ground plane is written.
  7. Disable lighting, set the color to be dark grey, and render the object again (not the ground plane). Only render the object points if the stencil buffer is set, and whenever you render, zero the contents of the stencil buffer. This slightly complex process will ensure that shadows are only cast on the ground plane, and that the shadow polygons are only rendered once.
  8. Pop the matrix and disable the polygon offset to tidy things up.

This technique, and others using multi-pass rendering, is explained in considerably more detail in an article by Mark J Kilgard of nVidia.


  • Version 1 issued on 7 Oct 04.


This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


About the Author

Dave Handley
Web Developer
United Kingdom United Kingdom
I started programming on 8 bit machines as a teenager, writing my first compiled programming language before I was 16. I went on to study Engineering and Computer Science at Oxford University, getting a first and the University Prize for the best results in Computer Science. Since then I have worked in a variety of roles, involving systems management and development management on a wide variety of platforms. Now I manage a software development company producing CAD software for Windows using C++.

My 3 favourite reference books are: Design Patterns, Gamma et al; The C++ Standard Library, Josuttis; and Computer Graphics, Foley et al.

Outside computers, I am also the drummer in a band, The Unbelievers and we have just released our first album. I am a pretty good juggler and close up magician, and in my more insane past, I have cycled from Spain to Eastern Turkey, and cycled across the Namib desert.

Comments and Discussions

SuggestionAlternative to Lines Pin
dsonck9216-Aug-11 12:13
Memberdsonck9216-Aug-11 12:13 
QuestionCross section? Pin
modati15-Oct-07 9:42
Membermodati15-Oct-07 9:42 
Questionhi Dave [modified] Pin
looole27-Apr-07 11:49
Memberlooole27-Apr-07 11:49 
AnswerRe: hi Dave Pin
Dave Handley27-Apr-07 13:17
MemberDave Handley27-Apr-07 13:17 
GeneralRe: hi Dave Pin
looole27-Apr-07 13:54
Memberlooole27-Apr-07 13:54 
Questionhello Pin
looole27-Apr-07 10:45
Memberlooole27-Apr-07 10:45 
AnswerRe: hello Pin
Dave Handley27-Apr-07 11:19
MemberDave Handley27-Apr-07 11:19 
Generalso expedite Pin
looole27-Apr-07 7:58
Memberlooole27-Apr-07 7:58 
GeneralRe: so expedite Pin
Rick York27-Apr-07 10:11
mveRick York27-Apr-07 10:11 
GeneralOutlines Pin
Anonymous14-Oct-04 21:12
MemberAnonymous14-Oct-04 21:12 
GeneralRe: Outlines Pin
Dave Handley14-Oct-04 21:58
MemberDave Handley14-Oct-04 21:58 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.