Click here to Skip to main content
12,894,930 members (59,557 online)
Click here to Skip to main content
Add your own
alternative version


7 bookmarked
Posted 16 Mar 2012

Taming the tessellator

, 18 Mar 2012 CPOL
Rate this:
Please Sign up or sign in to vote.
Hull, Domain shader


This article aims to show case the use of DX11 tessellator using Hull and Domain shader. Tessellation in DX11 allows subdivisions of surfaces (triangles) into smaller sub surfaces with new interconnecting triangles, these new triangles have their own set of vertices that can be manipulated to create effects.


It is a good idea to read up on DirectX11 and about tessellation, Level Of Detail (LOD), also basic mathematics is required to calculate the position of the points created by domain shader, also , knowledge of texture sampling for consumption of displacement map in domain shader is required but will not be used here, the displacement map is the most common way to create perturbation in the surface and is used in most games and demos.

Using the Code

Like all my previous articles, the code must be referenced at all times. Also this article will not discuss creation of DX11 device and rendering. It is assumed that the reader is well aware of DirectX11 device and device context.

The attached code has added support for of DX9 Mesh and makes use of mesh: tiger.x provided in DX-SDK. The code uses DX9 mesh support to build DX11 vertex/index buffer and its discussion goes beyond the scope of this article (this article mainly discusses the Hull and Domain shader).

Start by creating a DX11 device (Tessellator is available only on DX11).

You could use the following code:

D3D_FEATURE_LEVEL uFeatureLevel1;
  &uFeatureLevel, 1, D3D11_SDK_VERSION, &sd, &pSwapChain, 
  &pd3dDevice, &uFeatureLevel1,&pDeviceContext);
    hr=D3D11CreateDeviceAndSwapChain( NULL, D3D_DRIVER_TYPE_REFERENCE ,NULL, 0,
        &uFeatureLevel, 1, D3D11_SDK_VERSION, &sd, &pSwapChain, 
        &pd3dDevice, &uFeatureLevel1,&pDeviceContext);

Make sure that primitive topology is set to D3D11_PRIMITIVE_TOPOLOGY_3_CONTROL_POINT_PATCHLIST this is required by the HULL/Domain shader. This will tell the DX11 runtime that the input are control points to be manipulated by the HULL shader and are to be used by the tessellator to generate additional vertex points.


Now comes the best part, writing the HULL/Domain shader.

We first start by defining the structure that are to be used throughout the shader pipeline

struct VS_INPUT {float4 Pos : POSITION;float3 vNormal : NORMAL;float2 Tex : TEXCOORD0;};            
struct PS_INPUT {float4 Pos : SV_POSITION;float3 vNormal : NORMAL;float2 Tex : TEXCOORD0;};

The next thing is to create the structure that is to be sent to the fixed function tessellator, information about the level of tessellation required, the fixed function tessellator will generate the additional points on the surface using the control points.

struct HS_CONSTANT_DATA_OUTPUT { float Edges[3] : SV_TessFactor; float Inside : SV_InsideTessFactor; };

The Hull shader itself is divided into two parts:

Patch constant function

A patch constant function executes once for each patch to calculate any data that is constant for the entire patch (straight from MSDN)

The code below specifies tessellator factor at 1, increase the value to load up the tessellator and subdivide the surface. Since the input patch is available, the tessellation (and thus LOD) can be controlled depending on input patch values. (its a good idea to increase LOD only if object is closer to the view point).

HS_CONSTANT_DATA_OUTPUT ConstantHS( InputPatch<PS_INPUT, 3> ip, uint PatchID : SV_PrimitiveID ) 
    Output.Edges[0] = 1;Output.Edges[1] = 1;Output.Edge[2]= 1; 
    Output.Inside = 1; 
    return Output; 

Hull shader

This shader will run once for every point sent and is similar to the vertex shader.

We also specify to the tessellator about the control points and the type of tessellation required

In the code below, we specify the points to be a triangle ("tri") and the surface partitioning to be "integer" ( you can try other surface partitioning such as Fractional_odd,Fractional_even etc), the output topology is a triangle, there are three control points and also the patch function to be used by this hull shader.

PS_INPUT HS( InputPatch<PS_INPUT, 3> p, uint i : SV_OutputControlPointID, uint PatchID : SV_PrimitiveID ) 
    PS_INPUT Output; 
    Output.Pos = p[i].Pos; Output.vNormal = p[i].vNormal; Output.Tex = p[i].Tex;
    return Output; 

Domain shader

The next bit (and more important) is the domain shader, this shader works (in parallel) on every point that has been created by the tessellator. The domain shader generates the surface geometry from the transformed control points from a hull shader. Displacement maps can be used here to create perturbation in the surface (thus adding detail).

For Quad: [domain("quad")], the points location is provided in UV coordinates relative to the quad control points provided to the tessellator: float2 UV : SV_DomainLocation

For Triangle (used in attached code): [domain("tri")], the points location is provided in UVW barycentric coordinates relative to tri-patch control points provided. Notice that as per barycentric coordinates, the new position, normals, etc., are calculated using barycenter provided by domain location (in this case UVW).

         SV_DomainLocation, const OutputPatch<PS_INPUT, 3> quad ) 
    PS_INPUT Output; 
    Output.Pos = UVW.x * quad[0].Pos + UVW.y * quad[1].Pos + UVW.z * quad[2].Pos; 
    Output.vNormal= UVW.x * quad[0].vNormal + UVW.y * quad[1].vNormal + UVW.z * quad[2].vNormal; 
    Output.Tex= UVW.x * quad[0].Tex + UVW.y * quad[1].Tex + UVW.z * quad[2].Tex; 
    return Output;

The output of the domain shader is the input of the pixel shader.

The vertex, geometry, and pixel shaders are available in previous generation DirectX10 runtime

The geometry shader is not considered here as its optional.

Set the HULL and Domain shader as showed below

technique10 Main
    pass P0
        SetVertexShader ( CompileShader( vs_5_0, VS() ) );
        SetPixelShader ( CompileShader( ps_5_0, PS() ) );
        SetHullShader ( CompileShader( hs_5_0, HS() ) );
        SetDomainShader ( CompileShader( ds_5_0, DS() ) );

I do hope this article will help you understand the tessellator provided in DX11.

Points of Interest

The DX11 tesselator allows developers to scale LOD depending on position of the model. Displacement maps allow developers to create realistic rough surfaces without increasing the poly count of their models (the tessellator will do that), this saves the trouble of generating new vertices and not clog up bandwidth between CPU and GPU by sending huge data of vertices.


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

Asif Bahrainwala
Instructor / Trainer
India India
I have been working with computers since my eight grade, programming the ZX Spectrum. I have always had an interest in assembly language and computer theory (and is still the reason for taking tons of online courses), actively code using C/C++ on Windows (using VS) and Linux (using QT).

I also provide training on data structures, algorithms, parallel patterns library , Graphics (DX11), GPGPUs (DX11-CS,AMP) and programming for performance on x86.
Feel free to call me at 0091-9823018914 (UTC +5:30)

(All views expressed here do not reflect the views of my employer).

You may also be interested in...

Comments and Discussions

GeneralMy vote of 5 Pin
manoj kumar choubey16-Mar-12 19:31
membermanoj kumar choubey16-Mar-12 19:31 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.170424.1 | Last Updated 18 Mar 2012
Article Copyright 2012 by Asif Bahrainwala
Everything else Copyright © CodeProject, 1999-2017
Layout: fixed | fluid