Click here to Skip to main content
Click here to Skip to main content
Go to top

Loading and Rendering Milkshape 3D Models with Animation and Skinning

, 18 May 2011
Rate this:
Please Sign up or sign in to vote.
This article shows how to load Milkshape ms3d binary files, animate and display them with OpenGL

Introduction

Skeletal animation is a technique used in 3D rendering which uses an exterior shell composed of vertices and an internal skeleton for the animation. To avoid breaking the model, each vertex is assigned a set of bones and weights used to compute its final position based on the animation key frames.

In modern application, this technique is used frequently. Autodesk Media and Entertainment is one of the companies that produces software used in films, commercials and computer games. Among its current products, we can find Maya and 3DStudio MAX. The max file format is not very well documented and the 3Ds is just there for legacy purposes. In some cases, if you export and re-import from 3DSMax, the animation and models simply break. Autodesk offers developers SDK support for implementing their own exporters and importers but this takes a lot of time. On the other hand, MS3D is a complete documented file format with source code examples, in Milkshape 3D is a very rapid and easy to use application, making it easy to load and render their native file format. So if you want 3D modeling for your application, you can simply turn to Milkshape for creating and animating your objects.

The code is written in JAVA, but I also have a C# version with XNA or DirectX Managed. If you want the code, just let me know and I’ll email it or maybe just make it public.

milkshape_models.png

Further reading on skeletal animation and skinning can be done at http://en.wikipedia.org/wiki/Skeletal_animation. I recommend that you familiarize yourself more with this technique before you get started on the code.

Topics Covered by this Article

  • JOGL (java opengl)
  • Skeletal animation
  • Milkshape 3D model loading and rendering
  • GLSL
  • Hardware skinning

Specifications

The application includes the following features:

  • Vertex manipulation in GLSL
  • Loading and rendering Milkshape files
  • Skeletal animation
  • OpenGl rendering

Technical Information

  • Code written in JAVA Netbeans
  • Uses JOGL for rendering
  • Uses GLSL for hardware skinning

Requirements

  • Netbeans
  • OpenGL enabled video card with GLSL support (tested on Nvidia, hardware skinning fails on ATI and older Nvidia chipsets)
  • Milkshape 3D
  • JOGL

Using the Application

A demo of the application can be found at http://inline.no-ip.org/testing/java/ViewApplet2 (software skinning version) or http://inline.no-ip.org/testing/java/ViewApplet3 (hardware skinning version).

Using the Code

The provided code is meant to work in Java enabled browsers as a Java applet. The skeletal animation class is called MilkshapeModel and the constructor receives an absolute URL from which to download the required files. Textures have to be in the same directory as the model. The Boolean parameter represents the hardware skinning flag. If it’s set to true, it tries to download and compile the GLSL VertexShader for hardware skinning.

model = new MilkshapeModel(new URL
	("http://localhost:8080/WebApplication/dwarf2.ms3d"), drawable.getGL(), true);

The Basic Milkshape 3D File Format is:

  • Header
    • "MS3D000000" followed by version number (ver. 3 or 4)
  • Vertex Data
    • Coordinates for the vertices
  • Triangle Data
    • Pointers to vertices, as well as surface normals
  • Group Data (object/mesh)
    • Group name and pointers to triangles
  • Material Data
    • Color details
  • Bone data
    • Animation data

The supported file format is version 4. The model loader is based on the C++ binary milkshape loader example provided by http://chumbalum.swissquake.ch/.

Skeletal Animation

In this example, skeletal animation takes place in two steps. The bone setup in which the key frame interpolation is computed and matrices are updated and the vertex update, where vertex positions are obtained using attached bones and weights. This is done either software or hardware by using a shader. Software takes a lot of computing power and is dependent on the number of vertices. Also if we add more models in the same scene, the slowdown is visible. Shaders execute much faster and use GPU power, so we can use the CPU for other operations.

In our case, we have a hierarchy of bones, each having its own set of key frames. These give rotation and translation data. The first step is to locate two key frames that contain the given frame and interpolate the result. Also we need to concatenate results with the parent joint. Joints are preordered in the array so now when we evaluate a Joint, we know we have already evaluated its parent.

    //bone evaluation
    for (int i = 0; i < numJoints; i++) {
        EvaluateJoint(i, frame);
    }  
    private void EvaluateJoint(int index, float frame) {
        MilkshapeJoint joint = Joints[index];

        //
        // calculate joint animation matrix, this matrix will animate matLocalSkeleton
        //
        float[] pos = {0.0f, 0.0f, 0.0f};
        int numPositionKeys = (int) joint.positionKeys.length;
        if (numPositionKeys > 0) {
            int i1 = -1;
            int i2 = -1;

            // find the two keys, where "frame" is in between for the position channel
            for (int i = 0; i < (numPositionKeys - 1); i++) {
                if (frame >= joint.positionKeys[i].time && 
			frame < joint.positionKeys[i + 1].time) {
                    i1 = i;
                    i2 = i + 1;
                    break;
                }
            }

            // if there are no such keys
            if (i1 == -1 || i2 == -1) {
                // either take the first
                if (frame < joint.positionKeys[0].time) {
                    pos[0] = joint.positionKeys[0].key[0];
                    pos[1] = joint.positionKeys[0].key[1];
                    pos[2] = joint.positionKeys[0].key[2];
                } // or the last key
                else if (frame >= joint.positionKeys[numPositionKeys - 1].time) {
                    pos[0] = joint.positionKeys[numPositionKeys - 1].key[0];
                    pos[1] = joint.positionKeys[numPositionKeys - 1].key[1];
                    pos[2] = joint.positionKeys[numPositionKeys - 1].key[2];
                }
            } // there are such keys, so interpolate using hermite interpolation
            else {
                MilkshapeKeyFrame p0 = joint.positionKeys[i1];
                MilkshapeKeyFrame p1 = joint.positionKeys[i2];
                MilkshapeTangent m0 = joint.tangents[i1];
                MilkshapeTangent m1 = joint.tangents[i2];

                // normalize the time between the keys into [0..1]
                float t = (frame - joint.positionKeys[i1].time) / 
			(joint.positionKeys[i2].time - joint.positionKeys[i1].time);
                float t2 = t * t;
                float t3 = t2 * t;

                // calculate hermite basis
                float h1 = 2.0f * t3 - 3.0f * t2 + 1.0f;
                float h2 = -2.0f * t3 + 3.0f * t2;
                float h3 = t3 - 2.0f * t2 + t;
                float h4 = t3 - t2;

                // do hermite interpolation
                pos[0] = h1 * p0.key[0] + h3 * m0.tangentOut[0] + 
			h2 * p1.key[0] + h4 * m1.tangentIn[0];
                pos[1] = h1 * p0.key[1] + h3 * m0.tangentOut[1] + 
			h2 * p1.key[1] + h4 * m1.tangentIn[1];
                pos[2] = h1 * p0.key[2] + h3 * m0.tangentOut[2] + 
			h2 * p1.key[2] + h4 * m1.tangentIn[2];
            }
        }

        float[] quat = {0.0f, 0.0f, 0.0f, 1.0f};
        int numRotationKeys = (int) joint.rotationKeys.length;
        if (numRotationKeys > 0) {
            int i1 = -1;
            int i2 = -1;

            // find the two keys, where "frame" is in between for the rotation channel
            for (int i = 0; i < (numRotationKeys - 1); i++) {
                if (frame >= joint.rotationKeys[i].time && 
			frame < joint.rotationKeys[i + 1].time) {
                    i1 = i;
                    i2 = i + 1;
                    break;
                }
            }

            // if there are no such keys
            if (i1 == -1 || i2 == -1) {
                // either take the first key
                if (frame < joint.rotationKeys[0].time) {
                    AngleQuaternion(joint.rotationKeys[0].key, quat);
                } // or the last key
                else if (frame >= joint.rotationKeys[numRotationKeys - 1].time) {
                    AngleQuaternion(joint.rotationKeys[numRotationKeys - 1].key, quat);
                }
            } // there are such keys, so do the quaternion slerp interpolation
            else {
                float t = (frame - joint.rotationKeys[i1].time) / 
		(joint.rotationKeys[i2].time - joint.rotationKeys[i1].time);
                float[] q1 = new float[4];
                AngleQuaternion(joint.rotationKeys[i1].key, q1);
                float[] q2 = new float[4];
                AngleQuaternion(joint.rotationKeys[i2].key, q2);
                QuaternionSlerp(q1, q2, t, quat);
            }
        }

        // make a matrix from pos/quat
        float matAnimate[][] = new float[3][4];
        QuaternionMatrix(quat, matAnimate);
        matAnimate[0][3] = pos[0];
        matAnimate[1][3] = pos[1];
        matAnimate[2][3] = pos[2];

        // animate the local joint matrix using: matLocal = matLocalSkeleton * matAnimate
        RecConcatTransforms(joint.matLocalSkeleton, matAnimate, joint.matLocal);

        // build up the hierarchy if joints
        // matGlobal = matGlobal(parent) * matLocal
        if (joint.parentIndex == -1) {
            //memcpy(joint.matGlobal, joint.matLocal, sizeof(joint.matGlobal));
            for (int k = 0; k < joint.matLocal.length; k++) {
                System.arraycopy(joint.matLocal[k], 0, 
		joint.matGlobal[k], 0, joint.matLocal[k].length);
            }
        } else {
            MilkshapeJoint parentJoint = Joints[joint.parentIndex];

            RecConcatTransforms(parentJoint.matGlobal, joint.matLocal, joint.matGlobal);
        }
    }
    private void RecConcatTransforms(float in1[][], float in2[][], float out[][]) {
        out[0][0] = in1[0][0] * in2[0][0] + in1[0][1] * 
			in2[1][0] + in1[0][2] * in2[2][0];
        out[0][1] = in1[0][0] * in2[0][1] + in1[0][1] * in2[1][1] + in1[0][2] * in2[2][1];
        out[0][2] = in1[0][0] * in2[0][2] + in1[0][1] * in2[1][2] + in1[0][2] * in2[2][2];
        out[0][3] = in1[0][0] * in2[0][3] + in1[0][1] * 
			in2[1][3] + in1[0][2] * in2[2][3] + in1[0][3];
        out[1][0] = in1[1][0] * in2[0][0] + in1[1][1] * in2[1][0] + in1[1][2] * in2[2][0];
        out[1][1] = in1[1][0] * in2[0][1] + in1[1][1] * in2[1][1] + in1[1][2] * in2[2][1];
        out[1][2] = in1[1][0] * in2[0][2] + in1[1][1] * in2[1][2] + in1[1][2] * in2[2][2];
        out[1][3] = in1[1][0] * in2[0][3] + in1[1][1] * 
			in2[1][3] + in1[1][2] * in2[2][3] + in1[1][3];
        out[2][0] = in1[2][0] * in2[0][0] + in1[2][1] * in2[1][0] + in1[2][2] * in2[2][0];
        out[2][1] = in1[2][0] * in2[0][1] + in1[2][1] * in2[1][1] + in1[2][2] * in2[2][1];
        out[2][2] = in1[2][0] * in2[0][2] + in1[2][1] * in2[1][2] + in1[2][2] * in2[2][2];
        out[2][3] = in1[2][0] * in2[0][3] + in1[2][1] * 
			in2[1][3] + in1[2][2] * in2[2][3] + in1[2][3];
    } 

Hardware Skinning

After this, each vertex needs to be transformed according to the bones and weights. Milkshape uses a maximum of 4 bones per vertex, which is perfect for our vertex shader because we can store both bone ids and bone weights in two vec4 structures as vertex attributes.

GLSL Vertex Shader

attribute vec3 normal;
attribute vec4 weight;
attribute vec4 index;
attribute float numBones;
uniform mat4 matGlobal[100];
uniform mat4 matGlobalSkeleton[100];
uniform vec4 color;
uniform vec4 lightPos;
float DotProduct(in vec4 x, in vec4 y) {    
return x[0] * y[0] + x[1] * y[1] + x[2] * y[2];
}
void VectorRotate(in vec4 in1, in mat4 in2, out vec4 rez) {
    rez[0] = DotProduct(in1, in2[0]);
    rez[1] = DotProduct(in1, in2[1]);
    rez[2] = DotProduct(in1, in2[2]);
}
void VectorIRotate(in vec4 in1, in mat4 in2, out vec4 rez) {
    rez[0] = in1[0] * in2[0][0] + in1[1] * in2[1][0] + in1[2] * in2[2][0];
    rez[1] = in1[0] * in2[0][1] + in1[1] * in2[1][1] + in1[2] * in2[2][1];
    rez[2] = in1[0] * in2[0][2] + in1[1] * in2[1][2] + in1[2] * in2[2][2];
}
void VectorTransform(in vec4 in1,in mat4 in2, out vec4 rez) {
    rez[0] = DotProduct(in1, in2[0]) + in2[0][3];
    rez[1] = DotProduct(in1, in2[1]) + in2[1][3];
    rez[2] = DotProduct(in1, in2[2]) + in2[2][3];
}
void VectorITransform(in vec4 in1, in mat4 in2, out vec4 rez) {
    vec4 tmp; 
    tmp[0] = in1[0] - in2[0][3];
    tmp[1] = in1[1] - in2[1][3];
    tmp[2] = in1[2] - in2[2][3];
    VectorIRotate(tmp, in2, rez);
}

void main(){
    vec4 transformedPosition = vec4(0.0);
    vec3 transformedNormal = vec3(0.0);
    vec4 curIndex = index;
    vec4 curWeight = weight;
    for (int i = 0; i < int(numBones); i++)
    {
        mat4 g44 = matGlobal[int(curIndex.x)];
        mat4 s44 = matGlobalSkeleton[int(curIndex.x)];
        vec4 vert = vec4(0);
        vec4 norm = vec4(0);
        vec4 tmpNorm = vec4(0);
        vec4 tmpVert = vec4(0);
        VectorITransform(gl_Vertex, s44, tmpVert);
        VectorTransform(tmpVert, g44, vert);//g
        vert[3]=1;
        transformedPosition += vert * curWeight.x;

        vec4 preNormal=vec4(normal.xyz,1);
        VectorIRotate(preNormal, s44, tmpNorm);
        VectorRotate(tmpNorm, g44, norm);
        norm[3]=1;
        transformedNormal += vec3(norm.xyz) * curWeight.x;

        // shiftam sa avem urmatoarele valori pentru pondere si indice de os
        curIndex = curIndex.yzwx;
        curWeight = curWeight.yzwx;
    }
    gl_Position = gl_ModelViewProjectionMatrix * transformedPosition;
    transformedNormal = normalize(transformedNormal); 
    gl_FrontColor = dot(transformedNormal, lightPos.xyz) * color;
} 

The code above is also implemented in software version for those who don’t have GLSL support.

For more reading on GLSL, try http://www.opengl.org/registry/doc/GLSLangSpec.4.00.8.clean.pdf.

The GLSL is just for demonstration purposes. It doesn’t handle very well and wastes a lot of memory on the uniform matrices, failing on a lot of video cards because of this. However, if you want to test it, use a model with fewer bones and adjust the size of the matrices to suit your needs.
We animate the model using AdvanceAnimation:

if (lastTick == 0) {
            lastTick = Calendar.getInstance().getTimeInMillis();
        }
        long currentTick = Calendar.getInstance().getTimeInMillis();
        long delta = currentTick - lastTick;
        lastTick = currentTick;
        //model.AnimationFPS=120;
        model.AdvanceAnimation((float) delta / 1000);

Conclusion

This article is an introduction to OpenGL software and hardware skinning using Java and Milkshape3D model loading. It’s not optimized, but you can use a lot of it in your applications. Hope this helps and if you make any good changes to the code, please let me in on it Smile | :) .

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Tiberiu Boros
Other Research Institute For Artificial Intelligence, Ro
Romania Romania
No Biography provided

Comments and Discussions

 
QuestionExcllent article! Pinmemberbitzhuwei18-Jul-13 21:28 
QuestionDiablo? PinmemberReelix17-Aug-11 5:29 
AnswerRe: Diablo? PinmemberTiberiu Boros23-Nov-11 13:50 
GeneralThis file can't be opened in Visual Studio 2010: "MilkshapeViewerXNA" => "The project type is not supported by this installation" PinmemberRedDK26-May-11 13:22 
GeneralRe: This file can't be opened in Visual Studio 2010: "MilkshapeViewerXNA" => "The project type is not supported by this installation" PinmemberTiberiu Boros5-Jun-11 20:57 
GeneralThank you for the update! PinmemberMember 8837777-Feb-11 7:48 
Questionit was excellent Pinmemberaraelec29-Jan-11 22:54 
GeneralVery nice Article PinmemberKevinMO29-Jan-11 13:49 
GeneralRe: Very nice Article PinmemberTiberiu Boros6-Feb-11 21:19 
GeneralMy Vote of 5! PinmemberSharjith24-Jan-11 8:02 
GeneralMilkshape 3d models Pinmemberrpraver24-Jan-11 4:05 
GeneralVery Nice! PinmemberMember 88377721-Jan-11 22:46 
GeneralRe: Very Nice! PinmemberTiberiu Boros22-Jan-11 12:02 
GeneralRe: Very Nice! PinmemberMember 8837776-Feb-11 12:44 
GeneralRe: Very Nice! PinmemberTiberiu Boros6-Feb-11 21:13 
GeneralMy vote of 5! Pinmemberysliew77@yahoo.com21-Jan-11 3:28 
GeneralRe: My vote of 5! PinmemberTiberiu Boros21-Jan-11 4:06 
GeneralRe: My vote of 5! PinmemberTiberiu Boros6-Feb-11 21:19 
GeneralGood job PinmemberShane Story21-Jan-11 3:02 
GeneralRe: Good job PinmemberTiberiu Boros21-Jan-11 3:04 
GeneralMy vote of 5 PinmemberShane Story21-Jan-11 3:00 
GeneralMy vote of 4 PinmemberSlacker00721-Jan-11 0:31 
I like this kind of stuff. I also like lots of pictures of this kind of stuff. Thanks for sharing.

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web03 | 2.8.140905.1 | Last Updated 18 May 2011
Article Copyright 2011 by Tiberiu Boros
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid