How to programatically modify a joint in the Skeleton

See more:

C#

Kinect

Hi,

I am looking to achieve manipulation of the Kinect skeleton data on the fly. I have built an app that renders the Skeleton captured in 3D using the Helix toolkit. I use the XYZ position data for each joint to render the 3D model (a series of sphere's and cylinders to represent the joints and bones).

I have built a "Pause" button that when clicked, stops capturing in real time and keeps the last rendered skeleton information as well as the rendering on the screen.

What I am looking to do is rotate a selected joint (we will discuss the right elbow joint for the purpose of this) by a certain number of degrees around either an X, Y or Z axis, depending on what I choose.

I have built classes to carry out the necessary calculations to do this transformation (Matrix4x4, Matrix3x3, Vector3D, Point3D classes). I now need to apply the calculations to a selected joint to rotate as I require. Here's what I do:

for (int i = 0; i < lastSkeletons.Count(); i++)
{
  var item = lastSkeletons[i];

  // Generate an X axis rotation matrix.
  Matrix4x4 rotateMat = Matrix4x4.RotationAroundX(45); // Rotate 45 degrees for example
                            
  // Current matrix for bone. I can implicitly convert the Matrix4 from the Kinect

  // to my own Matrix4x4 class.
  Matrix4x4 sourceMat = item.BoneOrientations[JointType.ElbowRight].HierarchicalRotation.Matrix;

  // Multiply rotation matrix by bone source matrix.
  Matrix4x4 rotated = rotateMat * sourceMat;

  // Set back into the Kinect matrix after modifying its value.
  item.BoneOrientations[JointType.ElbowRight].HierarchicalRotation.Matrix = (Matrix4)rotated;
  
  // Update the skeleton with the modified item.
  lastSkeletons[i] = item;
}

// Call render method.
Render();

So I generate a rotation matrix, get the hierarchy matrix from the bonerotation within the Kinect and multiple these together to get the transformed matrix. I then set the Kinect hierarchy matrix to the value of the modified matrix. Firstly, is this the correct approach for manipulating the data?


Secondly, updating the Matrix obviously is not refreshing the position property of the joint. How do I either...

1.The problem I have is that the Render() method using the Position class to render the skeleton in 3D (item.Joints[JointType.ElbowRight].Position).
Pull the actual XYZ position from the modified Matrix? OR
2.Force the position info in the Kinect skeleton to refresh using the updated matrix info?


Thanks in advance! Also, if anyone has any pointers on how I can modify the rotation of the joint in a different way Id appreciate any pointers!

Thanks!

Rob