Kinect and WPF: Complete body tracking

Vangos Pterneas blog

Kinect and WPF: Complete body tracking

  • Comments 52
  • Likes

It's time for a really interesting and useful Kinect tutorial. We'll see how to achieve full body tracking using Kinect sensor, OpenNI library and Windows Presentation Foundation.

I found OpenNI C# samples a little messy, so I decided to develop a .NET 4 wrapper library which could be used into WPF applications whithout requiring .NET 2 staff like GDI+, System.Drawing, etc. I named it Nui.Vision and it's part of a larger framework I currently develop. Nui.Vision is a .NET 4 assembly which offers an easy-to-use body tracking API!

Update 20/04/2011

Nui.Vision is now compatible with the latest release of OpenNI framework (1.1.0.41). I have made some changes and bug-fixes to it, including the skeleton-display fix provided by roni26_wu (see comments below). An open-source version of Nui.Vision is coming soon!

Video demo

Here is a short video I created to demonstrate complete body tracking in pure WPF (up to 6 people can be recognized). You need to stand in "Psi" position in order to start tracking:

» Download binaries and source code.

Prerequisites

Using the library

Using Nui.Vision is a piece of cake. All body tracking is done in the background, so you only need to update your user interface when the proper events fire. Firstly, add a reference to OpenNi.net.dll and Nui.Vision.dll. Also import a valid configuration file to your project, as described here. Do not forget to type the corresponding using statement:

[code lang="c#"]using Nui.Vision;[/code]

Then declare a new NuiUserTracker object and initialize it in the constructor. Provide the path of the configuration file you previously imported (do not forget to paste the same file in the Debug/Release folders of your application):

[code lang="c#"]_skeleton = new NuiUserTracker("SamplesConfig.xml");[/code]

Just below that, you need to define the UsersUpdated event.

[code lang="c#"]_skeleton.UsersUpdated +=
          new NuiUserTracker.UserListUpdatedHandler(Skeleton_UsersUpdated);[/code]

A proper event handler is created. The NuiUserEventArgs parameter provides you with a collection of all the recognized users! You can now get the coordinates (X, Y and Z) of every body part of every user (OpenNI currently supports 15 body parts)!

[code lang="c#"]foreach (var user in e.Users)
{
    float headX = user.Head.X;
    float headY = user.Head.Y;
    float headZ = user.Head.Z;

    float neckX = user.Neck.X;
    float neckY = user.Neck.Y;

    // etc...
}
[/code]

Quite easy, huh?

Here is a list of all the available body parts:

  • Head
  • Neck
  • LeftShoulder
  • LeftElbow
  • LeftHand
  • RightShoulder
  • RightElbow
  • RightHand
  • Torso
  • LeftKnee
  • LeftHip
  • LeftFoot
  • RightKnee
  • RightHip
  • RightFoot

You may now start developing cool WPF Kinect applications and games. Imagination's the limit.

» Download binaries and source code.

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • Awesome work pal! Keep up the good work! :)

  • you are THE MAN!!!!

  • The depth camera is next to the colour one, and that is why the body marks are shifted a bit. It shouldn't be difficult to fix it :-)

    Excellent work!

  • Thank you everyone. I appreciate your comments.

    @Erevodifwntas: It is quite easy to fix this issue in the UI, but you may not need to display the color image all times.

  • Nice job :D

    Is there a way to control kinect motors with this wrapper?

  • @Nooby:

    Kinect motors cannot be controlled using OpenNI libraries. You can either use libraries such as CLNUI or wait for the official drivers release.

  • It look like very cool, but I though that the joints position of you got is not accurate. Try this, " DepthGenerator.GetAlternativeViewPointCap().SetViewPoint(ImageGenerator) "

  • Vangos! Very great stuff!

    I just have a question, how can I develop further applications in WPF using your developments?

  • @roni29_wu:

    Thank you very much for posting your correction!

    @Foxtrot:

    Simply create a new WPF project and add a reference to Nui.Vision. Then add a valid SamplesConfig.xml file and start developing your own Kinect apps!

  • How can I make .BVH files with it?

  • @Joe McPeek

    You need to use OpenNI library for .BHV files.

    Nui.Vision offers an easy-to-use API for managed user tracking.

  • Hi Vangos,

    thank you very much for providing this example.

    I'm very new to c#, just started a few weeks ago.

    With your code as 'base', I was even able to output

    the skeleton positions into a text file (sounds easy,

    but I'm happy that it works :-) - thank you again.).

    Still, I would like to ask you a question.

    As said, I save the seleton point positions as text file.

    If possible, I would also like to save a *.bmp/RGB-picture.

    1st: is this possible in an easy way, based on your code/dll ?

    2nd: If it's possible, can you give me a tip ? What to add where ?

    Thank you and best wishes,

    Dominik

  • Hi Dominik,

    Thank you for your comment. Of course you can save the point positions in a .bmp file using WPF. Here is an approach I came up with:

    Firstly, you need to handle the UserUpdated event and get the points' coordinates.

    Secondly, you need to display the coordinates on screen as XAML elements (eg ellipses, rectangles, lines, etc).

    Finally, you need to convert these XAML elements into a bitmap image. You can see how to achieve this here:

    www.ageektrapped.com/.../how-to-save-xaml-as-an-image

    And a more robust version here:

    www.xs4all.nl/.../Article_XAMLToBitmap_01.htm

    If you have trouble or need something more specific, contact me on vangos.eu/contact.

  • Exactly why I was looking for :-).

    I simply didn't know for what I have to search.

    Thank you very much,

    Dominik

  • Hi Vangos,

    Thank you very much for this, it really help me.

    I wanted to ask you if I can use an ".oni" file with your work. Or if there is another way you know about.

    I can do it with the UserTracker.net sample from Openni by replacing

    this.context = new Context(SAMPLE_XML_FILE);

    by :

    this.context = new Context();

    this.context.OpenFileRecording(FILE_PATH);

    but this sample isn't in wpf, and not as simple as your code.

    Thank you again.

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment