Kinect and WPF: Painting with Kinect using OpenNI

It's time for the second blog post considering Kinect and WPF. In the previous one, I show you how to obtain the raw and depth images from the Kinect cameras. This time, I developed a simple painting application in Windows Presentation Foundation (I implemented a "punch" gesture, too). Here is a video demonstrating the way it is used:

[youtube:2VTAz-o1W1E]

» Download binaries and source code.

How it works

Wave on camera and Kinect will start tracking the waving hand. While the hand is moving, color is drawn in the corresponding screen points. Punch forward to stop painting. Punch again to restart!

Prerequisites

Implementing the demo in C# and WPF

Step 0

Ensure that OpenNI is properly installed in your Windows operating system.

Step 1

  • Open Visual Studio and create a new WPF application (I named it "KinectPaint") and select .NET 4 as the target framework (you can use either version 4 or 3.5).
  • Add a reference to OpenNI.net.dll. OpenNI.net is found under C:\Program Files\OpenNI\Bin.
  • Add an existing item and load SamplesConfig.xml to your project. Remember to copy this file in your output folder (Debug / Release). SamplesConfig.xml is found under C:\Program Files\OpenNI\Data. You need to have the default XML file replaced with something like the one I provided in my "how-to" post.

Step 2

Download my NuiHandTracker & NuiPositionEventArgs classes. These classes use OpenNI internally to track a hand. The first one also implements a custom "punch" gesture!

Step 3

Navigate to MainWindow.xaml and add a canvas in your XAML code (yes, it is both a painting and a WPF one):

Step 4

Navigate to MainWindow.xaml.cs and create a new instance of NuiHandTracker providing the SamplesConfig.xml file path as a parameter. Also handle its Hover and Push events.

Step 5

We need a boolean member indicating whether the user is painting or not. Declare one and initialize it to true:

Step 6

It's time to paint! Fill-in the Hands_Hover and Hands_Push event handlers:


IMAGE_WIDTH and IMAGE_HEIGHT represent Kinect camera's resolution (640x480 respectively). DrawPixels method simply creates a new Ellipse object and adds it to the canvas element. X and Y coordinates, provided by OpenNI, range from -320 to +320 and -240 to +240. Canvas' coordinates start from 0, so we needed to convert the negative coordinates into positive ones properly. Here is the DrawPixels method:

You are done! KinectPaint is ready! I have added exta functionality in the demo I provide (such as random color generation and the ability to view the raw image from the camera).

Download the source code and have fun!

  • In first line I think you wanted to type show and not saw ;)

  • @infamous

    You are right, thanx ;-)

  • Anonymous
    Anonymous

    Nice sample app!

    Good work, looking forward for more sample projects for the .Net OpenNI library ;-)

  • Anonymous
    Anonymous

    Well depicted step by step procedural writeup, couldn't have asked for more!!..

  • Anonymous
    Anonymous

    how do you make de body tracking, ? in the xml? or how

  • @beat:

    I implemented a NuiHandTracker class which detects a hand. Complete body tracking will be presented in my next blog post.

    Thanks everyone :-)

  • Anonymous
    Anonymous

    Excellent example. Thanks!

  • Anonymous
    Anonymous

    Witch version avec OpenNI do you use, because I crash on start with error unable to find the entry point xnProductionNodeAddRef into the DLL OpenNI.

    The crash happen on line             _imageGenerator = _context.FindExistingNode(NodeType.Image) as ImageGenerator; into the constructor NUIHandTracker.

    Thanks for your help.

  • Hi Fabien,

    I use the latest unstable version for my samples.

  • Anonymous
    Anonymous

    I got the same error as Fabien. It seems like it crashes at some intry point in the openNI.net.dll. It is frustrating. I am not able to use that dll at all. any suggestions?

  • @Ank:

    Ensure that:

    - OpenNI samples are properly running (C:\Program Files\OpenNI\Samples).

    - You have copied SamplesConfig.xml in your Debug / Release project folder.

  • Anonymous
    Anonymous

    Thanks for sharing the code! Great work!

    Build succeded but I have this exception:

    The invocation of the constructor on type 'KinectPaint.MainWindow' that matches the specified binding constraints threw an exception.' Line number '3' and line position '9'.

    Any help will be great!

  • @MarCal

    You need to add a SamplesConfig.xml file (as the one I provide) in your Debug/Release folder of your application.

  • Anonymous
    Anonymous

    Hi Vangos thank you very much for sharing this code!

    I have problems to build it.

    VS always said: " The type or namespace name "xn" could not be found. ( Is missing a using directive or an assembly reference? )

    Is this a problem with the windows version? Cause i use XP instead Win7!

  • @Dinkman

    The latest version of OpenNI has replaced xn namespace with Openni namespace. So, simply define the correct namespace and it should be OK.