Motion detection in Silverlight - Vangos Pterneas blog - Site Root - StudentGuru

Motion detection in Silverlight

After the face detection post, it's time to dive a little deeper in Silverlight Computer Vision. We'll create a simple motion detection application using Silverlight webcam support!

You can view a demo of the application in my website. Download the source code too, in order to have a complete aspect of what's going on.

Silverlight motion detection

In this article, we'll focus on specific topics regarding both Silverlight 4 camera support and motion detection. For reasons of clarity, I won't demonstrate code details here, so I insist on downloading the source code.

The application

The application needs to do the following:

  • Open the default webcam and start capturing.
  • Retrieve the captured frames.
  • Process the frames.
  • Highlight the moving areas creating a new frame.
  • Display the new frames consecutively.

The user interface consists of a rectangle (displaying the camera stream) and two buttons (start / stop streaming). I have also added a slider indicating how sensitive the motion detector will be.

Points of interest

Starting the camera

Silverlight 4 offers integrated support for handling a camera, via CaptureSource object. We can either select from a list of the available capture devices or simply select the default one:

CaptureSource exposes straightforward methods for starting and stopping camera streaming:

Before Silverlight starts capturing, it needs to obtain user's permission. This is a critical issue, so do not forget to add the following lines of code:

Getting the current frame

CaptureSource contains CaptureImageAsync method in order to let us grab a specific frame. We need to process every single frame in our application. As a result, CaptureImageAsync needs to be called continuously (that's 60 times per second). How can we do this? Fortunately, every Silverlight page raises CompositionTarget.Rendering event! This event fires before its frame renders and its event handler is called once per frame:

Motion detection algorithm

Our motion detection algorithm is pretty simple. It compares two consecutive frames pixel-by-pixel and finds whether two pixels differ. Then, a new frame is created with the differing pixels marked as green.

Tip: Before comparing consecutive bitmaps, it helps turning them to grayscale, as it is easier to find out how much a pixel differs from black and white rather than specifying rules for RGB colors. We can also define a threshold, making our detector more or less sensitive.

Here's a code snippet:

Displaying the resulting frame

The above snippet is part of the method which returns a bitmap image. The only thing we have to do now is create an ImageBrush, set the resulting bitmap as its source and fill the gray rectangle. This is done in CaptureImageCompleted event handler:

Download full source code.


PS 1: The motion detection library is a Silverlight 3 assembly. This means that it can also be used in Windows Phone 7 applications. We are looking forward to accessing phone camera's streaming soon!

PS 2: It's been a long talk about whether Silverlight is dead. And it's been a tough battle between Silverlight and HTML 5 supporters. To be honest, HTML 5, CSS3 and JavaScript are great means for a richer web. As a web developer, I love HTML 5 new capabilities. But writing this post, I presented you some cool features not supported by HTML 5 (fair enough, camera support is hard even for desktop applications). In conclusion, Silverlight and HTML 5 are not enemies; they are complementary technologies, each one with its own advantages. I would never choose between them. I would choose between their features and use what suits best my work.

PS 3: This post was updated on January 14, after René Schulte's comment regarding the application's optimization.

  • Anonymous

    Nice post.

    You should cache the Pixels properties in local variables to avoid calling the properties inside the loop. This could result in a pretty decent performance improvement.

    BTW, XOR is usually used to get a difference image.

    Keep it up!

  • @Rene

    I really appreciate your comment. You are right considering the performance improvements that need to be done. I am going to update the source code.

    There can be applied some more optimizations too, but I intend to keep the code as clear and easy-to-read as possible.

  • Anonymous

    Nice blog post to aware people's about CCTV surveillance cameras. In today's world, its too necessary for anyone's security. So keep it up !

  • Anonymous

    Hi! Have you tried using this in Windows Phone 7? Because we tried to add your library as a reference and did not work (for WP7). Any advice on this one? I hope to hear from you soon. Appreciate it.

  • Anonymous

    Excitement  overloaded,here happy wheels is the most amazing place of unlimited fun a huge fans following it and http://happywheels.me hence thank you so  much for sharing the information.