Friday, June 8, 2012

Finger Tracking + Android 4.0 + Tegra 3 + Kinect + OpenNI + OpenCV

This tutorial we will show you how to implement the 'well-known' finger tracking algorithm on Kinect + Android + OpenNI + Tegra3 + OpenCV. The algorithm is indeed very simple and computationally efficient. We can achieve real-time performance easily on Tegra 3 without any s

We will be using the range camera (depth map) to create a contour of our hand. Then, we will perform a simple polynomial approximate on the contour, and extract the convex hull (the peaks) as our finger tips. All of these operations can happen in real-time on the Tegra 3 @ 25fps! That's pretty impressive.

The requirements are as follows:
  1. Tegra 3 or Tegra 2 Android device.
  2. Microsoft Kinect or Asus Xtion (not tested)
  3. NVIDIA NDK + OpenNI binary (see my previous tutorials)
  4. OpenCV binary (will be provided below)
  5. and 10 mins of your time :)

First of all, the method provided here are simplified for real-time usage, and we can get better results if we have a better way of extracting the hand location and filtering the depth map (See Limitation section at the end).  Right now, we are thresholding on a range, i.e., we assume that the hand of a person will be positioned closer to the range camera.

Contour Extraction.

Area and Bounding Boxes.

Polynomial Approximations and Convex Hulls 


My Setup. Tegra 3 Tablet + Kinect; powered by a 12V battery.
Coming up Soon...


  1. Wow this is amazing and the performance looks usable.

  2. Hi,

    Can you upload this Open Source Project ?

  3. it is on the svn. check my previous posts

  4. Hi ray :) .Can I have your mail ID .Because we need some information about the kinect project

  5. I saw this blog and I have benefited a lot. From this site I hope you can stomach the price of good quality and dighasthayi batteries. With the YABO Portable 12v UPS battery, 12v Lithium Ion battery, LED Strip and LED Lamp12v battery, 12v portable battery is also available in all types of batteries.
    For more information about their services, feel free to visit their site