We will be using the range camera (depth map) to create a contour of our hand. Then, we will perform a simple polynomial approximate on the contour, and extract the convex hull (the peaks) as our finger tips. All of these operations can happen in real-time on the Tegra 3 @ 25fps! That's pretty impressive.
The requirements are as follows:
- Tegra 3 or Tegra 2 Android device.
- Microsoft Kinect or Asus Xtion (not tested)
- NVIDIA NDK + OpenNI binary (see my previous tutorials)
- OpenCV binary (will be provided below)
- and 10 mins of your time :)
First of all, the method provided here are simplified for real-time usage, and we can get better results if we have a better way of extracting the hand location and filtering the depth map (See Limitation section at the end). Right now, we are thresholding on a range, i.e., we assume that the hand of a person will be positioned closer to the range camera.
Area and Bounding Boxes.
Polynomial Approximations and Convex Hulls
My Setup. Tegra 3 Tablet + Kinect; powered by a 12V battery.