I’ve decided to investigate some of the computer vision libaries that are already available that could possibly already do what I need. This is likely to save me a lot of time not having to re-invent the wheel.
I had the idea to look into OpenCV when I noticed a particular repository trending on Github the other day. The repo in question; maintained by Raphael Monnerat was titled SmartMirror and is designed to turn a two-way mirror into a gesture controlled smart device.
The potential for me to apply a similar algorithm to the one he was using to my project was almost too obvious, as it looked as though his implementation used some form of Hand Detection/Recognition. He even had a debugging tool packaged with his code that would display the gesture recognition information in real time while testing.
My first task was to install OpenCV on a system that I could use to test the SmartMirror code. I span up an Ubuntu 16.04.1 instance and ran the following code to setup the latest OpenCV version on the system.
## Install pre-requisites
$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install build-essential
$ sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
$ sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
## Change directory into a working folder
$ cd ~/Documents/
## Clone OpenCV repo and build from source
$ git clone https://github.com/opencv/opencv.git
$ cd opencv/
$ mkdir release
$ cd release/
$ cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
$ make
$ sudo make install
Now that I had the latest version of OpenCV installed I needed to make sure it was working. I also decided that my language of choice; particularly during the initial testing phases would be Python.
## Change into the OpenCV samples directory
$ ~/Documents$ cd opencv/samples/python/
## Either run each tool separately or use demo.py to list all demos
$ python demo.py
I had a play around with a number of the built in tools and was very impressed by the level of quality and detailed code each example provided. One of my personal favorites was the ‘edge.py’ example as I felt like it would provide a very good starting point for when I begin writing my hand detection method.
Now that I had OpenCV installed I decided I would attempt to demo the SmartMirror hand detection code to see if it was as good as it looked like it was.
I started out by cloning the repository using git and locating the ‘test.py’ file mentioned in the repo README.md.
## Change into SmartMirror/Motion directory
$ git clone https://github.com/Shinao/SmartMirror.git
$ cd SmartMirror/Motion
## Execute the test.py script
$ python test.py
The example was very finicky to get working, The hand detection itself works very well when it isn’t getting confused about the background lighting.
Difficulty assign, the implementing was detecting my palm, and there was even logic in place to handle hand movements (swiping in different directions).
Today was a productive session that really got me thinking about the possibilities that OpenCV has to offer. Now that I know that what I want to achieve is very possible with the OpenCV libraries I believe my next step will be to learn the OpenCV frameworks from scratch. My goal for the next week is to build up a small library myself so that I can begin to understand how other peoples code works without having to guess/hack a solution together.
I picked up a digital copy of Practical Python and OpenCV by Adrian Rosebrock as I’ve had it recommended to me before as a great practical reasource for learning the in’s and out’s of OpenCV.
Practical Python and OpenCV - https://www.pyimagesearch.com/practical-python-opencv/
SmartMirror - https://github.com/Shinao/SmartMirror