Plone, HTML 5 Canvas and Face Detection with Webcam

One of my latests articles was about HTML 5 Canvas and Webcam integration and in the same article I put all together in a Plone add-on for integrating portrait changes with Webcam.

Recently, following a retweet of one of the cool guys I follow on Twitter, I randomly hit an article that talk about a game that integrate user's webcam as a controller (unluckily I lost the original link, neither I remember the programming language used. The article were also generally introducing face recognition, and this captured my attention.
I asked myself how cool could be getting a face recognition feature with Python. It this something possible to do? Probably not so easily...

Introducing OpenCV

...or not? If you look for "face recognition" and "Python" on Google you'll always get references to OpenCV, the Open Source Computer Vision Python library.
I simply scratched the surface of this huge piece of technology as I'm totally a newbie about computer vision. What I get is that the library can really do a lot of stuff, and it's well documented.

Let me introduce some very-general information.

To use OpenCV for detecting faces on an image you must understand the difference between "face recognition" (find a know face on image or video) and "face detection" (find a face, in general). Obviously the second is a simpler task. Why? Because whatever is your task, OpenCV must be trained to find your target. For not simple task like face recognition you can (for example) train the software by providing a set of images where the object face can be found, and the result of the train is an XML file. After that you can you this file to implement something like Picasa, iPhoto or Facebook are doing when you submit new photos.

With face detection things are simpler because you can find one of those XML file online, already generated for you.

Another important informations about the library: recently a deep API changes has been performed so a lot of examples you can find online are broken (or must be fixed).

Finally, when you are able to do so, prefer the use of cv2 library instead of cv. They are more or less the same library but (for what I understand) cv2 is faster because is based on numpy, so C-compiled code.

Going back to what i did: I focused on face detection. Let see how.

Applying face detection to Plone (yes, I said it)

Meanwhile I were also fixing some minor issues in collective.takeaportrait. A new minor feature is the possibility to move the viewfinder by using mouse drag&drop (because must be forced to be in the middle of the screen is not comfortable).
Here came The Idea: how about a viewfinder that automatically center onto the face captured by the webcam?

Here my wish list:

• JavaScript check for server side availability of face detection feature (just because OpenCV is not a simple library to be installed... and let me be honest: this is a cool feature, but not really useful)
• With a not-so-long delay the whole Webcam image take by the canvas is sent to the server for a face detection view
• OpenCV on the server perform the face detection
• If a face is found, a rect is sent back to the JavaScript callback
• The viewfinder is centered on the face using the feature already implemented for drag&drop

The use of Plone here it's a bit unnatural but It was the simplest environment for my experiment because of the work already done with the webcam in the last article. Apart Plone itself I hope you'll the general idea: how simple can be the browser/webcam/face-detection integration, whatever will be your back-end Python framework.

As you can suppose, the experiment was a success!

Introducing collective.takeaportrait face detection feature

I don't think I can add more useful details. Let's see the video!