As a part of our project, we have to provide a way by which the user can point out the faces in a serie of Baroque paintings. Theses faces are previously calculated using face recognition algorithms and SimpleCV, a simplified API for OpenCV thar, of course, doesn’t work on my 64bits machine. In fact, there are differente methods, with advantages and drawbacks, used at the same time. But algorithms are not perfect, and in general they miss some “obvious” faces as long as they wrongly tag things likes dresses as faces.
Above these lines you can see the initial screen before starting to play. In the future we are thinking about including more than one game. The first one is focused on identifying faces through the input of the users. The second one could be, for example, a game in order to filter that previous selection, something like, once we hace faces selected and extracted, a Invaders-like game in which users has to shot just to faces of certain kind. Very useful to filter and provide even more information for our algortihms.
However, we need to ask for the user the minimal interaction ever, so we expect from the user just one single touch on the screen. And this raises a problem: how to, with a single point in the screen, get the face the user is trying to point. To resolve this we are exploring two approaches.
The first one is to wish the users are going to make a circle around the heads or faces of the characters on the painting. But this is against our main goal to request as minimal interactions as possible. Our idea is the users can punch the screen where they think faces are, not draw a circles around. We need to think in another approach.
So the next step is to provide a way to make even more usefull the input from the users. And we decided the creation of a second game. In one hand, we previously did a detection of some faces, so we can calculate how near are the single touchs to the already recognized faces. But we don’t know this information for new faces that our algorithms missed out. If this happens, all we have is a point in the painting and no way to figure out if it is relative to a face. In this scenario, we mark these points for second stage in our process. Thus a second game in which we show to the users these images properly cropped and we pretend they just poke eyes out. Then, we have information about almost the center of the face (the previous single points), the current position of the eyes, and information about the medium size of the rest of the heads our algorithms did detect before: we can now triangulate the shape of the face/head. So in this way we filter a lot of information just making the users play a different game. The common feature in both games? The violence: Punch and poke eyes out!