Calculating an average face

Example of average face

Example of average face

In the comings and goings of my thesis, now that I am certain that my main topic was way too big (my supervisor liked it but he says that it was more like a whole research curricula that a single thesis), I try to focus and cover only a fragment of the initial goal. Big Culture stills sounds in my head since that October of 2012 when I first thought of the idea. However, my thesis won’t be a monograph anymore, but a set of articles related to Big Data in Humanities.

One of these articles is already on process, and its topic is related to the representation of faces in world painting. An abstract has been sent to DH 2014, hosted at University of Lausanne, Switzerland. After a successful preliminary work for DH 2013, this time I have been working on deeper analysis from our proudly collected data set of 47k faces in paintings across time. As a part of the research process, and as usual in any paper conceived in the CulturePlex, there is some programming involved. In this case a lot of matplotlib, scipy, numpy, IPython, and Pandas (and even PIL/Pillow), a set of Python libraries and tools that quickly become our main stack for data analysis.

One interesting challenge that came from this research was the generation of an average face. The first thing I noticed was that my machine was not able to handle that amount of images due to its limited RAM (4GB), so I asked for help to SHARCNet and Prof. Mark Daley kindly offered me a user in one of the 32GB Mac machines available, so I managed to get installed IPython with all the libraries inside a virtualenv, I copied all the files needed, and then started the Notebook.

$ ipython notebook --pylab inline --no-browser --ip=<YOUR PUBLIC IP HERE> --NotebookApp.password=`python -c "from IPython.lib import passwd; print(passwd())"`

Among the features that I have available for a face (after applying face detection algorithms), there is the centroid of the face. From that point, and using height and width as well, I can trace a rectangle that delimits the boundaries of a face. Then I center all the faces by their centroid and resize all the images to have same height. In order to calculate the average face, I first implemented a solution that made use of opacity/alpha levels in matplotlob, but that seems to be limited to 256 layers (don’t know if can be increased) and works pretty slow. consuming all the resources of the machine really fast. After trying some other methods, I came with the idea that an average image is as simple as a standard statistical mean calculated for every single pixel. Images were in RGB color model, so corresponding matrices had 3 dimensions. If I were used grey-scale images the whole process would have been 3 times faster, although for the sizes of images that I am handling (faces of 200 by 200 pixels), there is almost no difference.

A simplified version of the code used is shown below, although is subjected to performance improvements.

def face_detail(face):
    mode = 'RGB'
    desired_height = 250
    center_at = [400, 400]
    img = faces.load_image(face)
    features = faces.load_features(face)
    center_pct = (
        features["center_x_pct"],
        features["center_y_pct"]
    )
    height = features['height']
    width = features['width']
    painting_height = features['painting_height']
    painting_width = features['painting_width']
    # Resizing
    pil_img = PILImage.fromarray(img, mode)
    resize_height = 1.0 * painting_height * desired_height / height
    resize_width = 1.0 * painting_width * desired_height / height
    resized_img = pil_img.resize(
        (int(resize_width), int(resize_height))
    )
    # Shifting
    shift_point = [
        center_at[1] - (center_pct[1] * resize_height / 100.0),
        center_at[0] - (center_pct[0] * resize_width / 100.0),
        [0]
    ]
    shifted_img = ndimage.shift(
        resized_img,
        shift_point,
        mode='constant',
        cval=256,
    )
    # Cropping
    xlim = slice(center_at[0] * 0.5, center_at[0] * 1.5)
    ylim = slice(center_at[1] * 0.5, center_at[1] * 1.5)
    cropped_img = shifted_img[xlim, ylim]
    return cropped_img

def get_average_face(faces):
    imgs = []
    center_at = [400, 400]
    for index, face in faces.iterrows():
        try:
            img = face_detail(face)
            if img is not None:
                # Adding images
                array_img = np.array(img)
                array_img.resize(center_at + [3])
                imgs.append(array_img)
        except Exception as e:
            msg = "Error found when processing image {}:nt{}"
            print(msg.format(face, e))
    # Averaging
    avgface = np.array(imgs).mean(axis=0)
    avgface = avgface.astype(numpy.uint8)
    return avgface

fig, ax = plt.subplots(1, 1, figsize=(10, 10), dpi=300, facecolor="none")
average_face = get_average_face(faces)
ax.imshow(average_face, interpolation='bicubic')

Some other problems still need to be addressed, i.e. face rotations. The use of affine and projective transformations can solve that, as well as replacing the method of resizing and shifting to re-center all the faces.

1 Comment

Filed under Tasks

One Response to Calculating an average face

  1. Pingback: The BaroqueArt Database: A Relaunch | The Official CulturePlex Blog

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>