CS 180: Project 3

Face Morphing

Meenakshi Mittal

Overview:

In this project, we walk through the process of producing morphs between faces. We use these ideas to compute the mean of a population of faces, extrapolate from a population mean to create caricatures of myself, and have some additional fun with face morphs and a beloved video game character.

Part 1: Defining Correspondences:

To start off, we will attempt to morph images of my friend Wiktor and me. The images were naively aligned and resized to match each other using the provided script from Project 2.

Our first step is to label correspondence points on the images of the faces we want to morph together. I wrote a script that allowed me to see exactly which points I labelled on each face, so I could match up the labels on the other face. I labelled 49 points on each image, including 8 points along the borders to ensure full image morphing. Here are the labelled images:

Part 2: Computing the "Midway" Face:

Now we will compute the midway face between these two images. Here is the process we follow:

1) Take the point-wise averages of all the correspondence pairs.
2) Compute the Delaunay triangulation of these midway points.
3) Use this triangulation to map each triangle of pixels from the original images onto their corresponding "midway" triangle, using an affine transformation. (In reality, we are mapping the triangles from the midway image onto the originals, interpolating from the surrounding points, and then mapping that back to the midway image)
4) Take the average of the 2 morphed images to get the midway colors.

Part 3. The Morph Sequence:

Using a similar process to the one described above, we can create a 45 frame animation that morphs from one face to the other. The only difference is that instead of taking the average shape and color, we will take 45 weighted averages of shape and color. For 0 < alpha < 1, this just looks like: alpha * im1 + (1-alpha) * im2. This applies for both the correspondences point and color calculations.

Just for fun, I repeated this process to morph a different photo of me to Christian Bale's iconic character Patrick Bateman in American Psycho.

Part 4. The "Mean Face" of a Population:

We can also use this technique to compute the average face of a population. I acquired a set of 200 neutral and smiling grayscale portrait images from https://fei.edu.br/~cet/facedatabase.html. Each image also came with a set of 46 correspondence labels each, to which I added the 4 corner points to capture the full images. I computed the average correspondences, morphed each image to the average, and then took the average of all the morphed images.

Here are a few examples of the original images and their morphs:

Here are the averaged faces for the two populations:

Now I will morph my own face to the average geometry of the neutral faces, and I will morph the average face to my own geometry. I will use a different image of myself from before, to better match the type of headshots used in this dataset.

Part 5. Caricatures: Extrapolating from the Mean:

Using the average face we found above, I can create caricatures of myself. I use the following formulas to compute this:

avg_to_me_vector = me - avg
me_caricature = me + alpha * avg_to_me_vector

This can be thought of as "adding alpha more of me to me".

This is similar to the process from earlier, but now we are using alpha outside the typical [0,1] range.

These definitely look... odd. I guess what this tells me is that I have a pointier-than-average chin, a smaller-than-average space between my nose and mouth, and a wider-than-average nose. Interesting.

Bells and Whistles:

We will try a few more face morphing ideas. First, let's see how I would look as a man.

Since I am ethnically Indian, I found an image of the average Indian man. Here is the resized image side-by-side with the image of myself that I will use.

Now I can use the same morphing techniques from before to achieve a few morph variations. I labelled correspondences on the image of the average Indian man to match the existing ones on the image of me. Below, I have done an average morph of the shape, color, and both.

For fun, I also tried to make a caricature using these images. I used the same approach from before, and I got what is supposedly the more feminine, less Indian version of me:

Moving on to our last experiment. The main character of the popular video game franchise "The Legend of Zelda", Link, has had many different iterations over the years. We will attempt to find the average Link. I found an image that compiled 30 different versions of the character, and subdivided it to get the 30 images I would use. I excluded a couple that had harsh angles / side profiles that I thought would be hard to label. I labelled 58 correspondences on each image and morphed them together. The results are displayed below:

That worked way better than I expected it to-- I thought the different angles and styles of the images might make the result contain more artifacts or be asymmetric. I suppose the large number of images and correspondence points helped iron these issues out. I think it's cool that the face was extracted very clearly, and other iconic features like his hair and hat are also somewhat visible.