Processing: Face Swap Next Iteration

My first iteration had some very messy code so my first step was to clean out the bits that weren’t being used to cut it down to the barebones of the parts I needed. The next step was to work out how I was going to swap the faces over for when there were 3 and 4 faces. At the moment I’m limiting it to 4 faces due to the stress it puts on my laptop, but for all intents and purposes I feel that 4 faces is enough. If the tracking detects any more than that it wont swapping the faces over and they’ll stay looking normal.

As a reference I drew out a table so I could keep track of which faces I was swapping. I tried doing it all in my head but kept getting it wrong so this is what I had to resort to. The faces along the top represent the faces in the array and the ‘swaps’ are which faces they will be swapped with for 2, 3, and 4 faces.

Screen Shot 2014-12-06 at 11.13.37

From here it was a simple matter of writing the if statements needed for up to 4 faces, resizing and placing the images appropriately:

//swap two faces over
  if (faces.length == 2) {
    //resize images to current tracked faces
    face0.resize(faces[1].width, faces[1].height);
    face1.resize(faces[0].width, faces[0].height);

    //place swapped faces
    image(face1, faces[0].x, faces[0].y);
    image(face0, faces[1].x, faces[1].y);
  }

  //swap three faces over
  if (faces.length == 3) {
    //resize images to current tracked faces
    face0.resize(faces[2].width, faces[2].height);
    face1.resize(faces[0].width, faces[0].height);
    face2.resize(faces[1].width, faces[1].height);

    //place swapped faces
    image(face0, faces[2].x, faces[2].y);
    image(face1, faces[0].x, faces[0].y);
    image(face2, faces[1].x, faces[1].y);
  }

  //swap four faces over
  if (faces.length >= 4) {
    //resize images to current tracked faces
    face0.resize(faces[2].width, faces[2].height);
    face1.resize(faces[0].width, faces[0].height);
    face2.resize(faces[3].width, faces[3].height);
    face3.resize(faces[1].width, faces[1].height);

    //place swapped faces
    image(face0, faces[2].x, faces[2].y);
    image(face1, faces[0].x, faces[0].y);
    image(face2, faces[3].x, faces[3].y);
    image(face3, faces[1].x, faces[1].y);
  }

The last if statement is greater than or equal to 4 faces so that if the camera picks up more than 4 faces, the first 4 will still swap but anything over that won’t be effected. One thing that bothers me with this code is that there is a lot of repetition of very similar lines of code for each statement. While it still works fine I would like to try to clean it up if at all possible to make it look a bit neater.

One thing I didn’t mention in my first post is the frame rate when it runs. In the example which comes with the OpenCV library (my basis for building this), the video is halved and then scaled back up to fit the window. This is so that the tracking is working on a lower resolution version of the video so that it can run faster and improve performance. In my attempts, the scaling kept confusing me while trying to grab the faces so I turned it off. This means that it is doing the tracking on a relatively high resolution video which takes longer to check for faces due to the larger number of pixels to check and affects the frame rate. My sketches currently run at about 4/5 frames per second which isn’t exactly very good. If it was run on a more powerful computer it could work a little better but I need to find a way to optimise the code and improve the frame rate up to a more reasonable level around 20/25 fps.

Below is it swapping 3 faces which works as expected. What I did notice is that sometimes it sees the faces in a different order, causing them to jump about a bit from face to face which could either be a good or bad thing. It could be considered a good thing as it makes it a bit more interesting as you have to keep track of where your face is going. Although it could get quite annoying and detract from the effect if the face doesn’t stay in one place long enough. Also the tracking isn’t exactly perfect. As you can see in the picture below there are 4 faces but one of them is being ignored for some reason. It could be a combination of lighting, positioning, the angle of the face or something completely different which is stopping it from being seen as a face by the sketch. At the moment I don’t think this is a problem with my code but rather the limitations of the Haar Cascade which is currently being used to detect the faces.

Screen-Shot-2014-12-05-at-14.20.13

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: