Category Archives: Other Processing

Processing: Masks

As a little experiment while I was working on the toping of identity and representation I decided to make a little sketch which would put a mask on all of the faces the video feed sees. Using the basic face tracking example as my starting point I worked out how I could go about doing this.

The idea was relatively simple; It required creating a PImage to store the mask image, loading the mask image into it, then resizing and drawing the mask for each face it sees. This is the mask picture it uses. It’s a .PNG so that it can have a transparent background to get the best effect possible so the eyes and part of the mouth can be seen through the mask. the mask uses the x, y, width and height for each face taken from the array which stores all the details. It took a little experimenting to get the eyes to line up right which ended up increasing the size of the mask by 30px each way, and shifting it up and left by 10px. This meant it would perfectly align on a forward facing face. mask

 

However when I made it, it ended up looking like this (below) where the mask gets blurrier and blurrier until it is unrecognisable. To solve this I realised that I needed to move the line which loaded the mask into Processing from the setup to the draw section. With it in the setup, the image was loaded once and left to be resized continually 30 times a second (or whatever frame rate it was running at) which made the quality continue to drop as the image is squashed and enlarged continually. With it at the start of the draw section, it meant that the original image would be loaded and the beginning of each frame to then be resized so that a fresh version of the mask was being used each time rather than the already diminishing quality one. This problem has actually been mentioned in my main project here with the problem of resizing the masking layer, and this little experiment is how I actually solved it (confusing backwards posting, i know).

[gfycat data_id=”ExcellentSpiritedCrocodileskink”]

Here is the fixed version, as you can see the mask quality stays the same throughout:

[gfycat data_id=”BaggyHorribleBarnowl”]

Processing: Capturing faces FIX

In my last post about capturing faces I had the problem that the output of each face captured was constantly being overwritten and I could only have 4 faces saved at once. I finally managed to work out a way around this, changing the name of each image saved so that it isn’t over written and are each saved with unique names.

//Before the setup
int number = 0 //number of picture 

//.... below written in the draw

// only runs if a face is seen
  // captures a face in the box
  // start timer
  if (passedTime > totalTime) {
    number++; // adds number to picture so they count up and not overwrite
    if (faces.length >= 1) {
      face1 = get(faces[0].x, faces[0].y, faces[0].width, faces[0].height);
      String number1 = "1_" + number + ".jpg";
      face1.save(number1);
      if (faces.length >= 2) {
        face2 = get(faces[1].x, faces[1].y, faces[1].width, faces[1].height);
        String number2 = "2_" + number + ".jpg";
        face2.save(number2);
        if (faces.length >= 3) {
          face3 = get(faces[2].x, faces[2].y, faces[2].width, faces[2].height);
          String number3 = "3_" + number + ".jpg";
          face3.save(number3);
          if (faces.length >= 4) {
            face4 = get(faces[3].x, faces[3].y, faces[3].width, faces[3].height);
            String number4 = "4_" + number + ".jpg";
            face4.save(number4);
          }
        }
      }
    }
    
    println( " 5 seconds have passed! " );
    savedTime = millis(); // Save the current time to restart the timer!
  }

Inside the timer still I have a new variable called ‘number’ which is the number of the image being taken. Every 5 seconds a value of 1 is added onto number making it count up each time the timer runs. This value is used in the name of the image being saves so that it changes each time.

For each image a string is created. For example with the first face it would output ‘1_1.jpg’, 1_2.jpg’, 1_3.jpg’ etc. continually counting until stopped.

With this there are still only 4 PImages in Processing that are continually being updated, it is just the name of the image being saved which is changed. This still means there are only 4 running images in processing and once it has been saved and overwritten, it isn’t seen by processing anymore. I may be able to save the images into an array instead or simply add more image variables but for now this will stay until I work out the best course of action.

Processing: Capturing Faces

In a recent post I started looking at OpenCV for processing, and more specifically the face tracking. Applying this to my idea of surveillance, I wanted to work out how to capture the faces it detects and save them. Before starting I knew that the sketch would be recognising faces every frame and if I just wrote some code to capture the faces straight into this, it would be capturing far too many images, at too high a speed. To get around this I decided to implement a basic timer from an example I found. The example printed a line in the log and changed the colour of the background every 5 seconds so you knew it was keeping track of time. I adjusted this so that it would save an image of a detected face ever 5 seconds instead.

import gab.opencv.*;
import processing.video.*;
import java.awt.*;

Capture video;
OpenCV opencv;

PImage face1 = createImage(0,0,RGB);
PImage face2 = createImage(0,0,RGB);
PImage face3 = createImage(0,0,RGB);
PImage face4 = createImage(0,0,RGB);

// Values for clock
int savedTime;
int totalTime = 5000;

void setup() {
  size(640, 480);
  video = new Capture(this, 640, 480, 15);
  opencv = new OpenCV(this, 640, 480);
  opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);  
  frameRate(15);
  video.start();
  savedTime = millis();
}

void draw() {
  scale(1);
  opencv.loadImage(video);

  image(video, 0, 0 );

  noFill();
  stroke(0, 255, 0);
  strokeWeight(3);
  Rectangle[] faces = opencv.detect();
  println(faces.length);


  // Calculate how much time has passed
  int passedTime = millis() - savedTime;


  for (int i = 0; i < faces.length; i++) {
    println(faces[i].x + "," + faces[i].y);
    rect(faces[i].x, faces[i].y, faces[i].width, faces[i].height);
  }

  //only runs if a face is seen
  // captures a face in the box
  if (passedTime > totalTime) {
    if (faces.length >= 1) {
      face1 = get(faces[0].x, faces[0].y, faces[0].width, faces[0].height);
      face1.save("output1.jpg");
      if (faces.length >= 2) {
        face2 = get(faces[1].x, faces[1].y, faces[1].width, faces[1].height);
        face2.save("output2.jpg");
        if (faces.length >= 3) {
          face3 = get(faces[2].x, faces[2].y, faces[2].width, faces[2].height);
          face3.save("output3.jpg");
          if (faces.length >= 4) {
            face4 = get(faces[3].x, faces[3].y, faces[3].width, faces[3].height);
            face4.save("output4.jpg");
          }
        }
      }
    }

    println( " 5 seconds have passed! " );
    savedTime = millis(); // Save the current time to restart the timer!
  }

  //display captured faces in corners for now 
  if (faces.length >= 1) {
    image(face1, 0, 0);
    image(face2, width-face2.width, 0);
    image(face3, 0, height-face3.height);
    image(face4, width-face4.height, height-face4.height);
  }
  
} // close draw

void captureEvent(Capture c) {
  c.read();
}

The timer loop contains another if statements to make sure that the face capturing part only runs if there is at least one face detected. This part took a long time to work out as to capture the faces I needed to access the array which the faces were stored in, however when there are no faces detected, it is trying to access a part of the array that doesn’t exist yet which causes it to crash. The series of nested if statements which follow detect up to 4 faces and save them using the get feature in Processing to capture the pixels within the rectangle and save them as PImages created before the setup. For now I’ve only got it saving 4 faces. OpenCV will be able to detect and track more but for now I feel that this is enough.

Currently there is one big flaw with this as when I save the output for each face, it always has the same name and this causes it to overwrite the face every 5 seconds with a new one. My next iteration will hopefully fix this flaw.

For now, I have the current captured faces appearing in the corner of the sketch as I’m not sure what I’m doing with them yet, It was just a more visual way of seeing what the code is doing as it runs.

Here is an example of it working:

[gfycat data_id=”UnrulyMediocreHellbender”]

The sketch runs at 15 frames per second as my laptop struggles to run it at a higher frame rate but currently its good enough for what I want it to do. Towards the end of the clip you can see it detecting a face on my wall where there isn’t one. This is one of the main problems with the face detection at the moment as sometimes it sees patterns which it thinks are faces where there are none. The best way i’ve found to get around this is to work in a well lit area, it’s a slight improvement but still is by no means perfect. Below is the very flattering image it captured and saved. From my experimenting with this it seems to capture the image when you least expect it (even though I know its coming), meaning you never really get time to pose for it leading to more natural representations of people.

output1

This intrigued me and made me wonder about what kind of faces I could capture when people don’t expect it. Even when I can see the video feed and know it is on a 5 second timer it still manages to capture images when I really don’t expect it, often of me concentrating reading through the code or mid-conversation. If I was to have something else on the screen, rather than the video feed, people would be completely unaware that they’re being watched (other than the presence of a camera of course) and I would be able to capture and save images of their faces without them ever being aware of it. Then with these images I could do just about anything I wanted (within reason) and they would be none the wiser.

Clock Update

Screen Shot 2014-11-07 at 10.34.25

 

After creating the basic clock, I decided to experiment with another way of displaying the time. My new, updated clock uses the hour, minute and time to make bars which are a visual representation of the time. The values for the time are multiplied by 4, to make the values bigger and then used to adjust the y values for the rectangles.

To do it this way i had to change the rectangle mode from the default setting to “CORNERS” which allows you to define the coordinates for a corner, and its opposite to draw the shape, rather than just one corner and the height and width.

My inspiration for this was a Pebble watch face I used to have on my watch which had bars to represent the progress through the year, month, week and day.

While still not exactly that interesting to look at, I feel this was a good experiment to play around with using different variables for adjusting shape, a method that may be useful in future projects.

Heres a little gif of it in action, sped up as second are too slow:

export2

Processing: Learning Time

Last week we were challenged with making a clock in Processing that will display the time and date.

for this it uses 6 basic functions which access the clock on the computer it’s running on:

  • hour();
  • minute();
  • second();
  • day();
  • month();
  • year();

These functions are pretty self explanatory, they look into the system and give you the current values for each of them.

The first step here was making a simple clock for the time. To do this I got made a variable for each hours, minutes and seconds and then the values were converted into strings to the text could be displayed. To display the text a string is formed which combines all the values and then is saved another variable to make it easier to read

String timevalue = hwith0 + “:” + mwith0 + “:” + swith0;

The string is then printed using text() and the background is refreshed each frame to make sure the numbers are written on a clean slate and not on top of each other.

The problem here was for single digit values it only showed a single digit. With the usual clock format we’re used to, it uses 01, 02, 03 etc. until it reaches 10. All credit here goes to Joel Trew for solving this problem simply. It uses an if statement to to check the length of the value of the seconds for example to see if it is equal to one (i.e the numbers 0-9). If this statement is true it then just adds a string (“0”) to the beginning to make them display at the desired 00, 01, 02, 03 and so on.

Screen Shot 2014-10-30 at 16.02.45

Having a basic understanding of how to make a clock i wanted to develop this further by making it into a class so it is neater and easier to read. for this I made a class called time() which took the variables timeX and timeY so when putting it into the main body of code it can easily be placed without editing the class. The clock here uses the same code as above.

Screen Shot 2014-10-30 at 16.15.44

Along with the clock I wanted to display the date, this was simple enough using the same methods as before to get the values from the computer’s clock, convert them into strings and display them as text. Obviously this was too easy and I wanted a little challenge. The first thing I went for was displaying the day of the week alongside the clock. My first thought here was to use an algorithm that worked out what day of the week it was based on the date itself, but this proved to be a lot harder than i thought. I found a Wikipedia article which outlined the methods but they were far too complex and I couldn’t be bothered to work them out.

Then I remembered that Processing is based on Java and that I could just import the utilities I need to use. There is a Calendar java utility which conveniently has a method for finding the day of the week and then give it as a number value. For example today is Thursday and it returned the value ‘5’ meaning it is the 5th day of the week (starting Sunday).

From here I created an array which contained a list of strings which were each the week days starting with Sunday to match the Java calendar. As when accessing an array the first value is 0 I had to subtract 1 from the value the calendar gave me to correct it.

When including this text I thought why not display the month as text as well to keep with the theme. This used the same method of having an array store the month strings and then just accessing them with the month value, but subtracting one to make sure it starts at 0 as with the array.

Having the date display Thursday 30 October 2014 didn’t really roll of the tongue very well so I wanted this to then be displayed as 30th to match the way we usually speak and write the date. The obvious hump here was the ones that stand out, 1st, 2nd and 3rd. The way around this I found was just to use ifstatements to see if the value for the day is 1, 2 or 3 and then add the appropriate suffix to string. An if statement was also used to add a ‘th’ to all the values greater than 3.
Screen Shot 2014-10-30 at 16.32.22

For the sake of simplicity I made the time and date functions in different tabs but still within the same processing document. This meant that the main body where we have our setup and draw is clear and isn’t cluttered with too much text. It was just a simple process of making sure the background is drawn each frame and then the date and time being placed in as the desired coordinates. 
Screen Shot 2014-10-30 at 16.15.20

Obviously the clock here doesn’t look very interesting, but hey it works and took a fair few hours to get my head around making it work. The next step from here is to develop upon this, experimenting with shape and colour like I have before to make a more visually interesting clock. For example I can have it change colour for each second or minute, or change location around the screen. The clock information could also be used to create a visual representation of the time rather than a digital clock, much like an analogue clock. This will be my next course of action to play around with different ideas related to this.

Screen Shot 2014-10-30 at 16.39.25

More Processing!

Over the week I’ve been busy practicing with processing, making some new experimental sketches to come to grips with how the if statements and for loops work.

One of my first ones was pretty similar to what i’ve made before, it was basically a spinning triangle which changed hue; nothing too interesting. I made the triangle using beginShape() as it allows you to plot the location of each vertex of the shape and lets you have as many vertexes as you want. Putting the frame rate up to 60 make it better for a drawing tool as the shape was drawn twice as many times a second as the default.

Screen Shot 2014-10-11 at 14.50.20

I play around with opacity and changing size for the shape to make it a bit more interesting, clearly it looks a bit more fun than the above image which is cool.Screen Shot 2014-10-11 at 13.25.10

With this tool i found that if I just held it in one place it would make some things which looked like flowers, but due to the way the size changed and the hue changed it limited the colour ranges of the flowers when making them, meaning I was only able to get some greeny/blue ones and some pink/yellow ones.Screen Shot 2014-10-11 at 13.24.22

This one is my first foray into using bezier curves. With these curves you plot a start point and an end point, as well as two other ‘control’ points with change the shape of the curve. As with before I started simply by making it rotate and change colour to see what patterns I could create. The bezier curve is a really good way to make more interesting shapes and I went on to experiment with it further.Screen Shot  2014-10-11 at 12.50.22

I decided I wanted to set myself a design challenge. Starting relatively simply I went for making a flower. For this flower i split it into 3 main classes, the stigmas, the petals and the leafs. Each class is the method for drawing one stigma, one petal and one leaf. In order to make the flower I had for loops running on click with various amounts in the loop. Having 3 different loops that ran meant I could have different numbers of each thing, e.g. 5 stigmas, 5 petals and 3 leaves. The loops randomly rotate each element each time its ran meaning the flowers are sort of randomly generated.Screen Shot 2014-10-17 at 14.38.54

due to this random rotation it also meant that sometimes the flowers didn’t really come out as intended and it sometimes took multiple attempts to get a flower that looked someone normal.
Screen Shot 2014-10-17 at 14.40.34

Because of the randomness making horrible looking flowers, i looked into a better way of working out the rotation of the shapes to get a more evenly spaced flower. I came up with a method that creates an evenly spaced flower but now I think each time its make it looks too similar to the last. Here is where I need to develop this idea and add an element of randomness back in, but limit it to keep the petals and leafs more or less spaced evenly. Screen Shot 2014-10-17 at 15.07.20

First Project: Processing

As I mentioned here, the first project we are doing is going to be made using processing. The ultimate aim is the create an interactive information graphic about 21st century media to be displayed on the screens in the foyer of the media building at uni.

For those who are keen viewers of this wonderful blog, you’ll remember that I’ve already done some work in processing, experimenting with generative design and the Twitter API. In friday’s workshop we got to grips using more complicated methods in processing such as for loops, if statements and even the odd modulo.

The first thing we made was a simple drawing tool which used an if statement to see if the mouse was clicked, and if it was it would then draw the circle and follow the position of the mouse. Obviously this was very basic stuff so we experimented a bit further by making the size of the drawing tool increase. For this we used a modulo e.g. x=(x+1)%50. What this does is adds 1 to the value of x over and over again until it reached 49 and then the modulo resets it to 0 so the counting can start again. I used a similar methods for the value of the hue which increased its value making the colour move up the spectrum and then loop back around. I like the effect this gave as its a gradual change in colour rather than just random values which we experimented with before.

Screen Shot 2014-10-11 at 12.17.50

Next up we played around with nested for loops in order to create a grid. This method is great for creating something which looks visually complex but can be written with very few lines of code. In my example below, I again added an element of interactive as the mouses x coordinate influenced the size of the grid and the colours. This meant that depending on the position of the mouse, the grid would be animated with waves of colours passing over it which looked pretty cool.

Screen Shot 2014-10-11 at 12.18.50

The final thing we did in the workshop was to play around with rotation. For this started by creating a rotating square around the centre of the canvas. This then progressed into making the square change colour with the same process i had used before to get the gradual hue changes. In this one the opacity of the square is lower so when it is redrawn over the old shapes, they can still be seen slightly, creating a different effect. We also made the square increase in size using the same modulo statement which resulted in the pattern created constantly evolving and moving. This created a rather psychedelic effect which I like a lot and will try to improve upon in the future.
Screen Shot 2014-10-11 at 12.14.48

For next week we’ve been tasked with playing around with some of the methods we’ve learned about and to make something cool to showcase in the workshop. So the plan for this weekend is to sit in the dark in my room, write a little code and watch some colourful shapes dance around on my screen.

Tagged ,