Advertisement
  1. Code
  2. Media
Code

Automatically Tag Photos with the AS3 Face Recognition Library

by
Languages:

The Face Recognition Library can be an incredible library if used correctly. It has the ability not only to detect faces in an image but also to recognize who the face belongs to in an image. The library's face detector object crops a single face within an image, but in this tutorial you will learn a quick workaround that will allow you to crop multiple faces within an image. We'll create editable tags that a user can adjust after loading an image into the application.


Final Result Preview

Let's take a look at the final result we will be working towards:

Click here to view the demo

Upload an image, and the demo will show you which areas the Face Recognition Library recognises as faces. (It's not always perfect!)

Since this is just a demo, there are a few restrictions: make sure you pick a landscape-oriented photo, and don't pick one with a large filesize. Of course, when you use this library in your own projects, you can work around these restrictions.

Need photos to test this on? Try Flickr!


Prerequisites

Download the FaceRecognitionLib.swc from the library's project page. You will also need the face.zip file from the project's supervision. Navigate to http://code.google.com/p/face-recognition-library-as3/source/browse/trunk/FaceRecognitionLib/src/face.zip. Click View raw file to download it.

You'll need the Flex SDK (version 3.5 or later) and must have Flash Professional CS5 configured to target compilation for Flash Player 10 or later. The last thing you will need is the latest version of TweenLite, which we'll be using to add simple animations to our application.


Step 1: Setting Up Flash

Open Flash CS5 Professional and create a new Actionscript 3.0 file.

new_project

Set the document class name to FaceTagger.

document_class

Click ActionScript Settings in the Properties panel. Click on the Library path tab in the ActionScript Settings window. Browse to the FaceRecognitionLib.swc to add it to the project. Repeat this process for the greensock.swc file.

add_button_component
add_button_component

Step 2: Add Color to the Stage

Click on the Stage. Change the Stage's color to a color of your choice.

add_button_component
color_picker

Step 3: Create the Main Buttons

Open the Components panel. Drag a button onto the top right corner of the stage.

components_panel

Set the instance name of the button to browseBtn in the Properties panel. Within the Component Parameters panel, change the button's label to Browse.

add_button_component

Add another button to the stage directly to the left of browseBtn. Change the label of the button to Add Tag and set its instance name to addBtn.

add_button_component

Step 4: Create the Remove Button

When our application detects a face, a tag that will appear over the cropped face will be generated. It will contain a button that will remove the tag when it is clicked. The tag itself will be drawn entirely with code but we need to draw the button that will remove the tag within manually.

Using the Oval tool (press the O key to switch to the oval tool), add an oval to the stage that is 36 pixels tall and 36 pixels wide (you can alt-click the stage to bring up a dialog for this). The oval should have a stroke of 2.0 and have a White line color and Dark Grey fill color as seen below.

add_button_component

Note: In this image, the layers containing our two buttons are locked and hidden. Do not remove them from the stage.

add_button_component

Using the Line tool (press the N key to switch to the line tool), draw a diagonal line on the stage while holding the shift key to give the line a fixed angle. Paste a copy of the line to the stage. In the top menu of Flash, select Modify, scroll down to Transfrom and select Flip Horizontal. Align the new line on top of the original line to form an X shape.

add_button_component
add_button_component

Drag both lines onto the oval shape. Select the oval shape along with each line. Press F8 to open the Convert to Symbol window. Set the name of the symbol to RemoveButton and the type to Movie Clip. Make sure the registration is set to the top left and that the folder is set to library root. Under the Advanced options, check Export for ActionScript. The Class field should read RemoveButton and the base class should flash.display.MovieClip. Click OK to create the symbol. An instance of the RemoveButton class doesn't need to exist at runtime. We will instantiate it when we need to, using code, so you can remove it from the Stage.

add_button_component

Step 5: Our First Code

If you remember from Step 1, the FaceTagger class is our document class. Most of the magic will happen in this class.

Create a new class and name it FaceTagger. The class should extend the flash.display.Sprite class.

create_new_class
create_face_tagger_class

Let's begin by importing the following classes:

Create the following variables and constants after the class declaration.

We need a TextField object to display messages to the user. The FileReference object allows the user to upload an image from the computer and the FileFilter object limits the file types the user can upload. The Loader object is needed to actually parse the image's data from the FileReference object. The darkBox variable is a Sprite that will be used to darken the screen when a tag is being edited. We'll do this to add contrast to our application. Doing so will let the user know that they are in some kind of Edit mode when the screen is dark and the current tag is in focus. Finally we'll need a Bitmap object to represent the actual image and a MovieClip to contain the bitmap image and all of the image's tags.

MIN_WIDTH and MIN_HEIGHT represent the minimum demensions that the users image should be in order for the face detection process to occur. MAX_WIDTH and MAX_HEIGHT represent the maximum demensions. FILE_TYPES holds the file types that we will allow the user to choose from when they're loading an image into our application.

Let's add some code to the class constructor.

We pass two parameters into the FileFilter contructor. The first parameter is string that contains the description of the filter which in this case is an image. We pass the FILE_TYPES constant within parentheses to display the allowed file types to the user when they are browsing for an image to load. The second parameter is a String that contains the allowed file types. We pass our FILE_TYPES constant into this parameter as will. Lasly, we call the init method which we will create later, to initialize the application.


Step 6: Initializing the Display

Let's create the init method.

We initialize the stage by setting its align property to TOP_LEFT and setting its scaleMode property to NO_SCALE. We call the onStageResize method when the stage is resized. We call the browse method, which will allow the user to browse images to load, when browseBtn has been clicked.

The addTag method is called when a user clicks addBtn. We need to listen for when the user selects a file so that we can load the file into memory. We also listen for when the file has been loaded and when the Loader object has finished parsing the loaded data into an image.

We initialize our textfield and apply a nice drop shadow filter to add depth to our text. Finally we draw graphics to the darkBox object, display the textfield, and we call the positionContents method which we will now create.

The positionContents method is self-explanatory. It positions each element based on the size of the stage. We resize the darkBox object to the size of the stage and only position the image object if it exists.

We want to call the positionContents method every time the stage is resized. Create the method onStageResize to handle this:


Step 7: Loading an Image

In order to look for faces within an image, we need an image in our possession. Let's create the browse method.

We simply call the FileReference object's browse method and this opens a native window which allows the user to browse for an image file. We need to apply our FileFilter object to limit the file types that the user can browse. We pass the filter into the first parameter of the browse method within an Array object to accomplish this.

Now the user can browse for an image to load. We'll have to load the file into Flash when the user selects a file. For this, let's create the onFileSelected method. This method is an event handler function that is called when the user selects a file.

We listened for the SELECTED event in the init method. We also listened for the COMPLETE event on the which is dispatched by the contentLoaderInfo object when the image data has been parsed. The onFileComplete method is called when this event occurs. Let's create this method also.

First thing we want to do once a file has been selected is disable the Browse button so that the user can't load a second file. We display the status to the user and finally we call the load method from the FileReference object. This loads the selected file into memory.

The onFileComplete method simply tells the Loader object to parse the bytes of the loaded file. We previously listened for loader.contentLoaderInfo to dispatch a COMPLETE event. When this event is dispatched the detectFaces method is called. Let's start coding this method.

For now we will only assign a reference to loader.content to our Bitmap object. We use type casting to tell Flash to treat loader.content as a Bitmap object, and then finally we display the image.


Step 8: Handling the Loaded Image

Before we can start the face detection process, we need to make sure that the demensions of the uploaded image are within a range for accurate face detection. If an image is to large it can cause our application too crash when we scan the image for faces. If the image is too tiny, the chances of the FaceDetector object not being able to detect a face within an image that actually has a face increases. We need to set limits here.

Let's create a static method called inRange.

The inRange method returns true if the specified width and height are not out of range. It returns false otherwise. We are going to call this method in the detectFaces method. Remove the line "addChild( bitmap );" from the detectFaces method. Then add the following lines of code.

Also we need to add com.greensock.TweenLite to the class path.

We pass the width and height of the Bitmap object into the parameters of the inRange method. If the inRange method returns false, we instantiate the image object, add bitmap to the image object and display the image underneath the user interface. The image fades in using TweenLite and we display an error message to the user that the image is out of range. Finally, we exit out of the function using a return statement since we don't want the code that will follow the if statement to execute.

Now that we have checked to see if the image is in a safe range for face detection we can now start detecting faces.


Step 9: Using the FaceDetector Class

Include the following classes in the FaceTagger classpath.

  • com.oskarwicha.images.FaceDetection.FaceDetector
  • com.oskarwicha.images.FaceDetection.Events.FaceDetectorEvent

Create a new variable underneath the class declaration named detector. The detector object should be of type FaceDetector. The beginning of the FaceTagger class should now look as follows.

Add the following lines of code to the detectFaces method.

First we add event listeners to the FaceDetector. There are two events that we need to listen for. The FACE_CROPED event and the NO_FACES_DETECTED event.

(Note: FACE_CROPED is not a typo. It is likely a spelling mistake by the author of the library.)

We want to call the onFacesDetected method whenever a face has been detected and the onNoFaces method whenever no faces were detected on the image. The last line of code loads the face image from our bitmap object containing the loaded image. We call the FaceDetector object's loadFaceImageFromBitmap method using the bitmap object as the parameter Bitmap that the method calls for.


Step 10: Detecting Multiple Faces

When the FaceDetector object's objectDetector detects faces in an image, the FaceDetector itself only crops one of those faces. If we want access to all of the detected faces we'll have to directly access the objectDetector. Create the onFacesDetected method and the onNoFaces method containing the following lines of code.

Let's take a look at the onFacesDetected method. The FaceDetector object's objectDetector contains an array of detected objects. These objects are Rectangle objects that each represent the location and size of each cropped face that was successfully found in the image.

We assign it to the faces variable which is an Array. If the image object doesn't exist yet we need to instantiate it. If it exists, we need to exit the function because the code has already executed once. We execute the same code the we did in the detectFaces method to display the image using a fade in animation. Then we clear the textfield.

The for each loop is used to iterate through each Rectangle in the faces array. When traced to the output panel, a detected face will look something like this:

The newTag method generates a new PhotoTag object to display on the image based on the parameter Rectangle object supplied. Before we can add a tag to the display, we'll need to first create our tag class. We'll create this class in the next step.

In the onNoFaces method we repeat the process but this time after we display the image, we notify the user that no faces were detected in the loaded image.


Step 11: Begin the PhotoTag Class

Create a new class named PhotoTag that extends Sprite.

add_button_component

Import the following classes to the PhotoTag class.

Create the following variables and constants.

Add the following code to the class contructor.

The class contructor accepts two parameters:

  • The first parameter is the rectangle object that contains data about the cropped face.
  • The second parameter is the MovieClip that contains the uploaded bitmap image.

We assign these to variables so that we can reference them throughout our code.

On the next line a new instance of the RemoveButton class is created. Use the inherited graphics object to draw a rounded rectangle that is the size of the Rectangle object's width and height. The rectangle's ellipse width and height come from the class constants as well as the rectangle's line attributes. We apply a drop shadow filter to add depth to the object and set the alpha to the constant INIT_ALPHA (the initial alpha). _editMode is a property that tells us whether or not the tag is in an editable state.

The last line in the contructor listens for the ADDED event then calls the onAdded method once the tag has been added to the display list. We do this because we'll need to access the stage object which is currently null. Once the object has been added we can safely access the stage.

The onAdded method is called when the object is added to the display list. The event listener is no longer needed so it is removed and the init method is called.


Step 12: Initializing the Photo Tag

Create the init method in the PhotoTag class.

The first line of code in the init method checks to see whether the tag was actually added to the image object. If it wasn't, the tag will add itself to the image.

Next the tag's x and y are set to the rectangle's x and y. This simply positions the tag over the cropped face. By default, the removeBtn's alpha property is set to 0 because we don't want it to be visible just yet. Later we will need to listen for a DOUBLE_CLICK event and the tag's doubleClickEnabled must be set to true in order for this event to be dispatched.

The last five lines of code add event listeners to the tag, to the stage, and to the removeBtn.

When the removeBtn is clicked we want to remove the tag from the display list. Create the remove method.

The method removes the tag from its parent object, if it has one. We set the _removed property to true since the object is being removed.

The _removed property is a private property but it needs to be accessed outside of the scope of this class. We only want to grant code, outside of the class, access to read the properties data. We also want to grant the same limited access to the _rectangle object, the _image object, and the _editMode property. Create the following getter methods.

Now code outside of this class can access these objects through the getter methods. The methods can be treated as properties, so each property is effectively read only.


Step 13: Adding Life to the PhotoTag Object

If we want our PhotoTag objects to have a little life in them we'll need to add some interaction and animation. Add the following methods to the PhotoTag class.

It may look like a lot of code but what it actually accomplishes is quite simple; all of the methods basically execute the same code.

Earlier we added an event listener for the ROLL_OVER event. We passed an event handler function to the eventHandler parameter called overState. This method adds a simple scale effect when the mouse hovers over the tag. The new width and height of the object is based on how much we want to increase the object's size by. In this case we want to increase the object's size by 40 percent.

We could have stopped right here and simply used TweenLite to animate the object being resized but this wouldn't look very appealing because the object's registration is in the top left corner. What we really want to do is transform the object from its center instead. If you are a Club Greensock member then you could just use the transformAroundCenter plugin to accomplish this with just one line of code and without having to make the calculations yourself, but for the rest of us we'll need a quick workaround.

We calculate the new x position based on the new width and we calculate the new y position based on the new height. Then we tween the x, y, width, and height properties to their new values. We also fade in the removeBtn and make the tag fully opaque.

We do the complete opposite in the outState method which is called everytime the mouse hovers off of the tag. The tag is tweened back to its original state.

The resize method accomplishes what the overState method accomplishes without affecting the alpha property and without any tweening. The resize method is called every time we want to resize the tag dynamically. The method takes one parameter which is the percentage to increase or decrease the tag's size by. The method adds a minimum size limit to the tag so that it does not become to small.

The onMouseWheel method is called whenever a user interacts with the wheel on their mouse. The onKeyDownE method is called everytime a key is pressed. Each method only executes code if the tag is currently in edit mode. And each method resizes the tag based on specific properties. The onMouseWheel method resizes the tag based on the MouseEvent's delta property's value. The onKeyDownE increases the tag's size when the Up key is pressed and decreases the tags size when the Down key is pressed.

Their is one last thing we need to fix in order to make our tag animate correctly. TweenLite changes the value of the width and height properties on each frame until the animation is complete. When you resize objects in Flash, the can become distorted. We don't want our tags looking distorted. To fix this problem we'll need to override the set width and set height methods from the parent class.

Whenever the width or height of the tag is set, we want to redraw the tag's graphics. Create the draw method.

The draw method should accept two parameters: the new width of the tag and the new height of the tag. The graphics are cleared then redrawn again based on the newWidth and newHeight parameters. Now our tag will animate beautifully.


Step 14: Entering and Exiting Edit Mode

We want our users to be able to edit each tag individually. When a tag is in edit mode, the user can manipulate the tag's size and reposition the tag on the image. Several things need be done to accomplish this.

First, let's create the enterEditMode and exitEditMode methods. Both of these methods will be public methods as we will need to call them from the document class.

The enterEditMode method executes the block of code within the if statement if the tag is currently not in edit mode. If the tag is not in edit mode the tag will enter edit mode, setting _editMode to true to allow the user to change the tag's properties. The ROLL_OUT event listener is removed as we don't wan't the method to accidently execute if the mouse hovers off of the tag. Finally, the draw method is called to display the original size of the tag.

The exitEditMode method sets the tag back to its initial state while positioning the tag at the _rectangle's new x and y position. We call the stopDrag method just in case the tag was being dragged by the user when edit mode has been exited.


Step 15: Creating a New Tag

Now let's go back to our Document class, FaceTagger. Finally we can create the newTag method now that we have a PhotoTag class. Create the newTag method.

The method creates a new PhotoTag object passing the rectangle parameter and the current image into the PhotoTag contructor method. The tag is added to the current image. We listen for several events that the tag will dispatch. The first event is the ROLL_OVER event. A method called onTagRollOver is called when the event is dispatched. We listen for the DOUBLE_CLICK event so that we can call the enterEditMode method, which will be different from the enterEditMode method from the PhotoTag class, and finally we listen for the REMOVED event that calls the onTagRemoved method whenever the tag is removed from its parent object.

Create an event handler method named onTagRollOver.

The method is very simple, as you can see. The method displays a message to the user if the tag that the mouse is hovering over is not in edit mode. The message notifies the user that if the double-clicks on the current tag, then the tag will enter edit mode.


Step 16: Granting the User Access to Edit

We now need to allow the user access to edit each tag. Create a new variable underneath the class delclaration of the FaceTagger class and name it currentTag. It should be a PhotoTag object. The beginning of the document class should now look like so:

Create the enterEditMode method within the FaceTagger class. This method is the event handler method that we called upon in the newTag method.

The method above notifies the user that they are currently in edit mode. The screen is darkened and the tag is removed from the image object and placed on the stage. The tag's x and y properties have to be set to values that are relevant to the image's position on the stage. If we didn't do this the tag would just sit in the top left hand corner of the screen. When this method is called, it won't even seem as if the tags position has changed at all but in reality it has changed quite a bit.

Finally we call the tag objects's enterEditMode method, assign a reference to the tag object to the currentTag variable and position the contents of the screen (particularly the statusTxt textfield).


Step 17: Saving the State of the Current Tag

After the user has finished editing the currentTag, the tag needs to be added back to the image object and correctly positioned. Create the exitEditMode method within the FaceTagger class.

Notice how the MouseEvent parameter is set to null by default. We are going to call this method from the onTagRemoved event handler method later and we won't have a MouseEvent object to pass as the method's parameter so we give the parameter a null default value. This makes the method more accessible.

The first task the method performs is hiding the darkBox object to give the user a sense of exiting edit mode. The block of code within the if statement executes if the currentTag exists. The textfield is set to a single space (" "). Remember that the textfield has its autoSize property set to resize from its center. To get the textfield to position itself correctly, we need to make sure that there is some kind of text present so that the textfield doesn't resize itself down to nothing. This can cause a potential problem if the user exits edit mode then hovers over a tag. The effect caused is that the textfield may appear off screen.

Next, the currentTag's rectangle object is given new x and y property values. These values are based on the tag's current position on the stage relative to the image's position on the stage. When the currentTag's exitEditMode method is called the currentTag is added back to the image object and positioned to the rectangle's new x and y position.

Last, we set the reference of the currentTag to null and position the contents on the screen.

Create the onTagRemoved method.

This method exits edit mode if the remove method of the targeted tag (e.target) has been called. Thus if the user has clicked the remove button on the targeted tag, we exit editMode. Without this method, if the user removed a tag while in edit mode, the screen would remain dark.


Step 18: Repositioning a Tag

Return to the PhotoTag class and add the following methods to the class.

Both methods require that the tag be in edit mode to execute completely. The onMouseUp method starts dragging the tag and limits the tag to only drag within the _image object. The onMouseUp method stops the dragging process.


Step 19: Adding Additional Tags

By now you're probably wondering what happened to addBtn. When this button is clicked a new PhotoTag object is added to the current image. Let's create the addTag method in the document class.

Note: You should remember this method from Step 6. It is the method that was passed as the eventhandler parameter when we added an event listener that listens for a CLICK event to be dispatched from the addBtn object.

If image exists, we call the newTag method as before. We create a new made up Rectangle object so that the user can customize the tag on their own. The tag is simply added to the image in the top left corner.


Step 20: Improving the Appearence of the RemoveButton Object

As we have seen, you can dramatically improve an object's appearance with a few lines of code. Let's improve the look of the RemoveButton object. Create a new class and name it RemoveButton. The class will have to extend the MovieClip class since it is linked to a library symbol of that type.

add_button_component

Save the class to the root folder. Import the following within RemoveButton:

All we're going to do now is add a simple line of code within the RemoveButton object's class constructor method.

We have just added a DropShadowFilter and a BevelFilter to the RemoveButton object. This will add a great deal of depth to our button.


Step 21: Testing Our Application

At this point, if we test our application, everything should run smoothly. But we will encounter a problem when we upload an image and the face detection process begins. If you do not put the face.zip file in the same directory as the SWF, the face detection process can not occur. It is very important to remember to do this. Once you have placed face.zip into the same directory as the SWF, you may then run the application.

final_result

Conclusion

Please take into consideration that the Face Recognition Library isn't perfect. There will be times where it won't detect any faces even though there are faces within the image that was uploaded. There will also be times where only some faces in an image are detected and times where patterns that look like faces (e.g., A cloud shaped like a face) will be detected as faces. This is why we allow for our users to edit tags. Generally, I found the library pretty amazing and I hope you will too. This concludes my tutorial. Thank you for reading.

Advertisement
Advertisement
Looking for something to help kick start your next project?
Envato Market has a range of items for sale to help get you started.