In the first part of this series, I introduced a class to handle single mouse stroke detection: MGesture
. This tutorial takes it a step further, by detecting a sequence of strokes.
We'll be using the class GManager
for this, in tandem with MGesture
. After briefing through GManager
functionality, an application demonstrating its use will be developed.
Final Result Preview
Let's take a look at the final result we will be working towards. To use the Flash presentation below, use your mouse to perform the gesture as indicated by the arrow in the top left corner. Gesture by pressing the left mouse button, moving the mouse while holding the button, and then releasing it.
Step 1: Recap Direction Numbering

First step, a revision on integers used to represent different directions from Part 1. Hopefully, you can form a mental picture of the diagram above as we shall refer to it heavily throughout this tutorial.
Step 2: Strokes Detected
As user makes a gesture by any pointing devices (mouse, tablet pen, etc), the vectors successfully detected over time actually form a series of integers. Refer to diagram below.

Notice the unique integers in the sequence are 0, 4, 1. However, I shall say unique integers are 0, 1. Lets face it: it's difficult to make an accurate stroke on my Bamboo tablet, and even more so with a mouse. Errors are inevitable. Our algorithm should then be based upon the high repetition of integers in a continuous string. They are 0, 1 here - 4 is probably due to inaccurate stroke. Therefore, this gesture can be uniquely identified as a sequence of 0, then 1.
I've included another few gestures with their accompanying unique sequences below.

Step 3: Introducing GManager
The purpose of this class is simple:
- To register unique stroke sequences to compare against.
- To capture sequence detected of current gesture made by user.
- To compare sequence in (2) against all stroke sequences in (1).
- To return search result.
Developer has to define a set of stroke sequences to detect initially. Once done, event listeners to capture user gesture at runtime can be programmed. If the current gesture's sequence successfully matches any one sequence in the predefined set, different operations can then be carried out.
Step 4: Variables
GManager
has the following variables:
Variable | Datatype | Purpose |
gestSeq |
Vector.<Vector.<int>> |
2D array to record different stroke sequences. |
gestName |
Vector.<String> |
1D array to record names of stroke sequences. |
strokes |
Vector.<int> |
1D array to record restriction on gestCurrent 's strokes when compared with stroke sequence. |
gestCurrent |
Vector.<int> |
1D array to record current stroke sequence. |
_order |
Boolean |
Determines whether gesture sets in gestSeq should be detected in order. |
orderCurrent |
int |
Current sequence to detect whether _order is turned on; starts at 0. |
Step 5: Methods
Methods in GManager
are as below:
Method | Input | Output | Description |
GManager |
void | void | Class initiation, gestSeq and gestName, initialised |
register |
Vector.<int>, String, int |
void | Register stroke sequence, name and stroke restriction (optional). |
remove |
int |
void | Remove selected stroke sequence |
removeAll |
void | void | Remove all registered stroke sequences |
start |
void | void | Prepare variable to record current stroke sequence, gestCurrent
|
populate |
int |
void | Populate gestCurrent with detected strokes (singular) |
tracer |
void | Vector.<int> |
Traces and outputs gestCurrent for debugging purposes. |
dropStrokes |
Vector.<int>, int |
Vector.<int> |
Eliminate unnecesary strokes in gestCurrent . Keep mains and diagonals (0), keep only mains (1), keep only diagonals (2). (Part 3 will further explain its use.) |
dropDuplicates |
Vector.<int>, int |
Vector.<int> |
Identify valid unique integer in current sequence and drop duplicates. Input minimum duplicates (apart from self) to be considered valid stroke |
checkMatch |
int |
int |
Checks if member of gestSeq matches with gestCurrent
|
end |
void | Array |
Returns result of search. Result is Array with index of matchedsequence in gestSeq and its name in gestName
|
Properties of GManager
are as below:
Property | Accessor Types | Purpose |
useOrder |
Getter/ Setter | Gets and sets whether gestSeq is to be detected in order |
length |
Getter only | Gets the length of gestSeq
|
Step 6: Register Predefined Stroke Sequences
As mentioned, the first step is to register unique stroke sequences that the entered gesture can be compared against. The constructor initates variables needed (highlighted) and methods that follow registers into and removes from predefined stroke sequences.
/** * Constructor to initiate gestSeq, stokes, gestName */ public function GManager(){ gestSeq = new Vector.<Vector.<int>>; gestName = new Vector.<String>; strokes = new Vector.<int>; } /** * Method to register a set/ sequence of gesture * @param sequence Numerical representation of gesture sequence * @param identifier Name representation of gesture * @param allowed Stokes allowed: Both (0), main four (1), diagonals (2) */ public function register(sequence:Vector.<int>, name:String, onlyStrokes:int = 0):void { gestSeq.push(sequence); gestName.push(name); strokes.push(onlyStrokes); } /** * Method to remove a specific set of gesture * @param selectSet Selected sequence to remove */ public function remove(selectSet:int):void { gestSeq.splice(selectSet, 1); gestName.splice(selectSet, 1); strokes.splice(selectSet, 1); } /** * Method to remove all predefined gestures. */ public function removeAll():void { gestSeq = new Vector.<Vector.<int>>; gestName = new Vector.<String>; strokes = new Vector.<int>; }
Step 7: Properties
Below are the properties specifically for set holding stroke seqeunces, gestSeq
:
/** * Property to get total sets of predefined gestures */ public function get length ():int { return gestSeq.length; } /** * Property to get current setting on sequential gesture detection */ public function get useOrder():Boolean { return _order; } /** * Property to determine whether sets of predefined gestures should be detected sequentially */ public function set useOrder(value:Boolean):void { _order = value; }
Step 8: Record Current Stroke Sequence
After registering stroke sequences to detect against, we can prepare gestCurrent
and record consequtive valid singular strokes into it. Methods below will provision for it. Tracer
is a method to trace the current gesture's sequence.
/** * Method to start recording current set of gesture made */ public function start():void { gestCurrent = new Vector.<int>; } /** * Method to populate gesture into current set of gesture * @param gestureCode Gesture detected from MGesture */ public function populate (gestureCode:int):void { //Accept only valid singular gesture (0~8) into current sequence if(gestureCode > -1) gestCurrent.push(gestureCode); } /** * Method used for debugging. Traces gestCurrent */ public function tracer():Vector.<int> { return gestCurrent; trace(gestCurrent); }
Step 9: Looking into gestCurrent

By now, we can already write a program to check out current gesture's sequence as mentioned in Step 2. You may download the source and look at the class CheckOut2
. Turn on your Output panel in FlashDevelop to view the sequence of integers committed by your current gesture.
I've also included a demo of it below and placed a TextField
to display the sequence instead of the Output panel in FlashDevelop.
Step 10: Implementation of CheckOut2
I have included source code of checkOut2
as below. Its not tough to understand the logic provided you have understood how event handlers are assigned. (Read this tutorial to find out more about event handlers.) I've also highlighted important points where GManager
's methods are being used.
public class CheckOut2 extends Sprite { private var ges:MGesture; //instance of singular gesture detection private var gesMan:GManager; //instance of sequence gesture detectio private var t:TextField; public function CheckOut2() { //initiating MGesture to allow singular gesture detection ges = new MGesture(stage); //initiating GManager to allow sequential gesture detection gesMan = new GManager(); t = new TextField() t.x = 50; t.y = 50; t.autoSize = TextFieldAutoSize.LEFT; addChild(t); stage.addEventListener(MouseEvent.MOUSE_DOWN, start); stage.addEventListener(MouseEvent.MOUSE_UP, end); } private function start(e:MouseEvent):void { //Start detecting singular and sequential gesture ges.start(); gesMan.start(); stage.addEventListener(MouseEvent.MOUSE_MOVE, check); //Start drawing gesture graphics.clear(); graphics.lineStyle(3); graphics.moveTo(mouseX, mouseY); } private function check(e:MouseEvent):void { gesMan.populate(ges.evalDirections()); //Feed singular gesture into a sequence graphics.lineTo(mouseX, mouseY) //Drawing gesture } private function end(e:MouseEvent):void { stage.removeEventListener(MouseEvent.MOUSE_MOVE, check); t.text = gesMan.tracer().toString() } }
Step 11: Clean Up gestCurrent
of Unnecessary Strokes
Cleaning up strokes in current gesture is important, as there are:
- Strokes that make difficult detection of gestures.
- Strokes that are duplicated
- Strokes that are invalid
I have divided the clean-up job into two phases, specifically dropStrokes
and dropDuplicates
. dropStrokes
eliminates strokes that made it difficult to detect of gestures. (This feature will be explained in detail in Part 3 when we detect alphabetical letters.) dropDuplicates
eliminates duplicates and invalid strokes.
dropStrokes
will only keep strokes in gestCurrent
according to the third paramenter, onlyStrokes
, in register
. It is defined when we register the gesture sequences initially. For further understanding, again, watch out for Part 3. But for now, just keep it at the back of your head. In fact, this tutorial does not make use of this feature just yet.
/** * Method to rid irrelevant strokes based upon condition * @param strokesCondition * @return Vector of */ private function dropStrokes(inputArray:Vector.<int>, strokesCondition:int):Vector.<int> { var xDiagonals:Vector.<int> = inputArray.slice(0,inputArray.length); //Keep mains only if (strokesCondition == 1) { for (var i:int = 0; i < xDiagonals.length; i++) { if (xDiagonals[i] > 3) { xDiagonals.splice(i, 1); i-- } } } else if (strokesCondition == 2) { for (var j:int = 0; j < xDiagonals.length; j++) { if (xDiagonals[j] <4) { xDiagonals.splice(j, 1); j-- } } } return xDiagonals }
Step 12: Clean Up gestCurrent
Swipe left-to-right to go to the next frame.
The second phase of clean up is to rid the duplicate and invalid strokes as mentioned in Step 2. The implementation is written below.
I've placed a Flash presentation above to ease your understanding of the algorithm. Make a right gesture using mouse to go to next frame, make a left to go back to previous frame. Up gesture brings you to the final frame, down gesture brings you to the first frame.
/** * Method to rid duplicates and possible mistaken gestures * @param minDuplicates Repetition to be considered high repetition * @return Vector array w/o duplicates and possible mistaken gestures */ private function dropDuplicates (inputArray:Vector.<int>, minDuplicates:int = 1):Vector.<int> { //Append end of line indicator to integer array var xDuplicates:Vector.<int> = inputArray.slice(0,inputArray.length); xDuplicates.push( -1); //Initiate variables var count:int = 1; var keepIndex:int = 0; //Ripping duplicates and invalid moves for (var i:int = 0; i < xDuplicates.length - 1; i++) { if (xDuplicates[i] == xDuplicates[i + 1]) count ++ else{ if (count > minDuplicates) { xDuplicates.splice(keepIndex, count - 1) keepIndex++ i = keepIndex count = 1 } else xDuplicates.splice(keepIndex, minDuplicates) i-- } } xDuplicates.splice(xDuplicates.length - 1, 1); return xDuplicates; }
Step 13: Check for a Match
After trimming down gesCurrent
, our next task is to find a proper match in the set of predefined gesture sequences. We can verify match between current gesture and a member of predefined gestures using checkMatch
. Input the index of gestSeq
you would like to check match with gestCurrent
. A return value of -1
indicates no match; the index of gestSeq
is matched otherwise.
/** * Method to check if gestCurrent matches with gestSeq * @param index Member of gestSeq * @return Match (1) or fail (-1) */ private function checkMatch(index:int):int { //Pessimistic search, assumes not found initially var matched:int = -1; //clean input first var cleaned:Vector.<int> = dropStrokes(gestCurrent, strokes[index]); trace(cleaned, " xstrokes"); cleaned = dropDuplicates(cleaned);trace(cleaned," xduplicates") //Check only those of same length if (cleaned.length == gestSeq[index].length) { var counter:int = cleaned.length; //Scroll through each integer of selected predefined gesture for (var member:int = 0; member < cleaned.length; member++) { if (cleaned[member] == gestSeq[index][member]) counter--; else break; } //if all integers matched, current index of gestSeq matches gestCurrent if (counter == 0 ) matched = index; } return matched; }
Step 14: Output the Result
Finally, we can output the result. The result will be based upon whether _order
is turned on or not. We check match between gestCurrent
with current gestSeq
only if _order
is turned on, otherwise we will need to search for match with any member in gestSeq
.
Regardless of the search outcome, we need to output the result. Search result will be -1
if there's no successful match; if there's a match, an array of stroke index coupled with its name will be returned. The result is -2
if gestSeq
is empty.
/** * Method to evaluate gesture according to preferred approach (in sequential order or not) * @return No predefined gestures (-2), No match of gesture (-1), array with index of match and its description */ public function end():Array { //Pessimistic search, -2 indicates that there are no predefined gesture var result:Array = [ -2, "No predefined gestures"]; //If there are predefined gestures if (gestSeq.length > 0) { //Find the match in order if (_order) { if (checkMatch(orderCurrent) > -1) { result = [orderCurrent, gestName[orderCurrent]]; orderCurrent++; } } //Find a match in no specific order else if (!_order) { //Scroll through all gestures for (var i:int = 0; i < gestSeq.length; i++) { if (checkMatch(i) > -1) { result = [i, gestName[i]]; } } } //Handle when cant find else { result = [ -1, "Gesture is invalid."] } } return result }
Step 15: Prepare Graphical Assets in Flash
With Step 14 we completed GManager
. We need to create graphical assets next. I have created the graphical assets in Flash Pro and will export them in .swc format. Press Ctrl + Shift + F12 to pop the Publish Setting window out in Flash and check "Export SWC" under Flash tab. I've made my assets available for download but feel free to develop your own.




Step 16: Importing Assets Into FlashDevelop

Bring your SWC into FlashDevelop's lib
folder as shown above. Create a class named "Main2" in FlashDevelop and instantiate the MovieClip that holds the graphics. I've highlighted the relevant ActionScript below.
private var ges:MGesture; private var gesMan:GManager; private var a:Arrows; private var seq:Boolean = false; public function Main2() { //Importing graphics in form of movieclip a = new Arrows(); a.stop(); a.x = 50; a.y = 50; addChild(a);
Step 17: Register Corresponding Gestures
We shall register our gesture sequences in the order that graphics are arranged in our published MovieClip. We do so to make easy scrolling of frames once the result has been attained. Note as well that I have highlighted line 33 of the implementation; you may uncomment this to in my source file to see matching done in order.
public function Main2() { //Importing graphics in form of movieclip a = new Arrows(); a.stop(); a.x = 50; a.y = 50; addChild(a); //initiating MGesture to allow singular gesture detection ges = new MGesture(stage); //initiating GManager to allow sequential gesture detection gesMan = new GManager(); //this.seq = true; //Detecting gestures in sequence if (seq) { a.gotoAndStop(2); gesMan.useSeq = true; } //Register gesture sequences in line with frames //Register gesture sequences in line with frames gesMan.register(new < int > [2, 1], "2 Stroke Down, CCW"); gesMan.register(new < int > [1, 0], "2 Stroke Right, CCW"); gesMan.register(new < int > [0, 3], "2 Stroke Up, CCW"); gesMan.register(new < int > [3, 2], "2 Stroke Left, CCW"); gesMan.register(new < int > [3, 2, 1], "3 Stroke Down, CCW"); gesMan.register(new < int > [2, 1, 0], "3 Stroke Right, CCW"); gesMan.register(new < int > [1, 0, 3], "3 Stroke Up, CCW"); gesMan.register(new < int > [0, 3, 2], "3 Stroke Left, CCW"); gesMan.register(new < int > [3, 0], "2 Stroke Right, CW"); gesMan.register(new < int > [0, 1], "2 Stroke Down, CW"); gesMan.register(new < int > [1, 2], "2 Stroke Left, CW"); gesMan.register(new < int > [2, 3], "2 Stroke Up, CW"); gesMan.register(new < int > [2, 3, 0], "3 Stroke Right, CW"); gesMan.register(new < int > [3, 0, 1], "3 Stroke Down, CW"); gesMan.register(new < int > [0, 1, 2], "3 Stroke Left, CW"); gesMan.register(new < int > [1, 2, 3], "3 Stroke Up, CW"); gesMan.register(new < int > [0, 3, 0, 3, 0],"Step Up"); gesMan.register(new < int > [0, 1, 0, 1, 0], "Step Down"); gesMan.register(new < int > [4], "South East"); gesMan.register(new < int > [5], "South West"); gesMan.register(new < int > [6], "North West"); gesMan.register(new < int > [7], "North East"); gesMan.register(new < int > [4, 5, 4, 5], "Zigzag"); stage.addEventListener(MouseEvent.MOUSE_DOWN, start); stage.addEventListener(MouseEvent.MOUSE_UP, end); }
Step 18: Handling Results
Recording of gesture sequence follows the same approach as CheckOut2
. I assume readers have gone through Step 10. So finally, we shall output the result. Upon successful match (> -1), we shall scroll to the appropriate frame. Again, this is subject to whether you have enabled the use of sequence by turning on seq
or not.
private function end(e:MouseEvent):void { stage.removeEventListener(MouseEvent.MOUSE_MOVE, check); //evaluate gesture and output result var output:Array = gesMan.end() if (output[0] > -1) { if (seq)a.nextFrame(); else a.gotoAndStop(output[0] + 2); //offset needed as first frame is empty. } }
Step 19: Publish Your Project
Press Ctrl + Enter to publish your project. Gesture with your mouse and what was detected. I've included both projects (one requests you to match a given gesture; the other displays the gesture you drew, if it matches one it knows). Have fun.
Step 20: Application Development
Inaccuracies in user gestures are common, and any application that incorporates gestures should consider this. In the next part of this tutorial, I shall attempt to use MGesture
and GManager
to develop an alphabet recognition application. I shall point out several details to tweak accordingly to improve gesture detection.
Conclusion
You should now find it easy to develop applications that detect gesture sequences of any combination. Hope this tutorial has helped you in some ways. Let me know of your comments, queries and bugs encountered. Terima kasih.
Subscribe below and we’ll send you a weekly email summary of all new Code tutorials. Never miss out on learning about the next big thing.
Update me weeklyEnvato Tuts+ tutorials are translated into other languages by our community members—you can be involved too!
Translate this post