Proceedings: GI 2010

A lightweight multistroke recognizer for user interface prototypes

Lisa Anthony, Jacob Wobbrock

Proceedings of Graphics Interface 2010: Ottawa, Ontario, Canada, 31 May - 2 June 2010, 245-252

  • BibTex

    author = {Anthony, Lisa and Wobbrock, Jacob},
    title = {A lightweight multistroke recognizer for user interface prototypes},
    booktitle = {Proceedings of Graphics Interface 2010},
    series = {GI 2010},
    year = {2010},
    issn = {0713-5424},
    isbn = {978-1-56881-712-5},
    location = {Ottawa, Ontario, Canada},
    pages = {245--252},
    numpages = {8},
    publisher = {Canadian Human-Computer Communications Society},
    address = {Toronto, Ontario, Canada},


With the expansion of pen- and touch-based computing, new user interface prototypes may incorporate stroke gestures. Many gestures comprise multiple strokes, but building state-of-the-art multistroke gesture recognizers is nontrivial and time-consuming. Luckily, user interface prototypes often do not require state-of-the-art recognizers that are general and maintainable, due to the simpler nature of most user interface gestures. To enable easy incorporation of multistroke recognition in user interface prototypes, we present $N, a lightweight, concise multistroke recognizer that uses only simple geometry and trigonometry. A full pseudocode listing is given as an appendix. $N is a significant extension to the $1 unistroke recognizer, which has seen quick uptake in prototypes but has key limitations. $N goes further by (1) recognizing gestures comprising multiple strokes, (2) automatically generalizing from one multistroke to all possible multistrokes using alternative stroke orders and directions, (3) recognizing one-dimensional gestures such as lines, and (4) providing bounded rotation invariance. In addition, $N uses two speed optimizations, one with start angles that saves 79.1% of comparisons and increases accuracy 1.3%. The other, which is optional, compares multistroke templates and candidates only if they have the same number of strokes, reducing comparisons further to 89.5% and increasing accuracy another 1.7%. These results are taken from our study of algebra symbols entered in situ by middle and high schoolers using a math tutor prototype, on which $N was 96.6% accurate with 15 templates.