Recognizing eyebrow and periodic head gestures using CRFs for non-manual grammatical marker detection in ASL

Jingjing Liu, Bo Liu, Shaoting Zhang, Fei Yang, Peng Yang, Dimitris N. Metaxas, Carol Neidle

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Scopus citations

Abstract

Changes in eyebrow configuration, in combination with head gestures and other facial expressions, are used to signal essential grammatical information in signed languages. Motivated by the goal of improving the detection of non-manual grammatical markings in American Sign Language (ASL), we introduce a 2-level CRF method for recognition of the components of eyebrow and periodic head gestures, differentiating the linguistically significant domain (core) from transitional movements (which we refer to as the onset and offset). We use a robust face tracker and 3D warping to extract and combine the geometric and appearance features, as well as a feature selection method to further improve the recognition accuracy. For the second level of the CRFs, linguistic annotations were used as training for partitioning of the gestures, to separate the onset and offset. This partitioning is essential to recognition of the linguistically significant domains (in between). We then use the recognition of onset, core, and offset of these gestures together with the lower level features to detect non-manual grammatical markers in ASL.

Original languageEnglish (US)
Title of host publication2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013
DOIs
StatePublished - 2013
Event2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013 - Shanghai, China
Duration: Apr 22 2013Apr 26 2013

Publication series

Name2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013

Other

Other2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013
CountryChina
CityShanghai
Period4/22/134/26/13

All Science Journal Classification (ASJC) codes

  • Computer Vision and Pattern Recognition

Fingerprint Dive into the research topics of 'Recognizing eyebrow and periodic head gestures using CRFs for non-manual grammatical marker detection in ASL'. Together they form a unique fingerprint.

Cite this