3D face tracking and multi-scale, spatio-temporal analysis of linguistically significant facial expressions and head positions in ASL

Bo Liu, Jingjing Liu, Xiang Yu, Dimitris Metaxas, Carol Neidle

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Essential grammatical information is conveyed in signed languages by clusters of events involving facial expressions and movements of the head and upper body. This poses a significant challenge for computer-based sign language recognition. Here, we present new methods for the recognition of nonmanual grammatical markers in American Sign Language (ASL) based on: (1) new 3D tracking methods for the estimation of 3D head pose and facial expressions to determine the relevant low-level features; (2) methods for higher-level analysis of component events (raised/lowered eyebrows, periodic head nods and head shakes) used in grammatical markings - with differentiation of temporal phases (onset, core, offset, where appropriate), analysis of their characteristic properties, and extraction of corresponding features; (3) a 2-level learning framework to combine low- and high-level features of differing spatio-temporal scales. This new approach achieves significantly better tracking and recognition results than our previous methods.

Original languageEnglish (US)
Title of host publicationProceedings of the 9th International Conference on Language Resources and Evaluation, LREC 2014
EditorsNicoletta Calzolari, Khalid Choukri, Sara Goggi, Thierry Declerck, Joseph Mariani, Bente Maegaard, Asuncion Moreno, Jan Odijk, Helene Mazo, Stelios Piperidis, Hrafn Loftsson
PublisherEuropean Language Resources Association (ELRA)
Pages4512-4518
Number of pages7
ISBN (Electronic)9782951740884
StatePublished - 2014
Event9th International Conference on Language Resources and Evaluation, LREC 2014 - Reykjavik, Iceland
Duration: May 26 2014May 31 2014

Publication series

NameProceedings of the 9th International Conference on Language Resources and Evaluation, LREC 2014

Other

Other9th International Conference on Language Resources and Evaluation, LREC 2014
Country/TerritoryIceland
CityReykjavik
Period5/26/145/31/14

All Science Journal Classification (ASJC) codes

  • Linguistics and Language
  • Library and Information Sciences
  • Education
  • Language and Linguistics

Keywords

  • ASL
  • Facial expression
  • Head position
  • Nonmanual Marker
  • Spatio-temporal

Fingerprint

Dive into the research topics of '3D face tracking and multi-scale, spatio-temporal analysis of linguistically significant facial expressions and head positions in ASL'. Together they form a unique fingerprint.

Cite this