Recognizing facial expressions by tracking feature shapes

Atul Kanaujia, Dimitris Metaxas

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Scopus citations


Reliable facial expression recognition by machine is still a challenging task. We propose a framework to recognise various expressions by tracking facial features. Our method uses localized active shape models to track feature points in the subspace obtained from localized Non-negative Matrix Factorization. The tracked feature points are used to train conditional model for recognising prototypic expressions like Anger, Disgust, Fear, Joy, Surprise and Sadness. We formulate the task as a sequence labelling problem and use Conditional Random Fields(CRF) to probabilistically predict expressions. In CRF, the distribution is conditioned on the entire sequence rather than a single observation. For the joint probability defined for the entire sequence, CRF does global normalization of the exponential model, as opposed to MEMM, for which the per state exponential distribution is locally normalized. Unlike generative models(HMM), no prior dependencies between the features are assumed. We adopt a simplistic approach to classify expressions without explicitly monitoring the change in shapes of the individual facial features. Instead, we allow CRF to learn the complex dependencies between the features and recognize the expressions directly. Experimental results demonstrate that accurately tracked feature shapes provide reliable discriminative cues to robustly recognize facial expressions for an image sequence.

Original languageEnglish (US)
Title of host publicationProceedings - 18th International Conference on Pattern Recognition, ICPR 2006
Number of pages6
StatePublished - 2006
Event18th International Conference on Pattern Recognition, ICPR 2006 - Hong Kong, China
Duration: Aug 20 2006Aug 24 2006

Publication series

NameProceedings - International Conference on Pattern Recognition
ISSN (Print)1051-4651


Other18th International Conference on Pattern Recognition, ICPR 2006
CityHong Kong

All Science Journal Classification (ASJC) codes

  • Computer Vision and Pattern Recognition


Dive into the research topics of 'Recognizing facial expressions by tracking feature shapes'. Together they form a unique fingerprint.

Cite this