Nonlinear dynamic shape and appearance models for facial motion tracking

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We present a framework for tracking large facial deformations using nonlinear dynamic shape and appearance model based upon local motion estimation. Local facial deformation estimation based on a given single template fails to track large facial deformations due to significant appearance variations. A nonlinear generative model that uses low dimensional manifold representation provides adaptive facial appearance templates depending upon the movement of the facial motion state and the expression type. The proposed model provides a generative model for Bayesian tracking of facial motions using particle filtering with simultaneous estimation of the expression type. We estimate the geometric transformation and the global deformation using the generative model. The appearance templates from the global model then estimate local deformation based on thin-plate spline parameters.

Original languageEnglish (US)
Title of host publicationAdvances in Image and Video Technology - Second Pacific Rim Symposium, PSIVT 2007, Proceedings
Pages205-220
Number of pages16
StatePublished - Dec 1 2007
Event2nd IEEE Pacific Rim Symposium on Video and Image Technology, PSIVT 2007 - Santiago, Chile
Duration: Dec 17 2007Dec 19 2007

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume4872 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other2nd IEEE Pacific Rim Symposium on Video and Image Technology, PSIVT 2007
CountryChile
CitySantiago
Period12/17/0712/19/07

Fingerprint

Motion Tracking
Nonlinear Dynamics
Generative Models
Template
Geometric transformation
Thin-plate Spline
Simultaneous Estimation
Particle Filtering
Model
Motion
Motion Estimation
Estimate
Motion estimation
Nonlinear Model
Splines
Model-based

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Keywords

  • Active appearance model
  • Adaptive template
  • Facial expression recognition
  • Facial motion tracking
  • Local facial motion
  • Nonlinear shape and appearance models
  • Thin-plate spline

Cite this

Lee, C. S., Elgammal, A., & Metaxas, D. (2007). Nonlinear dynamic shape and appearance models for facial motion tracking. In Advances in Image and Video Technology - Second Pacific Rim Symposium, PSIVT 2007, Proceedings (pp. 205-220). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4872 LNCS).
Lee, Chan Su ; Elgammal, Ahmed ; Metaxas, Dimitri. / Nonlinear dynamic shape and appearance models for facial motion tracking. Advances in Image and Video Technology - Second Pacific Rim Symposium, PSIVT 2007, Proceedings. 2007. pp. 205-220 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{d972e7e163a449658bb26facf9a4c57f,
title = "Nonlinear dynamic shape and appearance models for facial motion tracking",
abstract = "We present a framework for tracking large facial deformations using nonlinear dynamic shape and appearance model based upon local motion estimation. Local facial deformation estimation based on a given single template fails to track large facial deformations due to significant appearance variations. A nonlinear generative model that uses low dimensional manifold representation provides adaptive facial appearance templates depending upon the movement of the facial motion state and the expression type. The proposed model provides a generative model for Bayesian tracking of facial motions using particle filtering with simultaneous estimation of the expression type. We estimate the geometric transformation and the global deformation using the generative model. The appearance templates from the global model then estimate local deformation based on thin-plate spline parameters.",
keywords = "Active appearance model, Adaptive template, Facial expression recognition, Facial motion tracking, Local facial motion, Nonlinear shape and appearance models, Thin-plate spline",
author = "Lee, {Chan Su} and Ahmed Elgammal and Dimitri Metaxas",
year = "2007",
month = "12",
day = "1",
language = "English (US)",
isbn = "9783540771289",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "205--220",
booktitle = "Advances in Image and Video Technology - Second Pacific Rim Symposium, PSIVT 2007, Proceedings",

}

Lee, CS, Elgammal, A & Metaxas, D 2007, Nonlinear dynamic shape and appearance models for facial motion tracking. in Advances in Image and Video Technology - Second Pacific Rim Symposium, PSIVT 2007, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4872 LNCS, pp. 205-220, 2nd IEEE Pacific Rim Symposium on Video and Image Technology, PSIVT 2007, Santiago, Chile, 12/17/07.

Nonlinear dynamic shape and appearance models for facial motion tracking. / Lee, Chan Su; Elgammal, Ahmed; Metaxas, Dimitri.

Advances in Image and Video Technology - Second Pacific Rim Symposium, PSIVT 2007, Proceedings. 2007. p. 205-220 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4872 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Nonlinear dynamic shape and appearance models for facial motion tracking

AU - Lee, Chan Su

AU - Elgammal, Ahmed

AU - Metaxas, Dimitri

PY - 2007/12/1

Y1 - 2007/12/1

N2 - We present a framework for tracking large facial deformations using nonlinear dynamic shape and appearance model based upon local motion estimation. Local facial deformation estimation based on a given single template fails to track large facial deformations due to significant appearance variations. A nonlinear generative model that uses low dimensional manifold representation provides adaptive facial appearance templates depending upon the movement of the facial motion state and the expression type. The proposed model provides a generative model for Bayesian tracking of facial motions using particle filtering with simultaneous estimation of the expression type. We estimate the geometric transformation and the global deformation using the generative model. The appearance templates from the global model then estimate local deformation based on thin-plate spline parameters.

AB - We present a framework for tracking large facial deformations using nonlinear dynamic shape and appearance model based upon local motion estimation. Local facial deformation estimation based on a given single template fails to track large facial deformations due to significant appearance variations. A nonlinear generative model that uses low dimensional manifold representation provides adaptive facial appearance templates depending upon the movement of the facial motion state and the expression type. The proposed model provides a generative model for Bayesian tracking of facial motions using particle filtering with simultaneous estimation of the expression type. We estimate the geometric transformation and the global deformation using the generative model. The appearance templates from the global model then estimate local deformation based on thin-plate spline parameters.

KW - Active appearance model

KW - Adaptive template

KW - Facial expression recognition

KW - Facial motion tracking

KW - Local facial motion

KW - Nonlinear shape and appearance models

KW - Thin-plate spline

UR - http://www.scopus.com/inward/record.url?scp=38349033358&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=38349033358&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:38349033358

SN - 9783540771289

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 205

EP - 220

BT - Advances in Image and Video Technology - Second Pacific Rim Symposium, PSIVT 2007, Proceedings

ER -

Lee CS, Elgammal A, Metaxas D. Nonlinear dynamic shape and appearance models for facial motion tracking. In Advances in Image and Video Technology - Second Pacific Rim Symposium, PSIVT 2007, Proceedings. 2007. p. 205-220. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).