Optical flow constraints on deformable models with applications to face tracking

Douglas Decarlo, Dimitris Metaxas

Research output: Contribution to journalArticlepeer-review

240 Scopus citations

Abstract

Optical flow provides a constraint on the motion of a deformable model. We derive and solve a dynamic system incorporating flow as a hard constraint, producing a model-based least-squares optical flow solution. Our solution also ensures the constraint remains satisfied when combined with edge information, which helps combat tracking error accumulation. Constraint enforcement can be relaxed using a Kalman filter, which permits controlled constraint violations based on the noise present in the optical flow information, and enables optical flow and edge information to be combined more robustly and efficiently. We apply this framework to the estimation of face shape and motion using a 3D deformable face model. This model uses a small number of parameters to describe a rich variety of face shapes and facial expressions. We present experiments in extracting the shape and motion of a face from image sequences which validate the accuracy of the method. They also demonstrate that our treatment of optical flow as a hard constraint, as well as our use of a Kalman filter to reconcile these constraints with the uncertainty in the optical flow, are vital for improving the performance of our system.

Original languageEnglish (US)
Pages (from-to)99-127
Number of pages29
JournalInternational Journal of Computer Vision
Volume38
Issue number2
DOIs
StatePublished - Jul 1 2000
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Optical flow constraints on deformable models with applications to face tracking'. Together they form a unique fingerprint.

Cite this