Toward Personalized Modeling: Incremental and Ensemble Alignment for Sequential Faces in the Wild

Xi Peng, Shaoting Zhang, Yang Yu, Dimitris N. Metaxas

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


Fitting facial landmarks on unconstrained videos is a challenging task with broad applications. Both generic and joint alignment methods have been proposed with varying degrees of success. However, many generic methods are heavily sensitive to initializations and usually rely on offline-trained static models, which limit their performance on sequential images with extensive variations. On the other hand, joint methods are restricted to offline applications, since they require all frames to conduct batch alignment. To address these limitations, we propose to exploit incremental learning for personalized ensemble alignment. We sample multiple initial shapes to achieve image congealing within one frame, which enables us to incrementally conduct ensemble alignment by group-sparse regularized rank minimization. At the same time, incremental subspace adaptation is performed to achieve personalized modeling in a unified framework. To alleviate the drifting issue, we leverage a very efficient fitting evaluation network to pick out well-aligned faces for robust incremental learning. Extensive experiments on both controlled and unconstrained datasets have validated our approach in different aspects and demonstrated its superior performance compared with state of the arts in terms of fitting accuracy and efficiency.

Original languageEnglish (US)
Pages (from-to)184-197
Number of pages14
JournalInternational Journal of Computer Vision
Issue number2-4
StatePublished - Apr 1 2018

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence


  • Ensemble learning
  • Face alignment
  • Incremental learning
  • Personalized modeling
  • Sparse coding


Dive into the research topics of 'Toward Personalized Modeling: Incremental and Ensemble Alignment for Sequential Faces in the Wild'. Together they form a unique fingerprint.

Cite this