Sequential Face Alignment via Person-Specific Modeling in the Wild

Xi Peng, Junzhou Huang, Dimitris N. Metaxas

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Sequential face alignment, in essence, deals with nonrigid deformation that changes over time. Although numerous methods have been proposed to show impressive success on still images, many of them still suffer from limited performance when it comes to sequential alignment in wild scenarios, e.g., involving large pose/expression variations and partial occlusions. The underlying reason is that they usually perform sequential alignment by independently applying models trained offline in each frame in a tracking-by-detection manner but completely ignoring temporal constraints that become available in sequence. To address this issue, we propose to exploit incremental learning for person-specific alignment. Our approach takes advantage of part-based representation and cascade regression for robust and efficient alignment on each frame. More importantly, it incrementally updates the representation subspace and simultaneously adapts the cascade regressors in parallel using a unified framework. Person-specific modeling is eventually achieved on the fly while the drifting issue is significantly alleviated by erroneous detection using both part and holistic descriptors. Extensive experiments on both controlled and in-the-wild datasets demonstrate the superior performance of our approach compared with the state of the arts in terms of fitting accuracy and efficiency.

Original languageEnglish (US)
Title of host publicationProceedings - 29th IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2016
PublisherIEEE Computer Society
Pages1558-1567
Number of pages10
ISBN (Electronic)9781467388504
DOIs
StatePublished - Dec 16 2016
Event29th IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2016 - Las Vegas, United States
Duration: Jun 26 2016Jul 1 2016

Publication series

NameIEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
ISSN (Print)2160-7508
ISSN (Electronic)2160-7516

Other

Other29th IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2016
CountryUnited States
CityLas Vegas
Period6/26/167/1/16

Fingerprint

Experiments

All Science Journal Classification (ASJC) codes

  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering

Cite this

Peng, X., Huang, J., & Metaxas, D. N. (2016). Sequential Face Alignment via Person-Specific Modeling in the Wild. In Proceedings - 29th IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2016 (pp. 1558-1567). [7789684] (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops). IEEE Computer Society. https://doi.org/10.1109/CVPRW.2016.194
Peng, Xi ; Huang, Junzhou ; Metaxas, Dimitris N. / Sequential Face Alignment via Person-Specific Modeling in the Wild. Proceedings - 29th IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2016. IEEE Computer Society, 2016. pp. 1558-1567 (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops).
@inproceedings{4c312785cdd34000854ddc2f9dadae94,
title = "Sequential Face Alignment via Person-Specific Modeling in the Wild",
abstract = "Sequential face alignment, in essence, deals with nonrigid deformation that changes over time. Although numerous methods have been proposed to show impressive success on still images, many of them still suffer from limited performance when it comes to sequential alignment in wild scenarios, e.g., involving large pose/expression variations and partial occlusions. The underlying reason is that they usually perform sequential alignment by independently applying models trained offline in each frame in a tracking-by-detection manner but completely ignoring temporal constraints that become available in sequence. To address this issue, we propose to exploit incremental learning for person-specific alignment. Our approach takes advantage of part-based representation and cascade regression for robust and efficient alignment on each frame. More importantly, it incrementally updates the representation subspace and simultaneously adapts the cascade regressors in parallel using a unified framework. Person-specific modeling is eventually achieved on the fly while the drifting issue is significantly alleviated by erroneous detection using both part and holistic descriptors. Extensive experiments on both controlled and in-the-wild datasets demonstrate the superior performance of our approach compared with the state of the arts in terms of fitting accuracy and efficiency.",
author = "Xi Peng and Junzhou Huang and Metaxas, {Dimitris N.}",
year = "2016",
month = "12",
day = "16",
doi = "10.1109/CVPRW.2016.194",
language = "English (US)",
series = "IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops",
publisher = "IEEE Computer Society",
pages = "1558--1567",
booktitle = "Proceedings - 29th IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2016",
address = "United States",

}

Peng, X, Huang, J & Metaxas, DN 2016, Sequential Face Alignment via Person-Specific Modeling in the Wild. in Proceedings - 29th IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2016., 7789684, IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, IEEE Computer Society, pp. 1558-1567, 29th IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2016, Las Vegas, United States, 6/26/16. https://doi.org/10.1109/CVPRW.2016.194

Sequential Face Alignment via Person-Specific Modeling in the Wild. / Peng, Xi; Huang, Junzhou; Metaxas, Dimitris N.

Proceedings - 29th IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2016. IEEE Computer Society, 2016. p. 1558-1567 7789684 (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Sequential Face Alignment via Person-Specific Modeling in the Wild

AU - Peng, Xi

AU - Huang, Junzhou

AU - Metaxas, Dimitris N.

PY - 2016/12/16

Y1 - 2016/12/16

N2 - Sequential face alignment, in essence, deals with nonrigid deformation that changes over time. Although numerous methods have been proposed to show impressive success on still images, many of them still suffer from limited performance when it comes to sequential alignment in wild scenarios, e.g., involving large pose/expression variations and partial occlusions. The underlying reason is that they usually perform sequential alignment by independently applying models trained offline in each frame in a tracking-by-detection manner but completely ignoring temporal constraints that become available in sequence. To address this issue, we propose to exploit incremental learning for person-specific alignment. Our approach takes advantage of part-based representation and cascade regression for robust and efficient alignment on each frame. More importantly, it incrementally updates the representation subspace and simultaneously adapts the cascade regressors in parallel using a unified framework. Person-specific modeling is eventually achieved on the fly while the drifting issue is significantly alleviated by erroneous detection using both part and holistic descriptors. Extensive experiments on both controlled and in-the-wild datasets demonstrate the superior performance of our approach compared with the state of the arts in terms of fitting accuracy and efficiency.

AB - Sequential face alignment, in essence, deals with nonrigid deformation that changes over time. Although numerous methods have been proposed to show impressive success on still images, many of them still suffer from limited performance when it comes to sequential alignment in wild scenarios, e.g., involving large pose/expression variations and partial occlusions. The underlying reason is that they usually perform sequential alignment by independently applying models trained offline in each frame in a tracking-by-detection manner but completely ignoring temporal constraints that become available in sequence. To address this issue, we propose to exploit incremental learning for person-specific alignment. Our approach takes advantage of part-based representation and cascade regression for robust and efficient alignment on each frame. More importantly, it incrementally updates the representation subspace and simultaneously adapts the cascade regressors in parallel using a unified framework. Person-specific modeling is eventually achieved on the fly while the drifting issue is significantly alleviated by erroneous detection using both part and holistic descriptors. Extensive experiments on both controlled and in-the-wild datasets demonstrate the superior performance of our approach compared with the state of the arts in terms of fitting accuracy and efficiency.

UR - http://www.scopus.com/inward/record.url?scp=85010197289&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85010197289&partnerID=8YFLogxK

U2 - 10.1109/CVPRW.2016.194

DO - 10.1109/CVPRW.2016.194

M3 - Conference contribution

AN - SCOPUS:85010197289

T3 - IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops

SP - 1558

EP - 1567

BT - Proceedings - 29th IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2016

PB - IEEE Computer Society

ER -

Peng X, Huang J, Metaxas DN. Sequential Face Alignment via Person-Specific Modeling in the Wild. In Proceedings - 29th IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2016. IEEE Computer Society. 2016. p. 1558-1567. 7789684. (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops). https://doi.org/10.1109/CVPRW.2016.194