Low-Rank and Joint Sparse Representations for Multi-Modal Recognition

Heng Zhang, Vishal M. Patel, Rama Chellappa

Research output: Contribution to journalArticle

4 Scopus citations


We propose multi-task and multivariate methods for multi-modal recognition based on low-rank and joint sparse representations. Our formulations can be viewed as generalized versions of multivariate low-rank and sparse regression, where sparse and low-rank representations across all modalities are imposed. One of our methods simultaneously couples information within different modalities by enforcing the common low-rank and joint sparse constraints among multi-modal observations. We also modify our formulations by including an occlusion term that is assumed to be sparse. The alternating direction method of multipliers is proposed to efficiently solve the resulting optimization problems. Extensive experiments on three publicly available multi-modal biometrics and object recognition data sets show that our methods compare favorably with other feature-level fusion methods.

Original languageEnglish (US)
Article number7962177
Pages (from-to)4741-4752
Number of pages12
JournalIEEE Transactions on Image Processing
Issue number10
StatePublished - Oct 2017


All Science Journal Classification (ASJC) codes

  • Software
  • Computer Graphics and Computer-Aided Design


  • Multi-modal recognition
  • feature-level fusion
  • joint-sparse representation
  • low-rank representation

Cite this