Learning overcomplete representations from distributed data: A brief review

Haroon Raja, Waheed U. Bajwa

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Most of the research on dictionary learning has focused on developing algorithms under the assumption that data is available at a centralized location. But often the data is not available at a centralized location due to practical constraints like data aggregation costs, privacy concerns, etc. Using centralized dictionary learning algorithms may not be the optimal choice in such settings. This motivates the design of dictionary learning algorithms that consider distributed nature of data as one of the problem variables. Just like centralized settings, distributed dictionary learning problem can be posed in more than one way depending on the problem setup. Most notable distinguishing features are the online versus batch nature of data and the representative versus discriminative nature of the dictionaries. In this paper, several distributed dictionary learning algorithms that are designed to tackle different problem setups are reviewed. One of these algorithms is cloud K-SVD, which solves the dictionary learning problem for batch data in distributed settings. One distinguishing feature of cloud K-SVD is that it has been shown to converge to its centralized counterpart, namely, the K-SVD solution. On the other hand, no such guarantees are provided for other distributed dictionary learning algorithms. Convergence of cloud K-SVD to the centralized K-SVD solution means problems that are solvable by K-SVD in centralized settings can now be solved in distributed settings with similar performance. Finally, cloud K-SVD is used as an example to show the advantages that are attainable by deploying distributed dictionary algorithms for real world distributed datasets.

Original languageEnglish (US)
Title of host publicationCompressive Sensing V
Subtitle of host publicationFrom Diverse Modalities to Big Data Analytics
EditorsFauzia Ahmad
ISBN (Electronic)9781510600980
StatePublished - 2016
EventCompressive Sensing V: From Diverse Modalities to Big Data Analytics - Baltimore, United States
Duration: Apr 20 2016Apr 21 2016

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X


OtherCompressive Sensing V: From Diverse Modalities to Big Data Analytics
CountryUnited States

All Science Journal Classification (ASJC) codes

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Learning overcomplete representations from distributed data: A brief review'. Together they form a unique fingerprint.

Cite this