UNIVERSALITY OF REGULARIZED REGRESSION ESTIMATORS IN HIGH DIMENSIONS

Qiyang Han, Yandi Shen

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

The Convex Gaussian Min-Max Theorem (CGMT) has emerged as a prominent theoretical tool for analyzing the precise stochastic behavior of various statistical estimators in the so-called high-dimensional proportional regime, where the sample size and the signal dimension are of the same order. However, a well-recognized limitation of the existing CGMT machinery rests in its stringent requirement on the exact Gaussianity of the design matrix, therefore rendering the obtained precise high-dimensional asymptotics, largely a specific Gaussian theory in various important statistical models. This paper provides a structural universality framework for a broad class of regularized regression estimators that is particularly compatible with the CGMT machinery. Here, universality means that if a “structure” is satisfied by the regression estimator μ̂G for a standard Gaussian design G, then it will also be satisfied by μ̂A for a general non-Gaussian design A with independent entries. In particular, we show that with a good enough l.

Original languageEnglish (US)
Pages (from-to)1799-1823
Number of pages25
JournalAnnals of Statistics
Volume51
Issue number4
DOIs
StatePublished - Aug 2023

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Keywords

  • Gaussian comparison inequalities
  • Lasso
  • Lindeberg's principle
  • high-dimensional asymptotics
  • random matrix theory
  • ridge regression
  • robust regression
  • universality

Fingerprint

Dive into the research topics of 'UNIVERSALITY OF REGULARIZED REGRESSION ESTIMATORS IN HIGH DIMENSIONS'. Together they form a unique fingerprint.

Cite this