Abstract
The Convex Gaussian Min-Max Theorem (CGMT) has emerged as a prominent theoretical tool for analyzing the precise stochastic behavior of various statistical estimators in the so-called high-dimensional proportional regime, where the sample size and the signal dimension are of the same order. However, a well-recognized limitation of the existing CGMT machinery rests in its stringent requirement on the exact Gaussianity of the design matrix, therefore rendering the obtained precise high-dimensional asymptotics, largely a specific Gaussian theory in various important statistical models. This paper provides a structural universality framework for a broad class of regularized regression estimators that is particularly compatible with the CGMT machinery. Here, universality means that if a “structure” is satisfied by the regression estimator μ̂G for a standard Gaussian design G, then it will also be satisfied by μ̂A for a general non-Gaussian design A with independent entries. In particular, we show that with a good enough l.
Original language | English (US) |
---|---|
Pages (from-to) | 1799-1823 |
Number of pages | 25 |
Journal | Annals of Statistics |
Volume | 51 |
Issue number | 4 |
DOIs | |
State | Published - Aug 2023 |
All Science Journal Classification (ASJC) codes
- Statistics and Probability
- Statistics, Probability and Uncertainty
Keywords
- Gaussian comparison inequalities
- Lasso
- Lindeberg's principle
- high-dimensional asymptotics
- random matrix theory
- ridge regression
- robust regression
- universality