Abstract
In this paper we revisit the risk bounds of the lasso estimator in the context of transductive and semi-supervised learning. In other terms, the setting under consideration is that of regression with random design under partial labeling. The main goal is to obtain user-friendly bounds on the off-sample prediction risk. To this end, the simple setting of bounded response variable and bounded (high-dimensional) covariates is considered. We propose some new adaptations of the lasso to these settings and establish oracle inequalities both in expectation and in deviation. These results provide non-asymptotic upper bounds on the risk that highlight the interplay between the bias due to the mis-specification of the linear model, the bias due to the approximate sparsity and the variance. They also demonstrate that the presence of a large number of unlabeled features may have significant positive impact in the situations where the restricted eigenvalue of the design matrix vanishes or is very small.
Original language | English (US) |
---|---|
Pages (from-to) | 3443-3472 |
Number of pages | 30 |
Journal | Electronic Journal of Statistics |
Volume | 12 |
Issue number | 2 |
DOIs | |
State | Published - 2018 |
All Science Journal Classification (ASJC) codes
- Statistics and Probability
- Statistics, Probability and Uncertainty
Keywords
- High-dimensional regression
- Lasso
- Oracle inequality
- Semi-supervised learning
- Sparsity
- Transductive learning