Bounds on the Prediction Error of Penalized Least Squares Estimators with Convex Penalty

Pierre Bellec, Alexandre Tsybakov

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Scopus citations

Abstract

This paper considers the penalized least squares estimator with arbitrary convex penalty. When the observation noise is Gaussian, we show that the prediction error is a subgaussian random variable concentrated around its median. We apply this concentration property to derive sharp oracle inequalities for the prediction error of the LASSO, the group LASSO, and the SLOPE estimators, both in probability and in expectation. In contrast to the previous work on the LASSO-type methods, our oracle inequalities in probability are obtained at any confidence level for estimators with tuning parameters that do not depend on the confidence level. This is also the reason why we are able to establish sparsity oracle bounds in expectation for the LASSO-type estimators, while the previously known techniques did not allow for the control of the expected risk. In addition, we show that the concentration rate in the oracle inequalities is better than it was commonly understood before.

Original languageEnglish (US)
Title of host publicationModern Problems of Stochastic Analysis and Statistics - Selected Contributions in Honor of Valentin Konakov
EditorsVladimir Panov
PublisherSpringer New York LLC
Pages315-333
Number of pages19
ISBN (Print)9783319653129
DOIs
StatePublished - 2017
EventInternational Conference on Modern problems of stochastic analysis and statistics, in honor On the occasion of Valentin Konakov’s 70th birthday, 2016 - Moscow, Russian Federation
Duration: May 29 2016Jun 2 2016

Publication series

NameSpringer Proceedings in Mathematics and Statistics
Volume208
ISSN (Print)2194-1009
ISSN (Electronic)2194-1017

Other

OtherInternational Conference on Modern problems of stochastic analysis and statistics, in honor On the occasion of Valentin Konakov’s 70th birthday, 2016
CountryRussian Federation
CityMoscow
Period5/29/166/2/16

All Science Journal Classification (ASJC) codes

  • Mathematics(all)

Keywords

  • Group LASSO
  • LASSO estimator
  • Oracle inequality
  • Penalized least squares
  • SLOPE estimator

Fingerprint Dive into the research topics of 'Bounds on the Prediction Error of Penalized Least Squares Estimators with Convex Penalty'. Together they form a unique fingerprint.

Cite this