Variable selection and estimation with the seamless-L0 penalty

Lee Dicker, Baosheng Huang, Xihong Lin

Research output: Contribution to journalArticle

38 Scopus citations

Abstract

Penalized least squares procedures that directly penalize the number of variables in a regression model (L0 penalized least squares procedures) enjoy nice theoretical properties and are intuitively appealing. On the other hand, L0 penalized least squares methods also have significant drawbacks in that implementation is NP-hard and not computationally feasible when the number of variables is even moderately large. One of the challenges is the discontinuity of the L0 penalty. We propose the seamless-L0 (SELO) penalty, a smooth function on [0;∞) that very closely resembles the L0 penalty. The SELO penalized least squares procedure is shown to consistently select the correct model and is asymptotically normal, provided the number of variables grows more slowly than the number of observations. SELO is efficiently implemented using a coordinate descent algorithm. Since tuning parameter selection is crucial to the performance of the SELO procedure, we propose a BIC-like tuning parameter selection method for SELO, and show that it consistently identifies the correct model while allowing the number of variables to diverge. Simulation results show that the SELO procedure with BIC tuning parameter selection performs well in a variety of settings - outperforming other popular penalized least squares procedures by a substantial margin. Using SELO, we analyze a publicly available HIV drug resistance and mutation dataset and obtain interpretable results.

Original languageEnglish (US)
Pages (from-to)929-962
Number of pages34
JournalStatistica Sinica
Volume23
Issue number2
DOIs
StatePublished - Apr 1 2013

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Keywords

  • BIC
  • Coordinate descent
  • Oracle property
  • Penalized least squares
  • Tuning parameter selection

Fingerprint Dive into the research topics of 'Variable selection and estimation with the seamless-L<sub>0</sub> penalty'. Together they form a unique fingerprint.

  • Cite this