Evidence of Inflated Prediction Performance: A Commentary on Machine Learning and Suicide Research

Ross Jacobucci, Andrew K. Littlefield, Alexander J. Millner, Evan M. Kleiman, Douglas Steinley

Research output: Contribution to journalComment/debatepeer-review

4 Scopus citations

Abstract

The use of machine learning is increasing in clinical psychology, yet it is unclear whether these approaches enhance the prediction of clinical outcomes. Several studies show that machine-learning algorithms outperform traditional linear models. However, many studies that have found such an advantage use the same algorithm, random forests with the optimism-corrected bootstrap, for internal validation. Through both a simulation and empirical example, we demonstrate that the pairing of nonlinear, flexible machine-learning approaches, such as random forests with the optimism-corrected bootstrap, provide highly inflated prediction estimates. We find no advantage for properly validated machine-learning models over linear models.

Original languageEnglish (US)
Pages (from-to)129-134
Number of pages6
JournalClinical Psychological Science
Volume9
Issue number1
DOIs
StatePublished - Jan 2021

All Science Journal Classification (ASJC) codes

  • Clinical Psychology

Keywords

  • clinical psychology
  • data mining
  • machine learning
  • prediction
  • suicide

Fingerprint

Dive into the research topics of 'Evidence of Inflated Prediction Performance: A Commentary on Machine Learning and Suicide Research'. Together they form a unique fingerprint.

Cite this