Radiologist vs Machine Learning: A Comparison of Performance in Cancer Imaging

Machine learning (ML) has become a popular topic in Radiology, but practical implementation has been limited. This review summarized literature that compared predictive algorithms to radiologists to identify potential barriers to reproducibility and implementation of AI research. PubMed was searched for peer-reviewed manuscripts in English that compared performance of algorithms with that of radiologists. Full-text analysis was performed on 337 articles. Some manuscripts contained more than one comparison, resulting in 61 final manuscripts and 70 algorithm-to-radiologist comparisons. On average, algorithms performed comparably to radiologists; with most algorithms being comparable (0.00 difference) or marginally better (0.10 difference) than radiologist performance. Only eight algorithms included enough features to be replicable (features defined for this manuscript as model inputs containing relevant information such as coefficients, code, or variables). Despite these promising results, most publications did not contain enough information to replicate the algorithms in future studies. This study concluded that standardized metrics and benchmarks for development and reporting of ML algorithms in oncologic imaging are urgently needed.