Automating the ABCD method with machine learning

Gregor Kasieczka, Benjamin Nachman, Matthew D. Schwartz, David Shih

Research output: Contribution to journalArticlepeer-review

15 Scopus citations


The ABCD method is one of the most widely used data-driven background estimation techniques in high energy physics. Cuts on two statistically independent classifiers separate signal and background into four regions, so that background in the signal region can be estimated simply using the other three control regions. Typically, the independent classifiers are chosen "by hand"to be intuitive and physically motivated variables. Here, we explore the possibility of automating the design of one or both of these classifiers using machine learning. We show how to use state-of-the-art decorrelation methods to construct powerful yet independent discriminators. Along the way, we uncover a previously unappreciated aspect of the ABCD method: its accuracy hinges on having low signal contamination in control regions not just overall, but relative to the signal fraction in the signal region. We demonstrate the method with three examples: a simple model consisting of three-dimensional Gaussians; boosted hadronic top jet tagging; and a recasted search for paired dijet resonances. In all cases, automating the ABCD method with machine learning significantly improves performance in terms of ABCD closure, background rejection, and signal contamination.

Original languageEnglish (US)
Article number035021
JournalPhysical Review D
Issue number3
StatePublished - Feb 22 2021

All Science Journal Classification (ASJC) codes

  • Nuclear and High Energy Physics


Dive into the research topics of 'Automating the ABCD method with machine learning'. Together they form a unique fingerprint.

Cite this