Interrater reliability of ICU EEG research terminology

BACKGROUND: Standardized research terminology critical to the establishment of a multicenter intensive care unit (ICU) electroencephalogram (EEG) database was originally proposed in 2005 and has been modified many times since. However, interrater agreement (IRA) of the revised terminology has not been investigated. METHODS: After a brief tutorial, investigators of ICU EEG research centers (n = 16) took an 82-question EEG certification test comprising 10-second EEG samples, which assessed the use of main term 1 (pattern location), main term 2 (pattern type), and modifiers from the most recently revised terminology. RESULTS: Kappa values for main terms 1 and 2 were 0.87 and 0.92, respectively. Agreement was 93% for determination of amplitude and 80% for determination of frequency. Kappa values for each of the "plus" modifiers (fast, rhythmic, and sharp/spike activity) were 0.54, 0.62, and 0.16 respectively. CONCLUSIONS: Main terms 1 and 2 have high IRA and are reasonable for use in multicenter research. There is a suggestion that assessment of amplitude has good reliability, while assessment of frequency may have less reliability. The fast and rhythmic "plus" modifiers have moderate IRA, while sharp/spike modifier has only slight IRA implying that further refinement and assessment of terminology modifiers may be necessary.