Interobserver agreement: A preliminary investigation into how much is enough?

Nicole L. Hausman, Noor Javed, Molly K. Bednar, Madeleine Guell, Erin Schaller, Rose E. Nevill, Sung Woo Kahng

Research output: Contribution to journalArticlepeer-review

Abstract

Interobserver agreement (IOA) is important for research and practice, and supports the consistency of behavioral data (Kahng et al., 2011). Although general parameters for how much IOA is needed have been suggested (Bailey & Burch, 2018), it is unknown if the total number of sessions with IOA might impact the IOA coefficient. In this study, IOA was reanalyzed using functional analysis data at various cutoffs. Obtained IOA from these analyses was then compared to the original IOA. Overall, results suggested that, at least when using highly trained observers in a structured clinical setting, there were no significant differences in IOA across cutoffs. However, IOA was sensitive to overall rate of responding in the functional analysis. These data are encouraging, particularly for practitioners, because they provide preliminary support that the amount of sessions with IOA may not be as important as the consistency of the data.

Original languageEnglish (US)
JournalJournal of applied behavior analysis
DOIs
StateAccepted/In press - 2021

All Science Journal Classification (ASJC) codes

  • Philosophy
  • Applied Psychology
  • Sociology and Political Science

Keywords

  • data collection
  • interobserver agreement
  • reliability

Fingerprint

Dive into the research topics of 'Interobserver agreement: A preliminary investigation into how much is enough?'. Together they form a unique fingerprint.

Cite this