Interobserver agreement: A preliminary investigation into how much is enough?

Nicole L. Hausman, Noor Javed, Molly K. Bednar, Madeleine Guell, Erin Schaller, Rose E. Nevill, Sung Woo Kahng

Research output: Contribution to journalArticlepeer-review

16 Scopus citations

Abstract

Interobserver agreement (IOA) is important for research and practice, and supports the consistency of behavioral data (Kahng et al., 2011). Although general parameters for how much IOA is needed have been suggested (Bailey & Burch, 2018), it is unknown if the total number of sessions with IOA might impact the IOA coefficient. In this study, IOA was reanalyzed using functional analysis data at various cutoffs. Obtained IOA from these analyses was then compared to the original IOA. Overall, results suggested that, at least when using highly trained observers in a structured clinical setting, there were no significant differences in IOA across cutoffs. However, IOA was sensitive to overall rate of responding in the functional analysis. These data are encouraging, particularly for practitioners, because they provide preliminary support that the amount of sessions with IOA may not be as important as the consistency of the data.

Original languageEnglish (US)
Pages (from-to)357-368
Number of pages12
JournalJournal of applied behavior analysis
Volume55
Issue number2
DOIs
StatePublished - Mar 1 2022

All Science Journal Classification (ASJC) codes

  • Philosophy
  • Sociology and Political Science
  • Applied Psychology

Keywords

  • data collection
  • interobserver agreement
  • reliability

Fingerprint

Dive into the research topics of 'Interobserver agreement: A preliminary investigation into how much is enough?'. Together they form a unique fingerprint.

Cite this