Testing the Effects of Real-time Scaffolding of Science Inquiry Driven by Automated Performance Assessment

This project addresses how middle school students conduct scientific inquiry in complex domains. Expertise and methods from the learning sciences, science education, assessment, statistics, computer science, and educational data mining are combined in a program of work that: (1) uses knowledge engineering and data mining to develop methods for performance assessment of inquiry in earth and life sciences; (2) uses these performance algorithms to trigger a pedagogical agent that will provide individualized scaffolding to students in real time; and (3) studies the effects of that scaffolding. This project builds on an extensive body of prior research (including several NSF-supported projects) on STEM learning with simulations and assessments of STEM learning. The two main research questions seek to address the extent to which inquiry skills scaffolded by the pedagogical agent transfer across tasks within the same topic, and across topics within the same domain. Key activities to be conducted over the course of the three-year project include: (1) developing multiple types of scaffolds for the pedagogical agent; (2) pilot testing and analysis with students and teachers to drive iterative improvement; (3) developing (using knowledge engineered and data mined algorithms) and validating metrics to assess the inquiry skills of (a) hypothesis generation, (b) data collection, (c) data interpretation, and (d) warranting claims; (4) integrating assessment metrics with the pedagogical agent; and (5) data collection and analysis to address the research questions. Data collection will take place in five Massachusetts schools, comparing middle school students who do to those who do not receive inquiry scaffolding via the pedagogical agent (students randomly assigned to treatment conditions by topic). At the conclusion of the project, log data will be made freely available to scientific researchers for secondary research and teaching purposes via the Pittsburgh Science of Learning Center DataShop.