Training in Experimental Design (TED) aims to develop a computer-based intelligent tutoring system to improve science instruction in late elementary through middle school grades. The proposed intervention focuses on the conceptual understanding and procedural skills of designing and interpreting scientific experiments. The design of the computer-based instruction is based on an extensive accumulation of solid empirical evidence on how to teach these concepts and skills to children from third through eighth grades using both one-onone and whole-class instruction across high- and low-SES school contexts. The proposed technological and instructional development addresses two important challenges in our past research: to adapt to individual variability and to promote robust transfer to standardized-test outcome measures.

The R&D process iterates through increasingly computerized and adaptive TED modules. The modules will include simulations of experiments across many different domains, diagnostic capabilities to track students’ mastery and misconceptions, and adaptive algorithms that match task and feedback to diagnostic information. During each iteration, we conduct classroom validation studies of the instructional module in both highand low-SES school contexts. We will use up to 8 classrooms in each grade level from fifth to eighth grade, with approximately 120 - 160 students per grade level distributed across four low-SES and two high-SES schools. Students are assessed by researcher-defined outcome measures (high construct validity) immediately before and after instruction. In addition, they are assessed by standardized-test outcome measures (high external validity) following an extended delay (weeks or months). In between the immediate posttest and the delayed assessment, expert human tutors provide remedial instruction for those students who are “left behind” following the use of the TED module. Tutoring combines diagnoses of students’ misconceptions and difficulties during instruction with theoretically-guided instruction that match task and feedback variability to students’ state of knowledge.

Based on quasi-experimental between-module comparisons of immediate posttest measures (accounting for pretest and other prior measures), the classroom validation studies provide efficacy evidence for each module. Based on regression discontinuity analysis comparing delayed assessment between the non-mastery students who received remedial tutoring following initial training and the mastery students who did not receive tutoring, the human tutoring studies provide efficacy evidence of remedial diagnoses and instruction. If the remedial instruction is found to be effective, we incorporate the diagnostic and instructional strategies into the next iteration of the TED module. We will assess the efficacy of our instruction, focusing not only on its successes but also the extent to which it fails to meet the needs of all students. The discrepancy between planned instruction and learning outcomes will drive the improvement of the intelligent tutoring system and its adaptive instruction. Our aim is to deliver, by the end of the project, an Intelligent TED Tutor that can adapt to individual learners so as to enable them to achieve mastery measurable by a variety of internally and externally valid assessment instruments.