The NSF-funded Undergraduate Course Innovations (UCI) research project team has analyzed the quality of studies focusing on the effects of instructional innovations in undergraduate physics education and its impact on student learning. From more than 400 papers collected, only 118 met the methodological requirements necessary to analyze the impact of instructional innovations on student learning. From the 118 papers, 150 studies (some papers report more than one study) were analyzed. Our detailed analyses of these studies indicate consistent methodological weaknesses in the way innovations are being studied. Only a third of these 150 studies (n = 51) provide evidence that can be used to quantitatively synthesize the effects of these innovations using meta-analytic techniques.
Ruiz-Primo, M. A., Briggs, D., Shepard, L., Iverson, H., & Huchton, M. (2008). Evaluating the impact of instructional innovations in engineering education. In M. Duque (Ed.). Engineering education for the XXI Century: Foundations, strategies and cases (pp. 241-274). Bogotá, Colombia: ACOFI Publications.
Ruiz-Primo, M. A., Briggs, D., Shepard Lorrie, Iverson, H., Hutchton, M. (2008, June). Innovations in undergraduate science education: Lessons Learned. Invited talk as a Conference Plenary Session. International Conference of Active Learning in Engineering Education. Bogotá, Colombia.
See NSF ANNUAL REPORTS
An approach to be considered when designing studies to evaluate the impact of innovation on student learning.