The research design for this project is comparative and will generate evidence that is descriptive [case study and observational], associative [quasi-experimental], and causal [quasi-experimental]. The project collects original data using teacher artifacts, assessments of learning, observation, interviews [semi- structured interviews and structured cognitive interviews], and survey research [self-completion questionnaire]. The intervention includes educative supports for teachers added to Science, Technology, and Children (STC) curriculum materials, with teachers using regular STC materials (without educative supports) serving as the comparison condition.
We will use assessments of teachers' relevant subject matter knowledge and PCK. Through the ATLAST project (EHR-0335328), Smith and Horizon have developed an approach to measuring teacher content knowledge in the context of instructional practice. This work builds on research on the specialized content knowledge teachers hold (Ball et al., 2005). All ATLAST teacher and student assessment items share some common features: items are written in a multiple choice format, placing minimal burden on the test taker and researcher, and items are very precise in the content they target—each item typically targets one idea. All teacher assessment items are set in instructional contexts, acknowledging the unique work that teachers do with their knowledge. Teachers must understand the content to choose the correct answer.
To guide our classroom observations, we will use a protocol for characterizing the use of elements of effective science teaching (see Banilower, Cohen, Pasley, & Weiss, 2008), to allow us to see how the teacher enacts the curriculum materials and how well any adaptations are aligned with the developers' intent, the supports for scientific practices provided in the educative elements, and the elements of effective instruction. We will also use a system of structured field notes to allow us to focus on the most salient features of the teachers' practice and of the instructional interactions.
We will use or adapt a survey developed by RAND for the Mosaic II project (see http://www.rand.org/pubs/monograph_reports/MR1233/, the Science Grade 5 survey). The survey is intended to measure teachers' use of reform-oriented science teaching practices. It includes forced-response questions about typical classroom scenarios in which the teacher has the opportunity to incorporate scientific practices such as those emphasized in our educative overlays, allowing us to characterize teachers' knowledge and beliefs related to these practices.
We will use a range of appropriate analytic approaches for our set of studies. For example, using the teacher scores on the written assessments administered as a part of the quasi-experimental study, we will use repeated measures analysis of variance (ANOVA) to quantify teachers' subject matter knowledge and PCK, and to look at differences across conditions.
