Teacher Quality and Student Success: Testing the K20 Science Professional Development Model (K20 SCIENCE) for Science Teachers

Principal Investigator: 
Project Overview
Background & Purpose: 

This study seeks to empirically examine the impact of authentic research experiences, lesson study, and Professional Learning Communities (PLC) on teacher quality and student success. More specifically, it is hypothesized that:

H1: teachers completing the K20 Research Experiences for Science Teachers (KREST) institute will report significantly higher scores over control members on assessments of teacher knowledge, efficacy, collaboration/communication, authentic pedagogy, and student success;

H2: teachers completing lesson study will report significantly higher scores over control members on assessments of teacher knowledge, efficacy, collaboration/communication, authentic pedagogy and student success.

Setting: 

Sixth to twelfth grade science teachers and their classrooms from across the State of Oklahoma.

Research Design: 

The research design for this project is comparative, and is designed to generate evidence that is descriptive using observation, associative/correlational and causal using quasi-experimental methods. This project includes the intervention of attendance at the KREST institute, which is a two-week summer research program designed to provide authentic research experiences for teachers, lesson study and creating professional learning communities. The comparison condition is attendance at a one-day traditional professional development workshop.

This project collects original data using a variety of methods: diaries/journals/records kept by study subjects, assessments of learning/achievement tests, personal observation, and survey research including both paper and pencil and online self-completion questionnaires, as well as face-to-face semi-structured/informal interviews.

Instruments and measures used include:

Teacher Attitude and Motivation to Teach Science and Integrate Technology Pre/Post Survey
Teacher attitude and motivation to teach science and integrate technology into science teaching was measured through the use of an on-line pre/post survey. This survey was developed from an instrument previously tested by the K20 (the PDI used by Janis Slater), work compiled by Yager (2005), and items from the motivation research of one of the PI’s for measuring self-efficacy (e.g., Greene, Dillon, Crynes, 2003 & Greene et al. 2004). Therefore, all the items included on the survey instrument had been tested in other research and found to be reliable and valid for measuring attitudes and efficacy as they were being used in the present research.

Motivation and Aspiration Survey
All students, who were enrolled in the participating teachers’ classroom were asked to complete a short survey which included measures of: 1) Attitudes and efficacy for science and the inquiry method, 2) Self-efficacy for learning science, 3) Self-efficacy for using technology for science, 4) Use of constructivist approaches in the classroom, 4) Collaborative practices to support constructivist approaches in the classroom, 5) Expository and didactic approaches in classroom, and 6) Technology use in the classroom. This survey was modeled from the pre/post survey used to measure teacher attitudes, motivation and technology integration, thereby incorporating and modifying previous work by Slater (2007, 2008); Yager (2005); Greene, Dillon, Crynes (2003); and Greene et al., (2004).

Inquiry Process Test
To obtain a measure of student achievement, all students from selected comparison and experimental group teacher participants were asked to take a multiple choice scientific inquiry literacy test. Along with the student science motivation and aspiration survey, the instrument is administered at the beginning and end of the school year in Year 3 of the study. The K20 science team adapted the Scientific Literacy Inquiry Test (ScInqLiT) developed by Carl Wenning (2007) for use with Middle and High school students participating in the REESE study.

Classroom Observation
For the Spring 2008 pilot test of the classroom observation instrument, 11 teacher classrooms each from comparison and experimental group participants were selected for participation in classroom observations. One member from the evaluation/research team and one member from the program team or other qualified K20 staff visited the selected teacher classrooms and observed a lab or activity lesson in its entirety. The REESE observation teams observed and recorded student engagement and interactions with the teacher during the lesson. A teacher and student inquiry observation guide was developed jointly by the evaluation and program teams. Staff involved in classroom observations included members of the K20 REESE evaluation/research team, K20 REESE program team and members of the K20 staff who are certified science teachers in the State of Oklahoma and are trained in the use and teaching of the inquiry method. All individuals who conducted the classroom observations had completed the one day training given by Dr. Cate. The actual tool used in the classroom observations was adapted from Zubrowski, B. (2007) and Horizon Research, Inc (2000, 2005). The K20 observation tool includes measures for teacher use of inquiry methods and student level of engagement.

Concept Mapping
We also used Concept Maps with the KREST participants to capture and score teacher knowledge of the science concepts that were taught during the professional development sessions. These data are showing that the concept maps seem to be a good way to understand how teacher knowledge changed. Working mostly with the research of Graff (2005) and ideas of Shavelson, Ruiz-Primo, & Wiley (2005) a subgroup of the research and evaluation team developed a series of scores to use with the concept maps. They used the following metrics: Number of nodes minus one for the central concept; number of links or propositions, for a complexity score; links / nodes – 1 for a map density score; the longest link for a depth index; and the level with the greatest number of nodes for a width index. These metrics capture the complexity and structure of the knowledge represented in the maps. We are hoping that they will also capture change over time. At least three people scored and checked each map to ensure accuracy and reliability.

Return to Science (RTS) Semi-structured Interview
Interviews were conducted with the RTS KREST participants from the 2007 institute. The Interview protocol included questions about their participation in the institute, professional learning community development, on-line learning communities, implementation of inquiry teaching methods in the classroom, and the collaborative lesson study process.

This is a mixed methods study. For quantitative data analysis we are using correlational techniques (e.g., factor analysis) and multivariate analysis of variance for many of the tests or similar parametric statistics. We may also need non-parametric is some cases. Data and questions will determine actual tests used. Some of our data will likely require HLM procedures. For the qualitative data, standard constant comparison methods will be used. All data will be kept for five years in the event that other researchers ask to analyze our data.

Findings: 

Preliminary analysis of Pilot data from Teacher surveys, concept maps and student survey/tests indicates:

Teachers Knowledge

  • Findings from analysis of concept maps indicate a significant change in teacher knowledge over time in relation to amount and/or number of
    • content specific knowledge,
    • links between concepts on the map
    • integration of concepts
    • super-ordinate relationships identified, and
    • breadth and depth of knowledge.

Teacher Attitudes, Motivation, & Practice

  • Both groups reported increased endorsement of constructivist practices over time and decreased endorsement of didactic practices in their science teaching. No other differences were found.
  • There were no interactions found to suggest that the two-week group had more positive attitudes and motivation over time than the one-day group.

Student Knowledge

  • Overall students whose teachers participated in the K20 SCIENCE PD model scored significantly higher on the science process test than students whose teachers attended the one-day workshop.
  • Use of technology in the classroom and being in a suburban school was also associated with increased test scores.
Publications & Presentations: 

Journal Publications

O’Hair, M.J., & Reitzug, U.C. (2006). Working for social justice in rural schools: A model for science education. International Electronic Journal for Leadership in Learning, 10(28).
http://www.ucalgary.ca/~iejll/volume10/o%27hair_reitzug.htm .

Greene, B. A., Lubin, I.A., Slater, J., & Walden, S. E. (2009). Mapping Changes in Science Teachers’ Knowledge: Concept Maps and Authentic Professional Development. Paper submitted for publication in Journal of Educational Research.

Book Chapters

Atkinson, L., Cate, J.M., O’Hair, M.J., & Slater, J. (In press). K20 Model: Creating networks, professional learning communities, and communities of practice that increase science learning. In S. Mundry & K. Stiles (Eds.). Professional learning communities in science. San Francisco, CA: WestEd.

O’Hair, M. J., & Reitzug, U. C. (2008). A neglected dimension of social justice: A model for science education in rural schools. In A. Normore (Ed.), Leadership for social justice: Promoting equity and excellence through inquiry and reflective practice (pp. 149-165). Charlotte, NC: Information Age Publishing.

Working Papers

Greene, B. A., Walden, S. Slater, J., Lubin, I., and M. Keesee, I. (2008). Using Concept Maps to Capture Changes in Science Teachers’ Content Knowledge: Authentic Professional Development for Science Teachers

Greene, B. A., M. Keesee, I. Lubin, S. Walden and J. Slater (2008). Collaboration Between Science Teachers and University Scientists: Inquiry-oriented Professional Development.

Keesee, M.S., Slater, J. Greene, B., Cate, J., Atkinson, L., and O’Hair, M.J. Increasing Student Process Scores for K20 Science Teachers.

Reports

K20 Center. (2007). NSF REESE – KREST Institute 2007: Participant Evaluation of Two-Week Workshop.

K20 Center. (2007). NSF REESE – PIC Workshop 2007: Participant Evaluation of One-Day Workshop.

K20 Center. (working). NSF REESE – KREST Institute 2008: Participant Evaluation of Two-Week Workshop.

K20 Center. (working). NSF REESE – PIC Workshop 2008: Participant Evaluation of One-Day Workshop.

Presentations

  1. University Council for Educational Administration (UCEA) annual
  2. meeting, November 2006;
  3. K20 Center's 10th Annual Winter Institute, January 2007;
  4. K20 Center's Leadership Seminar, February 2007;
  5. New DEEL conference, Temple University, February 2007;
  6. Big 12 Educational Research Conference, March 2007;
  7. National Science Education Leadership Association, March 2007;
  8. Annual Report: 0634070
  9. K20 Center's Rural Science Initiatives (REESE) and the
  10. Amiens/Picardi Region of France, March 2007;
  11. American Educational Research Association presentation, April 2007
  12. OU's Deans' Council, April 2007;
  13. Cooperative Council for Oklahoma School Administrators, June 2007;
  14. University of Aberdeen, Scotland, June 2007;
  15. Oklahoma Alternative Schools Conference, June 2007.
  16. K20 Center, Open House, September 27, 2007
  17. Changing the Course of Science Education, 2007 National Leadership Development and Strategic Planning Symposium on K-16 Science Education for State Teams, September 30-October 2, 2007, Washington, DC.
  18. New Jersey Science Convention, Somerset, New Jersey, October 9, 2007
  19. Global Education Conference, Rose State College, November 3, 2007
  20. Annual meeting of the National Communication Association, Chicago, November, 2007
  21. University Council for Educational Administration, Washington DC, November 17, 2007
  22. K20 Center’s 11th Annual Winter Institute, January 2008
  23. New DEEL (Democratic Ethical Educational Leadership) Conference, Temple University, February 15, 2008
  24. National Science Education Leadership in Boston, March 26, 2008 in conjunction with the 2008 National Science Teachers Association (NSTA) Conference.
  25. American Educational Research Association Conference, New York, March 2008
  26. Phase I Leadership Seminar, K20 Center, April 2-3, 2008
  27. O’Hair, M. J., & Atkinson, L. Invited presentation to National Science Board, Washington D.C., May 2008
  28. Atkinson, L. "Integrating Authentic Research Experiences into Classroom Practices". Presented at the New Jersey Science Conference, Somerset, New Jersey, October 15, 2008.
  29. CCOSA Secondary Principal's Winter Conference, November 12, 2008.
  30. M. Keesee, B. Greene, J. Slater, J. Cate, L. Atkinson, M.J. O’Hair and H. D. O’Hair “Increasing Student Process Scores for K20 SCIENCE Teachers”. Presented at the K20 Center Engaged Research Conference, Norman, Oklahoma, November 14, 2008.
  31. B. Greene, M. Keesee, I. Lubin, S. Walden and J. Slater “Collaboration between Science Teachers and University Scientists: Inquiry-Oriented Professional Development”. Presented at the K20 Center Engaged Research Conference, Norman, Oklahoma, November 14, 2008.
  32. K20 Center's 12th Annual Winter Institute, January 2009.
  33. Greene, B. A., Lubin, I., Slater, J., Walden, S. E., & Keesee, M. S. (July, 2009). Using concept maps to capture changes in science teachers’ content knowledge: Authentic professional development for science teachers. Paper presented at the Sixteenth International Conference on Learning in Barcelona. Spain.
  34. Walden, S. E., Greene, B. A., Slater, J., Lubin, I., & Keesee, M.S. (August, 2009). Collaboration between science teachers and university scientists: Inquiry-oriented professional development. To be presented at the 2009 Biennial Conference meeting of the European Association for Research on Learning and Instruction, Amsterdam, the Netherlands.
Other Products: 

The intent of the study was not to develop marketable instruments or materials. However, both were developed to conduct the study. In particular the assessment tools will be products that will be useful beyond this study. Additionally, the professional development model studied was further articulated by this research for future applications and more evidence of the replicability of the model related outcomes.