CAREER: Data-Driven Instructional Systems-- Accessing How School Leaders Develop Local Capacity to Use Data to Influence Instruction

Principal Investigator: 
Project Overview
Background & Purpose: 

The Data-Driven Instructional Systems (DDIS) study examines how expert school leaders design systems of structures, people and practices to help teachers translate testing data into information for everyday use. The study documents the innovative ways in which teachers, school administrators, and district administrators use data to improve teaching and learning in math, science, and literacy. We strive to represent school leader practices so that other school leaders can learn how to create their local DDIS.

Setting: 

We studied data-driven instructional practices at nine urban, rural and suburban schools in southern Wisconsin.

Research Design: 

The research design for this project is comparative, and is designed to generate descriptive evidence utilizing case-study, design research, ethnography, observation, and survey. This project collects original data using school records/policy documents, assessments of learning/achievement tests, observation [personal observation, videography, and web logs], survey research [self-completion questionnaire (both paper and pencil, and online), face-to-face structured interviewer-administered questionnaire, and face-to-face semi-structured/informal interviews.

Interview and observational protocols were developed to capture the range of practices school staff used to collect, organize, interpret and use data in making instructional decisions. The interview protocols include structured questions to determine the degree to which data-driven practices have taken root in schools, and the variety of task in which leaders engage to guide teachers in data-driven practices. Our survey borrows from instruments designed by Anthony Bryk and Milbrey McLaughlin to investigate professional community and school climate, and on a social network survey developed by James P. Spillane. Secondary sources, student achievement records, local school climate and attitude surveys are also analyzed.

The plan involved the analysis of qualitative and quantitative data. A Data-Driven Instructional Systems Model was developed to guide qualitative data coding through the NVIVO program. This analysis resulted in a rich, qualitative data resource from which we are assembling a series of papers and research reports. The quantitative aspect of the study analyzes the results of a 84-item survey we administered to all DDIS schools. These data have been analyzed to establish a context for making sense of the qualitative data analysis.

Findings: 

We have developed findings in five research areas:

  1. A model for systemic data-driven instructional capacity within schools. Our model organizes previously fragmented activities into a coherent set of organizational functions through which information about student learning can flow;
  2. Social network measures of school leadership efficacy. Our survey findings analyze how leaders allocate resources in the form of roles and programs to facilitate data exchange in schools. Network surveys allow us to measure the degree to which information flows across the school and to determine which local actors serve as information hubs within the school;
  3. Formative Feedback Systems. We found tight, designed information loops at the heart of efforts to intentionally change and measure student learning. These formative feedback systems place staff at the center of a loop composed of interventions, artifacts and actuation spaces (legitimate opportunities for staff to make sense of and take action upon assessment information). We argue that such formative feedback systems are at the heart of school reform activities.
  4. Student services staff and data-driven instructional capacity. We found that existing models of individualizing student learning, originally developed in special education, serve as powerful organizing principles for how schools use data to address student learning needs. We found that the practices of special educators were used to structure these non-special education plans, and that the school psychologists and social workers were increasing pressed to serve as assessment experts (because of their data expertise) rather than focus on their counseling roles for students.
  5. Subject matter focus. The policy press to improve literacy development, especially at the elementary school level, created pressure in schools to collapse science and social studies curriculum into occasions for content-area reading skill development. This left the exploratory aspects of science and social studies investigation a relatively ignored aspect of the school instructional program.
Publications & Presentations: 

PEER REVIEWED PUBLICATIONS AND BOOK CHAPTERS:

Halverson, R. (under review). “School formative feedback systems.” Peabody Journal of Education.

Halverson, R., Grigg, J., Prichett, R. & Thomas, C. (under review). “Data-driven instructional systems: A model for understanding school capacity to meet high-stakes accountability policies.” Die Deutsche Schule.

Halverson, R., Feinstein, N. & Meshoulam, D. (under review). “School leadership for science education.” In G. DeBoer (Ed.) Handbook of Research in Science Education. Information Age Press: Charlotte, NC.

Halverson, R., Wolfenstein, M., Williams, C. & Rockman, C. (in press) “Remembering math: The design of digital learning objects to spark professional learning” E-Learning.

Halverson, R. (2008) “A distributed leadership perspective on how leaders use artifacts to create professional community in schools.” In L. Stoll and K. S. Louis (Eds.) Professional learning communities: Divergence, detail and difficulties. Maidenhead: Open University Press.

Halverson, R. & Thomas, C. (2008) “The roles and practices of student services staff as data-driven instructional leaders.” In M. Mangin and S. Stoelinga (Eds.) Instructional teachers leadership roles: Using research to inform and reform. Teachers College Press: New York.

Halverson, R., Grigg, J., Prichett, R., & Thomas, C. (2007) “The New Instructional Leadership: Creating Data-Driven Instructional Systems In Schools.” Journal Of School Leadership, 17(2), 159-194 (Http://Www.Education.Wisc.Edu/Elpa/People/Faculty/Halverson/
Halversongriggprichettthomas%20ncpea.Pdf
)

WISCONSIN CENTER FOR EDUCATIONAL RESEARCH WORKING PAPERS
Formative Feedback Systems and the New Instructional Leadership 
Richard Halverson, Reid B. Prichett, Jeffery G. Watson. 
May 2007 http://www.wcer.wisc.edu/Publications/workingPapers/Working_Paper_No_2007_03.pdf

The Roles and Practices of Student Services Staff as Data-Driven Instructional Leaders.
 Richard Halverson, Christopher N. Thomas. 
February 2007
http://www.wcer.wisc.edu/Publications/workingPapers/Working_Paper_No_2007_01.pdf

The New Instructional Leadership: Creating Data-Driven Instructional Systems in Schools. Richard Halverson, Reid Prichett, Jeffrey Grigg, Chris Thomas. 
September 2005
http://www.wcer.wisc.edu/Publications/workingPapers/Working_Paper_No_2005_9.pdf

STUDENT DISSERTATIONS
Prichett, R. (2007). How school leaders make sense of and use formative feedback systems. Unpublished dissertation: University of Wisconsin-Madison School of Education.

Thomas, C. (2007) Problem-solving teams and data-driven school leadership: A path toward the next generation of special education services. Unpublished dissertation: University of Wisconsin-Madison School of Education.

Other Products: 

A key part of the second phase of the study was to develop digital technologies that would communicate the findings of the DDIS study and to guide local practitioners in the best practices we have discovered in our investigation. We have developed a Teacher Evaluation Game (http://slg.gameslearningsociety.org/index.php) to guide leaders to recognize quality teaching practices; a set of learning objects (http://www.slg.gameslearningsociety.org/cesa_files/cesa.php) to guide experienced teachers to remember the math they had once learned; and the KidGrid project, a iPod Touch based application designed to facilitate teacher collection of formative feedback information on student learning.