Practical and Theoretical Foundations for Informative Classroom Assessment—A Synthesis of Cognitive Science, Curriculum, Instruction, and Measurement

Principal Investigator: 
Co-Investigator: 
Project Overview
Background & Purpose: 

The overarching goal of this proposed Knowledge Diffusion project is to strengthen the theoretical backing and practical foundations for understanding, developing and implementing assessments embedded in science, technology, engineering and mathematics (STEM) curricula, by interweaving threads from four research areas: 1) cognitive science, 2) curriculum development, 3) instructional practice and teacher preparation, and 4) educational measurement and psychometric modeling. We propose to outline a comprehensive theoretical and empirical approach for evaluating multiple aspects of validity of assessments that are embedded within STEM curricula, to understand: (a) what constitutes quality assessment in STEM disciplines and how it supports student learning in an ongoing and coherent instructional process tied to nationally defined standards, and (b) what teachers need to know to make effective use of such assessments in their practice, and c) what aspects of embedded assessments are most salient for diverse student populations. A primary project deliverable is a monograph targeted to researchers, evaluation specialists, curriculum developers, teacher preparation programs, and teachers that will unify and extend disciplinary understandings; identify important questions, issues and problems relevant to informative assessment for future research; and articulate a broad-based multidisciplinary research agenda for informative assessment.

Setting: 

No primary data collection is planned for this study. We do intend to consider sources of data gathered and analyzed by other projects. We will determine which data to consider in consultation with the Expert Panel. The chief criterion for selecting data analyses to consider is their relevance to validity issues for embedded assessments.

Research Design: 

This synthesis research employs meta-analysis. The particular foci of this research are the validity and practice aspects of curriculum embedded assessments in STEM areas. The extant research literatures devoted to embedded assessment validity, role and use are surprisingly sparse across the four areas of cognitive science, curriculum development, teacher preparation and instructional practice, and educational measurement. The substantive and education literature within the STEM disciplines are similarly muted with respect to aspects of embedded assessment. In addition to meta-analyses, this study will identify transformative studies within the literature, together with as broad a set of case examples, both real cases and hypothetical cases, that demonstrate and highlight what already is known in this area as well as what research questions deserve intensive attention.

Examination of secondary data sources typically will include existing external analyses as performed and reported within the literature. Our research here intends to inventory, interpret and critique existing analyses. In some situations we will offer alternative analytic approaches, including cognitive diagnostic psychometric modeling, grounded theory analyses of teacher and student data, and multi-tiered data collection and research designs that combine the strengths of qualitative small scale studies and psychometric large scale data analyses.

Findings: 

To date, this project has not generated findings.

Other Products: 

A primary deliverable will be a monograph with a cross-disciplinary consideration of all relevant aspects of understanding, developing and evaluating curriculum-based informative assessments, including a set of illustrative case examples, an inventory of existing research approaches, and a multidisciplinary research agenda for embedded assessment. This monograph will be targeted to both research and practitioner audiences. Supporting items will be made publicly available on a project web page including: transcriptions of expert panel discussions, portfolio of real and hypothetical examples of informative classroom assessment, appendices for the monograph, written reflections and materials collected by panelists, including annotated references, existing data analyses, and links to web pages. Papers will be submitted for publication in respected refereed journals, and one or more symposia or workshops will be proposed to professional meetings such as AERA, NCME, NCTM, NSTA.