Variance Almanac of Academic Achievement

Assigning groups to treatment conditions is increasingly common in social science research. In education, the groups assigned are often schools. Student achievement may vary systematically across schools. When it does, the intraclass correlation coefficient (ICC) measures how closely students within schools resemble each other with respect to their academic achievement. These ICCs are a key input to statistical power analyses (see Hedges & Hedberg, 2007).

This website provides access to the Variance Almanac of Academic Achievement (VA), a compendium of ICCs and related variance components from a variety of U.S. national datasets that span kindergarten through the 12th grade. The VA, developed by Professor Larry V. Hedges with support from the National Science Foundation, provides information for designing experiments that have adequate statistical power and precision to identify the effects of interventions on learning and instruction in specific locales.

Hedges and Eric Hedberg have developed an ARC online interface which enables users to access the ICC’s necessary for statistical power calculations using a six-step, menu-driven process. To use this resource you will need to specify the subject matter (reading or mathematics); region of the country in which your research will be conducted (Midwest, Northeast, South, West, or all regions); the "urbanicity" of the school setting (whether the schools in your project are located in urban, rural, suburban, or all urbanicities); any special subsample of interest (low achievement, low SES, or all schools); and grade level. You will also need to select a data source for the ICCs (i.e., which of a predefined list of major national datasets should be used to generate your design parameters). A complete set of instructions for utilizing this resource is provided in the VA User Guide. The Guide also provides information on how the ICCs were calculated, and instructions for downloading and using a Stata® program, RDPOWER, to calculate statistical power for cluster randomized trials using VA ICCs.

 

This research was supported by the National Science Foundation under Award Nos. 0129365 and 0815295. Additional support was provided to Hedges by the Institute of Education Sciences under Award No. R305D110032. Any opinions, findings, and conclusions or recommendations expressed here are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or Institute of Education Sciences.