SGER: Storytelling Alice and Game Design in Middle School

Principal Investigator: 
Co-Investigator: 
Project Overview
Background & Purpose: 

The purpose of the study was to collect pilot data to inform a larger study of how game design in pairs using Storytelling Alice (SA) can build IT fluency among middle school students. Our main research questions were: 1) Can Storytelling Alice (SA) be used by middle school students to design and program a game?; 2) In what ways does making a game with SA appear to promote fluency with information technology (IT)?; 3) Does SA motivate and engage middle school students from groups that are underrepresented in computer science (e.g., females and Latinos)?; 4) How can students be taught to use pair programming in conjunction with SA?; and 5) What kind of guidance or support do teachers need to provide students to create a game with SA?

Setting: 

During the summer of 2008, we conducted two courses for middle-school girls and boys. The two 2-week courses ran simultaneously for two hours each weekday; one was located at a community center in a town surrounded by a large agricultural area, and the other was located at a community center in a nearby small city.

Research Design: 

The research design for this project is comparative, and is designed to collect data that is descriptive using design research and observation. The project involves an intervention, which is computer game design in pairs. This project collects original data using assessments of learning/achievement tests, personal observation, survey data including online self-completion questionnaires, and face-to-face semi-structured/informal interviews, student games, and computer logging data.

Quantitative and qualitative data were collected in five main ways:

  • Student pre-post survey and skill assessment developed for this project, with some items adapted from a survey and assessment created by Caitlin Kelleher, creator of Storytelling Alice
  • Computer logging data to capture students’ actions while using SA; the logging program was provided by Dr. Kelleher
  • Game coding to assess “gameness” and aspects of IT fluency; we adapted our own rules of “gameness” from Juul’s six defining qualities for games
  • Recording of daily observations of student actions and interactions using structured observation templates
  • Face-to-face semi-structured interviews with the two course teachers about the success of our approach and goals
  • Face-to-face semi-structured interviews with a sample of 11 students about what they liked and disliked about the class.

Focusing on the five key research questions, quantitative data from the pre-post student survey were combined with qualitative data from interviews and observations to identify trends and themes relevant to each question. The survey was analyzed with SPSS by using paired t-tests to determine whether there was a significant difference from pre- to post-test among students who completed the class, and to compare those who dropped with those who remained in the class. The focus of the qualitative data analysis (e.g., interviews and observations) was to identify themes and sub-themes relevant to each of our five research questions. To assess the “gameness” of the 23 programs, each was coded using five defining qualities for games and two aspects of IT fluency. The two PIs coded each game separately, and disagreements were resolved by discussion. We also used computer logging data to understand the process through which programming a game can promote IT fluency. For each pair or solo, we identified two dates on which there were sufficient data to analyze: one at the beginning of the course, and one at the end. These data were parsed to reveal frequency of actions related to programming, scene layout, and housekeeping instructions.

Findings: 

Quantitative and qualitative data relevant to the key research questions indicate that: 1) SA motivates and engages middle school students; 2) the use of challenges (checklists guiding students through various aspects of game design) is an effective way to teach game design using SA; 3) training is needed for teachers to support students in learning game design using SA; and 4) pair programming is a useful approach to teaching students to design games using SA. In summary, the findings from this pilot study suggest that: 1) SA can be used by middle-school students with limited computer experience to build games; 2) this process can engage students in some critical aspects of IT fluency: algorithmic thinking, modeling, and abstraction; and 3) further research is needed to determine whether programming a computer game leads to gains computational thinking, and under what conditions pair programming produces greater gains.

Publications & Presentations: 

Papers have been accepted at four national conferences:

Denner, J., Martinez, J. & Werner, L. (2009, March). Teaching computer game design to middle school students. Game Developers’ Conference. San Francisco.

Werner, L., Denner, J., Bliesner, M. & Rex, P. (2009, April) Can middle-school students use Storytelling Alice to make games? Results of a pilot study. Fourth International Conference on the Foundation of Digital Games, Port Canaveral, FL.

Denner, J., Malyn-Smith, J. & Werner, L. (2009, April). Information and communications technology fluency: Defining and measuring standards in middle school. American Educational Research Association. San Diego, CA.

Denner, J., Werner, L., & Rex, P. (2009, June). Teaching computer game design using Storytelling Alice. National Educational Computing Conference. Washington, DC.

Other Products: 

We are finishing up a one-year curriculum for educators to teach students how to create games using Storytelling Alice.