S-CASTS: A System for Collaboration Among Students, Teachers and Software

Principal Investigator: 
Co-Investigator: 
Project Overview
Background & Purpose: 

This project embarks on a research program aimed at investigating the use of models of collaboration, especially as embodied in collaborative human-computer interface systems, in the augmentation of existing flexible software tools for mathematics education. The proposed effort will draw on prior work and expertise in two complementary arenas: (1) mathematics education: the development and study of representational tools for data and probability by math educators; and (2) AI multi-agent systems research: the development of a theoretical model of collaboration and its deployment in the design of collaborative human-computer interfaces. Bringing together results and software developed in these complementary arenas has the potential to create powerful tools that would support both the thoughtful analysis of probabilistic models by students and increased ability of teachers to identify those students who would benefit from teacher advice.

Research Design: 

The research design for this project is comparative, and is designed to generate evidence which is descriptive (design research). This project collects original data using videography and web log observations. Plan recognition algorithms address the problem of automatically inferring student activities through software interaction. We developed four plan recognition algorithms to determine whether students solved assigned problems with pedagogical software and to present students’ strategies in the case of success. Each algorithm compares user interaction histories to a set of potential strategies, or recipes, developed by domain experts. We compared two general types of algorithms, 1) those that use heuristics to find and describe user plans in polynomial time based on the length of the user log, and 2) complete algorithms that are guaranteed to find the right plan (if one exists) but in the worst case are exponential in the length of the log.

Findings: 

We developed and evaluated four original algorithms for recognizing user’s activities within solving probability problems using TInkerPlots a commercial pedagogical software used internationally to support statistics education. We carried out empirical evaluations of the four algorithms, based on 43 user interactions with TinkerPlots,. We found an expected tradeoff between computational complexity and accuracy; the polynomial algorithm identified user activities in 27 of 43 user interactions (63%), while the other three increasingly exponentially algorithms identified activities in 33 (77%), 40 (93%), and 40 (93%) interactions, respectively. We also examined the problem of expressing potential user strategies, or recipes, in a compact recipe language, and discussed the limitations arising from various recipe language assumptions.

Our laboratory study showed that under controlled conditions, plan recognition algorithms can be used pragmatically despite theoretical worst-case exponential complexity. Our next step is to see whether their performance can generalize to classroom settings involving middle school students using TinkerPlots to solve statistics problems of varying kinds and complexities. These conditions are more challenging in that students' interaction process with TinkerPlots is longer and more involved, and that the size of the recipe data base will be considerably larger than in our lab study.

Examining the results of the plan recognition process has also led us to consider two branches to our work that were not included in our original conceptualization of the project.

  1. Plan recognition, while it can accurately identify a student’s plan, essentially hides student actions that are not part of the plan. We also know that as the problems that students are solving become more complex, the recipe library becomes unwieldy, difficult to assemble and difficult to use. These two realizations have led us to investigate other representations of students’ work that do not rely on plan recognition. This will be the focus of the next year of the project.
  2. We have spoken to a number of people who do research on statistical thinking using TinkerPlots and they have expressed an interest in the kinds of results we can generate. It may be, in fact, that researchers will be able to take advantage of our work at least as much as teachers. We will thus be expanding our work to consider researchers as an audience as well as teachers.
Publications & Presentations: 

Y. Gal, E. Yamangil, A. Rubin, S. Shieber, and B.J. Grosz. Towards Collaborative Intelligent Tutors: Aautomated Recognition of User's Strategies. Proceedings of the Ninth International Conference on Intelligent Tutoring Systems. Montreal, Quebec, June 2008.

S. Reddy, “Constraint Satisfaction-Aided Plan Recognition,” A.B. Thesis, Harvard University, Cambridge, MA, 2009.

S. Reddy, Y. Gal, and S. Shieber. Recognition of User's Activities Using Constraint Satisfaction. Proceedings of the Seventeenth International Conference on User Modeling, Adaptation, and Personalization. Trento, Italy, June 2009.

Other Products: 

By the end of the project, we will have a set of “analysis and presentation” tools that will provide teachers and researchers with organized, useful information about students’ work with educational software. This information will be available both for individual students and for the students as a group (i.e. aggregate information).