An Integrated Model of Cognitive and Affective Scaffolding for Intelligent Tutoring Systems

Principal Investigator: 
Co-Investigator: 
Project Overview
Background & Purpose: 

This project explores a central question regarding one-on-one tutoring: how do expert tutors provide effective cognitive and affective scaffolding over the course of long-term tutorial interactions to improve learning? This question will be investigated through an extensive observational one-on-one human tutoring study from which an empirically grounded model will be machine-learned and then used directly in an intelligent tutoring system for introductory computer science. An experiment in the final year of the project with three versions of this tutoring system will identify the impacts of cognitive scaffolding alone, affective scaffolding alone, and combined cognitive plus affective scaffolding for students in particular subgroups of interest such as underrepresented groups and students with historically poor performance.

Setting: 

The students will be drawn from introductory computer science courses at the undergraduate level. The studies and experiments will be conducted across four institutions: one large public research university, one undergraduate minority-serving university, and two minority-serving colleges, all in the Raleigh, North Carolina area.

Research Design: 

This project has a comparative research design and will generate evidence that is descriptive [observational], associative or correlational [quasi-experimental], and causal [quasi-experimental using fixed-effects models]. Original data will be collected from undergraduate introductory computer science students using assessments of learning, observation [videography and web logs], survey research [self-completion questionnaires], and affect monitoring with affect detection bracelets.

Learning will be assessed with a pretest administered prior to the start of the experiment and a posttest administered at the conclusion of the experiment. The design of the pretest and posttest will be informed by the textbook utilized for the course, by the shared course objectives as stated in the syllabi at the primary institution and the three partner institutions, and by previous tests utilized in the course. The core concepts represented on the instrument will be aligned with the Association for Computing Machinery and IEEE Computer Society’s Computer Science Curriculum guidelines (ACM/IEEE Joint Task Force on Computing Curricula, 2008). The pretests and posttests will undergo expert review for content and difficulty by three instructors for introductory computer programming. The reliability will be established through an iterative process of testing, statistical analysis, and refinement during Years 1 and 2 of the project. The affective outcomes of interest include self-efficacy for computer science, computer science attitude, and intrinsic motivation. Domain-specific self-efficacy will be measured by a self-efficacy for computer science scale. It will be adapted directly from a self-efficacy for science scale (Bandura, 1997; Britner & Pajares, 2006). Computer science attitude will be measured by a computer science attitude survey developed by the co-PI (Wiebe, Williams, Yang, & Miller, 2003). Intrinsic motivation will be measured with an instrument adapted from the Intrinsic Motivation Inventory (IMI).

The experiment features a 2×2×2 full factorial between-subjects design. The underlying framework treats cognitive outcomes (learning) and affective outcomes (self-efficacy for computer science; computer science attitude; intrinsic motivation) as dependent variables. Additionally, students’ initial scores on the cognitive and affective instruments will be treated as covariates within the models, along with SAT-M scores as an indicator of general aptitude in computer science (Williams et al, 2002). A multivariate analysis of covariance (MANCOVA) will be used as the starting point in order to analyze the full model. The assumptions underlying MANCOVA will be verified before proceeding (Myers & Well, 2003), and any redundant covariates (i.e., those that are highly correlated with another covariate or that explain a very small amount of variance within the model) will be removed. Systematic differences in student populations between the partner institutions (described in detail above) with respect to initial cognitive and affective measures will also be conducted, and subsequent analyses will be conducted according to whether heterogeneity of these populations is present. For testing some of the hypotheses, the full MANCOVA model will be followed by appropriate ANCOVAs on individual dependent variables. Where appropriate, discriminant analysis will be used to further clarify the ANCOVA results.

Findings: 

Findings will be posted as they become available.

Other Products: 

JavaTutor intelligent tutoring system with cognitive and affective scaffolding.