Automated Feedback for Student Math Responses Based on Multi-Modality and Fine-Tuning

May 6, 2024

Automated Feedback for Student Math Responses Based on Multi-Modality and Fine-Tuning

Abstract

Open-ended mathematical problems are a commonly used method for assessing students’ abilities by teachers. In previous automated assessments, natural language processing focusing on students’ textual answers has been the primary approach. However, mathematical questions often involve answers containing images, such as number lines, geometric shapes, and charts. Several existing computer-based learning systems allow students to upload their handwritten answers for grading. Yet, there are limited methods available for automated scoring of these image-based responses, with even fewer multi-modal approaches that can simultaneously handle both texts and images. In addition to scoring, another valuable scaffolding to procedurally and conceptually support students while lacking automation is comments. In this study, we developed a multi-task model to simultaneously output scores and comments using students’ multi-modal artifacts (texts and images) as inputs by extending BLIP, a multi-modal visual reasoning model. Benchmarked with three baselines, we fine-tuned and evaluated our approach on a dataset related to open-ended questions as well as students’ responses. We found that incorporating images with text inputs enhances feedback performance compared to using texts alone. Meanwhile, our model can effectively provide coherent and contextual feedback in mathematical settings.

Download Complete PDF

Authors

Hai Li
University of Florida

Chenglu Li
University of Utah

Wanli Xing
University of Florida
wanli.xing@coe.ufl.edu

Sami Baral
Worcester Polytechnic Institute

Neil Heffernan
Worcester Polytechnic Institute

A Fair Clustering Approach to Self-Regulated Learning Behaviors in a Virtual Learning Environment

May 6, 2024

Return to Spotlight Research

A Fair Clustering Approach to Self-Regulated Learning Behaviors in a Virtual Learning Environment

Abstract

While virtual learning environments (VLEs) are widely used in K-12 education for classroom instruction and self-study, young students’ success in VLEs highly depends on their self-regulated learning (SRL) skills. Therefore, it is important to provide personalized support for SRL. One important precursor of designing personalized SRL support is to understand students’ SRL behavioral patterns. Extensive studies have clustered SRL behaviors and prescribed personalized support for each cluster. However, limited attention has been paid to the algorithm bias and fairness of clustering results. In this study, we “fairly” clustered the behavioral patterns of SRL using fair-capacitated clustering (FCC), an algorithm that incorporates constraints to ensure fairness in the assignment of data points. We used data from 14,251 secondary school learners in a virtual math learning environment. The results of FCC showed that it could capture six clusters of SRL behaviors in a fair way; three clusters belonging to high-performing (i.e., H-1. Help-provider, H-2) Active SRL learner, H-3) Active onlooker), and three clusters in low-performing groups (i.e., L-1) Quiz-taker, L-2) Dormant learner, and L-3) Inactive onlooker). The findings provide a better understanding of SRL patterns in online learning and can potentially guide the design of personalized support for SRL.

Download Complete PDF

Authors

Yukyeong Song
University of Florida

Chenglu Li
University of Utah

Wanli Xing
University of Florida
wanli.xing@coe.ufl.edu

Shan Li
Lehigh University

Hakeoung Hannah Lee
The University of Texas

Learner experience and motivational beliefs for a VR Lab in advanced undergraduate biology

May 6, 2024

Return to Spotlight Research

Learner experience and motivational beliefs for a VR Lab in advanced undergraduate biology

Abstract

Recently, interest in understanding the prospects of virtual environments, specifically virtual reality laboratories (VR Lab) – those that involve a first-person practica experience via a desktop computer or head-mounted display – in science education has increased. Seemingly ingrained in the process of implementing a VR Lab is a supposition of interest, intrinsic motivation, and confidence on the part of participants. These attributes influence the participant’s ability to start and complete the task and support sensemaking and meaningful learning. Thus, this study used expectancy-value theory with the intent of learning experience design to assess the relationship between task value beliefs and those for usability of a VR Lab. Beliefs were also compared to those for physical laboratories, where only a higher cost for the VR Lab was found. Increased cost beliefs suggest that participants perceived the VR Lab as requiring more from them, a negative consequence, which would decrease their potential for seeing it as a viable learning alternative. This finding suggests that a student’s level of content knowledge may influence thier VR Lab experience. Cost belief was significantly related to utility value, giving credence to the theoretical model where cost is a value component. The VR Lab was determined to be marginally usable, but usability was not related to any motivational beliefs.

Download Complete PDF

Authors

Shalaunda M. Reeves
University of Tennessee Knoxville

Charlotte A. Bolch
Midwestern University

Richard T. Bex II
Illinois State University

Kent J. Crippen
University of Florida
kcrippen@coe.ufl.edu

Using Ant Colony Optimization to Identify Optimal Sample Allocations in Cluster-Randomized Trials

May 6, 2024

Return to Spotlight Research

Using Ant Colony Optimization to Identify Optimal Sample Allocations in Cluster-Randomized Trials

Abstract

When designing cluster-randomized trials (CRTs), one important consideration is determining the proper sample sizes across levels and treatment conditions to cost-efficiently achieve adequate statistical power. This consideration is usually addressed in an optimal design framework by leveraging the cost structures of sampling and optimizing the sampling ratios across treatment conditions and levels of the hierarchy. Traditionally, optimization is done through the first-order derivative approach by setting the first-order derivatives equal to zero to solve for the optimal design parameters. However, the first-order derivative method is incapable of properly handling the optimization task when statistical power formulas are complex, such as those for CRTs detecting mediation effects under the joint significance test. The current study proposes using an ant colony optimization (ACO) algorithm to identify optimal allocations. We evaluate the algorithm’s performance for CRTs detecting main and mediation effects. The results show that the ACO algorithm can identify optimal sample allocations for CRTs investigating main effects with the same design efficiency as those identified through the first-order derivative method. Furthermore, it can efficiently identify optimal sample allocations for CRTs investigating mediation effects under the joint significance test. We have implemented the proposed methods in the R package odr.

Download Complete PDF

Authors

Zuchao Shen
University of Georgia
zuchao.shen@gmail.com

Walter L. Leite
University of Florida
walter.leite@coe.ufl.edu

Huibin Zhang
University Tennessee Knoxville

Jia Quan
University of Kansas

Huan Kuang
Florida State University

Automatically Detecting Confusion and Conflict During Collaborative Learning Using Linguistic, Prosodic, and Facial Cues

May 6, 2024

Return to Spotlight Research

Automatically Detecting Confusion and Conflict During Collaborative Learning Using Linguistic, Prosodic, and Facial Cues

Abstract

During collaborative learning, confusion and conflict emerge naturally. However, persistent confusion or conflict have the potential to generate frustration and significantly impede learners’ performance. Early automatic detection of confusion and conflict would allow us to support early interventions which can in turn improve students’ experience with and outcomes from collaborative learning. Despite the extensive studies modeling confusion during solo learning, there is a need for further work in collaborative learning. This paper presents a multimodal machine-learning framework that automatically detects confusion and conflict during collaborative learning. We used data from 38 elementary school learners who collaborated on a series of programming tasks in classrooms. We trained deep multimodal learning models to detect confusion and conflict using features that were automatically extracted from learners’ collaborative dialogues, including (1) language-derived features including TF-IDF, lexical semantics, and sentiment, (2) audio-derived features including acoustic-prosodic features, and (3) video-derived features including eye gaze, head pose, and facial expressions. Our results show that multimodal models that combine semantics, pitch, and facial expressions detected confusion and conflict with the highest accuracy, outperforming all unimodal models. We also found that prosodic cues are more predictive of conflict, and facial cues are more predictive of confusion. This study contributes to the automated modeling of collaborative learning processes and the development of real-time adaptive support to enhance learners’ collaborative learning experience in classroom contexts.

Download Complete PDF

Authors

Yingbo Ma
University of Florida

Yukyeong Song
University of Florida

Mehmet Celepkolu
University of Florida
mckolu@ufl.edu

Kristy Elizabeth Boyer
University of Florida
keboyer@ufl.edu

Eric Wiebe
North Carolina State University

Collin F. Lynch
North Carolina State University

Maya Israel
University of Florida
misrael@coe.ufl.edu