"The AIAI Challenge was a great success ... [It] spawned true collaboration between groups of researchers with different skill sets that are nonetheless committed to jointly tackling the same problem."
– James Drimalla, Assistant Professor of Education at the Herschend School of Education, Gordon College and organizer of the AIAI Challenge
The Why
For any complex skill, achieving excellence requires seeing performance as it actually occurred, not as practitioners experienced it or remember it. Just as athletes review game footage, or musicians listen to recordings, educators use classroom observation to help teachers hone their skills. Classroom observation can help educators objectively see critical moments that shape learning, like when an open-ended question revealed a misunderstanding, or when small group work would have worked better in a lesson than a whole-class activity.
In many schools, video classroom observation has replaced in-person observation, but processing classroom observation video remains time-consuming. Reviewers still need to watch the footage to make use of it.
The Solution
The goal of the Artificial Intelligence for Advancing Instruction (AIAI) Challenge was to build models to automate aspects of classroom observation and make it accessible to more teachers and teachers-in-training. Sponsored and organized by the Artificial Intelligence for Advancing Instruction Project and the University of Virginia, the challenge sought algorithms that could automatically label multimodal classroom observation data. Solvers developed machine learning models capable of identifying instructional activities and classroom discourse content from videos and anonymized audio transcripts.
The Results
The winning solutions in this competition pushed the boundary of the state-of-the-art in AI-assisted education, and established new collaborative networks among the organizers and winning teams. The top team achieved over 0.58 macro F1 score on the multi-phase test set. All three winning solutions used transformer-based architectures for both video and text data, and ensembled multiple models to improve performance.
See the results announcement for more information on the winning submissions and the teams who developed them. All of the prize-winning submissions from this competition are linked below and available for anyone to use and learn from.
RESULTS ANNOUNCEMENT + MEET THE WINNERS
Phase 1
AIAI Challenge - Phase 1
Classify instructional activities using multimodal classroom data. #education
Phase 2
AIAI Challenge - Phase 2
Classify instructional activities using multimodal classroom data. #education