Join us for a seminar by Marcus Gubanyi and Leen-Kiat Soh and learn about insights into programming behaviors during exams and implications for assessing programming ability in introductory computer science courses.
Thursday, October 31st 2-3 pm CT
Kiewitt Hall A251 or via Zoom
Title:
Compile-and-Run Prompts: Exploring Programming Process Data during Exams in CS1
Abstract:
Compile-and-run prompts are designed to authentically assess programming ability by providing an integrated development environment for students to write code to solve a problem. Recent literature supports the use of compile-and-run prompts to assess programming ability during exams. However, there is a lack of investigation into student programming behaviors during exams, which can provide valuable insights into the construct validity of these assessments in measuring programming ability. Our study examines the authenticity and validity of compile-and-run prompts on exams by exploring programming process data from assignments and exams in a CS1 course. While students program, keystroke and compilation, data are collected. We explore the student experience during programming exams by calculating compiler error metrics and analyzing student behaviors derived from the data. We find evidence that students behave differently during exams as compared to baseline assignments. We also find behavioral differences between lower-performing and higher-performing students during exams. Based on these differences in programming behaviors, we present important implications for future work and make recommendations for assessing programming ability in introductory courses.