Research
Educational research and case studies using PrairieLearn
- Actually achieving “A’s for All” (as time and interest allow), D. Garcia, A. Fox, P. Lopez, M. Silva, C. Zilles, and E. Ambrosio. SIGCSE 2026.
- A generalized framework for describing question randomization, R. Mahinpei, I. Xu, S. Wolfman, F. Moosvi. SIGCSE 2024
- How much deadline flexibility on formative assessments should we be giving to our students?, C. Zhao, M. West, and M. Silva. ASEE 2023.
- A' for All (as time and interest allow), D. Garcia. A. Fox, C. Zilles, M. West, M. Silva, N. Terrell, S. Russell, E. Ambrosio, and F. Shakir. SIGCSE 2023
- Integrating diverse learning tools using the PrairieLearn platform, M. West, N. Walters, M. Silva, T. Bretl, and C. Zilles. SIGCSE 2021
- A quantitative analysis of when students choose to grade questions on computerized exams with multiple attempts, A. Verma, T. Bretl, M. West, and C. Zilles. L@S 2020.
- Caches as an example of machine-gradable exam questions for complex engineering systems, S. Mahmood, M. Zhao, O. Khan and G. Herman. FIE 2020.
- A simple and efficient markup tool to generate drawing-based online assessments, N. Nytko, M. West, and M. Silva. ASEE 2020.
- Predicting the difficulty of automatic item generators on exams from their difficulty on homeworks, B. Chen, M. West, and C. Zilles. L@S 2019.
- Reducing difficulty variance in randomized assessments, P. Sud, M. West, and C. Zilles. ASEE 2019.
- Algorithmic grading strategies for computerized drawing assessments, M. Silva and M. West. ASEE 2017.
- PrairieLearn: Mastery-based online problem solving with adaptive scoring and recommendations driven by machine learning, M. West, G. L. Herman, and C. Zilles. ASEE 2015.
- How students focus their studying when offered exam re-takes, L. Flygare, D. H. Smith, G. Herman, M. Fowler, and C. Zilles. ITiCSE 2026.
- Frequent testing vs. second-chance testing: An exploration, G. Herman, K. Patel, C. Emeka, C. Zilles, and M. West. ICER 2025.
- Exploring different specifications grading policies, I. dos Santos Montagner, R. C. Ferrão, C. Zilles, and M. Silva. SIGCSE 2025.
- Evaluating mastery-oriented grading in an intensive CS1 course, I. dos Santos Montagner, R. C. Ferrão, A. Kurauchi, M. Silva, and C. Zilles. SIGCSE 2024.
- Determining the best policies for second-chance tests for STEM students, C. Emeka, D. Smith, C. Zilles, M. West, G. L. Herman, and T. Bretl. ASEE 2023.
- Second-chance testing as a means of reducing students' test anxiety and improving outcomes, C. Emeka, C. Zilles, M. West, G. Herman, and T. Bretl. ASEE 2023.
- Investigating the effects of testing frequency on programming performance and students' behavior, D. Smith, C. Emeka, M. Fowler, M. West, and C. Zilles. ASEE 2023.
- First try, no (autograder) warm up: motivating quality coding submissions, L. Butler and G. Herman. ASEE 2023.
- Students perceptions and behavior related to second-chance testing, C. Emeka, T. Bretl, G. Herman, M. West, and C. Zilles. FIE 2021.
- Comparison of grade replacement and weighted averages for second-chance exams, G. Herman, Z. Cai, T. Bretl, C. Zilles, and M. West. ICER 2020.
- Frequent mastery testing with second-chance exams leads to enhanced student learning in undergraduate STEM, J. Morphew, M. Silva, G. Herman, and M. West. Applied Cognitive Psychology 2019.
- Second-chance testing course policies and student behavior, G. Herman, K. Varghese, and C. Zilles. FIE 2019.
- Experiences with computer-based testing (CBT), J. Sosnowski, A. Fox, D. Garcia, F. Moosvi, M. Silva, M. West, and C. Zilles. SIGCSE 2025.
- Do centralized testing centers influence test anxiety for engineering students?, C. Emeka, M. West, J. Sosnowski, G. Herman, C. Zilles, and M. Silva. ASEE 2025.
- Measuring test anxiety of two computerized exam approaches, C. Emeka, C. Zilles, J. Sosnowski, M. West, G. Herman, and M. Silva. SIGCSE 2025.
- One solution to addressing assessment logistical problems: An experience setting up and operating an in-person testing center, K. Downey, K. Miller, M. Silva, and C. Zilles. SIGCSE 2024.
- Reflections on 10 years of operating a computer-based testing facility: Lessons learned, best practices, J. Sosnowski, J. Baker, O. Arnold, M. Silva, D. Mussulman, C. Zilles, and M. West. ASEE 2024.
- Unpacking the influence of computer-based testing modalities on student study behaviour and performance, R. Gulati, C. Zilles, M. West, and M. Silva. EDULEARN 2024.
- Computerized exam reviews: In-person and individualized feedback to students after a computerized exam, W. L. Chang, M. West, C. Zilles, D. Mussulman, and C. Sacris. ASEE 2020.
- Every university should have a computer-based testing facility, C. Zilles, M. West, G. Herman, and T. Bretl. CSEDU 2019.
- Student and instructor experiences with a computer-based testing facility, C. Zilles, M. West, D. Mussulman, and C. Sacris. EDULEARN 2018.
- Making testing less trying: Lessons learned from operating a computer-based testing facility, C. Zilles, M. West, D. Mussulman, and T. Bretl. FIE 2018.
- Using a computer-based testing facility to improve student learning in a programming languages and compilers course, T. Nip, E. Gunter, G. Herman, J. Morphew, and M. West. SIGCSE 2018.
- Measuring revealed student scheduling preferences using constrained discrete choice models, J. Bailey, M. West, and C. Zilles. ASEE 2017
- Modeling student scheduling preferences in a computer-based testing facility, M. West and C. Zilles. L@S 2016.
- Computerized testing: A vision and initial experiences, C. Zilles, R. T. Deloatch, J. Bailey, B. B. Khattar, W. Fagen, C. Heeren, D. Mussulman, and M. West. ASEE 2015.
- Student behavior in selecting an exam time in a computer-based testing facility, C. Zilles, M. West, and D. Mussulman. ASEE 2015.
- WIP: Low effort, high grades? Benchmarking LLMs on various engineering assignments, Y. Chen, S. Eggl, A. Alawini, M. Silva, M. Fowler, A. Umrawal, and M. Ornik. ASEE 2026.
- A case for Bayesian grading, C. Zilles, C. Zhao, Y. Chen, E. M. Matthews, and M. West. SIGCSE Virtual 2024.
- Plagiarism in the age of generative AI: Cheating method change and learning loss in an intro to CS course, B. Chen, C. M. Lewis, M. West, and C. Zilles. L@S 2024.
- Comparing the security of three proctoring regimens for Bring-Your-Own-Device exams, R. Gulati, M. West, C. Zilles, and M. Silva. SIGCSE 2024.
- Comparing student outcomes in online vs. in-person sections of an on-campus computer science course, R. Gulati, M. West, C. Zilles, and M. Silva. ASEE 2023.
- Are we fair? Quantifying score impacts of computer science exams with randomized question pools, M. Fowler, D. Smith, C. Emeka, M. West and C. Zilles. SIGCSE 2022.
- Learning to cheat: Quantifying changes in score advantage of unproctored assessments over time, B. Chen, S. Azad, M. Fowler, M. West, and C. Zilles. L@S 2020.
- Measuring the score advantage on asynchronous exams in an undergraduate CS course, M. Silva, C. Zilles, and M. West. SIGCSE 2020.
- Analyzing the decline of student scores over time in self‐scheduled asynchronous exams, B. Chen, M. West, and C. Zilles. Journal of Engineering Education, 2019.
- How much randomization is needed to deter collaborative cheating on asynchronous exams?, B. Chen, M. West, and C. Zilles. L@S 2018.
- Do performance trends suggest wide-spread collaborative cheating on asynchronous exams?, B. Chen, M. West, and C. Zilles. L@S 2017.
- Consistently good vs. occasionally great: A rubric for open-ended feedback quality from humans and machines, B. Chen, R. Haldar, M. Fowler, M. West, and C. Zilles. arXiv, 2026.
- Using LLM to autograde diagrams, R. C. Ferrão, I. dos Santos Montagner, M. Silva, and C. Zilles. ITiCSE 2026.
- Exploring intentional ambiguity to exploit generative AI grading, J. Gao, L. Flygare, and C. Zilles. AIED 2026.
- You don’t need a data center to Explain in Plain English! Comparing open-source and proprietary LLMs for EiPE grading, E. Jiang and M. Fowler. SIGCSE 2026.
- Automated grading of handwritten mathematics using vision-capable LLMs, J. Levine, M. Aenlle, C. Zilles, M. West, and M. Silva. AIED 2026.
- AI-supported grading and rubric refinement for free response questions, C. Zhao, M. Fowler, Y. Gertner, S. Poulsen, M. West, and M. Silva. SIGCSE 2026.
- A two-stage LLM pipeline for handwritten mathematics autograding, J. Levine, M. West, and M. Silva. SIGCSE 2026.
- Evaluating AI models for autograding Explain in Plain English questions: Challenges and considerations, M. Fowler, C. Emeka, B. Chen, D. Smith, M. West, and C. Zilles. ACM TiiS 2025.
- LLM agents for verifiable question generation and grading, J. Levine, M. West, and M. Silva. AIED 2025.
- Language models are few-shot graders, C. Zhao, M. Silva, and S. Poulsen. AIED 2025.
- Autograding mathematical induction proofs with natural language processing, C. Zhao, M. Silva, and S. Poulsen. International Journal of Artificial Intelligence in Education, 2025.
- Expanding the horizons of autograding: Innovative questions at UBC, J. Niu, J. Wong, C. Lake, J. Rahardjo, H. Zarkoob, O. Ola, P. Belleville, K. Mochetti, M. Allen, F. Moosvi, and S. Wolfman. SIGCSE 2025.
- Evaluating Large Language Model code generation as an autograding mechanism for “Explain in Plain English” questions, D. Smith and C. Zilles. SIGCSE 2024
- Counting the trees in the forest: Evaluating prompt segmentation for classifying code comprehension level, D. H. Smith, M. Fowler, P. Denny, and C. Zilles. ITiCSE 2025.
- ReDefining code comprehension: Function naming as a mechanism for evaluating code comprehension, D. H. Smith, M. Fowler, P. Denny, and C. Zilles. ITiCSE 2025.
- Am I wrong, or is the autograder wrong? Effects of AI grading mistakes on learnings, T. Li, S. Hsu, M. Fowler, Z. Zhang, C. Zilles, and K. Karahalios. ICER 2023.
- Peer-grading Explain in plain English questions: A Bayesian calibration method for categorical answers, B. Chen, M. West, and C. Zilles. SIGCSE 2022.
- Autograding Explain in Plain English questions using NLP, M. Fowler, B. Chen, S. Azad, M. West, and C. Zilles. SIGCSE 2021.
- How should we ‘Explain in Plain English’? Voices from the community, M. Fowler, B. Chen, and C. Zilles. ICER 2021.
- A validated scoring rubric for Explain-in-Plain-English questions, B. Chen, S. Azad, R. Haldar, M. West, and C. Zilles. SIGCSE 2020.
- An R autograder for PrairieLearn, D. Eddelbuettel and A. Barbehenn. arXiv, 2020.
- Implementing a tool for structured roles in hybrid collaborative learning environments, C. Zhao, Y. Chen, K. Feng, G. Herman, M. West, and M. Silva. ASEE 2025.
- Exploring computing students' sense of belonging before and after a collaborative learning course, M. Fong, S. Huang, A. Alawini, M. Silva, and G. Herman. SIGCSE 2024.
- Developing tools, pedagogies, and policies for computer-based collaborative learning activities, M. Fong, L. Butler, A. Alawini, G. Herman, and M. Silva. ASEE 2023.
- An analytic comparison of student-scheduled and instructor-scheduled collaborative learning in online contexts, G. Herman, Y. Jiang, Y. Jiang, S. Poulsen, M. West, and M. Silva. ASEE 2022.
- Enabling open educational resource adoption through integrated sharing in PrairieLearn, S. Poulsen, G. Herman, M. Silva, M. Fowler, D. Smith, L. Porter, N. Ritschel, C. Zilles, and M. West. SIGCSE 2026.
- "I don’t gamble to make my livelihood”: Understanding the incentives for, needs of, and motivations surrounding open educational resources in computing, M. Fowler, D. Smith, B. Chen, and C. Zilles. ICER 2023.
- On generating and validating erroneous examples in CS1 using LLMs, Y. Chen, C. Zhao, J. Levine, K. Feng, M. Fowler, and M. Silva. AIED 2026.
- Exploring LLMs for generating erroneous examples in CS1, Y. Chen, C. Zhao, K. Feng, J. Zhang, V. Malhotra, and M. Silva. SIGCSE 2026.
- A complete redesign of CS1 for engineering students, Y. Chen, C. Zhao, K. Feng, M. Beckman, and M. Silva. ASEE 2025.
- On teaching novices computational thinking by utilizing large language models within assessments, M. Hassan, Y. Chen, P. Denny, and C. Zilles. SIGCSE 2025.
- Dynamic, randomizable, autogradable visual programming simulations for Python using PrairieLearn, N. Chulo, G. Classon, A. Chiu, D. Garcia, A. Fox, and N. Norouzi. SIGCSE 2025.
- An interactive tool for randomized autogradable graph assessments, E. Hasanov, D. Ahluwalia, D. Garcia, N. Norouzi, and A. Fox. SIGCSE 2025.
- Evaluating how novices utilize debuggers and code execution to understand code, M. Hassan, G. Zeng, and C. Zilles. ICER 2024.
- Discovering, autogenerating, and evaluating distractors for Python Parsons problems in CS1, D. Smith and C. Zilles. SIGCSE 2023.
- On Students' Ability to resolve their own tracing errors through code execution, M. Hassan and C. Zilles. SIGCSE 2023.
- Reevaluating the relationship between explaining, tracing, and writing skills in CS1 in a replication study, M. Fowler, D. Smith, M. Hassan, S. Poulsen, M. West, and C. Zilles. Computer Science Education 2022.
- Investigating the impact of automated code quality feedback in an embedded systems course, R. C. Ferrão, I. dos Santos Montagner, R. Azevedo, M. Silva, and C. Zilles. Koli Calling 2025.
- Embedded-check: A code quality tool for automatic firmware verification, R. C. Ferrão, I. dos Santos Montagner, M. Silva, C. Zilles, and R. Azevedo. ITiCSE 2024.
- Micro-specialization as a solution to open-ended project, R. C. Ferrão, I. dos Santos Montagner, M. Silva, C. Zilles, and R. Azevedo. SIGCSE Virtual 2024.
- Teaching machine learning with repeated practice and rapid feedback in PrairieLearn, F. Fund and F. Moosvi. SIGCSE Virtual 2026.
- Paper or silicon: Assessing student understanding in a computer-based testing environment using PrairieLearn, J. Ardister, G. Recktenwald, and S. Roccabianca. ASEE 2024.
- Effects of integrating computational tools into an introductory engineering mechanics course, W. Chang, S. Ok, M. West, S. Hilgenfeldt, and M. Silva. ASEE 2024.
- Measuring the impact of a computational linear algebra course on students’ exam performance in a subsequent numerical methods course, H. Chen, M. West, S. Hilgenfeldt, and M. Silva. SIGCSE 2023.
- Innovating and modernizing a linear algebra class through teaching computational skills, M. Silva, P. Hieronymi, M. West, N. Nytko, A. Deshpande, J. Chuang, and S. Hilgenfeldt. ASEE 2022.
- A case study of early performance prediction and intervention in a computer science course, M. Silva, E. Shaffer, N. Nytko, and J. Amos. ASEE 2020.
- Disentangling the learning gains from reading a book chapter and completing Proof Blocks problems, S. Poulsen, Y. Gerner, H. Chen, B. Cosman, M. West, and G. Herman. SIGCSE 2024.
- Solving Proof Block problems using Large Language Models, S. Poulsen, S. Sarsa, J. Prather, J. Leinonen, B. Becker, A. Hellas, P. Denny, and B. Reeves. SIGCSE 2024.
- Using context-free grammars to scaffold and automate feedback in precise mathematical writing, J. Xia and C. Zilles. SIGCSE 2023.
- Efficiency of learning from Proof Blocks versus writing proofs, S. Poulsen, Y. Gerner, B. Cosman, M. West, and G. Herman. SIGCSE 2023.
- Efficient feedback and partial credit grading for Proof Blocks problems, S. Poulsen, S. Kulkarni, G. Herman, and M. West. AIED 2023.
- Proof Blocks: autogradable scaffolding activities for learning to write proofs, S. Poulsen, M. Viswanathan, G. Herman, and M. West. ITiCSE 2022.
- Benchmarking partial credit grading algorithms for Proof Blocks problems, S. Poulsen, S. Kulkarni, G. Herman, and M. West. AIED 2022.
- Evaluating Proof Blocks Problems as Exam Questions, S. Poulsen, M. Viswanathan, G. Herman, and M. West. ICER 2021.
- Uncovering patterns of SQL errors in student assignments: A comparative analysis of different assignment types, S. Yang, Z. Li, G. Herman, K. Cunningham, and A. Alawini. FIE 2023.
- Comparison of student learning outcomes among SQL problem-solving patterns, S. Yang, G. Herman, and A. Alawini. FIE 2023.
- Mining SQL problem-solving patterns using advanced sequence processing algorithms, S. Yang, G. Herman, and A. Alawini. DataEd 2023.
- Analyzing student SQL solutions via hierarchical clustering and sequence alignment scores, S. Yang, G. Herman, and A. Alawini. DataEd 2022.
- Insights from student solutions to MongoDB homework problems, R. Alkhabaz, S. Poulsen, M. Chen, and A. Alawini. ITiCSE 2021.
- Analyzing patterns in student SQL solutions via Levenshtein edit distance, S. Yang, Z. Wei, G. Herman, and A. Alawini. L@S 2021.
- A Quantitative analysis of student solutions to graph database queries, M. Chen, S. Poulsen, R. Alkhabaz, and A. Alawini. SIGCSE 2021.
- Insights from student solutions to SQL homework problems, S. Poulsen, L. Butler, A. Alawini, and G. Herman. ITiCSE 2020.
Contribute to this page
Do you have a paper that should be included on this page? Please send us the appropriate information at hello@prairielearn.com.
View demo course!
Explore the demo course to see how this all comes together