Getting a question wrong might be the best thing that can happen to a student, if they try again.Researchers at Carnegie Mellon University's School of Computer Science (SCS) revealed how small design changes in online tutoring platforms can help students push through their mistakes and keep learning.The research team made small tweaks to things like text and color to see how these alterations improved students' persistence.Researchers in the HCII found that small design changes in online tutoring platforms can help students push through their mistakes and keep learning."Being wrong tends to be a very strong signal that people interpret as, 'This is not working, I'm going to stop,'" said Paulo Carvalho, an assistant professor in the Human-Computer Interaction Institute (HCII). "In fact, it's the opposite. If you're getting a practice question wrong, you're probably going to get the feedback you need to understand your mistake in the next step. But learners usually interpret being wrong as, 'I'm not learning,' and we stop. But if you stop, then you definitely don't learn. This research looks at that issue."The CMU team partnered with the South African nonprofit Siyavula, which offers an intelligent AI-based tutoring system, to test these interventions. The partnership was made possible through HCII Associate Professor of Learning Sciences Amy Ogan'sLearning Sciences for Innovators program.The researchers' paper, "Will They Try Again? A Large-Scale RCT on Scaffolds That Support Persistence in an Intelligent Tutoring System," received an honorable mention at the recent Association for Computing Machinery Conference on Human Factors in Computing Systems (CHI 2026).Michael Asher, a project scientist in the HCII, said the team looked to psychology-inspired design to create the two changes: a written prompt and a nudge. The written prompt appeared after the student got a question wrong, encouraging them to try again. The nudge, which also appeared after an incorrect answer, provided a visual cue prompting a user to choose a certain option — something familiar to nearly everyone because it's baked into human-computer interaction design. (For example, when users close a Microsoft Word document, the "Save" button is highlighted in blue.) In this research, the button that read "Try an exercise like this again" was highlighted in bright orange, nudging students to try again."We move through the world making decisions and we process tons of information, so we rely on a lot of shortcuts," Asher said. "Default options are powerful ways to simplify things, because we assume that they're defaults for a reason. So there's a whole industry of nudge-based interventions, which often change defaults to help someone make a decision that is desired, hopefully for beneficial reasons."This study was a randomized controlled trial involving about 160,000 students who used Siyavula's online tutoring system to complete 17 million practice problems. Researchers randomly assigned some students to receive the nudge, the written prompt, neither or both. They found that the prompt and nudge increased how often students persisted after failure by 2% and 9%, respectively.These increases represent more than just statistics: they are thousands of instances of people learning after getting a problem wrong. Stacking the prompt and nudge together had a powerful additive effect, leading students to persist 11% more often than those who received no help at all."What we found here — and this is something that was only possible because we were able to work with a really large sample to test this precisely — is that these two interventions actually stack on top of each other really nicely," Asher said. "The prompt seems to make people more likely to try again, and the nudge has an even larger effect. When you put them together, it's the sum of the parts, which we can really see because of our sample size."Along with Asher, Ogan and Carvalho, the research team included HCII doctoral student Yumou Wei and the Siyavula Foundation's Adam Reynolds. Learn more in the research paper on the CHI 2026 website.
Design Tweaks That Keep Students Learning
Getting a question wrong might be the best thing that can happen to a student, if they try again.Researchers at Carnegie Mellon University's School of Computer Science (SCS) revealed how small design changes in online tutoring platforms can help students push through their mistakes and keep learning.The research team made small tweaks to things like text and color to see how these alterations improved students' persistence.Researchers in the HCII found that small design changes in online tutoring platforms can help students push through their mistakes and keep learning."Being wrong tends to be a very strong signal that people interpret as, 'This is not working, I'm going to stop,'" said Paulo Carvalho, an assistant professor in the Human-Computer Interaction Institute (HCII). "In fact, it's the opposite. If you're getting a practice question wrong, you're probably going to get the feedback you need to understand your mistake in the next step. But learners usually interpret being wrong as, 'I'm not learning,' and we stop. But if you stop, then you definitely don't learn. This research looks at that issue."The CMU team partnered with the South African nonprofit Siyavula, which offers an intelligent AI-based tutoring system, to test these interventions. The partnership was made possible through HCII Associate Professor of Learning Sciences Amy Ogan'sLearning Sciences for Innovators program.The researchers' paper, "Will They Try Again? A Large-Scale RCT on Scaffolds That Support Persistence in an Intelligent Tutoring System," received an honorable mention at the recent Association for Computing Machinery Conference on Human Factors in Computing Systems (CHI 2026).Michael Asher, a project scientist in the HCII, said the team looked to psychology-inspired design to create the two changes: a written prompt and a nudge. The written prompt appeared after the student got a question wrong, encouraging them to try again. The nudge, which also appeared after an incorrect answer, provided a visual cue prompting a user to choose a certain option — something familiar to nearly everyone because it's baked into human-computer interaction design. (For example, when users close a Microsoft Word document, the "Save" button is highlighted in blue.) In this research, the button that read "Try an exercise like this again" was highlighted in bright orange, nudging students to try again."We move through the world making decisions and we process tons of information, so we rely on a lot of shortcuts," Asher said. "Default options are powerful ways to simplify things, because we assume that they're defaults for a reason. So there's a whole industry of nudge-based interventions, which often change defaults to help someone make a decision that is desired, hopefully for beneficial reasons."This study was a randomized controlled trial involving about 160,000 students who used Siyavula's online tutoring system to complete 17 million practice problems. Researchers randomly assigned some students to receive the nudge, the written prompt, neither or both. They found that the prompt and nudge increased how often students persisted after failure by 2% and 9%, respectively.These increases represent more than just statistics: they are thousands of instances of people learning after getting a problem wrong. Stacking the prompt and nudge together had a powerful additive effect, leading students to persist 11% more often than those who received no help at all."What we found here — and this is something that was only possible because we were able to work with a really large sample to test this precisely — is that these two interventions actually stack on top of each other really nicely," Asher said. "The prompt seems to make people more likely to try again, and the nudge has an even larger effect. When you put them together, it's the sum of the parts, which we can really see because of our sample size."Along with Asher, Ogan and Carvalho, the research team included HCII doctoral student Yumou Wei and the Siyavula Foundation's Adam Reynolds. Learn more in the research paper on the CHI 2026 website.
