Conversational AI Tutors: When They Work and When They Don't

Conversational tutors built on large language models are the version of AI that most students actually encounter — a chat window, a question, an answer. They are also the version where the evidence is most polarized, because the exact same underlying model can help or hurt depending on how it is wrapped.

The most instructive single study makes the point in one experiment. Nothing about the model changed; only the design around it did, and the learning outcome flipped sign. This article uses that result as its spine: where conversational tutors demonstrably work, where they demonstrably fail, and the specific guardrails that separate the two.

The 2025 PNAS study by Bastani and colleagues is the clearest natural experiment available. Working with nearly 1,000 high school students in Turkey across four math sessions, the researchers compared two AI tutors built on GPT-4. One, "GPT Base," behaved like a standard chatbot and would supply answers. The other, "GPT Tutor," was prompted to withhold answers and give hints, incorporated teacher input, and was designed to guide rather than solve.

During the practice sessions, both helped — dramatically. Students with GPT Base improved their practice-problem scores by 48%, and students with GPT Tutor by 127%. Then the researchers took the tools away and tested students on their own. Here the results diverged sharply. Students who had used the unguarded GPT Base scored 17% worse than peers who never had AI access at all. The guarded GPT Tutor's students showed no such penalty.

The interpretation the authors give is blunt: without guardrails, students used the model as a "crutch." It inflated performance while present and eroded the learning it was supposed to build. This is the central risk of conversational tutoring, and it is not hypothetical — it is measured, in a large sample, with a clean control.

The same study, and others, show the upside is equally real when the design is right. The guarded GPT Tutor did not just avoid harm; it produced the largest practice gains of any condition. And the Harvard physics RCT by Kestin and Miller (2025) — a chat-based tutor built on GPT-4, wrapped in research-based pedagogy — delivered more than double the learning gains of a strong active-learning class, in less time and with higher engagement.

Reading across the positive results, conversational tutors add value in a consistent set of situations:

• On-demand availability. They answer at the moment of confusion, at any hour, without a scheduling constraint. Given that tutoring effects scale with dosage, this is not a minor convenience.
• Socratic questioning. When prompted to ask rather than tell, LLMs are capable questioners — the mode the Stanford Tutor CoPilot data showed improves outcomes (more probing questions, less answer-giving).
• Low-stakes practice and rehearsal. Students will ask a machine the "obvious" question they are too embarrassed to ask a teacher, and will practice explaining concepts back without fear of judgment.
• Language and accessibility. They translate, rephrase, and adjust reading level on demand, widening access for multilingual and differently-prepared learners.

The common thread is that the tutor is a practice and reasoning partner, not an answer vending machine. Every positive result in the literature shares that framing.

The failure modes are specific, and three of them are structural rather than incidental.

1. Hallucination

Unlike the rule-based tutoring systems of earlier research, generative models can state falsehoods fluently and confidently. The TruthfulQA benchmark (Lin, Hilton, and Evans) found that even leading models were truthful on only around 58% of adversarially selected questions where humans hold common misconceptions, against 94% for people. Broader surveys of hallucination document error rates that vary widely by domain but never reach zero. In a tutoring context a confident wrong answer is worse than no answer, because the student lacks the expertise to catch it — that is the entire reason they are being tutored.

2. Over-reliance and the crutch effect

The PNAS result is the documented case: an answer-giving tutor can substitute for the productive struggle that produces learning. The more capable the model, the more seductive the shortcut.

3. Gaming and answer-extraction

Students are resourceful. A tutor instructed to give hints can often be talked into giving answers through rephrasing, role-play, or persistence. Any guardrail that lives only in a system prompt is a guardrail a motivated student can probe. Design has to assume adversarial use, because Pew's 2026 data shows a majority of teens already reach for chatbots on schoolwork, and a meaningful share regard using them to solve problems outright as acceptable.

Design quality only matters if students use the tool, and here the numbers are humbling. Khan Academy's Khanmigo expanded from about 68,000 users in 2023-24 to more than 700,000 the following year, yet Khan Academy has reported that only around 15% of students with access actively use the tutor. As of late 2024 there was no published peer-reviewed study showing Khanmigo itself improves outcomes, though the underlying Khan Academy platform has a well-documented association with gains (roughly a 0.36 effect size across hundreds of thousands of students in earlier research on the exercise platform, not the AI tutor).

The lesson is that a conversational tutor left as an optional side feature tends to be ignored by the students who most need it. Effective deployments in the research — the Turkey study, the Harvard course — embedded the tutor into required coursework with teacher involvement, rather than offering it as a voluntary extra.

Synthesizing across the studies, a concrete design specification emerges. Conversational tutors that improved durable learning shared most of these traits:

• Withhold answers by default; give hints and questions. This is the single decision that separated harm from benefit in the PNAS study.
• Ground responses in vetted content. Constrain the tutor to course material and retrieval from trusted sources rather than open-ended generation, to limit hallucination on exactly the questions students can't fact-check.
• Incorporate teacher input. The guarded tutor that worked was shaped by the instructor's framing of the problem set, not a generic assistant.
• Design for adversarial use. Assume students will try to extract answers, and enforce the hint-first behavior at the system level, not by polite instruction alone.
• Keep a human in the loop. The strongest field results — Tutor CoPilot — used AI to support a human tutor rather than to stand in for one.
• Embed it in required work. Optional tools reach roughly one in seven students; effective deployments made the tutor part of the assignment.

Conversational AI tutors are neither the breakthrough nor the menace they are alternately sold as. The evidence, still young but increasingly rigorous, shows a technology whose learning effect is set almost entirely by design. The same GPT-4 that doubled learning gains in a well-built Harvard course left students measurably worse off when deployed as a bare answer engine to high schoolers.

For institutions, that reframes the decision. The question is not whether to adopt conversational tutoring — students have already adopted it on their own — but whether the version they meet in a course is engineered to make them think or engineered to think for them. Bloom's forty-year-old challenge was to scale the effect of individual attention. Conversational AI can begin to do that, but only under guardrails, with teachers in the loop, and with a design that treats the student as the one who has to do the learning.