A primer
Education AI has raised learning in math and science for decades, and reached millions of students. It only worked where people built the content by hand, one subject at a time.
Large language models remove that limit. They also arrive without the teaching skill and the evidence the older tools earned. This primer walks that gap in plain terms.
Sources · Watters 2021; Bloom 1984; Anderson et al. 1995; Brown et al. 2020; LearnLM Team 2024
In 1984 Benjamin Bloom found that one-to-one tutoring with mastery learning moved the average student about two standard deviations, to roughly the 98th percentile.
One tutor per child doesn't scale, so the field spent forty years trying to reach that bar with software. Good intelligent tutors got partway, close to a human tutor in math, never all the way.
Sources · Bloom 1984; VanLehn 2011
A tutor for algebra knows only algebra, because a person encoded that subject step by step. That is slow and expensive, and none of it carries over to the next subject.
The limit was never how smart the system was. It was breadth, and the cost of building content one subject at a time.
LLMs are the first technology that gets past both. They generalize across subjects, and they need no hand-built content. That is the promise. So what is an LLM?
Sources · Aleven et al. 2016; Baker 2016
Two steps make a modern chatbot. Pretraining on huge amounts of text loads broad knowledge into the model. Then human feedback tunes it to give answers people like: helpful, direct, complete.
That second step matters. The model is built to be a helpful assistant first. Good teaching often withholds the answer so the student works for it, which runs against the assistant's grain.
Sources · Brown et al. 2020; Ouyang et al. 2022
Reinforcement learning from human feedback rewards the answer people want. Good tutoring often does the opposite. It holds the answer back and lets the student struggle toward it.
So the default behavior of a chatbot works against the default behavior of a teacher. Every LLM tutoring product is, underneath, an argument with that default.
Sources · Ouyang et al. 2022, InstructGPT
Benchmarks like MMLU and GSM8K check whether a model gets the answer. A tutor's job is to get the answer out of the student, so a high score doesn't tell you the model can teach.
Give students an unguarded chatbot and they lean on it, then score worse once it's gone. Getting the answer and learning to get it are not the same thing.
Sources · Hendrycks et al. 2021; Cobbe et al. 2021; Bastani et al. 2025
A few efforts now score teaching behavior instead of correctness, such as LearnLM's criteria, MathTutorBench, and the MathDial dataset of labeled tutoring dialogues.
They work by rubric. Experts write down what good teaching looks like, then rate the model against it or pick between the model and a human tutor. Demszky did this for teacher feedback, Shetye for Khanmigo.
The method is sound, but samples stay small, and every one scores the tutor's behavior. None connect to whether the student learned more. That top rung is the open problem.
Sources · Jurenka et al. 2024; Mačina et al. 2025, 2023; Demszky et al. 2023; Shetye 2024
This is what the older tools could never do. One model works across every subject, gives feedback and coaching to any number of students at once, and can be shaped toward good teaching rather than just good answers.
It isn't only theory. In a Harvard physics course, a well-designed AI tutor beat in-class active learning, and Google's LearnLM shows teaching behavior can be trained into a model on purpose.
Sources · Kestin et al. 2025; Jurenka et al. 2024; LearnLM Team 2024
Because the model is built to help, it will happily do the work for the student. Given an unguarded chatbot, high-schoolers did better while they had it and worse on the exam once it was gone.
Left to lean on it, learners offload their own thinking, which weakens the skills school is meant to build. Tutoring quality also swings widely with how the product is designed.
Sources · Bastani et al. 2025; Fan et al. 2024; Darvishi et al. 2023
The strongest early signal of the two. But the flagship product, Khan Academy’s Khanmigo, still has no efficacy trial behind it, only small qualitative studies.
The fastest-spreading use, and it plausibly saves teachers real time. Yet no study links an AI-planned lesson to what students actually learn.
Both are worth using now. Neither has cleared the evidence bar the classical tools eventually did.
Sources · Shetye 2024; Hu et al. 2025; Choi 2025
The tools that scaled shared a plain mechanism. Deliver mastery learning, get students to use it enough, and make sure it's used as designed.
How much a student used the tool predicted whether it helped. Seats sold predicted nothing. So measuring adoption means watching the whole funnel, from access to real use to whether people come back.
Sources · Ritter et al. 2016; Sales, Wilks & Pane 2016
Sources · Ritter et al. 2016; Sales, Wilks & Pane 2016; Feng et al. 2014
Lab effect sizes shrink in the field. Ambitious integrated programs tend to fail on rollout rather than on features. Over-helping teaches students to game the system instead of learning.
The failure modes are human and operational far more often than they are algorithmic. Plan for the rollout, not just the product.
Sources · Pane et al. 2014; Bingham et al. 2018; Baker et al. 2006
Districts buy against the ESSA evidence tiers and the What Works Clearinghouse. The top of the ladder is a randomized trial against a real alternative, measured on an independent test weeks later.
With LLMs, add one new worry. Students in the comparison group may quietly use ChatGPT too, which shrinks the measured effect and can hide a real one.
Sources · Feng et al. 2014; ESSA tiers; What Works Clearinghouse
The playbook that made classical tools work still holds. Mastery, dosage, and fidelity, the adoption funnel, and randomized trials against a real alternative all carry over unchanged.
What has to change is the measurement.
Sources · Fan et al. 2024; Bastani et al. 2025; Mačina et al. 2025; Heffernan & Heffernan 2014
Set everything side by side and the balance is simple. Real reach on one side, unfinished proof on the other.
Aleven, V., McLaren, B. M., Sewall, J., Velsen, M. V., Popescu, O., Demi, S., et al. (2016). Example-Tracing Tutors: Intelligent Tutor Development for Non-programmers. International Journal of Artificial Intelligence in Education, 26(1), 224–269. https://doi.org/10.1007/s40593-015-0088-2
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive Tutors: Lessons Learned. Journal of the Learning Sciences, 4(2), 167–207. https://doi.org/10.1207/s15327809jls0402_2
Baker, R. S. (2016). Stupid Tutoring Systems, Intelligent Humans. International Journal of Artificial Intelligence in Education, 26(2), 600–614. https://doi.org/10.1007/s40593-016-0105-0
Baker, R. S., Corbett, A. T., Koedinger, K. R., Evenson, S., Roll, I., Wagner, A. Z., et al. (2006). Adapting to When Students Game an Intelligent Tutoring System. In Proceedings of the 8th International Conference on Intelligent Tutoring Systems (pp. 392–401). https://doi.org/10.1007/11774303_39
Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122. https://doi.org/10.1073/pnas.2422633122
Bingham, A. J., Pane, J. F., Steiner, E. D., & Hamilton, L. S. (2018). Ahead of the Curve: Implementation Challenges in Personalized Learning School Models. Educational Policy, 32(3), 454–489. https://doi.org/10.1177/0895904816637688
Bloom, B. S. (1984). The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring. Educational Researcher, 13(6), 4–16. https://doi.org/10.3102/0013189x013006004
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., et al. (2020). Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems (NeurIPS). https://doi.org/10.48550/arxiv.2005.14165
Canfield, W. (2001). ALEKS: a Web-based intelligent tutoring system. Mathematics and computer education, 35(2), 152–158.
Choi, Y. (2025). Integrating ChatGPT into the Design of 5E-Based Earth Science Lessons. Education Sciences, 15(7), 815. https://doi.org/10.3390/educsci15070815
Cobbe, K., Kosaraju, V., Bavarian, M., Mark, C., Heewoo, J., Lukasz, K., et al. (2021). Training Verifiers to Solve Math Word Problems. arXiv. https://doi.org/10.48550/arxiv.2110.14168
Cosyn, E., Uzun, H., Doble, C., & Matayoshi, J. (2021). A practical perspective on knowledge space theory: ALEKS and its data. Journal of Mathematical Psychology, 101, 102512. https://doi.org/10.1016/j.jmp.2021.102512
Darvishi, A., Khosravi, H., Sadiq, S., Gašević, D., & Siemens, G. (2023). Impact of AI assistance on student agency. Computers & Education, 210, 104967. https://doi.org/10.1016/j.compedu.2023.104967
Demszky, D., Liu, J., Hill, H. C., Jurafsky, D., & Piech, C. (2023). Can Automated Feedback Improve Teachers’ Uptake of Student Ideas? Evidence From a Randomized Controlled Trial in a Large-Scale Online Course. Educational Evaluation and Policy Analysis, 46(3), 483–505. https://doi.org/10.3102/01623737231169270
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E., Jeyaraj, A., Kar, A. K., et al. (2023). Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642
Every Student Succeeds Act (ESSA), Pub. L. No. 114-95 (2015). Evidence tiers (I–IV) as defined in U.S. Department of Education non-regulatory guidance (2016).
Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., et al. (2024). Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance. British Journal of Educational Technology, 56(2), 489–530. https://doi.org/10.1111/bjet.13544
Fang, Y., Ren, Z., Hu, X., & Graesser, A. C. (2018). A meta-analysis of the effectiveness of ALEKS on learning. Educational Psychology, 39(10), 1278–1292. https://doi.org/10.1080/01443410.2018.1495829
Feng, M., Roschelle, J., Heffernan, N. T., Fairman, J. C., & Murphy, R. F. (2014). Implementation of an Intelligent Tutoring System for Online Homework Support in an Efficacy Trial. In Proceedings of the 12th International Conference on Intelligent Tutoring Systems (pp. 561–566). https://doi.org/10.1007/978-3-319-07221-0_71
Gallifant, J., Afshar, M., Ameen, S., Aphinyanaphongs, Y., Chen, S., Cacciamani, G., et al. (2025). The TRIPOD-LLM reporting guideline for studies using large language models. Nature Medicine, 31(1), 60–69. https://doi.org/10.1038/s41591-024-03425-5
Heffernan, N. T., & Heffernan, C. (2014). The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching. International Journal of Artificial Intelligence in Education, 24(4), 470–497. https://doi.org/10.1007/s40593-014-0024-x
Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., & Steinhardt, J. (2021). Measuring Massive Multitask Language Understanding. International Conference on Learning Representations (ICLR).
Hu, B., Zhu, J., Pei, Y., & Gu, X. (2025). Exploring the potential of LLM to enhance teaching plans through teaching simulation. npj Science of Learning, 10(1), 7. https://doi.org/10.1038/s41539-025-00300-x
Jurenka, I., Kunesch, M., McKee, K. R., Gillick, D., Zhu, S., Wiltberger, S., et al. (2024). Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approach. arXiv. https://doi.org/10.48550/arxiv.2407.12687
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., et al. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274
Kestin, G., Miller, K., Klales, A., Milbourne, T., & Ponti, G. (2025). AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting. Scientific Reports, 15(1), 17458. https://doi.org/10.1038/s41598-025-97652-6
Kulik, C. C., & Kulik, J. A. (1991). Effectiveness of computer-based instruction: An updated analysis. Computers in Human Behavior, 7(1-2), 75–94. https://doi.org/10.1016/0747-5632(91)90030-5
LearnLM Team, Modi, A., Veerubhotla, A. S., Rysbek, A., Huber, A., Wiltshire, B., et al. (2024). LearnLM: Improving Gemini for Learning. arXiv. https://doi.org/10.48550/arxiv.2412.16429
Lo, C. K. (2023). What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature. Education Sciences, 13(4), 410. https://doi.org/10.3390/educsci13040410
Mačina, J., Daheim, N., Chowdhury, S. P., Sinha, T., Kapur, M., Gurevych, I., & Sachan, M. (2023). MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems. Findings of the Association for Computational Linguistics: EMNLP 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.372
Mačina, J., Daheim, N., Hakimi, I., Kapur, M., Gurevych, I., & Sachan, M. (2025). MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors. arXiv. https://doi.org/10.18653/v1/2025.emnlp-main.11
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., et al. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems (NeurIPS). https://doi.org/10.48550/arxiv.2203.02155
Pane, J. F., Griffin, B. A., McCaffrey, D. F., & Karam, R. (2014). Effectiveness of Cognitive Tutor Algebra I at Scale. Educational Evaluation and Policy Analysis, 36(2), 127–144. https://doi.org/10.3102/0162373713507480
Phillips, A., Pane, J. F., Reumann-Moore, R., & Shenbanjo, O. (2020). Implementing an adaptive intelligent tutoring system as an instructional supplement. Educational Technology Research and Development, 68(3), 1409–1437. https://doi.org/10.1007/s11423-020-09745-w
Ritter, S., Yudelson, M., Fancsali, S. E., & Berman, S. (2016). How Mastery Learning Works at Scale. In Proceedings of the Third (2016) ACM Conference on Learning @ Scale (pp. 71–79). https://doi.org/10.1145/2876034.2876039
Roschelle, J., Feng, M., Murphy, R. F., & Mason, C. A. (2016). Online Mathematics Homework Increases Student Achievement. AERA Open, 2(4). https://doi.org/10.1177/2332858416673968
Sales, A., Wilks, A., & Pane, J. F. (2016). Student Usage Predicts Treatment Effect Heterogeneity in the Cognitive Tutor Algebra I Program. In Proceedings of the 9th International Conference on Educational Data Mining (pp. 207–214).
San Pedro, M. O. Z., & Baker, R. S. (2021). Knowledge inference models used in adaptive learning. In Computational psychometrics: New methodologies for a new generation of digital learning and assessment (pp. 61–77). Springer.
Shetye, S. (2024). An Evaluation of Khanmigo, a Generative AI Tool, as a Computer-Assisted Language Learning App. Studies in Applied Linguistics and TESOL, 24(1). https://doi.org/10.52214/salt.v24i1.12869
VanLehn, K. (2011). The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems. Educational Psychologist, 46(4), 197–221. https://doi.org/10.1080/00461520.2011.611369
Watters, A. (2021). Teaching Machines: The History of Personalized Learning. MIT Press.
What Works Clearinghouse. (2022). Procedures and standards handbook (Version 5.0). U.S. Department of Education, Institute of Education Sciences.
Yan, L., Greiff, S., Teuber, Z., & Gašević, D. (2024). Promises and challenges of generative artificial intelligence for human learning. Nature Human Behaviour, 8(10), 1839–1850. https://doi.org/10.1038/s41562-024-02004-5