Ethan Mollick

New randomized, controlled trial of students using GPT-4 as a tutor in Nigeria. 6 weeks of after-school AI tutoring = 2 years of typical learning gains, outperforming 80% of other educational interventions.

And it helped all students, especially girls who were initially behind 

No working paper yet, but the results and experiment are written up here. They used Microsoft Copilot and teachers provided guidance and initial prompts: https://blogs.worldbank.org/en/education/From-chalkboards-to-chatbots-Transforming-learning-in-Nigeria

To make clear the caveats for people who don't read the post: learning gains are measured in Equivalent Years of Schooling, this is a pilot study on narrow topics and they do not have long-term learning measures. And there is no full paper yet (but the team is credible)

And since it isn't clear to everyone who doesn't read the blog post - the fact that this is teacher-led is likely very important. We know that independent use of AI as a tutor can harm learning in some circumstances, because it gives the illusion of learning.

No one reads the thread and now I see viral quotes of the first post in the thread saying “teachers are going away”

That is the opposite of the message of every RCT we have in AI &amp; education so far! You really need instructor guidance for today’s LLMs to be useful for learning.