Graduate student Pavlos Sakellaridis has developed and tested an LLM-powered agent capable of running Dungeons & Dragons sessions, training it on transcripts from the popular streaming show Critical Role. His research provides the first rigorous evaluation of AI's potential to guide players through collaborative storytelling adventures.
Using a custom-built text-based RPG environment, Sakellaridis compared player experiences with both AI and human Dungeon Masters. The study's questionnaire revealed surprising strengths in the AI Dungeon Master's performance. While human DMs maintained a slight edge in most categories, the AI excelled at creating immersive environments, scoring 4.13 out of 5 compared to humans' 3.35.
Both AI and human DMs kept players engaged and comfortable, with similarly strong scores in positive experiences (4.03 vs 4.2) and equally low frustration levels. Traditional DMs held only modest leads in areas like narrative flow (4.0 vs 3.76) and player competence (4.4 vs 4.06).
Most striking was how both DM types maintained appropriate challenge levels without overwhelming players, suggesting the AI has mastered one of game-mastering's trickiest balancing acts. Particularly impressive was the AI's consistent portrayal of non-player characters, like Meepo the kobold, who maintained unique speech patterns and personality traits — a mix of cowardly behavior balanced with helpful determination — across multiple sessions.
"This research opens new possibilities for enhancing tabletop gaming experiences," Sakellaridis writes in his thesis conclusion, "though the results suggest LLMs will likely serve as tools to augment human DMs rather than replace them entirely."
Previously:
• Should a Dungeon Master change a game campaign to be 'cruelty free?'
• Gamer gifts ultimate homemade Dungeon Master screen
• Combat Heroes: the 1980s RPG gamebook craze got so wild they were implementing multiplayer grid-based dungeon crawlers in print
• Meet a professional D&D dungeon master. Yes, that's his main gig.
• How to plan and execute a truly exciting, character-driven game of D&D
• Why every Dungeon Master needs to read Aristotle's Poetics
• How much game can you fit in 32×32 pixels?
