A Stanford Law study put AI tutors up against actual law professors, and the machines won. Sixteen professors wrote answers to 40 contract law questions, then graded a stack of answers blind, not knowing which came from a colleague and which from a large language model. Across nearly 3,000 comparisons, the AI responses won 75% of the head-to-head matchups.
The graders also flagged AI answers as potentially misleading or harmful only 3.5% of the time, compared with 12% for human-written ones.
Co-author Sarath Sanga of Yale wrote, "In most fields where AI gets tested, there's a right answer. In law, there often isn't" — yet the AI still met the standard lawyers use to judge each other's arguments. Lead researcher Julian Nyarko isn't calling for "wholesale adoption," but says "blanket skepticism may be equally unwarranted."
Previously:
