Since time immemorial, many students have invoked cliches to compose the most terrible essays. Maybe the GPT-3 text generator can help them. It certainly can't do any worse, right?
From The Next Web:
[Education resource site EduRef] hired a panel of professors to create writing prompts for essays on US history, research methods, creative writing, and law.
They fed the prompts to GPT-3, and also gave them to a group of recent college graduates and undergrad students.
The anonymized papers were then marked by the panel, to test whether AI can get better grades than human pupils.
Its human rivals earned similar marks for their history papers: a B and a C+. But only one of three students got a higher grade than the AI for the law assignment.
GPT-3 also received a solid C for its research methods paper on COVID-19 vaccine efficacy, while the students got a B and a D.
EduRef has a lot more data and graphs comparing the results:
Even without being augmented by human interference, GPT-3's assignments received more or less the same feedback as the human writers. While 49.2% of comments on GPT-3's work were related to grammar and syntax, 26.2% were about focus and details. Voice and organization were also mentioned, but only 12.3% and 10.8% of the time, respectively. Similarly, our human writers received comments in nearly identical proportions. Almost 50% of comments on the human papers were related to grammar and syntax, with 25.4% related to focus and details. Just over 13% of comments were about the humans' use of voice, while 10.4% were related to organization.
For GPT-3, the feedback was more diverse. While GPT-3 was praised for some excellent openings and transitions, it was criticized for being vague, too blunt, and awkward. GPT-3 also slipped up with its citations, at one point not providing references at all. But the awkward writing, lack of citations, and bluntness didn't cause GPT-3 to fail – it's inability to craft a strong narrative did. GPT-3's F-rated assignment received comments calling the writing cliche, too personal, and bland. The AI failed to craft a strong narrative incorporating the five senses, and telling-not-showing essays don't cut it in creative writing classes.
How long until some savvy student finds a way to sell GPT-3 essays to their classmates?
Who writes better essays: College students or GPT-3? [Thomas Macauley / The Next Web]
Image: Mike Mackenzie / Flickr (CC-BY-SA 2.0)