I ran across reports of two studies relating to AI today that combine to paint a sad picture. Kids are using LLMs to complete schoolwork, and the LLMs aren't even good at it.
A recent poll from the Pew Research Center shows more and more teens are turning to ChatGPT for help with their homework.
3 things to know:
NPR
- According to the survey, 26% of students ages 13-17 are using the artificial intelligence bot to help them with their assignments.
- That's double the number from 2023, when 13% reported the same habit when completing assignments.
- Comfort levels with using ChatGPT for different types of assignments vary among students: 54% found that using it to research new topics, for example, was an acceptable use of the tool. But only 18% said the same for using it to write an essay.
"The main takeaway from this study is that LLMs, while impressive, still lack the depth of understanding required for advanced history. They're great for basic facts, but when it comes to more nuanced, PhD-level historical inquiry, they're not yet up to the task," said Maria del Rio-Chanona, one of the paper's co-authors and an associate professor of computer science at University College London.
…
Why are LLMs bad at answering technical historical questions, when they can be so good at answering very complicated questions about things like coding? Del Rio-Chanona told TechCrunch that it's likely because LLMs tend to extrapolate from historical data that is very prominent, finding it difficult to retrieve more obscure historical knowledge.
TechCrunch
AI isn't very good at details like history because it generalizes and makes things up based on what patterns indicate versus actually known details. Also, more schools allow students to use AI to create work and teachers to grade it. A storm is brewing.
Previously:
• Machine learning models keep getting spoofed by adversarial attacks and it's not clear if this can ever be fixed
