At The New Yorker, Kyle Chayka explores perhaps the most common reminder that AIs don't know what they're drawing and have no model of anything real to guide them: the mangled hands.
As others have reported, the hand problem has to do, in part, with the generators' ability to extrapolate information from the vast data sets of images they have been trained on. When a user types a text prompt into a generator, it draws on countless related images and replicates the patterns it has learned. But, like an archaeologist trying to translate Egyptian hieroglyphs from the Rosetta Stone, the machine can deduce only from its given material, and there are gaps in its knowledge, particularly when it comes to understanding complex organic shapes holistically. Flawed or incomplete data sets produce flawed outputs. As the linguist Noam Chomsky and his co-authors argued recently in a recent Times Op-Ed, machines and humans learn differently. "The human mind is not, like ChatGPT and its ilk, a lumbering statistical engine for pattern matching, gorging on hundreds of terabytes of data," they wrote. Instead, it "operates with small amounts of information; it seeks not to infer brute correlations among data points but to create explanations."
The teeth are less obvious, but I think they offer a superior frisson when you notice it. By then you'd definitely be within range.
Midjourney's forthcoming fifth version reportedly has found its way past the hands and teeth problem, but still can't do feet.
Meanwhile, professors are "in denial about the depth of the AI plagiarism problem" of students using GPT-3 and such to write their papers. There are a few prose "nightmare appendages" to watch out for, though…