Why video game eyes are creepy

When it comes to the uncanny valley of video game characters, the eyes have it. Even as digital characters become increasingly (hyper)realistic, the eyes lag behind. At FastCo Design, Mark Wilson looks at the technological and perceptual challenges of designing eyes with personality:

The initial problem with rendering eyes is simply that of light and structure. While the eye looks simple to, um, the naked eye, when you actually examine its structures, you realize it’s actually a mostly clear object. All of these clear layers manipulate light differently, and in reaction to one another, through a spherical structure (but notably, not a perfect sphere!). On top is the cornea. It’s not just a transparent lens. It’s a transparent lens that bulges out from the eyeball. It might reflect light like a mirror, or refract light, warping it like a water droplet on a windshield. Indeed, every structure you see within someone’s eye—like the colorful iris—has been distorted by their cornea.

"The transitions of each of these things, from one to the next, needs to be handled properly," says (Brian Karis, senior graphics programmer at Epic Games). "How light interacts with all those things has to be handled."

The white of the eye is particularly tricky. Known as the sclera, it’s actually the layer that wraps around most of your eye like an orange skin. Light "scatters" from the sclera through the clear gel that comprises most your eye—which is the same phenomenon that gives a glass of milk its particular glow.

Assuming all of this is rendered correctly, there’s one final problem remaining: caustics, or an envelope of refracted light we see frequently in the real world. "Imagine you’re at a fancy restaurant. You have a glass of wine in front of you, and the light hits it," says CTO Kim Libreri. "You’ll see on the tablecloth a little red, a little white." Caustics occur within the eye, too.

"It’s a subtle little effect you might not notice if it wasn't there," continues Karis. "But it won’t seem like there was as much complexity happening in the eyeballs."

"The Impossibly Complex Art Of Designing Eyes"