Conquering the Uncanny Valley: An Uphill Battle in Vain?

Remember that googly eyed face staring you down, or your AI partner’s uncanny ventriloquist performance? Ever wonder where your character got their dental work done, or what skin lotion they're using? Maybe their mouth's just like that, or maybe they just don't get that much sun. If you do, then you’ve probably traveled down the “uncanny valley” one time too many.

The next-generation of video-games keep promising to revolutionize the way we play our games, but few can be said to consider the experience. Whether we notice it or not, the ways we all feel, perceive, care, and understand the characters we see on our screens is something all the more precious. Few, however, can deny that this generation’s quest for “next-gen” graphics is a divisive, if not criticized goal. Many of us would freely admit the times we’ve been turned off by our AI partner’s ridiculous Jar Jar Binks sense of grace or some passerby’s amazing ventriloquist abilities. Is it because they can’t and never will be human enough?

If you’ve ever thought that, you’re certainly not alone. Roboticist Masahiro Mori’s published the same thing in his 1970 article entitled, “Bukimi No Tani” (or "Uncanny Valley” in English) detailing his personal observations on humanlike robots. He noted that the more closely robots imitated human appearance and behavior, the more familiar they seemed to a human observer. That is, until the unfortunate point by which they resembled humans so closely, yet imperfectly, that people eventually experienced a negative reaction to them thanks to feelings of eeriness, discomfort, or repulsion to their appearance. His so-called “uncanny valley,” emerged as a term referencing a later graph depicting the correlation between familiarity and human likeness that, dipping suddenly and drastically, would cut short robots’ abilities at perfectly mimicking the human appearance.

As video-games enter their eight generation and continue to push the limit of graphics and motion capture, the realism of our characters and their worlds are dangerously approaching Mori’s own description along with the entertainment media as a whole. Why do too many increasingly “real” looking characters seem so discomforting when we play them? Only recently have scientists and roboticists begun to uncover the science of the uncanny valley, and how we can possibly conquer it.

Associate professor Karl MacDorman, the very human head of the human-computer interaction program at the School of Informatics at Indiana University has worked with the uncanny valley hypothesis, using human participants during its course. MacDorman’s scientific support for the hypothesis doesn’t pin a single factor on the theory, but the one dominating factor he believed to be the most crucial wasn’t in creating proportionately realistic humans, but their timing and interactive character movement. "I suspect the hard part will not be in finding the right physical proportions, but making the movements seem natural and well-timed, especially during interaction," he says. The incredible challenge for video games is then to do so in sequences where the player has total control over the characters for the majority of the time. Characters have to then look and react realistically and anticipate players’ input.

This can be seen in the most recent of demos that industry has shown off in just these past few years. One could easily point to any number of Quantic Dream’s recent, and quite eery, tech demos. “The Sorcerer” and “Kara” have both demonstrated impressively emotional depth from their protagonists in question, humorously and dramatically. While quite technologically amazing, they can be accused of achieving something uncannily human and decidedly unsettling.

Demos like them display great technique, but their designers exaggerate the depth of field to enhance their realism. Kara and the Sorcerer are facially rendered in focus, but with their hair and neck out of focus, their body’s disproportionally tangible. That’s really, really disconcerting when you look at the whole picture. Every time that a game like Heavy Rain had Ethan give himself a good ‘ole shave (only one apparently), the stiff, plasticky hairs hitting the sink would shout to me that I was playing a game and not a movie, unlike what David Cage would wish me to believe.

It’s additionally true that people “hear” with their eyes as much as their ears. Play an identical sound while looking at a person's lips. The synchronization of their speech and lip movements can make you to hear it differently. It can be seen as easily in a remarkable well-rendered game like The Last of Us or as bad a game as 2010’s Ironman 2. The former’s character dialogue can be heard the moment they mouth them, as their dialogue flows naturally while the latter’s leave us staring at a bunch of ventriloquist mannequins.

Some could say that beauty is only skin-deep, but it’s one of the holy grails of computer animation. Characters’ complexions are another disturbance to player immersion and the facial qualities create a kind of sociopathy that’s difficult to relate to. Sociopaths frequently seem odd to the average person, not only in their behavior, but in their facial asymmetries. Their often blank expressions can embody an appearance of developmental disorders -- compare the bugged out eyes of The Force Unleashed’s Secret Apprentice or LA Noire’s twitchy Cole Felp’s to photographs of known psychiatric patients.

It’s not always the graphics that are the culprits behind the uncanny valley’s depth. Companies like Nintendo have demonstrated great integrity in retaining the cartoony art stylings of their flagship characters. Why do we not feel disturbed by the likes of Mario and Kirby? Simply put, it’s because they don’t threaten our sense of reality. Chubby cartoon plumbers and pink puffballs don’t try to associate themselves with our reality, so we feel far more in control of identifying their fantastical origin. We gladly suspend our disbelief for them because we know what they are. Borderlands and Ratchet & Clank are unapologetically theatrical for comedic purposes and viewers are hardly insulted by it going into them expecting such. Alien’s Colonial Marines and Ride to Hell’s Jake Conway are meanwhile guilty of pathetically trying to deceive us as real, and that betrayal is far more unsettling.

In addition, the animators themselves often face the bulk of the difficulty in spotting uncanniness in their own games. The more time they spend with their characters, the more desensitized they become to their uncanniness. As artists, animators naturally rely on their own instincts for aesthetics. The dilemma is that they’ve lost the “common sense” working alone that the rest of us inherently have. This is why game franchises like Naughty Dog emphasize the critical importance of why animators need to work on their games in groups with fresh pairs of eyes available. Group development can prevent the loss of objectivity designers have at the start of a game’s production. Feedback, especially the artistic feedback, is an essential to maintaining a standard of believability.

It takes more than just feedback to get climb out of the uncanny valley, though. It’s something that distinguishes reality from believability, an equally if not more worthwhile goal. Animating a believable protagonist allows you to texture a movement than simply making something that's realistic. A realistic character will check off a list of human qualities, but a believable one will display subtleties and quirks that make them feel unique and alive. In short, perfection is the opposite of what animators need to sell us on a character. Capturing the imperfections of Nathan Drake’s falls or Big Boss’s limps are what tell us their human, not rag-dolls.

The uncanny valley is indeed a mile-high place to climb out of. Are graphics and technology the rope we hang ourselves with or what we use to climb out of it? It’s ever more ironic that the the most critically acclaimed games of this generation weren’t ever thanks to computers: they were thanks to people. More and more do we see games with A-list voice talent behind them and top notch writers. It was grafting the very real motions and voices of Nolan North and Troy Baker that made Naughty Dog’s two latest successes of The Last of Us and Uncharted.

Well or poorly rendered, what mattered to John Marston or Commander Shephard were the journeys they took. Look back to the Final Fantasies of the nineties and you won’t remember triple A graphics; you’ll remember stories. I still laugh at the slapstick gags from a red and green plumber. I still fall in love with a point-eared boy in tights that sommersaults everywhere. It’s when games don’t try so hard to be technical perfectionists and try to be something personal, something human, that they resonate with us. No matter what E3 press conferences preach to us, horsepower is a tool, maybe a magnificent one, but you can’t captivate minds with a brush or an easel. You paint something beautiful with them that lasts in us forever.

[Toon_Link]

Awesome blog!! I had a great time reading it.