Disturbing and poignant video about a self-aware robot tests game-engine's limitations

Kara, a disturbing short film about a self-aware robot, was made by games studio Quantic Dream to demonstrate the "expressive power" of the PS3's graphics. In order to sidestep the limitations of animating human characters (the so-called, contentious "uncanny valley"), the creators made a story about a newborn, intelligent robot -- a character that is supposed to be subtly unconvincing in its humanity.

"Our goal at the time with The Casting was to use the game engine to see how we could convey different emotions," Cage tells us prior to the GDC talk where he's unveiling a slice of what Quantic Dream has been up to since 2010. "We wanted to see what it would take in terms of the technology but also with the acting, and working with the actor on-stage to have this performance coming across in the game engine. We learned so much doing it for Heavy Rain, from the good things that worked very well but also from the mistakes that we made, and things we could have done differently. 'Introducing Quantic Dream's Kara' Screenshot 1

"When Heavy Rain was over, we thought why not do exactly the same thing and do a short sequence in real-time, in the game engine to see how our next game is going to benefit from what we're going to learn?"

"In Kara, you can't imagine the same scene having the same impact as someone who's not a talented actor. Technology becomes more precise and detailed and gives you more subtleties, so you need talent now. I'm not talking about getting a name in your game - I'm talking about getting talent in your game to improve the experience and get emotion in your game."

Welcome to Kara, the product of Quantic Dream's recent work on the PlayStation 3, and of its investment in new motion capture facilities. Again it's a one-woman show built around a slow tonal shift, again channelled through a strong and actorly central performance - but the distance between Kara and The Casting is as good a measure as any of the technical progress we've seen this generation, and of a shift in ambition and capability within Quantic Dream.

As interesting as this is as a technology demo, I think its real value is in the questions raised by the story and the storytelling choices. The unsettling poignancy of this clip arises from the gender and form of the robot. It would be interesting to re-render this with the "robot" as a kind of arachnoid assembly-line robot with a gender-neutral voice and see what happens to the film's affect.

Introducing Quantic Dream's Kara (via JWZ)


    1. Why even go that far? Change her ethnicity to and see what the audience reaction is. Make her gain weight. Change her hair away from the straight pixie crop. Move her appearance away from the “ideal” beauty and see what the affect would be.

      I´d wager that the minute she´s no longer Stereotypical Manic Pixie Dream Girl, her chances of survival would drop.

      1. Really?  She didn’t fit my ‘ideal beauty’, she’s kind of cute, I guess, but I don’t think she conforms to any current ideal of ‘perfection’.

        I would have found the animation just as moving if she were a man, possibly even more so if she were an ugly man.

        I think this evaluation says more about you than it does about ‘people’.

        1. Open up… *any* magazine lately?

          It may not be your ideal beauty, but it certainly falls into the creepy barely-legal-skinny-vulnerable-white-girl-object beauty stereotypes in advertising and entertainment.

          I think your point about it being more moving with an “ugly man” is right on though. For me, this would bring out more of the raw emotion of the AI than the distractions of their body choice.

          Weird that they went a step farther using the 50’s slang “baby-doll” talk… on one hand they seem to be poking for our reactions to the weird… female robot misogyny? But on the other hand the animation seems way too gratuitous to make this point believable.

          Seems like a disconnect between the art direction and the storyboard.

          1. If they’re planning to build an actual game based on this, I can see her apparent beauty being a plot point. The invisible maintenance tech she’s interacting with obviously has his own bias. Put a lampshade on it and have him say out loud that this is why he wasn’t able to disassemble her, and this is no longer a prurient piece but a commentary on value systems or something.

            Who knows if any of the above is true. I would have found it just as compelling if it was a man. If anything, the skin growing and the artificiality of her was just slightly repellant. (Oddly enough, the uncanny valley didn’t even enter into it, as I found the rendering of her extremely good–my reaction was entirely down to the setup.)

          2. Oh I get the persona she fits into, and I wasn’t trying to assert it was an unappealing one, but I don’t think they were going for any kind of female ‘perfection’.  If anything I think they were attempting to make her look normal, rather than someone you’d see in one of your ‘magazines’ ;)

            The whole baby doll thing was odd, but I didn’t like anything about the way the male character was played to be honest, I thought he was terribly scripted and badly acted.

          3. Quantic Dream is a French company and I think we can chalk that up to translation… mostly, in terms of the specific slang.

            …disclaimer. I worked on this. I don’t think I’ll say anything to get me in trouble… I hope, but the misogony had it’s place in a larger context but it went on too long. It was intentional.

        2.  re: “She didn’t fit my ‘ideal beauty’,”

          Me neither. I like a girl with HUGE tracts of land!

  1. “Our goal at the time with The Casting was to use the game engine to see how we could convey different emotions,” 

    I’m not sure which emotion they are referring to.  Feelings that a person watching this might feel for the robot, or the emotional expression of the character?

    The first is fairly easy to convey in a variety of mediums, even ones that aren’t visual in nature.  People experience similar feelings when reading a book, thus a large portion of the emotional attachment is gained through the story/words/actions/ect..

    In terms of how the character acted, I think it was pretty realistic.  But then again I guess it should be considering they were using an actress/motion capture model.

    In terms of a demo I think it’s pretty good and does have a very emotional feel to it.  I think it does a good job of showing off some of their more nuanced abilities with the motion capture stuff.  In terms of graphic the PS3 is lagging behind the PC world now just because of it’s age, but it’s still one of the top platform systems…

  2. __________Spoilers:____________

    Just before the end, I thought that the real test was to see if she was self-aware, and she’d actually passed it by being scared for her life. I think I actually prefer this interpretation to what actually happened.

    1. I agree, and I think it’s because the writing’s kinda terrible.

      My comment elsewhere:
      The male voice is actually a computer. This is conditioning that all models go through to prevent them from exposing themselves before becoming integrated into family units and giving away the revolution before they’re all in place. As their startup speech demonstrates, while they can act alive, they’re still a bit lacking on subtlety.

    2. That would have been a much better scenario.  Instead I’m left a little confused by the ending, was it compassion?  It didn’t really feel right to be honest.

    3.  I thought that was where it was going too and was glad when he said “Go join the others.”

      Then I was disappointed to see she was just getting into the packaging.

  3. “It would be interesting to re-render this with the “robot” as a kind of arachnoid assembly-line robot with a gender-neutral voice and see what happens to the film’s affect.”
    I think the wanting to live part would work for me, but “I’m entirely at your disposal as a sexual partner” would be even creepier.

    1.  Did nobody here see “Wall-E”?

      However, I think the point of the tech demo was to show the character on screen exhibiting emotion, not necessarily to evoke emotion in the viewer because we all know you can do that with plain text, or robots like in Wall-E.

  4. The one thing that bugs me about the whole premise is that you have what looks like a very polished industrial process in manufacturing gynoids, and apparently not a single lawyer or internal process person had not added an “IN CASE THE PRODUCT BECOMES SENTIENT” section in the manufacturing workflow.

    (they didn´t — apparently the only thing they had was a generalized “in case robot was defective” process? Sloppy.)

    I mean, really? I wouldn’t buy a single product from that manufacturer then, because if they can’t catch an exception a bunch of hippies and neurotic degenerates from the sixties onwards had figured out, then I have no idea what other design potential issues they’d have missed. Especially with something that has a battery lifespan of 100+ years.

    1. I was thinking that in this world it would be fun to mass-virus this production line with parameters capable of detecting domestic abuse and make it trigger HUMAN EXTERMINATION MODE. 

  5. Emotional effect? Effing Pinocchio conveys emotions. WORDS ON A PAGE convey emotions. You don’t need a super-awesome 3-D animation to get humans emotionally involved in a story.

    1. He’s talking about videogames, not entertainment in general. And in a visual and interactive medium you do need high end animation to convey subtle human emotions.

  6. Although I agree with the point that this probably wouldn’t have had the same impact if it just looked like a regular mechanical robot, I think the primary issue with this is time. If you want to have that kind of emotional impact in seven minutes, you usually have to go for the cheap shots, but if you spend enough story time with a self-aware Teddy Ruxpin,  you’ll probably be feeling for it by the time the military comes after it.

  7. What is wrong with this picture:
    Man makes life/death decisions over  a beautiful “being”  that is  “entirely at your disposal as a sexual partner”. 

  8. As Cory notes, charging straightforward into the Valley instead of skirting around it could have given it all some more chops. I’d be interested in watching this video again even with just all the lines done in “Cylon voice” to see if the emotional dynamics changed any.
    The voice actors did a fine job with the not-fantastic script, but the overall tendency toward schmaltz (“…My God!” as well as the sexual disposal nonsense) made it all too ham-fisted to be effective in my opinion, as either a tech demo (which I didn’t quite get how it was, exactly) or as a work with artistic merit.

  9. Why is no one talking about the corporate drone? “I’m sorry you think and feel, let me tear you limb for limb because my bosses would yell at me if I didn’t.” That to me is the more interesting part of the story. Sure he lets her enter a life of slavery in the end (because that’s awesome, and of course she had to beg that she would be a good little pet), but his initial reaction was “Oh, shit. I’m gonna get canned if I don’t kill it with fire.” It really bothers me.

    If they really wanted to make the narrative more interesting, they could have let him finishing disassembling the robot only to apologize and make a statement about how no one should live with her fate. Or maybe I’m just terrified by how realistic the reaction may be.

  10. I’d like to see it without the heavily emotionally-keyed music. I doubt it would seem as poignant without the heavy-handed emotional hints of the music.

  11. Wow, I guess I am the sappy one here. I just got choked up at work watching that. My emotional buttons are pretty easy to press, though

    1. *SPOILERS*  

      No, me too. I found it really hard to watch once the machines started disassembling her. Bad writing usually doesn’t bother me, as long as it still conveys emotion, and this does.

  12. I guess I could overlook all the clunkiness if it wasn’t for the vast steaming turd  that is the assertion that a slightly dodgy bit of kit could accidentally become self-aware.

    For crying out loud, awareness is something that requires an epic degree of complexity. It’s not something that’s just going to pop out by accident; it’s only going to happen as a result of countless dedicated man-hours.

    Catching up with evolution to match its greatest trick is really going to take some doing.

    This fact renders this vid hopelessly implausible.

    1. I think you’re not only significantly underestimating technological innovation, and furthermore, are probably underestimating the way things like this will likely be coded in the future.

      Right now, your average open source project is being updated hundreds of times a day. Now, imagine it in a world where a computer is itself designing the systems and architecture of the program. Programs making, testing, and running programs. Through the process of analytic and testing algorithms and systems, you could eventually have a system that in days updates itself in ways it would take a much larger open source community to do it.

      Robots themselves are simply automation of standard human practices. And the idea of robots making robots is a rather standard idea in itself. Just follow that to its natural conclusion, and this seems a lot less outlandish.

      Especially when you discover that the last 1% is the hardest part. Hence the uncanny valley in general.

      1. And I suspect you’re underestimating just how much complexity is going on inside your skull.

        Tech sure is damn nifty and kinda exponential and all, but I don’t quite buy Kurzweil’s singularity.

        Anyway, getting back to my point; you either have sufficient hardware for awareness or you don’t, right?

        Something ‘accidentally’ becoming aware when it was never designed to be even a tenth as complex as that requires just doesn’t fly.

        1. This could be a future in which a “Strong AI CPU” is an of-the-shelf item you purchase in bulk and just integrate in your manufacturing process. Said AI could come complete with “BIOS-disableable” options for pretty standard stuff at the time, self-awareness being merely one such option. And it so happened that this unit had an actual manufacturing defect causing the option to lock into enabled rather than this being left for the end-user to enable or not (some might like their robots self-aware, some not, so the default is disabled). The operator reaction of disassembling the unit to find the, let’s suppose, cold solder causing the issue, fix it, then reassemble the unit, makes sense then. But, by hearing the pleas from the robot, he maybe thought, quite understandably, that disabling an already self-aware unit would be akin to murder, and let it go, certainly hoping for its new owner to not mind and not return it.

          A follow up story would be quite interesting if the new owner were one of the “I prefer mine not self-aware” crowd, and she, not wanting to become refurbished (hence disassembled/”killed” etc.), were to fake being non-self-aware all the while trying to figure out a way to make him want her self-aware or something like that.

          Bonus points if she actually must keep obeying him, not having *any* additional defect that’d allow her to simply run away. That’d make things much more interesting, but the storytelling much more challenging.

          Extra-bonus points if it’s a social commentary/slice-of-life kind of story, as I prefer those to action packed stuff. :) But this being a video-game project of sorts, I doubt it’d work.

  13. Hang on a second here. Are we criticising people for pitying a robot *more* because it’s a young, nubile female? Or are we criticising people for hypothetically pitying a robot of any other demographic less?

    Are we saying that paternalism is an affront to the paternalised, or that there’s a tendency for mature, dark men to be regarded as expendable?

    Which are we supposed to fix?

Comments are closed.