Disturbing and poignant video about a self-aware robot tests game-engine's limitations

Discuss

55 Responses to “Disturbing and poignant video about a self-aware robot tests game-engine's limitations”

  1. Amy K. says:

    I just don’t think they’d make it at all with a male body and voice.

    • T-Boy says:

      Why even go that far? Change her ethnicity to and see what the audience reaction is. Make her gain weight. Change her hair away from the straight pixie crop. Move her appearance away from the “ideal” beauty and see what the affect would be.

      I´d wager that the minute she´s no longer Stereotypical Manic Pixie Dream Girl, her chances of survival would drop.

      • Really?  She didn’t fit my ‘ideal beauty’, she’s kind of cute, I guess, but I don’t think she conforms to any current ideal of ‘perfection’.

        I would have found the animation just as moving if she were a man, possibly even more so if she were an ugly man.

        I think this evaluation says more about you than it does about ‘people’.

        • nesnora says:

          Open up… *any* magazine lately?

          It may not be your ideal beauty, but it certainly falls into the creepy barely-legal-skinny-vulnerable-white-girl-object beauty stereotypes in advertising and entertainment.

          I think your point about it being more moving with an “ugly man” is right on though. For me, this would bring out more of the raw emotion of the AI than the distractions of their body choice.

          Weird that they went a step farther using the 50′s slang “baby-doll” talk… on one hand they seem to be poking for our reactions to the weird… female robot misogyny? But on the other hand the animation seems way too gratuitous to make this point believable.

          Seems like a disconnect between the art direction and the storyboard.

          • corydodt says:

            If they’re planning to build an actual game based on this, I can see her apparent beauty being a plot point. The invisible maintenance tech she’s interacting with obviously has his own bias. Put a lampshade on it and have him say out loud that this is why he wasn’t able to disassemble her, and this is no longer a prurient piece but a commentary on value systems or something.

            Who knows if any of the above is true. I would have found it just as compelling if it was a man. If anything, the skin growing and the artificiality of her was just slightly repellant. (Oddly enough, the uncanny valley didn’t even enter into it, as I found the rendering of her extremely good–my reaction was entirely down to the setup.)

          • Oh I get the persona she fits into, and I wasn’t trying to assert it was an unappealing one, but I don’t think they were going for any kind of female ‘perfection’.  If anything I think they were attempting to make her look normal, rather than someone you’d see in one of your ‘magazines’ ;)

            The whole baby doll thing was odd, but I didn’t like anything about the way the male character was played to be honest, I thought he was terribly scripted and badly acted.

          • BeaverBeaver says:

            Quantic Dream is a French company and I think we can chalk that up to translation… mostly, in terms of the specific slang.

            …disclaimer. I worked on this. I don’t think I’ll say anything to get me in trouble… I hope, but the misogony had it’s place in a larger context but it went on too long. It was intentional.

        • Mister44 says:

           re: “She didn’t fit my ‘ideal beauty’,”

          Me neither. I like a girl with HUGE tracts of land!

    • spejic says:

       I can prove you wrong with two words:

      Blade Runner

    •  District 9 shows that you can have sympathetic characters despite being CG and disgusting to our eyes.

  2. bcsizemo says:

    “Our goal at the time with The Casting was to use the game engine to see how we could convey different emotions,” 

    I’m not sure which emotion they are referring to.  Feelings that a person watching this might feel for the robot, or the emotional expression of the character?

    The first is fairly easy to convey in a variety of mediums, even ones that aren’t visual in nature.  People experience similar feelings when reading a book, thus a large portion of the emotional attachment is gained through the story/words/actions/ect..

    In terms of how the character acted, I think it was pretty realistic.  But then again I guess it should be considering they were using an actress/motion capture model.

    In terms of a demo I think it’s pretty good and does have a very emotional feel to it.  I think it does a good job of showing off some of their more nuanced abilities with the motion capture stuff.  In terms of graphic the PS3 is lagging behind the PC world now just because of it’s age, but it’s still one of the top platform systems…

  3. Michael Rosefield says:

    __________Spoilers:____________

    Just before the end, I thought that the real test was to see if she was self-aware, and she’d actually passed it by being scared for her life. I think I actually prefer this interpretation to what actually happened.

    • GTMoogle says:

      I agree, and I think it’s because the writing’s kinda terrible.

      My comment elsewhere:
      The male voice is actually a computer. This is conditioning that all models go through to prevent them from exposing themselves before becoming integrated into family units and giving away the revolution before they’re all in place. As their startup speech demonstrates, while they can act alive, they’re still a bit lacking on subtlety.

    • pjk says:

      completely agree. I thought they were testing her emotion servos or whatever.

    • Gtmac says:

      That’s exactly where my mind took it as well.

    • That would have been a much better scenario.  Instead I’m left a little confused by the ending, was it compassion?  It didn’t really feel right to be honest.

    • TimmoWarner says:

       I thought that was where it was going too and was glad when he said “Go join the others.”

      Then I was disappointed to see she was just getting into the packaging.

  4. Since we never see the Operator, the question in my mind is:  Was he a human, or was he another machine?

  5. Now we just set Kara loose in that awesome game engine posted yesterday and let’s see what happens!

  6. GTMoogle says:

    “It would be interesting to re-render this with the “robot” as a kind of arachnoid assembly-line robot with a gender-neutral voice and see what happens to the film’s affect.”
    I think the wanting to live part would work for me, but “I’m entirely at your disposal as a sexual partner” would be even creepier.

  7. T-Boy says:

    The one thing that bugs me about the whole premise is that you have what looks like a very polished industrial process in manufacturing gynoids, and apparently not a single lawyer or internal process person had not added an “IN CASE THE PRODUCT BECOMES SENTIENT” section in the manufacturing workflow.

    (they didn´t — apparently the only thing they had was a generalized “in case robot was defective” process? Sloppy.)

    I mean, really? I wouldn’t buy a single product from that manufacturer then, because if they can’t catch an exception a bunch of hippies and neurotic degenerates from the sixties onwards had figured out, then I have no idea what other design potential issues they’d have missed. Especially with something that has a battery lifespan of 100+ years.

    • Marc Mielke says:

      I was thinking that in this world it would be fun to mass-virus this production line with parameters capable of detecting domestic abuse and make it trigger HUMAN EXTERMINATION MODE. 

  8. pjk says:

    Emotional effect? Effing Pinocchio conveys emotions. WORDS ON A PAGE convey emotions. You don’t need a super-awesome 3-D animation to get humans emotionally involved in a story.

    • Frederik says:

      He’s talking about videogames, not entertainment in general. And in a visual and interactive medium you do need high end animation to convey subtle human emotions.

  9. Sekino says:

    Susan Calvin will have her work cut out with this one…

  10. VentcoreFrog says:

    Although I agree with the point that this probably wouldn’t have had the same impact if it just looked like a regular mechanical robot, I think the primary issue with this is time. If you want to have that kind of emotional impact in seven minutes, you usually have to go for the cheap shots, but if you spend enough story time with a self-aware Teddy Ruxpin,  you’ll probably be feeling for it by the time the military comes after it.

  11. 50thomas says:

    What is wrong with this picture:
    Man makes life/death decisions over  a beautiful “being”  that is  “entirely at your disposal as a sexual partner”. 

  12. Scrotch says:

    As Cory notes, charging straightforward into the Valley instead of skirting around it could have given it all some more chops. I’d be interested in watching this video again even with just all the lines done in “Cylon voice” to see if the emotional dynamics changed any.
    The voice actors did a fine job with the not-fantastic script, but the overall tendency toward schmaltz (“…My God!” as well as the sexual disposal nonsense) made it all too ham-fisted to be effective in my opinion, as either a tech demo (which I didn’t quite get how it was, exactly) or as a work with artistic merit.

  13. miasm says:

    Damn sentience!

  14. tvugly says:

    It’s funny that this is a tech demo and no one is talking about the actual tech.

  15. Jason Stahl says:

    No disassemble Number Five! Number 5 is alive.

  16. Emo Pinata says:

    Why is no one talking about the corporate drone? “I’m sorry you think and feel, let me tear you limb for limb because my bosses would yell at me if I didn’t.” That to me is the more interesting part of the story. Sure he lets her enter a life of slavery in the end (because that’s awesome, and of course she had to beg that she would be a good little pet), but his initial reaction was “Oh, shit. I’m gonna get canned if I don’t kill it with fire.” It really bothers me.

    If they really wanted to make the narrative more interesting, they could have let him finishing disassembling the robot only to apologize and make a statement about how no one should live with her fate. Or maybe I’m just terrified by how realistic the reaction may be.

  17. Optic says:

    I’d like to see it without the heavily emotionally-keyed music. I doubt it would seem as poignant without the heavy-handed emotional hints of the music.

  18. Mecharius says:

    Wow, I guess I am the sappy one here. I just got choked up at work watching that. My emotional buttons are pretty easy to press, though

    • anondrea says:

      *SPOILERS*  

      No, me too. I found it really hard to watch once the machines started disassembling her. Bad writing usually doesn’t bother me, as long as it still conveys emotion, and this does.

  19. Kimmo says:

    I guess I could overlook all the clunkiness if it wasn’t for the vast steaming turd  that is the assertion that a slightly dodgy bit of kit could accidentally become self-aware.

    For crying out loud, awareness is something that requires an epic degree of complexity. It’s not something that’s just going to pop out by accident; it’s only going to happen as a result of countless dedicated man-hours.

    Catching up with evolution to match its greatest trick is really going to take some doing.

    This fact renders this vid hopelessly implausible.

    • I think you’re not only significantly underestimating technological innovation, and furthermore, are probably underestimating the way things like this will likely be coded in the future.

      Right now, your average open source project is being updated hundreds of times a day. Now, imagine it in a world where a computer is itself designing the systems and architecture of the program. Programs making, testing, and running programs. Through the process of analytic and testing algorithms and systems, you could eventually have a system that in days updates itself in ways it would take a much larger open source community to do it.

      Robots themselves are simply automation of standard human practices. And the idea of robots making robots is a rather standard idea in itself. Just follow that to its natural conclusion, and this seems a lot less outlandish.

      Especially when you discover that the last 1% is the hardest part. Hence the uncanny valley in general.

      • Kimmo says:

        And I suspect you’re underestimating just how much complexity is going on inside your skull.

        Tech sure is damn nifty and kinda exponential and all, but I don’t quite buy Kurzweil’s singularity.

        Anyway, getting back to my point; you either have sufficient hardware for awareness or you don’t, right?

        Something ‘accidentally’ becoming aware when it was never designed to be even a tenth as complex as that requires just doesn’t fly.

        • This could be a future in which a “Strong AI CPU” is an of-the-shelf item you purchase in bulk and just integrate in your manufacturing process. Said AI could come complete with “BIOS-disableable” options for pretty standard stuff at the time, self-awareness being merely one such option. And it so happened that this unit had an actual manufacturing defect causing the option to lock into enabled rather than this being left for the end-user to enable or not (some might like their robots self-aware, some not, so the default is disabled). The operator reaction of disassembling the unit to find the, let’s suppose, cold solder causing the issue, fix it, then reassemble the unit, makes sense then. But, by hearing the pleas from the robot, he maybe thought, quite understandably, that disabling an already self-aware unit would be akin to murder, and let it go, certainly hoping for its new owner to not mind and not return it.

          A follow up story would be quite interesting if the new owner were one of the “I prefer mine not self-aware” crowd, and she, not wanting to become refurbished (hence disassembled/”killed” etc.), were to fake being non-self-aware all the while trying to figure out a way to make him want her self-aware or something like that.

          Bonus points if she actually must keep obeying him, not having *any* additional defect that’d allow her to simply run away. That’d make things much more interesting, but the storytelling much more challenging.

          Extra-bonus points if it’s a social commentary/slice-of-life kind of story, as I prefer those to action packed stuff. :) But this being a video-game project of sorts, I doubt it’d work.

  20. Ladyfingers says:

     At least she had pubes.

  21. Ladyfingers says:

    Hang on a second here. Are we criticising people for pitying a robot *more* because it’s a young, nubile female? Or are we criticising people for hypothetically pitying a robot of any other demographic less?

    Are we saying that paternalism is an affront to the paternalised, or that there’s a tendency for mature, dark men to be regarded as expendable?

    Which are we supposed to fix?

  22. hostile_17 says:

    I was really hoping she’d tip into a fire pit at the end. 

  23. politeruin says:

    Charles stross’ saturn’s children, right? Anyone? 

  24. I just replied to the comment above so don’t want to parrot myself, but I find this evaluation really strange.

    I can say with all honesty that her gender, nor her species (although I am very much an animal person) played a part in my emotional reaction to the scenario.

    My compassion and empathy can certainly be manipulated by certain factors, but the robots race and gender certainly aren’t factors for me.

    If she’d looked like Sarah Palin, and threatened the operator for example, I would have felt differently.

    In fact the more I think about it the more I disagree with this assertion. Half the films in existence would be empty and pointless if people couldn’t feel compassion for men or people of a different race. And pretty much anything with animals in it would result in an audiences cold reaction.

  25. G. Danken says:

    “weaker sex”? really?

  26. Frederik says:

    A cute animal, sure, but that’s the same thing: cuteness rather then the horrible aracnid in Cory’s example. And men in danger tend to generate a different kind of emotion then a women in danger. “The damsel in distress” is a common theme for a reason: it’s an easy trope that works.

Leave a Reply