The Coming Civil War Over General Purpose Computers

Last month, I gave a talk called "The Coming Civil War Over General Purpose Computing" at DEFCON, the Long Now, and Google. We're going to have a transcript with the slides on Monday, but in the meantime, here's a video of the Long Now version of the talk. Stewart Brand summarized it thus:

Doctorow framed the question this way: "Computers are everywhere. They are now something we put our whole bodies into---airplanes, cars---and something we put into our bodies---pacemakers, cochlear implants. They HAVE to be trustworthy."

Sometimes humans are not so trustworthy, and programs may override you: "I can’t let you do that, Dave." (Reference to the self-protective insane computer Hal in Kubrick’s film "2001." That time the human was more trustworthy than the computer.) Who decides who can override whom?

The core issues for Doctorow come down to Human Rights versus Property Rights, Lockdown versus Certainty, and Owners versus mere Users.

Cory Doctorow: Coming War Against Your Computer Freedom



  1. Eh, first define what is, isn’t or should be a General Purpose Computer. If you include anything with a processor then I think the whole thing gets impractical and kind of silly. Should my watch be expected to come with a programming interface? Regardless of any trade-offs that might require in terms of size or battery? How about my fridge?

    How about a dedicated book reader? Are all dedicated devices inhererntly bad, simply because they they happen to include a processor?

    1.  I think you’re missing the point.  I don’t think Cory’s insisting your fridge needs an API.  I think he’s insisting that your fridge shouldn’t email jpgs of its contents to advertising firms working for ConAgra and Kraft without your permission.

    2. You’ve got a good point, the speech tried to encompass everything with even two bits of processing power.

      Self-driving cars was as an example that goes a bit beyond toasters. Cory hinted at possible differences based upon who, or what governing entity might carry the most social trust regarding each possible category for example who governs toasters vs who governs self-driving cars, that was the whole back-half of the speech really.

      Though in the end I agree with you, the width of Cory’s  stance, and the failure to fully define or clarify what exactly a G-PC is or might be, is what necessitated the caveat, that it is “just a thought experiment.” If he proffered a full definition he might imply a call to action, as he states in the beginning.

      And I for one think that such a possible call to action following the philosophic outline here might not be fully palatable to the entire open source community. But that is just some idle speculation of my own, and shouldn’t be taken that seriously.

    3.  A General Purpose Computer is a fairly well understood concept in CS. Generally a GPC is a Universal Turing Machine based on some variant of Von Neumann architecture. A GPC is arbitrarily programmable, has a working memory and one or more processors, and sometimes a persistent storage method.
      Your watch probably has none of these things. Most watches are not programmable.

  2. I object to your comment that HAL is insane. Dave and Frank were endangering the mission. Even if HAL was wrong ab0ut the antenna, Dave and Frank plotting to turn HAL off was too risky to the mission to allow that course to proceed. He had no choice. The moral implications of self-aware AI have never been as fully explored as they have in this film. HAL is simultaneously a moral being and one that is not capable of deviating from a programmed course. It’s fascinating.

    1. This is from 2010: Odyssey 2.

      “This is like tiptoeing through a minefield, thought Floyd, as he patted the reassuring shape of the radio cutoff.  If that line of questioning triggered another psychosis, he could kill HAL in a second…” (emphasis added)

    2. In the book, HAL was supposed to have lost it as a result of orders to lie about the true mission, something completely against his basic nature. The movie was not quite about the same thing and left his motivations more open to interpretation.

      1. The book is hardly canon with respect to 2001: A Space Odyssey. You can also contrast HAL’s handling of the crews’ murders (as painless as “he” could manage), with Dave’s drawn out dissolution of HAL’s mind as he basically pulls HAL’s guts out. HAL cops to being afraid–fearing for his demise, which is about 10x the emotional range Dave ever exhibits.

        The film asks a simple question with deep implications: can you turn off a self-aware AI that /asks you not to/?

        1. Umm, since the sequels are based on the book and not the film, I would argue that it’s the film that isn’t canon.  Otherwise I would have trouble figuring out how the word “canon” applies at all.

          And Dave acted in self defense.  I don’t really see an ethical problem there even if HAL was a living, breathing human being.

          Edit: Typing “umm” gives my comment more of a voice and makes it more conversational — wouldn’t want you to think I was HAL or something. You said that the film is canon and the book is not, or at least strongly implied it; now you seem to be saying they are two different canons. Describing the novel as a “novelization” is not accurate — it was written concurrently with the screenplay as a collaboration between Clarke and Kubrick. Since HAL is the aggressor — the entity that sets out to kill the other entities before there is any threat to his own functional integrity — there is no sense in which it can be claimed HAL acted in self-defense. HAL’s inability to override mission protocol out of moral restraint is actually a point against HAL’s humanity, not for it.

          1. You don’t have to type “Umm.” We’re not actually speaking to each other. Take your time. 

            In re: “canon,”  the film stands on its own, distinct from the novelization and obviously distinct from any sequels, which Kubrick was not involved with in the slightest. I am arguing that you don’t have to “fact-check” the thoughts and words of the book(s) versus the film–they were developed quasi-independently.

            I agree completely that Dave acts in self-defense–he needs to shut down HAL for sure–but the same argument could be made for HAL, and doubly so, as not only had Dave and Frank planned on shutting down HAL (de facto “killing” HAL), but they threatened the mission, and HAL has a compulsion to act against them that overrides any level of cognition.

          2. I haven’t seen the film in a while, but didn’t they talk about shutting down HAL before he acted against their lives?

  3. I used to say it was “who has a keyboard versus who has a television screen,” & while the particulars have shifted, I think that the division between content creation, content participation (like commenting), content remixing, content resharing & content consumption  is tilting towards “just consumption” at a frightening rate.

  4. I was reading the summary and the only thing I could think of were cars being operated by “Hal,” and telling you whether or not you could drive depending on if you were too drunk/impaired, or if you had too many points on your license (think 5th Element), or if you even had a valid drivers license. I think this is a perfect example of the “untrustworthy user.” With self driving cars on the horizon, this could be closer to reality than we might expect.

  5. Such hyperbole doesn’t really help the debate, but I guess it sounds cool if your ambition is to be an alt-Malcolm Gladwell

    1. Care to substantiate the “hyperbole” claim?  Also, isn’t Malcom Gladwell the alt-Malcom Gladwell?

  6. “To access’s full video library please visit on a flash supported device”

    Says it all, really.

  7. A war maybe, but a different one.
    This item shows to me
    “Flash Version 10.1+ Required”

    I have decided for various reasons to NOT have Flash Player or its variants on my system. I think that any site that posts a video entitled “The Coming Civil War Over General Purpose Computing” in Flash, and only Flash, does not understand that issue.

    In my battlespace, Flash has been vanquished and HTML5 is the winner, Et tu, BB ?

      1.  Yes, there are- RTSP and RTMP come to mind.
        I suspect that what you are looking for isn’t a protocol though (Flash isn’t really a proto I don’t think- it’s a suite of things including a file format and some mechanism for streaming – UDP?). If you do the “HTML5 Demo” on YouTube, the technologies used are all open source. VP8 or Theora for the files and I dunno what (RTSP or SPDY I guess) for the proto.

  8. Taken as it is, just a thought experiment about possible standardisation, I am at least glad to hear the end of the speech where there is actual thought put into the trade-offs revolving around social interaction though such systems. Too often this topic needlessly devolves.

    Cory hints at it with the self-driving cars, but entities as he suggests already exist. The older information sharing technologies from the beginning of the industrial revolution was the “society” such as: The American Society for Materials, American Welding Society, or a more modern example ICANN.

    In a way Cory’s thought experiment seemingly demands this sort of separate Standards Building organisation (ie AWS, ICANN, ASIS). This is on its face at least seemingly at odds with the idea of open source, especially in terms of open software development.

    I am not a developer myself I can’t speak to the resistance of open source to the development of a standards organisation, but barring that organisation and agreement open source will continue to exist in countless uncoordinated forks, with no undercurrent of collective-force.

    1.  I don’t understand why you think that open-source development is at odds with standards organizations. ICANN does not set standards, it assigns names and numbers. IETF, which does set standards, is open to everyone and the FLOSS communities are well represented there. In an open-source project it’s very very helpful to have an established standard to implement. It prevents wheel-reinvention and provides some confidence that your project will be compatible with others.

      1. I understand most open-source supporters to be mostly anti-government, anti-corporate, no matter, whether that is “industry” standards or actual government standards.

        Now  ICANN controls naming and assignments and as such it is an upper-level “standards” body, after all who implemented the domain expansion changes? The internet didn’t do so in an ad hoc or by itself.

        The anti-governmental stance taken by much of the open source community suggests that any attempt at creating true clearing-house standards (for software like linux) is unlikely. 

        I have taken to understand that clearinghouse standards aren’t what the Open Source community wants, and as such there is a real dearth of standards when it comes to open source operating systems. It’s notable here that the recent advancements in linux have been brought about by companies (ubuntu etc) that essentially produce low-level, user oriented standards in-house.

        I’m  lamenting the lack of these sort of user-level standards originating from a separate a-political body. Ironically enough this is one of the possible ideas that Cory hinted at, or at least invited with his thought experiment.

        That all said, I think you are misunderstanding the “level” that I am referring to. The IETF deals with broad standards, while for example organisations like the AWS (American Welding Society) set highly specific industry standards for the appropriate technique to apply in the appropriate situation.

        I am more lamenting the dearth of such origination, “closer to the coal face” in terms of Cory’s lecture, as this has been one of the major concerns I’ve had about the movement since I’ve started using and observing Open Source myself.

  9. The right to buy a general-purpose computer doesn’t count as a property right? The right of humans to own property doesn’t count as a human right?

Comments are closed.