The Coming Civil War Over General Purpose Computers

Last month, I gave a talk called "The Coming Civil War Over General Purpose Computing" at DEFCON, the Long Now, and Google. We're going to have a transcript with the slides on Monday, but in the meantime, here's a video of the Long Now version of the talk. Stewart Brand summarized it thus:

Doctorow framed the question this way: "Computers are everywhere. They are now something we put our whole bodies into---airplanes, cars---and something we put into our bodies---pacemakers, cochlear implants. They HAVE to be trustworthy."
Sometimes humans are not so trustworthy, and programs may override you: "I can’t let you do that, Dave." (Reference to the self-protective insane computer Hal in Kubrick’s film "2001." That time the human was more trustworthy than the computer.) Who decides who can override whom?
The core issues for Doctorow come down to Human Rights versus Property Rights, Lockdown versus Certainty, and Owners versus mere Users.

Cory Doctorow: Coming War Against Your Computer Freedom

I write books. My latest is a YA science fiction novel called Homeland (it's the sequel to Little Brother). More books: Rapture of the Nerds (a novel, with Charlie Stross); With a Little Help (short stories); and The Great Big Beautiful Tomorrow (novella and nonfic). I speak all over the place and I tweet and tumble, too.

MORE: Civlib • computer science • drm • human rights • videos

More at Boing Boing

Ants and Stars: Bruce Sterling and Jasmina Tesanovic visit the Sardinia Radio Telescope in Italy

The Snowden Principle

Ryan_T_H

Eh, first define what is, isn’t or should be a General Purpose Computer. If you include anything with a processor then I think the whole thing gets impractical and kind of silly. Should my watch be expected to come with a programming interface? Regardless of any trade-offs that might require in terms of size or battery? How about my fridge?

How about a dedicated book reader? Are all dedicated devices inhererntly bad, simply because they they happen to include a processor?
- wysinwyg
  
  I think you’re missing the point. I don’t think Cory’s insisting your fridge needs an API. I think he’s insisting that your fridge shouldn’t email jpgs of its contents to advertising firms working for ConAgra and Kraft without your permission.
- http://www.openbuddha.com/ Al Billings
  
  You seem to be missing the point of the speech.
- Yacko
  
  I dunno. Yesterday I went a few rounds with a toaster and boy are my knuckles bruised.
- csforstall
  
  You’ve got a good point, the speech tried to encompass everything with even two bits of processing power.
  
  Self-driving cars was as an example that goes a bit beyond toasters. Cory hinted at possible differences based upon who, or what governing entity might carry the most social trust regarding each possible category for example who governs toasters vs who governs self-driving cars, that was the whole back-half of the speech really.
  
  Though in the end I agree with you, the width of Cory’s stance, and the failure to fully define or clarify what exactly a G-PC is or might be, is what necessitated the caveat, that it is “just a thought experiment.” If he proffered a full definition he might imply a call to action, as he states in the beginning.
  
  And I for one think that such a possible call to action following the philosophic outline here might not be fully palatable to the entire open source community. But that is just some idle speculation of my own, and shouldn’t be taken that seriously.
- https://launchpad.net/~googoleyes koanhead
  
  A General Purpose Computer is a fairly well understood concept in CS. Generally a GPC is a Universal Turing Machine based on some variant of Von Neumann architecture. A GPC is arbitrarily programmable, has a working memory and one or more processors, and sometimes a persistent storage method.
  Your watch probably has none of these things. Most watches are not programmable.
http://www.facebook.com/profile.php?id=1197774709 Jeb Adams

I object to your comment that HAL is insane. Dave and Frank were endangering the mission. Even if HAL was wrong ab0ut the antenna, Dave and Frank plotting to turn HAL off was too risky to the mission to allow that course to proceed. He had no choice. The moral implications of self-aware AI have never been as fully explored as they have in this film. HAL is simultaneously a moral being and one that is not capable of deviating from a programmed course. It’s fascinating.
- Yacko
  
  The passive-aggressive voice made HAL seem crazy.
- wysinwyg
  
  This is from 2010: Odyssey 2.
  
  “This is like tiptoeing through a minefield, thought Floyd, as he patted the reassuring shape of the radio cutoff. If that line of questioning triggered another psychosis, he could kill HAL in a second…” (emphasis added)
- chenille
  
  In the book, HAL was supposed to have lost it as a result of orders to lie about the true mission, something completely against his basic nature. The movie was not quite about the same thing and left his motivations more open to interpretation.
  - http://www.facebook.com/profile.php?id=1197774709 Jeb Adams
    
    The book is hardly canon with respect to 2001: A Space Odyssey. You can also contrast HAL’s handling of the crews’ murders (as painless as “he” could manage), with Dave’s drawn out dissolution of HAL’s mind as he basically pulls HAL’s guts out. HAL cops to being afraid–fearing for his demise, which is about 10x the emotional range Dave ever exhibits.
    
    The film asks a simple question with deep implications: can you turn off a self-aware AI that /asks you not to/?
    - wysinwyg
      
      Umm, since the sequels are based on the book and not the film, I would argue that it’s the film that isn’t canon. Otherwise I would have trouble figuring out how the word “canon” applies at all.
      
      And Dave acted in self defense. I don’t really see an ethical problem there even if HAL was a living, breathing human being.
      
      Edit: Typing “umm” gives my comment more of a voice and makes it more conversational — wouldn’t want you to think I was HAL or something. You said that the film is canon and the book is not, or at least strongly implied it; now you seem to be saying they are two different canons. Describing the novel as a “novelization” is not accurate — it was written concurrently with the screenplay as a collaboration between Clarke and Kubrick. Since HAL is the aggressor — the entity that sets out to kill the other entities before there is any threat to his own functional integrity — there is no sense in which it can be claimed HAL acted in self-defense. HAL’s inability to override mission protocol out of moral restraint is actually a point against HAL’s humanity, not for it.
      - http://www.facebook.com/profile.php?id=1197774709 Jeb Adams
        
        You don’t have to type “Umm.” We’re not actually speaking to each other. Take your time.
        
        In re: “canon,” the film stands on its own, distinct from the novelization and obviously distinct from any sequels, which Kubrick was not involved with in the slightest. I am arguing that you don’t have to “fact-check” the thoughts and words of the book(s) versus the film–they were developed quasi-independently.
        
        I agree completely that Dave acts in self-defense–he needs to shut down HAL for sure–but the same argument could be made for HAL, and doubly so, as not only had Dave and Frank planned on shutting down HAL (de facto “killing” HAL), but they threatened the mission, and HAL has a compulsion to act against them that overrides any level of cognition.
      - retepslluerb
        
        I haven’t seen the film in a while, but didn’t they talk about shutting down HAL before he acted against their lives?
http://mordicai.livejournal.com Mordicai

I used to say it was “who has a keyboard versus who has a television screen,” & while the particulars have shifted, I think that the division between content creation, content participation (like commenting), content remixing, content resharing & content consumption is tilting towards “just consumption” at a frightening rate.
cymk

I was reading the summary and the only thing I could think of were cars being operated by “Hal,” and telling you whether or not you could drive depending on if you were too drunk/impaired, or if you had too many points on your license (think 5th Element), or if you even had a valid drivers license. I think this is a perfect example of the “untrustworthy user.” With self driving cars on the horizon, this could be closer to reality than we might expect.
http://twitter.com/MrAaronSwainEsq Aaron Swain

Best thing I’ve listened to all day in spite of the countless mic problems.
- TrevorSweet
  
  Everything was going well until the tech guy decided to add a sixth microphone ;-).
slabman

Such hyperbole doesn’t really help the debate, but I guess it sounds cool if your ambition is to be an alt-Malcolm Gladwell
- wysinwyg
  
  Care to substantiate the “hyperbole” claim? Also, isn’t Malcom Gladwell the alt-Malcom Gladwell?
Paulwh80

“To access FORA.tv’s full video library please visit FORA.tv on a flash supported device”

Says it all, really.
http://pulse.yahoo.com/_CGUZJZ3IVNH66EXXB6JY23VPR4 vdev

A war maybe, but a different one.
This item shows to me
“Flash Version 10.1+ Required”

I have decided for various reasons to NOT have Flash Player or its variants on my system. I think that any site that posts a video entitled ”The Coming Civil War Over General Purpose Computing” in Flash, and only Flash, does not understand that issue.

In my battlespace, Flash has been vanquished and HTML5 is the winner, Et tu, BB ?
- wysinwyg
  
  This is a pretty good point. Are there any open source streaming protocols?
  - https://launchpad.net/~googoleyes koanhead
    
    Yes, there are- RTSP and RTMP come to mind.
    I suspect that what you are looking for isn’t a protocol though (Flash isn’t really a proto I don’t think- it’s a suite of things including a file format and some mechanism for streaming – UDP?). If you do the “HTML5 Demo” on YouTube, the technologies used are all open source. VP8 or Theora for the files and I dunno what (RTSP or SPDY I guess) for the proto.
- https://launchpad.net/~googoleyes koanhead
  
  You could try playing it in VLC or Gnash Player. Very few videos that nominally require Flash *really* require Flash.
csforstall

Taken as it is, just a thought experiment about possible standardisation, I am at least glad to hear the end of the speech where there is actual thought put into the trade-offs revolving around social interaction though such systems. Too often this topic needlessly devolves.

Cory hints at it with the self-driving cars, but entities as he suggests already exist. The older information sharing technologies from the beginning of the industrial revolution was the “society” such as: The American Society for Materials, American Welding Society, or a more modern example ICANN.

In a way Cory’s thought experiment seemingly demands this sort of separate Standards Building organisation (ie AWS, ICANN, ASIS). This is on its face at least seemingly at odds with the idea of open source, especially in terms of open software development.

I am not a developer myself I can’t speak to the resistance of open source to the development of a standards organisation, but barring that organisation and agreement open source will continue to exist in countless uncoordinated forks, with no undercurrent of collective-force.
- https://launchpad.net/~googoleyes koanhead
  
  I don’t understand why you think that open-source development is at odds with standards organizations. ICANN does not set standards, it assigns names and numbers. IETF, which does set standards, is open to everyone and the FLOSS communities are well represented there. In an open-source project it’s very very helpful to have an established standard to implement. It prevents wheel-reinvention and provides some confidence that your project will be compatible with others.
  - csforstall
    
    I understand most open-source supporters to be mostly anti-government, anti-corporate, no matter, whether that is “industry” standards or actual government standards.
    
    Now ICANN controls naming and assignments and as such it is an upper-level “standards” body, after all who implemented the domain expansion changes? The internet didn’t do so in an ad hoc or by itself.
    
    The anti-governmental stance taken by much of the open source community suggests that any attempt at creating true clearing-house standards (for software like linux) is unlikely.
    
    I have taken to understand that clearinghouse standards aren’t what the Open Source community wants, and as such there is a real dearth of standards when it comes to open source operating systems. It’s notable here that the recent advancements in linux have been brought about by companies (ubuntu etc) that essentially produce low-level, user oriented standards in-house.
    
    I’m lamenting the lack of these sort of user-level standards originating from a separate a-political body. Ironically enough this is one of the possible ideas that Cory hinted at, or at least invited with his thought experiment.
    
    That all said, I think you are misunderstanding the “level” that I am referring to. The IETF deals with broad standards, while for example organisations like the AWS (American Welding Society) set highly specific industry standards for the appropriate technique to apply in the appropriate situation.
    
    I am more lamenting the dearth of such origination, “closer to the coal face” in terms of Cory’s lecture, as this has been one of the major concerns I’ve had about the movement since I’ve started using and observing Open Source myself.
jhertzli

The right to buy a general-purpose computer doesn’t count as a property right? The right of humans to own property doesn’t count as a human right?