Who should know what's happening in your computer? Who should control it?

My latest Locus column is "What’s Inside the Box," a discussion of whether owners, users or third parties should be able to know and/or control what their computers are doing:

The answer to this that most of the experts I speak to come up with is this:

The owner (or user) of a device should be able to know (or control) which software is running on her devices.

This is really four answers, and I’ll go over them in turn, using three different scenarios: a computer in an Internet cafe, a car, and a cochlear implant. That is, a computer you sit in front of, a computer you put your body into, and a computer you put in your body.

Cory Doctorow: What’s Inside the Box



  1. Cory: 

    Does “users control what is running on their computers”  imply “users may modify what is running on their computers??” 

    If yes, then I think we need a caveat in a couple of these situations.

     By all means, patch and modify your desktop and tablet to your heart’s desire, but if you choose to develop your own home-brew patches for the computer which controls the anti-lock braking of your car, I’m not quite sure I want to be sharing the road with you. 

    And if I decide to tinker with the software that controls the timbre and volume of my hearing aid’s output,  I should bear some responsibility if I fail to hear your car coming my way.

  2. Cory,
    As a tinkerer, I recognize the value of the “users control” model.  However, a few years ago I made a deliberate decision to be very selective about what I tinkered with and where I wanted to be bleading edge, because things I tinker with frequently break.  Users control devices seem to have a higher failure rate than owners know or owners control  paradigms.  Granted, the state malware or carrier IQ possibility are very real.  I still get the sense that if I had an artificial organ, I would want a company with lots of full time QA engineers whose livelihood depends on me not dieing because their software was buggy.  They are going to do a better job securing the system than I will.  Same with my car software and probably even my phone.  I want to agree with you, but as a busy parent, I don’t have time to guard all my freedom as well as I should.

    1.  Mmm, sometimes it’s not the companies pushing the safety aspect, it’s Gov regulations. Settling with (or out-litigating) a few private individuals is nothing compared to your device being removed from sale entirely.

  3. I agree that users should be able to control  the software running on their device if they choose to do so. However in practice this always translates into “users NEED to know what software runs and be familiar with the arcane rules governing computing.” which is the whole reason the Apple “just take care of it for me”-style walled garden is popular in the first place. If the choice is between me needing to micromanage all my devices or giving up some measure of control I’ll choose the second, and I say that as a sysadmin.

  4. You. You and Lawrence Lessig. I can imagine the future again. Thank you!

    Implementationwise, UEFI, Apple, etc are trademarks, so a hardware curation organization that identifies good “users control” hardware should also be a trademark?

    VicqRuiz, people can physically alter the brakes of the cars they drive. Innate goodness and fear of prosecution if they hurt someone mostly prevents people from abusing this ability. 

    Edward Fine, if you like you can choose someone you trust not to abuse you, and follow their recommendations for hardware and software. The important thing is that this is your choice.

  5. From a performance standpoint, I want a higher level of control. The more unnecessary processes that are removed, the better the performance of the device  (I say this as someone who firmly believes that many tech companies are pushing forced obsolescence down our collective throats; the amount of tech waste is increasing, even though many devices are still capable of functioning, albeit perhaps a little slower).
    On the other hand, many users don’t want to be bothered with details, as others here have written here. As Paul mentioned, it is a choice, and it is a trade off.

  6. Cory,
    I like “users control” but reality is “manufacturers and developers control” and “nobody knows what the hell is really going on”. It’s safe to assume that companies and governments and hackers are spying on us all the time, and the only thing going for us is that the eavesdroppers are overwhelmed with noisy information. Want convenience? Use smartphones etc. Want privacy? Use primitive tech.

    Meaningful user control is possible, with open-source hardware. Say you build a computer out of 20-year-old discrete logic chips, and it has 16k RAM and 1k program code, which you hand-coded in machine language: it’s only capable of doing one simple task, but you can be certain it’s not doing anything nefarious. If you have the time and money, you can order custom chips from a fab facility, and inspect them under an ordinary optical microscope to make sure no ‘bugs’ have been added. In a few years, I imagine that serious DIYers will be etching their own silicon… not 33nm, but even 300nm would be useful.

    Software is more problematic, even in a 100% open-source world. If you’re a geek like me, you can write your own little OS and compiler to run your own little programs. To run other peoples’ apps in a secure AND useful manner, you need fine-grained control over permissions. Android perms are a half-assed version of the right idea. The permission system needs to be comprehensible to developers or they won’t use it. And to be confident that it works, you need to be able to read and comprehend the entire OS code… which means it’s gotta be pretty damn simple. If you’re not a geek, it’s okay to rely on your geek friends to tell you it works. But today’s open-source software is so complex, there’s not one geek in the world who understands it. All we can say is, it’s not good enough.

    You want control? Be prepared to give up all the bells and whistles.

    Your story “Human Readable” captures the idea pretty well.

Comments are closed.