Thoughts on augmented realities

 3545 3334569530 73507Ca066

CC-licensed photo from robinmochi's Flickr stream

Augmented Reality is definitely trending up the Hype Cycle in a big way. The past year has seen explosive growth in this nascent field buoyed by the rise of gps-enabled, cloud-aware smart phones. The marketing hype has, of course, been even more resounding, like a wailing chorus of virtual vuvuzelas trumpeting the next great wave of advertising (I couldn't resist). But beneath the hype and the fluff is a thriving community of innovators & designers working to weave this technology into the very fabric of our lives.

As a quick review, augmented reality is a context-aware UI layer rendered over a camera stream or other transparent interface. This is typically mediated by geo-location, orientation, physical markers (those funky UPC-like symbols), and visual recognition. In this manner AR is able to reveal visually the hidden data shadow of our world, like showing you the nearest coffee shops or details about the air quality in your city. The mobile device gets info about where you are and what direction you're facing, goes to the cloud to look up data appropriate for the vicinity, then renders it over the camera stream in a way that updates as you move.

A whole industry has been born around this premise, dragging in images, annotations, and data to overlay on the camera stream of our mobiles. But the really interesting stuff is yet to come. As standardization issues, hardware issues, and numerous UI design challenges sort out in the next couple of years, concurrent with the development of AR-specific devices, our interaction with visualized data will become more and more specialized and appropriate to our individual needs. The clutter of markups that currently plagues many AR apps will be attenuated by algorithms that know our interests and affinities and block out the elements we wish to avoid. Just like Amazon makes recommendations based on your click & purchase history, AR apps will screen out the noise and provide us only with the data we need.

When paired with the massive deployment of embedded sensors AR becomes a lightweight visualization layer for interfacing with the instrumented world. Civic workers could see underground cables and pipelines. Homeowners could see real-time energy & network use. Police and early responders could post visual warnings cordoning streets and alerting to hazards. Ecologists could determine water & air quality at-a-glance. Ecosystems begin to have a voice, communicating soil contamination to observers. Public facilities like park benches, utility poles, and street signs could hold annotations & links created by community members, made public or gated by in-group permissions. Geographic social annotations could mark up our cities with tags and content. Virtual worlds might break out of the box and overlay on the physical plane. The environment suddenly becomes much richer - and potentially much nosier - with a flood of information. Augmented reality promises to exteriorize the cloud, drawing it out across the world canvas and making visible our social fabric. But it doesn't promise to mediate or regulate that content.

We risk myopia, disconnection, visual occlusion, fragmented realities, reinforced tribalism. Consider the seemingly-inevitable future where eyewear mediates a cloud-aware augmented interface with the world. Perhaps you opt to obscure ethnicities or anyone not connected to the net. Ghettos look much nicer when painted over with high-res colors and dancing sprites. The world you experience is really only shared by the other people running your default layer set. Maybe you see paycheck information or health records or political affinities of those you pass, measuring up the once-private lives of your community. Perhaps the most popular layers are hacked to display swastikas or porn or spam swarms or simply to black out your view in the middle of the morning commute. How does the layered world enable crime, gang affinities, and political or religious extremism? What inevitable inequities might arise between those able to purchase such access and those condemned to the dark poverty of quiet disconnection? Do the wealthy become even more enhanced & capable compared to the underclass? And what are the risks of getting lost in the virtual glitz? Are there considerations for how these augmented realities will bring us closer to the natural world in which we're embedded? And just what is "real" or "natural" anymore?

As connected social computing devices get smaller & smaller and nearer & nearer to us, the weight of the cloud gets lighter. We carry around immense computational power and almost immediate access to the global repository of information. The mobile phone will eventually pair with head's-up eyewear displays just as more and more people avoid catastrophic disease & injury through the aid of embedded brain-computer interfaces. As computation moves next to and into our bodies, the cloud is breaking out of the screen and washing onto our world. We grow more augmented with computation while our environment is getting smarter and more aware and increasingly able to communicate with us. It may very well be that in 5, 10, 20 years the world is a much more visual, dynamic, and communicative place than we can even imagine.

For more of my explorations of this subject check out my articles Breaking Open the Cloud: Heads in an Augmented World and Cognition & Computation: Augmented Reality Meets Brain-Computer Interface.


  1. Considering the website you link to is named “,” it’s ironic that you neglect to mention the most likely use of augmented reality devices: to have everything you do and everywhere you go tracked by private companies, building up a marketing profile on you for sale to the highest bidder.

    I for one can’t wait for flash ads to appear in real life, especially ones tailored explicitly to me using the information trails I leave behind like pheromones. I can’t freaking contain my joy just thinking about the day when my augreal app tells all my friends, “Hey! Your buddy Beelze has stepped into Jim’s Discount Smut Shoppe on 3rd & Holly, just two blocks down! Wanna join him?” It’s going to be so awesome getting a automated ticket in the mail for some minor infraction I committed with my glasses on.

    If the future’s that bright, I think I’ll wear shades.

      1. Shyeah, barely. Imagine how much better it’ll be with AR, when Mars, Inc. can display Snickers ads every time I pass a candy aisle, keying off my history of past Snickers purchases and the fact that I haven’t eaten recently. Why wait, ‘yknow?

    1. that’s dead right. Our shopping experiences will be enhanced in terms of speed, ease, accuracy etc. but this is really only benefitting the retailers. For the rest of us it’ll be like our simple corner store is turning into a Marrakech bazaar – except that the merchant has been reading our diary, while we’re negotiating blindfolded, behind a curtain, through a translator.

    2. I’m all for individuality and choice, but is it all that awful to have your buying behaviour sent to marketers?
      It’s only information and you still have free will to react, or not react to advertisements.

  2. I was just at Gettysburg and took their $24 2-CD audio tour-in-your-own-car. Great stuff. But AR would be an outstanding app: view the scale of the troops (instead of just regiment markers), the actions of the battle played out in faster-than-real-time (three hours instead of three days is probably about right), etc.

    I can see Layar, Google Goggles, etc. having paid “channels” that you can subscribe to, to view the AR at a given physical space, and later review them in virtual space. I’d buy a “National Park Virtual Pass” to help plan out a trip to Yellowstone and understand the course of the Civil War (separate trips, most likely).

    It’s coming, it’s unavoidable, and it’s wonderful.

    Next step is the cheap knockoff versions sold in sunglasses at the roadside, completely with coupons for a discount Gettysburger.

  3. Media design student here. When I visualized AR, I imagined having the power to ERASE ads in the physical world. Since you can overlay anything over the image you see, why not paste big whiteouts on billboards, brand names, magazine advertisements, or radio ads?

    Not EVERYTHING to do with AR is going to be push-data from some big media company. At least, not if you guys developing the future get on the fucking boat and row it in the right direction.

    PS: captcha is “murks Pentagon.” Should we discuss the implications AR has for the fog of war?

    1. “not if you guys developing the future get on the fucking boat and row it in the right direction.”

      no doubt the killer app will involve something related to xray vision.

  4. There’s an inherent “making our marks” aspect to any form of media or even a blank wall. The reality of that goes back perhaps to the Altamira cave art and forwards to the CCCKC Hackerspace’s “Laser Graffiti” shows.

    Augmentation of our senses is one of those concepts we’ve barely scratched the surface layers of. And it’s going to be an interesting ride as we deploy some uses of AR. Rule 34 intersecting AR is likely the most certain prediction it’s possible to make.

  5. couple of years back I had this plan for a two page comic spread about a guy who witnesses theft in a convenience store. He pursued the assailant on foot down the back alleys in a montage of shots, nearly getting himself killed in the process.
    Long story short, the thief turned out to be an AR advertisement for a security or insurance group.

    I’d hate to think of what other ads/pop-ups could occur in a fully saturated AR social environment

  6. I once had a dream about being in an ARG. Yeah, I do some of my most creative thinking while I’m sleeping or while I’m not thinking about the problem that I’m trying to solve. So sue me.

    Anyway, from what I recall, each player used an iPhone or something similar. The game was in a city, and as you wandered around you could tell where other players were (GPS on the phone). Each player or set of players would be given a different objective which translated IRL to finding objects and photographing them. Presumably one could do that by photographing objects with barcodes or QR codes.

    The video stream would be altered to show you different aspects of the game, maybe overlaying players, storefronts, objects.

    Players could interact by gesturing with their phone (accelerometers), and collecting objects would give you access to different gestures.

    Anyway, those were the mechanics. I only remember it as “fun”, and as we all know, in game design plots are cheap.

Comments are closed.