Thoughts on augmented realities

 3545 3334569530 73507Ca066

CC-licensed photo from robinmochi's Flickr stream

Augmented Reality is definitely trending up the Hype Cycle in a big
way. The past year has seen explosive growth in this nascent field
buoyed by the rise of gps-enabled, cloud-aware smart phones. The
marketing hype has, of course, been even more resounding, like a
wailing chorus of virtual vuvuzelas trumpeting the next great wave of
advertising (I couldn't resist). But beneath the hype and the fluff is
a thriving community of innovators & designers working to weave this
technology into the very fabric of our lives.

As a quick review, augmented reality is a context-aware UI layer
rendered over a camera stream or other transparent interface. This is
typically mediated by geo-location, orientation, physical markers
(those funky UPC-like symbols), and visual recognition. In this manner
AR is able to reveal visually the hidden data shadow of our world,
like showing you the nearest coffee shops or details about the air quality in your city.
The mobile device gets info about where you are and what direction
you're facing, goes to the cloud to look up data appropriate for the
vicinity, then renders it over the camera stream in a way that updates
as you move.

A whole industry has been born around this premise, dragging in
images, annotations, and data to overlay on the camera stream of our
mobiles. But the really interesting stuff is yet to come. As
standardization issues, hardware issues, and numerous UI design
challenges sort out in the next couple of years, concurrent with the
development of AR-specific devices, our interaction with visualized
data will become more and more specialized and appropriate to our
individual needs. The clutter of markups that currently plagues many
AR apps will be attenuated by algorithms that know our interests and
affinities and block out the elements we wish to avoid. Just like
Amazon makes recommendations based on your click & purchase history,
AR apps will screen out the noise and provide us only with the data we

When paired with the massive deployment of embedded sensors AR becomes
a lightweight visualization layer for interfacing with the
instrumented world. Civic workers could see underground cables and
pipelines. Homeowners could see real-time energy & network use. Police
and early responders could post visual warnings cordoning streets and
alerting to hazards. Ecologists could determine water & air quality
at-a-glance. Ecosystems begin to have a voice, communicating soil
contamination to observers. Public facilities like park benches,
utility poles, and street signs could hold annotations & links created
by community members, made public or gated by in-group permissions.
Geographic social annotations could mark up our cities with tags and
content. Virtual worlds might break out of the box and overlay on the
physical plane. The environment suddenly becomes much richer – and
potentially much nosier – with a flood of information. Augmented
reality promises to exteriorize the cloud, drawing it out across the
world canvas and making visible our social fabric. But it doesn't
promise to mediate or regulate that content.

We risk myopia, disconnection, visual occlusion, fragmented realities,
reinforced tribalism. Consider the seemingly-inevitable future where
eyewear mediates a cloud-aware augmented interface with the world.
Perhaps you opt to obscure ethnicities or anyone not connected to the
net. Ghettos look much nicer when painted over with high-res colors
and dancing sprites. The world you experience is really only shared by
the other people running your default layer set. Maybe you see
paycheck information or health records or political affinities of
those you pass, measuring up the once-private lives of your community.
Perhaps the most popular layers are hacked to display swastikas or
porn or spam swarms or simply to black out your view in the middle of
the morning commute. How does the layered world enable crime, gang
affinities, and political or religious extremism? What inevitable
inequities might arise between those able to purchase such access and
those condemned to the dark poverty of quiet disconnection? Do the
wealthy become even more enhanced & capable compared to the
underclass? And what are the risks of getting lost in the virtual
glitz? Are there considerations for how these augmented realities will
bring us closer to the natural world in which we're embedded? And just
what is "real" or "natural" anymore?

As connected social computing devices get smaller & smaller and nearer
& nearer to us, the weight of the cloud gets lighter. We carry around
immense computational power and almost immediate access to the global
repository of information. The mobile phone will eventually pair with
head's-up eyewear displays just as more and more people avoid
catastrophic disease & injury through the aid of embedded
brain-computer interfaces. As computation moves next to and into our
bodies, the cloud is breaking out of the screen and washing onto our
world. We grow more augmented with computation while our environment
is getting smarter and more aware and increasingly able to communicate
with us. It may very well be that in 5, 10, 20 years the world is a
much more visual, dynamic, and communicative place than we can even

For more of my explorations of this subject check out my articles
Open the Cloud: Heads in an Augmented World
and Cognition & Computation: Augmented Reality Meets Brain-Computer Interface.