Real life version of Minority Report's user interface

At TED2010, John Underkoffler gave a demo of his remarkable g-speak user interface. He was the science advisor for Minority Report.

Remember the data interface from Minority Report? Well, it's real, John Underkoffler invented it -- as a point-and-touch interface called g-speak -- and it's about to change the way we interact with data.
John Underkoffler led the team that came up with this interface, called the g-speak Spatial Operating Environment. His company, Oblong Industries, was founded to move g-speak into the real world. Oblong is building apps for aerospace, bioinformatics, video editing and more. But the big vision is ubiquity: g-speak on every laptop, every desktop, every microwave oven, TV, dashboard. "It has to be like this," he says. "We all of us every day feel that. We build starting there. We want to change it all."

John Underkoffler points to the future of UI

Mark Frauenfelder is the founder of Boing Boing and the editor-in-chief of MAKE and Cool Tools. Twitter: @frauenfelder. Come and hear Mark speak at the ALA conference in Chicago on July 1.

More at Boing Boing

Eurovision 2013: An American in London

The technology that links taxonomy and Star Trek

Anonymous

I can’t see any job where this type of use would be beneficial. Imagine using those arm gestures for more then the 15 minutes of this TED talk. In the media editing example, imagine having to wave your arms around for 8 hours a day. These types of interfaces look wonderful in demos, but they just seem impractical for actual use (to me). What am I missing?
Robert

>:( User interfaces should become less physical, not more physical!
Clay

It was cool to see one actual practical application of G-Speak, but most of the presentation was yet more academic tech demos that didn’t seem to really approach any practical use case.

I still vastly prefer something flat and less sexy like http://10gui.com .
- robulus
  
  Nice.
weaponx

This guy is the Quentin Tarantino of user interface design.
Daemon

Nice toy. Notice that it takes a lot more physical effort to do what you can already do with a mouse & keyboard.

Good for presentations though.
Bucket

Man, Ted talks sure can make any random idea seem innovative and brilliant.

You get someone up there with a stylish headset, spewing whatever in a nonstop, rat-a-tat-tat style delivery, give the voice a nice big-hall reverb, show clips of a breathless audience who gasp or laugh on cue, and bingo! Whatever the speaker is presenting seems totally amazing!

I would love to see a Ted talk about this thrilling new invention the Toothbrush and how rubbing it around in your mouth every morning will make you 10 times smarter and fart rainbows.
se7a7n7

Does this mean I’m going to have to buy a wood ball printer?
untonetdemi

Sorry to be off topic, but I thought Mitch Hedberg’s accent was a personal one, not a regional one. But this guy talks like him too. So in which region of America do people talk like this?
dchamp

Hmmm… this guy has about zero percent of his historical facts straight… like there was no such thing as a network, that the Macintosh invented the GUI…
JohnByron

Life imitating art: the virtual reality scene in the ’94 movie Disclosure, made real. And no less goofy. I can definitely see how Oblong’s patented gestures will make my microwaving experience so much more enjoyable then pressing Popcorn…Start.
- Ed Frome
  
  make my microwaving experience so much more enjoyable then pressing Popcorn…Start
  
  Wut? You have to press two buttons? Contemporize, man. Today’s microwaving ovens need only “popcorn” to be pressed.
Anonymous

I don’t see the value.

He makes these bold claims of cooperative efforts, yet his demonstration fails to show the multiple users accomplishing anything other than manipulating the display components awkwardly.

He talks about the value of tangible spatial manipulation, but there is no tactile feedback, no precision, nothing but visually impressive ways of displaying data.

Ultimately, it seems like a lot of work for no extra reward. There is no increase in efficiency, rather a decrease. Countless additional components needed to track hand movements, lenses and detectors, parts of all sorts which will need maintaining and cleaning and replacing and energy. Data manipulation is slow and confusing.

Watching the bit with browsing through the images was visually arresting, but conceptually confusing. I don’t want to have to visualize my data three dimensionally in addition to all the other attention I have to devote to it. I want to scroll through my images quickly and linearly, with good cross referencing and indexing to make it easy to find specific things out of many.

It all sounds very familiar if I stop and think back to thirty years ago, when the concept of Cyberspace caught the imaginations of people everywhere. Look at vintage games, movies, comics, and everything else. Cyberspace was going to be the new revolution, a virtual world of data in three dimensions. This just seems like the same sort of silly, ineffective romantic dreaming.

We don’t want pretty computers, we want operable computers. I’m reminded of Zaphod Beeblebrox, trying in vain to choose a radio station by waving his hands subtly through the air, having to hold unerringly still to get a half decent reception of the station he actually wants.

I mean, for goodness sake, as Eddie Izzard has pointed out, we can’t even make toasters and showers that work the way we want them to reliably.

~D. Walker
Anonymous

Why do people feel the need to have data represented in 3d. This is no different then that cheesy interface that they slapped on UNIX in jurassic park. http://www.youtube.com/watch?v=dFUlAQZB9Ng

Anything that they did in this video could have been done just as easily with a mouse, without the need to remember all kinds of weird hand gestures. Wearing gloves with tracking sensors stuck all over it is basically just turning your hand into a mouse anyway.

I don’t see this ever becoming common place. You can’t just throw a few fancy graphics up and pretend it’s something amazing. This is just another case of reinventing the wheel.

Having said all that GUIs that look good are pleasing to the eye. I would agree that many of the current GUIs that we use consist of nothing more then boxes, which makes sense when you think about how TVs project data the same way.

I would like to see GUIs get a facelife but I don’t see the need for the hand gestures.
bshock

As I recall, the Minority Report interface was the most awkward, impractical, and generally useless cinematic example of futurism I ever saw. Ideally, I shouldn’t have to do more than think “make it happen” for the computer to do its job.
Anonymous

http://okcancel.com/comic/3.html
Orchestra Spy

That Bill Gates question at the end (not to mention the video game jab) asking when this technology will be released is because Billy boy himself has developed similar technology and is pushing for the holiday rush!

http://www.xbox.com/en-ca/live/projectnatal/
Bucket

In conclusion:

Information wants to be flat.
- The Hamster King
  
  “Information wants to be flat.”
  
  Mostly flat. We live in a 3-D world but our retinas are 2-D and our arms are short relative to the physical space that surrounds us. Which means that when it come to interacting with the real world on a moment-to-moment basis, left/right & up/down are more important spatial distinctions than near/far. “Far” means I have to get up off my ass and spend time walking over to it. I don’t want that inconvenient aspect of reality incorporated into my user interface. I want my UI to prioritize the left/right and up/down motions that arise naturally from the limitations of existing as a physical body at a specific location in 3-D space.
  - Anonymous
    
    On that note, perhaps a middle ground could be reached? Instead of a long depth of field, have everything be very shallow, a sort of low relief?
    
    We already sort of do that with Windows, which overlap each other spatially with a very slight illusion of depth. If that was deepened to something closer to say, an inch of depth, could something useful be done with that? I’m inclined to think not really, but it seems more useful than a full room display that tracks large arm motions.
    
    ~D. Walker
Anonymous

How do you type?
dculberson

I was gonna say, everyone will look as much like a toolbag as Tom Cruise did when he was using it. This sort of interface is not exactly elegant or simple. My biggest complaint about that movie was the ridiculous user interface on the investigator’s computer.
Lobster

Poor Philip K. Dick. He wrote this great story about the dangers of precognition and preemption, and what do people always talk about? The neato computer.
Willsan

Wow just watching that made my arms tired.

e
headfirstonly

Willsan’s comment at #12 is spot on. There’s a risk that user fatigue with interfaces can be overlooked until people start using the thing for extended amounts of time in real applications.

Over a decade ago I had a customer who was keen to incorporate big, sexy touchscreen interfaces for a project we developed. At the critical design review we rented the largest touchscreen monitor we could find, and let the customer loose on it so they could work through the content.

They gave up after ten minutes, moving the discussion on to the next agenda item. And touchscreens were never mentioned again.

g-speak looks wonderful and it would probably be fun to use for about ten minutes, but I wouldn’t want to use it intensively all day.
Kai Pak

One term: Gorilla arm. This interface breaks about every single rule for effective ergonomic design. Unless this was an application I was only using for a few seconds at a time, I can’t imagine how I could withstand interacting with something like this without experiencing extreme pain and fatigue in my upper-body.

As had been said in earlier posts, this type of interface looks cool in movies, but is completely impractical for real-world use.
The Hamster King

I worked as a programmer for a company back in the 90′s that had the grand idea of creating a new 3-D UI to replace the existing mouse/desktop metaphor. It was going to be much simpler and intuitive to use because it would mimic how human beings negotiate real-world spaces. It sounded brilliant and visionary in theory, but in practice it was a pain in the ass to use: large muscle groups asked to make fine-grained movements, disorienting context switches, lack of haptic feedback, awkward and slow navigation, and rapid user fatigue and frustration. Underkoffler’s demo looks very familiar to what I worked on 15 years ago, and it’s broken in similar ways.
slippy0

Got to play with most of these things (on display at MIT media lab). The Minority Report gloves are realllly fun.
robulus

I am thinking the personal computer version of this will be a version of today’s gesture based touch-screen technology, without the touch-screens.

So we drop the big arm gestures, and just stick with small hand gestures, but in 3D and without actually contacting the screen. This would negate a lot of the concerns mentioned above, and provide a very powerful and compelling personal experience.

I think the big arm gesture thing may still have a place in certain applications, such as air traffic control, but is really best for demonstration.
hansk

I’ve seen Jeff Han (of multi-touch fame http://www.ted.com/talks/jeff_han_demos_his_breakthrough_touchscreen.html ) blow a gasket when someone relates his work to Minority Report. He hates that style UI and cites several of the reasons mentioned above. We need to get these two on the same stage to battle it out :)

Btw – Why no reference to the Power Glove? :)

http://en.wikipedia.org/wiki/Power_Glove

That was release in 1989!
wylkyn

“It has to be like this,” he says. “We all of us every day feel that. We build starting there. We want to change it all.”

Sounds like wishful thinking to me, along the lines of “The Segway is going to revolutionize transportation!!!” I mean, when I’m sorting through photos, I’m not thinking “Gosh, if only I could be standing up with all these photos spinning in front of me in 3D while I’m making grandiose, arcane gestures to control it all.” People want simple. Keep it simple, stupid.
EH

I could save more than minutes per day if my mouse-wheel’s “side to side” scrolling functionality worked more than 10% of the too-wide places I try to use it.
Anonymous

Why is nobody mentioning the similarity to the Wii or project Natal? Similar waving the arms about sort of interface, at least initially :)

Loved the early media lab stuff though – the urban planning stuff in particular looked fascinating. This guys innovations might ultimately be subsumed into what the AR guys are doing.

Finally: He’s talking about making “the process of computation” (right down to the OS) integrated with the physical environment the computer user resides in, which is great for tasks like copy and paste. But, IMHO, very little I do with computers actually maps to the physical location of the PC, heck, very little of it even maps to three dimensions. Great to watch but seems to have little real application.
Sethum

I agree with most people that this interface is over-the-top special effects with zero or negative improvement in efficiency. However, there definitely are some situations where 3D hand gestures are better than a mouse. Namely, any situation where you combine the keyboard with mouse inputs. This is especially relevant when you are navigating a 3D space, like Google Earth.