Zoom for audio enables you to hear a single conversation in huge crowd

Two physicists developed a new technology called AudioScope that apparently enables you to zoom in on sounds in huge, loud places like sports arenas or lecture halls. Physicists Morgan Kjølerbakken and Vibeke Jahr, formerly of the University of Oslo, were were experimenting with sonar when they hit upon the idea for the AudioScope, which is based on a circular array of 300 microphones and a video camera. They've now launched a company, Squarehead, to commercialize the system. From New Scientist:

The AudioScope software then calculates the time it would take for sound emanating from that point to reach each microphone in the circular array, and digitally corrects each audio feed to synchronise them with that spot. "If we correct the audio arriving at three microphones then we have a signal that is three times as strong," says Kjølerbakken. Doing the same thing with 300 microphones can make a single conversation audible even in a stadium full of sports fans.

"Audio zoom picks out lone voice in the crowd" (via @chris_carter_)

David Pescovitz is Boing Boing's co-editor/managing partner. He's also a research director at Institute for the Future. On Instagram, he's @pesco.

MORE: Technology

More at Boing Boing

Ants and Stars: Bruce Sterling and Jasmina Tesanovic visit the Sardinia Radio Telescope in Italy

The Snowden Principle

ackpht

The stadium example is probably a bit exaggerated, but only by degree. No reason this thing shouldn’t work on some scales, though. It probably helps to have all the sound sources in the plane of the microphones. A 3-D solution would take more math and present some aliasing issues.

But once you have the data from all the microphones, you can listen to any of the individual conversations at your leisure.

I recall a system with distributed microphones that police departments can use to quickly locate the position of gunshots within a city.
- Oren Beck
  
  One of the most deployed gunfire location systems is:
  
  http://www.shotspotter.com/
  
  As for the multi microphone arrays in the lead article, I’d argue for comparing it to phased array radar in “passive or ambient target energy modes” since there’s no need for a “target painting beam” if the target=a sound source. The articles of simplest description often mention using ambient RF like Broadcast or Cell transmitters. And of course- Individual devices that transmit signals. The “reason” for my mentioning RF devices in connection with this Zoom tech is that the most unsuspected cruft anyone might have on their person could be RF excited in Cavity Resonator modes akin to the infamous bugged “Great Seal” embassy exploit.
  
  Yes, your car’s keyfob remote in concept at least- could be exploited at minimum for aiming “confirmation” of this Zoom system. Or even as – yes, a microphone. Which renders the precautions of powering off phones etc no longer good enough. After all- if you’re an evil entity with zero regard for human life there’s no worry in using insane power levels to resonate a car’s keyfob. It’s not paranoia to speculate when there’s tech legacy reminding us that real world spy games did such things years ago. As sadly- there are quite real evil folks in our mundane world.
Kev

How is this new, besides the uber-complicated 300 microphones? Has nobody heard of a simple parabolic microphone?

http://en.wikipedia.org/wiki/Parabolic_microphone
- Man On Pink Corner
  
  It’s different from a parabolic microphone in that it records the data from the individual mics and allows the operator to “aim” the microphone at any person within their field of hearing — even after the fact.
  
  A single parabolic mic can intercept sound from only one emitter at a time.
MandoSpaz

I for one welcome our new all-hearing overlords.
Teller

Well I would love to hear the trash-talk during an NBA game w/o having to buy courtside seats.
RedSun

Some aren’t seeing the upside to this. Better live band recordings, finally being able to get the vevuzela out of football game broadcasts, etc.
JonStewartMill

Can they build this into something the size of a hearing aid? ‘Cause I would buy it.
Anonymous

Here’s hoping these things will be ridiculously expensive, so that only the government can afford them.
- pwandz
  
  God, it’s the governments that we DON’T want to have these devices. Privacy of conversation in public places => destroyed.
  
  This is something you’d expect to show up in Little Brother if Mr Doctorow was writing it now.
adam

This product may be new, but this technology has been available for several years… since at least January 2008.
sic transit gloria C.F.A.

And the spooks probably had something similar years before that.
- imag
  
  That’s what I was thinking. “The Conversation” popped into my head. If the spooks didn’t have it before, they do now.
  
  The implications for this are pretty staggering, and not for professional sports.
- Anonymous
  
  And how exactly are the spooks going to place 300 microphones in the middle of a room?
Powell

I am sure this will only be used at sporting events. Go to back to sleep America.
Anonymous

I’m pretty sure every company that has been involved in sonar has though of exactly this system, I know the one I work for did prototypes a few years ago.

Cool for them finessing it into a product though.

It doesn’t make clear whether the array is a ring or a plane though?
CC

Pah! I had a toy in the 1980′s called The Big Ear which was a plastic, rifle shaped device with a microphone set in a dish at one end and earphones at the other. It could pick up distant conversations. They might have the accuracy, technical superiority and gain but I had the shiny red plastic casing.
- knoxblox
  
  I was going to say something similar…a more sophisticated version of the wrapping paper tube placed against the ear.
- secretlab
  
  That was the only toy I ever owned which actually scared some of the neighbors (think “death ray”.) It was fun being mis-perceived as a mini mad scientist.
Xenu

Oh yay, more creepy stalking tools. Just what the world needs.
wiredfool

Hello panopticon.

Given enough computer power, if they can do one, they should be able to pick out all the conversations in a room, and their location. And it doesn’t need to be realtime, so long as they have the time coded feeds from all the microphones.

Creepy.
- Anonymous
  
  wiredfool..you have just scared me. The only real limitation that I can see is that the microphones have to be spread over a wide area, preferably as wide or wider than the crowd.
- anansi133
  
  Now I hope they decide to reboot _The Conversation_ using this kind of technology as the McGuffin.
  
  If you were in such a place, though, I wonder what kind of countermeasures would be possible, to generate a kind of cone of silence?
  
  I’m more concerned that cell phone mics could be activated remotely without the subject’s knowing. You could pull the battery out of your own phone, but what of the person standing next to you?
Anonymous

This is obviously sound interferometry. It’s the same concept that is applied to electromagnetic waves in radio and optical telescope arrays (Very Large Array, Very Large Telescope, etc.). The use of the same technique for sound flows so naturally from even a basic understanding of the principles of thing, I doubt there’s anything really innovative in this new system. Marketing ploy…
Anonymous

SOSUS is an underwater sonar array for tracking submarines. The algorithms for beam forming have been used by the US Navy for a loooooong time. The technology takes all the mikes as inputs and then places the “sound events” (like a bubblegum pop) on the geographic grid based on the time the sound arrives at each mike. Once you locate a contact, then you can tune the listening point back to it. In real-time or later on. It is NOT like a megaphone, cardboard tube, etc. The array does NOT have to be larger than the crowd size. You manage to achieve stereoscopic vision with two eyes that are three inches apart, not on 100′ stalks. The required separation of the mikes is a function of the wavelength, speed of sound (in air, water, etc), and your sampling resolution. With enough computing power you can turn all the noise into signal.
Patrick Dodds

What could possibly go wrong with this….?
IOOIIIOIIO

This was one of the demos that Inmos built when the Transputer first came out.
I think one of their undergraduate trainees developed it.
GyroMagician

I simulated one of these for an undergraduate physics project in about 1995! Sadly I never got as far as building one. It’s a very cool idea, if you ignore the creepiness.

@wiredfool – yeah, especially if you think how cheap storage is now, and how small audio is compared to video. Put the microphone array on continuous record, come back when you need it.
Bouillion Cube

The ability to steer a “beam” based on timing across an array is not new, either in theory or in practice. The US Navy’s SOSUS is the largest example. Several hundred microphones feeding a computer that can scan for “sound events” like a submarine opening torpedo tube doors, and then calculate the position based on when that sound arrived at each of the microphones. Or it can be used to listen to a single point in space by timing the microphones tracks. It is NOT amplifying a sound like a cardboard tube. The array does NOT have to be larger than the crowd. Your eyes are not wider than the things you see, but you do manage to achieve stereoscopic vision.
- Tdawwg
  
  Tom McCarthy’s new novel C, partly set in WWI, has a part where the Brits do the same thing: they’re able to triangulate where German gun emplacements are based on a set of mics, and by calculating when the sound of firing hits each month, plotting this on a map, etc. (Except he describes it way better.)
monument

Yes, yes. This is all well and good, but did you notice the bubble gum?
ubarch

What the video describes is pretty much textbook acoustic beam-forming, and this technology has been well known for decades. When you go to the airport, you can even see radar versions of this setup. They look just like scaled-up versions of the device shown in the video. There’s even a company in Germany making a parallel version they call an Acoustic Camera:

http://www.acoustic-camera.com/

This makes it a little odd, since I would assume that these scientists have done something to advance the state of the art, but they don’t seem to be very keen on talking about it. Maybe it’s just a very well-engineered beam former.

It isn’t that hard to make low-end versions of these things (i.e. without the capability to process and stream data continuously) with ordinary electret microphones, LM386 op-amps, and ATMegas.
seanmc23

The New England Patriots had this years ago.
- Pliny the Elder
  
  Gah! Beat me to it! (I was going to say “No one tell Bill Belichick”)
toresbe

I work nextdoor to SquareHead; in the only other tenant in the building (main tenant was a film development lab; their moving-out made for an absolutely epic dumpster dive).

Their microphone is a big circular plate. They have a demo they don’t always power off, which has the system connected to a video conferencing system, and circles people who are speaking.
penguinchris

If one checks my boingboing comment history, it’s obvious I like to be cynical and trash-talk the subject of BB articles as much as the next guy, but I’m a little surprised this time – everyone’s saying how it’s old news and so on, but still I thought it was very cool!

Indeed I did try out the parabolic microphone toy they had at the Discovery store in the mall when I was 11 or 12, and thought it was cool. I never managed to convince my parents to ever buy me anything from there (it was an expensive store), but yet they took me there frequently (I suppose we were going to the mall anyway) – some kind of torture I suppose. Now I’m getting off-topic, but I’ve decided I’m going to keep my kids out of toy stores and the like as much as possible, if/when I have kids that is ;)

Anyway, the parabolic microphone was cool but fairly limited in practicality. I remember reading about the laser systems that pick up sound waves from window glass. I always like to read about or see in action this kind of stuff. I loved “The Conversation,” though not just because of the surveillance aspect – great film all around.

I can imagine plenty of uses, both cool and creepy/invasive, for this kind of technology and I think they chose a great setting for their demo. I don’t generally watch sports, but when I have I always wondered what people were saying – they’ll show close-ups of people on the sidelines with no audio all the time.

My defense from this is that I naturally have a very soft voice. In loud environments I tend to only talk when absolutely necessary, because it’s uncomfortable to raise my voice. I’m sure there’s a Seinfeld-esque way to describe me. I don’t think I’ll have to worry too much about the surveillance possibilities here ;)
regularfry

…and now, I’m kicking myself. Being more of a theatrical than sporting type, when I was at college, I sketched out the design of a system remarkably similar to this for stage use, so that you could do away with radio microphones. Then I calculated how much it would actually cost, and figured that at that price there was a market for “5, maybe 6 worldwide…” and ditched it.

This is the one and only time I’ve ever regretted not being more enthusiastic about spectator sports.