Brain scans reveal our mind movies?

Discuss

48 Responses to “Brain scans reveal our mind movies?”

  1. kmoser says:

    is that an image of Jesus at 0:20?

  2. Kaffenated says:

    Just think – one day we can Facebook our brain! Zuck is going to love this.

  3. alfanovember says:

    Spiffy.  Now it’s only a matter of time before this too becomes a routine part of air travel.

  4. Dean Putney says:

    Finally everyone else can see how fantastic I know I really am.

  5. JIMWICh says:

    While it’s cool that reality has finally caught up with Wim Wenders brain scan movies, I’d probably trade this for software as cool as detective Winter’s Nintendo-style people tracking software or the Russian detective’s Bounty Bear program (which ran on “Vietnamese chips”).

    …just a minute… I’m searching… searching…

  6. Walt Guyll says:

    Wait ’till someone hooks one up to a 3-D fabricator.

  7. bardfinn says:

    I’ll take one.

  8. Nicholas Calderon says:

    think of how this will affect relationship dynamics. Over the long-term I can see benefit in this kind of transparency but damn will it F some peoples ideals of “love” and “sanctity”. Whatever those things mean, anyway, they’re sure to change.

  9. Private Private says:

    In the future, could this technology+hypnotism=very accurate witness testimony?

  10. This is the future of filmmaking. Anyone with an imagination will be able to rival Pixar.

  11. bcsizemo says:

    Am I the only one that sees a catch-22 with this?

    If you can only scan for things that have been known to have been seen, then does that mean they are using the original source of video/picture to help recreate the neural image?

    In a way it’s like the chicken and the egg.  I can understand if they need to calibrate the hardware/software to your brain, but after a few dozen tests shouldn’t it work by itself?

    • Yes, they say clearly they ARE using original pictures to recreate this images- a giant database of frames from youtube.

      I don’t really know what’s going on here, but it’s not showing a picture of what’s inside the brain. It’s matching fMRI data with youtube frames.

      Anyone know wtf this is?

      • hypnosifl says:

        Based on the Berkeley press release I think the way it works is this. First, they showed a few movie trailers to a person while scanning their brain, then the computer program was fed both the visual information from the trailers and the information about the brain activity, and it tried to figure out the correlations between them. Then based on that, the computer made predictions about what the brain activity might look like for a huge set of brief segments of youtube videos it has in its archive (I think each segment was only 1 second long although the wording wasn’t totally clear in the article)–these youtube clips weren’t actually shown to the person. After that, they then showed the person a different set of movie trailers, and this time the computer had no access to visual information from the trailers themselves, it was only given information about the person’s brain activity while watching them. For each brief segment of brain activity (1 second if I was reading it correctly), the computer compared the real activity during this time to the simulated brain activity it had predicted for all the youtube clips. Based on this, for each short segment it picked the 100 youtube clips where the simulated brain activity best matched what was actually going on in the person’s brain when they watched the second set of movie trailers, and averaged those 100 youtube clips together to get the final result.

      • Will Tingle says:

        As best I can tell:

        1. People shown clip.
        2. Scan taken.
        3. Frame-by frame scan compared to database of youtube clips that DOES NOT include the clip show prior to the scan.
        4. Tech uses best-match frames to attempt to reconstruct original clip.

        • hypnosifl says:

          That’s a little off, as I said in an earlier comment it’s more like:

          1. People shown movie clips while brain scan is being taken
          2. Info about both clip and scan is fed into computer
          3. Computer figures out correlations between the two, and uses that to predict what brain activity might look like for database of youtube clips that weren’t actually shown to people
          4. People shown different movie clips while another brain scan is being taken
          5. Computer is fed info from second brain scan (not from second set of clips)
          6. Computer finds best match between second brain scan and predicted brain activity in youtube clips, averages together best-match youtube clips to get final result

  12. usonia says:

    I’ve been obsessed with this since being haunted by that Wim Wenders movie when I was a teen. I don’t see some of the Orewllian bad news in this some of my pals do. And I think it’s a huge, amazing step towards the goal of interpreting brain activity, but this is pretty rudimentary: they’re matching the MRI info of image-calibrated brains to existing footage. It isn’t strict visual information, just impulses related to already-seen information. Which is still amazing, but I’m impatient for the real goal. I want to get to SEE my dreams again.

  13. Also similar to Until The End of the World is the satellite falling from the sky. 

  14. “Until the End of the World” looks fascinating but it’s not available on DVD in the US. Worth $25 + s/h? 

    • William says:

      Abbie, there’s two versions of Until the End of the World by Wim Wenders. The regular cut on 1-DVD, and an epic several-hour long version on 3-DVDs. Both are great. Both only exist as imports, and the long version is missing English subtitles for the moments when non-English is spoken.

      For many years, people hoped that the film company Criterion would license and release the long version on DVD in the United States. Then they hoped they’d release it on BluRay. Then they hoped other companies would. Then they gave up. Wim Wenders gets no respect in the US.

  15. Nylund says:

    I remember an old SF story….William Gibson maybe?…that involved the ability to record the music in people’s minds and thus the best “musicians” didn’t even have to be able to play an instrument.  There was a passage in there along the lines of, “imagine all the people in the history of the world that made wonderful music in their mind, but lacked the technology to extract it, and all the beautiful music the world will never hear because of it.”

    Or something like that.

  16. Bass says:

    If anything this is more like the film Brainstorm.  The death scene at the end is a trip!
    http://en.wikipedia.org/wiki/Brainstorm_%281983_film%29

  17. William says:

    Or the movie Final Fantasy the Spirits Within. Recording dreams is a popular desire.

  18. RobDobbs says:

    So… if neither Netflix nor Apple is renting a copy of Until the End of the World, where does one rent a dvd these days? Sounds like a legitimate case for finding a copy on the torrents. 

    • Jim Moskowitz says:

      Duncan Creamer wrote “So… if neither Netflix nor Apple is renting a copy of Until the End of the World, where does one rent a dvd these days?”

      How about a good library?

  19. Matt Shapiro says:

    Does anyone know if the clip above represents the entirety of the presented and reconstructed clips?  If this is unedited, it’s pretty spectacular.  If it’s edited I don’t see how this is much different from the researchers giving themselves a Rorschach test or watching the Wizard of Oz to the Dark Side of the Moon :)

  20. Private Private says:

    I’m tweaking on this story. I’m a visual artist and I can tell you that my creative process involves me recreating the fully realized visions of paintings that I dream. These mental images are very clear to me. If it is now possible to see those images before I paint them…well…holy eff.

    I have also just had a wave of terror wash over me thinking about the possibilty of seeing our dreams, as one YouTube commenter has postulated about this technology. I can’t imagine how seeing the sometimes horrific/random images that we do in our sleep would be anything but psychologically scarring. Call me a luddite, but I don’t want to see my psyche, thanks very much.

    Apologies for the double-post, but I’m bowled over by this.Thanks David, for the link and the nightmares.

  21. rbdewell says:

    I haven’t read the research in detail, but here is quick description of the paper. 3 people each viewed 2 hours of movies while in an fMRI. The researchers then modeled the changes in the fMRI to the movies and tested the model by showing the subjects 9 minutes of new movies (not in the first 2 hours). The images on the right are the result of averaging the movies which produced the most similar fMRI signals.
    Its a interesting idea, but their claim that this could be used for decoded dreams or halucinations is very far fetched.

  22. The Chemist says:

    When cutting edge AI is more about blowing people to smithereens, does anyone really think this will be used for anything remotely humane before we get dream recorders?

  23. Mark Dow says:

    Retinal information projects onto visual cortex, from which this information is inferred (using fMRI).

    Dreams likely influence primary visual cortex, but are not similar to retinal projections. They have components from across many parts of the brain, and aren’t played like a movie onto visual cortex.

    A better way to record dreams would be to induce people to speak while dreaming.

    • hypnosifl says:

      Well, all experiences have “components from across many parts of the brain”, and dreams have plenty of subjective elements that aren’t visual–but if we’re interested in the purely visual aspects of dreams, is the brain activity in the visual cortex noticeably different from that of a waking experience?

      • Mark Dow says:

        Yes. Although they can’t be compared directly, all indications are that visualization results in very different patterns of activity in the visual cortex compared to visual stimuli.

        • hypnosifl says:

          But when you say “visualization”, are you talking about waking visualization? When I try to visualize something in my imagination it’s not remotely as vivid and detailed as a real object in my visual perception, whereas dream objects seem to come pretty close (I sometimes try to investigate this in lucid dreams, the level of detail in things like tree branches really is pretty amazing). Are there studies that show specifically that activity in the visual cortex during dreams is very different from activity during waking perception?

          • Mark Dow says:

            I don’t know of any neuroimaging with respect to dream imagery. Of course the difficulties of gathering data about brain function and the details of the percieved imagery are problematic, and will be problematic in any attempt to correlate brain activity with perception during dreaming. So it is not possible to entirely discount or confirm the possibility that activity during dream imagery is not strongly correlated with visual cortex activity. Expert lucid dreamers are hard to come by, and harder to certify.

            Wikipedia’s “Cognitive neuroscience of dreams” entry has a good overview of what (little) is known about brain activity related to perception during dreaming:
            http://en.wikipedia.org/wiki/Cognitive_neuroscience_of_dreams#Perceptual_processing

            But many waking tasks involve visual imagery (e.g. mental rotation). There is some evidence that the vividness of imagery is different for different people for different tasks, and some evidence that “vividness” correlates with activation levels in visual cortex (but not necessarily with the same spatial or temporal organization as a similar visual stimuli).

            Here’s a relavant reference, “Cortical activation evoked by visual mental imagery as measured by fMRI””
            http://www.kyb.mpg.de/fileadmin/user_upload/files/publications/pdf2443.pdf

  24. Nick Gold says:

    Until the End of the World is one of the absolute best freaking movies, import it now, and try to get the long version.  I bought the long version and haven’t watched it yet, I’m waiting for the absolute right 4 hour stretch!  :)  It’s just so good, and yes, oddly synchronistic in terms of the falling satellite.  Well good, dammit, if life is becoming more like Until the End of the World, I’m happy!  :)

  25. subhan says:

    While cool, this is not really that amazing of a thing.  This is not reading memory of any sort. It is reading neural activity in the visual cortex, *while* an image is being viewed.  If you know much about the visual cortex, you may recall that there is a very well-defined mapping between the retina & the primary visual cortex.  I’d be amazed if this mapping is not what is driving the process in question.  

  26. The averaging of the 100 ‘best fit’ youtube clips to produce the result looks exactly like a moving version of the average Playboy centerfold

    http://salavon.com/work/EveryPlayboyCenterfoldDecades/

    or the average Vogue cover

    http://gizmodo.com/5715176/meet-vogue-magazines-cover-girl-of-2010

  27. Neat. Things I can do with this:

    1. Create pictures and diagrams for presentations.

    2. Create my own movie (starring me, of course).

    3. Show people what I’m talking about in video-chat (yes, live, real-time updates).

  28. autark says:

    So what if they’re only matching fMRI scans to YouTube frames currently? They’re building a library of what scans “look” like.  Once they have a large enough sample database, they can begin synthesizing the playback images to create an original image.

    Think about audio resolution as an analogy, you probably could not discern a synthesized piano, guitar or drum sound from a well recorded multi-sampled version.  There may be a few indicators at the edges (such as synthesizing some guitar techniques like palm muting or different kinds of sliding), but it’s convincing enough most non-musicians won’t notice.

Leave a Reply