The McGurk Effect

Discuss

37 Responses to “The McGurk Effect”

  1. Anonymous says:

    Weeelll, the effect tends to work less well with subjects from countries in which foreign TV programs are dubbed, because people in these countries are accustomed to hearing something that does not correspond to what they see.

  2. mermaid says:

    Bet if you tried this with a Tagalog speaker it wouldn’t work.

  3. Anonymous says:

    Olive Juice lipreads as “I love you”

    • Donald Petersen says:

      Olive Juice lipreads as “I love you”

      As does “elephant shoes.”

      I deal with ADR and sound sync as part of my work in TV post production, and it seems to me, too, more like the “fah” picture is slightly out of sync with the audio, as opposed to the B sounding like an F.

  4. OldRipbeak says:

    That’s bucking cool!

  5. HarlanH says:

    That’s not the McGurk effect. The McGurk effect is where you perceive a sound that’s neither what you see or what you hear. The standard example is when you hear “ba” and see “ga”, you often hear “da”.

    • Anonymous says:

      Both of those count as the McGurk Effect. Consonants are composed of three phonological dimensions: voicing, manner, and place of articulation. When you’ve got both auditory and visual inputs, especially when they conflict, the auditory signal contributes the voicing and manner information, and the visual signal contributes the place because that’s much harder to tell from sound alone. The McGurk Effect is just the combining of these two signals, which sometimes results in a percept that’s different from both the auditory and visual signal, and sometimes that’s the same as the visual.
      For example,
      aud /ba/ + vis /ga/ = perceived /da/
      and
      aud /ba/ + vis /da/ also = perceived /da/

      -A cognitive scientist

    • Xopher says:

      I’d be reluctant to argue with the UCR guy, myself, but the effect you describe makes sense too. /b/ is at the front of the mouth and /g/ at the back, so splitting the difference (/d/ is the closest stop to the middle of the mouth) is a fairly sensible thing for the brain to do.

      I have to watch with captions on, and this is related to the reason why.

  6. Sparrow says:

    I still hear bah, but I get a distinct impression that the ADR is off. I really notice this effect in movies when the overdub doesn’t match what was originally said or when the sound isn’t quite in sync with the mouth movements.

  7. Anonymous says:

    I need to figure out how to write to this man. My daughter and I, both neurologically typical, were astonished by this. But my high-functioning autistic son always heard “bah bah bah” no matter which face he saw. A possible test for autistic spectrum disorders? Or just something weird about my son, in particular?

    • SamSam says:

      My reaction to the video being an empathetic sense of “feeling” the sound “eff” would make me say, as a non expert (but a BA in cognitive science, for whatever that’s worth) that it’s completely plausible that there would be a significant difference in people with autistic spectrum disorder. It’s worth researching more, and I bet this person would be interested in researching it if there isn’t anything already.

  8. Anonymous says:

    Bah, humbug…

  9. Bucket says:

    Hm.

    I still hear ‘bah’ and get the feeling like the video is out of sync with the audio.

    I also have a really hard time conversing in loud environments, even though a recent hearing test showed that I have pretty good hearing.

    I wonder if the two are related.

  10. Anonymous says:

    Doesn’t seem to work for me. I hear the bah both times. Just doesn’t look like it is him saying it the second time.

  11. Anonymous says:

    Is this guy David Cross’s more erudite clone?

  12. andmit says:

    What does it mean if all you hear is “ba” the whole time? I didn’t get what was going on when my boyfriend first showed me this because I don’t hear “fa” at all. Even when he made me stare at his mouth. All I hear is “ba”. Someone said it means my brain isn’t working right.. O_o

  13. mellowknees says:

    I was really hoping that this was a post about the coach from Home Movies.

    I am disappoint.

  14. marc anthony says:

    At 56 seconds, when the guy first starts changing the movements of his mouth, I detect a slight “f” sound, even when completely looking away from the video. I don’t think you can change the shape of your mouth without slightly reshaping the sound. The effect is increased with the visual, but I detect the change without it.

    • Anonymous says:

      @marc anthony
      I agree it does seem to change… I can hear a “popping” noise when he does the Bah, and it seems the audio has been edited in the second loop to mute the “pop” of the b. Seems one time he actually does say “Fah” after all. I think the guys in the studio maybe took a liberty to increase the effectiveness of this video? But the point it well made.

      Which, like HarlanH says, is a bit misconstrued, because if you wiki McGurk Effect it relates that two nodes of communication can be misconstrued to create a third unrelated effect; which is much more interesting if you think about it, than just a confused “flip flopping” (or blip blopping?) between the two F and B nodes.

    • Anonymous says:

      You do understand that both videos use the “bah” audio? Interesting that you hear a change that isn’t there. Or if you look away when it shows him saying “bah”, do you hear “fah” there too?

  15. Robert says:

    Betrayed! I closed my eyes and heard B both times. Then I opened them and heard F the second time. Betrayed, I tell you!

  16. SamSam says:

    That was great. When I was watching the supposed “fah fah fahs” after having been told what was happening, I was acutely aware of my own mind supplying the missing Fs. It was less an auditory sensation as a physical one — by seeing his mouth make an F sound, I was making an F sound myself. I was almost feeling what I would be feeling if I were the one making the Fs. I had almost the same sensation watching the video with the volume off, but not as strongly.

  17. recoiled says:

    I always thought this was called the Two Party Effect

  18. RHK says:

    So that’s how politicians do it!

  19. yragentman says:

    “… a co-hear-rent view of the world…” ugh

  20. JohnBerry says:

    Opening and closing my eyes really drove home the point – you can’t trust your eyes (or ears, or (sometimes) brains). Amazing!

  21. shava says:

    I have crosstalk deafness — a cognitive deafness that means I can’t parse one strand of speech in the presence of conflicting signals, like a person who can’t make out the separate instruments playing in a piece of music. As a result, in a place where many people are talking or there’s non-white background noise — a subway car, a networking meeting, the lobby at a theater — I have to lip read.

    Whether it’s strong conditioning to lip reading, or the cognitive bits, or both, this illusion doesn’t work on me at all. Interesting.

  22. rebdav says:

    Woah!!
    Its like my visual center snuck some shoes, a nail file, and a bottle of milk onto my auditory center airplane and demanded that we fly to Cuba!!

  23. wylkyn says:

    How weird. I didn’t experience the illusion. What I got was a feeling of dissonance. I was hearing the “b” sound and seeing the “f” sound, and my brain was setting off “this is wrong” alarms, even though I was trying to go with it. I wish that I had seen this before it was explained so my rational brain would have been taken off-guard. Just like when the hypnotist couldn’t put me under. Stupid rational brain always spoils my fun. :(

  24. Crispinus211 says:

    A similar effect happens with the subtitled STSanders “shreds” videos — I know Mick Jagger is singing (for example) “You make the grown me cry,” but the dubbing and the subtitles trick me into something else.

  25. Anonymous says:

    @mermaid
    I had a college professor who studied anthropology in Micronesia. His name was Bill but the local people there didn’t have a “b” or short “i” sound, and never ended words with a consonant. So “Bill” became “Peelly”. Even when he said his name, they would “hear” their version in their mind.

  26. Bill Barol says:

    Thanx for the nightmares, Mark!

  27. dross1260 says:

    I prefer Señor Wences

  28. Anonymous says:

    Anyone also hear two people saying something when they do the split screen? I thought at first they had laid down two audio tracks over each other.

  29. pepik says:

    This strikes me as the result of a kind of error correction technique. Very often, it can be hard to discern the initial sound of a word or syllable starting with the f sound. Seeing someone’s mouth make the sound helps fill in the gap.

    I have had the distinct experience of hearing sounds that don’t quite make sense, and having my brain go back and ‘fix’ the sounds into a word that makes sense in context. It’s something very cool that our brains can do, I think.

Leave a Reply