McGurk effect

Key takeaways

McGurk effect belongs to science; place it in that map before memorising details.
Learn the definition first, then one example that makes the definition concrete.
Connect McGurk effect to a quantity you can measure, compute or draw — that is where exam questions come from.
Reproduce the core statement of McGurk effect from memory before moving on to harder problems.

Reference excerpt

The McGurk effect is a perceptual phenomenon that demonstrates an interaction between hearing and vision in speech perception. The illusion occurs when the auditory component of one sound is paired with the visual component of another sound, leading to the perception of a third sound. The visual information a person gets from seeing a person speak changes the way they hear the sound. If a person is getting poor-quality auditory information but good-quality visual information, they may be more likely to experience the McGurk effect. Integration abilities for audio and visual information may also influence whether a person will experience the effect. People who are better at sensory integration have been shown to be more susceptible to the effect. Many people are affected differently by the McGurk effect based on many factors, including brain damage and other disorders.

Background It was first described in 1976 in a paper by Harry McGurk and John MacDonald, titled "Hearing Lips and Seeing Voices" in Nature (23 December 1976). This effect was discovered by accident when McGurk and his research assistant, MacDonald, asked a technician to dub a video with a different phoneme from the one spoken while conducting a study on how infants perceive language at different developmental stages. When the video was played back, both researchers heard a third phoneme rather than the one spoken or mouthed in the video. This effect may be experienced when a video of one phoneme's production is dubbed with a sound-recording of a different phoneme being spoken. Often, the perceived phoneme is a third, intermediate phoneme. As an example, the syllables /ba-ba/ are spoken over the lip movements of /ga-ga/, and the perception is of /da-da/. McGurk and MacDonald originally believed that this resulted from the common phonetic and visual properties of /b/ and /g/. Two types of illusion in response to incongruent audiovisual stimuli have been observed: fusions ('ba' auditory and 'ga' visual produce 'da') and combinations ('ga' auditory and 'ba' visual produce 'bga'). This is the brain's effort to provide the consciousness with its best guess about the incoming information. The information coming from the eyes and ears is contradictory, and in this instance, the eyes (visual information) have had a greater effect on the brain, and thus the fusion and combination responses have been created. Vision is the primary sense for humans, but speech perception is multimodal, which means that it involves information from more than one sensory modality, in particular, audition and vision. The McGurk effect arises during phonetic processing because the integration of audio and visual information happens early in speech perception. The McGurk effect is very robust; that is, knowledge about it seems to have little effect on one's perception of it. This is different from certain optical illusions, which break down once one "sees through" them. Some people, including those that have been researching the phenomenon for more than twenty years, experience the effect even when they are aware that it is taking place. With the exception of people who can identify most of what is being said from lip reading alone, most people are quite limited in their ability to identify speech from visual-only signals. A more extensive phenomenon is the ability of visual speech to increase the intelligibility of heard speech in a noisy environment. Visible speech can also alter the perception of perfectly audible speech sounds when the visual speech stimuli are mismatched with the auditory speech. Normally, speech perception is thought to be an auditory process; however, our use of information is immediate, automatic, and, to a large degree, unconscious and therefore, despite what is widely accepted as true, speech is not only something we hear. Speech is perceived by all of the senses working together (seeing, touching, and listening to a face move). The brain is often unaware of the separate sensory contributions of what it perceives. Therefore, when it comes to recognizing speech the brain cannot differentiate whether it is seeing or hearing the incoming information. It has also been examined in relation to witness testimony. Wareham and Wright's 2005 study showed that inconsistent visual information can change the perception of spoken utterances, suggesting that the McGurk effect may have many influences in everyday perception. Not limited to syllables, the effect can occur in whole words and have an effect on daily interactions that people are unaware of. Research into this area can provide information on not only theoretical questions, but also it can provide therapeutic and diagnostic relevance for those with disorders relating to audio and visual integration of speech cues.

Factors

Internal

Brain damage Both hemispheres of the brain make a contribution to the McGurk effect. They work together to integrate speech information that is received through the auditory and visual senses. A McGurk response is more likely to occur in right-handed individuals for whom the face has privileged access to the right hemisphere and words to the left hemisphere. In people that have had callosotomies done, the McGurk effect is still present but significantly slower. In people with lesions to the left hemisphere of the brain, visual features often play a critical role in speech and language therapy. People with lesions in the left hemisphere of the brain show a greater McGurk effect than normal controls. Visual information strongly influences speech perception in these people. There is a lack of susceptibility to the McGurk illusion if left hemisphere damage resulted in a deficit to visual segmental speech perception. In people with right hemisphere damage, impairment on both visual-only and audio-visual integration tasks is exhibited, although they are still able to integrate the information to produce a McGurk effect. Integration only appears if visual stimuli is used to improve performance when the auditory signal is impoverished but audible. Therefore, there is a McGurk effect exhibited in people with damage to the right hemisphere of the brain but the effect is not as strong as a normal group.

… excerpt ends here. Continue reading the full article.

Worked examples

Example 1 — a first encounter with McGurk effect

Start with the simplest possible case. Write down what McGurk effect claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In science, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to McGurk effect before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about McGurk effect ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of McGurk effect

In research: McGurk effect appears in science research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry: Engineering practice reuses McGurk effect in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom: McGurk effect is common in secondary-school and first-year university syllabi. It links to neighbouring topics Auditory illusions, Perception, Speech processing, so understanding it makes those chapters shorter.
In everyday life: Look for McGurk effect outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

How to study McGurk effect in 20 minutes

Read the reference excerpt below once, without taking notes.
Close the page and write down what McGurk effect means in your own words.
Compare your version with the excerpt and mark what you missed.
Work through the three examples above with pen and paper.
Explain McGurk effect out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is McGurk effect in simple terms?

The McGurk effect is a perceptual phenomenon that demonstrates an interaction between hearing and vision in speech perception. The illusion occurs when the auditory component of one sound is paired with the visual component of another sound, leading to the perception of a third sound.

Why does McGurk effect matter?

Because it connects several science ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study McGurk effect?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on McGurk effect.