Seeing Videos Improved Sound Detection in Noise

TL;DR: A 2026 study in The Journal of the Acoustical Society of America found that seeing a video tied to the sound source, such as motion from a train, drum, keyboard, bird, or canoe paddle, made faint sounds easier to detect through noise. Closing the eyes raised sound-detection thresholds by 1.32 dB, while videos tied to those sounds lowered thresholds by 2.98 dB.

Source: The Journal of the Acoustical Society of America (2026) | Ni et al.

Hearing in noise is usually framed as an ear problem: the sound is too faint, the background is too loud, or attention is too scattered.

Ni and other researchers tested whether vision changes that task by giving the listener a picture or video connected to the sound source.

Here, “matched” means the visual cue belonged to the same event as the target sound.

A drum sound came with drum-related visual information; a train sound came with train-related visual information; a keyboard sound came with keyboard-related visual information.

The surprising part is the direction of the result. People often close their eyes when trying to hear something faint.

In this experiment, that habit did not improve noisy detection. It made participants need a slightly stronger target sound before they could detect it.

25 Listeners Heard 5 Sounds Through 70 dB Noise

The behavioral task was simple enough to understand but demanding enough to test a real listening problem.

Participants heard target sounds while a 70 dB(A) pink-noise mask played in the background, and the targets were recognizable everyday sounds rather than abstract beeps: a canoe paddle, a drum, a lark chirping, a train, and a keyboard.

For each sound, the researchers measured the detection threshold: how loud the target needed to be before the participant could detect it through the noise.

Lower thresholds mean better sensitivity.

Higher thresholds mean the listener needed more sound energy to notice the target.

The study used the blank-board condition as the baseline.

That baseline is important because it separates “eyes open” from “helpful vision.” A blank board gives visual input in the most minimal sense, but it does not provide a picture or motion cue related to the sound.

4 Visual Conditions Separated Eye Closure From helpful Vision

The researchers did not just compare eyes open with eyes closed. They used four visual conditions that moved from no visual input to a moving scene matched to the sound:

1. Eyes closed: participants listened without visual input.
2. Blank board: participants kept their eyes open, but the visual scene gave no information about the sound.
3. Static matching image: participants saw a still picture related to the target sound, such as an image connected to the object or event producing it.
4. Dynamic matching video: participants watched moving visual content that corresponded to the sound, giving the brain timing and source cues instead of a frozen hint.

Dynamic video means the visual cue unfolded over time.

A still image can tell the listener what kind of source to expect; a video can also suggest when the relevant sound event is about to happen.

That design is stronger than a simple eyes-open versus eyes-closed comparison.

It asked whether the auditory system benefited from any visual state or from informative visual content, and the matched dynamic video produced the largest benefit.

Closing the Eyes Raised Sound-Detection Thresholds by 1.32 dB

Against the blank-board baseline, closing the eyes elevated sound-detection thresholds by 1.32 dB on average.

In practical terms, the target sound had to be made louder before participants could detect it through the pink noise.

A 1.32 dB shift is not the difference between normal hearing and deafness; it is a controlled laboratory shift in sensitivity.

The direction is still helpful because it runs against the everyday assumption that shutting out vision automatically frees the auditory system.

The finding leaves room for situations where closing the eyes helps someone concentrate.

The narrower point is specific: when the task is to detect target sounds in a complex noisy soundscape, removing visual input can remove information the brain could have used.

That distinction helps explain why personal experience can vary.

If you are trying to focus on a quiet voice in a room with no relevant visual cue, closing your eyes may be calming.

If you are trying to detect an event that has a matching visual source, such as a train, a drum hit, or a keyboard action, vision can help the brain narrow the search.

Dynamic Video Lowered Thresholds More Than Static Images

The strongest behavioral effect came from matching video. Compared with the blank-board baseline, videos corresponding to the target sound lowered sound-detection thresholds by 2.98 dB on average.

Still pictures related to the target sound also helped, lowering thresholds by 1.60 dB.

The step from static image to video is the informative comparison.

Both are visually relevant, but the video can show change over time.

Timing is especially important for hearing because sound unfolds moment by moment, so a moving picture can prepare the listener for when an auditory event is likely to arrive.

For a drum, the visual cue might imply an impending strike.

For a train, motion can help define the object producing the noise.

For a keyboard, visible movement can make a faint sound less ambiguous.

The paper does not prove each mechanism separately, but the pattern fits a basic idea: matched motion supplies timing cues and source cues before the sound is fully clear.

Hearing in noise improved when the visual scene supplied relevant timing or source information.

EEG Linked Eye Closure to a 22.3% to 45.2% Criticality Shift

The study also recorded electroencephalography (EEG) in 27 participants and analyzed a measure called the avalanche critical index.

Researchers reported that eye closure reduced this index by 22.3% to 45.2% across the 5 auditory stimuli compared with blank visual stimulation.

“Criticality” can sound abstract, but the basic idea is not mystical.

Researchers use it to describe a regime where neural activity is balanced between too much order and too much instability.

A brain state near that boundary may be flexible enough to respond to weak or uncertain inputs without turning every fluctuation into noise.

Here, the EEG result gives the behavioral finding a brain-dynamics angle.

Eye closure was not only a change in attention or comfort; it was associated with a measurable shift in cortical dynamics during the auditory task.

The study does not establish that the scalp-recorded brain readout alone caused the threshold differences.

The behavioral and EEG results support the same direction: visual engagement changed how listeners performed in noisy detection, and the brain-state measure moved with the visual condition.

Sound-Matched Visual Cues Are Different From General Distraction

The result should not be read as “more stimulation is always better.” The study tested relevant visual stimulation: images and videos connected to the target sounds.

The next experiment is straightforward. If someone hears a drum but sees a bird, does the visual input still help?

That test tests whether the benefit depends on the auditory and visual streams belonging to the same event. The AIP release notes that the researchers want to test incongruent pairings like that.

This is the difference between visual engagement and random visual clutter.

A noisy bar, a busy screen, or an irrelevant video could easily interfere with listening.

A matched face, instrument, machine, or movement pattern can help because it tells the brain what kind of sound to expect.

• Matching visual information: pictures or videos tied to the same object or event as the sound, likely to help when they carry source or timing cues.
• Blank visual input: helpful as a baseline, but not the same as a meaningful scene.
• Irrelevant visual input: still needs direct testing in this paradigm.

In noisy environments, relevant visual information can improve detection of faint target sounds, but the study does not overturn every case where people close their eyes to concentrate.

The paper itself frames the issue as a difference between noisy target detection and quieter auditory segregation.

In a quiet setting, visual disengagement might still help someone focus on an auditory stream.

In a noisy scene, matched visual cues can guide the detection process.

That is a more realistic model of everyday hearing. People usually do not listen as disembodied ears.

They watch mouths, hands, instruments, vehicles, tools, and movement. The brain can combine those streams when the scene gives it enough structure.

For hearing research, the study is a reminder to treat visual context as part of the experiment, not background decoration.

For everyday life, it suggests a simple rule: when the visual source is relevant, watching can help more than shutting your eyes.