Illusory sound texture reveals multi-second statistical completion
in auditory scene analysis

Richard McWalter & Josh McDermott
Nature Communications, vol.10 pp. 5096, Nov 2019

Introduction

Here you will find some sound examples demonstrating the phenomenon of "illusory sound texture" (characterized more fully in the paper). Illusory sound texture occurs when a loud foreground sound interrupts a quite background sound texture but the sound texture is perceived to continue even when it is not physicially present in the stimuli during the periods of the foreground sound. For instance, when a foreground speech signal interrupts a background applause sound texture, the applause sound texture is perceived or heard to continue. Listen to the demo below to hear it for yourself!

Demo 1: Illusory sound texture

In the first demo, we show that a sound texture is heard to continue when interrupted by noise. The stimuli consist of 2 seconds of sound texture, followed by 2 seconds of white noise, and then another 1 second of the texture. Note that the sound texture is not physically present during the intermediate noise segment (from 2-4 seconds), and the noise segment is sufficiently intense to mask the texture were they presented simultaneously. The three sound textures (Applause, Pheumatic drill and Rattlesnake) are synthetic (see McDermott and Simoncelli [2011]).

Applause texture Pneumatic drill texture Rattlesnake texture


Demo 2: Multi-second illusory continuity is unique to textures

This demo shows that many types of sounds (e.g. textures, speech, environmental sounds etc...) can be heard to continue over short masker durations, but only textures elicit extended, multi-second illusory continuity. In the following sound examples, we vary the duration of the interrupting noise from short (125ms) to long (2s). The first 2 seconds is the "inducer" sound, while the last 4 seconds interleaves 2 seconds of noise and 2 seconds of the inducer. For the short masker durations the noise occurs as a series of pulses interspersed with the inducer. For the longest masker duration there is a single segment of noise followed by the inducer. The inducers are real-world sound recordings.

These examples are drawn from the four sound categories used in an experiment described in our paper. Listeners typically hear only the texture to continue during the longer noise bursts. One might imagine that sound textures are inferred over long temporal extents due to their predictable nature (e.g. the fact that their statistical properties are stable over time). However, periodic sounds that are also quite predictable are not heard to continue over the longer durations. This suggests that the extent of illusory continuity might be related to the likelihood that a sound continues for long durations in the world, rather than merely whether the sound characteristics are predictable.

Noise condition Texture:
Applause
Speech/Music:
Male speech
Speech/Music:
Bluegrass music
Environmental Sound:
Church bells
Periodic/Rhythmic:
Sawing wood


Demo 3: Illusory texture is eliminated by gaps

In this demo, we show that illusory texture occurs only when the texture could have continued during the interrupting sound. If a gap occurs between the texture and the interrupting noise the illusory phenomenon is eliminated. The following examples are drawn from four conditions with silent gaps in different positions. The textures are synthetic, the noise is Gaussian (white) and the silent gaps are 200ms in duration. The level difference (signal-to-noise ratio) between the texture and noise was set so that the noise would mask the texture, were they presented simultaneously.

Inducer Texture-Noise Contiguous
Silent Gap Before/After Noise
Silent Gap After Noise
Silent Gap Before Noise
Applause
Pneumatic drill
Rattlesnake


Demo 4: Illusory texture occurs when a texture could be masked

In this demo, we show that illusory texture only occurs when the interrupting noise is sufficiently intense to mask the texture, were they presented simultaneously. The Noise louder than inducer stimuli consist of interrupting noise that is +18dB relative to the texture. The Inducer louder than noise stimuli consist of interrupting noise that is -6dB relative to the texture.

Inducer Noise louder than inducer
(illusory texture)
Inducer louder than noise
(no illusory texture)
Applause
Pneumatic drill
Rattlesnake


Demo 5: Illusory texture occurs for a diverse set of maskers

In this last demo, we show that illusory texture can occur for masker sounds other than white noise. In the following demo, we use a "mean" noise masker, synthesized from the statistics averaged over a large set of natural sounds (e.g. textures, speech/music, environmental sounds). Illusory continuity also occurs for the "mean" masker, demonstrating that it does not depend strongly on the masker being stationary, or on the similarity of the inducer to the masker.

Applause texture Pneumatic drill texture Rattlesnake texture


About the paper

Here is a link to the paper.

Email me with questions or requests for code.