[Press Release] Evaluating Music Beyond Sound: Understanding Visual Influence Across Genres
2025.07.11
Musical training shapes the sight-over-sound effect; auditory expertise reduces visual bias, highlighting the need to balance sensory cues in evaluation.
Although music is considered a purely auditory experience, visual cues can unconsciously influence performance judgements--a phenomenon known as the sight-over-sound effect. To explore how one's musical expertise can affect this bias, researchers from Japan conducted a controlled study using recordings from Japanese high school brass band competitions. Their results reveal that the effect predominantly appears in musicians without genre-specific experience, highlighting how specialized training can shape multisensory evaluation in music perception and judgement.
When we listen to a symphony, a jazz performance, or a pop song, we naturally tend to assume that our ears are the primary judges of quality. For many people, music is primarily an auditory experience, and they instinctively believe that the true essence of a musical performance lies in factors such as its acoustic fidelity, proper intonation, or rhythmic precision. Research has, however, revealed an intriguing phenomenon that challenges this assumption. Dubbed as the "sight-over-sound effect," this phenomenon suggests that visual aspects of a performer--from their stage presence to subtle body language cues--can unconsciously sway our judgment, sometimes even more powerfully than the music itself.
Despite its fascinating implications, the replicability of the sight-over-sound effect has remained debatable across musical styles and performance settings. Previous research has struggled with accurately assessing visual influences, experiencing challenges with inconsistent filming angles, musical pieces, and definitions of musical experience among evaluators. Consequently, the understanding of the sight-over-sound effect and its extent remains unclear.
Against this backdrop, a research team led by Associate Professor Shinya Fujii, Director of the Neurosciences and Music Laboratory at Keio University and the Research Center for Music Science at Keio University Global Research Institute, Japan, conducted a comprehensive study to investigate the replicability of this effect. The study was co-authored by Mr. Tomohiro Samma from the Graduate School of Media and Governance at Keio University and published online in the PLOS One journal on April 29, 2025. The team meticulously investigated the replicability of the sight-over-sound effect, with a special focus on the role of the evaluators' specific musical background.
To address the limitations of prior research, the researchers designed an experiment using recordings from Japanese high school brass band competitions. This allowed for significant control over the experimental stimuli. All selected bands had received gold awards, minimizing differences in skill level. Most importantly, all performances within a comparison set featured the same musical piece filmed with consistent camera angles. 301 adults were included in the study, who were categorized into three groups: brass band musicians (BMs) with direct experience in the genre, non-brass band musicians (NBMs) with general musical experience but no brass band background, and non-musicians (NMs) with no formal musical training. These groups evaluated brass band performances under audio-only, visual-only, or audio-visual conditions, selecting the band they believed would achieve the best competition results.
The study's results revealed a markedly nuanced picture of the sight-over-sound effect. When analyzing the overall participant sample, the researchers found no significant evidence of the effect, suggesting that a careful control over visual elements and musical pieces could indeed reduce visual dominance. However, detailed subgroup analyses suggested a dependency on the evaluator's musical experience. Specifically, the sight-over-sound effect was observed in NBMs, who showed significantly higher accuracy in identifying winners from visual-only information. In contrast, the effect was absent in BMs, who demonstrated superior accuracy in the audio-only condition, indicating their ability to make precise judgments based solely on sound. The NM group did not exhibit the sight-over-sound effect either.
These findings carry significant implications for our understanding of multisensory integration and the practical aspects of music and music competitions. Firstly, the absence of the sight-over-sound effect in the overall sample highlights the importance of methodological rigor in future studies. More importantly, varying results across the three groups underscore that specialized musical training can profoundly shape how individuals process and prioritize sensory information. "Our study found that the sight-over-sound effect was observed only in NBMs, indicating its dependence on the evaluator's specific musical experience, while BMs' auditory training potentially mitigating visual influence," explains Mr. Samma.
Additionally, the results are also relevant in social psychology and cognitive science in general, since they suggest that expertise in a given domain can alter one's weighting of sensory inputs. For instance, visual dominance observed in the NBM group suggests that while they possess musical skills, they may still rely on visual cues for evaluation if they lack experience in a given genre. "By shedding light on the underexplored aspect of how evaluators' musical experience influences the evaluation process in multisensory integration, this work has significant implications for various real-world aspects, such as music education, performance, and competition judging," concludes Dr. Fujii.
Further work on this intriguing topic will hopefully help us improve music education and make competitions fairer.
Image
Title: Visual perception and musical experience are important when assessing a band's performance
Caption: In musical evaluations, the "sight-over-sound" effect--where visual information overrides auditory input--is frequently observed, calling into question the assumption that sound is the dominant factor in these judgments. Yet, it was unclear whether this was the case across various musical genres and settings. Now, a recent study focusing on Japanese brass band competitions reveals that individuals lacking specific experience in the genre are more influenced by visual cues than those with relevant expertise. These results highlight the need to balance visual and auditory elements in music education and training to help them process and prioritize sensory information.
Credit: Dr. Shinya Fujii from Keio University, Japan
License type: Original Content
Usage restrictions: Cannot be reused without permission
Details of Journal Article
Title : Sight-over-sound effect depends on interaction between evaluators' musical experience and auditory-visual integration: An examination using Japanese brass band competition recordings
Authors : Tomohiro Samma, Kazuaki Honda, Shinya Fujii
Journal : PLOS One
DOI : 10.1371/journal.pone.0321442
About Associate Professor Shinya Fujii from Keio University
Dr. Shinya Fujii obtained MS and Ph.D. degrees in Human and Environmental Studies from Kyoto University in 2007 and 2010, respectively. He currently serves as an Associate Professor at the Faculty of Environment and Information Studies, Keio University. He is also the Director of the Neurosciences and Music Laboratory at Keio University and the Research Center for Music Science at Keio University Global Research Institute. Dr. Fujii specializes in perception neuroscience and its relationship with music. He has over 75 papers credited to his name.
https://neuromusic.sfc.keio.ac.jp/
https://www.kgri.keio.ac.jp/en/project/research-centers/2025/A25-27.html/
https://www.k-ris.keio.ac.jp/html/100012635_en.html/
https://researchmap.jp/7000025148?lang=en
About Tomohiro Samma from Keio University
Tomohiro Samma is a Ph.D. student at the Graduate School of Media and Governance, Keio University. He specializes in musical neuroscience and has three papers credited to his name. He is a member of the Society for Music Perception and Cognition and the Society for the Neural Control of Movement.
https://researchmap.jp/tsamma?lang=en
About Kazuaki Honda from Communication Science Laboratories, NTT, Inc.
Dr. Kazuaki Honda obtained a Ph.D. degree from the Graduate School of Media and Governance, Keio University, in 2024. He is a research scientist at NTT Communication Science Labs. Dr. Honda specializes in perceptual and cognitive neuroscience and its relationship with music.
https://www.rd.ntt/e/cs/team_project/human/embodied_intelligence/
Funding information
This work was supported by Taikichiro Mori Memorial Research Grants, the Japan Society for the Promotion of Science Grant-in-aid (16K16483 and 20H04092), Japan Science and Technology Corporation (JST COI-NEXT Grant: JPMJPF2203), Keio University Academic Development Funds, and Toyama Memorial Music Research Foundation.
[Enquiry]
Contact: Tomohiro Samma
E-mail: tomo.windbrass@keio.jp
