Spatial Audio Virtualization and Gamification for Hearing Assessment and Enhancement

Auditory perception can be both challenging and rewarding. In a noisy crowded restaurant, listeners must quickly recalibrate their perception to the acoustic characteristics of the room and constantly switch attention to new speakers throughout a conversation. At the same time, the listening experience is motivating, e.g., when participating in an interesting conversation or enjoying the sounds of the surrounding environment. A major challenge in auditory neuroscience is that the laboratory training procedures and testing tools used to assess hearing abilities are neither rewarding nor representative of the multifaceted experiences of real-world auditory scenes. The goal of the current grant is to use virtual reality and computer game technologies to create advanced tools and models for accurately assessing auditory abilities and their underlying neural mechanisms, and for enhancing them in people with normal and impaired hearing. First, a series of experimental studies will be performed to determine 1) how to achieve a strong percept of externalization in virtual environments, 2) how virtualization can improve our understanding of the cognitive factors that influence performance on auditory perceptual tasks, and 3) how gamified training in virtual environments can enhance spatial and speech perception. Then, models will be developed to describe how gamification and virtualization influence auditory perception in the examined tasks. The results will be used to develop new modules for existing auditory assessment and enhancement tools - PART and Listen - which were developed by members of the consortium. Finally, workshops will be organized to train new researchers in the use of virtualization and gamification in auditory neuroscience, and to share and disseminate the new results of the project.