Mixed Reality Environment for Immersive Experience of Art and Culture

Engagement with the art and music can have a range of positive effects on physical and mental health. However, access to the many benefits of musical experience can be limited due to various reasons: (1) the geographic location of the venue, (2) personal limitations in access, (3) limitation on the schedule of the concerts and performances, (4) cost of the experience. Due to these limitations the effort and cost of getting access to the concerts and performances override additional value the participant would get from the experiences. In XTREME project, we remove these limitations by developing a human-centered mixed reality (MR) solution that, instead of requiring the participant to be physically present in the venue, brings the concerts and performances to a remote location via MR technology while retaining the immersive experience, and even giving additional dimensions to it by integrating real and virtual contents. Our solution also allows for virtually shared, realistic experience together with selected social circles of the user. The solution removes geographic limitations as by the MR solution the concerts and performances can be accessed virtually from anywhere and they hence are more inclusive for people with limited access. Likewise, providing a global access to the art and music provides a wider audience for musicians and artists, not limiting the number of seats by the size of the physical venue. As this solution removes the necessity to travel to concerts and performances, it also provides a greener alternative. In contrast to the current state-of-the-art, our solution targets at making an MR solution to the concerts and performances where those are in in 3D, both for visuals and sound, blended into the room of the users. We will also provide maximised experience of presence via reconstruction of the users and their behaviour in the shared access, and enhance the immersion by providing an additional visual layer to the sound.