Research Project
Sandscape -->Soundscape
An Embodied Approach to co-creating Spatial AR sounds
Keywords
Audio-based Augmented Relaity, Creativity Support, Embodied Cognition, Tangible User Interfaces, Co-creativity
Publication
Sandscape -> Soundscape: An embodied approach to co-creating spatial AR soundscapes Under review
Team
Yiwei Huang, Yanwen Dong, Zhenfang Chen, Daragh Byrn, Dina El-Zanfaly
Project description
Spatialized Audio-based Augmented Reality (AAR) has been widely explored in the past few years to create immersive experiences. For example, AAR has been used to enrich audio walks in cultural heritage settings. Yet, prototyping spatial AAR remains challenging; it is immaterial, intangible, and needs to be closely coordinated with site-specific, local conditions. We investigate how an embodied interface can overcome these limitations and support rapid sketching and co-creation of spatial AAR. We also present a system consisting of: UWB indoor positioning; an augmented headset; and an embodied interface that tracks hand gestures and object tokens on a sand table. Placing a marker adds an audio loop to the environment, while sketching in the sand generates sounds in the environment moving along a corresponding path. Findings of a qualitative study (n=18) indicate that embodying spatial sounds aids perception of sound location and collaboration in prototyping spatialized sound.
A sand table is used to place sound markers and sketch sound paths. An augmented headset tracks head orientation and play back spatialized AR. Ultra
wide Band sensors localize users in the space.
A sand table is used to place sound markers and sketch sound paths. An augmented headset tracks head orientation and play back spatialized AR. Ultra
wide Band sensors localize users in the space.
A sand table is used to place sound markers and sketch sound paths. An augmented headset tracks head orientation and play back spatialized AR. Ultra
wide Band sensors localize users in the space.
System setup of the interaction
Other Projects
CUIs for Creative CollaborationProject type
Origami SensiProject type
Sand PlaygroundHuman-AI collaboration for creative practices on sand
Mixed reality In-situ WeldingProject type
Sandscape→SoundscapeProject type
Penumbra of PrivacyProject type
Memory PortalExperiment and Observation on Augmented Spatial Experiences
Remote EncounterMediating ambient presence for remote social groups
Here/ThereExperiments in telepresence and remote experiences co-creation
Intelligence in EnvironmentsAn investigation about presence, temporality, and memory
Memory MatrixProject type
I3 Imitation, Iteration, ImprovisationEmbodied Interactions
From Nature to FabricationThe simple logic of nature creates complex forms
Contact
Lab Director:
Dina El-Zanfaly
delzanfa@andrew.cmu.edu
School of Design, MM207
Carnegie Mellon University
Pittsburgh, PA 15213
USA