Auditory-Tactile Narratives: Designing New Embodied Auditory-Tactile Mappings Using Body Maps

The increasing number of haptic interfaces for musical applications offers opportunities to create new auditory-tactile mappings. However, the innate, bodily correspondences between music and vibrotactile stimuli remain unexplored and a challenge to articulate. In this paper, we investigate how new mappings can be created based on the somatic, felt experiences from vibrotactile devices. We utilize an embedded HCI method to collect, articulate, and document such bodily experiences through body maps. Through an exploratory workshop with mixed-background participants and a focus group analysis, the participants reported their bodily experiences using two categories of haptic devices on the body maps and ideated new mappings through co-design. Based on the results of these sessions, we report three annotated auditory-tactile mapping schemes, each of which offers different strategies. The themes that emerged in the workshop interactions help us examine implicit associations of auditory-tactile stimuli on bodily sensations and how body maps can be used to articulate these sensations while offering an opportunity to generate new ones.