Designing for Embodied Interaction with Sound

The development surrounding digital music interface (DMI) technologies have begun to con- sider embodied and enactive design approaches [1, 2]. This has largely been driven through the need to address ongoing problems that are common across many DMI’s [1, 3, 4]. This shift in design perspective is in following traditional Human Computer Interaction (HCI) which is now thought to be moving towards a third paradigm, which is considering phenomenological approaches within its practice [5].

This research intends to build on the work that has already been completed within the field that has centred on embodied and inactive approaches to the design and contraction of DMI’s and New Interfaces for Musical Expression (NIME). However the focus of this research will be centred around the user of such interfaces. Even with the emerging shift to consider embodied ideas, the focus of design is still heavily centred towards the interface itself. This can be seen throughout the field with much comparison being carried out between traditional acoustic instruments, and digital musical instruments [3, 6, 7]. In many of these cases the understanding of the experience of the player is assumed, or incorporated through the inclusion of fundamental feature of acoustic instruments [3].

In focusing on the user within a musical interaction, brings forth many phenomenological questions relating to musical experience, and music cognition in relation to musical instruments and interaction. This research can therefore seen as being positioned within the field of embodied music cognition [8]. As much of the experience of playing musical instruments is situated within the learning of it this research also has an interest and focus within learning and musical education, an area in which the potential of such interfaces has already been demonstrated in research from: [9, 10, 11, 12, 13].

Keywords: New Interfaces for Musical Expression, Embodied Interaction, Embodied Music Cognition, Flow, User Experience Design, Learning, Musical Skills.


  • [1]  N. Armstrong, An enactive approach to digital musical instrument design. PhD thesis, Prince- ton University, 2006.
  • [2]  G. Essl and S. O’Modhrain, “An enactive approach to the design of new tangible musical instruments,” Organised Sound, vol. 11, p. 285, Nov. 2006.
  • [3]  P. R. Cook, “Remutualizing the Instrument : Co-Design of Synthesis Algorithms and Con- trollers Computer Science Dept . ( also jointly in Music ),” Most, vol. c, no. Smac 03, pp. 6–9, 2003.
  • [4]  N. Bernardini, X. Serra, M. Leman, and G. Widmer, eds., A roadmap for Sound and Computer Music. The S2S2 consortium – Sound to sense, Sense to Sound, v1.0 ed., 2007.
  • [5]  S. Harrison, D. Tatar, and P. Sengers, “The three paradigms of HCI,” Alt. Chi. Session at the SIGCHI . . . , pp. 1–18, 2007.
  • [6]  S. J. Puig, “Digital lutherie-crafting musical computers for new musics’ performance and improvisation,” 2005.
  • [7]  C. Cadoz, “Supra-Instrumental Interactions and Gestures,” Journal of New Music Research, vol. 38, pp. 215–230, Sept. 2009.
  • [8]  M. Leman, Embodied Music Cognition and Mediation Technology. Cambridge, MA/London: The MIT Press, 2008.
  • [9]  A. N. Antle, G. Corness, S. Bakker, M. Droumeva, E. van den Hoven, and A. Bevans, “Design- ing to support reasoned imagination through embodied metaphor,” Proceeding of the seventh ACM conference on Creativity and cognition – C&C ’09, p. 275, 2009.
  • [10]  A. Antle, G. Corness, and M. Droumeva, “Human-computer-intuition? Exploring the cogni- tive basis for intuition in embodied interaction,” International Journal of Arts and Technology, vol. 2, no. 3, pp. 235–254, 2009.
  • [11]  A. Antle and G. Corness, “Exploring Embodied Metaphors for Full Body Interaction,” pp. 1– 4, 2009.
  • [12]  L. Nijs, P. Coussement, C. Muller, M. Lesaffre, and M. Leman, “THE MUSIC PAINT MA- CHINE,” 2007.
  • [13]  L. Nijs, B. Moens, M. Lesaffre, and M. Leman, “The Music Paint Machine : Stimulating Self- monitoring Through the Generation of Creative Visual Output Using a Technology-enhanced Learning Tool,” Journal of New Music . . . , no. August, pp. 37–41, 2012.