Co-Creative Artificial Intelligence of Music, CCREAIM
The main objective of the project, CCREAIM , is to enable the next generation of AI models that stimulate creative processes and enhance music practitioners’ creative strength. The joint research project, Co-Creative Artificial Intelligence of Music – CCREAIM, in collaboration with SOPI Research Group Aalto University and Google Brain Team (Magenta), will fo- cus on creativity with a wider social practice approach on mutual interactions between computational technologies and human musicians. The CCREAIM project will investigate the practical implementation of AI enabling new possibilities of connecting technology to artists to open the door to say something new.
Digital Musical Interactions – Instruments – Performances, DMI – I – P
Academy of Finland Research Fellow Project, pr. 316549
The main objective of the project, Digital Musical Interactions – Instruments – Performances, is to articulate a framework to better understand the principles of emerging technologies and cultural constraints behind digital musical interactions, specifically how they appropriate and give meaning to the evolving nature of the relations between performer, musician, instrument, audience, instrument designer and environment in digital music practices. In this macro-view understanding of digital musical interaction, digital music performance’s disparate elements will be connected and interdisciplinary design factors of new musical instruments, technological and cultural constraints will be structured. The project’s framework will help us to better understand the interaction principles from a cultural and the technology-interaction perspective, which can open up new set of values for us to explore and imagine that do not yet exist in our interaction with digital musical instruments.
AI-terity
AI-terity is a deformable, non- rigid musical instrument that comprise computational features of a particular AI model for generating relevant audio samples for real-time audio synthesis.It has been our research interest to explore and integrate generative AI models applied in audio domain into development of new musical instruments. This in turn has the potential to bring an alternative synthesis of knowledge about musical instruments, as well as enhancing the ability to focus on the sounding features of the new instruments.
Past Projects
InSpace with the Otherness
InSpace with the Otherness is a part of SOPI research group’s project, utilising Google Magenta team’s AI algorithm in a composition that develops through the performer’s choices to generate new sounds that no one ever heard before. The piece is performed through a novel collaborative environment in which the visitor of the exhibition becomes the performer who is not only free to move and interact with the instrumental sounds of the composition, but where her decisions contribute to the sonic outcome of the composition. This interactive music system is designed for co-located collaboration with AI deep learning algorithm and provides embodied and spatial opportunities to exhibition visitors for musical exploration.
Sounding Brush
Sounding Brush is a tablet tool enabling users to express themselves by creating sound and painting at the same time. It is an app aimed at giving access to sound experimentation, editing, learning and performance without prior music or visual arts experience. By linking sound to drawing, painting and mark making Sounding Brush offers an intuitive digital alternative to traditional sound making interfaces. The project aims to provide an inclusive platform to make sound, music and performance more accessible for anyone wanting to become active in this field.
Vibrating Instruments in Virtual Reality (VIVR): Creative Content Production in VR with Cohesive Sense of Reality
In Vibrating Instruments in Virtual Reality (VIVR) research, we aim to a) study the needs and problems behind the key features in musical practices with virtual reality, b) understand the notion of virtual reality, determining its NIME design principles in music production, c) build VIVR virtual reality toolkit, reflecting the principles that were deployed in the role of emerging VR technologies in music practices. In a three-stage inquiry into the new vibrating instruments in virtual reality, we will deliver a VR toolkit in which we will design and build interactive VR musical instruments, reflecting the essential features of our findings. These instruments will be used in the generation of artistic content with music production by the team members.
Laser String Kantele
Laser String Kantele is a musical instrument that finds its roots in a traditional Finnish plucked string instrument, Kantele. This musical instrument gives an opportunity to explore a wider range of sonic worlds that go beyond the sonic characteristics of the traditional one. Its massive sculpture-body enables multi-user interaction to happen with the laser beams that provide an optical control through sound producing gestures using both hands. Its relation to traditional Kantele is with the sound synthesis module; each laser beam can be controlled individually to playback particular notes in real time that are synthesised using physical models of Kantele strings. The sculpture design of the LASER STRING KANTELE also reflects the very early invention of the Kantele instrument, built by using a pike (Hauki) chin as told in Kalevala, the collection of ancient Finnish runes. Laser string structure also gives an opportunity to extend the traditional Kantele tuning, having several alternative tuning structures available in the moment of playing. This project is a joint collaboration between audio researchers and students in Aalto University from Department of Signal Processing and Acoustics School of SCI and Department of Media School of ARTS.
KET – audiovisual artproject
KET is an audiovisual performance with Thomas Bjelkeborn and Koray Tahiroğlu as musicians and video artists on stage for a live sound and video experience. The work involves music and video created live from images and sound of an old woman singing on the language KET. Her singing gets expanded and evolves into a contemporary audiovisual performance. Both Thomas and Koray are involved in the development of new ways for human interaction for live performance.
The Archi-VR: Archived Memories with 3D audio content
Our Cross-platform artistic design research in Archi-VR proposal is condensed in the ultimate form of an artistic work – an immersive auditory space with virtual reality sound. The artistic outcome of this research will be targeted towards Finland’s 100 year celebration, where the theme of Finnishness is reused & remixed with archive materials from three broadcasting networks – Finnish YLE, Swedish SVT and Norwegian NRK. Building on the tradition of classic radio plays, we aim to create an immersive space where diegetic sounds from three country’s archive organizations are used in a single dramaturgy.
Network Of Intelligent Sonic Agents – NOISA
The research Networks of Intelligent Sonic Agents – NOISA studies a new way to inquire the engaging relationship we have with digital artefacts and new media practices. The main argument here is that NOISA provides an intelligent system, that acts to maintain and deepen the user’s engagement by learning from the user behaviour while interacting with digital environments.
The Notion of Participative and Enacting Sonic Interaction (PESI)
The Notion of Participative and Enacting Sonic Interaction (PESI) deals with designing new ways of interacting physically with digital sound in more aesthetically engaging forms. The research aims to increase the potential affordances of digital technologies to create interactions that challenge the notion of expression in collaborative music performances and result in making new forms of music. – Academy of Finland (pr. 137646).We have been also developing a repertoire to be performed with the PESI system.
Flexible Interactions and Devices (FLINT)
This research Flexible Interactions and Devices (FLINT) consists of explorative work on interactions technologies which are especially designed for flexible and deformable form factors. The research activities are divided into two topical areas and tasks: continuous and dynamical audio and tactile feedback in deformation-based interactions, and deformation-based input and output in non-verbal communication and in other implicit interaction applications. – joint research project with Nokia Research Center.
New Interaction Solutions (NIS)
New Interaction Solutions (NIS) focuses on expressiveness of multimodal interaction. Whereas typically interaction research aims to optimize the interaction in terms of efficiency and ergonomics, here the aim is to use the interactive technologies for expressing emotions and feelings. – joint research project with Nokia Research Center.
Designing for Embodied Interaction with Sound
This research Designing for Embodied Interaction with Sound intends to build on the work that has already been completed within the field that has centred on embodied and enactive approaches to the design and construction of Digital Musical Interfaces (DMI’s). However, the focus of this research is centred around the experience of users of such interfaces.
Human Emotional Interaction (HEI)
Human Emotional Interaction (HEI) project aims to take yet another step further in the evolution of communication by focusing on the emotional aspect. The intention is to develop a platform that enables research and development of emotional communication and interaction. – joint project with Nokia Research Center.
Raja – Interactive Dance Performance
Raja – Interactive Dance Performance is a joint project together with Nokia Research Center, utilizing motion-tracking technology with the creation of real-time music of the dance piece.
AaltoWindows
In AaltoWindows project, the students and researchers of the three Aalto schools work and study at geographically separated locations, and therefore rarely interact with each other on a daily basis. To increase the feeling of social unity, this project opens a virtual window between the separate Aalto campuses. – joint research project with Jari Kleimola, Palash Mukhopadhyay and Dipti Sonawane.
Design Fest
Design Fest is a 3-D service platform implementation for virtual customer interaction. Design Fest project is a new kind of mix of web portal, webshop and hi-quality campaign site that celebrate Finnish brands who are part of Helsinki World Design Capital 2012 project. The project was a collaboration between Media Lab Helsinki/ Department of Media, Media Factory and Innofactor. Heikki Sillanpää designed the sound effects and background ambience. The main sound design strategy was to combine elements of synthesis from natural and unnatural sound sources. In the final version, the system itself acted as a musical instrument.
