Machine Learning as Design Material for Music-Making

We present a Research-through-Design exploration of Machine Learning (ML) as design material in music-making. We designed Picnic, an interactive musical installation that augments everyday objects in a picnic basket into a loop-based sampler which allows users to build rhythms with a variety of percussive, harmonic and more-than-human sounds. Embracing ML’s inherent uncertainty, we intentionally used an underfitted real-time classification model to create a playful and ambiguous music-making experience with the system. Through an evaluation with 23 participants of varying musical expertise and AI interest, we found that the system’s misclassifications made participants engage in a creative dialogue, constantly adapting to its unpredictability. Furthermore, when errors occurred, participants tended to criticise themselves rather than the system, indicating a tendency to overtrust the system. Our findings contribute with insights into the potential for using ML as design material for music-making and other creative domains.