Agnes Mercedes Kloft

Recent advancements in artificial intelligence (AI) have driven innovative applications across performative arts, especially in music practices involving real-time content generation and accompaniment. Systems such as Yamaha’s AI pianist and Google’s Magenta show AI’s potential in music, yet most function as reactive partners that follow the performer without shaping the performance collaboratively. Current systems remain largely one-way, tool-like roles where AI reacts but lacks the mutual interaction and shared creative agency needed for meaningful artistic exchange.<br/>The Augmented Performer project proposes a novel paradigm for real-time human–AI co-creation in live music, developing a generative audio AI system that listens to and responds to a pianist, co-composing in real time. The system will integrate expressive performance analysis—capturing timing, dynamics, and articulation—and generate harmonically and rhythmically adaptive responses, enabling a fluid exchange of musical ideas. Unlike conventional accompaniment tools, our approach keeps the human at the creative center, while allowing mutual influence between musician and AI.<br/>