This Austrian Artificial Intelligence Podcast episode provides an in-depth look at Beat Shaper, an AI co-pilot for music producers developed by a startup co-founded by Taylor Peer.
Beat Shaper is an AI co-pilot for music producers, that is characterised by its bottom-up approach to music generation, distinguishing it from "top-down" systems like Suno that produce finished audio pieces. Instead, Beat Shaper supports the iterative process of music production, generating elements such as drum beats, baselines, effects, and synthesizer settings. This provides users with fine-grained control and the ability to edit the generated components in external digital audio workstations (DAWs).
The system architecture combines a Variational Autoencoder (VAE) and a Transformer, where the VAE is used to create high-dimensional embeddings that represent user preferences, which are controlled via interface sliders affecting musical attributes like genre or "vibe". These embeddings guide the Transformer’s autoregressive generation of a custom symbolic music notation, that encodes instructions for playing music, including notes and instrument settings, and is inspired by OpenAI’s MuseNet.
A significant aspect of development was creating a synthetic training dataset in the custom symbolic notation, as such data was not readily available. This proprietary dataset, enables excellent results, while applying comparable small transformer models in the range of 1-5 Billion parameters) that enable near real-time audio generation.
Beat Shaper targets both beginner producers and is currently under development. An early better is available through a web platform using text prompts and real-time adjustable sliders. A first release of Beat Shaper is planned for summer 2025.
References
- https://www.beatshaper.ai/
- OpenAI MuseNet – https://openai.com/index/musenet/