Generative systems present new opportunities for composers, but it can be unclear how to integrate such systems into creative workflows. We put forward a vision for a generative audio workstation, in which the composer can work with generative expressions much like ordinary audio or MIDI items, seamlessly mixing static and dynamic musical material. We present our research prototype in this direction, LambDAW, which takes the form of an extension to REAPER (a popular digital audio workstation) that executes Python expressions directly in the timeline, and we discuss possibilities for integration with generative models and machine learning libraries.
generative music, end-user programming, music composition, digital audio workstation
© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License formula, a name begins with = to indicate that it contains
Attribution 4.0 International (CC BY 4.0).
proach afords limited support for generative music. To
and work with disparate materials. However, this ap- draw on computational notebooks such as Jupyter [
Australia (J. Freeman) (J. Freeman) htp:/ceur-ws.org ISN1613-073
CEUR
Workshop Proceedings (CEUR-WS.org) 1They can also put the generative system in a plugin, but this is hidden in the FX chain, exists for all time, and has limited ability to interact with the DAW. as DAWs are with (static) digital audio. In our vision, code is not something separate to be executed outside of the DAW, nor is it timeless in the FX chain. Instead, it is right in the timeline, alongside the other musical materials. Furthermore, it can connect to other materials by reference, enabling the composer to create meaningful links through the piece. In this model, the composer need not give up all control to generative systems, nor reject them entirely. Instead, they retain ultimate control of the piece and can bring in generative systems as they see fit, like any other source of material.
code fragments into a music tracker), Ossia Score [
REAPER called LambDAW (“lambda” + “DAW”).2 With in the timeline to generate audio or MIDI output directly in the DAW, as shown in Fig. 1. These expressions are stored in take names, so that both the code and its generated output are visible in the timeline. Like a spreadsheet
an expression to be evaluated. (The = may be further preifxed to give it a name for later reference, as in f o o = b a r ( ) .) When LambDAW detects a new or updated expression, it We believe LambDAW ofers a useful way to compose automatically evaluates it and puts the generated output with AI and ML by bringing code into the familiar inin the associated item. The user can also re-evaluate ex- terface of the DAW. Because LambDAW allows arbitrary pressions on demand, e.g. to get diferent outputs from a Python expressions in the timeline and provides the model. project module as a place to import libraries and load as
Expressions can refer to other items in the timeline as sets, it enables the user to take advantage of Python’s rich
variables. For example, if there is a MIDI item in the time- ecosystem. Expression items can serve as user-specified
line named m y _ c o o l _ r i f f , an expression item can refer “blanks” for an AI to fill in, which the user can re-evaluate
to it with an expression like = t r a n s p o s e ( m y _ c o o l _ r i f f , to generate new output. The ability of LambDAW
expres5 ) . This feature allows the composer to establish connec- sions to refer to other items in the timeline provides
tions between musical material; if they later modify the a convenient way to work with models that transform
original rif, the derived item can be updated simply be or continue input material, and for the user to pre- or
re-evaluating it (unlike a transformed copy, which “for- post-process model data.
gets” its relationship to the original material). Referring Looking ahead, we plan to further explore integrations
to items in expressions also facilitates the transformation with generative libraries and models to find how they can
of recorded material with code. (Expression items can ift into compositional workflows. Some initial work in
also reference other expression items, as in the example this direction is depicted in Fig. 1, which features
expreswith c a r p in track 1 of Fig. 1.) sions that invoke MusicVAE [
Linking expressions by reference enables the user to symbolic data and audio, respectively. We also aim to divide up complexity between expressions and so pro- make architectural changes to improve LambDAW’s usvides a way to manage complexity and expression length; ability (especially with ML libraries), such as performing the user project module provides another.3 LambDAW evaluation in a separate process and allowing to user to loads a user-defined module for each project in which the interrupt long-running operations.4 Finally, we hope to user can define functions, import useful libraries, load present our system to composers to get their feedback resources, etc., for use in timeline-embedded expressions. about the quality of the integration, the degree to which The user can also customize how LambDAW converts it facilitates their use of generative systems, and how it DAW items to/from Python objects. The user project afects their creative process. module thus supports concision and customizability, enabling the user to choose their own set of abstractions for composing and maintain the brevity of expressions in the timeline.
advisable to keep it brief so that the whole thing can be seen at a glance and without excessive zooming.
REAPER’s embedded Python support when using libraries such as NumPy, TensorFlow, and PyTorch, which required workarounds. Moving evaluation out to a separate process (as in Jupyter kernels) would avoid these issues, enable interrupting/killing the interpreter without restarting the DAW, and prevent evaluation from blocking the UI thread.