Improvisation plays a cardinal role in the arts and is acknowledged to be a typical manifestation of creativity. In performing arts, an impromptu consists in playing extemporaneous sequences of actions, i.e. notes or movements, in accordance to some rules and constraints. Typically, a good improviser masters those constraints and can produce meaningful paths in the feasible space of allowed actions and can also explore some areas in the adjacencies of this space. From a computational perspective, one of the possible ways to capture this creative production is to make use of statistical learning mechanisms, which are also believed to be involved in human musical improvisation. At the basis of statistical learning are transitional probabilities between segments of a sequence and their following segments of symbols. In this paper we present preliminary results of a statistical learning model in which a transitional probability graph is computed from a set of sample pieces of music. This graph is subsequently generalized by applying a node similarity mechanism. This generalized graph is used for generating melodies that resemble improvisations in a given musical style.
According to the Grove Music Online [
Recent works address musical improvisation in the context of statistical learning [
Inspired by the SL literature, especially regarding computational approaches like [
In implicit sequence learning, such as in language, music or movements, initial acquisition of
implicit sequences may arise from SL [
The algorithm proceeds through three subsequent phases: learning, generalization, and
generation (see Figure 1). In the learning phase, we introduced TPs at two specific levels:
between symbols, as a cue for segmenting the incoming input into small segments (or chunks),
and between these formed chunks. After the learning phase, the graph of TPs between chunks
undergoes a generalization phase. This phase draws on the distributional learning hypothesis [
Finally, this new generalized graph is employed to generate novel, structured sequences using an ad hoc Monte Carlo creative walk.
The model can be applied both on unsegmented and segmented corpora.
The learning phase consists of two mechanisms: tracking the transitional probabilities between symbols (second order TPs) to be used as cues to segment the input into words (or units,
LEARNING segmentation tracking
GENERALIZATION nodes SimRank
GENERATION MonteCarlo walk chunks), and tracking TPs between those words (first order transitions) to form a graph made of transitions between chunks.
At each perception cycle, TPs between the observed symbols are stored. Initially, the algorithm
tries to use stored TPs to find a drop in the transitions between symbols that would determine
the boundary of a word. On the other hand, if no TPs cue is found, a syllable is perceived
(two consecutive symbols). The segmentation strategy used in this work is one of the simplest
where a boundary is detected if the transitional probability of the upcoming symbol drops
under a certain threshold, so if > +1 + . In the present study, we used = 0.2 as an
empirically selected threshold. However, various strategies could be exploited (see [
After segmentation, TPs between the resulting ordered units are recorded. Note that this per se represents an abstraction intended to grasp the dynamics, the transitions, between formed words—not between symbols.
The output of the learning phase is a graph where nodes represent units/words and edges represent transitional probabilities between words. To construct the generalized graph, the procedure first computes the form classes, using similarity between nodes, and then generates some sequences (with the TPs graph) that are parsed to build the higher-level graph. The similarity between nodes is computed using a SimRank [16] measure over inward and outward edges. SimRank is a graph-theoretic measure that says "two objects are considered to be similar if they are referenced by similar objects". In this case, we used a slightly modified version where "two objects are considered to be similar if they are referenced by similar objects . . . and refer to similar objects". That is, nodes are grouped if they have similar inward and outward edges, so if they have a similar neighborhood. Precisely, we group nodes with both inward and outward SimRank greater than a threshold value . In this case, we used = 0.5 as, in our experiments, it provided convincing similarity values over known samples. So for each node we calculate = { ∩ } where, for each node :
= { : ( , ) ≥ } = { : ( , ) ≥ }
The formed groups represent what in language acquisition is called form classes [17]. Once calculated, the form classes are used to parse some generated sequences (using the TPS graph), and the new generalized graph is then built. Transitional probabilities between formed (form) classes are computed as well, counting transitions over the parsed sequences.
The generalized graph is then used to produce novel sequences. In the present experiment, we opted for a simple Monte Carlo choice over the edge probabilities to traverse the graph. At each visited node, as in general it may contain words that can be used in the same position in the construction, a word is picked randomly (the nodes that contain alternative words are called here choice nodes). Another possibility for selecting a word is to use the weights (the frequencies) of the words to employ another Monte Carlo choice at each node.
A prominent context for improvisation is of course music. Since the system we developed is mainly focused on sequences of symbols, we opted for melodic pieces of music. Therefore, we provided the system a set of melodies belonging to a given style (e.g. Irish music) upon which the TPs graph and the generalized graph can be built. The latter provides then the basis for the generation of new melodies in the style of the repertoire provided, but with variations and explorations in the implicit boundaries set by the examples. The resulting melodies are characterized by improvisation flavor, as they have not the structure of a complete piece of music, but capture the main stylistic features of the original compositions, like a musician making extemporaneous explorations around a given style.
To test the system we chose two diferent styles: Irish melodies and the six preludes from solo cello sonatas by J.S. Bach. Irish melodies have been retrieved from Henrik Norbeck’s abc tunes [18]. All the 136 melodies in the key of G have been gathered (including variations of the same song) and the abc notation symbols, which encode the music in textual form, have been directly used as sequence symbols. The second repertoire of melodic music, instead, has been retrieved in MIDI format from David J. Grossman’s J.S. Bach page [19]; the MIDI files have been converted to an intermediate textual representation by means of PyPianoroll [20] and transposed to the same key, so as to have sequences composed of symbols representing the intervals from a common base note. In both the corpora of examples, a symbol in a sequence represents both pitch and duration.
44
We are interested here in the features of the generalized graph and the characteristics of the melodies it produces. The number of nodes containing alternative choices and the number of choices estimate the amount of “controlled exploration” around the musical style learned. For example, a typical choice node in the generalized graph of Irish music may have the following alternatives: B A B A G G3 | F G2 G2 | G B A A G G2 | c A G2 G A, represented in Figure 2 in score notation. In general, the segments difer in start and end note, as well as total duration; therefore, they do not represent equivalent alternatives, but rather diferent sub-paths that can be used to compose a new path which is likely to combine fragments of melodies in an original way, yet keeping the flavor of the melodies in the repertoire. The generalized graph built from Irish music has 151 nodes, of which 19 are choice nodes. The choices in each node are distributed between 2 and 8, with a median of 3. The resulting melodies are similar to the ones in the repertoire, but characterized by a considerable degree of originality. The interested reader can find audio excerpts and score transcriptions at [21].
The generalized graph of Bach preludes for cello solo substantially difers from the one related to Irish music, as it it composed of a greater number of nodes (526) and a lower number of choice nodes (10), all with just 2 choices except one with 4 choices. Another remarkable diference with respect to the previous case is that the melodic segments in each choice node are longer. The musical diference between the two repertoires is wide and this has of course strong impact on the properties of the generalized graph. Irish traditional music is characterized by simple elements: almost all the notes used belong to the scale of G major and the maximal diference in pitch is about two octaves. Moreover, the melodies are often composed of long sequences of notes at small intervals and few large steps (e.g. of an octave or a fifth). Conversely, the preludes for cello solo by Bach span a wider range of pitches and the use of chromatisms is extremely common. In addition, the examples available are much less than the Irish ones, so the probability of overlaps between portions of melodies is much lower. The features of the two graphs reflect the musical properties of the two styles in that the richness of Bach’s style and, above all, the hierarchical structure of his compositions limit the adjustable interchangeability of melodic segments which is expressed by the generalized graph. However, the musical result of artificial improvisations in the style of Bach’s cello preludes is appreciable (audio and score excerpts are available at [21]).
We presented a generative model that uses a stochastic walk on a topological generalization of variable Markov Chains (TPs graph), to produce novel musical sequences. The presented work is intended to be a seminal, basic module of a more extensive system conceived for emulating the learning of implicit sequences. It is intentionally domain-general and symbolic since it is intended to model various phenomena: from music and language to movements and social interactions [22]. This learning system assimilates implicit knowledge that becomes the basis for modeling implicit, automatic behaviors. In these regards, we envision adding a short memory module, to model higher-level phenomena such as attention, for example. However, even if in this case the focus was on the learning system, the ultimate goal is, in fact, that of producing creative outputs. In this perspective, the next step will be to use an ad hoc, creative Monte Carlo walk in place of the simpler stochastic one, that is, to give the model the ability to explore creative paths instead of the most (or the least) probable ones. We believe a model built in this way could also provide a place, an environment, for simulating and studying a variety of behaviors in cognitive science and creativity. [16] G. Jeh, J. Widom, Simrank: a measure of structural-context similarity, in: Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, 2002, pp. 538–543. [17] J. F. Schwab, K. D. Schuler, C. M. Stillman, E. L. Newport, J. H. Howard Jr, D. V. Howard, Aging and the statistical learning of grammatical form classes., Psychology and Aging 31 (2016) 481. [18] H. Norbeck’s abc tunes, http://www.norbeck.nu/abc/, last accessed February 2021. [19] D. J. Grossman’s J.S. Bach page, http://www.jsbach.net, last accessed August 2022. [20] H.-W. Dong, W.-Y. Hsiao, Y.-H. Yang, Pypianoroll: Open source python package for handling multitrack pianorolls, in: Late-Breaking Demos of the 19th International Society for Music Information Retrieval Conference, 2018. [21] Music excerpts for the paper titled “Machine improvisation through generalized transition probability graphs” by M. Barbaresi and A. Roli, https://tinyurl.com/n4bc6x73, last accessed October 2022. [22] C. Monroy, M. Meyer, S. Gerson, S. Hunnius, Statistical learning in social action contexts, PloS one 12 (2017) e0177261.