Musical composition typically requires several years of experience and knowledge in music theory that includes but is not limited to chord progression, melody note theory, and an understanding of whole-step/half-step passing tones among others. Due to the complexity of musical composition, even experienced musicians may find the writing process challenging and time consuming. Songwriters may find the need to employ trained pianists to aid in the theory behind musical composition. We present Lumanote, an interactive real-time music composition tool that aids songwriters in their writing endeavors by presenting real-time suggestions on appropriate melody notes and chord progression. Lumanote's color-coded design provides chords appropriate to chosen scales, and melody notes appropriate to given chords in real-time as musicians play chords and melody notes. We performed a qualitative study to gauge the effectiveness of this tool, including both experienced musicians and complete beginners to assess the intuitiveness of the system at various skill levels. Qualitative results helped in identifying potential issues on differing interpretations of color-coded suggestions, and yielded insights on interface improvements and polish.
INTRODUCTION Computer-aided musical composition remains a promising technology for the creation of the newest generation of music. Many current technologies, however, tend to focus ©2018. Copyright for the individual papers remains with the authors. Copying permitted for private and academic purposes.
MILC ’18, March 11, 2018, Tokyo, Japan
on the development of automated or semi-automated music
composition, taking with it some of the agency and in turn
some of the identity of music creation. Songwriters often hire
the assistance of professional pianists to aid in composing
a song, providing insights on chord progression and melody
theory to harmonize with the given chords. Since musical
composition is inherently a human endeavor, we believe
significant care must be placed to develop software that helps
composers create melodies they can still consider completely
their own.
Tools for Musical Analysis
Demand for digital tools that analyze music has risen in recent
years. Bello et al. details a system that extracts rhythmic
and harmonic information without attempting to transcribe the
entirety of a song [
Tools for Musical Composition
Several technologies have been developed to help musical
composition through procedural generation, with increasingly
more complex methods of input and parameters that affect
the resulting output for more musically interesting material.
Diaz-Jerez leveraged his previous technology of FractMus into
a project called Melomics in order to assist in the procedural
creation of music based on biologically inspired musical
evolution [
Teaching Systems
Computerized interactive teaching tools have promisingly
emerged as low-cost, convenient solutions for learning about
music theory and instrument playing. Nilsson et al. has
explored the integration of professional computer musical
sequencing programs with the creative composition process of
1Hookpad: https://www.hooktheory.com/hookpad/new
children, and found that students were successful in developing
music of sound form and structure [
DESIGN In this research we present Lumanote, an interactive online tool that integrates MIDI input to suggest chords and melodic notes depending on the scale and type of user input. This project was conceived to fill the need to enhance the music composition process for beginner and intermediate composers. The software is intended for users to create music with appropriate chord progressions and melodic choices within only minutes of starting to use the software. Careful consideration of the existing digital composition tools and technologies has yielded the following set of design requirements:
Create a practical piece of composition software that can fully function with minimal cost, installation time and effort Allow users to interface with the software with or without a musical instrument with equivalent levels of productivity Intuitively present appropriate chords and accompanying melodic notes to a user who may not have had previous music composition experience Guide users toward the creation of a melody without taking control from the composer or sense of authorship System Setup In order to minimize software setup requirements, Lumanote is hosted on the website www.lumanote.net for access with any computer that has an active internet connection. It is recommended that users access the website through the Google Chrome browser. The system’s first screen directs the users to an optional video tutorial on how the use the software in four minutes. The application then scans the USB input devices for MIDI-type devices. Users who wish to instead use the software with a computer keyboard or mouse will be presented with instructions on which keyboard keys are mapped with which notes. Afterward, the user will see the main interface screen.
Interface Design Notes are represented through the standard keyboard layout due to the instrument’s naturally intuitive note representation of one key per note, as well as the ease of mapping a USB keyboard’s inputs with that of a digital on-screen keyboard. The top bar controls the mode, musical key, a toggle for microphone input for singing, the synthesizer’s sound, and the color palette. The bottom left buttons are six chords that correspond to the selected key. The bottom right buttons include a function to shift the "split point" (the note that separates the chord notes with the melodic notes), a "panic" key that resets the interface who have become confused at their setting changes or encountered any interface bugs, a function that highlights all the notes of the selected musical key, and zoom-in and zoom-out buttons to alter how many keys are shown on the screen at once. The system is also able to recenter the digital keyboard’s view in case a note that is out of the screen’s view is played on the keyboard.
Note Highlighting Users can active Lumanote’s suggestion system in one of two fashions: play a base chord, or a melody tone. Any keys pressed by the user will be denoted by a column of color that will extend vertically, clearly indicating to users which keys they are currently playing. If a chord is played that is one of the chords of the chosen musical key, any melody key that the user then presses will highlight all adjacent notes that are appropriate for that chord and base melody note. Similarly, if a melody note is played first, Lumanote highlights all root chord notes for that note’s accompanying chords. This allows users to create music in both possible forms of composition: if the user has a melody as inspiration, they can quickly receive suggestions on appropriate chord progression, and can similarly receive melody note suggestions if their inspiration is a chord progression. This ensures that Lumanote accommodates for both forms of the creative composition process.
In musical composition there exist varying degrees of what is an "appropriate" note. While some notes are considered "safe" (one of the chord notes), other notes like whole-step, halfstep passing tones, and 7th/9th degree notes are used to strain the melody before its resolution. Lumanote’s note and chord suggestion color codes the highlighted notes to indicate which suggested notes are "safer" than others, so that composers can adeptly alternate between safe notes, passing tones, and strained notes as they compose their music.
The specific color scheme that differentiates between the safety level of suggested notes depends on the "Theme" chosen. The "beginner" theme highlights all in-scale notes the same color, whereas "Intermediate" and "Advanced" modes integrate more colors of the rainbow so that the least "safe" notes are closest to red. "Rainforest" highlights the notes in a green and blue gradient, "Ocean" highlights notes in various shades of blue, and the "Sunset" palette integrates tones of red, yellow and purple. Differing color schemes are for the preference of the composer only and do not affect the suggested notes or their level of safety.
Musical Composition Process The musical composition process involves the user exploring suggested notes, which in turn suggests appropriate chords and notes that accompany the pressed note. This allows composers to evolve their composition and progress their composition from one "step" to the next. Users may begin the composition with a chord and a single melody note, and follow the suggested highlighted notes until the user desires to change the chord. Pressing the single melody tone at that point will then suggest new chord root notes. The user can then pick one of the root notes, play the chord appropriate to that root note, and the new chord combined with the existing melody note will suggest new melody notes that the composer can then use. Lumanote works best when the composer utilizes the suggested chords to select melody notes, and then in turn use those melody notes to select appropriate chords to advance the progression of the composition.
The musical composition process can be completed with a musical MIDI controller such as a digital keyboard, but can also be used with a computer typing keyboard. If using a computer keyboard, users can press the numerical keys from 1 to 6 to have Lumanote automatically play every note in the corresponding chord. Users may also interact with the software with a mouse by selecting melody notes for chord suggestions, hovering over and clicking on listed chords on the bottom left to identify the appropriate chords, and hovering over the "Keys" button on the lower right to highlight all the keys appropriate to any given scale. A singer songwriter can also interface with the system by enabling the "Sing" option, which takes a singed note via microphone as a melodic input and Lumanote will suggest accompanying chords. The numerous ways to interface with Lumanote have been developed with the intention to provide as flexible a piece of software as possible.
SYSTEM EVALUATION Composition Activity A system evaluation comprised of a user study of 20 participants. Of these, 12 were non-pianist musicians, 6 were pianist musicians, and 2 were non-musicians. They wwere first asked to create a "simple melody" without the use of Lumanote. We did not provide specifics on melody type, key, or composition. Non-musicians had the most difficulty with creating a melody without using Lumanote, generally creating melodies that had no structure and followed no key. More experienced musicians required some time to remember specific chord progressions.
After each participant completed what they considered a simple melody, they were asked to repeat the process using Lumanote. They were given the choice to start with a new melody or refine their existing work. Nearly all participants opted to start a new composition. They were given no instructions on how to use the system in order for us to gauge its level of intuitiveness. Researchers answered the participants’ questions when asked. All participants, including experienced users, showed considerable improvement in the quality and musicality of their composition.
Suggestions and Changes A four-minute instructional video was created as a need to address the most common questions from participants, including questions about color scheme, explanation of the Split Point line, questions about genre, and choosing notes. We initially had one single color palette, but soon observed most users had different interpretations of colors for suggestions. This prompted us to develop various color palettes to suit more individual’s personal interpretations.
Several UI additions were made for various use cases. Key widths can now be changed to accommodate various computer screen sizes and resolutions, the Split Point line location can be now changed at the request of more advanced participants, and hovering over each chord to display the notes was added at the request of the intermediate users who otherwise needed reminders on which chords to play. Finally, we added nonMIDI input devices with keyboard, mouse, and note singing since participants frequently reported not having a MIDI keyboard at their disposal if they were to use this system at home.
FUTURE WORK AND CONCLUSION Lumanote’s powerful and flexible melody and chord suggestion system presents significant potential in leveraging digital technology to aid composers in creating music quickly and efficiently. Careful design decisions were made to place all composition decisions on the side of the user, to ensure the author preserves their identity and can call the composition their own. Future iterations of this system will include a menu to select different styles of music, and UI elements for beginners to help them understand what makes a "good" melody, so that they know how often to pick safe vs. unsafe notes.