Why
Wikipedia defines synesthesia as "a perceptual phenomenon in which stimulation of one sensory or cognitive pathway leads to involuntary experiences in a second sensory or cognitive pathway." One of the most common variations of this phenomenon is a connection between color and sound. Although this is a rare form of perception to have, many wish that they did perceive the world in such a vivid and connected way, with certain colors evoking certain sounds and vice versa. It is an interesting concept that can tell us a lot about how our brains work. Although these senses are picked up and processed by different mechanisms, there is clearly some crossover in how we make sense of sound and visual stimulation. Both sound and visual art can evoke a wide range of similar emotions.
The concept of synesthesia is the inspiration for my latest project, Color Chords. The concept of the project is to randomly generate a color, then synthesize a sound based on the emotive effect of that color.
If you prefer a video, I made one on this topic which you can find here:
Setup
I started out the project by creating a vanilla javascript Vite project. You can do this by running the command "npm create vite@latest" in the directory where you want your project to live. After naming the project, select vanilla javascript. After your project is created, you will also need to install Tone.js with the command "npm install tone." Open up your project in an IDE.
Color Generation
I wanted the sounds to be based on a randomly generated color, so that's the first step. I used rgb format, so all I need to do is randomly generate three integers between 0 and 255 in the format of "rgb(Int1, Int2, Int3)". Here is how I did that.
Now that we have a random color generated, we will set the background of the div to that color and put it as a heading as well so that we can see the color displayed with its rgb formatted label inside.
I set the background color not only when the page loads but also in the click event for the play button so that the color resets whenever the button is pushed. You might have also noticed that I separated the r, g, and b into separate variables, which will be important for creating the chord.
Sound Generation Setup
Now that we are generating random colors with rgb values, we can create rules for these values that generate sound with Tone.js. There are many possible ways you could do this since interpreting sound from color is a subjective process. I decided that each chord would be made up of three notes which I am calling the root, second, and third. The root note is based on the level of red, the second note is based on the amount of blue, and the third note is based on the amount of green.
The first thing I did was summarize the emotions and thoughts that red, green, and blue tend to evoke. Here are the qualities I ended up with.
Since the root note is basically the key of the chord, I also did some research into the emotive qualities of musical keys and ended up with this summarization:
I then mapped out these qualities to how much they aligned with the qualities of red to rank all the root notes in terms of how "red" they are. I then mapped these rankings to their equivalent value ranges from 0 - 255. So if the color's red value is 0 - 20, the root note will be G, for example.
For the blue and green values, there are 6 options for each. I used blue for the second note, because this is the easiest place in the chord to define the emotive quality, and blue has a pretty clear emotive quality in western culture, representing sadness, melancholy, and calmness. So for the highest blue level, it was clear the second note would be a minor third up from the root note. For the lowest blue level, the interval is obviously a major third. I used major and minor sixths, perfect fourths, and perfect fifths, to fill out the rest of the value ranges for blue.
The level of green defines the third and final note in the chord. The third noted will either be a minor 9th, minor 7th, perfect 5th, major 9th, or major 7th. It is set using the same number ranges as the second note.
Now that we know how the notes will be set, I need to create JavaScript objects for each key so that the program knows which notes to play based on the root note and the intervals(distance) between the root and other notes. In a file called data.js, I stored the necessary intervals for all keys. Here is an example for the key of A#:
I also made a rootToKey() function in this file, so that that format compatible with Tone.js ("A#") would return the format compatible with my objects ("Asharp").
Sound Generation
Now we are ready to write functions to return each of the three notes. To generate the root note, we follow the rules previously established based on the level of red.
Then we do the same with our established rules for determining the second and third notes, using the key established by the root note to form the right intervals.
The octave is added to the end of the note in the return statement of all the note-generation functions. The octave is determined by the combined sum of the r, g, and b values, with a higher sum resulting in a higher octave. For some of the intervals in the third note, the octave is raised by 1.
Finally, to put it all together, we create a Tone.js polysynth, pass the rgb values into the note generation functions, and play the chord. All of this happens inside our button's click event.
Conclusion
Now, you should have a functional application that generates a new color and chord when the button is pressed. My next steps are to further customize the sound by adding variations to the timbre. I also plan to improve the interface design, among other fun things. Stay tuned!