THE VISION

I think I've finally got my thoughts in order for this Interactive Exhibit class. I see...A VISION. THE vision. I've already indicated on my website that I'm interested in creating something that conveys information through music. Admittedly, I had absolutely no idea HOW I would go about doing this, or even WHAT such an exhibit would look like, what kind of information it would be transmitting, or how people could use it. Enter: the vision.

I envision an exhibit that allows visitors to VIEW how people have historically used vocal expression to convey meaning. Not everyone is musically inclined and can't quite understand how music, or even sound, is really a rich language to be interpreted. I plan on taking a look at a couple different vocal media; for example, Inuit throat singing, opera, or perhaps even political speeches. The idea is that the visitor would be allowed to look at the visualization of the sounds and gather information about what the vocalist is trying to say without necessarily understanding the language of the words they are using. Sound does not need words to communicate. Perhaps by seeing the visualization of the sound, visitors who never had before will learn to Listen.

Ideally, the interactive part of the exhibit would be to create a way for visitors to visualize their own voices. This would require a real time translation of sound into visual design.

I'm not entirely sure HOW I'm going to pull all of this together. At the moment, I'm just trying not to overwhelm myself, so I'm taking "Processing: A Programming Hangbook for Visual Designers and Artists" by Casey Reas and Ben Fry, a day at a time. Thus far, I've been reading about the components of sound and essentially trying to re-learn a lot of high school physics (amplitude, frequency, oscillators, etc.). I think it helps a bit that I have a strong musical background and a general understanding of musical vocabulary like pitch, octaves, modulation, transposition, etc.

As far as digital representation of sound and music, this is where it starts getting a little complicated for me, as it combines a lot of math and digital code (there is a reason why I study history). However, I'm getting the general idea of sound synthesis and processing. For example, from what I understand, sampling allows you to synthesize all kinds of frequencies just by processing a single sound. Then you can either sustain the sound, modulate it, or even create echo and reverberation effects.

I've just begun tapping into the world of Audio Analysis, or the derivation of information from digitized sound. This allows us to create interactive systems that "listen" to audio input and tell us what it means. The most obvious example of this type of technology is speech recognition.One thing I've found that will probably be useful is the idea of "attack detection", which is programming a computer to trigger an event at the onset of a sound, or when the sound reaches a certain frequency or amplitude. In other words, you could create a program that would synchronize a visual action with just the percussive sounds of a recording. Another concept that I'll probably find useful is "timbral analysis," which allows us to transform audio signals into data that can then be mapped to interactive computer events. One way to derive data from a signal is to count the number of times the signal turns from positive to negative - or even filtering out the loudest frequencies to test their harmonics or timbre. Of course, all that I've mentioned thus far are nothing but concepts in a book, so I only very vaguely understand them. If I can get my hands on the technology and actually see, hear, and feel how these concepts work, I think this could be something I might be good at.

As I'm reading along, I haven't really come across anything that is going to map out exactly what I have to do with this project. It's going to be a learning experience that I'll have to piece together myself - and perhaps just let my interests take me where they will. I'm really interested in how digital technology has changed the face of the music industry, for example, but I've never understood how sound engineering works. I'm learning now though, because of this project (which only illustrates further the beauty of this degree).

I suppose it's only natural that I lean toward sound for this project - my mother being both an audiologist and a musician on the side. Conversations of music theory and hearing aides were not uncommon around my dinner table. Though I can't say that I ever foresaw myself studying these things seriously...or the convergence of several of my passions in one class!