A spectrogram is a three-dimensional picture of sound – any sound. The three dimensions are time, frequency, and amplitude. Spectrograms usually look abstract, like successions of clumsy paint strokes or stills from Tron. They’re useful for sound engineers, but not all that good to look at. However, some software can also conduct spectral analysis in reverse, translating images into sound. In this case, the images are clear and the audio typically abstract.

The sound clip above, for example, was created from the picture(s) of me, moving from left to right and reading discrete clusters of color and texture as frequencies. The line of my thumb moves diagonally upward from finger to nail, causing its attendant sound to rise in pitch.

Photosounder.com, home of Photosounder, has a great demo video of how their program works with various images:

This kind of graphic sound creation has actually been around for several years (though newer programs are increasingly sophisticated), and musicians and sound artists have toyed with it extensively as a way to create strange noises that don’t otherwise exist.

I bring it up today not only because it’s fun, but because of the rhetoric of truth and discovery it inspires. People encountering this software for the first time quickly figure out that it can be used to encode information. As an experiment, you could take a screen shot of an email, turn it into “abstract” spectrographic sound, and then send it as an .MP3 to a friend. If your friend knew how you’d created the sound, she could then translate it back into an image, and read your original message. Likewise, you could embed translations of any image or text in a song or movie soundtrack as an ambient layer. The mind races at the probability that this has already been done many times, as a prank, an easter egg, a subtle political gesture, or even a means of legitimately covert communication.

Says one blogger about Photosounder:

I can see people … maybe hid[ing] messages in photographs or art work.

The hermeneutic exercise can, of course, encompass not only human communication, but universal structures as well. Several posters on Photosounder’s Youtube page imagine the applications of spectral analysis for science or pseudo-science:

The potential in this concept is far more than many people realize.

You could, for example, create hidden coded messages insides pictures/fractals, and/or DECODE incredible secrets of the Universe with very little modification. ;)

It would be interesting to hear whether electronic voice phenomena (EVP) could be detected in these complex sounds.

These posters understand implicitly that visualization is the standard best mode of reading data, that sound as we hear it fares poorly against image in revealing patterns and broad trends. Thus, they assume that phenomena in the world (say, ghosts) may exist under our noses, present but undetectable until we invent a light that can shine on them.

For both the spiritual and science-minded, this suggests that natural sound could be worth divining spectrographically in search of patterns we haven’t been able to pick up with our ears. Sonic images that appear orderly invite claims of design, intelligent or incidental.

A recent photoessay of whale and dolphin sounds, rendered with a program similar to Photosounder and published in the London Telegraph, is a great example:

These images may look like just pretty patterns, but they are visual representations of songs sung by whales and dolphins

The sounds were recorded by American engineer Mark Fischer and transformed into visuals using a mathematical tool called wavelets

Mark used to work on US Navy sonar and software for defence and aerospace companies but he now records the underwater conversations between whales and dolphins and transforms the waves into art

Mark then uses a branch of maths called wavelets which creates these intricate structures

"To look at a spectrogram you will see a simple, boring blur with a few harmonics," he said....

"With wavelets, however, there was an image that displayed extraordinary structure. Something was going on with this sound, even if we are not quite sure what"

The effect is even more apparent when colour is applied and the graph transformed from rectangular to polar coordinates, forming a circular graph

Atlantic spotted dolphin, wavelet graph

Recent research shows that humpback whale songs differ in local dialects and contain complex grammatical rules, showing a higher level of communication than first thought

The risk of this approach, of course, is in romantically imagining that the software has no bearing on the data represented. The complex beauty of the pictures may lead us to forget that the computational processes used to render them were designed by people who probably share many of our own standards of beauty – formal symmetry, clear coloration, sharp lines, etc. Faced with images like these, we’re inclined to imagine that god, or evolution, or some other force, created a perfectly-patterned world, one that can ultimately be “read” and understood. But no matter who or what is in charge, that isn’t the case.

Get Adobe Flash player