Or How I Learned to Stop Worrying and Love the Glitch

This week I attempted to recreate the results of glitching files as demonstrated in this blog post by Trevor Owens. As we shall see, I ran into a few difficulties in reproducing this experiment exactly. But first what is a glitch? According to Wikipedia, “A computer glitch is the failure of a system, usually containing a computing device, to complete its functions or to perform them properly.” In this post, I chronicle my attempts to create glitches by using files in ways other than their intended purpose to reveal what we can learn about the formats themselves.

A Textual Audio Experience

I started trying to view an .mp3 as a .txt file. I could not use the same audio file as in the original blog post because the Library of Congress does not provide direct download any longer, having switched to streaming-only for access. Instead, I randomly selected an .mp3 of the Rush classic Tom Sawyer. From here I changed the file extension to a .txt file and opened it with my computer’s text editor. Here is the result:

rush
A real toe tapper

Just as with the audio file Owens used, much of the information in the .mp3 is a confused mess and the result of the text editor’s attempt at interpreting the bits as alphanumeric sequences. However, along the top there is some embedded metadata such as information on the writers of the song: Alex Lifeson, Geddy Lee, Neil Pert, and Pye Dubois. These bits are meant to be read as text and therefore can be read by the program.

Where the Trouble Began

In the next step, I tried to view an .mp3 and .wav file as .raw images. Because I did not use the same audio file as the original blog post, I did not have a .wav file to accompany my .mp3 when trying to replicate this part. Rather than simply changing the extension on my Tom Sawyer .mp3 I used a media encoder and converted the file to a .wav file. From here, I changed the extension on each to .raw and attempted to view them in an image editor. Unfortunately these files would not open in any of my image editing software. Borrowing a computer that had Photoshop, I was able to view the results seen below:

01 Tom Sawyer wav and mp3 raw photoshop
On the left: .mp3 as .raw, on the right: .wav as .raw

Just as above, an image editor can do no better than a text editor when attempting to read the audio files in a visual manner. Unlike Owens’s results, my two images look largely the same. The .wav as .raw did produce a large black bar at the top of image, which I am assuming is due to the difference in original format. I thought the similarity might have been because I converted my .mp3 into a .wav, so I downloaded a different .wav audio file directly from the web and repeated the steps and yet it still yielded the same results.

Complete Failure

While I was able to replicate most of the outcomes in the preceding section, I failed at the next step of editing an image with a text editor. The link Owens listed for the image in his post was broken, but luckily the original image was also available in the post. I downloaded this image and changed the extension from .jpg to .txt. I opened the file in the text editor, deleted some of the text, and changed it back it into a .jpg. Unfortunately, the file would not open in any of the image software I tried, including Photoshop. I kept receiving error messages that the file was unsupported, corrupted, etc. I tried these steps again but with copying and pasting parts of the text back into itself or even deleting only one character. I even attempted using a different image entirely and doing all the same steps again. Alas, all my attempts failed to produce a glitched image that could open.

Tom Hanks typing

Conclusions

While I was not able to reproduce all the tasks that Owens accomplished in his blog post, I was still able to see his main point that screen essentialism masks that digital objects are more than what they appear to be on the screen. The different ways the files can be read demonstrates the different structures of the formats, even if they look the same on the screen. My failure in this process has made me realize how much the public is pushed to a limited understanding and shielded by the programs that are meant to read certain files in certain ways. Perhaps my failures are just a result of well working computer software that allows you only to produce the desired outcome of these files. I encourage everyone to try glitching some files. Can you do it?

UPDATE: I was able to fix the problems I mentioned in this post. Here are the results:

Desert (2)
Glitched Image
doh
Correct comparison, .mp3 as .raw on left, .wav as .raw on right

To see how my issues were fixed, see the comments section below. Many thanks to Nick Krabbenhoeft for helping me fix the problems.

7 Replies to “Or How I Learned to Stop Worrying and Love the Glitch”

  1. If you want to recreate the audio as image differences that Trevor has, you need to start with a WAV and convert it to MP3. WAV is an uncompressed audio format. It represents the sound waves of a song as accurately as possible. On the other hand, MP3 is a compressed, lossy format. It represents the sound waves of a song accurately enough for the human ear, but uses mathematical compression to save space at the expense of true accuracy.

    As an example, row your boat:
    – as WAV: row, row, row your boat, gently down stream
    – as MP3: (row, )*3 yr boat, gntly dwn th stream
    The MP3 version takes up less space and is recognizable, but it is forever missing the full information of the WAV. So if you convert the MP3 to WAV, you don’t recover any of that information. The reason that you see the smoother black-gray-white gradients on the WAV image is that it is representing the smoothing curves of true audio wave. The MP3 has chunked that audio wave into discrete yes-no pieces, and its image is of a bunch of black and white chunks.

    On the JPEG experiment, what chunks of the file were you removing? Some bits are more important that other, especially in compressed bitstreams because the decoding of any one bit depends on the information in other bits. Losing bits at the very beginning that describe the encoding parameters means that none of the other bits can be decoded.

    1. Ah ok, that makes sense now. Converting from .wav, I was able to do the the .wav/.mp3 comparison. Thanks for the help!

      I’m unsure of what the issue is with the jpeg part. I repeated the process over and over (returning to an unedited file each time), randomly deleting or duplicating sections of different length throughout the file. Many times the chunks were nowhere near the very beginning.

  2. What OS and editor are you using for the jpeg experiment? It might be reformatting the encoding without telling you, causing the error. For example, Microsoft Word, Wordpad, and TextEdit all do weird things to txt files. Notepad is fine, and as are SublimeText and Notepad++.

    And here’s a great episode of 99% Invisible that speaks to some of the issues of prioritizing ease-of-use over understanding how computers work. http://99percentinvisible.org/episode/of-mice-and-men/

    1. Yea I had been using Notepad. Downloaded SublimeText and had no trouble glitching the image file. Thanks again!

      Also, great article/episode. I think it also speaks a lot about path dependency and corporations’ knowledge that it’s easiest to sell consumers products that are at least somewhat familiar, especially when it comes to technology. Which can link back to how things are displayed on the screen, with items (PDF Documents, eBooks, etc.) mirroring how their tangible counter parts would look in real life.

  3. So, I was thinking about the metadata embedded in the .mp3. Since it is there and shows up as text in the text editor, is it possible to glitch the song in the same way that the .jpgs can be? I look at the mesa image with its glitched colors and wondering what would happen if you edited the text version the same way as the .jpgs; working from what Nick Krabbenhoeft said about the .jpgs, starting the glitch from the back because the front is where the instructions about how to read it are located, could you change the way the song sounds? Could you glitch it as a .txt and then turn it back into an .mp3 and still have it work?

    1. Most likely yes. It’s worth doing an experiment.

      Beyond corrupting a data stream, you can also start hiding other data inside of an existing stream. Most famously, there are image in spectrograms of Aphex Twins’s songs. For example, https://www.youtube.com/watch?v=M9xMuPWAZW8&t=278

      From there you can dive into the world of steganography, https://en.wikipedia.org/wiki/Steganography

      And even placing gifs inside of mp3s. https://thewebmonkey.wordpress.com/2009/12/29/hide-mp3-incopy-picture-gif-b-audio-mp3-b-combined-gif-an-image/

Leave a Reply

Your email address will not be published.