Tag: sonification

Posted on

Task 7: Mode-Bending

Sonifying Task 1: What’s In Your Bag?

For this task I tried something very different (different to me, anyway): sonification.

Keeping Task 7 in mind as I read the New London Group’s A Pedagogy of Multiliteracies: Designing Social Futures, the following passage caught my attention: “The Redesigned is founded on historically and culturally received patterns of meaning. At the same time, it is the unique product of human agency: a transformed meaning” (New London Group, 1996, p. 76; emphasis added). I wanted to transform the meaning of Task 1, but I was also interested in retaining the original text and story within Task 1 as well.  How could I do both?  I decided to use the New London Group’s idea of metalanguage in order to describe Task 1 “….in various realms.  These include the textual and the visual, as well as the multimodal relations between the different meaning-making processes that are now so critical in media texts and the tests of electronic media” (New London Group, 1996, p. 77).

Hello Google!

I started Googling various combinations of: “text”, “analysis”, “tools”, “visualization”, “open source”, “music”, and “audio” (among others).   The first tool I came across that looked promising, was Voyant Tools (https://voyant-tools.org/).  It’s pretty interesting and I highly recommend it if you’d like to visualize your text.

Voyant Tools

I copied and pasted the text component of Task 1 into Voyant Tools to see what would happen.  It turns out, there is quite a lot of information to be gained by visualizing one’s text.  It was pretty neat: the interactive chart (below) is from Voyant Tools. I noticed that the original text was cut into 10 equal-sized pieces with the most frequent terms from Task 1 plotted, showing the trend in discourse from the start to the end of the original blog post.

This was an interesting way to visual my text, but it had nothing to do with sound.  Though I learned a lot about the text I’d written weeks ago, I still hadn’t integrated sound…yet.

Sonification

Enter:  sonification.  Again, Google was my go-to tool as I tried to figure out how to turn my data into sound or music or something audio-based.  That’s where I came across “sonification”.  I’d heard of the term before, but never really explored the concept.  My Google search for sonification yielded two interesting results: TwoTone Data Sonification (https://twotone.io/, a free web-based app that turns your data into sound/music) and programminghistorian.org (more on this later).

TwoTone

I exported the data from Voyant Tools and placed it into an Excel sheet. The exported data was divided into the same ten pieces shown in the graph/chart (above) indicating the trends of the words I’d used most frequently in the description of my image in Task 1.  I then pasted the Excel data into TwoTone to see what would happen: the results were really cool!  The text I’d written describing the photo of my bag had been sonified:  I was now listening to my text!  Check it out below:

Reflection

In preparing for this task, I was hoping to not only produce something that satisfied the audio requirement, but I also wanted to move beyond Audio and Linguistic Designs toward a more multimodal process (New London Group, 1996).
The first step in the process toward sonifying my text was to visualize the text (Visual Design).  Using Voyant Tools, I was able to see my text as a word cloud (not included in this post), as a series of knots (which can also be used to sonify the text), and the graph I included above.  Using the graph, I could see the five words I repeated most often in the written piece: bag, bags, items, kids, and text.  I was also able to use the graph to view the trends of my word-use throughout Task 1 and view each word spatially (Spatial Design) in comparison to other words: the graph indicated where, in Task 1, I focused my attention on a particular word, and when that focus drifted toward another word (compare “kids” earlier in the written piece versus “bag” closer to the end).

When I transferred my data to TwoTone, I could also see the different “voices” representing the five most common words repeated in Task 1; as the music plays, different columns light up across the screen.  (The image below displays the progress of the words “bag” and “items” during the 10 second sonification: note the yellow/orange coloured blocks indicating the notes being played).

An image showing the progression of text that has been sonified
The lighter ‘columns’ or blocks represent the notes being played; darker (blue/green) blocks indicate notes/words that are not being played

When I listen to my sonification, I can hear the repetition of certain notes that are pleasing to the ear.  This repetition of sound suggests that in my writing, I return to common themes, thoughts and ideas (or that I’m quite repetitive…).  The rhythmic nature of the notes suggests an interconnectedness between the ten pieces of the chart I created in Voyant Tools (above).  The rhythm also allows the listener to hear how different words ebb and flow as the story progresses.

….teachers need to develop ways in which the students can demonstrate how they can design and carry out, in a reflective manner, new practices embedded in their own goals and values.  They should be able to show that they can implement understandings acquired through Overt Instruction and Critical Framing in practices that help them simultaneously to apply and revise what they have learned.  (New London Group, 1996, p. 87)

This task encourages students to take what we produced in Task 1 and, through reflecting on course material, stretch our learning by applying new knowledge to an “old” task in order to create something completely new.  Sounds pretty transformative to me!

NOTE:  The Programming Historian

I found this resource when I was trying to figure out how to analyze my text and convert it to an audio format.  Though it’s beyond the scope of this course (and I don’t have time to read it all), I found Graham’s description of sonification quite helpful.

References

Graham, S. (2016). The Sound of Data (a gentle introduction to sonification for historians). Programming Historian. https://programminghistorian.org/en/lessons/sonification

Sonification. (2020). In Wikipedia. https://en.wikipedia.org/w/index.php?title=Sonification&oldid=954085563

The New London Group.  (1996). A pedagogy of multiliteracies: Designing social futures. (Links to an external site.)  Harvard Educational Review 66(1), 60-92.

Tools

TwoTone Data Sonification:  https://twotone.io/
Posted on

Final Project

It’s finally here!  I’ve managed to finish my very first podcast.  Let’s just say I’d call it a work in progress.  In keeping with my self-appointed goal for this course, I tried another new-to-me tool to create this podcast:  Audacity.  It proved to be fairly intuitive and a lot of fun to use.  The sound within the different components of my recording was a little inconsistent, but I attribute that to the fact that I’ve never actually created a podcast before so I’m still learning how to set up the microphone and use the gain and volume settings on it as well.

Podcasting:  The Process

As expected, this podcast required quite a lot of prior planning.  I first needed to sketch out my idea and determine whether it was doable within the scope of a 10-minute podcast.  When those ideas were well on the way, I created a script for most of the podcast, but left my conversation with my daughter free-flowing and unscripted.  I found another app (TwoTone) to create an introduction and some music to start the podcast, but the rest was was completed on Audacity.

Generating Ideas – 7.2:  Digital Literacy

As I reread the course notes and some of the assigned readings, the paragraph below from from section 7.2: Digital Literacy from within our course readings initially sparked my idea for the final project.  (I believe I might have erroneously said “7.1” within the podcast; it should be 7.2):

Unlike critics such as Tapscott and Prensky, Dobson and Willinsky do not define digital literacy as a revolution in literacy practices resulting in rifts between those “in the know” and those left behind. Rather, they see the onset of digital literacy as an evolution, taking place—with both continuities and extensions of traditional print-literacy practices—more in line with Bolter’s idea of remediation, or McLuhan’s idea that the content of any new medium is an older medium. Digital literacy is not positioned as standing in opposition to print; it is part of a continuum in technologies for knowledge diffusion across which we can observe signs of the remaking of old relationships under new conditions of digital and global, networked cultures.

With a rough idea in place, I had to make sure my ideas were situated within literature that related to this course.  This part of the task was fantastic!  Because the task was so open-ended, I felt I had more freedom not only to explore literature that related to what we’ve learned within the course, but I was also able to look at ideas and literature that pushed the boundaries of what I’d learned thus far in ETEC 540.  What’s interesting to note though, is that in examining current technologies and how the meaning of a message can be conveyed and perceived differently based on media selection, it was only through digging into the past, that my project idea finally began to take shape.

Hinted at in our course notes above, McLuhan and his “the medium is the message”  supposition from Chapter 1 in Understanding Media: The extensions of Man contributed to the starting point for my project.  In addition though, throughout the course, we’ve looked at the evolution of text technologies and how humans have responded accordingly to these changes and advances in technologies.  I was particularly interested in how the message we are trying to convey might change depending on the type of technology we choose to use to communicate the message in the first place.  This idea lead me to the “Father of Information Theory”, Claude Shannon and his 1948 paper, A Mathematical Theory of Communication.  Shannon was particularly interested in how the meaning of a message can be lost or misconstrued when “noise” interrupts a message’s signal as it is transmitted from its source to its receiver.  This is *exactly* what I was interested in, but on a much smaller scale.

Putting it Together

I decided I would examine the use of the following modes of communication to determine how the meaning of a message is affected by the use of a particular medium:  oral-based instructions, text, cursive writing, emoji, and sonification.  I used the image below (from the 2019 film, The Bit Player, documenting Claude Shannon’s life and the modern-day effects of his Information Theory) to guide me through my project. My idea was to take information (a message or instructions) and communicate it to my daughter, Carys.  However, I wanted to see how the noise (my daughter’s brain) interferes with the message depending on how I chose to communicate the message.

Image from The Bit Player film depicting Claude Shannon's information theory
Image taken from The Bit Player (available on Amazon Prime). Image courtesy of Levinson, 2019

Disability

My daughter, Carys, has a learning disability.  She has difficulty processing text-based tasks (whether that’s putting her ideas down on paper, or decoding words and processing text-based tasks).  Through examining different modes of communication with my daughter who has known difficulties with processing text tasks, I was hoping to determine whether meaning can become misconstrued if the communication mode wasn’t a good ‘fit’.

The Podcast

Here it is: the final product and my final project.  Thanks for listening!  References are below the podcast.

References

Bolter, J. D. (2001). Writing space: Computers, hypertext, and the remediation of print (2nd ed.). Mahwah, N.J: Lawrence Erlbaum Associates. doi:10.4324/9781410600110
Collins, G. P. (2002). Claude E. Shannon: Founder of Information Theory. Scientific American. https://www.scientificamerican.com/article/claude-e-shannon-founder/
Horgan, J.  (2016 ). Claude Shannon: Tinkerer, Prankster, and Father of Information Theory – IEEE Spectrum. IEEE Spectrum: Technology, Engineering, and Science News. Retrieved August 10, 2020, from https://spectrum.ieee.org/tech-history/cyberspace/claude-shannon-tinkerer-prankster-and-father-of-information-theory
Levinson, M. (2019). The Bit Player. Prime Video. https://www.primevideo.com/detail/The-Bit-Player/0Q3B8F95AAOY3FEK1L8WINFASM
McLuhan, M. (2001 [1964]). Understanding Media: The extensions of man. Taylor & Francis Group. Retrieved from:  http://ebookcentral.proquest.com/lib/ubc/detail.action?docID=4643237
Shannon, C. E. (1948). A Mathematical Theory of Communication. M The Bell System Technical Journal, 27, 55.  Retrieved from: http://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf
Soni, J., & Goodman, R. (2017). Betty Shannon, Unsung Mathematical Genius—Scientific American Blog Network. Scientific America. https://blogs.scientificamerican.com/voices/betty-shannon-unsung-mathematical-genius/
The New London Group.  (1996). A pedagogy of multiliteracies: Designing social futures. (Links to an external site.)  Harvard Educational Review 66(1), 60-92.

Spam prevention powered by Akismet