This week’s task involved changing the semiotic mode of our week 1 task. I’ve opted to redesign my original task by creating an audio-driven trivia game using Twine. During the game, users are invited to guess the items in my bag based on the sounds that they make.
Twine Game: Play ‘What’s in My Work Bag’ on itch.io
Project File: Download ‘What’s in My Work Bag’ (*.zip)
Development
Audio Recording
In the process of changing the semiotic mode of my original task, I took it upon myself to record the sounds made by each of the items in my bag. To do so, I used the Voice Notes app on my iPhone to create the recordings. By positioning and manipulating each item close to my phone’s microphone, I attempted to capture each sound in a distinguishable way. I then brought each sound into Audacity to edit out any excess noise. The only items that I did not include from my original task #1 were the face mask and the laptop charger, as the recorded sounds were not distinguishable enough.
Twine Game
To tie each sound together and create a unified multimodal experience, I decided to produce a simple hypertext trivia game in Twine. This was an opportunity for me to learn and experiment with Twine for the first time, especially since I had more bandwidth this week to do so. Along the way, I found myself relying on my knowledge of HTML and CSS, the Twine user-guide, and findings that my peers made with their own Twine objects. For example, Elaine Lee’s (2022) task #5 post references Fry’s (2017) video on hosting and embedding images in Twine, which I also drew upon to add some visuals from the Noun Project.
To integrate my sound effects into Twine, I uploaded each sound to Google Drive and then pasted a public embed link for each sound.
In order to add the score counter, I referred to a discussion post by Chapel (2017) which introduced me to the $score variable. While this was originally written for Twine Sugarcube, I was able to modify Chapel’s (2017) approach for Harlowe with the help of Twine’s debug tool.
Reflection
In my original week #1 task, I shared the contents of my bag using a combination of modes including visual (photo), linguistic (text), and spatial (content arrangement/page layout). While approaching this task, I was reminded of the New London Group’s (1996) “designs of meaning” (p. 73) framework, which speaks to the ever-changing and contextually-driven relationship between “available designs, designing, and the redesigned” (p. 74). I can see now how my use of visual, linguistic and spatial modes in task #1 was drawing on my existing familiarity and knowledge of “available designs” (NLG, 1996, p. 74) for sharing information in a blog environment.
As such, the “designing” (NLG, 1996, p. 73) process challenged me to reimagine the representation of task #1 by modifying the mode of communication. With my “redesigned” (NLG, 1996, p. 73) task, I relied more on aural modes (rather than solely linguistic and visual) to do the ‘heavy lifting’ of meaning-making. Rather than describing the object using visuals and text, the user must interpret the sound of each object to guess what it is.
The use of hypertext allowed me to modify the arrangement and presentation of audio clips in a more participatory way. While my redesigned task is more of a proof-of-concept, I think it successfully demonstrates how users can express their existing knowledge in a new way (i.e., drawing on knowledge of different sounds and the trivia game rules and format) when compared to the more passive nature of the original task.
Thinking through this mode-changing exercise and its broader connection to digital literacy, I’m reminded of Dobson & Willinsky (2009) suggestion that digital literacy has the potential to “[further] educational and democratic as well as creative and literary ends” (p. 303). In particular, I can see how the practice of mode-changing can work towards advancing 21st-century skills and student-centred learning by providing exposure to modes of representation and expression that go beyond written text. After completing this exercise, I feel more confident with approaching a multimodal redesign of my own instructional content in a way that would support digital literacy.
References
Chapel (2017, June 29). Keeping track of points [Online forum post]. Twine Forum. https://twinery.org/forum/discussion/comment/23447/#Comment_23447
Dobson, T., & Willinsky, J. (2009). Digital literacy. In D. R. Olson & N. Torrance (Eds.), The Cambridge handbook of literacy (pp. 286-312). Cambridge University Press. https://go.exlibris.link/7G5xqPXN
Fry, J. (2017, March 27). Twine – Adding Pictures [Video]. Youtube. https://www.youtube.com/watch?v=DRETz9wnOkk
Lee, E. (2022, February 13). Task 5: Twine story. ETEC 540 Reflections. https://blogs.ubc.ca/etec540elainelee/2023/02/13/115/
The New London Group. (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review 66(1), 60-92. http://newarcproject.pbworks.com/f/Pedagogy%2Bof%2BMultiliteracies_New%2BLondon%2BGroup.pdf