In my everyday life, I am used to externalizing my thoughts and code my meanings through writing. The challenge this week was to use the affordances of a different mode – images, to code a message that I would normally disseminate through the mode of writing (Kress, 2005).

As Bolter (2001) claims, the prose and print of today are attempting to remediate the use of images, by readjusting the ratio between text and image. This week, we were challenged to push this idea to the extreme, and code a full message without using a single printed word, moving from “pure text to pure pictorialism” (p. 52). 

We played with the limited choice of emojis on hand to arrange them in space in the way that we would normally arrange words. Even though we were working with images, the affordances of depictions, such as size, placement, or colors were limited (Kress, 2005). An additional challenge was that being constrained to using emojis, we were not able to name the relations between the concepts, such as states, belonging, past or present tense, etc. Instead, we had to find a way to use the limited affordances of the medium – the text box – to position the images in space in order to represent those relationships for our audience. 

I chose to work with a baby book for this task. What makes young children’s literature unique is that images have a much more important role in it, compared to text. In fact, some storybooks that my daughter owns, are completely wordless, allowing for a certain degree of freedom of interpretation. Another great thing about infant books, just like our emoji stories, is that they can be understood by young readers around the world, regardless of the language they speak.

My strategy was to code the whole story using emojis to represent words, so that someone who has only heard or read it once, could use these visual notes to retell the story, as close to the original as possible. I felt like an inventor of the alphabet – I had to decide whether I was going to use emoji to represent a single phoneme, a morpheme, or the whole word (Schmandt-Besserat, 2009). Perhaps, if my goal was to create a long-term solution, I would have taken the length to create an alphabet (e.g. using an image of an apple for the letter A, and a bear for B, etc.). However, I wanted to stay away from the habitual mode of representation and use the affordances of the emojis as they were designed – to represent a concept or a whole word.

To conclude, this task made me wonder if picture writing is indeed going to replace text, and how that will affect our thinking and our behaviors if it does. Looking forward to learning more on remediation of reading and the effect it has on our thinking in the next few weeks. 

References:

Bolter, J. D. (2001). The Breakout of the Visual. In Writing space: Computers, hypertext, and the remediation of print (2nd ed.). Mahwah, N.J: Lawrence Erlbaum Associates. doi:10.4324/9781410600110

Kress (2005), Gains and losses: New forms of texts, knowledge, and learning. Computers and Composition, Vol. 2(1), 5-22.

Schmandt-Besserat, D. (2009). “Origins and Forms of Writing.” In Bazerman, C. (Ed.). Handbook of research on writing: History, society, school, individual, text. New York, NY: Routledge.

The Process

Being very new to hypertext writing and HTML, I initially tried to create my game entirely on paper. However, as the complexity of my story was increasing, the limited affordances of a pencil and eraser quickly became apparent. Thus, I was forced to switch to the computer, which became my ‘clerk’ helping me to externalize my thoughts in “symbolized concepts” (Englebart, 1963, p. 6).  Not only I was able to focus on little steps without having to remember every iteration I already had written in my game, but due to the permanence of the computer text, I was also able to test-run, revise, and edit the game as many times as I wanted. It is hard to argue with Englebart (1963) that such use of technology increased the level of the information I could handle, shortened the time, and thus, turned me into an “augmented” story designer.

I wrote a game about myself and the choices that I make every day, representing these choices through hyperlinks. Twine makes these choices visible to myself, the author, and you, my reader, who is invited to take a journey into that space (Bolter, 2001).

The Game

Readings, assignments, deadlines – they are a part of every graduate student’s life. Many struggle with balancing school, career, family, and self-care.  

What choices would you make if you were a graduate student? Try the game to understand what choices and decisions students face. There are several options, but only one path leads to the successful completion of the game.

Click the link below to download the game on your computer. Extract the HTML file and run it in your browser. Good luck!

The Checklist

References

Bolter, J. D. (2001). Hypertext and the Remediation of Print. In Writing space: Computers, hypertext, and the remediation of print. Routledge.

Englebart, D. (1963). A conceptual framework for the augmentation of man’s intellect. In Hawerton, P.W. and Weeks, D.C. (Eds.), Vistas in information handling, Volume I: The augmentation of man’s intellect by machine. Washington, DC: Spartan Books.

Prior to starting the assignment, I spent a considerable amount of time thinking of a word I was about to bring to material life (Haas, 2013). Besides the time and effort I was going to invest in the process, I also wanted to select a special and meaningful word to share with the world. In the end, I chose to carve the name of the city where I spent my youth and got educated – Minsk. (Ironically, my home country Belarus is often humorously referred to as ‘Potato Land’).

When I think of studying Belarusian history and culture at school, I immediately reminded of Francysk Skaryna, the first book printer in Eastern Europe. His contribution to our culture was so important that a busy street in the capital city, Minsk, is named after him. When the new library building was finished a few years ago, a statue of Francysk Skaryna was placed in front of it.

It took me just over an hour to create two stamps – 40 minutes for the first one, and 20 for the second one. Even though the second stamp took half the time to make, if my goal was to print a leaflet or a poster, the process would have probably taken me a few days.

Most of the letters in my word consist of straight lines intersecting at different angles. The letter shapes in the English alphabet go back to the original alphabet invented in Lebanon in 1700 BCE (Schmandt-Besserat, 2009). While carving the letters with my craft knife, I visualized our ancestors incising the first letters on tablets, and appreciated the simplicity of the shapes.

While ‘potato press’ printing was calming and the resulting letters look unique and beautiful, the handcrafting process did take a lot of time. The temporal investment, coupled with eye fatigue and the inconsistent results makes me feel grateful and relieved that we live in a world where handwriting is remediated by print technologies (Bolter, 2001) and access to education is not limited to a wealthy few.

P.S. A video clip of my daughter Alexandra trying to pronounce the word ‘Minsk’ for the first time. Soon she will start to understand that the signs she sees on paper match the sounds of language, and the words these sounds represent carry semantic meaning. For now, though, she is only starting to develop her orality, and her thinking is not yet affected by writing (Haas, 2013).

References

Bolter, Jay David. (2001). Writing space: Computers, hypertext, and the remediation of print [2^nd edition]. Mahwah, NJ: Lawrence Erlbaum.

Haas, C. (2013). “The Technology Question.” In Writing technology: Studies on the materiality of literacyLinks to an external site.. Routledge. (pp. 3-23)

Schmandt-Besserat, D. (2009). “Origins and Forms of Writing.” In Bazerman, C. (Ed.). Handbook of research on writing: History, society, school, individual, text.Links to an external site. New York, NY: Routledge.

For this task, I used the ‘Dictation’ tool preinstalled on my Windows 10 laptop’. I talked, unscripted, using my computer microphone, about a research article I recently read. Here is the screenshot of the resulting script, followed by my analysis. You can also access the script through this link. 

Deviations from conventions of written English

A quick glance over the text was enough to say that there are some rules and standards of written text that this script does not follow. The biggest challenge for comprehending the text is that the ideas are not organized into sentences and paragraphs. Informal vocabulary (e.g. I’m gonna) and filler words (e.g. Umm, like) are other examples of ‘inadequate’ writing.

What is “wrong” in the text? What is “right”?

Writing is a communicative activity (Bauman & Sherzer, 1989), and in my view, this text is not adequately performing the communicative function due to the multiple mistakes and deviations from the norms of written language.

Among the features of the text that are “right” are the grouping of symbols into words divided by single spaces and representing the text in rows from left to right. 

Most common “mistakes” 

The two most common mistakes that made the whole script very hard to decipher are punctuation and spelling. 

The lack of punctuation and not knowing where one idea ends and the next one begins makes the text nearly impossible to understand, even for myself.

Spelling

Being an ESL speaker, I speak with an accent. English vowels are the most difficult for me, as the differences in their pronunciation can be very subtle. When analyzing the script, I found a few instances when my wrong pronunciation (that I was unaware of ) was displayed on the computer screen:

However, my accent aside, I found more instances when the automated script was “simply a record of uttered sounds” (Gnanadesikan, 2011, p. 10), and not a correct encoding of my speech. For example, when I paused, repeated the same word twice, or used a filler, the software encoded the word(s) as something else. For example:

Some words and word linking were written phonetically:

At some points, this phonetic encoding was impossible to decipher even for me, the speaker:

• carduus (times 2!), eserver, peterlee, anumati, berthaut.

A third, less common error, which does not impede understanding to the same extent as spelling and punctuation, is capitalization.

The software only recognized and capitalized 

• a few common proper names, such as Suzanne, Canada, US;
• the personal pronoun ‘I’

Other proper names and acronyms were transcribed in lower-case letters. 

Lastly, there were also a few words that got both misspelled and erroneously capitalized:

• Cairns, Cyril

If I had “scripted” the story…

If I had a chance to script this story prior to dictating it to my computer, I would have had time to organize my ideas using a correct chronological progression, and adding headlines, signposts, and punctuation. When dictating my script to the software, I would have pronounced the symbols as words” – e.g. ‘comma’, ‘period’, to ensure better accuracy of encoding my oral text (Gnanadesikan, 2011). Of course, I would have had time to make sure the grammar and vocabulary were more appropriate as well. The script would also have helped me to pronounce words loudly and clearly and avoid using long pauses and fillers, which would result in a much more readable text. 

On the other hand, I expect some errors made by the software to stay the same – such as mistakes in capitalization and spelling.   

Oral vs.  written storytelling 

One big difference between oral and written storytelling is that we usually know quite well who our audience is when we talk. In writing, however, our audience can be unknown to us. Furthermore, while oral storytelling disappears as we speak unless we record ourselves, a written story is much more permanent and is able to reach its audience across space and time (Gnanadesikan, 2011).

Another difference is that oral storytelling is not strictly sequential. It is quite common for the speaker to go back in time when information important for the story has been left out. We also repeat words or parts of the sentence, correct ourselves, and use fillers while searching for a suitable word, remembering what happened next, or returning to the story after getting distracted for a moment. 

Next, in oral storytelling there are a variety of ways to help the listener follow our ideas as we speak: we use pauses, changes of tone, intonation, and the voice, to emphasize certain information or signal to the listener that we are changing the subject or going back in time. In written storytelling, these technologies are much more limited.

In oral storytelling, just as the speaker can help the audience make sense of the story, the listeners would be more forgiving of his/her pronunciation mistakes. Where software erroneously wrote some of the words I mispronounced as something else, a live audience would be able to understand me much better in context. There is also a possibility of a dialogue and a chance to ask questions or clarify information in live communication. On the other hand, in oral communication, one cannot delete or edit a thought in the way that writing affords it.   

Last but not least, we can be emotional as we speak, while a reader has to read between the lines to decipher our feelings from the vocabulary choices, word order, and other writing techniques. 

Final thoughts

This writing assignment was material because it required using material technologies – the computer, microphone, pixels, and others that I am unaware of (Haas, 2013). It also resulted in the creation of a visual artifact. I am wondering whether being aware of the fact that my discourse was being moved from an aural to a visual realm (Ong, as cited in Haas, 2013) to be later analyzed by myself, my professor and peers, altered the way I talked. 

References

Bauman, R., & Sherzer, J. (Eds.). (1989). Explorations in the Ethnography of Speaking (2nd ed., Studies in the Social and Cultural Foundations of Language). Cambridge: Cambridge University Press. doi:10.1017/CBO9780511611810

Gnanadesikan, A. E. (2011).“The First IT Revolution.” In The writing revolution: Cuneiform to the internet. (Vol. 25). John Wiley & Sons (pp. 1-10).

 Haas, C. (2013). “The Technology Question.” In Writing technology: Studies on the materiality of literacy. Routledge. (pp. 3-23).

In her fascinating presentation, Dr. Lera Borditsky talks about how the language(s) we speak shape(s) our thinking patterns, the way we view the world, remember things and identify ourselves. The CLAS annotation activity was rich with discussion and thought-provoking questions, and I hope my contributions added some value to the collective learning experience this week.

As I return from a walk with my fifteen-month-old daughter, put her down for a nap and unload the contents of the stroller caddy (my ‘purse’ since she was born) to her small play table, I am feeling skeptical about being able to write something about myself based on what I find there. Especially, something interesting, unique, and related to text technologies. However, as I watch the table filling up with things, ideas start coming and I suddenly feel more confident. Challenge accepted!

A few of the items are there for my daughter Alexandra – her sippy cup, snack jar, and tissues. These items, just like the bag itself, represent the new literacy I am just starting to develop – parenting. 

The next couple of things – the hand sanitizer and the mask, will probably be found in everyone’s picture. This is where we are all connected nowadays. 

The key lanyard contains a hint about my workplace. I work with newcomers to Canada, teaching them English and settlement skills. Next, by taking a closer look at the keys themselves, you can tell that I live in an apartment, ride a bike, and take occasional yoga classes at Yyoga studio. Yoga is the literacy that is a big part of my life now, but that was not always the case. After coming to Canada eight years ago, I was introduced to it by a colleague and a friend. Slowly and gradually I developed my understanding of the body and mind, and how they are interconnected. While I am not always able to find time to practice, I always feel much better when I do.

My phone is perhaps the most obvious “text” technology in the bag. It is my notebook, shopping list, news portal, and so much more. Not less important are the wireless earbuds – always with me, always fully charged. They are perfect for going for a run, which I am trying to get back to after a long break. I also need them on my walks, as this is the only time I have to call or connect with my friends living in other countries. With the time difference and everyone’s busy schedules, I choose to connect with them by exchanging asynchronous voice messages. Messenger apps such as WhatsApp and Telegram make it super easy to do – once you swipe the mic up, it starts recording, no need to keep holding the thumb on the button, which means I can keep pushing my daughter’s stroller as I speak. The bottom line is, talking has become so much more convenient than texting for me, because of its immediacy, ease of use, and not having to worry about my emotions being interpreted right, as it is often the case with written messages.

To continue on the subject of ‘text technologies’, I want to add that to me, ‘text’ is getting more and more oral as I choose to listen to audiobooks when I walk outside or drive, instead of reading printed books as I used to do.  Even course materials, which at the beginning of my MET journey I used to print out and read on paper, are much easier digested now when I hear them first using @Voice app. 

Moving on to the chocolate wrappers which are the evidence of my lifelong love of chocolate. One of the wrappers is from a Peruvian chocolate candy shared by a friend a few days ago. The other one, whose name is translated from Russian as a ‘Bird’s Milk’, would be familiar to anyone who grew up in the former USSR. This wrapper is giving away my ethnic background (Eastern European) and polylingualism. 

There was also an old receipt in the bag that I hadn’t thrown away. The receipt has a tracking number for an online return. While I do my best to be a mindful shopper and support local businesses, online shopping is still a part of my reality, especially nowadays when it is safer to stay at home.

Taking a final look at the photo, I also just realized I had forgotten my wallet on that walk, which also represents my current lifestyle quite well. For someone who likes to plan and be organized, parenting can become a powerful lesson in being flexible, resourceful, and not getting upset or surprised when things don’t go as planned. You also learn to rely on people – I have forgotten my wallet twice before, which I only realized when I was trying to pay for my coffee or groceries. I was saved each time by a friend who paid for my purchases. So yes, one more thing about me is I have some great people in my life.