For my final project I have researched Speech-to-Text (STT) technology and the implications it is having on culture, literacy, and education. I chose this technology because it provides an opportunity to reflect on many of the concepts covered in this course. It touches on oral versus written language, the purpose of writing systems, how we adopt technologies, how education integrates them, the concerns and optimism around such technologies, what drives the technological changes and how using certain technologies for writing potentially changes how we think. I also chose this technology because my youngest daughter is one of many children for whom handwriting and to a lesser extent spelling pose specific challenges. While she does well in many aspects of education, her struggle with handwriting negatively impacts her self-esteem and enjoyment of school. To help her with her writing we have been getting her to use STT. I was curious to find out what the academic literature says about providing such education technology supports in the classroom.

STT is made possible because of advances made in Speech Recognition technology. It is a combination of highly advanced Speech Recognition computer programming and relatively simple word processing. The development of the technology was largely driven by nation state’s security interests and corporation’s desire to automate services, with powerful governments listening in at home and abroad needing to transcribe all of that content, as well as quickly highlight specific utterances, and corporations seeking to replace manual labor with machines in an effort at creating greater efficiencies and larger profits. These forces are like those that drove the development of early writing which as Hass points out: “… was used in state administration and bureaucracy, in trade and commerce, and in religion.” (1996). Although an extremely advanced technology in comparison to early forms of writing STT is used for many of the same purposes, exerting influence, amplifying conformity, recording transgressions and increasing efficiency.

Speech recognition technology has improved significantly in the past few years thanks to the application of deep learning and its use of big data. Deep learning is a subset of machine learning; a type of artificial intelligence built on complex neural networks that when trained using sufficiently large amounts of data can perform tasks that we once thought of as only humanly possible with incredible speed and accuracy. When a person converses with a virtual assistant like ‘Alexa’ or ‘Siri’ and has what feels like a seamless interaction it is in large part thanks to the deep learning algorithms that support these technologies. While the performance provided by these technologies is impressive it is not without its problems. Biases prevalent in for example ‘Western’ societies which disadvantage certain groups in those societies are sometimes replicated in the deep learning algorithms, such as the case with racial disparities identified in automated speech recognition in studies done in the USA. (Koenecke, A. et al, 2020). There appears to be little financial incentive for the few large corporations who dominate this technology to address these issues given the often-greater buying power of the dominant groups in those societies. The ubiquity of these popular speech recognition technologies may well lead to even greater homogenization of language and the loss of vernaculars across the world. Given that, as Boroditsky points out ‘the languages we speak shape our perceptions of the world’ (2011) the loss of vernaculars brought about by the ever-greater influence of speech recognition technology looks likely to reinforce the existing hegemonic state of the world.

An additional concern with the use of speech recognition technology is that given the enormity of the computing power required to facilitate these humans to machine ‘conversations’ it almost always means interacting with the Cloud. Otter, for example, which is one highly regarded STT tool and the one my daughter and I have been trialing uses AWS services. Reading the Privacy Policy provided by companies like Otter it serves mostly to remind the user that they are in essence giving up their right to privacy by using the product, listing as it does the many circumstances under which consumer data may be shared (2020). Otter is also the tool used to provide transcription services in Zoom meetings. The ability to provide closed captioning and transcription is highly beneficial to businesses in terms of the efficiencies it provides as well as for individuals who might face physical or mental challenges and might otherwise find participating in online meetings more difficult. However, it also, for better or worse, changes the way people communicate in meetings; making people more circumspect in what they say, and it of course does not capture what is intimated by body language and pauses. It is also worth noting that one of the main uses of STT is Compliance Management, particularly so in the financial sector which must monitor the calls and trades made by its employees (Fortune Business Insights, 2020). Continuous monitoring and scrutiny influence how people communicate. Moderating what we say contexts is not new or either simply bad or good, but what is clear is that STT technology applied in these contexts is shaping how we use language and influencing our thoughts.

Education

In their review of speech-to-text recognition technology for enhancing learning Shadiev and their fellow authors note that STT has been applied to enhancing learning in some areas of education since at least the late 1990s (2014). Initially it was mostly used with learners who had cognitive or physical disabilities, and learners receiving instruction in a second language. Studies carried out on those learners and their use of STT have shown modest positive results in relation to increasing the quality and quantity of written text (MacArthur, 2009; Peterson-Karlan, 2011). It is worth noting that at the time STT was being used as part of some of these studies the technology performed significantly less well than it does currently in relation to, for example Word Error Rate and Command Success Rate and required mastering a much less intuitive user interface compared with the current versions of the technology. Additionally, as is the case with the implementation of all education technology in a classroom any benefits gained from its use are as a result of several factors, (e.g., the teacher’s pedagogy, parent support, school culture, availability of resources, etc.) and as such it can be challenging to analyze the impact of the technology.

STT in education is most often framed as an aid to helps learners overcome certain barriers whether they be physical, mental, or other (Svensson, I., Nordström, T., Lindeblad, E., Gustafson, S., Björn, M., Sand, C., Almgren/Bäck, G., & Nilsson, S., 2019;). It is rarely discussed as simply a preference. Which is probably reflective of how it is perceived outside of the classroom, in society in general. Most people still think of writing as something done by hand, at least for any kind of meaningful writing. We do not imagine novelists composing their books aloud, although some do, with Agatha Christie being a famous example of an author who dictated many of her novels (Daily Writing Tips, 2021). Framing STT as a work around in education rather than a positive choice limits its adoption. Studies such as the one carried by Haug and Klien in which they investigated whether STT can be used to learn a writing strategy helps make the case that STT is a viable choice for all students (2018). Obviously, cost is still a prohibitive factor for many schools when it comes to using STT. However, any classroom that provides learners with the opportunity to type on screens could just as easily provide the option to use STT.

There is also still a good deal of support for the notion that teaching handwriting skills to young learners is important for reasons beyond simply mastering the ability to transcribe with some suggesting that the act of handwriting prepares the brain for learning (Ose Askvik, E., van der Weel, F. R. (Ruud), & van der Meer, Audrey L. H., 2020). Such arguments when carried over to the public domain are often framed as a zero-sum game; as if educators are being made to choose between something like STT and abandoning handwriting, or perhaps the thought is that if learners can compose with ease using STT they themselves will give up on handwriting. However, it is better to think of STT working alongside other forms of writing, similarly to how the New London Group describes multiliteracies creating: “a different kind of pedagogy, one in which language and other modes of meaning are dynamic representational resources, constantly being remade by their users as they work to achieve their various cultural purposes.” Cazden, C., Cope, B., Kalantzis, M., Luke, A., Luke, C., Nakata, M., & New London Group. (1999;1996; p.72) Writing using STT has the potential to change how individuals express themselves in writing in ways in which we have probably not yet begun to see.

Anecdotally, my daughter’s teachers will often refer to the notion that her hand cannot keep up with her brain and that is why she struggles with handwriting. However even allowing for the fact that there is probably room for improvement in her fine motor skills, handwriting exercises to improve this skill, while perhaps of some limited benefit, will not provide the solution. Of most concern is the fact that laboring over handwriting exercises is a drain on her time in return for small gain and detracts from the joy of learning in general and more specifically that associated with composing a piece of writing. A favorite soundbite in non-scholarly articles on the topic of STT is some approximation of the following: the average human can speak 150 words per minute, but the average person can write only 40 in the same time (Boyd, C., 2018)). The suggestion being that STT will free us from such physical limitations. For some people, notably those with physical disabilities, it may to an extent do that, but that is only one part of what it provides. Using STT compared with handwriting or typing helps learners reduce some of the cognitive load placed on a learner’s working memory resources when they are trying to compose and transcribe at the same time (Acorn, N., Klein, P.D. & Domboroski, J.D., 2017) This appears to be the case for my daughter. There are signs that she has challenges with her visual working memory which taxes her ability to transcribe and compose at the same time more than many of her peers. It is early days for my daughters use of STT. As with any technology there is a learning curve and there are frustrations. What is evident though is her delight at being able to quickly put into text her ideas. Tasks that would exhaust her when attempted with pencil and would quickly lead to her giving up she now engages with for longer and more fruitfully.

From our current view point it is difficult to imagine STT becoming as widely used as the more traditional forms of writing; but it is also not hard to imagine that people used to handwriting felt the same way before typing became commonplace. Many office workers around the world have recently begun to experience more flexible working arrangements. No longer tied to an office desk, could the next step be freedom from a keyboard and the ability to write using STT while walking? In education, providing greater opportunities for all learners to use STT would be a good example of Universal Design for Learning, helping to ensure that learners are provided with multiple options for expression and communication (CAST, 2021). I am a long way from being ready to switch from typing to STT, but I am trying to at least use it some of the time in an effort to show my daughter that it is as viable a text technology as any other.

References

Arcon, N., Klein, P. D., & Dombroski, J. D. (2017). Effects of dictation, speech to text, and handwriting on the written composition of elementary school english language learners. Reading & Writing Quarterly, 33(6), 533-548. https://doi.org/10.1080/10573569.2016.1253513

Boroditsky, L. (2011). How language shapes thought. Scientific American, 304(2), 62-65. https://doi.org/10.1038/scientificamerican0211-62

Cazden, C., Cope, B., Kalantzis, M., Luke, A., Luke, C., Nakata, M., & New London Group. (1999;1996;). A pedagogy of multiliteracies designing social futures. In B. Cope, & M. Kalantzis (Eds.), (pp. 60-92). Routledge. https://doi.org/10.4324/9780203979402-6

Daily Writing Tips (2021) Can you write a book or a novel with speech recognition software? Retrieved from: https://www.dailywritingtips.com/book-speech-recognition/

Fortune Business Insights (2020) Speech-to-Text API Market Size.Retrieved from: https://www.fortunebusinessinsights.com/speech-to-text-api-market-102781

Haas, C. (1996;1995;). Writing technology: Studies on the materiality of literacy. L. Erlbaum Associates.

Haug, K. N., & Klein, P. D. (2018). The effect of speech-to-text technology on learning a writing strategy. Reading & Writing Quarterly, 34(1), 47-62. https://doi.org/10.1080/10573569.2017.1326014

Koenecke, A., Nam, A., Lake, E., Nudell, J., Quartey, M., Mengesha, Z., Toups, C., Rickford, J. R., Jurafsky, D., & Goel, S. (2020). Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences – PNAS, 117(14), 7684-7689. https://doi.org/10.1073/pnas.1915768117

MacArthur, C. A., & Cavalier, A. R. (2004). Dictation and speech recognition technology as test accommodations. Exceptional Children, 71, 43–58. doi:10.1177/001440290407100103

Ose Askvik, E., van der Weel, F. R. (Ruud), & van der Meer, Audrey L. H. (2020). The importance of cursive handwriting over typewriting for learning in the classroom: A high-density EEG study of 12-year-old children and young adults. Frontiers in Psychology, 11, 1810-1810. https://doi.org/10.3389/fpsyg.2020.01810

Otter.ai (2020) Privacy Policy, retrieved from: https://otter.ai/privacy

Peterson-Karlan, G. R. (2011). Technology to support writing by students with learning and academic disabilities: Recent research trends and findings. Assistive Technology Outcomes and Benefits, 7, 39–62.

Shadiev, R., Hwang, W., Chen, N., & Huang, Y. (2014). Review of speech-to-text recognition technology for enhancing learning. Educational Technology & Society, 17(4), 65-84.

Speech recognition (2021, December 3). In Wikipedia. https://en.wikipedia.org/wiki/Speech_recognition

CAST (2021, December 5). The UDL Guidelines. https://udlguidelines.cast.org/

I am linking my Task 12 ‘Speculative Futures’ to Melissa Guzzo’s. Melissa does not state specifically that her first Speculative Future is a dystopian one or that her second is utopian (this is my assumption). I like that as it made me think more about their first story and the view they were taking. I had to read it twice before concluding that this is not a happy future. The choice not to label their stories as either utopian or dystopian makes me regret my own. I now think it would have been better to let readers draw their own conclusions about my stories.

Like me, Melissa chose to tell a story about how AI would work as some kind of an assistant to or in service of humans. In both our stories it is an uneasy relationship between human and machine. Ostensibly the partnerships in both our stories should benefit the human. We both make the point that humans do not always want what is best for them. Machine like efficiency and precision can very quickly become tiresome for most humans. In my story the father probably knows he should work diligently with AI Harvester and in Melissa’s the person can probably recognise that reflecting on their inappropriate comment is a good thing to do, but they still resent being asked to do it.

Melissa’s stories raised interesting questions for me about what rights we might have to images of ourselves captured in public and what use those images might be put to with our permission, similar to questions about data ownership. The story also I believe makes the point that future technology will most likely not be used in an overtly evil way (e.g.,machines initiating a war with humans) but will instead be used to subtly manipulate us and shape the way we think and behave. I also liked how Melissa’s story taps into some of the current conversation around ‘cancel culture’ and political correctness and how that may be policed/enforced in the future.

Finally, reading the story reminded me of what is currently happening in China with their Social Credit System. It is not hard to imagine that something like Melissa described could be integrated into that system.

A Note About My Speculative Futures

I do not think my utopian future is all that far fetched. We already use prosthetic limbs. I can easily imagine that we will try to develop ‘prosthetic’ brains and with what seems like ever increasing rates of diseases like althzeimers and dementia clearly there may be a need (assuming some other miracle cure is not found). A lot of occupational therapy goes into supporting individuals who have begun to use prosthetic limbs; to teach them to live with and use that limb. I can imagine that even greater levels of support and training would be needed to help someone integrate supports for their brain. One of many complicating factors is that diseases such as dementia do not necessarily progress in a gradual or linear fashion. It would be difficult to determine at what point the AI is expected to provide support and when it remains in the background. I also appreciate that diseases like dementia affect people in more ways than memory loss; that’s just the element I chose to focus on in my story. Diseases aside, I think there are also interesting questions to be asked about what we are supposed to forget as we get older. I do not think we are meant to have total recall throughout our lives. Would the ability to have perfect access to all our memories be a good thing? I am not sure.

For my dystopian future I chose to consider what the dark side of something that I am mostly very much in favor of might be… a world with greater levels of empathy. I note that VR is currently being employed to give individuals empathetic experiences and is something that, for example, a number of charities have started using to raise awareness and illicit more donations. Which seems like a good idea when it is something that an individual can choose to experience. But what if it wasn’t a choice? What if it was a mandatory part of a prisoner release process or what if parents could sign their kids up for it like some do for ‘weight loss camps’? My wife and I do try to teach our kids empathy and we do complain when we do not think they show as much empathy as we would like. We try to create real world experiences for them to develop their empathy (e.g., have them volunteer) and have conversations with them about it. Most parents are doing some form of social engineering with their kids. But it still feels like crossing a line to me to force a child to undergo some kind of VR supported empathy enhancing process. In saying this I recognize that I do have some bias when it comes to accepting technological solutions to certain human problems, like the development of empathy.

Vaulter

I came across this old conversation recently when uploading artifacts to my Vaulter. I am now near the age my father was then when we were having that conversation about him. He hated interacting with his own version of the Vaulter. At that time, it was just a prototype, clunkily referred to as an AI Harvester 2027. Back when calling something artificial was not frowned upon. We thought ourselves lucky to get Dad access to the technology, only doing so through my sister Matilda’s connections. We would have tried anything to spare him and us the ravages of dementia. But he was a little bit more philosophical about it; always ready to ask what are the trade offs; and what about the natural order of things. I could never be that brave.

View conversation 1

He said it was like spending time with the Spanish Inquisition; answering a barrage of questions, being made to account for meaningless actions sprinkled randomly throughout his life and ‘always with the feelings’. I think that is what he disliked the most. He wasn’t trained to access and talk about his feelings the way our generation was. Ironically, he probably shared more emotions with his Harvester than he ever did with any of the members of his family. Those early researchers didn’t understand how draining it would be for patients to interact with machines in that way. They were racing against time. There weren’t then truly empathetic AIs. It was quick and dirty, gathering information (memories, emails, texts, images, etc.) as relentlessly as they could, desperately trying to achieve critical mass and the ability to offset the inevitable neurological shortfalls in the patient’s biological brain.

View conversation 2

There’s a very old song by Radiohead “Where I End And You Begin” when I listen to it now, it always makes me think about Dad and his relationship with his AI Assistant. That’s not me he would say. This thing has eaten me alive and absorbed me into some cloud. In a way he was right. Achieving seamless blending between AI and human executive functioning was a long way from being harmoniously realised when Dad was working with his prototype. He glitched often and the buffering could be excruciating. There could be a deadness in his delivery that meant even though the words were right the tone might be wrong. It frustrated him to function in that way.He clamed his thoughts echoed, calling attention to themselves in an unsettling way. He could never seem to appreciate that when he was without his AI Assistant, things were so much worse for us. He always maintained even with all that he lost access to, he only ever really felt his true self when he was disconnected from the machine.

I signed up for the full Vaulter package shortly after Dad passed, giving it full access to my entire digital footprint and making a practice of uploading for 20 minutes daily from my biological brain. The technology has improved significantly since Dad first tried it. The interface is much better; less inquisition, more friend, counsellor and priest. Dad had to interact with a harvester and wear his AI assistant. My Vaulter is a combination of the two and embedded in my brain. My doctors tell me that without it, given the deterioration in my biological neural pathways caused by the many years of social media consumption during the early decades of this century (if only we had known then what we know now), I would be practically non-communicative. I am grateful, I think…at least, I hope this is me, that these really are my thoughts.

When working my way through this week’s podcasts and readings and the influence of risk based algorithms I was reminded of my previous career as an immigration officer with the Canadian government. Approving visitor visa applications (tourist, business, workers and students) is a risk management process. When reviewing applications immigration officers are trying to determine what the likelihood is that the individual applying will overstay their visa, work illegally or make a bogus refugee application after entering Canada. If it is determined that there is a reasonable likelihood that they may fail to leave at the end of their visa’s duration their application is refused. Determinations are made on scant documentary evidence, officer intuition and a good deal of undocumented (i.e., never included in the officer’s official notes) guidance and advice shared among officers.

It is an arbitrary and flawed system, highly subjective and inconsistent. The people making the decisions are for the most part foreigners living in a part of the world where they may have little or no local knowledge or cultural understanding. The workload, in places like, for example, China or India, is enormous and unrelenting. Consider for example that pre-Covid Canada was receiving approximately 30,000 student visa applications a year from individuals in mainland China. That means an individual immigration officer is making determinations on thousands of applications every year.

It is, therefore, not hard to see why the Canadian government would be interested in using AI technology to remove some of the decision making responsibility from human officers, as they began doing in 2014 (CBC Radio, 2018). Regrettably, the approach taken by the Canadian government at that time does appear to be a good example of the 3 factors associated with ‘bad’ algorithms highlighted by Dr. Cathy O’Neil in her interview on the 99 Percent Invisible podcast. The algorithm used was widespread, potentially influencing hundreds of thousands of immigration decisions, decisions that can have huge consequences for the individuals applying; the process was secretive for a number of years; and destructive, disproportionately disadvantaging certain groups (put simply, wealthy individuals have much less trouble obtaining visas to Canada). It should be noted that the Canadian government has recently established what it calls an ‘algorithmic assessment tool’ which is in part an attempt to address the type of criticism put forward by advocates such as Dr. O’Neil.

However, I think there are additional concerns when it comes to deploying AI in areas like immigration, beyond those outlined by Dr. O’Neil. For example, it could be assumed that the algorithms would be built using very problematic data. As I alluded to earlier, many of the factors that have historically gone into an immigration officer’s decision making process are left out of the official record. What is left out is usually the very obvious biases that the officer or groups officers in a particular office have built up over time. The algorithm would bring consistency and efficiency, but it would also replicate and perhaps amplify many of the biases inherent in the traditional system. 

Pocaro, in his article regarding ‘Detain/Release: simulating algorithmic assessments at pretrial’, states that: “Software has framing power: the mere presence of a risk assessment tool can reframe a judge’s decision-making process and induce new biases, regardless of the tool’s quality.”(2019) I can easily imagine this playing out in a similar fashion among immigration officers. There are usually no official guidelines as to how many applications an officer should approve or refuse, but informally there is feedback provided by managers and co-workers. Overtime officers generally fall in line. The AI risk assessment tool would increase the level of conformity and leave less room to question the status quo.

The immigration algorithm and those like it are presented as solutions to complex problems, assuming that what is needed is greater efficiency and consistency. However, the answer to complex problems, like for example how to equitably and transparently manage immigration into Canada is not solved by simply providing greater efficiency and consistency based on flawed data.

References

CBC Radio (2018, November 16). How artificial intelligence could change Canada’s immigration and refugee system. The Sunday Magazine. https://www.cbc.ca/radio/sunday/november-18-2018-the-sunday-edition-1.4907270/how-artificial-intelligence-could-change-canada-s-immigration-and-refugee-system-1.4908587

Nalbandian, L. (2021, April 28). Canada should be transparent in how it uses AI to screen immigrants. The Conversation. https://theconversation.com/canada-should-be-transparent-in-how-it-uses-ai-to-screen-immigrants-157841

Pocaro, K. (2019, January 8). Detain/Release: simulating algorithmic assessments at pretrial. Medium. https://medium.com/berkman-klein-center/detain-release-simulating-algorithmic-risk-assessments-at-pretrial-375270657819

Vogt, P. J., & Goldman, A. (Hosts). (2019, May 12). The Age of the Algorithm (No. 274) In Reply all. Gimlet. https://gimletmedia.com/shows/reply-all/brho4v/274-the-age-of-the-algorithm

In what way is the ‘User Inyerface’ game designed to manipulate my attention and responses?

This game might be the answer to weaning myself off my internet addiction! If all my interactions online involved this level of frustration I would take up knitting. The game’s graphic user interface (GUI) is horrible. Interacting with it is incredibly frustrating. I made slow progress through the various form filling activities, figuring out cheats as I repeated my efforts, before eventually giving up after 10 or more attempts. I got as far as the third page but concluded I was never going to be quick enough to match the date of birth with the number slider on that page.

According to thedarkpatterns.uxp2.com website: “Brignull defines “dark patterns” as instances where designers use their knowledge of human behavior (e.g., psychology) and the desires of end users to implement deceptive functionality that is not in the user’s best interest.” (2021) I appreciate what Brignull is trying to do and the examples he provides are helpful. I have personally fallen victim to several of the dark patterns listed on his website (e.g., Privacy Zuckering, Hidden Costs, Forced Continuity). However, there is something about his definition and the notion of what is in a ‘user best interest’ that throws me off a little. I do not think we should or do expect all businesses to act in our best interest. There is a whiff of nannyism about such an idea. However, when a business markets itself as caring or a ‘public good’ but acts in another way, it is reasonable to shine a light on how their actions contradict their public statements.

In order to reflect on how web and interaction designers employ practices to lead the attention of people towards or away from certain elements in digital environments, and to promote or discourage certain kinds of behavior I have decided to examine my own journey from occasional craigslist user of many years to recently regular facebook Marketplace (fM) user (despite the fact that several years ago I gave up using the main facebook site for any kind of social media). I used craigslist when I had a specific need to buy or sell an item locally. I bought and sold, for example, my kids’ bikes on craigslist. In all the time I used craigslist I never felt like I was spending a lot of time on the site or that I was being distracted by the machinery it used to display the items. Its user interface might be described as functional, plain, text heavy and not the simplest to navigate. Its pages are mostly white, when images of items are displayed they are not attractively framed and the font used, at least to my eye, is not very appealing. In contrast fM has a much more appealing design layout; using a good combination of pleasing colors and text, images that are nicely arranged and search bars that are easy to navigate. This is even truer if we compare both platforms displays on mobile devices. fM is aesthetically more appealing than craigslist but this is not the reason that it has stolen my attention.

fM personalizes the buying and selling experience much better than craigslist. It provides a small amount of information about how long the person you are buying or selling from has used the site; what other items they have for sale; which groups they belong to; more precise mapping of where they are located and; if you have friends in common on the site it will inform you of this. This information gives a buyer a little more trust in the seller; this may of course be a false impression. Additionally, communicating (messaging) with an individual is typically quick and seamless on fM. Craigslist, by contrast, does very little to dispel the impression that you are buying from an anonymous stranger. fM seems like a smarter way to shop than craigslist but this is not the reason it has stolen my attention.

The reason fM has stolen my attention is because of its sophisticated use of recommender systems. facebook is constantly gathering information on its users and using this to among other things suggest additional content. While shopping for a used couch it might also be suggested to a user that they consider related items such as armchairs. Every time you log on to the site the homepage will display an array of potential purchasing items based on your most recent searches. Some of these items you may have directly searched out and some may be only tangentially related. From armchairs to camping chairs that you did not know you wanted until you saw them displayed on your feed! I do not have push notifications turned on for fM, which protects me from even greater distraction, but even without those reminders I still find myself drawn to the site much more than I ever did when using craigslist. Somehow I allow this to happen despite knowing what I do about facebook and listening as I have to the Wall Street Journal’s recent expose, the facebook files.

References

ux2p Dark Patterns. (2021, November 14). ‘The dark side of UX Design’ https://darkpatterns.uxp2.com/

I am linking my Task 8 ‘Golden Record Curation’ to DeeDee Perrot’s. In selecting her songs DeeDee used 3 criteria. She selected based on mood, geographical location and whether a song was instrumental or contained vocals; aiming to have songs that represented a range of moods, diverse areas of the world and a list that had both instrumental and songs with vocals. When I embarked on this task I also thought about trying to select songs based on mood. I abandoned that approach because I could not divide the tracks into 10 distinct moods (which for reasons I can’t recall now was important to me). I appreciate how DeeDee used a particular methodology for discerning mood. Considering music based on pitch, timbre and rhythm seems somewhat of an objective measure, but I am not sure that intensity is anything other than subjective. It is interesting to consider how we have come to tacitly accept that certain moods are communicated via music, while at the same time understanding that our feelings towards music are very subjective. It seems that we are hardwired to sort and categorize; an instinct which probably propelled the development of language and text.

It’s not clear to me why DeeDee selected based on moods, geography and instrumental/vocals. What does a selection of music based on human moods, diverse locations and instrumental/ vocals communicate about the people that live on this planet? The same question could be asked about my selection process. I chose only tracks that included vocals and represented unique languages. My stated reason was that hearing a range of human voices and languages might provide beings from another planet some kind of insight into humans, but of course that is a very fanciful idea. Perhaps this task is a reminder that when we set out to communicate anything we do so with a large amount of trust and hope that whoever is receiving our communication will interpret it as we have intended, or at the very least will not ascribe ill intent on our communications. I can’t help but wonder if we were to post our song selections on Twitter and attempt to explain how we chose our tracks, how much opprobrium and scorn might we receive.

Reviewing DeeDee’s list, I realized that I missed a song with vocals when making my own selection; ‘Jaat Kahan Ho – India – Surshri.’ and I think, DeeDee incorrectly identified the  ‘Panpipes and drum song’ from Peru as a song with vocals.

My understanding is that Group 3 (Natalie, Delian, et al) are the most well connected group. They each have a list of song choices that connects them to more individuals in more ways than the people in Groups 2 and 1. I am making this assumption based on the graph below which I created by selecting as Source – ‘Curators’ and according to – ‘Sum of Communities’. We can see that the circles associated with each individual’s name are sized according to the size of community they relate to. Group 3 contains curators with the largest sum of communities, Group 2 the second largest and group 1 the smallest.

But what does this mean? Do the individuals in Group 3 have a knack for selecting songs that will create more pathways between them and other members of this class? Is it a reflection of their taste in music? I wondered if their lists of songs contained more of the most popular songs than those of us in Groups 2 and 1.The following graph groups song based on how many curators chose that track; with the size of the circle reflecting its popularity.

The 3 most popular songs are ‘Melancholy Blues’, ‘El Cascabel’, and ‘Fifth Symphony (First Movement)’. Four individuals in Group 3 have selected 2 or more of the the 3 most popular songs. However, Amy did not select any of the 3 most popular tracks. Amy is an outlier in her group in that she has less songs in common with the others in her group (see graph below). This suggests that selecting the most popular songs does not mean that you will necessarily be the most connected person in the class, but it probably helps.

Interestingly, in Group 2, no one selected the ‘Fifth Symphony (First Movement)’ and only 2 individuals selected both of the other most popular songs, while  in Group 1, 2 individuals selected all 3 of the most popular songs. I might hastily conclude from this that selecting the most popular songs in your list does not in and of itself mean you will be well connected to the rest of the group. I could of course be wrong about all of the above, which would be an interesting outcome to, saying something, as it would, about how we can be seduced by graphs and our own vague understanding of them.

What to make of the political implications of such groupings and all that is unsaid in the graphs? We know that the criteria individuals used to choose songs for Task 8 would be quite idiosyncratic and I think on one level it would seem quite random. At the same time, we do know quite a bit about the people that chose these songs. We know that they are pursuing a Master of Education Technology degree through a university in western Canada. We might conclude from this that they have an interest in education and are educated to at least undergraduate level, they are at least somewhat technologically proficient and given which university they are attending more likely than not they probably lean a little to left on the political spectrum (This being an online degree, with individuals from all over the world attending, this might not hold up quite as well compared to a similar course pursued in person at the university). With the above information we might surmise that the individuals choosing the songs would want to exhibit a sense of fairness in their selections. One way in which they might make an effort at fairness would be to choose songs that represented peoples and cultures from different parts of the world. There are 6 pieces of Western classical music on the original list. I bet even the most hardcore classical music fan would avoid picking more than 3 of these in the interest of fairness.

When it comes to making use of networks and identifying how groups connect it can sometimes seem that there is less effort made to understand why these connections exist compared to figuring out how best to exploit these patterns. We do not always understand why certain pathways are created, but we do know that under certain conditions these pathways are the likely outcome. We see this done on social media platforms where political ads and campaigns run. Messages and images are constantly tweaked and monitored in order to figure out which combination of spin makes something most viral. In our example we might experiment by reordering the original list of songs, or we might choose to highlight the fact the song was composed by a male or female person. Does the order in which songs are listed, or highlighting the gender of the composer, impact which songs are more likely to be selected? It might. By applying this type of methodology we learn how people are influenced and the pathways that are created by their choices. Lessons learned from how networks are formed in this type of assignment might on another level be used to help with the dissemination of knowledge or persuasion of people in relation to, for example, vaccines and pandemics, or it could be used to encourage people to share conspiracy theories. 

I am linking my Task 7 ‘Mode Bending’ to Amy Trainor’s. I was attracted to Amy’s post as it seems to me a bold reimagining of the first task. It is audio without words; something I could not have imagined doing. I was also impressed that she took the opportunity to weave in teachings from the ‘First People’s Principles of Learning’.

Amy used a soundscape to communicate information about the items in her bag. In her reflection she described “…“whole-body” listening as multiliteracy and culturally diverse way of knowing…” Connecting multilieracies to cultural diverse ways of knowing seems entirely appropriate to me and very much in keeping with the ideas being presented by the New London Group. Amy’s approach recognizes that communication is both about how it is transmitted (sound) and also how it is received (whole-body).

Her soundscape brought the items in her basket (bag) to life in a visceral way and yet it was simply done. There was no showiness or trickery, no attempt to dazzle. She let the items speak for themselves and it drew me in. It was for me a whole-body listening experience. Hearing a marker rub against paper I felt the sensation in my chest in addition to processing the sound aurally. Listening to this I was reminded how my youngest daughter enjoys watching and listening to Autonomous Sensory Meridian Response (ASMR) videos on YouTube. These videos communicate a sense of calm or well being to my daughter and I think I got some of that from Amy’s video too. Amy’s soundscape communicated clearly how she uses some of the objects in her bag, like for example the sound of her camera taking a picture (although interestingly having the camera on our phones make a sound when taking a picture is a choice these days), but for some objects it was harder for me to connect the sound with the object. Perhaps the difficulty in attaching the sound to a particular object encourages better listening, which would in itself be a pretty useful outcome from this activity.

In contrast, for my version of Task 7, I think I had an interesting idea and story to tell, but I did not feel confident enough to do that without also dressing it up with a hodge-podge of visual elements. In contrast to Amy’s approach mine is cluttered, with some of the images I chose perhaps distracting from the story rather than working in support of it. With her soundscape Amy created an immersive experience, whereas my approach keeps the receiver at a distance.

Tracks

1. Java, court gamelan, “Kinds of Flowers,” recorded by Robert Brown. 4:43
2. Zaire, Pygmy girls’ initiation song, recorded by Colin Turnbull. 0:56
3. Australia, Aborigine songs, “Morning Star” and “Devil Bird,” recorded by Sandra LeBrun Holmes. 1:26
4. Mexico, “El Cascabel,” performed by Lorenzo Barcelata and the Mariachi México. 3:14
5. “Johnny B. Goode,” written and performed by Chuck Berry. 2:38
6. Mozart, The Magic Flute, Queen of the Night aria, no. 14. Edda Moser, soprano. Bavarian State Opera, Munich, Wolfgang Sawallisch, conductor. 2:55
7. Georgian S.S.R., chorus, “Tchakrulo,” collected by Radio Moscow. 2:18
8. Bulgaria, “Izlel je Delyo Hagdutin,” sung by Valya Balkanska. 4:59
9. Navajo Indians, Night Chant, recorded by Willard Rhodes. 0:57
10. Peru, wedding song, recorded by John Cohen. 0:38

Explanation

I picked the 10 tracks listed above. I made my selection based on the fact that all of these tracks contain singing. I hope that by including the human voice and a range of languages in each of my 10 tracks it would provide beings from another world a little more insight into who we humans are. Our voices are an essential part of human language and expression. 

Would the beings from another world be able to recognize the difference between a human voice and any other musical instrument? Humans are usually pretty good at this, so I am going to assume our friends from other planets will too.

Based on my review of the 27 pieces of music included in the Voyager Golden Records there were 11 tracks that contained singing. To solve this problem I decided that no language could be represented more than once. There were two songs in English. I  opted to omit “Dark Was the Night” by Blind Willie Johnson, because “Johnny B. Goode” has an electric guitar and I thought its invention says something profound about human ingenuity….or maybe I just like ‘Back to the Future’.

Having made my selection, it occurs to me now that my list skews slightly towards a Euro-centric representation of the world if we consider that English, German, Spanish, Bulgarian and Georgian (close to Indo-European language) are all represented on the list, although I think that is in part a legacy of the original record compilers. I am also cognizant of the fact that by privileging the human voice I am neglecting other aspects of human identity and communication that are equally valuable.