Task 11: Detain/Release

It’s scary how, despite all the warnings about the issues with algorithms in the various podcasts and videos for this module, I ended up heavily relying on just three colour-coded lines of “fail to appear,” “commit a crime,” and “violence” to make decisions in Porcaro’s 2019 simulation Detain/Release. Rather than mulling over all the aspects of each case such as what the prosecutor recommends or the defendants’ age and statement, I gravitated towards green/yellow/red words that highlighted the risks of each defendant and without thinking about the accuracy behind it, used those to inform my choices. This personal experience with Detain/Release reminds me of the concept of “thin-slicing,” and in this reflection I will focus on how thin-slicing creates a positive feedback loop when we throw algorithms in the mix.

Almost two decades ago I was taking a undergraduate cognitive psychology class, and Gladwell’s 2005 book, Blink, was the assigned reading. In it Gladwell argues that with life experience, rather than processing all pieces of information available, people develop subconscious heuristics relying on a few key pieces of information to make decisions/judgements. Gladwell calls this “thin-slicing,” and gives numerous examples such as how rather than agonizing over all the aspects of a property, people buying a new home only need a few moments with the property to realize it’s “the one,” or how a psychologist with decades of experience working with married couples can accurately predict whether a couple will stay married 15 years down the road with 90% accuracy after spending 15 minutes with them. Gladwell also points out that often thin-slicing is affected by prejudices or biases, and do not lead to correct decisions, giving the example of blind vs face-to-face orchestra auditions that was also given by O’Neil in the 2016 Talks at Google video.

Two decades later, I believe that thin-slicing is more prevalent. With easy access to far more data, one could spend hours gathering data or something to make a choice, or one could selectively place their attention on a few pieces of key information. This attention selection could be either deliberate or primed subconsciously through UI design elements such as colour, movement, and soud, discussed in last week’s module. For example, in an assignment for ETEC 511 a year and a half ago I reflected on how I purposefully only considered price, rating, images of the room, and location when I was selecting hotels for my vacation, whereas in Detain/Release, I suspect that I was heavily influenced by the colour-coding of information to more attention to it. The prevalence of video games could also contribute to more thin-slicing; Green & Bavelier (2012) was one of the readings in the aforementioned assignment for ETEC 511, and in it they discuss how video games “enhance attentional and executive control. By facilitating the identification of task relevant information and the suppression of irrelevant, potentially distracting sources of information, improvements in attentional control could enable individuals to more swiftly adapt to new environments or to more quickly learn new skills.”

Alongside the various discussions in this week’s module of the issues with algorithm such as how data was used in a metric rather than a tool to inform decisions, privacy issues regarding training data, creating epistemic bubbles, positive feedback loops and self fulfilling prophecies were two themes I noted throughout. In Vogt’s The Crime Machine podcasts, CompStat was shown to perpetuate the summons issued in a particular area. Crime is reported -> police logs the crime -> algorithm now shows the area having more crime and sends more police there -> police now need to continue issuing summons to meet their quota. O’Neil in the 2016 Talks at Google video provides another example with recidivism algorithms: biased judge makes a ruling -> ruling is used to train algorithms -> algorithms informs other judges to make rulings. My personal experience with Detain/Release supports O’Neil’s claims while highlighting how factors such selective attention and thin-slicing help reinforce this positive feedback loop. By distilling all the information into colour coded words about someone’s supposed likelihood of running away, recommitting a crime, or violence, key pieces of data to inform decisions are no longer obtained through observation through a lens shaped by personal experience, but obscured through a lens provided by an algorithm.

References