The detain/release game is incredibly conflicting, which I think is a testament to how well put together it is. It’s so difficult making each decision, and the game forces one to balance a variety of conflating factors, from your fictional position as the judge, the recommendation of the prosecution, the defendant’s statement, and the algorithm’s ratings. Though the photos are obscured, there is sufficient information for unconscious bias to seep into the decision making regardless: Gender and age are provided, and the photos provide an – albeit vague – notion of race.
I played through the game three times, and each proved to be a slightly different experience.
The gamification of the decision making was initially a major influence for me. I found myself overly focused on maintaining a low fear rating and an acceptable prison population. To do this, I placed significant weight on the algorithm’s ratings. This resulted in high detentions. At the same time, my fear rose as well. Defendants with low risk ratings would fail to appear, or commit additional crimes.
On subsequent playthroughs I focused on balancing the advisement of the prosecution and the statements of the defendants, along with the algorithm’s rating. Still, my fear level rose just as quickly as my prison population.
On none of my playthroughs did I feel as though my decision-making was having any measurable impact. No matter my decision, detain or release, neither felt as though it had an outcome much better than random. Releasing even low rated offenders lead to risk of re-offence or flight. Detention was always the safe decision, assuming the population wasn’t climbing too high.
The incentives seem misaligned, the information insufficient. It makes it easy to defer to the algorithm’s recommendation. The algorithm seems as ineffectual as the prosecutors recommendations, which itself seems to have as little influence as the defendant’s statement.
The way I picture it is very much like the network graph of our collective Golden Record track selections – a network graph that many algorithms would have been trained on for purposes of evaluating potential detain/release risk. There’s simply too much missing context for this algorithmic data to be used for such grave and consequential decision making.
As is explained in The Crime Machine I and II, over-reliance on algorithms for decision-making can lead to extremely skewed, outright harmful outcomes. In detain/release, these outcomes don’t feel much different than outright random selection.
References
Vogt, P. (2018, October 12a). The Crime Machine, Part I (no. 127) [Audio podcast episode]. In Reply All. Gimlet Media.
Vogt, P. (2018, October 12b). The Crime Machine, Part II (no. 128) [Audio podcast episode]. In Reply All. Gimlet Media.
Leave a Reply