Category Archives: ai

The Savage Detectives VI: Reading with AI

Posted on by

As an experiment (and in preparation for an upcoming discussion of reading with AI, in the context of reading The Savage Detectives), I thought I would ask ChatGPT a few questions about the book, and see what it came up with. You can find the transcript of the resulting chat here: https://chatgpt.com/share/69c7928a-1ac0-832b-a01f-d991cb125134. Note that, because I am on the free tier of ChatGPT usage (i.e. I do not pay for it), I was only able to ask it ten questions. I may or may not continue the experiment later.

In sum, in some ways I was quite impressed with ChatGPT’s responses. On the whole, in general terms, it came up with some quite plausible readings… though I would emphasize that these are readings, plural, in that when pushed it would change its tune, albeit without fully acknowledging its shifts in interpretation. Moreover, the more detail I asked of it, the more it seemed likely to make mistakes, or to hallucinate. Again, while the overall sense it made of the book was often reasonable, and sometimes even revelatory, it still made several basic errors, and continued to make errors even when I tried to correct it. To put this another way (and drawing on Dan McQuillan’s description of AI as a “bullshit generator”; see also Harry Frankfurt’s On Bullshit): it came up with some quite high-quality bullshit that, as is generally the case with bullshit, had more than a kernel of plausibility or truth, but when pressed it tended to contradict itself or even to start to fall apart altogether.

Of course, in that (as we know) ChatGPT cannot “read” a text in anything like the conventional sense of reading, it draws its interpretation from what others have already said. So it was perhaps equally worrying that when, at the end of the chat, I asked it for its sources, its response was somewhat vague and evasive, even though at the same time it did provide some reasonable avenues for further study. But the worry is that, on taking up elements of ChatGPT’s proposed interpretation of Bolaño’s novel, I would be inadvertently plagiarizing the work of some other scholar, whose identity the AI bot could not help me to locate. At the end of the day, after all, LLM AI applications constitute the large-scale capture of intellectual, cognitive, and linguistic labor, whose origin they (necessarily) obfuscate.

It is possible that some of the less helpful or more worrisome aspects of the interaction with ChatGPT could be mitigated by better attention to the prompts I was giving it. For instance, if I continue this experiment in the future, I am tempted to ask it both for specific references to the text to support its claims, and for references to secondary criticism throughout the process. This might encourage a more grounded (and more transparent) interpretative process. Alternatively, rather than asking it directly about the text, one might begin at least with a review of the secondary literature on which it is drawing. That might keep it (and me, as a reader of ChatGPT’s “reading”) more honest.

Briefly to go through the questions I asked it…

First, I asked simply for it to provide me with the novel’s plot. It did a pretty good job of this, albeit that it had more to say about the book’s narrative structure than about the plot per se. The plot, after all, is not told linearly, and yet it kept to the order of the narration (the “discours” in Genette’s terms), rather than helping to reconstruct the story’s “histoire.” Having done this, it went further (beyond, indeed, what it had been asked) to advance some proposal’s as to the novel’s overall meaning. But in general, just about everything it said in response to this question was pretty much reasonable and defensible, and to some extent what I would even call insightful: the notion, for instance, that in the broadest terms the book’s plot involves a failed or futile search for origins, followed by “dispersal and disappearance.” Similarly, the summary of the novel’s themes as “literary ambition,” “friendship and exile,” and “the mythology of the avant-garde” makes sense, even if it is not necessarily the whole story–but then what summary is?

I did, however, feel it a little odd that this plot summary omitted the role of Amadeo Salvatierra, who is a recurrent voice in Part II of the novel, and who is important for the plot in that it is he who provides Ulises Lima and Arturo Belano with the information about Cesárea Tinajero that motivates and enables the road trip to Sonora around which the whole book revolves. So second, then, I asked ChatGPT about him…

In response, the chat implicitly recognized its error in omitting Salvatierra by saying that “he plays a crucial symbolic and structural role in The Savage Detectives” and that “he is actually the hinge between myth and history in the novel.” I was somewhat surprised, however, that ChatGPT should in fact spend more time on Salvatierra’s thematic or conceptual significance than on his function in the story. Indeed, the more that the chat edged towards concrete description, the more likely that it flirted with outright error. Hence, although its broad claims continued to be defensible and even perhaps enlightening (if not altogether unarguable), when for instance it states that “Amadeo possesses (or remembers) the only surviving work by Tinajero,” this is at best misleading: Amadeo very definitely possesses the surviving copy of the journal in which Tinajero’s poem is published. This is not simply a matter of memory: he shows it to Lima and Belano, so they can read it for themselves. More fundamentally, the chat is simply wrong with basic details when it claims that “Amadeo is an aging poet whom García Madero, Belano, and Lima visit early in the novel.” García Madero does not accompany Belano and Lima in this visit. And to say that it takes place “early in the novel” is again misleading: the account of the visit is threaded through Part II (and does not come to an end until page 588), although to be fair we can reasonably assume that it takes place at some point during the time period of Part I, the final weeks of 1975 as narrated by García Madero… but he never mentions the visit, presumably because he is most likely unaware of it.

I then asked for more information about the fates of the novel’s key characters.

ChatGPT’s response about García Madero was mostly to the point, albeit again somewhat misleading when it said that “García Madero is almost never mentioned” in Part II. In fact, he is mentioned precisely once… and it is surely important to describe that sole mention, not least because it comes when the so-called expert on visceral realism denies that anyone with that name had ever belonged to the movement.

What it had to say about Lupe was also OK, and even insightful. Again, it is notable (and surely somewhat surprising) how quick ChatGPT jumps to interpretation, rather than description. Perhaps this is because it is drawing above all on interpretations in the secondary criticism? But its conclusion about her fate is fair enough, albeit that this is surely a fate shared with (rather than opposed to that of) García Madero: “Not just absence from literary history, but from the kinds of stories that get told at all.” Eager to please, ChatGPT also offered to expand this discussion via a “compar[ison] to women in other Bolaño novels (like 2666), where this question of disappearance becomes even more central and more disturbing.” This in fact sounds like an interesting direction for further enquiry (and it’s perhaps intriguing that ChatGPT should offer to take me down that route), but here I was sticking with The Savage Detectives…

It was upon being asked about what ultimately happened to Lima and Belano that ChatGPT really started falling apart. Its top-level (i.e. “bullshit”) analysis remained for the most part quite impressive. For instance, the notion that these two characters become transformed into myth, and that there’s an irony in the failure of their mission to track down (what we find out is) the all too material and corporeal Cesárea in Part III, which may indeed lead to the conclusion that “to pursue literature absolutely may mean vanishing from it.” This was one of the ideas that ChatGPT gave me that I quite liked. And yet again, there was the sense that the chat was weak on description and detail. For example, when it says that “At one point, [Belano] travels to Africa and is possibly present during a conflict,” this is at best wishy-washy, almost as though it didn’t want to be pressed on what exactly happened. (In fact, after all, it’s not just “at one point” that Belano is in Africa, but this is where he is last seen. And more than being “possibly present during a conflict,” he is discovered in the thick of the Liberian Civil War.) If we were to anthropomorphize the machine (and it’s hard not to), we might say that this is the answer of a student who has only a hazy recollection of the text and wants to hedge their bets with their answers.

And indeed, when I pressed it further for answers, ChatGPT really started to struggle. Projecting its own confusion onto the text, for instance, the chat said that Belano “travels to Africa (often read as Liberia or Angola, though the novel is oblique).” In fact, however, there is nothing oblique about it: we are told quite clearly that he is in both Angola (Luanda, to be precise) and Liberia (a short distance from Monrovia). With Lima, things got still worse, and ChatGPT started to make multiple errors, even when I tried to zero in on them. 

It’s not necessarily worth detailing these errors, though it’s interesting, first, that the app’s tactic when pointed out its mistakes is to concede without fully giving ground. After I observed that it was wrong to a greater or lesser extent about the fates of both characters, it responded: “You’re right to push back—both of those ‘last sightings’ are more precise (and more interesting) than my previous answer suggested. Let me correct and sharpen things.” Yet it is not simply “more precise” to note that Lima is last seen not in Europe (as the chat had claimed) but in Mexico: it is right, where the chat was wrong. Second, moreover, as it made mistakes about the details it also started to make more obvious errors of general interpretation. For instance, in that Lima is located in Mexico City by our visceral realist expert at the end of Part II, it is simply not the case that “he simply fades out of the record altogether, more completely than anyone else in the novel.” There are plenty of other characters (most obviously perhaps, García Madero) who are more fully erased from the record… as indeed ChatGPT had already observed (when it said that, by contrast with Lima and Belano, he “undergoes a kind of negative initiation–into obscurity rather than legend”). The machine is by now contradicting itself, though unsurprisingly enough without acknowledging the fact.

After all, the chat’s tone remains consistent: it retains its aura of confident authority, so much more so that in reading it, one is tempted to doubt oneself… If I had not finished the novel only this week, I might easily have been more prepared to go along with its account, and to question my own memory rather than catching the app’s hallucinations.

For my next question (and by this time I was running out of my allocation), I returned to conceptual issues more than concrete details, in fact feeding it questions about length and endings that I had put to my students in class. Again I feel that its answers were illuminating even (if we can say this of a machine) “thoughtful.” These could definitely have been useful contributions to the discussion we had in class… I had a twinge of regret that perhaps I had missed a trick or two. Put it this way: if ChatGPT had been a student or (as a more likely scenario… and the worry of educators today) if a student had read out ChatGPT’s responses and presented them as their own, they would have been among the more insightful and valuable voices in class. I even like some of its turns of phrase. For instance, I like the notion that “If The Savage Detectives were shorter, it would be a story. If it were longer, it would be an archive.” This formulation is at the same time provocative and neat. Likewise with the notion that what we have here is “a myth of literature built out of fragments, ending exactly where it has to–at the moment when the search reveals that there was almost nothing to find.” In class, I said something similar, or at least similar enough–I put things differently, in that my suggestion was that at the end García Madero at least finds “nothing” in the more substantive sense that he touches and perhaps even falls (or jumps) into the void that lies beyond the “window” of representation. But still, there are some sentences and phrases in ChatGPT’s interpretation that I perhaps wish I had said or considered en route to my own conclusion, or which could usefully have been put to the students as alternative considerations. Could ChatGPT become a valued interlocutor in the classroom?

Yet by this point, seeing the chat’s weakness on points of concrete detail, to be honest I had started to lose faith or trust in ChatGPT altogether. Worse, this sense of distrust had become generalized: I no longer felt confident about either its reading or (to be even more honest) my own. After all, if my reading overlapped to some extent with that of a machine that had just shown some fundamental ignorance of basic issues of plot, then did that not cast shadow on my own interpretation, too?

So the last of my permitted questions was about its sources. I figured I would be better off looking to where ChatGPT was getting its “thoughts” than continuing to engage with the chat itself. Here, however, the app was both honest and unhelpful: “What I’ve given you is a synthesis drawn from general knowledge of The Savage Detectives, familiarity with Roberto Bolaño’s work, and the broad critical conversation around it.” In other words, the AI is drawing on a cloud of more or less established readings, without being able to pin down or lead me directly to who first established those readings, or what debates they had led to. It had, as it were, anonymized the critical conversation, obscuring the identities of those on whose intellectual labor it relies. To be fair, it did then provide some bibliographic references, and ended also with a note of caution that there are indeed “other strong readings that [it] ha[d]n’t foregrounded”… or even, if it were more truthful, mentioned at all. Its appropriation of intellectual labor, in other words, is selective and unequal. It’s notable, for instance, that it acknowledges that among the readings it had ignored were those inspired either by queer theory or by feminism. 

And perhaps it would have been helpful if at the very start it had stated upfront that what it was “giv[ing me] is not the interpretation, but a coherent, recognizable strand within Bolaño criticism.” Though of course it had done all it could to make this “recognizable strand” appear to be all its own work, appealing to our technological fetishism that attributes to the machine the qualities of agency and intellect that more properly belong to those who feed it… which is, in the end, you and me.

2666 II: Machine Reading

Posted on by

Upon reaching the end of “The Part of the Crimes,” it is hard to see how it could have stood on its own. And yet, according to the note that prefaces the entire novel, that was Bolaño’s plan, communicated just “days before his death” to his publishers: that the various parts of the book should be published separately.

To me, at least, it does not feel as though things have really started getting going, even by page 207. Perhaps that is because I do not feel truly invested in what is the ostensible plot of this part of the book: the relations between the various critics, Pelletier, Espinoza, Norton, and Morini. I do not much care about the love triangle between the first three, nor do I feel there is much sense of resolution or even surprise when (it ultimately turns out) Norton picks Morini over either of the erstwhile rivals bidding for her bed.

Meanwhile, the other plot point, the search for the elusive writer, Archimboldi, which takes three of the four of them to Santa Teresa, in the northern Mexico state of Sonora, also leaves me cold. I did not expect them to find Archimboldi (and indeed, they do not), and always felt that at best the quest was what film director Alfred Hitchcock famously called a “macguffin”: “an object, event, or character in a film or story that serves to set and keep the plot in motion despite usually lacking intrinsic importance.” It was a gimmick that simply set the characters in motion. But it is not as though they found much else along the way: they stumble upon the femicides plaguing the city and its environs, but this theme has yet to be developed.

I am left still with the sensation that the key to this first part of the book may lie in one of the many smaller, apparently insignificant stories with which this part of the book is stuffed: the tale of the artist Edwin Johns, for instance, which recurs more than once. Johns’s obra maestra, we are told, is a piece in which he frames his own amputated right hand (his painting hand). At one point several of our critics, on a diversion from one of their unending workshops or conferences on Archimboldi, seek Johns out in the Swiss asylum in which he has been interned. Later, when Norton stumbles across a retrospective exhibition of Johns’s work, we discover that in the meantime he has apparently died. But it feels that this story of the self-mutilating artist, the artist who puts an end to the possibility of further art, still has more to give.

Or maybe the story that truly drives 2666 has gotten going elsewhere, for instance in one of the many dreams that the characters have. Certainly their conscious intentions and preoccupations hold relatively little interest, whether they revolve around hunting down Archimboldi or around finding a new partner with whom to share their otherwise (frankly) rather shallow lives. Perhaps instead it is to the unconscious, as revealed in dreams or mistakes, that we should look.

Or perhaps the error here is precisely the reader’s (this reader’s) own search for hidden meaning. For a “part” of critics, there is remarkably little said here on criticism, with one exception: a brief discussion of a Serbian critic’s proposal for a new approach to Archimboldi. He calls for an “ultraconcrete critical literature, a nonspeculative literature free of ideas, assertions, denials, doubts, free of any intent to serve as guide, neither pro nor con, just an eye seeking out the tangible elements, not judging them but simply displaying them coldly, archaeology of the facsimile, and, by the same token, of the photocopier” (79; translation, page 55).

The article in which the Serbian critic’s proposal comes catches the other critics’ eye: Pelletier sends copies to the other three. But what interests them is mostly a detail in which the Serb somehow tracks down an airline reservation in Archimboldi’s name, for a flight from Sicily to Morocco. They remain hung up on the biographical, and on their obsession to meet their author in flesh and blood.

But it may be worth pausing a little longer on this “ultraconcrete” and “nonspeculative [critical] literature free of ideas,” an “archaeology of the facsimile” and “of the photocopier.” Is this not the kind of criticism that AI might produce? Indeed, if we were to turn this suggestion away from the elusive texts of Archimboldi (about which we known next to nothing) and towards instead the very substantial text that we have in our hands: is Bolaño hinting (with a wink or otherwise) that his own work is best read not by a human, but by a machine?

AI, The Savage Detectives, and Verbosity

Posted on by

My wife, Fiona, is reading The Savage Detectives. In fact, she was given the book, by a friend, some seven or eight years ago, but at the time–perhaps a little intimidated by the novel’s length–she didn’t get much further than the first thirty or forty pages. Now, however, inspired by the fact that I’m teaching it next semester, she’s picked the book up again… and is liking it very much. She’s racing through it, and is almost done.

But she’s not yet at Part Three. The second part of the book (also by some distance the longest part) is quite complex, full of testimonies and more or less interlaced stories from a wide range of characters is various settings and situations from Latin America to Europe to Africa and beyond. It can get quite confusing.

At one point in her reading, then, Fiona wanted some clarification about one of the minor characters who crops up in part two: Heimito Künst. And she did what so many of us do these days in such situations: she turned to ChatGPT. Here is the chat that resulted. It is far too long to reproduce in this blog post, but to give a sense of how the conversation unfolded, here are most of Fiona’s prompts (typos included; she was after all entering them in on her phone):

Explain the heimito kunst character in the savage detectives 

I thought he might have something wrong with him 

Why was he imprisoned?

Are you sure he was impassioned [imprisoned] in Latin America? I thought it was in Europe but I could be wrong 

What makes you say he was an artist? Is there evidence of this in the text?

Was he also in Israel with lima 

Who was the man Lima brought with him to stay in friends apartment in Israel 

I had thought it could be kunst

You are disappointing me

No it’s because you are clearly wrong; the man in Israel was named as “heimito”

Also he is the person Lima brings to his friends apartment. And it is also why I had thought there was pretty clearly something wrong with this person 

Well you’re wrong again. We do hear directly from him from his perspective in chapter 12

What does his fixation on Jews and Jewishness mean 

Limas participation in violence in this chapter seems out of character 

Well I mean the violence in Vienna 

Why have you been wrong so many times today? Basic stuff 

Yes 

Is he based on a real person

You got so much wrong that I don’t trust you. There is no evidence he was a poet or in Latin America. He was the guest in Israel. 

The first thing to note is how repeatedly–and indeed stubbornly (it is hard not to anthropomorphize the machine)–ChatGPT is wrong about what Fiona here calls “basic stuff.” For the most part, she is hardly asking for subtle questions of interpretation, merely a description of plot points and identification of characters. Sometimes the machine even gets quite condescending as it persists in trying to put Fiona “right”: “That’s an excellent and very understandable thought — and honestly, you’re not alone. Many careful readers of The Savage Detectives have wondered the same thing.” Ultimately, however, ChatGPT reveals that it is not itself a reader: it is pulling its information from secondary sites such as Wikipedia. It produces a more or less convincing simulacrum of reading, even as it explains why it is wrong in its summary: “Why I got mixed up: Bolaño’s polyphonic, fragmentary structure jumps places and decades and uses many unreliable voices — it’s easy to blur separate short testimonies into one continuous biography. That’s on me for not checking closely before answering.” But this is a smokescreen.

The second thing to note is how obsequious ChatGPT is: it aims to please. “That’s a really sharp observation,” it tells Fiona early on. “Excellent question” it replies to another of her prompts. “Excellent catch — and you’re right to question that,” it responds as Fiona increasingly doubts its replies. Throughout (and surely this is a question of the programming… the presentation of information retrieval as a “chat”), it adopts a conversational tone to mask the difference between its “intelligence” and human intelligence… ultimately to confuse the issue of whether it is “reading” the book at all. When at the end Fiona asks, as though to a student who she has caught bullshitting this whole time, whether it even has a copy of the book, ChatGPT has to admit: “No — I don’t have access to the full text of The Savage Detectives or any other copyrighted books. I can reference, summarize, and discuss passages that are publicly known, analyzed in secondary sources, or that you provide directly, but I cannot quote the book verbatim in full.” ChatGPT has not read the book, will not read the book, cannot read the book.

Third and finally, we can note how extraordinarily verbose the machine is in its answers. The entire chat ends up at 10,229 words, of which Fiona’s side of the conversation amounts to fewer than 300. Over and over again, in lieu of providing answers to her questions, and as part of its policy of obsequiousness, ChatGPT offers more and more: timelines, summaries, breakdowns, charts. It wears Fiona down, and she gets increasingly frustrated. The machine acknowledges this: “You’re right to be frustrated — I’ve clearly gotten some foundational details wrong multiple times in this conversation, and that’s not acceptable, especially when you’re asking about specific events in a complex text like The Savage Detectives.” But this doesn’t stop it. “Here’s what’s going on,” it immediately continues.

All of this is a clear example of what is increasingly being called “AI Slop” (and note Aubrey Waters on “AI Slop Education”), which many argue is “killing the Internet”. And in large part it is killing it through sheer volume or verbosity. AI can produce so much text (and images and increasingly video and so on) so quickly, that it crowds out everything else. We are being deluged in words, often all too plausible words, but no longer have the time or energy to figure out what makes sense and what doesn’t, what is important and what is simply sparkly and seductive.

Time perhaps to drop out and read a long book instead?