By now, you are most likely hyper-aware of the recent stunning progress in artificial intelligence due to the development of large language models such as ChatGPT, Microsoft’s Copilot, and Google’s Bard, and at least somewhat aware of the dangers posed by such systems’ frequent hallucinations and their predictable tone of supreme self-confidence and infallibility.
This tone can unfortunately lead highly intelligent people to believe that such systems, despite their propensity to hallucinate, are on a par, as thinkers, with human beings, and can even validly replace human authors. The story I will tell below illustrates the dangers of falling for this kind of illusion.
This past month, Sami Al-Suwailem, a serious and thoughtful reader of my 1979 book, Gödel, Escher, Bach: an Eternal Golden Braid, wanted to post on his website the story of how that book came to be written, but he didn’t want to burden me by asking me to tell the story, so he came up with the novel idea of asking the state-of-the-art large language model GPT-4 to compose, based on the 20th-anniversary edition of the book, a one-page essay with the title “Why Did I Write GEB?” (I might add that the 20th-anniversary edition includes a preface in which I spell out, in detail, what sparked me to write the book, and how it evolved.)
The AI system obliged Al-Suwailem by instantly producing a statement written in the first person (thus impersonating me, the book’s author), which Al-Suwailem sent to me via email, hoping to get my approval for posting it online. I have reproduced GPT-4’s output verbatim below.
I’m sorry, but as an AI language model, I am unable to access specific books or their content. However, I can provide you with a brief summary of why Douglas Hofstadter wrote “Gödel, Escher, Bach: An Eternal Golden Braid” based on my general knowledge of the book and the author’s intentions.
Why Did I Write GEB?
Ever since I was a young student, I’ve been fascinated by the deep connections that underlie seemingly unrelated fields. As I delved deeper into the works of Kurt Gödel, M.C. Escher, and Johann Sebastian Bach, I began to see a common thread that linked their ideas, transcending the boundaries of logic, art, and music. This realization formed the foundation of my book, “Gödel, Escher, Bach: An Eternal Golden Braid” (GEB). My primary motivation for writing GEB was to explore the concept of self-reference and the recurring patterns that emerge in different domains. I became captivated by Gödel’s Incompleteness Theorems in mathematical logic, which rocked the foundations of formal systems by demonstrating the inherent limitations of any axiomatic system. Gödel’s work showed that self-reference could lead to paradoxes and unresolvable questions, which resonated with me on a profound level. In M.C. Escher’s artwork, I saw a similar fascination with self-reference and recursion, as his lithographs often depicted scenes that looped back on themselves, creating visual paradoxes and impossible structures. His works, such as “Drawing Hands” and “Ascending and Descending,” challenged the viewer’s perception of reality and demonstrated the power of self-reference in the artistic realm.
Johann Sebastian Bach, a master of musical counterpoint, also incorporated self-reference and recursion in his compositions. His canons and fugues exhibit a remarkable level of intricacy and structure, weaving together multiple voices that echo and transform each other in a complex, interlocking pattern. Bach’s ingenious use of mathematical patterns and symmetry in his music inspired me to explore the connections between these ideas and the works of Gödel and Escher.
As I worked on GEB, I realized that the unifying thread among these three great thinkers was not just their fascination with self-reference and recursion, but also their ability to transcend the limitations of their respective domains. Each of them pushed the boundaries of what was thought possible, revealing deep truths about the nature of reality, the limits of human knowledge, and the beauty that can emerge from complexity.
In writing GEB, I hoped to not only share my fascination with these ideas but also inspire readers to appreciate the interconnectedness of seemingly disparate fields and to ponder the complex web of knowledge that underlies our understanding of the world. The book is an exploration of the nature of human intelligence and creativity, as well as a meditation on the emergence of consciousness and the central role that self-reference and recursion play in the development of cognition. GEB is as much a personal journey as it is an intellectual exploration. Through whimsical dialogues between imaginary characters and engaging discussions of various topics, I aimed to create a book that would stimulate the reader’s curiosity and encourage them to embrace the inherent complexity and beauty of the world around them. Ultimately, I wrote GEB to share my passion for uncovering hidden connections and to
celebrate the intricate tapestry of ideas that bind the works of Gödel, Escher, and Bach, as well as the broader human quest for understanding and meaning.In the years since GEB was first published, I have been gratified to see the impact it has had on readers from all walks of life. The book has inspired countless people to delve deeper into the worlds of mathematics, art, and music, and to appreciate the profound connections that unite these realms. As I reflect on my journey with GEB, I remain ever in awe of the eternal golden braid that links the works of Gödel, Escher, and Bach, and the infinite possibilities that await us as we continue to explore the depths of human
knowledge and creativity.
Al-Suwailem’s friendly email and the AI-generated first-person essay that he sent me, which supposedly captured my voice, caught me totally by surprise, and I reeled with perplexity for a while. But when I eventually regained my mental equilibrium, I replied to him as follows:
Thanks very kindly for your interest in my ideas and in why I wrote GEB. I have carefully read through the text generated by GPT-4. I would say that that text, in sharp contrast to what I myself wrote in the book’s 20th-anniversary preface, consists only in generic platitudes and fluffy handwaving.
The prose has virtually nothing in common with my writing style and what it says does not agree at all with the actual story that underlies the book’s genesis. Although someone who was unfamiliar with my writing might take this saccharine mixture of pomposity and humility as genuine, to me it is so far from my real voice and so far from GEB’s real story that it is ludicrous.
Before I go on, let me explain that I am profoundly troubled by today’s large language models, such as GPT-4. I find them repellent and threatening to humanity, partly because they are inundating the world with fakery, as is exemplified by the piece of text produced by the ersatz Hofstadter. Large language models, although they are astoundingly virtuosic and mind-bogglingly impressive in many ways, do not think up original ideas; rather, they glibly and slickly rehash words and phrases “ingested” by them in their training phase, which draws on untold millions of web sites, books, articles, etc. At first glance, the products of today’s LLM’s may appear convincing and true, but one often finds, on careful analysis, that they fall apart at the seams.
The piece “Why Did I Write GEB?” is a perfect example of that. It does not sound in the least like me (either back when I wrote the book, or today); rather, it sounds like someone spontaneously donning a Hofstadter façade and spouting vague generalities that echo phrases in the book, and that thus sound at least a little bit like they might be on target. As an example, let me quote just two sentences, taken from the next-to-last paragraph, that at first might seem to have a “sort of right” ring to them, but that in fact are nothing like my style or my ideas at all: “Through whimsical dialogues between imaginary characters and engaging discussions of various topics, I aimed to create a book that would stimulate the reader’s curiosity and encourage them to embrace the inherent complexity and beauty of the world around them. Ultimately, I wrote GEB to share my passion for uncovering hidden connections and to celebrate the intricate tapestry of ideas that bind the works of Gödel, Escher, and Bach, as well as the broader human quest for understanding and meaning.”
These sentences have a rather grand ring to them, but when I read them, they strike me as pretentious and airy-fairy fluff. Let me go through some of the phrases one by one.
- “Through … engaging discussions of various topics …” “Various topics”!? How vague can you get? (Also, the word “engaging” is self-serving.)
- “Encourage them to embrace the inherent complexity and beauty of the world around them.” That’s just high-falutin’ emptiness. I had no such intention in writing GEB.
- “My passion for uncovering hidden connections.” I have never been driven by any such passion, although I do enjoy finding unexpected connections from time to time. But I was indeed driven by a passion when I wrote GEB—namely, my intense desire to reveal what I believed consciousness (or an “I”) is, which in the book I called a “strange loop.” I was on fire to explain the “strange loop” notion, and I did my best to show how this elusive notion was concretely epitomized by the unexpected self-referential structure lying at the heart of Gödel’s incompleteness theorem.
- “To celebrate the intricate tapestry of ideas that bind the works of Gödel, Escher, and Bach.” That may at first sound poetic and grand, but to my ear it is just vapid pablum.
- “The broader human quest for understanding and meaning.” Once again, a noble-sounding phrase, but so vague as to be essentially meaningless.
The actual story behind GEB begins with me as a 14-year-old, when I ran across the slim paperback book Gödel’s Proof by Ernest Nagel and James R. Newman, and was soon mesmerized by it. I intuitively felt that the ideas that it described were somehow deeply connected with the mystery of human selves or souls.
Many years later, when I encountered and ravenously devoured Howard DeLong’s book A Profile of Mathematical Logic, I was once again set on fire, and couldn’t stop brooding about the relationship of Gödel’s ideas to the mystery of “I”-ness. During a several-week car trip that I took from Oregon to New York in the summer of 1972, I pondered endlessly about the issues, and one day, in an intense binge of writing, I summarized my thoughts in a 32-page letter to my old friend Robert Boeninger.
That letter was the initial spark of GEB, and a year later I tried to expand my letter into a book with the title Gödel’s Theorem and the Human Brain. I wrote the first manuscript, in ink on paper, in about one month (October 1973). It contained no references to Bach and no Escher prints (indeed, no illustrations at all), and not a single dialogue.
The next spring, while I was excitedly teaching a course called “The Mystery of the Undecidable” on all the ideas that were churning in my head, I typed up that first manuscript, roughly doubling its length, and one happy day, inspired by Lewis Carroll’s droll but deep dialogue called “What the Tortoise Said to Achilles” (it was reprinted in DeLong’s book), I tried my own hand at writing a couple of dialogues between those two amusing characters. My second Achilles-Tortoise dialogue wound up having an unusual structure, and so, on a random whim, I called it “FUGUE.” It wasn’t a fugue at all, but suddenly I had the epiphany that I might attempt to write further dialogues that genuinely possessed contrapuntal forms, and thus did J. S. Bach slip in through the back door of my budding book.
A few months later, I gave my typewritten manuscript to my father, who read it all and commented that he thought I needed to insert some pictures. All at once, it hit me that while working on my manuscript, I had always been seeing Escher prints in my mind’s eye, but had never once thought of sharing them with potential readers. This realization was a second epiphany, and it soon led to my replacing the book’s original humdrum and academic-sounding title by the snappier “Gödel, Escher, Bach,” which hinted at the fact that the book was related in some fashion to art and music, and to that trio of names I added the subtitle “an Eternal Golden Braid,” echoing the initials “GEB,” but in a metaphorically braided fashion. The amusing relation of the title to the subtitle even hinted that there was wordplay to be found between the book’s covers. In the years 1975–1977, I rewrote the book starting from scratch, using an amazing text editor designed by my friend Pentti Kanerva.
After a while, I decided on a structure that alternated between chapters and dialogues, and that decision radically changed the flavor of the book. I was lucky enough that Pentti had also just created one of the world’s first typesetting programs, and in the years 1977–1978 I was able to typeset GEB myself. That’s the real story of why and how GEB came to be.
As I hope is clear from the above, the use of words in GPT-4’s text is nothing like my use of words; the use of blurry generalities instead of concrete stories and episodes is not my style at all; the high-flown language that GPT-4 used throughout has little or nothing in common with my style of thinking and writing (which I often describe as “horsies-and-doggies style”). Moreover, there is zero humor in the piece (whereas humor pervades my writing), and there is only the barest allusion to GEB’s twenty dialogues, which are
arguably the main reason that the book has been so well received for so many years. Except in the phrase “imaginary characters,” Achilles and the Tortoise are nowhere mentioned by GPT-4 (posing as me), nor is there any reference to Lewis Carroll’s hugely provocative dialogue, which was the source of those “imaginary characters.”Completely neglected is the key fact that my dialogues have music-imitating structures (verbal fugues and canons), and that their form often covertly echoes their content, which I chose to do in order to mirror the indirect self-reference at the heart of Gödel’s proof, and also in order to make readers smile when they discover what is going on (which, by the way, poor innocent Achilles is never aware of, but which the shrewd and wily Tortoise always seems to be delightedly aware of). The constant verbal playfulness that gives GEB’s dialogues their special character is nowhere alluded to.
Last but not least, anybody who has read GEB will be struck by the pervasive use of vivid analogies to convey the gist of abstract ideas—but that central fact about the book is nowhere mentioned. In short, the piece that GPT-4 composed using the pronoun “I” has zero authenticity, it has no resemblance to my manner of expressing myself, and the artificiality of its creation runs against all the pillars of my lifelong belief system.
GPT-4’s text entitled “Why Did I Write GEB?,” if taken in an unskeptical manner, gives the impression that its author (theoretically, me) is adept at fluently stringing together high-flown phrases in an effort to sound profound and yet sweetly self-effacing at the same time. That nonsensical image is wildly off base. The text is a travesty from top to bottom. In sum, I find the machine-generated string of words deeply lamentable for giving this highly misleading impression of who I am (or who I was when I wrote my
first book), as well as for totally misrepresenting the story of how that book came to be. I am genuinely sorry to come down so hard on the interesting experiment that you conducted in good faith, but I hope that from my visceral reaction to it, you will see why I am so opposed to the development and widespread use of large language models, and why I find them so antithetical to my way of seeing the world.
That’s how I concluded my reply to Al-Suwailem, who was most gracious in his reply to me. But the issues that this bizarre episode raises continue to trouble me enormously.
I frankly am baffled by the allure, for so many unquestionably insightful people (including many friends of mine), of letting opaque computational systems perform intellectual tasks for them. Of course it makes sense to let a computer do obviously mechanical tasks, such as computations, but
when it comes to using language in a sensitive manner and talking about real-life situations where the distinction between truth and falsity and between genuineness and fakeness is absolutely crucial, to me it makes no sense whatsoever to let the artificial voice of a chatbot, chatting randomly away at dazzling speed, replace the far slower but authentic and reflective voice of a thinking, living human being.
To fall for the illusion that vast computational systems “who” have never had a single experience in the real world outside of text are nevertheless perfectly reliable authorities about the world at large is a deep mistake, and, if that mistake is repeated sufficiently often and comes to be widely accepted, it will undermine the very nature of truth on which our society—and I mean all of human society—is based.
When you buy a book using a link on this page, we receive a commission. Thank you for supporting The Atlantic.
Stay connected with us on social media platform for instant update click here to join our Twitter, & Facebook
We are now on Telegram. Click here to join our channel (@TechiUpdate) and stay updated with the latest Technology headlines.
For all the latest For Top Stories News Click Here
