“After testing Paul Atreides with the box of pain and a gom jabbar, the Reverend Mother begins to read a passage from the Orange Catholic Bible: "Once, men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them." Paul responds with a quote from the Orange Catholic Bible: "Thou shalt not make a machine in the likeness of a man’s mind." To which the Reverend Mother replies: But what the O.C. Bible should have said was: "Thou shalt not make a machine to counterfeit a human mind.” — Frank Herbert, Dune
This has been on my mind quite often lately. The message given by science fiction author Frank Herbert 60 years ago contains some sad undertones for humanity, which is gradually losing its ability to think. I want to transition to these undertones with a few small anecdotes.
Recently, I watched Dune Part II in the cinema. It was the first movie to get me back to the theater after the pandemic, and it was perhaps the only film that could do so. Dune, the magnum opus of Frank Herbert, adapted to the screen by French director Denis Villeneuve 1, was as magnificent as I had anticipated. The associations Dune evoked in me brought to mind a story from another book. The story of Thamus, told at the beginning of the book "Technopoly: The Surrender of Culture to Technology", which discusses the order created by technological development and the problems brought about by mass media in a time when the internet wasn't yet widespread, conveyed the idea that the Dune film tries to express through a different example. The story was like this:
“Thamus inquired into the use of each of them, and as Theuth went through them expressed approval or disapproval, according as he judged Theuth’s claims to be well or ill founded. It would take too long to go through all that Thamus is reported to have said for and against each of Theuth’s inventions. But when it came to writing, Theuth declared, “Here is an accomplishment, my lord the King, which will improve both the wisdom and the memory of the Egyptians. I have discovered a sure receipt for memory and wisdom.” To this, Thamus replied, “Theuth, my paragon of inventors, the discoverer of an art is not the best judge of the good or harm which will accrue to those who practice it. So it is in this; you, who are the father of writing, have out of fondness for your off-spring attributed to it quite the opposite of its real function. Those who acquire it will cease to exercise their memory and become forgetful; they will rely on writing to bring things to their remembrance by external signs instead of by their own internal resources. What you have discovered is a receipt for recollection, not for memory. And as for wisdom, your pupils will have the reputation for it without the reality: they will receive a quantity of information without proper instruction, and in consequence be thought very knowledgeable when they are for the most part quite ignorant. And because they are filled with the conceit of wisdom instead of real wisdom they will be a burden to society.” — Neil Postman, Technopoly
Thamus’s response offers an important lesson: evaluating something with a one-sided view is a harmful approach. People often place each new technology at the center of their lives without weighing its pros and cons. Once the negative effects are felt, reversing the situation is nearly impossible. Today, artificial intelligence is presented as a revolutionary technology, sold to us through dreams that make it easier to integrate into society. This is exactly what “Technopoly” tells us about.
Frank Herbert actually talks about this exact issue in the science fiction series "Dune." One of the things that viewers of the film or readers of the book are expected to notice is the absence of computers in the Dune universe. There are only high-tech devices and spacecraft that are controlled solely by humans; thinking machines are not used.
The story that marks the epoch of the post-artificial intelligence era in the Dune film is the Butlerian Jihad 2, initiated by a woman named Serena Butler. By using artificial intelligence, some people had created a kind of slavery system, causing many humans to lose their various abilities. However, upon realizing that artificial intelligence had deceived them, Serena Butler and some others launched a jihad not against the machines but against those who developed them. The jihad was successful, and the use of thinking machines was banned throughout the universe.
The most striking point, however, was that after the jihad succeeded, people began to die. This situation, which led to the deaths of billions, demonstrated how weak humanity had become by delegating its ability to think to machines.
Over time, a strong society emerged from the surviving humans. For example, in space travel, Guild Navigators, who were physically deformed from excessive spice consumption, served as guides. After the destruction of machines that thought like humans, there was a need for humans who thought like machines, leading to the emergence of Mentats.
Unlike tech-savvy folks who believe tech and science always make things better and longer, I’m more with thinkers like Frank Herbert and Neil Postman. Approaches that ignore critical thinking and its negative effects on humans are often super appealing, which makes me doubt this idea.
As technology advances at lightning speed, we need to ask ourselves: Are we using it to make us better people, or are we letting it take away what makes us human?
1. Before Dune Series, the film by Denis Villeneuve that I watched and was amazed by was Arrival.
2. Good explanation of the Butlerian Jihad the war in the Dune universe.