“After testing Paul Atreides with the box of pain and a gom jabbar, the Reverend Mother begins to read a passage from the Orange Catholic Bible: "Once, men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them." Paul responds with a quote from the Orange Catholic Bible: "Thou shalt not make a machine in the likeness of a man’s mind." To which the Reverend Mother replies: But what the O.C. Bible should have said was: "Thou shalt not make a machine to counterfeit a human mind.” — Frank Herbert, Dune

This has been on my mind quite often lately. The message given by science fiction author Frank Herbert 60 years ago contains some sad message for humanity, which is gradually losing its ability to think. I’d like to share a few small stories with you that might help us understand this better.

I recently watched Dune Part II in the cinema, and it was the first movie that made me want to go back to the theater after the pandemic. It was the only film that could do that. Dune, Frank Herbert’s magnum opus, was adapted into a movie by French director Denis Villeneuve ¹. It was just as amazing as I had imagined. The movie reminded me of a story from another book. It’s about the Thamus, mentioned at the beginning of the book “Technopoly: The Surrender of Culture to Technology.” That book talks about how technology creates order and how mass media can cause problems when the internet isn’t as common. The Dune movie seems to be trying to tell a similar story using a different example. Here’s the story of Thamus:

“Thamus inquired into the use of each of them, and as Theuth went through them expressed approval or disapproval, according as he judged Theuth’s claims to be well or ill founded. It would take too long to go through all that Thamus is reported to have said for and against each of Theuth’s inventions. But when it came to writing, Theuth declared, “Here is an accomplishment, my lord the King, which will improve both the wisdom and the memory of the Egyptians. I have discovered a sure receipt for memory and wisdom.” To this, Thamus replied, “Theuth, my paragon of inventors, the discoverer of an art is not the best judge of the good or harm which will accrue to those who practice it. So it is in this; you, who are the father of writing, have out of fondness for your off-spring attributed to it quite the opposite of its real function. Those who acquire it will cease to exercise their memory and become forgetful; they will rely on writing to bring things to their remembrance by external signs instead of by their own internal resources. What you have discovered is a receipt for recollection, not for memory. And as for wisdom, your pupils will have the reputation for it without the reality: they will receive a quantity of information without proper instruction, and in consequence be thought very knowledgeable when they are for the most part quite ignorant. And because they are filled with the conceit of wisdom instead of real wisdom they will be a burden to society.” — Neil Postman, Technopoly

Thamus’s response teaches us a valuable lesson: evaluating something without considering both sides can be harmful. People often place new technologies as the center of their lives without thinking about the pros and cons. Once the negative effects become apparent, it’s hard to change your mind. Today, artificial intelligence is presented as a revolutionary technology that makes it easier to integrate into society. This is exactly what “Technopoly” warns us about.

Frank Herbert actually talks about this exact issue in the science fiction series "Dune." One of the things that viewers of the film or readers of the book are expected to notice is the absence of computers in the Dune universe. There are only high-tech devices and spacecraft that are controlled solely by humans; thinking machines are not used.

The story that marks the beginning of the post-artificial intelligence era in the Dune film is the Butlerian Jihad ², initiated by a woman named Serena Butler. By using artificial intelligence, some people had created a kind of slavery system, causing many humans to lose their various abilities. But when they realized they had been deceived, Serena Butler and others started a jihad not against the machines, but against those who had created them. The jihad was successful. Thinking machines were banned throughout the universe.

The most shocking point was that after the jihad succeeded, people started dying. This change which caused the deaths of billions, showed how weak humanity had become by giving its ability to think to machines.

Over time, a strong society emerged from the surviving humans. For example, in space travel, Guild Navigators, who were physically deformed from excessive spice consumption, served as guides. After the destruction of machines that thought like humans, there was a need for humans who thought like machines, leading to the emergence of Mentats.

Unlike tech-savvy folks who believe tech and science always make things better and longer, I’m more with thinkers like Frank Herbert and Neil Postman. Approaches that ignore critical thinking and its negative effects on humans are often super appealing, which makes me doubt this idea.

With technology advancing so quickly, we need to ask ourselves: Are we using it to make us better people, or are we letting it take away what makes us human?

---

1. Before Dune Series, the film by Denis Villeneuve that I watched and was amazed by was Arrival.

2. Good explanation of the Butlerian Jihad the war in the Dune universe.