Can computers think?

Why passing the Turing Test is not enough

Introduction

In the past few years multiple claims have been made that a machine, specifically a large language model (LLM), passed the Turing Test. Turing himself predicted that by the year 2000 machines would play the test well enough that an interrogator would correctly identify them no more than 70% of the time when questioned for five minutes ¹. The Turing Test is widely seen by the public as one of the main metrics to measure the general intelligence of AI Systems, and by extension computers ²³.

One might conclude then that this is proof that the main question of this essay has already been answered and there is nothing more to discuss.

I will argue that passing the Turing Test cannot demonstrate that computers can think.

The Imitation Game

Turing acknowledged that the question "Can machines think?" is too vague to answer meaningfully, since the words "machine" and "think" resist clear definition. Rather than getting stuck in a philosophical debate, he replaced the original question with a concrete test.

An interrogator (C) sits in a separate room and exchanges written messages with two hidden participants A and B, communicating only through typed text so that voice and appearance give nothing away. A is a man and B is a woman and C must determine which is which. The man (A) tries to deceive the interrogator (C) into the wrong identification, while the woman (B) tries to help C identify her correctly.

Turing then proposed: What happens when a machine takes the part of A? The machine succeeds if the interrogator decides wrongly as often as when the game is played between a man and a woman. Essentially this answers the question if a machine can imitate a human convincingly enough that C can no longer reliably tell the difference. Turing named it the Imitation Game. This test does not answer the original question as we can not conclude on the basis of this test that the machine actually behaves like a human but Turing argues that if the machines responses are indistinguishable from the human ones, we have no good grounds to deny it is "thinking" regardless of how it produces those responses ¹.

The Imitation Game's primary purpose is of philosophical nature to replace an elusive question with an operational one rather than to define an engineering benchmark. It enabled Turing to argue in favor of machines eventually being capable of thinking.

Its intuitive appeal and the later label "Turing Test" contributed to a popular-culture shift in which it is often treated as a definitive measure of general intelligence ²³.

Adequacy as a measure

Searle formulated the Chinese Room as a critique of the simplification that mere imitation is enough to prove that a computer can think.

The Chinese Room argument builds a scenario in which an English native speaker is locked in a room with appropriate instructions that enable them to appear to the outside world like a Chinese native speaker despite not understanding any Chinese. This construction makes it impossible for a third party to decide if the human in the room can understand Chinese or is just mindlessly following instructions based just on the observations alone. Searle argues on this basis that imitation itself is no proof that a system understands as formal symbols carry no intrinsic meaning ⁴.

Harnad

Passing the Turing Test

With the invention and recent improvements of LLMs humans are capable of building machines that are able to pass the Turing Test. Jones & Bergen showed in their paper that GPT 4.5 with a humanlike persona was judged to be the human 73% of the time, significantly more often than interrogators selected the real human participant. Without the persona prompt its win rate dropped to about 36% ⁵. As a result we can conclude that LLMs are capable of passing the Turing Test under controlled conditions.

The inability of LLMs

Systems, which are trained on text alone can never learn meaning, because meaning arises from the relationship between language and the surrounding world ⁶.

LLMs are trained by feeding astronomically large amounts of text into Transformer Models. They were never designed to think but to predict the next token based on statistical patterns. I argue that they serve as the realization of the hypothetical digital computer mentioned in the Chinese Room argument where the rule book is given by the trained weights, the Chinese symbols are equivalent to the processed tokens and the room being the transformer architecture.

Conclusion

It seems like the focus on Turing Tests was pointless and we gained nothing from it. This is false because we now have strong reasons to think that mere imitation is not equivalent to thinking however we may define this term. Following the argument of equivalence of LLMs to the hypothetical digital computers in the Chinese room thought experiment we can deduce that next-token prediction alone is not thinking. A truly intelligent system has to have access to more knowledge about the world than just in textual form following the theory of symbol grounding. There is also a need for better evaluation methods to test the intelligence of improved LLMs or completely novel systems.

---