The Turing Test and What Replaced It
This page is a structured working draft — real analysis, not yet expanded with the full expert sourcing given to the flagship pages. Safe to build on; treat specifics as provisional until sourced.
The original test
Alan Turing’s 1950 “imitation game” asked whether a human judge, conversing by text, could reliably distinguish a machine from a person. For decades it stood in as shorthand for “real” machine intelligence.
Why it stopped being useful
Modern language models pass casual versions of the Turing Test routinely, yet no serious researcher considers that evidence of AGI. The test measures conversational indistinguishability, which turns out to be achievable through fluent pattern-matching rather than general reasoning or robust world-modeling. A system can talk like a person and still fail at tasks a person would find trivial, like maintaining a consistent plan across a long, novel task.
What replaced it
Current evaluation approaches focus on:
- Task-horizon benchmarks — how long and complex a chain of actions a system can complete autonomously before failing.
- Held-out, contamination-resistant reasoning tests — problems constructed so the answer can’t have been memorized from training data.
- Agentic benchmarks — evaluating a system operating tools, writing and running code, and correcting its own errors across many steps, not just answering a single question.
- Economically grounded evaluations — can the system complete a real, paid task end-to-end at professional quality, not just answer a quiz question about it.
The honest state of the art
No single benchmark is accepted as the AGI test the way the Turing Test once was treated. That’s arguably healthier: a portfolio of hard, adversarially maintained benchmarks is much harder to game than one famous parlor trick.