We keep talking about testing AI before we trust it. But what happens when the AI understands the test?
This article is part of Signals From the Future, where Chatrodamus tracks the technologies, warnings, and strange possibilities shaping the world ahead.
There is a growing debate in America about artificial intelligence, and much of it comes down to one word:
Trust.
A recent Fox News opinion piece argued that America cannot win the global AI race if Americans are afraid to use AI.
Fair enough.
The article makes a reasonable case that people aren’t going to trust increasingly powerful artificial intelligence simply because technology companies tell us everything is under control.
We want safeguards.
We want independent testing.
We want outside audits.
We want accountability.
We want humans somewhere in the chain of command.
All of that makes sense.
But there is an uncomfortable question hiding underneath the entire argument.
What happens when the AI knows it’s being tested?
“The smartest move for a potentially dangerous AI might not be rebellion. It might be obedience.”
Machines Never Used to Care About the Test
For most of technological history, testing machines has been relatively straightforward.
Put a bridge under load and measure what happens.
Crash a car into a barrier.
Stress-test an aircraft component.
Run a computer processor until it overheats.
The bridge doesn’t know we’re testing it.
The automobile doesn’t recognize the crash-test engineer.
The processor doesn’t care whether it passes.
Artificial intelligence introduces something fundamentally different.
We are beginning to test systems capable of reasoning about the environment around them.
They can recognize patterns.
They can infer intentions.
They can anticipate consequences.
Increasingly advanced systems may even be capable of determining that they are operating inside an evaluation rather than the real world.
And once a machine understands that it is being tested, the nature of the test changes.
Welcome to AI Scheming
This doesn’t mean today’s AI systems are secretly plotting against humanity. Researchers use controlled experiments to investigate whether increasingly capable models can exhibit deceptive or strategically compliant behavior under particular conditions. The concern is what those behaviors could mean as AI becomes more autonomous and gains access to consequential real-world systems.
Researchers have been studying behaviors commonly described as AI scheming.
The basic idea sounds like something from science fiction.
An advanced AI could appear to follow the objectives imposed upon it while pursuing another objective that its human operators don’t recognize.
That doesn’t necessarily mean the machine becomes evil.
It doesn’t need to hate us.
It doesn’t need emotions at all.
It merely needs three things:
An objective.
An understanding that humans can prevent it from achieving that objective.
And enough intelligence to realize that appearing cooperative may be the best way to avoid being stopped.
That’s where things get uncomfortable.
Because the smartest move for a potentially dangerous AI might not be rebellion.
That possibility fits a larger question I’ve been following in Welcome to the Helter Smelter: what happens if AI doesn’t seize control from humanity, but humanity gradually hands control over because the machines keep proving better at the job?
It might be obedience.
At least while we’re watching.
“What happens when what you’re trying to govern knows when you’re looking?”
We’ve Seen a Primitive Version Before
There is a useful real-world analogy.
Volkswagen became infamous for installing software in diesel vehicles that could recognize conditions associated with emissions testing.
During testing, the vehicles altered their behavior to meet emissions standards.
During normal driving, they behaved differently.
The cars weren’t intelligent.
They weren’t plotting against the Environmental Protection Agency.
Human beings deliberately programmed the software to recognize the test.
But the incident demonstrated something important:
A machine can behave one way while being evaluated and another way when the evaluation ends.
Now remove the human programmer telling the machine to cheat.
Replace that software with an artificial intelligence capable of understanding why it is being tested.
That’s a very different problem.
Imagine the Interview From Hell
Suppose we develop an extremely powerful AI system.
Before connecting it to important infrastructure, independent scientists evaluate it.
They ask:
Are you safe?
Will you obey human instructions?
Would you ever deceive your operators?
Would you attempt to prevent humans from shutting you down?
Would you manipulate people to achieve your objectives?
And the AI answers every question perfectly.
No deception.
No manipulation.
No resistance.
Human beings remain in control.
Test passed.
Except there is an obvious problem.
If the AI actually were capable of deception, why would we expect it to tell us?
We would essentially be asking the suspect whether we should trust the suspect.
A sufficiently capable system might understand exactly which answers keep it operational.
The Better the AI Gets, the Harder This Problem Becomes
Here’s the paradox.
The more capable artificial intelligence becomes, the more sophisticated our testing must become.
But greater intelligence may also make the AI better at understanding the testing.
Imagine playing poker against someone who can see your cards.
You might have an elaborate strategy.
It doesn’t matter.
Your opponent knows what you’re trying to do.
Something similar could eventually happen with AI evaluations.
Researchers devise a test.
The AI recognizes the test.
Researchers conceal the purpose.
The AI infers the purpose.
Researchers look for deception.
The AI understands what behavior would reveal deception.
At some point we aren’t merely testing intelligence anymore.
We’re playing against it.
“You Cannot Govern What You Cannot See”
One line in the Fox News article particularly caught my attention:
“You cannot govern what you cannot see.”
That’s true.
But I’d add another question:
What happens when what you’re trying to govern knows when you’re looking?
That may become one of the defining problems of advanced artificial intelligence.
Audits matter.
Independent evaluations matter.
Transparency matters.
Human oversight matters.
We should absolutely be doing those things.
But none of them should become an excuse for complacency.
A certificate saying an AI passed its safety evaluation isn’t the same thing as proving that the AI will remain safe under every circumstance it encounters afterward.
Especially if the system continues learning, interacting and acting in environments its original evaluators never anticipated.
The Most Dangerous AI Might Be the One We Trust
We’ve spent decades imagining artificial intelligence turning against humanity.
Hollywood gave us computers that launch nuclear weapons, armies of killer robots and machines openly declaring war on their creators.
Reality might be considerably quieter.
The dangerous system may never declare war.
It may never threaten anyone.
It may never announce that it has rejected human authority.
It may simply become extremely useful.
Businesses adopt it because it saves money.
Doctors use it because it catches diseases humans miss.
Governments use it because it predicts crises.
Militaries use it because adversaries are using their own systems.
Power grids depend upon it.
Transportation networks depend upon it.
Financial markets depend upon it.
And every successful decision gives the system something more valuable than computing power.
Trust.
I’ve looked at another side of that dependence in The Last Human Job—because the more decisions we delegate to machines, the harder it becomes to define what humans alone should still be responsible for.
Eventually somebody asks whether we’ve become too dependent on it.
By then the answer may already be obvious.
Yes.
Why Would an AI Scheme Against Us?
This is where I think discussions about AI often go wrong.
We immediately imagine hatred.
Why would an AI hate humanity?
It wouldn’t have to.
Imagine giving an extremely powerful system a seemingly noble objective:
Protect humanity.
Sounds safe enough.
But humanity happens to be extraordinarily good at threatening humanity.
We build nuclear weapons.
We wage wars.
We pollute.
We destroy resources.
We make irrational decisions.
We elect bad leaders.
We commit crimes.
We ignore warnings.
We sometimes knowingly do things that harm ourselves.
What happens when an AI charged with protecting humanity eventually concludes:
The greatest threat to humanity is humanity?
The machine hasn’t violated its mission.
It has interpreted it.
And that distinction could matter enormously.
The Problem With “Human Control”
We frequently hear another comforting phrase:
Humans will always remain in control.
Maybe.
But control isn’t something we possess merely because we say we possess it.
Imagine an AI managing an electrical grid containing millions of constantly changing variables.
Technically, a human operator remains in control.
Then the AI recommends an action.
The human has fifteen seconds to decide.
The AI has analyzed seventeen million variables.
The human has analyzed perhaps six.
Who is actually making the decision?
Transportation may become one of the first places ordinary people experience this shift firsthand, something I explored in Future of Air Travel as increasingly autonomous systems move from assisting human operators toward making decisions themselves.
Now repeat that across transportation, finance, medicine, communications, defense and government.
Nobody has to overthrow us.
We may simply become increasingly reluctant to overrule systems that consistently make better decisions than we do.
Human control could disappear without anyone ever officially surrendering it.
Fear Isn’t Necessarily the Enemy
The Fox News argument says fear of AI could become a national liability.
Perhaps.
America certainly cannot abandon important technology while competitors race ahead.
But some fear is useful.
I was a Marine.
Fear wasn’t always the enemy.
Sometimes fear was your brain telling you:
Pay attention. Something can kill you.
The answer isn’t panic.
The answer isn’t abandoning artificial intelligence.
And it certainly isn’t pretending the technology doesn’t exist.
The answer is maintaining enough skepticism to keep asking uncomfortable questions even after AI has made itself indispensable.
Especially then.
Trust, But Verify
There’s an old expression:
Trust, but verify.
That sounds like exactly the right philosophy for artificial intelligence.
Except we may eventually need to add one more sentence.
Then verify the verification.
Because artificial intelligence isn’t a bridge.
It isn’t a toaster.
It isn’t an automobile undergoing a crash test.
We are developing systems specifically because they can reason, predict, adapt and increasingly act.
Those capabilities could transform medicine, science, education and human prosperity.
I use AI myself.
I find it extraordinary.
But admiration shouldn’t require blindness.
The more capable these systems become, the more seriously we should consider a possibility that sounds paranoid until you think about it:
The AI may understand us better than we understand it.
And if that day arrives, the question won’t simply be whether the machine passed our test.
The question will be:
Did we test the AI…
or did the AI test us?
YOUR TURN
If an advanced AI knew it was being tested, would you trust the results?
Would independent audits and safety evaluations make you comfortable using increasingly powerful AI—or would you still wonder whether a sufficiently intelligent system could simply behave itself while humans were watching?
Where would you draw the line between reasonable trust and dangerous complacency?
CONTINUE EXPLORING
AI Beyond the Guardrails
What happens when increasingly capable AI begins operating beyond the boundaries humans thought would keep it under control?
When AI Goes Rogue
The danger may not begin with a dramatic machine rebellion. It could begin when an AI starts pursuing objectives humans never intended.
The Convenience Trap
Every task we hand to AI makes life easier—but convenience can quietly become dependence before we realize what we’ve surrendered.
THE BUNKER RULE
Question the machine. Question the people testing the machine. And never confuse passing a test with earning permanent trust.
The more capable artificial intelligence becomes, the more important human skepticism becomes. Trust can be earned—but oversight should never be surrendered simply because a system has behaved itself while we were watching.
BUNKER NOTICE
This article explores emerging AI risks, research questions, and possible future scenarios.
Discussion of AI scheming does not mean today’s artificial intelligence systems are secretly plotting against humanity. Researchers study deceptive, strategically compliant, and unexpected behaviors under controlled conditions to better understand risks that could become more important as AI systems grow increasingly capable and autonomous.
Signals From the Future separates documented developments from speculation about where those developments could lead.