When I was growing up, a PSA used to run before the news every night. “It’s 10 p.m. Do you know where your children are?” It ran for decades, a nightly gut check for parents.
Now, I’d ask CEOs the same question about their AI agents.
Do you know where your agents are?
A.k.a., do you know what’s operating on your behalf in your business right now?
I bet many wouldn’t be able to answer confidently.
I’m not suggesting AI agents are children. They can make decisions on their own, they don’t have a bedtime, and they never ask to borrow the car.
But recent AI developments have convinced me leadership needs to be watching them more closely: what they can access, what they’re allowed to do, and who is accountable for them.
Some agents are formally reviewed before deployment. Others arrive inside a workflow platform, a browser plug-in, or a quick script written by an employee trying to save time.
OpenAI benched its own model
On August 7, OpenAI said its next model, code-named Astra, had become capable enough at autonomous hacking that the company could not rule out its worst-case risk category.
In that scenario, the model could find and exploit security holes in real systems without human direction.
They paused parts of the work and added new safeguards. The decision was unusual enough to draw immediate coverage from Axios and TechCrunch.
When a company benches its own product, I pay attention.
Salesforce data: the average company runs 12 AI agents
According to Salesforce’s 2026 Connectivity Benchmark Report, the average company already runs about 12 AI agents. That number is expected to hit 20 by 2027.
Half of those agents operate in isolation, disconnected from other systems.
In plenty of cases, they operate without much oversight either. Deloitte found that roughly 80% of companies still lack a mature governance model for the agents they have already deployed.
An agent that routes tickets or reconciles invoices can do useful work, often faster than a person. The risk rises when no one has defined what it may access, what actions it may take, or who must review its work.
The FDA allows AI to diagnose without a doctor reviewing each result
Now, you may be wondering why I’m reaching back to a 2018 FDA decision.
The first two stories show companies struggling to decide how much autonomy is too much. This one shows what it looks like when the boundary is clear.
Since 2018, the FDA has allowed IDx-DR, now called LumineticsCore, to diagnose diabetic retinopathy from a retina photo without a doctor reviewing each result. Two more systems have since won the same approval.
Regulators approved it after evidence showed that, on this specific task, it caught more cases than trained specialists had in earlier studies.
This decision from the medical industry offers a useful model for technology companies: grant autonomy task by task, and only where the evidence supports it.
Takeaway 1: Protect the judgment needed to supervise AI
For consequential work, autonomous AI needs someone who can recognize a bad answer before it becomes a bad decision.
A recent study had people solve logic puzzles with and without AI help. The more they leaned on the AI, the worse they got at solving them without it.
Meanwhile, the agents themselves are getting better and improving on their own. One example is the Darwin Gödel Machine, which rewrote its own code to improve how it edited files and checked its work.
This creates a risk: people may lose the judgment needed to supervise AI as the systems they oversee become more capable.
One fix: make people form their own answer before they see what the AI produced. Compare the two.
Think of it like a new analyst on your team. They can build a beautiful chart with a conclusion that makes no sense.
Before I let an analyst run numbers for me, I make them walk me through their logic first. That way, they catch their own mistakes before I do.
Whoever owns an AI agent needs that same habit.
Takeaway 2: Give each agent a different rope
I have a well-earned reputation on my team for disliking bureaucracy.
Autonomous AI is making me rethink that.
The same capability that defends your company can attack it. Assume bad actors will try.
Not every use carries the same risk.
Here’s a simple decision tree to help you work through that, one agent at a time.
AI drafting an email is different from AI sending one. Forecasting cash flow is different from wiring it. Match your controls to the consequences of getting it wrong.
In practice, you need to:
- Limit each agent’s access to what the task requires
- Monitor what the agent does
- Require a named human to approve any action involving customers, money, or sensitive data.
A few examples:
Use case | Let AI do | Human must |
Personnel decisions | Summarize performance data | Make the call, document the reasoning |
Financial transactions | Prepare the transaction | Approve payments above a set threshold |
Work assigned to new hires | — | Learn it manually first, then speed up with AI |
Sales forecasting | Produce the forecast | Review any change big enough to affect hiring, spending, or targets |
Marketing | Draft, refine, distribute | Set the message and verify every claim |
Takeaway 3: Find every agent, name an owner, repeat
Start by finding every agent already running in your company. Not just the tools you paid for. Anything that can act without a person directing each step.
For each one, write down three things: what it does, what it can see, and what it’s allowed to do. Then name a human owner.
Sort them by consequence. An agent summarizing documents can run alone. An agent moving real money needs human approval every time.
Set a review cadence according to the agent’s access and the consequences of an error. Re-audit it whenever its access changes. A harmless assistant can become high-risk when it connects to payroll.
Your agents need clear limits, and leadership needs to set them
AI was introduced to most businesses as a tool humans would be in charge of. Agents are beginning to change that.
Now a model has become capable enough at autonomous hacking that its own maker slowed it down.
Leadership’s job is to decide, task by task, how much autonomy the evidence supports. Limit access, monitor activity, and require human approval where the consequences are high. Protect the judgment people need to catch what AI misses.
With clear limits, you can automate low-risk work while keeping people responsible for the results.
Every kid eventually earns a later curfew and the car keys. But only once they've shown they can handle it. Give your agents the same deal: access they've earned.
It's 10 p.m. Do you know where your agents are?
.avif)
.avif)
