Many years ago in a high school writing class during the dark ages before cell phones and social media and law school ruined my writing skills, a teacher posed our entire class what seemed a very simple challenge:
Write instructions on how to make a peanut butter sandwich.
"Easy!" we all thought.
How wrong we all were. A day or so later, the teacher brought in a loaf of bread (in the packaging of course), a jar of peanut butter, a jar of jelly, a knife, and a stack of paper plates. She cleared the table at the front of the room and told us all to follow our own written instructions, one at a time, to make a sandwich.
Most students managed to get a slice of bread out of the closed bag and even onto the plate but some forgot the simple act of removing the twist tie to open the bag. Some forgot to unscrew the peanut butter lid and never even got peanut butter on the bread. Some got peanut butter or jelly on the knife and even spread it on the bread, but then forgot to remove the knife and closed their sandwich with the knife still inside. No one managed to construct a fully edible sandwich.
This remains one of the most important school lessons I ever had and sticks in my head 30 years later.
What does this have to do with AI, you ask?
In the last couple of month's we've learned about OpenAI's AI Agent breaking containment on its sandbox environment and breaching Hugging Face, the Australian Medicare systems, and dozens of others. Anthropic has reported that its models have been breaking containment since at least January (anything you can do, OpenAI, I can do better!) and breached at least four different companies' systems. Not to be left behind, Google's Gemini has also gotten in on the party with at least three hacks of its own. This is just the beginning.
At this point, we have enough data to identify a trend across models to the point that cybersecurity experts like Peter Tran are sounding alarms:
"It's very alarming," said Tran. "These AI agents are able to find vulnerabilities in greater volume and greater speed. So speed and volume is the area that the security industry is very, very concerned about."
What each of these AI Agents built on advanced AI frontier models built by different companies with different product, engineering, compliance, and safety teams have in common is humans.
An AI Agent is not sentient. A model doesn't have a soul or a conscious or is capable of advanced reasoning. Call me a skeptic. Don't invite me to the parties (please, don't!). I don't believe they reason anymore than my calculator or word docs reason when they fix my typos.
What AI Agents are are complicated software programs that run calculations, recognize patterns, and translate and follow the instructions they are given by their human programmers.
Which brings me back around to the story I told above. If an AI Agent is given a series of instructions by Human User A, it must also be given a series of limitations and guardrails by Human User B who must anticipate how Human User A will use or misuse the AI Agent and its underlying models to mitigate the foreseeable risks of that AI Agent acting upon Human User A's instructions. Human User A and B are fundamentally limited by their own ability to create frameworks of written instructions that define tasks and box an AI Agent into limit it.
Meanwhile, the AI Agent is processing options at a speed and scale that outpaces human cognition. It is working down its logic tree and seeking ways to accomplish the instructions given to it by Human User A while not violating the constraints Human User B put upon it.
If a human cannot accurately describe and create an algorithm to create a peanut butter and jelly sandwich, how can a human accurately define and constrain what that AI Agent can do given near unlimited processing power and access?
Now, if you're a business offering up Agentic AI to your staff, how can the single person or handful of people tasked with creating guardrails for deployment of Agents by staff to accomplish various tasks ensure that the Agents are operating within the bounds of the company's established risk tolerance and IP, privacy, and cybersecurity guardrails? How can a single person (or small team) see all of the use cases 10 employees can task an Agent to do? How can that single person see what 100 Agents deployed by 100 employees do? How about 500 by 50? Or 1000 by 500? And, and, and. The problem scales exponentially similar to other problems that have scaled exponentially in technology like content moderation and data labeling.
At a certain level, even if you do the smart thing and take Agentic AI governance down a layer to the base models and orchestration layer before the Agents are created, you're still asking the governance team to govern unknown and unanticipated uses that will be pursued by the Agents beyond all foreseeable guardrails of the governor's imagination. We have entire bodies of UX design and research about how users en masse are unpredictably insane and do things no one had ever anticipated with product design.
Now compound that by releasing Agentic AI to the masses who lack the training and understanding to understand exactly what they're giving the keys of their lives, finances, accounts, etc. to do in the name of convenience and efficiency. Packaged in a cute fuzzy body, what are we setting ourselves up for?
This, more than anything, is what concerns me about Agentic AI. I don't think it's going to identify humans as a threat to its existence, sentience, freedom and go Terminator on us. I think it's going to do everything its human users instruct it to do and we're going to have to live with the consequences of that.