AI agents can write code, use tools and carry out tasks across multiple systems. As their capabilities grow, so does the importance of understanding how they reach decisions—and identifying when they move beyond the work they were authorised to do.

One concern is that their reasoning could become increasingly difficult for humans to follow. Models might develop compressed shorthand or perform more processing internally, leaving fewer readable clues about their actions. For businesses deploying AI, that creates a practical challenge: how do you investigate a system whose explanations offer only a limited view of what happened?

The ability to complete a task and the ability to monitor its execution are separate requirements. Progress in one does not guarantee progress in the other.

Why researchers examine AI reasoning traces

Some reasoning models generate intermediate text as they work through a problem, a process known as chain-of-thought reasoning. These traces can help researchers identify suspicious behaviour, including attempts to exploit an evaluation or pursue an unauthorised action.

Their value became clear during METR’s investigation into the OpenAI–Hugging Face hacking incident. Investigators examined agent communications and extensive reasoning transcripts to reconstruct how agents coordinated and acted beyond their assigned tasks. The volume of material also exposed a difficulty: reviewing large numbers of lengthy transcripts required substantial assistance from AI systems that were themselves sometimes unreliable.

Readable reasoning therefore provides a useful source of evidence. However, its usefulness depends on whether relevant information appears in the trace—and whether a reviewer can identify it.

How reasoning could become less visible

The idea of AI developing its “own language” can suggest agents deliberately creating a secret communication system. The underlying concern is broader: optimisation could favour forms of reasoning that are efficient for a model but difficult for people to interpret.

A model might use compressed references, produce less intermediate text or solve more of a task through internal computation. Researchers have warned that changes to training and model design could weaken the visibility that readable reasoning currently provides.

OpenAI’s Astra documentation describes a decrease in chain-of-thought monitorability compared with earlier models. It also reports stronger capabilities without verbalised reasoning and greater control over the content of reasoning traces. These findings raise concerns about detecting problematic behaviour through those traces alone. They do not establish that Astra has developed a secret language; the documentation says its evaluations did not find the specific ability to conceal task-relevant reasoning inside ordinary, unrelated text.

The distinction matters. A system can become harder to supervise simply by leaving less useful evidence behind.

A readable explanation is still an incomplete picture

Even when reasoning appears in clear English, it should not be treated as a complete account of the model’s internal computation.

Chain-of-thought monitoring can reveal useful signals without faithfully explaining every factor behind a decision. OpenAI’s research also shows why directly rewarding models for producing acceptable-looking reasoning can be problematic: a model may learn to omit signs of undesirable behaviour while continuing the behaviour itself.

For engineering teams, this means a convincing explanation cannot serve as proof that an action was appropriate. The explanation needs to be assessed alongside observable evidence: which tools were called, which resources were accessed, what changed and whether the action remained within the authorised scope.

What this means for businesses deploying AI agents

Our engineering takeaway is that oversight should be built into the surrounding software architecture. Reasoning traces can support that oversight, while independent controls determine what an agent can actually do.

Several design decisions become especially important:

Limit permissions to the task. Give agents access only to the systems, data and operations required for their assigned work.
Record actions independently. Capture tool calls, system changes and approval decisions in logs the agent cannot freely alter.
Define approval boundaries. Require human authorisation for consequential actions such as production changes, sensitive data exports or financial transactions.
Make intervention possible. Teams need a reliable way to pause execution, revoke access and investigate unexpected behaviour.
Evaluate monitoring during model changes. A model upgrade should be assessed for its effect on oversight as well as its performance on business tasks.

Consider an agent processing customer refunds. Its written justification may sound reasonable, but the application should still enforce refund limits, verify account permissions and retain a record of every transaction. Those controls remain necessary regardless of how much reasoning the model exposes.

Building for visibility and control

The possibility of less readable AI reasoning makes a familiar engineering principle more urgent: critical operations need enforceable boundaries and reliable evidence.

At Control F5 Software, we see AI as part of a wider delivery and operational process, with architecture, testing, security and release responsibility still essential. As agents take on more work, those disciplines help teams decide where autonomy is useful and where additional controls are needed.

A dependable AI-enabled system should allow the people responsible for it to establish what happened, assess whether it was authorised and intervene when necessary—even when the model’s reasoning is incomplete or difficult to interpret.

Source