As technology leaders debate whether AI development should slow down, Palo Alto Networks CEO Nikesh Arora argues that a coordinated pause would be difficult to sustain. For businesses adopting AI, the discussion raises a practical question: how do you keep innovation moving while maintaining control over what you deploy?

Speaking on CNBC’s The Tech Download podcast, Arora described efforts to slow AI development as “unrealistic.” His reasoning centres on competition: even if some companies agree to limit progress, others will continue pushing ahead. He also characterised the probability of AI causing human extinction as “extremely small.”

His comments highlight a growing disagreement over how the industry should manage increasingly capable systems—and who should take responsibility when those systems fail.

The debate intensified after an Anthropic researcher suggested that AI could pose a 10% chance of causing human extinction, while another researcher departed amid concerns about responsible development. Anthropic CEO Dario Amodei subsequently called for the industry to moderate the pace of improvements in AI capabilities, attracting support from other prominent technology leaders, according to CNBC’s reporting.

Arora initially questioned the motives behind those calls. In his interpretation, companies advocating a slowdown could gain regulatory goodwill and seek protection from future liability. That remains his assessment of their intentions, rather than an established explanation for the proposals.

His broader objection is about feasibility. A shared commitment would depend on companies with different incentives, priorities and competitive pressures following the same rules. It would also require a workable definition of what slowing development actually means and a way to measure compliance.

Instead, Arora argues that developers should withhold products they cannot confidently release safely.

That position echoes comments from Nvidia CEO Jensen Huang, who has emphasised testing and continued engineering before release. U.S. Treasury Secretary Scott Bessent has also argued that AI developers should take responsibility for their products rather than expect government protection from liability.

The disagreement over extinction risk deserves careful interpretation. Neither the 10% estimate cited in the debate nor Arora’s much lower assessment establishes a verified probability. Both express judgments about uncertain future developments. Arora’s confidence should therefore be read as his position in the debate, rather than evidence that the underlying safety questions have been resolved.

International cooperation adds another layer of complexity. Amodei’s proposals include coordination across borders, including cooperation with China. Bessent said the U.S. and China had discussed an AI dialogue, with the U.S. proposing a notification mechanism for national security incidents involving the technology.

Arora questioned how far such coordination could progress while American policymakers and technology executives still disagree over their own objectives.

For businesses building and adopting AI, our reading is that uncertainty at the industry level makes clear deployment decisions even more valuable.

A company may have little influence over the speed of frontier research. It can still decide which systems access its data, which actions require approval, and what evidence is needed before an AI feature reaches customers.

For CTOs and product teams, that means addressing concrete questions:

What can the system access and change? Define permissions around the specific task and the sensitivity of the information involved.
What must it demonstrate before release? Establish evaluation criteria for accuracy, security and behaviour when inputs are ambiguous or unexpected.
Where is human approval necessary? Identify consequential actions that require review before execution.
How will failures be detected and contained? Assign ownership for monitoring, escalation and recovery.

These decisions should reflect the workflow. An assistant that retrieves internal documentation needs different controls from an agent that updates customer records or initiates transactions.

At Control F5 Software, we see this debate as a reminder that AI adoption requires the same engineering discipline as any other important operational change. Start with a clear business purpose, understand the consequences of failure, and expand deployment as evidence supports it.

The pace of AI research may remain contested. The criteria for allowing AI into a business-critical workflow should be explicit.

Source