Warnings about the speed of artificial intelligence development are spreading across Silicon Valley, but market competition continues to push the industry in the opposite direction. Anthropic, which has called for a global effort to moderate the development of advanced AI, is considering another model as it seeks to keep pace with OpenAI. The move exposes a fundamental weakness in voluntary restraint: even companies built around AI safety face pressure to keep advancing when competitors continue to release more capable systems.

That tension has now moved beyond the technology industry. At high-level talks in New York, the United States proposed that Washington and Beijing establish a mechanism to notify each other about significant AI incidents that could raise national security concerns. The proposal marks a shift in the AI safety debate from voluntary corporate commitments toward government-to-government communication and risk management.

The proposal remains at an early stage. China has not publicly accepted the mechanism, and the New York talks produced no broader AI agreement. Export controls on advanced AI chips and semiconductor manufacturing equipment were also excluded from the discussions.

The significance lies in what comes next. The central question is no longer whether AI companies and governments recognize the risks. It is whether competitors that have strong incentives to keep accelerating can establish minimum rules for what happens when increasingly powerful AI systems create risks that cross corporate and national boundaries.

Corporate Restraint Meets Competitive Reality

Anthropic Chief Executive Dario Amodei recently proposed a phased framework for moderating the pace of frontier AI development.

“The first step is something Anthropic is unilaterally committing to. The second step requires industry-wide coordination. The third step requires global coordination,” Amodei said.

His framework identifies the central problem confronting AI safety. A company can impose safeguards on itself, but unilateral restraint becomes difficult to sustain when competitors operate under different standards.

Anthropic is considering a new model following OpenAI’s release of GPT-6 Astra. No launch decision or timetable has been confirmed. The deliberations nevertheless show how quickly commercial competition can collide with voluntary efforts to moderate development.

If one company slows while a rival continues releasing more capable models, customers, developers, investors and talent can move with the technology. In frontier AI, model performance also influences data-center investment, financing capacity and corporate valuation. Unless competitors operate under comparable safety standards, the company that slows first risks putting itself at a market disadvantage.

The same dilemma exists at the national level.

The Trump administration has emphasized maintaining US leadership in AI while resisting broad restrictions that could slow domestic development. David Sacks, who has led White House policy on AI and cryptocurrency, summarized that position when the administration unveiled its AI Action Plan.

“The United States must win the AI race,” Sacks said.

Competition with China makes unilateral restraint even harder. Washington increasingly treats AI as a strategic technology affecting economic competitiveness, industrial capacity and national security. Policies designed to reduce AI risks therefore operate alongside policies intended to preserve the technological advantage of US companies.

Companies struggle to slow down because competitors continue developing. Governments face the same constraint when rival countries continue accelerating. That is why the outcome of the New York talks matters.

US-China AI Safety Moves From Debate to a Proposed Mechanism

US Treasury Secretary Scott Bessent and Chinese Vice Premier He Lifeng concluded their New York talks after discussing artificial intelligence and trade ahead of the planned meeting between US President Donald Trump and Chinese President Xi Jinping.

The US side proposed creating a bilateral AI dialogue that would include a notification mechanism for incidents serious enough to raise national security concerns. Bessent said greater transparency between the two leading AI powers was increasingly important.

“Moving from opaque to more transparency between the number one and the number two AI powers in the world is very important,” Bessent said.

The proposal is far narrower than an international AI safety treaty. It would not require either country to stop developing advanced models or give the other side access to sensitive technology. Instead, it begins with a more practical question: what should happen when a powerful AI system creates a risk that could affect both countries?

Potential areas of concern include rogue AI agents, misuse by non-state actors and biological or other national-security threats. Further US-China discussions are expected as officials explore how communication protocols could work in practice.

The distinction is important. Washington has not offered to relax restrictions on China’s access to advanced computing technology as part of the safety dialogue. US officials said export controls on advanced AI chips and semiconductor manufacturing equipment were not discussed in connection with the proposed mechanism.

The United States is therefore beginning to pursue two AI policies toward China at the same time: strategic competition over advanced technology and limited cooperation over risks that neither side may be able to contain alone.

That separation could become one of the defining features of the next phase of US-China AI relations.

OpenAI Pushes the Debate Toward International Standards

A second development reinforces this shift.

OpenAI has called on the United States to lead an international effort to establish technical standards for frontier AI, including increasingly autonomous systems and models capable of improving their own capabilities. The company is calling for standardized measurements, safety evaluations and incident-reporting protocols that would allow governments and AI laboratories to assess advanced systems using comparable criteria.

The proposal points toward a different model of AI governance from a blanket slowdown.

Instead of asking every company and country to develop AI at the same speed, governments could begin by agreeing on how dangerous capabilities are measured, which incidents must be reported and what thresholds should trigger additional safeguards.

That approach also addresses one of the weaknesses in voluntary corporate commitments. Safety pledges are difficult to compare when companies use different evaluation methods and disclose different information. Common technical standards could make external scrutiny more meaningful without requiring competitors to follow identical development schedules.

It would not eliminate competition. It could establish minimum rules within which competition takes place.

This is where the US-China dialogue becomes particularly important. If Washington and Beijing can agree even on a narrow incident-notification system, it would provide an initial test of whether strategic competitors can cooperate on AI risks without resolving their wider technology rivalry.

The First Guardrails May Be Communication, Not a Slowdown

The debate that began with calls to slow AI development is therefore changing.

A global agreement requiring frontier AI companies to develop at the same pace remains difficult to envision. Companies have commercial incentives to move faster than rivals, while governments have strategic incentives to prevent competitors from gaining technological advantage.

The first workable international safeguards may consequently be more limited: common safety measurements, independent evaluations, major-incident reporting and direct communication channels between governments.

That does not make them insignificant.

Nuclear powers developed communication mechanisms not because geopolitical rivalry disappeared, but because certain failures could become catastrophic for everyone involved. Advanced AI is beginning to raise a related governance question: whether intense competition can coexist with a narrow set of rules designed to prevent accidents, miscalculation or uncontrolled escalation.

The analogy has limits. AI is developed largely by private companies, its capabilities change far faster than nuclear technology, and governments do not yet share a common definition of what constitutes a major AI incident. Those differences make technical standards, reporting thresholds and independent evaluations particularly important.

The next test will be whether the US proposal survives beyond diplomatic language.

China has yet to publicly endorse the American notification mechanism. The Trump-Xi meeting could show whether the proposal receives political support at the highest level. Subsequent talks will need to determine what kinds of AI incidents require notification and how information could be exchanged without exposing national-security information or corporate secrets.

Anthropic’s consideration of another model exposed the limits of corporate restraint. The New York talks have now produced the beginnings of a different response.

AI companies understand the risks, and governments increasingly recognize them as well. But neither companies nor countries find it easy to slow down first while competitors continue to accelerate.

The emerging answer may therefore be more pragmatic than a global slowdown. If competitors cannot agree to stop or even slow the race, they may first have to agree on how to measure the risks, what incidents must be reported and how to communicate when something goes wrong.

That could become the first practical guardrail of the frontier AI era.