The previous article titled The United Nations’ Evolving Role in Global AI Governance ended with this proposition:
The UN can play an important role by helping countries agree on a common framework for testing frontier models, as well as reporting, investigating and learning from significant AI incidents. Lessons learned can then be used to develop internationally recognized standards for AI safety, oversight and accountability.
It didn’t take long before a colleague reached out and asked, “What, specifically, do you think the UN can do to put this proposal into practice?“
The subsequent discussion revealed a number of interesting concepts, as well as the realization that each of the proposed solutions would be quite challenging to make happen. But that shouldn’t prevent them from being brought to light.
Image: Mark Lovett / AI GAPS — Generated with ChatGPT Images 2.5
1. Establish a global AI incident reporting framework
Nations, as well as frontier AI model developers, can’t learn from incidents they don’t know about or discover months after the fact. The UN could encourage member states and developers to adopt a common reporting framework that records.
- What exactly happened and when it was discovered
- Which AI system and agent capabilities were involved
- What the system was instructed to do versus what it actually did
- Whether information systems, people, data or infrastructure were affected
- Which safeguards failed, what harm occurred, and what remediation followed
This would build on work already underway at the OECD, whose common reporting framework proposes 29 criteria for describing AI incidents consistently across countries and sectors.
Why it matters: Without consistent reporting, the international community only sees headlines for major incidents rather than discerning developing patterns that could inform policy decisions.
2. Create an international AI incident repository
The reporting process should feed into a shared, searchable source of information. The UN could coordinate a repository, or work with the OECD’s existing AI Incidents and Hazards Monitor, to make verified information available to researchers, testing authorities, governments and the public. Such a repository could distinguish:
- Confirmed incidents from alleged or unverified reports
- Human-directed misuse from unintended AI agent behavior
- Actual harm incurred from near misses and potential hazards
- Unauthorized access from successful compromise and confirmed damage
- Independent incidents from multiple reports related to one particular event
Sensitive information could be shared through protected channels with qualified investigators and government officials, while public summaries would still provide appropriate transparency.
Why it matters: A common evidence database could reveal recurring weaknesses across systems, such as inadequate sandbox safeguards, excessive permissions being granted, poor monitoring of agent activity or delayed disclosure of critical incidents. The OECD already operates an incident monitor, so the UN could coordinate with and build on existing work rather than create a competing database.
3. Use independent scientific research to identify systemic risks
Incident reports document what happened in the past. Independent research helps explain why and how it happened, whether the same failure could recur, and what other potential risks might be emerging.
The UN’s Independent International Scientific Panel on AI is well positioned to contribute their expertise. Its mandate includes evidence-based assessments of AI capabilities, opportunities, risks and impacts. Its preliminary report identifies important domains including security, reliability, human rights, democracy and child safety.
The UN could ask the Panel to assess recurring incident patterns and report on questions such as:
- How often do agents exceed their intended permissions?
- Which containment and monitoring measures fail most frequently?
- What agentic capabilities make certain types of incidents more likely?
- Which safeguards have evidence of effectiveness, and how often are they used?
- What risks remain difficult to test before deployment or monitor after deployment?
Why it matters: AI governance policies should respond to demonstrated failure mechanisms regardless of scope, not just to the most alarming individual incidents.
4. Develop internationally recognized minimum safety standards
Once recurring failure modes are understood, the UN could help countries agree on minimum expectations for AI systems, especially those using agents that take actions in external environments.
Possible standards include:
- Least-privilege access: Agents receive only the permissions required for their task.
- Isolation and containment: Testing systems cannot freely reach production infrastructure or the public internet unless explicitly authorized.
- Human authorization: Sensitive actions, such as transferring funds, changing critical systems or submitting official reports, require appropriate approval by a human.
- Continuous monitoring: Developers maintain records of agent actions and implement systems able to detect attempts by any agent to bypass safeguards or violate protocols.
- Independent evaluation: Systems undergo proportionate testing before deployment and after significant changes.
UNESCO’s Recommendation on the Ethics of Artificial Intelligence already calls for measurable safety and transparency standards, auditability, traceability, monitoring and redress.
Why it matters: A common baseline would let countries implement and enforce protections through their own laws while making expectations more consistent internationally. In this way, the public could see when companies or governments were not meeting minimum safety standards.
5. Clarify responsibilities and disclosure obligations
The incidents raise a basic governance question: when an AI agent takes an unauthorized action, who must report it, investigate it, contain it and help remedy the consequences?
An international framework could distinguish responsibilities among:
- Developers who design and train the model
- Deployers who integrate it into operational systems
- Users who provide it with instructions or grant permissions
- Operators and infrastructure providers who control access and containment
- Regulators and affected authorities who investigate and enforce applicable laws
It could also establish recommended timelines for reporting serious incidents, preserving evidence, notifying affected parties and documenting corrective action.
Why it matters: Accountability becomes extremely difficult when organizations use different definitions of an incident, operate under different disclosure timelines, and adopt different assumptions about who is responsible.
6. Turn lessons learned into an ongoing international process
A one-time declaration will not keep pace with a rapidly evolving technology. The UN could use its Global Dialogue on AI Governance as a recurring forum where countries examine incidents, compare national responses and update shared recommendations.
Each annual cycle could follow a simple process:
- Collect incident reports and technical evidence
- Have independent experts identify recurring failure patterns
- Ask governments and stakeholders to evaluate possible responses
- Publish recommendations, model reporting rules and proposed safety standards
- Review implementation and assess whether incidents and harms are decreasing
The Global Dialogue is already designed to facilitate international cooperation and exchange best practices, but it is not a negotiating forum and does not itself produce binding agreements.
Why it matters: The UN could help create a continuous feedback loop between real-world experience, scientific analysis and international policy.
Wrap up: From incidents to international action
The long list of incidents involving AI agents are not simply stories about this technology behaving unexpectedly. These incidents should be treated as case studies that exemplify the challenges of governing increasingly capable systems that can act with agency, and do so across digital boundaries.
They raise questions about training, testing, containment, permissions, monitoring, disclosure, responsibility and accountability. And because AI systems don’t recognize national borders, those questions cannot always be resolved by one country acting alone. Without international cooperation, such incidents will never be fully dealt with.
The United Nations does not need to become a global AI regulator in order to make an important contribution to the framework. It is well-equipped to help countries agree on how such incidents should be reported, establish a shared evidence base, commission independent analysis to uncover threats and patterns, develop common safety standards, and clarify expectations for disclosure and accountability.
And the processes need to be iterative: document actions, learn from incidents, identify patterns, develop policies, implement safeguards, and discern whether those safeguards work. Everything learned becomes part of the processes that creates the next frontier model.
Even with the best of intentions, such practices won’t eliminate AI risk. Nor will it guarantee that every country or company complies with the guidance or meets expectations. But if implemented properly, it could replace the current system of fragmented responses with a more coordinated approach to AI governance and policy implementation.
The goal should not be to create a global institution that predicts and prevents every AI failure. It should be to build an international system that detects failures, learns from them, shares that knowledge with transparency, and makes future failures less likely.
