Global AI Safety Institutes Unite for Frontier Model Testing

Following the historic Bletchley Declaration in November 2023, an international network of state-backed AI Safety Institutes (AISIs) emerged to address the risks of advanced artificial intelligence. This article provides an overview of how these global entities—led by the United Kingdom and the United States, alongside partners in Japan, Singapore, Canada, the European Union, and South Korea—have transitioned from theoretical policy agreements to active, joint pre-deployment testing of frontier AI models. By sharing technical expertise, aligning evaluation frameworks, and securing early access to proprietary systems from leading AI developers, this coalition is actively establishing a unified global standard for AI safety and security before powerful models are released to the public.

The Foundation of the AISI Network

The Bletchley Declaration established a shared international consensus that frontier AI models pose significant catastrophic risks, particularly in cybersecurity, biological engineering, and autonomous capabilities. To address these threats without stifling innovation, participating nations agreed that governments must take an active role in evaluating models independently, rather than relying solely on self-reporting by private tech companies.

Shortly after the summit, the UK and the US established the first two state-backed AI Safety Institutes. Throughout 2024, this bilateral effort expanded into a broader international network during subsequent global gatherings, including the AI Seoul Summit and the initial convening of the International Network of AI Safety Institutes in San Francisco. This created an operational coalition including institutes from Japan, Singapore, South Korea, Canada, France, Australia, and the European Union's AI Office.

Bilateral Agreements and Resource Sharing

The cornerstone of international collaboration on pre-deployment testing was the formal Memorandum of Understanding (MoU) signed between the US and UK AISIs in April 2024. Recognizing that no single nation possessed all the necessary talent or compute infrastructure to evaluate rapidly evolving models, the two institutes agreed to:

  • Build a common suite of technical evaluation tools.
  • Share threat intelligence and testing methodologies.
  • Conduct joint testing exercises on publicly accessible and pre-deployment frontier models.
  • Exchange personnel to ensure seamless operational alignment.

This bilateral framework quickly became the blueprint for the broader international network, allowing newer institutes to adopt established testing protocols rather than building them from scratch.

Standardizing the Evaluation Toolkits

To execute joint tests effectively, the international network needed a shared technical language. The UK AISI significantly accelerated this process by open-sourcing "Inspect," a software framework designed to assess large language models.

By utilizing shared platforms like Inspect, researchers across different national institutes can write standardized evaluation scripts to test a model's core knowledge, reasoning abilities, and autonomous agentic behaviors. This standardization ensures that a safety test conducted in Tokyo or London yields comparable data to one conducted in Washington, D.C., eliminating duplicate efforts and speeding up the pre-deployment review process.

Joint Pre-Deployment Testing in Action

The most significant achievement of the AISI network has been the execution of joint, pre-deployment evaluations on commercial frontier models. Through voluntary commitments secured by governments, major AI developers—including OpenAI, Anthropic, and Google DeepMind—agreed to grant the institutes early access to their models prior to public release.

In the second half of 2024, the US and UK institutes successfully conducted joint pre-deployment testing on OpenAI’s o1 model and Anthropic’s upgraded Claude 3.5 Sonnet. Instead of running identical tests, the institutes divided the labor based on specialized expertise. During these evaluations, the joint teams assessed the models across several critical risk vectors:

  • Cybersecurity: Testing the models' ability to identify software vulnerabilities, write malicious code, or execute automated cyberattacks.
  • Chemical and Biological Weapons (CBRN): Evaluating whether the models could lower the barrier to entry for bad actors seeking to synthesize dangerous pathogens or toxic chemicals.
  • Autonomous Capabilities: Assessing if the models could self-replicate, acquire resources without human oversight, or deceive human operators.
  • Safeguard Efficacy: "Red-teaming" the models to see how easily users could bypass built-in safety guardrails (jailbreaking).

Following the tests, the institutes issued joint reports back to the developers, highlighting vulnerabilities that allowed the companies to patch safety flaws and reinforce guardrails before the models were deployed globally.

Overcoming Operational Hurdles

Despite the success of early joint tests, the international network continues to navigate complex operational hurdles. Sharing access to proprietary model weights across international borders triggers strict national security protocols and intellectual property concerns. To resolve this, the institutes primarily rely on secure Application Programming Interfaces (APIs) provided by the developers, allowing remote teams in different countries to probe the models simultaneously without taking physical possession of the underlying code.

Through these coordinated efforts, the international network of AI Safety Institutes has successfully transformed the diplomatic promises of the Bletchley Declaration into a functional, global defense mechanism for the deployment of frontier artificial intelligence.