AI Safety Institute Landscape
A Global Briefing on Government-Backed AI Evaluation Bodies
Date: 2026-05-17 | Type: Research | Status: Complete
14+ Countries · 3 Summit Generations · 1 International Network · 24 Sources
3
Summits (Bletchley→Seoul→Paris)
2025-26
Safety → Security pivot
Executive Summary
The AI Safety Institute model — government-backed bodies evaluating frontier AI models — emerged in 2023 as the first global institutional framework for AI governance. By 2026, 14+ countries have established or announced institutes coordinated through an international network. However, the model faces a critical inflection point: the two founding members (US and UK) have pivoted from "safety" to "security/innovation," while the EU has become the primary binding regulatory force.
Global Network of AI Safety Institutes
Operational Institutes
🇺🇸 US — CAISI (was AISI)
NIST. Founded Nov 2023. Renamed CAISI June 2025. Mission shifted from safety evaluation to innovation promotion and guarding against foreign regulation. Budget: ~$10M. Director: Elizabeth Kelly. 200+ orgs in AISIC consortium.
🇬🇧 UK — AI Security Institute
DSIT. Founded Apr 2023 as Frontier AI Taskforce (£100M). Renamed Feb 2025. Scope narrowed to security risks only — bias and free speech removed. Chair: Ian Hogarth. Open-sourced "Inspect" eval framework. SF office opened May 2024.
🇯🇵 Japan — J-AISI
Founded Feb 2024 under IPA. ~23 staff. Director: Akiko Murakami (ex-IBM Japan). Released comprehensive evaluation guide Oct 2025. Builds on Hiroshima AI Process (G7).
🇸🇬 Singapore AISI
Founded Jun 2022 as Digital Trust Centre. Renamed May 2024. NTU/IMDA. S$10M/yr. AI Verify governance testing framework — 11 principles.
🇨🇦 Canada — CAISI
Founded Nov 2024. CIFAR. CA$50M/5 years. Solution Networks for AI safety in legal systems and linguistic equality.
🇫🇷 France — INESIA
Created Jan 31, 2025. National Institute for AI Evaluation and Security. LNE involved in evaluation. Announced pre-Paris Summit.
🇰🇷 South Korea AISI
Founded Nov 2024 under ETRI. Bundang, Seongnam. 30+ staff. KRW 10-20M/yr budget.
🇪🇺 EU — AI Office
Founded May 2024. The only AISI with binding regulatory power. Administers EU AI Act Code of Practice. Enforcement begins Aug 2026.
🇦🇺 Australia AISI
Created Nov 25, 2025 by Albanese government. Early-stage.
Announced / In Development
🇮🇳 India — IndiaAI Safety Institute
Announced Jan 2025. ₹20 crore from IndiaAI Mission. Hub-and-spoke with IITs and UNESCO. Focus on standards-setting and Indian social/linguistic context.
🇰🇪 Kenya
Only African state in network. Agreed to join; no formal institute yet.
The Safety → Security / Innovation Pivot
"The AI future is not going to be won by hand-wringing about safety" — VP JD Vance, Paris AI Action Summit, Feb 2025
Both founding AISI members underwent dramatic rebranding in 2025:
- US (June 2025): AISI → CAISI. Commerce Secretary Lutnick: "For far too long, censorship and regulations have been used under the guise of national security. Innovators will no longer be limited."
- UK (Feb 2025): AISI → AI Security Institute. Bias and free speech explicitly removed. "Societal impacts" → "societal resilience". "Public accountability" → "public safe and secure".
- Paris Summit (Feb 2025): Safety relegated to side event. US and UK refused to sign communique. Innovation vs. regulation binary framing.
Critics: Elizabeth Seger (Demos): "attention to bias explicitly cut out." Michael Birtwistle (Ada Lovelace): "risks leaving a whole range of harms unaddressed."
How Frontier AI Evaluation Works
| Method | Description |
|---|
| Automated Capability Assessment | Systematic testing of model capabilities across safety-relevant domains |
| Red-Teaming | Adversarial testing to elicit harmful outputs, CBRN knowledge, dangerous capabilities |
| Human Uplift Evaluation | Assessing how AI enhances harmful human capabilities (bioweapons, cyber, fraud) |
| AI Agent Evaluation | Testing autonomous goal-seeking behavior, info leaks, multi-step attacks |
| Responsible Scaling Policies | Capability thresholds that trigger enhanced safety measures (Anthropic RSP, OpenAI Preparedness, DeepMind Frontier Safety) |
Key limitation: Evaluations are "easy to manipulate or game by training models on the evaluation dataset" (Ada Lovelace Institute). Small changes to AI products cause unpredictable behavior shifts.
Emerging Regulatory Framework
| Law | Jurisdiction | Threshold | Key Obligations | Effective |
|---|
| CA SB 53 | California | >10²⁶ FLOPs + $500M rev | Framework, reports, incident reporting (15d/24h), whistleblower | Jan 2026 |
| EU AI Act (CoP) | EU | >10²⁵ FLOPs | Risk assessment, evals, security, incident tracking, governance | Aug 2025/26 |
| NY RAISE Act | New York | >10²⁶ FLOPs + $500M rev | Detailed framework, 72h incident reporting | Jan 2027 |
CoP signatories: OpenAI, Anthropic, Google, xAI. Notable holdout: Meta (must demonstrate alternative compliance). External evaluators get ≥20 business days pre-release access.
Criticisms & Debates
💰
Tiny Budgets
US AISI: $10M. UK: £100M one-time. Labs spend billions. NIST chronically underfunded.
⚖️
No Enforcement Power
Except EU, institutes can evaluate but cannot block deployment. Voluntary compliance only.
🎯
Gaming Evaluations
Red-teaming datasets trainable. Model changes cause unpredictable shifts. Cannot certify safety.
🔄
Regulatory Capture
200+ industry orgs in US consortium. Lab employees seconded to government. Revolving doors. (Springer Nature 2025)
🌍
Geopolitical Fragmentation
US/UK innovation-first vs. EU precautionary. China not in network. Regulatory arbitrage.
⏰
Can't Keep Pace
Model velocity outstrips evaluation capacity. No standardized benchmarks across institutes.
Key Takeaways
- The AISI model is the first global institutional framework for AI governance — but at a crossroads
- The US/UK pivot has split the network's original mission
- The EU is now the primary binding regulatory force — enforcement starts Aug 2026
- California SB 53 is the first US law with teeth — NY's RAISE Act follows in 2027
- Evaluation science is still nascent: no standardized benchmarks, easy to game
- Regulatory capture is the existential risk for AISIs
- China's absence means the framework covers only part of the frontier AI landscape
Sources
Primary: NIST AISI Vision | NIST CAISI | UK AISI/DSIT | EU AI Office | J-AISI | Singapore IMDA | Canada CIFAR | Seoul Declaration | International AI Safety Report
Analysis: CSIS (Allen & Adamson 2024) | All Tech Is Human (Kore & Dhawan 2025) | Brookings | METR | Ada Lovelace Institute | Springer Nature | EPC | Oxford | FAF
Reference: Wikipedia (AI Safety Institute, AI Action Summit, AI Seoul Summit)
NotebookLM: Open notebook →