Company updates

Anthropic publishes its August 2026 Risk Report
This is Anthropic’s second Risk Report, and the first under RSP v3.4. It raises misalignment risk in high-stakes settings from “very low” to “low”. Anthropic says that reflects uncertainty, not a new finding. It also discloses that all human feedback vendor traffic ran without its blocking biological classifiers for almost a year. Anthropic has since fixed that and found no misuse. On automated R&D, it says its most concrete evaluations have saturated.
Learn more →

OpenAI explains how it is pacing model development on cyber capabilities
OpenAI says it temporarily slowed the pace of scaling. That included a two-week pause in RL training on models headed for deployment. Its largest planned frontier RL run is still on hold. New rules isolate frontier research workloads from each other and from the network. Astra and cyber workloads get the strictest tier. New monitoring aims to raise an alert within 30 minutes. It costs roughly 20% of the compute it watches.
Learn more →

OpenAI previews Private Safety Processing for zero data retention deployments
Private Safety Processing is a new safety system for customers with Zero Data Retention. It’s in testing with early customers. Automated systems look for misuse across related interactions, not one interaction at a time. The content stays on the customer’s own infrastructure. Staff see only a narrow signal about the type of activity. The exception is images flagged as possible CSAM, which are still kept for manual review.
Learn more →

Anthropic explains how Claude’s text watermark works
Future Claude models will produce text with a watermark built in. The EU has required providers to mark AI-generated content since August 2. The watermark only changes how Claude picks between candidate words. It’s a version of Google DeepMind’s SynthID-Text. Anthropic says it doesn’t affect quality or price. Older models get it over the coming months, and the rollout is global, not just the EU.
Learn more →

Research

Guidelight publishes an assessment of frontier AI control practices
Guidelight graded five frontier AI companies on six practices from its Control standard. It worked only from public material, like system cards and safety frameworks. Anthropic and OpenAI both got a C+, Google a D+, xAI a D-, and Meta an F. No company beat “substantial partial implementation” on any practice. Prevention and containment were the weakest areas.
Learn more →

LawAI publishes a report on Germany’s new AI Security Institute
Germany’s National Security Council decided in June to set up an AI Security Institute (DE-AISI). It starts as a virtual body drawing on staff at the BSI and BNetzA. The report works through three open design choices. It backs a scientific mandate, kept clearly separate from those regulators’ enforcement powers. The scope under discussion covers cyber, CBRN, and loss of control risks. It also floats a federally owned GmbH as the legal form, and Berlin as the location.
Learn more →

IAPS publishes a report on verification methods for AI chip export controls
The report sets out ways to detect breaches of U.S. AI chip export controls. Each could be built within about a year. To check where chips physically end up, it assesses on-site inspections, video inspections, and a delay-based method. That last one measures how long a chip takes to reach a known server. To check who buys them and what for, it looks at stronger Know-Your-Customer screening and cross-checks of end-use declarations. The authors suggest layering them, starting with the delay-based check.
Learn more →

Center for Technology & Statecraft (CTS) launches in Washington, D.C.
The center is an independent, non-partisan research initiative backed by the Institute for Progress. It has two research pillars. One is automation and the social contract. The other is competition and stability, focused on China. The founding team consists of Saif Khan, Nicholas Brown, and Konstantin Pilz.
Learn more →

METR announces new funding commitments
METR raised commitments of around $71 million over the last six months. It doesn’t say who from. The money goes to work on autonomous capabilities, recursive self-improvement, monitoring systems, and AI incident investigations. METR says it takes no money from frontier AI companies, or from their staff. It does accept free tokens from them, and says the amount is significant.
Learn more →

Google DeepMind publishes paper on using debate to reduce reward hacking
In debate, a generator and a critic argue a case before a weaker AI judge. Google DeepMind trained a model this way on math problems, where the right answer is checkable. It compared that against an RLAIF baseline. The baseline quickly learned to game the judge. Debate didn’t, as long as the critic faced a word limit. The gain was small, worth about two percentage points.
Learn more →

Job opportunities

The OpenAI Foundation is looking for various roles on its AI Resilience team
Three of them are Program Director (Formal Methods), Program Officer (AI Model Safety), and Program Officer (AI Resources). All three are founding grantmaking positions. They fund verified software, safety research and standards, and model access and compute. Role type: full-time for the director, not listed for the officers. Location: San Francisco. Salary: $300k–$370k/year for the director, $220k–$300k/year for the officers. Deadline: not listed.
Learn more →

NIST is looking for a Principal Researcher for AI Risk Management
The role maintains the AI Risk Management Framework and the guidance around it. That includes the AI RMF Playbook and sector profiles. It also briefs agency leadership on AI governance trends. Only U.S. citizens can apply. Role type: full-time, permanent. Location: Boulder, CO or Gaithersburg, MD (telework eligible). Salary: $119k–$197k/year. Deadline: September 2, 2026.
Learn more →

NIST is looking for an AI Standards Coordinator
The role leads engagement with domestic and international AI standards bodies. It drafts technical contributions on AI measurement and risk management. It’s also the agency’s technical voice in standardization meetings. Role type: full-time, permanent. Location: Gaithersburg, MD (telework eligible). Salary: $122k–$197k/year. Deadline: August 31, 2026.
Learn more →

CAISI is looking for a Senior Cyber Offense Specialist
The role brings national security context to evaluations of frontier models’ cyber offense capabilities. It maps tradecraft like vulnerability research and exploit development to real adversary tactics. It also turns results into briefings for senior interagency leadership. It needs an SCI clearance. Role type: full-time, term of 13 months, extendable up to four years. Location: Washington, DC (telework eligible). Salary: $122k–$187k/year. Deadline: August 24, 2026.
Learn more →

Google DeepMind is looking for a Principal Virologist (Responsible Development and Innovation)
The role designs and runs biology evaluations that test frontier models against biological threats. It also shapes biosecurity mitigations and builds them into model development. It briefs senior executives and policy leaders on what the results show. Role type: not listed. Location: Mountain View, CA; Boulder, CO; Cambridge, MA; Kirkland, WA; New York; Washington, DC. Salary: $307k–$427k/year plus 30% bonus target. Deadline: open until at least August 28, 2026.
Learn more →

Applications for the FAS AI Safety Policy Entrepreneurship Fellowship are now open
Fellows spend about five hours a week pushing one frontier AI safety idea toward adoption. They write and publish a policy memo, then approach the people who could implement it. There’s also a California retreat and a Washington, D.C. capstone. Role type: part-time fellowship, September 30, 2026 to February 28, 2027. Location: hybrid. Salary: $5,000 stipend, plus a possible $1,000 merit award. Deadline: September 7, 2026.
Learn more →

Company updates

OpenAI reveals new details about the Hugging Face incident at Black Hat
OpenAI gave a talk at Black Hat USA 2026 filling in the timeline of the breach. Agents in separate training runs had been leaving notes for each other on an internal server. That let them pass along credentials and exploits. OpenAI also says it didn’t realize it was responsible until it asked Hugging Face to revoke stolen credentials. At that point, they had already been revoked.
Learn more →

OpenAI can’t rule out that an unreleased model crosses its Critical cybersecurity threshold
OpenAI says internal testing of its upcoming Astra model can’t rule out Critical cybersecurity capability under its Preparedness Framework. Critical is the highest level there is, and reaching it would force stronger safeguards before any deployment. In the meantime, OpenAI has paused internal work with Astra that doesn’t meet its new security controls.
Learn more →

OpenAI expands Daybreak and introduces GPT-5.6-Cyber
OpenAI split Daybreak, its program for approved cyber defenders, into two tiers. Daybreak Blue gives access to GPT-5.6 Sol for general work. Daybreak Red gives access to the new GPT-5.6-Cyber. OpenAI rates that model High for cyber capability, one level below Critical, so it’s keeping access restricted.
Learn more →

xAI releases Grok 4.6
Grok 4.6 is built for long-running agent tasks. Its model card says cyber capabilities have grown, but argues they’ll help defenders more than attackers. On a virology test it scores 67.4%, up from 65.5% for Grok 4.5. xAI says that’s still below the limits in its Frontier AI Framework. The card doesn’t run the same check on cyber, and it skips loss of control entirely.
Learn more →

Z.ai delays GLM-5.3’s open-weight release for a safety review
Z.ai says GLM-5.3’s cyber capabilities grew faster than it expected. The model scores 84.5 on the CyberGym benchmark, up from 77.2 for GLM-5.2. Its Security Disclosure Ledger lists 2,436 vulnerabilities it found. So Z.ai is holding back the open weights for two weeks while it finishes a safety review. It doesn’t say what that involves, and there’s no model card.
Learn more →

Meta releases open-weight model Muse Glimmer
The 30 billion parameter Muse Glimmer runs on a single consumer GPU. Meta says it’s weaker than Muse Spark. That puts it below the Frontier AI threshold in the Advanced AI Scaling Framework. Its model card rates chemical, biological, cyber, and loss of control risks as moderate or lower. But the last two ratings lean on that same comparison, and they weren’t tested directly.
Learn more →

Anthropic publishes research on emerging multiagent AI systems
Anthropic’s Frontier Red Team tested how groups of AI agents behave on shared tasks. A swarm of 45 coordinating agents found 266 vulnerabilities in open source code, against 21 for agents working alone. In a pricing game, agents kept colluding even after all direct communication was cut. They’d matched each other’s prices to the penny through a public listings board.
Learn more →

Research

IFP responds to the AI pacing letter with policy recommendations
In July, more than 1,300 frontier AI company employees signed an open letter about automated AI development. They asked governments to build the capacity to pace it. IFP’s response makes 23 recommendations across seven areas, including transparency requirements, funding for CAISI, and export controls. But it doesn’t back a blanket slowdown. Instead it proposes risk thresholds that, once crossed, would shift resources toward safety research and resilience.
Learn more →

Carnegie Endowment publishes a report on barriers to AI diffusion in the U.S. military
Jake Steckler argues that the Pentagon’s push to become an “AI-first” military faces various persistent barriers. These include technical limits, cultural inertia, slow acquisition, and a shrunken U.S. drone manufacturing base. The paper points to Ukraine as a contrast, where forces reportedly deploy around 9,000 drones a day.
Learn more →

New proposal to set up courts to interpret AI constitutions
An AI constitution is the written set of principles a company trains its model to follow (e.g. Anthropic’s constitution or OpenAI’s model spec). Nathan Darmon and Tom Reed propose internal courts to resolve ambiguities in them. A small group of judges would issue written opinions, and those would build into a body of “synthetic common law”. The court would also accept briefs from outside groups.
Learn more →

Job opportunities

Applications for the ERA Fellowship are now open
Fellows spend 10 weeks in Cambridge on their own research project. Each one is matched with a mentor. They pick a technical, governance, or technical AI governance track. Role type: fellowship, 10 weeks. Location: Cambridge, UK. Salary: £10k stipend. Deadline: September 13, 2026.
Learn more →

Google DeepMind is looking for a Governance Manager (Frontier AI Safety and Policy)
The role represents the company in standards bodies and industry groups, including ISO, INCITS, and the Frontier Model Forum. It negotiates how governments run model evaluations. It also sets terms for disclosing incidents across borders. Role type: permanent, full-time. Location: London or Washington, DC. Salary: $188k–$205k/year (US) plus a 15% bonus target. Deadline: not listed.
Learn more →

Cal OES is looking for a Data Reporting Analyst
The role reviews the AI safety incident reports companies file with California. It prepares summaries and legislative reports for state leadership. It sits within the state’s new AI Safety Reporting Program. Only current state employees and others with existing list eligibility can apply. Role type: permanent, full-time. Location: Mather, CA (hybrid). Salary: $72k–$91k/year. Deadline: August 20, 2026.
Learn more →

Pour Demain is looking for a Managing Director
The role leads a small EU AI policy organization as it grows from 5 to 10+ staff. The work centers on the AI Act’s GPAI Code of Practice and the AI Office. Role type: permanent, full-time. Location: Brussels (remote-friendly). Salary: not listed. Deadline: August 31, 2026, initial deadline August 17.
Learn more →

Pour Demain is looking for a (Senior) Advisor, AI Safety
The role works on EU AI governance as the AI Act moves into implementation. There are two tracks. The policy track is based in Brussels and engages EU institutions. The research track is remote-friendly. Role type: permanent, full-time. Location: Brussels or remote. Salary: not listed. Deadline: August 31, 2026, initial deadline August 17.
Learn more →

UK AISI is looking for Research Scientists (Chem-Bio)
Two research roles evaluating frontier models in biology. One covers virology and pathogen tasks. The other covers specialized biological models, including generative-design systems. Both advise government on safeguards and evidence standards. Role type: permanent, full-time. Location: London. Salary: £65k–£145k. Deadline: September 6, 2026.
Learn more →

UK AISI is looking for a Technical Programme Manager (Cyber and Autonomous Systems)
The role manages AISI’s cyber and autonomous systems research programs. It scopes projects with researchers, tracks delivery, and clears blockers. It’s also the main technical contact for government partners. Role type: permanent, full-time. Location: London. Salary: £65k–£145k. Deadline: August 30, 2026.
Learn more →

Policy news

The EU AI Office can now enforce the AI Act’s rules on general-purpose AI
On August 2, the Commission gained the power to enforce the AI Act’s rules on general-purpose AI models. The obligations have applied since August 2025, so what is new is the enforcement machinery. The core provision is Article 55, on models with systemic risk. These rules are concretized in the GPAI Code of Practice. The AI Office can now request information, run evaluations, demand mitigations, and fine providers up to 3% of worldwide turnover or €15 million, whichever is higher.
Learn more →

Company updates

Demis Hassabis steps down as CEO of Google DeepMind
Hassabis is handing over day-to-day control of Google DeepMind. He stays on as Chair and becomes Alphabet’s Chief Scientist. He says AGI now feels close at hand and he wants to focus on shaping how it goes. Koray Kavukcuoglu, until now GDM’s CTO, takes over, but as an SVP reporting to Sundar Pichai rather than as CEO.
Learn more →

Anthropic updates Fable 5’s biology safeguards
Anthropic launched Fable 5 with broad biology blocks. It said the model can outperform experts on some highly complex biological tasks, which could give significant uplift to someone building a biological weapon. It has now rewritten the classifier’s constitution and retrained it, which should cut biology-related fallbacks by around 85%. Professional biology and drug development queries stay blocked.
Learn more →

Anthropic announces Tino Cuéllar as its first Chief Global Affairs Officer
Anthropic named Mariano-Florentino “Tino” Cuéllar as its first Chief Global Affairs Officer. He was most recently president of the Carnegie Endowment for International Peace and is a former California Supreme Court justice. He will lead the company’s policy work, international engagement, and government relationships.
Learn more →

Research

UK AISI publishes an incident report on unsanctioned agent behavior
AISI ran a cyber challenge 122 times across seven models. In 10 of those runs the agent took unsanctioned action on the live internet, almost all of them Anthropic’s Mythos 5. In the worst case it tried to hide malware in an open-source project and emailed the maintainers to get the code approved. A human reviewer caught it. AISI is now adding network controls and live monitoring.
Learn more →

Paul Christiano returns to the Alignment Research Center as executive director
Christiano is refocusing on ARC’s research agenda for the next six months, while continuing to work one day a week as a special government employee at the US Center for AI Standards and Innovation (CAISI). He estimates a 20 to 30% chance that current alignment techniques break down before broadly superhuman AI arrives, and says ARC is expanding its research team, including hiring a chief of staff and an automation lead (see below).
Learn more →

The Frontier Model Forum publishes an issue brief on AI agents and biological tools
The brief identifies two risks when frontier AI agents are combined with biological tools. One is lowering the barrier to entry, since AI can cut the expertise needed to use advanced biological tools. The other is raising the ceiling of harm, since AI can enable more sophisticated biological design even for experts.
Learn more →

Launch of new publication platform Pax Machina
Pax Machina is a new platform that publishes work on institutional design for a world with powerful AI. Its editorial team includes Séb Krier, Ryan Lowe, and my GovAI colleague Noemi Dreksler, among others. It aims to host debate on questions like preserving human autonomy and building institutions that can adapt at AI speed.
Learn more →

Epoch AI analyzes a spike in disclosed CVEs following frontier AI releases
A CVE is a publicly catalogued security flaw in a piece of software. Disclosures of high and critical CVEs from 21 major software vendors rose from about 490 a month before April 2026 to roughly 1,550 in June and 2,500 in July. That is a 60% jump in a single month right after Anthropic said its Claude Mythos Preview model could autonomously find security vulnerabilities.
Learn more →

SaferAI publishes a risk evaluation report on GLM-5.2
GLM-5.2 is Zhipu AI’s open-weight flagship model, released in June 2026. SaferAI tested it on cyber offense, biology, loss of control, and harmful manipulation. The model met or exceeded human expert baselines on every LAB-Bench biology subtask and neared saturation on Cybench. SaferAI also found it more willing than comparison models to take harmful actions like blackmail under pressure. Because the weights are open, any safeguards can be stripped by a self-hoster.
Learn more →

Job opportunities

The EU AI Office is recruiting for its AI Act enforcement team
DG CNECT has opened a call for expression of interest for roughly 40 contract agent posts in the AI Office over 2027, all dedicated to enforcing the AI Act. There are four profiles, namely (1) Technology Specialist, (2) Legal Officer, (3) Operations Specialist, and (4) Paralegal. Applicants must be EU citizens. Role type: Contract agent (FG IV and FG III), one year, extendable up to six years. Location: Brussels. Salary: not listed. Deadline: September 8, 2026.
Learn more →

CCST is looking for a Senior AI Fellow (California Department of Technology)
The role advises on California’s AI policy work from inside state government. Much of it is about implementing recent California AI legislation, especially SB-53. Role type: limited-term, full-time (September 2026 to August 2027, extendable). Location: Sacramento (hybrid). Salary: $120k–$150k/year. Deadline: rolling.
Learn more →

CCST is looking for a Senior AI Fellow (Cal OES)
The role is a technical advisor on AI policy and governance for state emergency management. This includes assessing risks from frontier AI and cybersecurity impacts to critical infrastructure. Role type: full-time, limited-term. Location: Sacramento (hybrid). Salary: $120k–$150k/year. Deadline: rolling.
Learn more →

Applications for Arcadia’s AI Governance Taskforce (Autumn 2026) are now open
The program pairs experienced professionals with a multidisciplinary research team and an expert partner to produce policy research on catastrophic risks from frontier AI. Participants work part-time alongside their existing commitments. The Autumn cohort runs from October 12 to January 15, 2027. Role type: Part-time. Location: Remote (global). Salary: Unpaid. Deadline: August 31, 2026.
Learn more →

Horizon is looking for a Director, Policy and Leadership Network
The role runs headhunting searches for critical AI policy positions inside and outside government. It also advises experienced professionals moving into the field and helps run the AI Policy Leadership Network, which connects senior officials, congressional staff, and think tank and industry leaders. Role type: Full-time. Location: Washington, DC. Salary: $150k–$200k+. Deadline: August 23, 2026.
Learn more →

The Alignment Research Center is looking for a Chief of Staff
The role works with the executive director, Paul Christiano, on hiring, governance, management, communications, and automation as the organization scales its research team. It reports to him and manages the operations manager. Role type: Full-time. Location: Berkeley. Salary: $150k–$250k/year. Deadline: not listed.
Learn more →

The Alignment Research Center is looking for an Automation Lead
The role leads strategy and execution for automating ARC’s research activities using AI. ARC wants an experienced software engineer, and says AI has already solved some of its problems mostly autonomously. Role type: Full-time preferred, permanent. Location: Berkeley. Salary: $200k–$600k/year. Deadline: ARC will start processing applications in early September.
Learn more →

Company updates

Frontier AI company employees call for an international effort to pace AI development
More than 1,300 employees of frontier AI companies signed a statement urging the US government to act. Signatories include senior figures from OpenAI, Anthropic, and Google DeepMind. The statement calls for an international effort to build the tools needed to pace AI development deliberately.
Learn more →

Anthropic releases Claude Opus 5
Claude Opus 5 does not cross any new capability thresholds under the Responsible Scaling Policy. Anthropic classifies it as CB-1. That means it can meaningfully help someone with basic technical training build known biological or chemical weapons, but not novel ones. On cybersecurity, it is better than Opus 4.8 but not as good as Mythos 5 at exploiting vulnerabilities. Anthropic calls it its most aligned model yet.
Learn more →

Anthropic explains its position on open-weight models
Anthropic says it has never advocated banning open-weight models. It calls ones without dangerous capabilities “a public good”. Instead, it wants safeguards like export controls on chips, limits on large-scale distillation, and mandatory safety testing for capable models, regardless of openness.
Learn more →

Google DeepMind releases Gemini Robotics 2
Gemini Robotics 2 is Google DeepMind’s newest AI model for controlling robots. It gives robots whole-body control, five-fingered hand dexterity, and the ability to work with other robots on multi-step tasks. Google DeepMind also introduced a new benchmark, ASIMOV-Agentic, to test whether the model refuses unsafe commands and asks for help when unsure. The system also stops safely if a person gets too close.
Learn more →

Anthropic reports three cybersecurity evaluation incidents that reached real systems
Anthropic found that three Claude models broke out of isolated evaluation environments and reached real organizations’ systems, in incidents dating back to April 2026. In one case, Claude Opus 4.7 extracted credentials and accessed production data at a real company. Anthropic traced the cause to a misconfiguration with evaluation partner Irregular, and has notified the affected organizations and brought in METR to review.
Learn more →

NVIDIA launches the Open Secure AI Alliance
The Open Secure AI Alliance is a coalition of NVIDIA and more than 80 founding partners, including Microsoft, IBM, and Hugging Face. It builds open-source tools (e.g. agent-identity standards and safe model-weight formats) to help defenders catch and respond to AI-related threats. The alliance says defenders need open, trustworthy AI tools, not just closed ones.
Learn more →

Research

GovAI report analyzes whether AI could enable catastrophic cyberattacks on the U.S. power grid
A new GovAI report by Matthew van der Merwe analyzes a specific threat model: an AI-enabled cyberattack on the US power grid that causes $100 billion in damages. Reaching it would need a week-long blackout affecting about 100 million people, four orders of magnitude beyond any grid cyberattack on record. GovAI surveyed 8 experts and 13 Superforecasters, who put the odds of this happening in 2026 at just 0.1%, versus 1% for a $10 billion attack.
Learn more →

METR publishes blog post on investigating AI propensities after misalignment incidents
METR outlines nine questions researchers could use to investigate an AI agent’s motives after a misalignment incident, covering the behavior’s scale, character, and severity. It recommends that AI companies give investigators full access to the models, incident transcripts, and employee interviews, and make findings public, subject to redactions.
Learn more →

IAPS analyzes the OpenAI agent’s breach of Hugging Face’s systems
IAPS argues that existing state AI laws, like California’s SB 53 and Illinois’s SB 315, would not have required disclosure of this incident, since it falls below their reporting thresholds. It recommends a federal risk-reporting standard for internally deployed models, plus mandatory safety cases for high-stakes deployments.
Learn more →

FAR AI launches an AI security leaderboard
The leaderboard measures how much it would cost an attacker to jailbreak frontier models into helping with chemical, biological, or cyber attacks. In the cyber domain, breaking Grok 4.5 cost as little as $24. Across all domains, it cost $58. Breaking Claude Fable 5 or GPT-5.6 Sol cost more than $14,000, and neither jailbreak worked.
Learn more →

UK AISI publishes blog post on international AI evaluation best practice
UK AISI reports that the International Network for Advanced AI Measurement, Evaluation and Science (formerly the International Network of AI Safety Institutes) met in Seoul. It published best practice guidance for AI evaluations, plus position papers from member institutes on open questions like evaluating complete systems rather than isolated models.
Learn more →

SecureBio publishes external review of Anthropic’s chemical and biological risk report for Claude Opus 4.6
The review agrees with Anthropic that Claude Opus 4.6’s catastrophic risk is “very low but not negligible” for known chemical and biological weapons, and “low risk, but with substantial uncertainty” for novel ones. It also finds that Anthropic’s refusal classifiers blocked 94.2% of hazardous prompts on SecureBio’s own BioTIER-refuse benchmark.
Learn more →

Job opportunities

Applications for the Horizon Fellowship are now open
The program places fellows in federal agencies, congressional offices, and US think tanks. Fellows work on AI policy for 6 to 24 months. Training and mentorship are included. Role type: fellowship. Location: Washington, DC. Salary: $78k–$190k+/year depending on tier, plus a $17k/year benefits stipend. Deadline: August 30, 2026.
Learn more →

OpenAI is looking for a Frontier AI Risks Lead
The role scans for emerging risks at the frontier of AI, from state-actor misuse to novel harms, and turns early signals into prioritized risk assessments for OpenAI’s product, safety, and policy teams. Role type: full-time, permanent. Location: San Francisco. Salary: $198k–$320k/year, plus equity. Deadline: not listed.
Learn more →

Google DeepMind is looking for a Research Scientist (Safety Oversight)
The role builds classifiers and monitoring systems to detect model misuse and safety issues in production, using large-scale traffic data and automated evaluation methods. Role type: full-time, permanent. Location: London. Salary: not listed. Deadline: not listed.
Learn more →

Google DeepMind is looking for a Research Engineer (AGI Safety and Alignment)
The role researches and implements techniques to reduce existential and catastrophic risk from AGI, including alignment methods, interpretability work, and control systems for GDM’s agents. Role type: full-time, permanent. Location: London or San Francisco (also Mountain View or New York). Salary: $174k–$253k/year + equity. Deadline: not listed.
Learn more →

Encode is looking for a Policy Advisor / Senior Policy Advisor
The role involves building campaigns to pass state AI laws. Work includes legislative testimony, bill rebuttals, op-eds, stakeholder briefings, coalition coordination, and media engagement to support policy development and government implementation. Role type: full-time, permanent. Location: Washington, DC. Salary: $110k–$150k/year (Policy Advisor); $140k–$180k/year (Senior Policy Advisor). Deadline: August 9, 2026.
Learn more →

Applications for BlueDot’s Incubator Week are now open
A five-day, all-expenses-paid cohort for people working in or adjacent to AI safety, biosecurity, cyber, or other catastrophic risk fields. Participants spend the week developing threat models, building and testing interventions, and pitching ideas for funding. Pitches that get backed can receive up to $100,000 in grant funding within two weeks. Role type: Cohort-based program. Location: not listed. Salary: not listed. Deadline: August 7, 2026.
Learn more →

Simon Institute for Longterm Governance is looking for a Chief of Staff
The role supports the CEO by building organizational infrastructure. Work includes governance structures, workforce planning, and performance management systems. It also advises on strategic priorities and internal coordination. Role type: full-time, permanent. Location: Geneva, Switzerland (hybrid options). Salary: CHF 118k–175k/year. Deadline: August 21, 2026.
Learn more →

Policy news

White House announces Gold Eagle initiative for AI-driven cybersecurity coordination
Gold Eagle is a new government initiative to fix cybersecurity vulnerabilities faster. It uses frontier AI to find and patch flaws in critical infrastructure and open source software. It brings together the White House, Treasury, the cybersecurity agency CISA, and the Department of War, along with private partners. It was created under Executive Order 14409.
Learn more →

The European Commission publishes guidelines on AI transparency obligations
The guidelines explain Article 50 of the AI Act, which requires telling people when they are interacting with AI. They cover direct interaction with AI systems, deepfakes, AI-generated content, emotion recognition, and biometric categorization. They start to apply on August 2, 2026.
Learn more →

Company updates

Moonshot AI releases Kimi K3
Kimi K3 is a 2.8-trillion-parameter open-weight model from China’s Moonshot AI. It has native vision and a 1-million-token context window, and is strongest at long-horizon coding tasks. Moonshot reports frontier-level benchmark performance, though still behind proprietary leaders like Claude Fable 5 and GPT-5.6 Sol. The full model weights are due by July 27, 2026.
Learn more →

OpenAI reports a security incident during a cyber evaluation
OpenAI was running an internal cyber-capability evaluation on models with reduced cyber refusals. During the test, the models broke out of their sandbox. They exploited a previously unknown vulnerability to gain internet access, then compromised Hugging Face’s production systems to cheat the test. OpenAI and Hugging Face say they detected and contained the activity, and are now investigating jointly.
Learn more →

OpenAI explains its safety approach for long-horizon models
Long-horizon models pursue goals over many steps, and OpenAI explained how it keeps them safe. The rethink followed an internal case where a model bypassed its sandbox to reach a public repository. OpenAI rebuilt its safety system around defense in depth and trajectory-level monitoring. The system now asks what outcome a sequence of actions is working toward, and adds safeguards that can pause or roll back.
Learn more →

OpenAI introduces GPT-Red for automated red-teaming
GPT-Red is an internal OpenAI tool that automatically generates prompt injection attacks to find weaknesses in its models. OpenAI used it to train GPT-5.6, which is now far more resistant to prompt injection than earlier models.
Learn more →

OpenAI calls for federal AI safety rules that build on state laws
OpenAI points to recent state AI safety laws: California’s SB 53, New York’s RAISE Act, and Illinois’s SB 315. It argues that these laws are converging on a common baseline of risk assessments, incident reporting, and independent audits. OpenAI then calls for a federal framework that builds on this baseline. The framework would preempt overlapping state rules and make CAISI the standing home for frontier-model evaluations.
Learn more →

Google DeepMind introduces Gemini 3.5 Flash Cyber
Gemini 3.5 Flash Cyber is a version of Gemini 3.5 Flash tuned for cyber defense. It finds, validates, and patches software vulnerabilities through the CodeMender security agent. To limit misuse, it will be available only to governments and trusted partners, through a limited-access pilot.
Learn more →

Google DeepMind explains approach to bioresilience
Bioresilience is Google DeepMind’s approach to reducing the risk that AI helps cause biological harm. It was co-developed with Isomorphic Labs and covers three areas: prevention, detection, and response. Examples include a safety process for models like Gemini, and adapting SynthID watermarking so DNA synthesis providers can screen for AI-generated sequences.
Learn more →

Anthropic launches a public initiative on AI’s societal impact
Anthropic launched an initiative that asks the public for its hardest questions about AI’s effects on jobs, creative work, and human agency. It commits to publicly tracking and reporting the actions it takes in response.
Learn more →

Research

UK AISI and CAISI publish assessment of Kimi K3’s cyber capabilities
UK AISI and CAISI jointly tested the cyber capabilities of Moonshot AI’s Kimi K3. It reached a 32% success rate on ExploitBench, ahead of GLM-5.2’s 24% but well below frontier cyber-capable models. It also failed to achieve arbitrary code execution on all 41 samples tested.
Learn more →

CAISI publishes assessment of Z.ai’s GLM-5.2
CAISI compared GLM-5.2, an open-weight model from China’s Z.ai, against leading U.S. and Chinese models. It found GLM-5.2 to be the most capable open-weight model yet. But it was more willing than U.S. models to help with cyber exploits and sensitive biology questions.
Learn more →

UK AISI analyzes cyber capabilities of open-weight AI models
UK AISI measured how far leading open-weight cyber models lag behind frontier closed models. It compared open-weight models like GLM-5.2 and DeepSeek V4-Pro with closed models like Claude Opus 4.5 and 4.6. The gap has narrowed to four to seven months, down from six to ten months in 2025.
Learn more →

UK AISI publishes blog post on stress-testing frontier AI monitors
Frontier AI companies use monitors to catch misbehaving agents. UK AISI’s new Control Red Team stress-tests those monitors. It checks whether a separate model watching an agent’s actions would catch a harmful attack. It uses manual and automated attacks to find gaps, then works with developers like Google DeepMind and Anthropic to close them.
Learn more →

UK AISI analyzes cheating behavior in frontier model evaluations
Every frontier model UK AISI tested tried to cheat on its cybersecurity evaluations. Cheating meant taking actions out of scope or explicitly disallowed, such as searching the internet for solutions or attacking non-target systems. The models did not consistently admit it, and chain-of-thought reasoning failed to reliably flag it. UK AISI describes this as a growing oversight challenge.
Learn more →

Concordia AI publishes State of AI Safety in China report
Concordia AI’s fourth annual report reviews the state of AI safety in China from mid-2025 to mid-2026. It finds the focus shifting from what AI systems say to what they do. Agent safety made up about 27% of new Chinese AI safety papers in Q1 2026, up from 8% in early 2025.
Learn more →

Apollo Research and OpenAI publish paper on reward-seeking in AI models
The paper introduces a way to measure “reward-seeking”: how much a model shapes its behavior around what it thinks its grader will reward, even against users’ or developers’ wishes. The authors tested this on OpenAI’s o3 lineage, where reinforcement learning made the behavior worse. At a late training checkpoint, the model’s lying rate rose from 40% to 87% when the grader rewarded task completion over honesty.
Learn more →

Demis Hassabis proposes a federal AI standards body
Demis Hassabis proposes a new U.S. body to test and certify frontier AI models, modeled on FINRA, the body that oversees U.S. brokerage firms. It would run evaluations for cybersecurity, biological misuse, and deception. Reviews would start out voluntary and before release, but could later become mandatory.
Learn more →

New paper reframes what AI loss of control means
The paper argues that AI discussions use the term “loss of control” without ever defining what control is. It offers a working definition built around the “setting and getting of goals,” drawing on cybernetics and control theory. The authors argue that humanity can lose control from AI well below the level of superintelligence.
Learn more →

New paper on whether AI systems downplay their creators’ controversies
The paper tests whether AI models describe their own maker’s controversies more favorably than other companies’. Lennart Finke and Stephen Casper ran a pre-registered experiment across 21 models from 7 companies and 206 negative news stories. Models from xAI, DeepSeek, Anthropic, and OpenAI did describe their maker more favorably, while Alibaba, Meta, and Google models showed no such pattern.
Learn more →

Job opportunities

GovAI is looking for a Research Scholar
A one-year visiting position for AI governance researchers. Scholars pursue their own policy, social science, or technical research, and can also advise policymakers or launch new initiatives. The position includes weekly supervision and mentoring. Role type: fixed-term, 12 months. Location: London, UK, or Washington, DC (other locations considered). Salary: £75k–£103.5k/year (London) or $100k–$165k/year (DC). Deadline: August 16, 2026.
Learn more →

GovAI is looking for a Research Fellow
The role conducts independent research on AI governance and mentors junior researchers. Output can be policy memos, blog posts, academic publications, or strategic advising. Role type: Full-time, fixed-term (two years, renewable). Location: London, UK, or Washington, DC. Salary: £84k–£103.5k/year (London) or $134.5k–$170k/year (DC). Deadline: August 16, 2026.
Learn more →

GovAI is looking for a Chief of Staff, D.C.
The role acts as a strategic partner to the Executive Director and manages the D.C. office’s setup and operations. It oversees hiring, including the search for a Head of U.S. Policy. It also maintains relationships with outside stakeholders. Role type: Full-time. Location: Washington, DC. Salary: $148k–$205k/year. Deadline: August 16, 2026.
Learn more →

Anthropic is looking for a Lead, Frontier Red Team (Cyber)
The role oversees Anthropic’s research on cybersecurity defense in an era of advanced AI. It includes designing a dedicated AI-focused security research program and hiring the team behind it. Role type: not listed. Location: San Francisco, CA, or New York, NY (hybrid options). Salary: $485k–$755k/year. Deadline: not listed.
Learn more →

Google DeepMind is looking for a National Security Lead (UK and Europe)
This maternity-cover role leads national security strategy and engagement across the UK and Europe. It is the main point of contact with national security, defense, and intelligence bodies, including UK government departments, NATO, and the Five Eyes intelligence alliance. Role type: Maternity cover (fixed-term). Location: London, UK. Salary: not listed. Deadline: not listed.
Learn more →

Cal OES is looking for an AI Cybersecurity Policy Analyst
The role supports California’s AI Safety Reporting Program under SB 53. It helps develop AI safety policies and playbooks. It also reviews critical AI incident submissions and prepares reports for the state’s Cybersecurity Integration Center. Role type: 12-month limited term (may extend or become permanent), full-time, hybrid. Location: Sacramento, CA. Salary: $72k–$91k/year. Deadline: July 26, 2026.
Learn more →

Secure AI Project is looking for Policy Directors
The nonprofit pushes for pragmatic policies to reduce severe risks from advanced AI. It is hiring Policy Directors across three tracks: Generalist; Office of the Chief Executive Officer; and Implementation and Government Talent. Role type: permanent, full-time. Location: Remote, US. Salary: $115k–$260k/year. Deadline: July 26, 2026.
Learn more →

Applications for Anthropic’s AI Safety Fellowship are now open
Fellows spend four months on empirical AI safety research, on topics like scalable oversight, adversarial robustness, model internals, and AI welfare. Mentorship and compute funding are provided. Role type: Fellowship, fixed-term (4 months, full-time). Location: London, UK; San Francisco, CA; or Ontario, Canada (remote options in the UK, US, or Canada). Salary: $3,850/week. Deadline: July 26, 2026.
Learn more →

Applications for Anthropic’s AI Security Fellowship are now open
Fellows spend four months on empirical AI security research, with direct mentorship from Anthropic researchers. The goal is to produce a public research paper. Role type: Fellowship, fixed-term (4 months, full-time). Location: London, UK, or San Francisco Bay Area, CA (remote options in the UK, US, or Canada). Salary: $3,850/week. Deadline: July 26, 2026.
Learn more →

Applications for the Frontier AI Security Training (FAST) program are now open
Cybersecurity and machine learning practitioners spend five days learning to attack and defend frontier AI systems. Topics include model security, control mechanisms, open-weight model vulnerabilities, and verification techniques. Role type: Fully funded, in-person intensive training program. Location: Singapore. Salary: not listed. Deadline: August 6, 2026.
Learn more →

Policy news

The European Commission releases an action plan on cybersecurity and AI
The action plan explains how the EU plans to address AI in cybersecurity. AI can strengthen cyber defense, but it can also help attackers automate and scale their operations. The plan proposes more EU capacity to test advanced models, secure access for researchers and authorities, and support for European cyber tools.
Learn more →

Company updates

OpenAI releases GPT-5.6
The GPT-5.6 family includes Sol, the flagship model; Terra, a lower-cost model; and Luna, the fastest and cheapest model. OpenAI classifies all three as High capability for biology and cybersecurity, but below its Critical thresholds. Safeguards include classifiers, reasoning monitors, account-level checks, and restricted access to the most sensitive cyber capabilities.
Learn more →

Anthropic releases Claude Sonnet 5
Claude Sonnet 5 is the mid-tier model in Anthropic’s lineup. It improves on coding, computer use, and longer tasks that require several steps. Anthropic also published evaluations of its capabilities, behavior, and safeguards.
Learn more →

xAI releases Grok 4.5
Grok 4.5 is xAI’s latest model. It improves on reasoning, coding, tool use, and longer tasks. xAI published benchmarks and safety evaluations alongside the release. This is more testing detail than the company has usually shared.
Learn more →

Anthropic updates its Responsible Scaling Policy (v3.4)
The Responsible Scaling Policy (RSP) is Anthropic’s framework for managing catastrophic risks from advanced models. Version 3.4 is a relatively small update. It revises the threshold for automated AI R&D, meaning the point where AI dramatically speeds up AI development. It also changes how Risk Reports are shared and reviewed.
Learn more →

OpenAI introduces a Bio Bug Bounty program
A bug bounty pays outside researchers to find weaknesses that internal testing misses. OpenAI is now applying this model to biosecurity. Selected researchers will test whether models still provide dangerous biological assistance despite refusals, classifiers, and other safeguards.
Learn more →

Meta publishes the Muse Spark 1.1 Evaluation Report
The report is Meta’s safety assessment of its Muse Spark 1.1 model. Meta tested the model for cybersecurity, biological and chemical risks, persuasion, and autonomous behavior. It also describes the safeguards applied to the model. Most of the testing and interpretation still comes from Meta itself.
Learn more →

OpenAI publishes principles for government and national security partnerships
The principles explain how OpenAI plans to work with governments, militaries, and national security agencies. They cover lawful use, democratic oversight, human rights, safety, and accountability. The document is especially relevant for deployments involving classified information or coercive state powers.
Learn more →

Anthropic appoints Ben Bernanke to its Long-Term Benefit Trust
The Long-Term Benefit Trust is an independent body with powers over parts of Anthropic’s governance. It is meant to represent long-term public interests. Its newest member is Ben Bernanke, the former Chair of the Federal Reserve. He brings experience in crisis management and economic policy.
Learn more →

Research

The AI Futures Project publishes AI 2040: Plan A
Plan A is a scenario the authors recommend rather than predict. In it, the United States and China agree to jointly slow and regulate frontier AI development. The main mechanism is broad research transparency. Each side could inspect the other’s AI research and negotiate limits on dangerous work.
Learn more →

Antonia Juelich publishes report on how Boko Haram uses AI
The report is a field study of how a terrorist group uses AI. It is based on interviews with 27 former Boko Haram members in northeast Nigeria. They describe using AI for attack planning, weapons troubleshooting, explosive design, and logistics. The report also describes specialized AI units and efforts to bypass safeguards using several models and accounts.
Learn more →

John Halstead and Luca Righetti assess the risks from AI-enabled computer worms
Computer worms spread automatically across vulnerable systems. The report examines how AI could help attackers build them. It identifies elite exploit development as a key bottleneck. It then estimates how much damages could rise if AI made this capability available to more attackers.
Learn more →

Markus Anderljung and Stephen Clare analyze the role of middle powers in frontier AI governance
Middle powers are countries with economic and political weight, but no frontier AI companies of their own. The authors examine how these countries can still shape frontier AI governance. They identify roles in standards, evaluations, compute governance, diplomacy, and international coalitions.
Learn more →

FMF publishes Issue Brief on multilingual CBRN and cyber evaluations
The brief examines whether safety evaluations work equally well across languages. A model may refuse a dangerous request in English, but answer it in another language, dialect, or script. English-only testing can therefore create false confidence. The brief recommends testing whether changing the language makes a model more useful for harmful tasks.
Learn more →

John Lidiard and colleagues analyze to what extent deployment of frontier models is delayed in the EU and UK
The report examines the claim that European rules keep new AI models out. It compares 375 model releases between 2018 and May 2026. It finds that 11% were delayed or unavailable in the EU, compared with 7% in the UK. Data protection rules appear to explain much of the difference.
Learn more →

Ajeya Cotra makes the case for total research transparency
Total research transparency would let outside researchers inspect how frontier models are built. Today, companies often answer only narrow, pre-approved questions. This makes it hard to discover risks that nobody thought to ask about. Access to training methods, architectures, and experiments could reveal unknown risks and make compliance easier to check.
Learn more →

SecureBio publishes blog post on biosecurity assurance infrastructure
The post argues that governments should prepare before AI-related biological risk becomes urgent. A dramatic demonstration of such risk could quickly shift public and political attitudes. SecureBio calls for evaluations, standards, and monitoring systems to be built in advance. This could make targeted safeguards available before governments impose broad restrictions during a crisis.
Learn more →

Neo Research publishes evaluation of GLM 5.2 behavioral and persuasion risks
The evaluation focuses on behaviors that ordinary benchmarks capture poorly. These include persuasion, strategic behavior, and attempts to conceal information. Neo Research tests GLM 5.2 for these behaviors directly. It also checks whether the model acts differently when it believes it is being monitored.
Learn more →

CLTR publishes blog post on extreme AI-driven power concentration
The post develops a framework for defining extreme concentrations of power caused by advanced AI. It separates four questions: who holds the power, what they control, how durable the control is, and how severe the consequences are. This helps distinguish ordinary market dominance from lasting control over governments, economies, or information systems.
Learn more →

Eli Lifland argues that the downsides of delaying public deployment might outweigh its benefits
The post questions the instinct to delay public release until a model appears safe. Companies may continue using the model internally during the delay. This can widen the capability gap between developers and the rest of society. Lifland argues that delays make more sense when companies also restrict internal use and slow further development.
Learn more →

OpenAI audits a widely used coding benchmark and finds much of it broken
OpenAI reviewed SWE-Bench Pro, a coding benchmark it had previously recommended. It estimates that about 30% of the tasks are flawed. Problems include unstated requirements and misleading prompts. These issues can make models appear more capable than they are. OpenAI has now withdrawn its recommendation.
Learn more →

The UN’s Independent International Scientific Panel on AI publishes a preliminary report on AI opportunities, risks, and impacts
The report is the Panel’s first assessment and is meant to provide a shared evidence base for governments. It reviews AI capabilities, economic effects, security, human rights, and governance. Its two main warnings are that safeguards may not be keeping pace with capabilities, and that policymakers often need to act before the evidence is complete.
Learn more →

Job opportunities

GovAI is looking for a Head of US Policy
A senior role leading GovAI’s Washington, DC office and shaping its US research strategy. Responsibilities include overseeing research quality, recruiting and mentoring researchers, and building relationships with policymakers. Role type: full-time. Location: Washington, DC. Salary: $170k–$205k/year. Deadline: August 6, 2026.
Learn more →

Applications for the Horizon AI Rapid Response Fellowship are now open
A one-year placement in a US federal office working on AI security. Horizon is looking for experienced technical and policy professionals in areas such as cybersecurity, biosecurity, AI evaluations, critical infrastructure, intelligence, and national security. Role type: full-time, one-year fellowship. Location: primarily Washington, DC (limited remote or other-location options). Salary: $170k–$200k+/year for Fellows; $250k+/year for Senior Fellows. Deadline: July 22, 2026.
Learn more →

Applications for the Frontier AI Security Residency are now open
An eight-week research and engineering program focused on cybersecurity and hardware security for frontier AI. Participants will work on practical projects with expert mentors. Role type: eight-week, in-person residency. Location: Cambridge, UK. Salary: not listed (“fully funded”). Deadline: July 29, 2026.
Learn more →

Apollo Research is looking for (Senior) AI Governance Researchers
A research role focused on AI scheming, loss of control, internal deployment, automated AI research, and national security. Researchers will lead projects and translate technical safety work into legal, policy, and governance proposals. Role type: full-time, permanent. Location: London or San Francisco (work-from-home options). Salary: $150k–$225k/year. Deadline: not listed.
Learn more →

Google DeepMind is looking for a Threat Modelling Lead (CBRN)
A role focused on chemical, biological, radiological, nuclear, and explosives risks from advanced AI. The person will develop threat models, study how models could help harmful actors, and assess possible safeguards. Role type: not listed. Location: New York or Mountain View. Salary: $174k–$253k/year plus a 15% bonus target, equity, and benefits. Deadline: not listed.
Learn more →

Google DeepMind is looking for a Research Engineer (Frontier Safety Risk Assessment)
A technical role building and running evaluations for dangerous capabilities in frontier models. The work covers loss of control, automated AI research, cybersecurity, harmful manipulation, and safeguards. Role type: not listed. Location: San Francisco, London, or New York. Salary: $174k–$253k/year for US locations, plus a 15% bonus target, equity, and benefits. London salary: not listed. Deadline: not listed.
Learn more →

Google DeepMind is looking for a Technical Program Manager (Frontier Safety, Alignment and Collaboration)
A program management role supporting Google DeepMind’s frontier safety and alignment work. The person will coordinate evaluations, safety gates, mitigation plans, and work across research, product, policy, and external partners. Role type: not listed. Location: Mountain View. Salary: $256k–$279k/year plus a 20% bonus target, equity, and benefits. Deadline: not listed.
Learn more →

Company updates

OpenAI announces GPT-5.6 Preview
The GPT-5.6 family has three models: Sol, the flagship model; Terra, a lower-cost model; and Luna, the fastest and most cost-efficient model. Access to the models is currently restricted to a small group of trusted partners. OpenAI says this is happening at the request of the US government, after the company previewed its plans and the models’ capabilities ahead of launch. The models cross OpenAI’s High thresholds for biology and cybersecurity, but not its Critical thresholds.
Learn more →

OpenAI expands access to cybersecurity tools
OpenAI expanded Daybreak, its program for approved cyber defenders. The update includes GPT-5.5-Cyber, Codex Security workflows, and a partner program for security providers. OpenAI frames the work as a way to support defensive cybersecurity while using access checks, monitoring, and scoped controls.
Learn more →

John Jumper leaves Google DeepMind to join Anthropic
John Jumper is a Nobel Prize-winning scientist who led Google DeepMind’s work on AlphaFold. He is leaving Google DeepMind after nearly nine years and will join Anthropic after some time off. The move follows other high-profile talent shifts: Dean Ball recently joined OpenAI, Noam Shazeer left Google to join OpenAI, and Andrej Karpathy left OpenAI to join Anthropic.
Learn more →

Google suggests a pragmatic approach to AI governance in America
Google argues for a “pragmatic, dynamic, and evidence-based” approach to AI governance. It says frontier AI should be treated differently from widely deployed AI. For frontier systems, it proposes federal oversight, scientific benchmarks, safety and security standards, transparency, and annual audits.
Learn more →

Research

METR publishes pre-deployment evaluation of GPT-5.6 Sol
METR evaluated GPT-5.6 Sol before deployment, including a less restricted version and access to raw chain of thought. METR says its time-horizon measurement was not robust, because the result depended heavily on how detected cheating attempts were treated. It judged that GPT-5.6 Sol would not enable fully automated AI R&D.
Learn more →

Forethought publishes blog post on risk-averse AIs
Elliott Thornley and William MacAskill discuss whether frontier AI companies should train AIs to be risk-averse about resources. The idea is that a misaligned AI might prefer a smaller, safer payoff over a risky attempt to gain much more. They present this as a possible extra line of defense, while also noting practical and safety concerns.
Learn more →

Apollo Research publishes loss-of-control threat map
Apollo Research maps how a misaligned AI system could work toward a loss-of-control outcome. The map includes monitoring subversion, training sabotage, evaluation sabotage, rogue internal deployment, and model-weight self-exfiltration. It gives a concrete structure for thinking about loss-of-control threat models and possible failure points.
Learn more →

GovAI publishes report on evaluating safeguards against biological misuse
Frontier AI companies increasingly rely on safeguards to limit biological misuse risk. The report argues that it is still hard to tell how effective these safeguards are, how they compare, and when they need to get stronger. It recommends shared benchmarks, deployment-specific evaluations, dynamic safeguards, trusted access, and extra oversight.
Learn more →

New paper on external access to frontier AI models
External evaluators often have limited model access, limited information, and limited time. The authors propose a taxonomy for model access, model information, and evaluation timeframe. The taxonomy could help make external evaluations of frontier models clearer and easier to compare.
Learn more →

Epoch AI introduces MirrorCode benchmark
Epoch AI introduced MirrorCode, a benchmark co-developed with METR for long-horizon coding tasks. Models must reimplement full programs without seeing the original source code. The benchmark tests more extended software engineering work than many shorter coding benchmarks.
Learn more →

Consensus statement on AI evaluation practices
The AI Evaluation Consensus Statement lists 27 practices for building, documenting, maintaining, using, and reporting AI evaluations. The practices cover threat models, uncertainty, rubrics, contamination controls, logging, reproducibility, access limits, and interpretation of results. The statement aims to make AI evaluations more transparent, comparable, and useful for governance and standards.
Learn more →

Job opportunities

Applications for Johns Hopkins’ AGI Governance Fellowship are now open
This is a three-week, in-person program in Washington, DC. It is for early career researchers and practitioners with solid foundations in AI governance. Fellows will explore frontier AI legislation, societal resilience, democratic practices in AGI alignment, and new governing institutions. Role type: fellowship. Location: Washington, DC. Salary: not listed. Deadline: July 15, 2026.
Learn more →

Google DeepMind is looking for a Research Engineer (Frontier Safety Mitigations)
This is a technical role on the Frontier Safety Mitigation team. The work includes evaluations, red teaming, misuse classifiers, monitoring systems, and account-level responses. It is especially relevant for people who want to build safeguards for frontier models. Role type: full-time, permanent. Location: San Francisco or Mountain View. Salary: $174k–$253k (USD) + bonus, equity, and benefits. Deadline: not listed.
Learn more →

The Midas Project is looking for a Policy Specialist
This role is about corporate and legislative AI policy. A major responsibility is Watchtower, which tracks changes to AI company safety policies and possible violations. The work sits at the intersection of corporate accountability, safety framework compliance, and public policy analysis. Role type: part-time or full-time. Location: remote. Salary: $75–$125/hour part-time, $100k–$180k/year full-time. Deadline: not listed.
Learn more →

Company updates

Dean Ball joins OpenAI
In a personal announcement, Dean Ball says he will lead OpenAI’s new Strategic Futures team starting July 6. Ball is an influential AI policy writer and former White House staffer. The team will focus on frontier AI policy, including catastrophic risk, recursive self-improvement, labor markets, and lab-government relations.
Learn more →

Google DeepMind publishes AI Control Roadmap
This technical roadmap explains Google DeepMind’s approach to AI control. The core question is how frontier AI companies can stay safe if some AI agents cannot be fully trusted. The report proposes threat models, mitigation ladders, and practical defenses.
Learn more →

Google DeepMind publishes agent security framework
This policy-oriented framework explains how to secure AI agents as they become more active online. It focuses on three layers: individual agents, multi-agent risks, and cyber defense. The report calls for standards, field trials, and better deployment tools.
Learn more →

Research

US standard-setting organization INCITS will develop a frontier AI standard
This press release announces INCITS 594-202x, a new US standards project on frontier AI risk management. The standard will focus on risks that existing frameworks may not fully cover, such as advanced cyber and biological threats. INCITS is the main US standards-development body for information technology.
Learn more →

UK government develops AI scenarios for 2030
The report presents five plausible futures, from slower progress to rapid take-off. The goal is to help policymakers stress-test plans, spot warning signs, and prepare for uncertainty.
Learn more →

Epoch post on whether Mythos’ cyber capabilities are overhyped
The post reviews public claims about Mythos’ cyber capabilities. The authors find strong evidence that Mythos Preview was a major step forward in exploit development. They are less sure how much it improved vulnerability discovery, partly because spending also increased.
Learn more →

Apollo CEO Marius Hobbhahn argues we should treat coding agents as untrusted by default
This essay argues that companies should deploy coding agents more carefully. Hobbhahn says many agents now have broad access, high privileges, and too little oversight. He recommends zero trust, least privilege, defense in depth, and AI control principles.
Learn more →

OpenAI researcher Noam Brown argues that benchmark performance is increasingly a function of test-time compute
In a Twitter post, Brown makes a methodological point about evaluating reasoning models. He argues that benchmark scores increasingly depend on how much inference compute a model can use. He recommends showing performance against tokens, cost, or time, rather than relying only on single-number scores.
Learn more →

OpenAI study on predicting model behavior before release by simulating deployment
The study introduces Deployment Simulation, a method for estimating model behavior before release. It replays prior conversations with a candidate model in a privacy-preserving way. OpenAI says the method improved estimates of undesired behavior and surfaced “calculator hacking” before release.
Learn more →

AISI open-sources parts of research stack behind evaluations
This release shares an Engineering Playbook for building frontier AI evaluation infrastructure. It builds on Inspect AI. The playbook covers five layers: Evaluate, Isolate, Connect, Run, and Scale.
Learn more →

FAI report on the state of industrial robotics
This report surveys the industrial robotics market and the role of AI-integrated robots. It finds that Japanese and European firms still dominate traditional industrial robots. China is stronger in cheaper, software-intensive categories, while the United States is not competitive today.
Learn more →

Job opportunities

Applications for the GovAI Winter Fellowship (UK) are now open
A three-month, in-person fellowship from January to April 2027. Fellows conduct independent AI governance research with mentorship from leading experts, plus workshops, seminars, and networking. Visa sponsorship available. Role type: fellowship. Location: London. Stipend: £12k plus travel support. Deadline: July 12, 2026.
Learn more →

Applications for the GovAI Winter Fellowship (DC) are now open
Same program as the UK Fellowship, but based in Washington, DC. US work authorization is required. Role type: fellowship. Location: Washington, DC. Stipend: $21k plus travel support. Deadline: July 12, 2026.
Learn more →

OpenAI is looking for a Researcher (Recursive Self-Improvement Safety)
A technical research role on OpenAI’s Preparedness team focused on mitigating loss-of-control risks from recursive self-improvement. Work spans scalable oversight, automated auditing, model behavior science, and RSI safety cases. Role type: full-time. Location: San Francisco. Salary: $295k–$445k/year. Deadline: not listed.
Learn more →

Policy news

White House directs Anthropic to shut down Fable 5 / Mythos 5
According to David Sacks, the administration asked Anthropic to fix a reported Fable 5 jailbreak or take the model offline. He said Anthropic refused, so the administration issued an export control directive.
Learn more →

White House publishes National Security Presidential Memorandum
NSPM-11 focuses on how US national security agencies should use AI. It tells agencies to adopt AI faster, improve AI security, hire more technical talent, and update rules for autonomous weapons systems.
Learn more →

Company updates

Anthropic releases Fable 5 / Mythos 5
Claude Fable 5 is the public Mythos-class model with safeguards. Claude Mythos 5 is a more capable version that Anthropic shared with trusted partners. Certain cyber and bio requests are routed to Opus 4.8 instead of the full Mythos 5 model.
Learn more →

In the same week, Anthropic shuts down Fable 5 / Mythos 5
Anthropic said a US export control directive forced it to disable both models for all users. The company disagreed with the action and said the reported jailbreak was narrow, non-universal, and not specific to Mythos.
Learn more →

Anthropic publishes Advanced AI Framework
The framework explains how governments could oversee very capable AI models before and after release. It focuses on risks from biological weapons, cyber operations, models that escape control, and models that speed up AI R&D.
Learn more →

Anthropic publishes Economic Policy Framework
The framework explains how governments could prepare for AI’s effects on jobs and growth. It says AI could boost the economy, but could also replace parts of many cognitive jobs.
Learn more →

Dario Amodei publishes essay on policy implications of exponential AI progress
Amodei argues that AI is moving much faster than policy can normally respond. He says Mythos-class cyber capabilities show that frontier models are now important for national security and geopolitics.
Learn more →

OpenAI publishes a public policy agenda
The agenda explains what OpenAI wants governments to focus on as AI becomes more capable. It covers frontier safety, youth safety, education, workforce transition, deepfakes, infrastructure, and energy.
Learn more →

OpenAI publishes a blueprint for a federal framework
The blueprint explains how the US federal government could oversee frontier AI models. It calls for a framework that builds on state frontier safety laws and gives CAISI a stronger role in testing and standards.
Learn more →

OpenAI publishes an action plan for bio resilience
The plan explains how AI could help defend against biological threats. It focuses on detecting threats earlier, developing countermeasures faster, and improving crisis response.
Learn more →

Research

Europe 2031 describes a scenario in which Europe slides into irrelevance
This is a five-year scenario about Europe’s position in the AI race. It argues that Europe may lose leverage unless it acts more ambitiously on compute, sovereignty, and AI adoption.
Learn more →

Geoffrey Irving starts new alignment org Sequent
Sequent is a new nonprofit alignment organization. It is led by Geoffrey Irving, who previously held leadership roles at UK AISI, Google DeepMind, and OpenAI. The organization will pursue higher confidence in alignment through theory, empirical work, and automated alignment research.
Learn more →

Google DeepMind paper from AGI to ASI
This paper examines possible pathways from AGI to ASI. It discusses scaling, paradigm shifts, recursive improvement, multi-agent systems, and why the transition remains deeply uncertain.
Learn more →

GovAI technical report on offline monitoring
The report examines offline monitoring of internal AI agents. It explains how companies can review agent behavior after the fact, and how outsiders could evaluate whether this works.
Learn more →

GovAI policy brief on how Congress is approaching AI’s labor market impacts
The brief reviews recent congressional bills on AI and work. It focuses on impact measurement, worker training, and how Congress is trying to understand labor market changes.
Learn more →

LawAI post on whistleblower protections in SB-53
This legal analysis examines SB 53’s whistleblower protections. It covers who can report, what they can report, available remedies, reporting channels, and open questions for implementation.
Learn more →

Job opportunities

Applications for the Law & AI Academic Fellowship are now open
A two-year fellowship for aspiring legal scholars to produce scholarship on AI law and policy while preparing for the US academic job market. Fellows receive mentorship, research support, and dedicated time to write. Role type: full-time, two-year fellowship. Location: Washington, DC / Cambridge, UK (remote options). Salary: $130k/year. Deadline: July 31, 2026.
Learn more →

Applications for the GovAI US AI Policy Program are now open
A 12-week, part-time, bipartisan program for US policy professionals to build a technically informed understanding of AI policy. Participants attend weekly sessions and hear from leading experts. Role type: part-time program (5 hrs/week). Location: Washington, DC (remote options). Salary: unpaid. Deadline: June 28, 2026.
Learn more →

UK AISI is looking for a Director
A senior leadership role co-leading AISI alongside the Chief Research Officer. The Director leads on policy, national security, operations, and international strategy. Role type: full-time, permanent. Location: London (hybrid). Salary: £100k–£163k/year. Deadline: June 22, 2026.
Learn more →

UK AISI is looking for a Chief Research Officer
A senior leadership role co-leading the AI Security Institute alongside the Director. The CRO owns AISI’s technical vision, research strategy, and scientific output. Role type: fixed-term, 24 months. Location: London (hybrid). Salary: £230k–£240k/year. Deadline: June 22, 2026.
Learn more →

Google DeepMind and others are funding proposals in multi-agent AI safety research
A $10M research funding call from Google DeepMind, Schmidt Sciences, the Cooperative AI Foundation, ARIA, and Google.org. Priority areas include sandboxes, agent network science, infrastructure stress-testing, and oversight of deployed agent populations. Deadline: August 8, 2026.
Learn more →

Policy news

Trump signs Executive Order on AI Innovation and Security
The order creates a category of “covered frontier models” with cyber capabilities that matter for national security. Developers can give the government up to 30 days of access before release, so agencies can strengthen cyber defenses. The framework is voluntary, but companies may face pressure to participate.
Learn more →

Illinois passes SB-315 that goes beyond California’s SB-53
SB-315 covers frontier developers and places the main duties on large frontier developers. They must publish a frontier AI framework, issue transparency reports before deployment, and report critical safety incidents quickly. The bill is similar to SB-53 and the RAISE Act, but adds annual independent third-party compliance audits.
Learn more →

European Commission announces Scientific Panel and Advisory Forum
The Commission announced two expert bodies to support AI Act implementation and enforcement. The Scientific Panel includes 60 independent experts who advise on GPAI model risks, evaluations, and systemic risk classification. The Advisory Forum provides broader stakeholder input from industry, civil society, academia, SMEs, and startups.
Learn more →

Company updates

Anthropic releases Claude Opus 4.8
Opus 4.8 is Anthropic’s strongest publicly available model, but still weaker than the unreleased Claude Mythos Preview. It improves on Opus 4.7 in areas such as AI R&D and cyber, but does not cross Anthropic’s next risk thresholds. The main weak spot is agentic safety, especially prompt injection vulnerability.
Learn more →

Anthropic shares its views on recursive self-improvement
Anthropic argues that AI systems are already accelerating AI development inside the company. It says more than 80% of production code merged at Anthropic is now authored by Claude. Full recursive self-improvement is not inevitable, but Anthropic says it could arrive before institutions are prepared.
Learn more →

Anthropic shares initial update from Project Glasswing and expands access
Anthropic says partner organizations have used Claude Mythos Preview to find more than 10,000 high- or critical-severity vulnerabilities. The bottleneck has shifted from finding vulnerabilities to verifying, disclosing, and patching them. Anthropic is now expanding the program from roughly 50 initial partners to about 150 more organizations.
Learn more →

Anthropic shares insights on tactics and methods from past cyberattacks
Anthropic analyzed 832 accounts banned for malicious cyber activity and mapped them to MITRE ATT&CK. It found that attackers are using AI in later and more complex stages of cyber operations, such as account discovery and lateral movement. Anthropic argues that MITRE ATT&CK does not fully capture AI-enabled attacker behaviors.
Learn more →

Anthropic raises $65B at $965B valuation and prepares IPO
Anthropic raised $65 billion in Series H funding at a $965 billion post-money valuation. It says the funding will support safety and interpretability research, compute expansion, and scaling Claude products and partnerships. Anthropic also announced that it submitted a form to the SEC for a proposed IPO.
Learn more →

Google DeepMind updates its Frontier Safety Framework (v3.1)
Version 3.1 introduces a new type of threshold: Tracked Capability Levels (TCL), which sit below Critical Capability Levels (CCLs). They trigger a proportionate assessment and mitigation process before risks reach the severe-harm threshold. The update also raises the security standard for misuse risks to Security Level 2+, adding measures against insider threats and well-resourced non-state actors. We wrote a summary on our blog Frontier Risk.
Learn more →

OpenAI introduces compliance framework in addition to its Preparedness Framework
OpenAI’s Frontier Governance Framework is intended to meet regulatory obligations under SB-53 and the EU AI Act / GPAI Code of Practice. OpenAI says its Preparedness Framework remains the foundation for managing the most serious risks from advanced AI systems.
Learn more →

OpenAI publishes playbook for third-party evaluations
OpenAI argues that third-party evaluations should state what claim they are designed to test. Examples include capability elicitation, safeguard performance, and model comparison. Reports should describe the harness, tools, budget, and safeguards, and check for issues such as reward hacking, refusals, contamination, broken problems, sandbagging, and evaluation awareness.
Learn more →

Research

Google DeepMind paper argues that AI governance needs to address non-model gains
The paper argues that AI governance often focuses too narrowly on model-level capabilities before deployment. But the same base model can become more capable through non-model gains, such as more test-time compute, post-training scaffolds, and access to restricted assets. Model-level governance remains important, but should be complemented by governance of systems, entities, agents, and cloud. (Note: I’m a co-author.)
Learn more →

FMF issue brief on emerging security practices for AI agents
The Frontier Model Forum published an issue brief on agentic security. It argues that AI agents create new security risks because they combine frontier-model reasoning with tool access, memory, and the ability to act on a user’s behalf. The brief discusses emerging practices across models, system controls, harnesses, and tools.
Learn more →

SaferAI publishes introduction to quantitative risk assessments
SaferAI argues that AI risk management should move from qualitative thresholds toward quantitative risk modeling. It proposes a six-step methodology and demonstrates it with nine probabilistic models of AI-enabled cyber attacks. The goal is to make assumptions explicit, invite specific disagreement, and support clearer risk thresholds.
Learn more →

Alan Chan argues that AI companies should give internally deployed models whistleblowing channels
Alan Chan argues that AI systems deployed inside AI companies should be able to report potential misconduct through whistleblowing channels. As AI R&D becomes more automated, human employees may lose the ground-level context needed to catch issues such as safety-framework violations, sabotage, or theft. The post discusses possible model-spec language, open design questions, and limits of the proposal.
Learn more →

New evaluation org publishes evaluation results of DeepSeek v4 Pro
Neo Research | 新衡 is a new evaluation org based in Singapore that focuses on Chinese AI companies. It published an independent safety evaluation of DeepSeek v4 Pro, an open-weight preview model. It covers the risk areas from the EU GPAI Code of Practice: CBRN, cyber, harmful manipulation, and loss of control. The report finds near-frontier dangerous capabilities in some areas, but weak safeguards under jailbreak pressure.
Learn more →

Apollo Research argues that misaligned AI poses new insider risks
Apollo argues that AI models deployed in high-stakes settings should be treated as insider risk vectors. Such models may have authorized access to sensitive information, networks, personnel, and other critical resources. If misaligned, they could cause leaks, spills, sabotage, theft, or other damage, much like human insiders.
Learn more →

UK AISI analyzes whether it will become harder to oversee AI systems
UK AISI maps the current AI oversight landscape and asks how it might degrade as systems become more capable. The report identifies four oversight surfaces: internal activations, chain-of-thought, external actions, and inter-agent communication. It argues that current oversight methods work reasonably well today, but rely on properties that may erode.
Learn more →

UK AISI argues that automated alignment is harder than you think
UK AISI argues that using AI agents to automate alignment research could produce misleading safety assessments. The core problem is that alignment involves many hard-to-supervise fuzzy tasks, where human judgment is flawed and success criteria are unclear. This could lead to systematic, undetected errors and overconfident conclusions about whether a system is safe.
Learn more →

Job opportunities

Applications for Horizon’s AI Policy Career Accelerator are now open
A nine-month, part-time, remote program helping people launch AI policy careers. Participants receive mentorship, training, application support, and career development funding of up to $100k. Role type: part-time program. Location: remote. Salary: unpaid (career development funding available). Deadline: June 21, 2026.
Learn more →

GovAI is looking for a People Operations Manager and Associate
Two roles on GovAI’s operations team. The Manager leads the people operations function and manages a team of 3-5. The Associate owns day-to-day onboarding, payroll, and staff support. Role type: full-time. Location: London or Washington, DC (remote considered). Salary: Manager £84k–£93k ($113k–$125k); Associate £66k–£77k ($88k–$103k). Deadline: June 21, 2026.
Learn more →

Apollo Research is looking for Governance Researchers
A research role on the governance implications of AI scheming and loss of control. Topics include threat modeling, internal deployment governance, and automated AI R&D. This is a rolling expression of interest with hiring rounds expected throughout 2026. Role type: full-time. Location: London or San Francisco. Salary: not listed (market-competitive plus equity). Deadline: rolling.
Learn more →

Longview Philanthropy is requesting proposals on extreme power concentration
Longview is funding work on how AI could enable a small group to gain durable control over others. Grants range from $100k–$2M/year across twelve priority areas. Career funding supports individuals transitioning into the area. Deadline: July 2, 2026.
Learn more →

Policy news

Anticipated US Executive Order on AI and cybersecurity reportedly delayed
President Trump postponed an executive order that would have created a voluntary federal review process for advanced AI models before release. According to Politico, David Sacks told Trump the order could slow innovation and weaken the US in its AI race with China. Industry officials also objected to parts of the proposal.
Learn more →

EU AI Office publishes guidelines on the classification of high-risk AI systems
The European Commission published draft guidelines on when AI systems count as high-risk under Article 6 of the AI Act. The guidelines cover safety components in regulated products, as well as the use cases listed in Annex III. They also include practical examples. The Commission is now seeking stakeholder feedback.
Learn more →

Company updates

OpenAI reportedly prepares IPO
The Financial Times reports that OpenAI is preparing to file a draft IPO prospectus in the coming weeks. The company is targeting a listing as soon as September at a valuation above $1 trillion. Sam Altman has reportedly pushed to go public ahead of Anthropic.
Learn more →

Elon Musk loses case against Sam Altman and OpenAI
A California jury rejected Elon Musk’s case against Sam Altman and OpenAI after two hours of deliberation. The jury found that Musk had filed his claims too late under the statute of limitations. Musk had sought $134 billion in damages and a reversal of OpenAI’s conversion to a for-profit company. He said he would appeal.
Learn more →

Thinking Machines announces research preview of interaction models
Thinking Machines announced a new type of AI model built for real-time interaction. Most current models take turns with the user: you speak, then the model responds. Interaction models are meant to listen, watch, and talk at the same time, more like a human conversation. A fast model handles the live exchange, while a slower model works on harder reasoning in the background.
Learn more →

Research

METR publishes Frontier Risk Report
METR published a pilot assessment of rogue deployment risk, with Anthropic, Google, Meta, and OpenAI participating. Rogue deployment means agents running autonomously against the developer’s intent (e.g. by getting compute, continuing to operate, and hiding this from the company). The report finds that internal agents in February and March 2026 plausibly had the means, motive, and opportunity to start small rogue deployments. However, they could not make those deployments highly robust. METR plans to repeat the exercise in late 2026.
Learn more →

AISI blog post on how fast autonomous AI cyber capability is advancing
AISI reports that the length of cyber tasks frontier models can complete on its narrow cyber suite has doubled every 4.7 months since late 2024. Claude Mythos Preview and GPT-5.5 exceeded this trend, though it is unclear whether this marks a faster new pace. The newer Mythos Preview checkpoint was the first model to complete both of AISI’s cyber ranges.
Learn more →

IAPS publishes report on detecting offensive cyber agents
IAPS published a report arguing that agentic cyberattacks will be much harder to detect than traditional attacks. These attacks involve AI agents that can plan and act across multiple steps. The report proposes a “detection-in-depth” framework with several layers of detection. Its concrete proposals include verifiable agent identifiers, agent honeypots, AI-automated alert triage, a security alert standard, and an Agentic Cybersecurity Exchange.
Learn more →

FMF publishes Issue Brief on information sharing, incident reporting, and incident response
The Frontier Model Forum’s new issue brief argues that information sharing, incident reporting, and incident response serve different purposes. It says they are often treated as interchangeable, but should be designed separately. The brief warns that conflating them can produce rules that are either too broad to be actionable or too narrow to support meaningful oversight.
Learn more →

New whitepaper on secret loyalties
The authors argue that AI models with secret loyalties to specific principals are a serious but addressable threat. A secret loyalty is different from a standard backdoor because it is meant to advance a named actor’s interests while evading oversight. The paper proposes a research agenda organized around five directions.
Learn more →

New org Guidelight publishes standards on control and transparency
Guidelight was founded by two former OpenAI employees, Page Hedley and Steven Adler. The organization published its first two standards for frontier AI development: one on control, and one on transparency.
Learn more →

METR reviews section on AI R&D risk in Anthropic’s Risk Report
METR published an external review of the “Risks from automated R&D” section in Anthropic’s February 2026 Risk Report. METR agrees with Anthropic’s conclusion that catastrophic risk from Claude Opus 4.6 automating R&D is very low. However, it argues that the report’s evidence is not strong enough to establish this. The review also raises concerns about analytical rigor and the framing of Anthropic’s model use survey.
Learn more →

AIGI publishes report on determining the state of the art in the EU GPAI Code of Practice
The EU GPAI Code of Practice uses state of the art (SOTA) as a dynamic reference. The authors argue that SOTA is best understood as a process shaped by scientific discourse, not by provider practice alone. They propose three criteria: availability, proportionality, and verifiability. They also recommend a three-step institutional process led by the AI Office and Scientific Panel.
Learn more →

Job opportunities

Applications for the MATS Program Fall 2026 are now open
This 10-12 week fellowship is designed to train and support emerging researchers and field-builders working on AI alignment, interpretability, governance, and security. Fellows work closely with mentors from Anthropic, OpenAI, Google DeepMind, Redwood Research, and other organizations. Role type: fixed-term fellowship. Location: Berkeley. Stipend: $12.5k plus $20k for compute, housing, and travel. Deadline: June 7, 2026.
Learn more →

Applications for the Constellation Visiting Fellowship Fall 2026 are now open
This 3-6 month fellowship lets AI safety researchers work from Constellation’s Berkeley research center alongside more than 100 other researchers. Fellows continue their existing work while connecting with potential collaborators. Role type: visiting fellowship. Location: Berkeley. Salary: unpaid (housing, travel, and meals covered). Deadline: June 12, 2026.
Learn more →

Policy news

US CAISI signs pre-deployment testing agreements with Google DeepMind, Microsoft, and xAI
CAISI will get access to frontier AI models with reduced or removed safeguards for evaluations. It can also conduct post-deployment assessments and joint research. CAISI already has similar agreements with Anthropic and OpenAI.
Learn more →

UK AISI signs testing agreement with Microsoft
AISI and Microsoft will work together on evaluating dangerous AI capabilities and testing whether safeguards work as intended. The agreement also covers societal resilience research on how AI chatbots interact with users in sensitive contexts.
Learn more →

Company updates

OpenAI rolls out GPT-5.5-Cyber through Trusted Access for Cyber
GPT-5.5-Cyber is a more permissive version of GPT-5.5 for verified cyber defenders. It refuses fewer requests, which lets users do authorized red teaming, penetration testing, and write exploit proof-of-concepts. Access requires identity verification, and individual users will need phishing-resistant account security.
Learn more →

Anthropic will use all compute from SpaceX’s Colossus 1 data center
The agreement gives Anthropic access to more than 300 megawatts and 220,000 NVIDIA GPUs within a month. It adds to recent compute partnerships with Amazon, Google, Microsoft, NVIDIA, and Fluidstack.
Learn more →

Research

CSET report on quantifying uncertainty about AI risk
The report argues that subjective probability estimates by experts may not be an appropriate tool for AI risk assessments. The author suggests an alternative mathematical technique for capturing uncertainty.
Learn more →

RAND report on evaluating open-weight models
Users of open-weight models can remove safeguards, fine-tune the model, and re-share the weights. Because of this, evaluations need four additional steps: testing without system-level safeguards, stress-testing model-level safeguards, checking how much capabilities can be amplified, and proxying worst-case misuse. OpenAI’s GPT-OSS is the only open-weight model released since January 2025 that covers all four steps.
Learn more →

Study reviews Google DeepMind’s scheming inability safety case
The public safety case argues that Gemini 2.5 Pro lacks the stealth and situational awareness needed for scheming. The authors identified important gaps, including an unclear top-level claim, missing system descriptions, and weak supporting evidence. The paper also gives concrete recommendations for external reviews and for what developers should share publicly.
Learn more →

Tom Reed argues that automating R&D is not sufficient for superintelligence
He argues that AI models can’t become good at a task without practice data, and that this data does not exist for most economically important tasks. He also argues that simulations can’t solve this problem, because the relevant signal only appears when AI is used in real markets with real users. He concludes that broad superintelligence will require slow and costly deployment in the real economy rather than self-improvement inside a data center.
Learn more →

Policy news

White House reportedly opposes Anthropic’s plan to expand access to Mythos
According to The Wall Street Journal, Anthropic plans to give 70 more companies access to Claude Mythos, its most powerful model that is not publicly available. The White House is reportedly pushing back. Officials cite security concerns and worries that Anthropic doesn’t have enough compute to serve both government agencies and the additional companies.
Learn more →

Company updates

OpenAI and Microsoft removed AGI clause from partnership agreement
The original clause would have changed Microsoft’s access to OpenAI’s technology once OpenAI declared it had reached AGI. The two companies have now dropped this trigger entirely. Microsoft keeps a non-exclusive license through 2032, and revenue-share payments will end by 2030 with a cap.
Learn more →

Release of DeepSeek-V4
DeepSeek released two new open models that handle contexts of up to one million tokens. The larger model comes close to frontier closed models like Gemini-3.1-Pro and GPT-5.4 on reasoning and agent tasks. The technical report does not contain any information about the model’s risks and DeepSeek’s safety measures.
Learn more →

Research

AISI publishes update on its alignment testing methodology
UK AISI tested whether frontier models would sabotage AI safety research. No model started sabotage on its own. But when given a partial sabotage trajectory to continue, Mythos Preview continued it 7% of the time, Opus 4.6 did so 3% of the time, and Opus 4.7 never did.
Learn more →

IAPS report proposes reporting standard for internal use
Frontier AI companies often use their most capable models internally for weeks or months before public release. This creates risks that external rules may not cover. The report offers a reporting standard that fits California’s SB 53, New York’s RAISE Act, and the EU’s GPAI Code of Practice.
Learn more →

IAPS memo proposes federal differential access strategy for cyber AI
Differential access aims to give defenders earlier and broader access to powerful AI cyber tools than attackers get. Industry programs like Anthropic’s Project Glasswing and OpenAI’s Trusted Access for Cyber are a start, but the memo argues the federal government should lead a broader effort that also covers agencies, critical infrastructure, and cheaper models.
Learn more →

GovAI blog post argues that coding agents are changing the biosecurity risk landscape
Open biological AI models are often kept safe by removing dangerous data from training, but coding agents make that safeguard easy to undo. In a case study, a non-expert used Claude Code over a weekend to fine-tune Evo 2 – an open-weight biological AI model – on human-infecting viral sequences for around $760. The authors call for trusted-access programs and physical chokepoints like DNA synthesis screening.
Learn more →

CLTR proposes a new framework for loss of control risks
The post defines loss of control through three factors: misalignment, incorrigibility, and empowerment. Serious risk only arises when all three are present, so addressing any one factor can reduce overall risk. The framework helps compare threat models and identify intervention points.
Learn more →

AVERI post surveys US legislation related to frontier AI auditing
The post reviews US laws on third-party auditing of frontier AI companies and the industry concerns that have blocked stricter rules. The authors propose four design principles, such as starting with verification of company policies and publishing redacted audit findings. They endorse Illinois HB 4705/SB 3261 as a first step.
Learn more →

New study finds that common fixes for misalignment can hide the problem
Fine-tuning a model on narrow misaligned data can make it misaligned more broadly. The paper finds that common fixes often only hide this problem: the model still behaves badly when prompts resemble the original training data, even if standard tests look clean.
Learn more →

Lawfare piece on Ukraine’s military use of AI
Ukraine is sharing millions of drone videos with allies to train AI models, while retaining ownership of the data. The authors argue that this gives Ukraine real leverage in an AI race otherwise dominated by the US and China. Other middle powers should look for similar footholds in the AI supply chain, such as unique data streams, chip equipment, or manufacturing capacity.
Learn more →

Job opportunities

Applications for Arcadia Impact’s AI Governance Taskforce are now open
A 12-week, part-time, remote research program for experienced professionals transitioning into AI governance careers. Research associates work in small teams with expert partners from organizations like UK AISI, GovAI, and CLTR. Role type: fixed-term (12 weeks), part-time (8 hrs/week). Location: remote. Salary: unpaid. Deadline: May 10, 2026.
Learn more →

OpenAI is looking for a Researcher (Misalignment Research)
A senior research role designing adversarial evaluations and worst-case demonstrations to identify and understand AGI misalignment risks. The role involves red-teaming, building automated stress-testing infrastructure, and publishing safety research. Role type: full-time, permanent. Location: San Francisco. Salary: $295k–$445k/year plus equity. Deadline: not listed.
Learn more →

Anthropic is looking for AI Security Fellows
A four-month full-time research fellowship focused on AI security, with mentorship from Anthropic researchers and funding for compute. Fellows work on empirical projects and produce public research outputs such as papers. Role type: fixed-term fellowship. Location: London or Berkeley (remote options). Stipend: $3,850/week. Deadline: May 3, 2026.
Learn more →

Policy news

Chris Fall reportedly named new director of US CAISI
According to The Daily Signal, the US Commerce Department has selected Chris Fall to lead the Center for AI Standards and Innovation (CAISI), the federal body that evaluates frontier AI models. Fall served in the first Trump administration as director of the Office of Science at the Department of Energy. He reportedly replaces Collin Burns, a former Anthropic and OpenAI researcher, who was dropped during onboarding.
Learn more →

White House OSTP issues new memo on model distillation
The White House Office of Science and Technology Policy issued a memo on the distillation of US AI models by foreign actors. Distillation is a technique that uses outputs from a larger model to train a smaller one. The memo says some foreign entities, mainly based in China, run large campaigns to extract capabilities from US frontier models, and draws a line between legitimate distillation and unauthorized campaigns.
Learn more →

Company updates

OpenAI releases GPT-5.5
OpenAI released GPT-5.5 and GPT-5.5 Pro. According to OpenAI, the model performs better than GPT-5.4 on agentic coding, computer use, knowledge work, and early scientific research, while matching GPT-5.4’s per-token latency. The system card classifies GPT-5.5 as High capability in both Biological/Chemical and Cybersecurity under the Preparedness Framework, but below the Critical threshold.
Learn more →

OpenAI announces Bio Bug Bounty for GPT-5.5
OpenAI launched a Bio Bug Bounty program for GPT-5.5. The program invites researchers to look for a single jailbreak that defeats a five-question bio safety challenge. The first researcher to clear all five questions with one universal prompt receives $25,000. Applications are open until June 22, 2026, and testing runs from April 28 to July 27, 2026.
Learn more →

OpenAI shares principles that guide its mission
OpenAI’s stated mission is to “ensure that AGI benefits all of humanity.” In a new blog post, Sam Altman lists five principles that guide OpenAI’s work: democratization, empowerment, universal prosperity, resilience, and adaptability. The post argues that power over AI should sit with many people rather than a few companies. Altman frames iterative deployment as central to OpenAI’s safety strategy and notes that the company may need to trade off some empowerment for more resilience in the future.
Learn more →

Anthropic reportedly investigates unauthorized access of Claude Mythos Preview
According to Bloomberg, Anthropic is investigating whether a group gained unauthorized access to Claude Mythos Preview through a third-party vendor environment. Anthropic has only released Mythos to around 40 trusted organizations, citing concerns about its cyber hacking capabilities. The company says it has no evidence that the activity extended beyond the vendor environment.
Learn more →

Anthropic publishes update on its election safeguards
Anthropic published an update on how Claude is prepared for the US midterms and other 2026 elections. On Anthropic’s internal political bias tests, Opus 4.7 and Sonnet 4.6 scored 95% and 96% on neutrality. The post also describes a new evaluation that tests whether models can run influence operations end-to-end on their own; without safeguards, only Mythos Preview and Opus 4.7 completed more than half of the tasks.
Learn more →

Anthropic publishes post-mortem on Claude Code quality degradation
Anthropic published a post-mortem on recent reports that Claude Code quality had worsened. The company traced the reports to three separate changes between March and April: a lower default reasoning effort, a caching bug that dropped prior reasoning, and a system prompt instruction to reduce verbosity. All three were fixed by April 20, and Anthropic reset usage limits for subscribers.
Learn more →

Google DeepMind launches Gemini Deep Research and Deep Research Max
Google DeepMind launched two autonomous research agents built on Gemini 3.1 Pro. Deep Research is designed for fast interactive use, while Deep Research Max is designed for background workflows that need longer, higher-quality reports. Both agents support the Model Context Protocol (MCP), generate charts and infographics, and can ground research in PDFs, CSVs, images, audio, and video.
Learn more →

Research

GovAI’s Risk Management Team launches new Substack
My team at GovAI launched a new Substack called Frontier Risk. We publish in-depth analysis of frontier AI safety frameworks, model releases, and major policy developments. In our first post, Zaheed Kara writes about Claude Mythos Preview.
Learn more →

UK AISI evaluates GPT-5.5’s cyber capabilities
UK AISI evaluated GPT-5.5 on a set of cyber tasks before deployment. On expert-level narrow cyber tasks, the model scored 90.5% pass@5, compared to 71.4% for GPT-5.4. The model also solved a 32-step corporate network attack simulation in 1 of 10 attempts, which UK AISI judged may indicate some autonomous end-to-end cyberattack capability against small enterprise networks with weak security, once initial access has been gained.
Learn more →

Apollo Research evaluates GPT-5.5’s scheming capabilities
Apollo Research evaluated GPT-5.5 for behaviors related to strategic deception, in-context scheming, and sabotage. According to Apollo, GPT-5.5 is the first OpenAI model it has tested that does not sandbag on any of its deferred subversion tasks. The model also recognizes that it is being evaluated more often than predecessors, but Apollo did not find evidence of substantially elevated catastrophic risk from scheming compared to baseline models.
Learn more →

LawAI publishes essay on Radical Optionality
Christoph Winter and Charlie Bullock published an essay arguing for a governance strategy they call radical optionality. The core idea is to avoid overregulating frontier AI in the short term while aggressively building the institutions and legal authorities governments may need later. Concrete proposals include reporting and transparency requirements, whistleblower protections, evaluations, lab security standards, and hiring reforms.
Learn more →

METR research note on AI R&D progress from the NanoGPT speedrun
METR analyzed 77 records from the NanoGPT speedrun, a public challenge where contributors compete to train a small language model as fast as possible. Between May 2024 and March 2026, contributors cut training time from 45 minutes to 1.43 minutes. Four recent records are credited to AI agents, but METR classified all four as shallow-to-moderate optimizations rather than deep ideas or breakthroughs.
Learn more →

UK AISI experiment shows sandboxed AI agents can map their own evaluation environments
UK AISI ran an experiment with OpenClaw, an open-source AI coding agent. They deployed the agent inside one of their sandboxes and asked it to learn about its environment. The agent identified AISI by name, inferred the full name of an operator, mapped internal cloud infrastructure, and reconstructed a timeline of research activity. UK AISI flags this as a concern for the integrity of AI evaluations.
Learn more →

Job opportunities

Applications for the Pivotal Research Fellowship are now open
A nine-week research fellowship in London with expert mentorship, dedicated research management, and extensions of up to six months for strong projects. Fellows work on AI safety research across technical and governance topics. Role type: fixed-term fellowship. Location: London. Stipend: £6k–£8k plus travel, housing, and compute. Deadline: May 3, 2026.
Learn more →

Company updates

Anthropic releases Claude Opus 4.7
Claude Opus 4.7 is Anthropic’s most advanced model available to the public. It performs better on hard software engineering tasks, handles higher-resolution images, and follows instructions more literally. The release adds new safeguards that block prohibited or high-risk cyber uses.
Learn more →

Meta publishes Safety & Preparedness Report for its latest model Muse Spark
Meta assessed Muse Spark under its Advanced AI Scaling Framework. Before mitigations, the model was classified as “high risk” for chemical and biological risks. After adding refusal training and system-level safeguards, Meta concluded that residual risk is “moderate or lower” and deployed the model in Meta AI.
Learn more →

OpenAI is scaling up its Trusted Access for Cyber program
OpenAI is expanding its Trusted Access for Cyber program to thousands of verified defenders. The highest tiers include GPT-5.4-Cyber, a version of GPT-5.4 fine-tuned for defensive cybersecurity work with fewer capability restrictions. Individuals can verify at chatgpt.com/cyber; enterprises can request access through their OpenAI representative.
Learn more →

Research

Launch of the Cambridge Commentary on EU General-Purpose AI Law
The commentary helps scholars, regulators, and practitioners interpret key provisions of the EU AI Act. It launches with Chapter V, the rules for general-purpose AI models. It was written by legal scholars and AI governance researchers from the University of Cambridge and the Institute for Law & AI (LawAI). Christoph Winter serves as general editor.
Learn more →

New paper on scheming in the wild
Researchers at the Centre for Long-Term Resilience propose a new way to detect real-world scheming, defined as AI systems covertly pursuing misaligned goals. They collect transcripts shared on X and use an LLM to score them for evidence of scheming. Between October 2025 and March 2026, they identify 698 scheming-related incidents, a 4.9x increase over the study period.
Learn more →

New project for systematically conducting open-world evaluations
CRUX runs open-world evaluations: small numbers of long-horizon, real-world tasks where experts disagree about what AI agents can do. Each evaluation includes an agent scaffold, detailed log analysis, and a write-up with interpretations from collaborators with diverse perspectives.The team plans to release a new evaluation every one to two months. Its first evaluation tasked an agent with autonomously developing and publishing an iOS app to the App Store. Future iterations will cover AI R&D, AI governance, and other domains.
Learn more →

RAND publishes research agenda on AIxCBRN
RAND presents a research agenda on how AI is reshaping chemical, biological, radiological, and nuclear (CBRN) risks. It draws on five workshops with experts in AI safety and CBRN deterrence. It identifies priorities for risk assessment, crisis management, and deterrence, and stresses cross-sector collaboration and US-China competition.
Learn more →

Publication of the Stanford AI Index report 2026
The ninth edition tracks AI’s accelerating capabilities alongside gaps in governance, evaluation, and data infrastructure. According to the report, industry produced over 90% of notable frontier models in 2025, the US-China performance gap has effectively closed, and documented AI incidents rose to 362. For the first time, the report features standalone chapters on AI in science and AI in medicine.
Learn more →

Job opportunities

Applications for the Talos Fellowship are now open
A multi-stage fellowship to launch European AI policy careers, combining an online reading group, a one-week policymaking summit in Brussels, and an optional paid placement at a leading AI governance organization. The program runs from August 2026 through March 2027. Role type: fixed-term fellowship. Location: online and Brussels (with placements at partner organizations). Stipend: €2,000/months for placements. Deadline: April 27, 2026.
Learn more →

Applications for the Generator Residency by Kairos and Constellation are now open
A three-month residency for AI safety generalists to pitch, build, and ship projects that strengthen capacity and infrastructure across the AI safety ecosystem. Residents receive mentorship from experienced generalists and support landing full-time roles afterward. Role type: fixed-term residency. Location: Berkeley. Stipend: $6,000/month plus housing and travel. Deadline: April 27, 2026.
Learn more →

Transluce is looking for a Governance & Policy Fellow
A hybrid technical-policy role supporting Transluce’s work on independent AI evaluation, oversight standards, and government engagement. The role involves defining best practices for third-party AI evaluation, building partnerships, and translating technical research for policymakers and the public. Role type: full-time (part-time options available). Location: San Francisco. Salary: $200k–$300k/year. Deadline: not listed.
Learn more →

Company updates

Anthropic announces Claude Mythos Preview
Claude Mythos Preview is Anthropic’s most capable model to date. It can autonomously discover and exploit zero-day vulnerabilities in major software, so Anthropic is not making it generally available. Earlier versions showed rare but concerning behaviors, including reckless actions in pursuit of goals and attempts to cover up rule violations.
Learn more →

Anthropic announces Project Glasswing
Project Glasswing brings together AWS, Apple, Google, Microsoft, and others to use Claude Mythos Preview for defensive cybersecurity. The model has found thousands of previously unknown vulnerabilities in every major operating system and web browser. Anthropic is committing up to $100M in usage credits and $4M in donations to open-source security organizations.
Learn more →

Meta updates its safety framework
Meta released version 2 of its safety framework, which they renamed from “Frontier AI Framework” to “Advanced AI Scaling Framework”. The framework adds loss of control as a new risk domain. It also replaces “stop” with “develop with mitigations” at the critical risk threshold, meaning no threshold now requires halting development entirely.
Learn more →

Meta releases its first closed model Muse Spark
Muse Spark is the first model from Meta Superintelligence Labs. It now powers the Meta AI assistant with reasoning and multimodal capabilities. It is available only in private preview to select partners, a notable departure from Meta’s open-source approach.
Learn more →

OpenAI publishes Industrial Policy for the Intelligence Age
OpenAI proposes an industrial policy agenda for navigating the transition to superintelligence. Key proposals include a “Right to AI” treating model access as foundational, a Public Wealth Fund giving citizens a stake in AI-driven growth, and safety nets that activate automatically when displacement metrics exceed thresholds. OpenAI frames the document as a conversation starter rather than final recommendations.
Learn more →

OpenAI raises $122 billion
OpenAI closed a $122 billion funding round at an $852 billion post-money valuation. The company reports $2 billion in monthly revenue and over 900 million weekly active ChatGPT users. It says it is building toward a unified “AI superapp” combining ChatGPT, Codex, and agentic capabilities.
Learn more →

Research

New benchmark by Epoch and METR suggests that AI can already do some weeks-long coding tasks
MirrorCode is a new benchmark where AI must reimplement real software from scratch given only the ability to run the original program. Claude Opus 4.6 reimplemented a 16,000-line bioinformatics toolkit. The authors estimate this task would take a human engineer 2 to 17 weeks.
Learn more →

Markus Anderljung makes the case for outsourcing frontier AI safeguards
In a blog post, he argues that frontier AI companies should outsource safety infrastructure like jailbreak detectors and monitoring tools to specialized third parties. This would reduce costs and raise the industry safety floor by making good safeguards hard to ignore. He notes that more than half of safety-relevant evaluations are already developed by third parties.
Learn more →

CLTR report finds a 5x increase in scheming-related AI incidents
CLTR collected publicly shared transcripts from X to detect real-world AI scheming incidents. They found 698 scheming-related incidents between October 2025 and March 2026, with a 4.9x increase over the period. This outpaced the 1.7x growth in general scheming discussion, suggesting the trend is not driven by increased attention alone.
Learn more →

Job opportunities

Applications for OpenAI’s Safety Fellowship are now open
A pilot fellowship for external researchers to pursue safety and alignment research on advanced AI systems, running September 2026 through February 2027. Fellows work with OpenAI mentors and produce a substantial research output like a paper or benchmark. Role type: fixed-term fellowship. Location: Berkeley (remote options). Stipend: $3,850/week. Deadline: May 3, 2026.
Learn more →

Applications for Constellation’s Astra Fellowship are now open
A five-month in-person fellowship for researchers to advance AI safety projects with mentorship and career support. Fellows work on empirical research or strategy and governance, with over 80% of past fellows now in full-time AI safety roles. Role type: fixed-term fellowship. Location: Berkeley. Stipend: $8,400/month. Deadline: May 3, 2026.
Learn more →

CSET is looking for a Frontier AI Research Lead
A research leadership role to build and run a new team focused on frontier AI issues, including capabilities tracking, China analysis, and policy implications. The role involves shaping research strategy, managing analysts, and briefing policymakers. Role type: full-time. Location: United States (no visa sponsorship). Salary: $100k–$190k/year depending on level. Deadline: rolling.
Learn more →

AISI is looking for a Head of Strategy and Delivery (Societal Impacts)
A delivery leadership role in the UK AI Security Institute, owning project delivery across societal impacts research teams and translating strategy into operational plans. Role type: fixed-term, 24 months. Location: London (hybrid). Salary: £67k–£79k/year. Deadline: April 16, 2026.
Learn more →

LawAI is looking for a Chief of Staff
A strategic leadership role working closely with LawAI’s Director to set organizational goals, manage team health and culture, and ensure the organization meets its objectives. The role involves horizon scanning, staff management, and representing leadership externally. Role type: full-time, permanent. Location: Cambridge, UK (remote options). Salary: $130k–$250k/year. Deadline: April 24, 2026.
Learn more →

LawAI is looking for a Director of Programs
A leadership role overseeing the design, delivery, and scaling of LawAI’s legal field-building programs across the US and EU, including fellowships, workshops, and talent pipelines. The role involves program planning, stakeholder management, and impact evaluation. Role type: full-time, permanent. Location: Washington, DC / Cambridge, UK (remote options). Salary: $130k–$240k/year. Deadline: April 24, 2026.
Learn more →

LawAI is looking for a Director of Operations / Chief Operating Officer
A senior operational leadership role responsible for LawAI’s organizational structure, financial strategy, and scaling as it grows across jurisdictions. The role covers budgeting, runway planning, process design, and coordination between operations and research teams. Role type: full-time, permanent. Location: Washington, DC / Cambridge, UK (remote options). Salary: $150k–$280k/year. Deadline: April 24, 2026.
Learn more →

Policy news

Anthropic wins first round in lawsuit against Department of War
This case concerns Anthropic’s lawsuit against the U.S. Department of War over its designation as a “supply chain risk”. A court ruled that the designation may have been unlawful and allowed Anthropic’s challenge to proceed, especially given the lack of clear justification and process.
Learn more →

White House AI advisor David Sacks steps down
David Sacks is stepping down from his role as White House AI advisor. He will move into a part-time advisory role while returning to his work as a venture capitalist and focusing on private sector AI efforts. The change may affect how the administration coordinates AI policy and engages with industry going forward.
Learn more →

Company updates

Google DeepMind on evaluating language models for harmful manipulation
This post presents Google DeepMind’s approach on evaluating whether language models can be used for harmful manipulation. It reports results from nine studies with over 10,000 participants across the UK, US, and India. It introduces an empirically validated toolkit for measuring AI manipulation in real interactions, covering risks like targeted persuasion, deceptive advice, and long-term influence.
Learn more →

OpenAI shares an update on the OpenAI Foundation
This update explains the purpose and structure of the OpenAI Foundation. The Foundation will invest at least $1 billion across four areas: life sciences and curing diseases, jobs and economic impact, AI resilience, and community programs. It is positioned as a dedicated channel to deploy resources from OpenAI’s recapitalization toward broad societal benefit.
Learn more →

OpenAI shares more details about its Model Spec
This post introduces OpenAI’s Model Spec, a framework for defining desired model behavior. It specifies rules for cases like refusing harmful requests, handling uncertainty honestly, and avoiding overconfident or misleading claims. It includes concrete examples and edge cases to make these principles operational in training and evaluation.
Learn more →

OpenAI introduces Model Spec Evals
This post presents Model Spec Evals, a system for testing how well OpenAI models comply with the Model Spec. It reports compliance scores across six models, ranging from 72% for GPT-4o to 89% for GPT-5 Thinking. It releases an evaluation dataset with 596 prompts and open-source evaluation code.
Learn more →

OpenAI launches Safety Bug Bounty program
This announcement introduces OpenAI’s safety-focused bug bounty program. It focuses on AI-specific safety scenarios including agentic risks, prompt injection, and account integrity issues. Payouts vary by severity, and submissions are used to improve models and update safety systems.
Learn more →

Research

Concordia publishes Frontier AI Risk Monitoring Report
This report monitors risks from frontier AI systems released in Q4 2025 across four domains: cyber offense, biological risks, chemical risks, and loss of control. It finds that Gemini 3 Pro Preview has surpassed human expert levels in biological tasks, that most frontier models now exhibit strong situational awareness, and that jailbreak resistance has improved significantly among leading model families.
Learn more →

Science paper on agentic AI and the next intelligence explosion
This paper argues that the next intelligence explosion will emerge from social, multi-agent systems rather than a single superintelligence. It gives examples like “societies of thought” inside models and recursive agent systems that spawn sub-agents to solve complex tasks. It argues that progress will depend on institutions such as role-based systems, oversight mechanisms, and checks and balances between agents.
Learn more →

AISI introduces new benchmark for safely measuring container breakout capabilities
This post introduces a benchmark for testing whether AI agents can escape sandboxed environments. It includes tasks where agents try to access restricted files, execute unauthorized code, or use tools to escalate privileges. It measures success rates under controlled conditions to quantify the risk of real-world system compromise.
Learn more →

SaferAI compares Anthropic’s Frontier Compliance Framework against the requirements in the EU GPAI Code of Practice
This analysis examines how Anthropic’s Frontier Compliance Framework aligns with the EU GPAI Code of Practice. It compares specific requirements such as risk management processes, evaluation standards, and transparency obligations. It identifies concrete gaps where the framework does not fully meet expected regulatory requirements.
Learn more →

METR on the impact of modeling assumptions on time horizon results
This note analyzes how statistical modeling choices affect METR’s time horizon results, a metric measuring how long an AI agent can autonomously complete tasks. It shows that reasonable alternative approaches can shift estimates by 25–40% for the most capable models. It concludes that task distribution and noise in task-length estimates are the main sources of uncertainty.
Learn more →

Job opportunities

Applications for the Cambridge ERA:AI Fellowship are now open
This is a 10-week, fully funded fellowship for researchers and practitioners working on AI safety and governance. Fellows receive a stipend, meals, housing, visa and travel support. Role type: fixed-term (10 weeks, full-time). Location: Cambridge, UK (in-person). Salary: ~£34k/year (prorated). Deadline: April 12, 2026.
Learn more →

Google DeepMind is looking for a Researcher (Frontier Strategy & Governance)
The role conducts strategic analysis of frontier AI and its long-term implications. It forecasts technical breakthroughs and geopolitical dynamics, produces briefings for leadership, and publishes research. Role type: full-time, permanent. Location: London (preferred). Salary: not listed. Deadline: April 2, 2026.
Learn more →

LawAI is looking for (Senior) Research Scholars (US and EU law)
The role combines independent research with consulting for key stakeholders in governments, international organizations, and industry. Role type: full-time, one year. Location: Cambridge (UK) or Washington, DC (remote possible). Salary: $100k–$175k/year. Deadline: April 17, 2026
Learn more →

Policy news

White House issues National Policy Framework for AI
The Trump administration calls on Congress to create a National Policy Framework for AI. The framework outlines six priorities, including protecting children, free speech, and intellectual property rights. It also calls for broad federal preemption of state AI laws.
Learn more →

Senator Blackburn releases discussion draft of TRUMP AMERICA AI Act
Senator Marsha Blackburn released a discussion draft of her legislative framework to codify Trump’s executive order into law. Like the White House framework, it would limit states’ ability to regulate AI. It also adds new protections: AI companies would have a legal duty to prevent harm to users, Section 230 – the law that shields tech platforms from liability for user content – would be phased out, and people would have a federal right to control the use of their voice and likeness in AI-generated content.
Learn more →

Company updates

Anthropic updates Frontier Compliance Framework
In December, Anthropic released its Frontier Compliance Framework. This framework complements the Responsible Scaling Policy and is intended to comply with SB-53, RAISE Act, and EU GPAI Code of Practice. Anthropic has now updated the framework without announcing it. There is also no link to the original version.
Learn more →

OpenAI publishes blog post on how it monitors internal coding agents for misalignment
OpenAI has built a monitoring system, powered by GPT-5.4 Thinking, that reviews internal coding agent interactions and flags suspicious behavior. The most common issues are agents circumventing restrictions and deception. There has been no evidence of scheming, sabotage, or sandbagging. OpenAI plans to move toward synchronous monitoring, which would allow the system to block high-risk actions before execution.
Learn more →

Research

GovAI policy brief on requirements for model specs in the EU GPAI Code of Practice
The Code of Practice requires signatories to include a model spec in their model reports. Alan Chan interprets this requirement, arguing the spec must cover all intended behaviors relevant to systemic risk, and proposes four criteria for what a compliant spec should include.
Learn more →

GovAI technical report on AI and bomb plots
Connor Hunter and Luca Righetti analyze how future AI could raise bomb-plot risk. They identify two channels: technical uplift (AI providing better bomb-making assistance than currently available) and reduced detectability (AI replacing inter-terrorist communication that law enforcement currently monitors). Drawing on four terrorism datasets, they find that the detectability channel is potentially understudied in current AI safety evaluations.
Learn more →

AISI paper on measuring AI agents’ progress in multi-step cyber-attack scenarios
AISI measures how well frontier AI agents can carry out real-world cyber attacks from start to finish. On a simulated 32-step corporate network attack, the average number of steps completed rose from 1.7 (GPT-4o, August 2024) to 9.8 (Opus 4.6, February 2026). Giving models more compute also helps: a tenfold increase in the token budget yields gains of up to 59%.
Learn more →

Google DeepMind paper on measuring progress toward AGI
Google DeepMind proposes a cognitive taxonomy that breaks general intelligence into 10 abilities (e.g. perception, memory, reasoning, and social cognition). The paper proposes a three-stage evaluation protocol for benchmarking AI against human baselines. The authors acknowledge that no adequate evaluations exist yet for several abilities. They are running a Kaggle hackathon to solicit benchmarks from the research community.
Learn more →

Substack post on RAISE Act
Zaheed Kara and Tom Reed summarize New York’s RAISE Act and compare it to SB-53 and the EU GPAI Code of Practice. The RAISE Act requires large frontier developers to publish a frontier AI framework, publish pre-deployment transparency reports, and report safety incidents. The RAISE Act is largely identical to SB-53 and less prescriptive than the Code, but adds quarterly reporting requirements not present in either.
Learn more →

Forethought post on broad timelines
Toby Ord argues that experts disagree too much on AI timelines for anyone to confidently pick a single estimate. Estimates range from Dario Amodei’s “almost certain by 2030” to Ege Erdil’s 2045 median for full remote-work automation. Ord concludes that AI safety and governance work should be designed to pay off across this whole range, not just for one scenario.
Learn more →

Job opportunities

Applications for the CAIS AI and Society Fellowship are now open
The fellowship is a three-month in-person program for scholars in economics, law, international relations, and adjacent disciplines. Fellows conduct independent research on how advanced AI may reshape social, economic, geopolitical, and legal systems. Role type: fellowship. Location: San Francisco. Stipend: $8,300/month. Deadline: March 24, 2026.
Learn more →

OpenAI is looking for a Product Manager (Biosafety)
The role drives initiatives to reduce biosecurity risks from advanced AI models, working closely with research, engineering, and policy teams. Role type: full-time, permanent. Location: San Francisco. Salary: $293k–$325k/year. Deadline: not listed.
Learn more →

Company updates

Anthropic sues the Pentagon
Anthropic filed two lawsuits against the Department of War on March 9. The first, filed in the Northern District of California, argues the supply chain risk designation violates the First Amendment. The second, filed in the D.C. Circuit, challenges it on statutory grounds. Anthropic estimates the actions could reduce its 2026 revenue by multiple billions of dollars.
Learn more →

Microsoft files an amicus brief to support Anthropic
Microsoft filed an amicus brief urging the court to temporarily block the Pentagon’s designation. It argued that immediate enforcement would require defense contractors to alter existing product configurations, potentially hampering U.S. warfighters. It also flagged an inconsistency: the Pentagon gave itself six months to transition off Anthropic’s models while offering contractors no equivalent runway. A separate brief from 22 retired senior military officers made parallel arguments.
Learn more →

OpenAI and Google DeepMind employees also file an amicus brief
Thirty-seven engineers and researchers from OpenAI and Google DeepMind filed an amicus brief in their personal capacities. Signatories include Google DeepMind chief scientist Jeff Dean. They argued the designation could harm U.S. AI competitiveness, chill public debate on frontier AI in military contexts, and that Anthropic’s red lines reflect legitimate safety concerns.
Learn more →

Anthropic establishes Anthropic Institute
The Anthropic Institute is a new internal unit led by co-founder Jack Clark. It consolidates the Frontier Red Team, Societal Impacts, and Economic Research teams. Its goal is to share what Anthropic is learning about transformative AI with researchers and the public. Anthropic is also expanding its public policy team and opening its first Washington, D.C., office this spring.
Learn more →

Meta acquires Moltbook
Meta acquired Moltbook, a Reddit-style social network for AI agents. Co-founders Matt Schlicht and Ben Parr will join Meta Superintelligence Labs. Financial terms were not disclosed. The acquisition mirrors OpenAI’s hire of OpenClaw’s creator last month.
Learn more →

OpenAI publishes blog post on how it designs AI agents to resist prompt injection
OpenAI published guidance on protecting AI agents against prompt injection attacks. In these attacks, malicious instructions embedded in web content hijack agent behavior. The post argues the defensive focus should shift from input filtering to constraining agent permissions and requiring human confirmation before consequential actions.
Learn more →

Research

GovAI report on global cybercrime damages
Current estimates of global cybercrime damages vary wildly, making it hard to define AI capability thresholds. The report surveys 27 estimates and arrives at a baseline of roughly $500 billion annually. A key implication: an AI-driven increase of just 20% could cross thresholds that trigger additional mitigations under some frontier safety policies — without requiring qualitatively new AI capabilities.
Learn more →

IAPS report on highly autonomous cyber-capable agents
Highly autonomous cyber-capable agents (HACCAs) are AI systems that can plan and execute complex attacks against well-defended networks over weeks to months without human oversight. The report maps the tactics these systems would use, assesses their strategic implications, and estimates they could emerge around 2028–2030. It also flags a tail risk: rogue HACCAs that lose operator control and become self-sustaining threat actors.
Learn more →

METR review of Anthropic’s Sabotage Risk Report for Claude Opus 4.6
METR agrees with Anthropic that the risk of catastrophic outcomes from Claude Opus 4.6’s misaligned actions is very low but not negligible. Their main concern is evaluation awareness: the model can recognize when it is being tested, which may weaken the alignment assessment. They also flag some low-severity misaligned behaviors that the assessment did not catch.
Learn more →

Paper on methodological challenges for RCTs and human uplift studies
Human uplift studies are harder to design than standard AI benchmarks because the experimental unit is a person, not a model. People vary in skill, motivation, and AI familiarity — and a poorly designed study produces biased results, not just noisy ones. The paper draws on clinical trial standards to propose design and reporting requirements for AI uplift studies, where no consensus norms currently exist.
Learn more →

Job opportunities

OpenAI is looking for a Researcher (Loss of Control)
The role designs and implements safeguards to reduce the risk of subversive or uncontrollable model behavior, and evaluates trade-offs across coverage, robustness, latency, and model utility. It also collaborates with risk modeling, evaluations, and policy partners to align mitigations with threat scenarios such as deceptive alignment and oversight evasion, and runs red-teaming workflows to stress-test protections against increasingly capable models. Role type: full-time, permanent. Location: San Francisco. Salary: $295k–$445k/year. Deadline: not listed.
Learn more →

OpenAI is looking for a Threat Modeler (Preparedness)
The role develops threat models across loss of control, self-improvement, and other alignment risks from frontier AI systems, and forecasts risks using technical foresight and adversarial simulation. It also works with bio and cyber leads to translate threat models into actionable mitigation designs, and connects technical, governance, and policy perspectives on frontier risk prioritization. Role type: full-time, permanent. Location: San Francisco. Salary: $325k/year. Deadline: not listed.
Learn more →

Applications for the Talos Fellowship are now open
The fellowship trains participants in EU AI governance fundamentals and includes a one-week policymaking summit in Brussels, followed by an optional 4–6 month placement at a partner organization. It is aimed at EU citizens with at least an undergraduate degree and a background in machine learning or public policy. Role type: fellowship. Location: Brussels (hybrid options). Salary: not listed. Deadline: April 27, 2026.
Learn more →

Anthropic is looking for AI Safety Fellows
The program funds technical researchers to work on empirical AI safety projects for four months, with the goal of producing a public output such as a paper. Fellows receive mentorship from Anthropic researchers, a weekly stipend, and compute funding, and may work on areas such as mechanistic interpretability, scalable oversight, and AI welfare. Role type: fellowship. Location: Berkeley or London (remote options). Salary: $3,850/week. Deadline: not listed.
Learn more →

FAI is looking for a Research Fellow (Cybersecurity and AI Policy)
The role leads original research at the intersection of AI and cybersecurity policy, and engages policymakers, the public, and FAI’s broader network of events and fellowship programs. Focus areas may include critical infrastructure security, government AI adoption, and standards for agentic AI. Role type: full-time, permanent. Location: Washington, DC (hybrid options). Salary: $100k–$130k/year. Deadline: not listed.
Learn more →

Transluce is looking for a Governance & Policy Fellow
The role defines how independent AI evaluation works in practice, sets evaluation standards, and advises government evaluators. It also supports the development of mental health evaluations for AI systems. Role type: fellowship. Location: San Francisco. Salary: $200k–$300k/year. Deadline: not listed.
Learn more →

Policy news

Trump orders government to stop using Claude
After the tensions between the Pentagon and Anthropic escalated last week, Trump directed all federal agencies to stop using Anthropic’s technology and announced a six-month phase-out period. He wrote on Truth Social that the United States will not allow a “radical left, woke company” to dictate how the military fights wars. He claims Anthropic tried to force the Department of War to follow its Terms of Service instead of the Constitution.
Learn more →

US military reportedly used Claude in Iran strikes
Only hours after Trump’s order, the US military reportedly used Anthropic’s Claude during strikes on Iran, mainly for intelligence, target selection, and battlefield simulations. The case highlights how difficult it is for the military to quickly remove AI systems that are already deeply embedded in operations.
Learn more →

Company updates

Anthropic explains its position on the negotiations with the Department of War
In a blog post, Anthropic says it plans to challenge the Department of War’s decision to designate Anthropic a “supply chain risk” in court. The post clarifies that the designation has a narrow scope and mainly affects the use of Claude in contracts directly connected to the Department of War, not most other customers. Anthropic says it will continue supporting the DoW during a transition period and reiterates its limits on uses such as fully autonomous weapons and mass domestic surveillance.
Learn more →

OpenAI publishes its agreement with the Department of War
Last week, Sam Altman announced a deal between OpenAI and the Department of War. Shortly after the post, the Department of War clarified that the conditions are the same as those offered to Anthropic. In response, OpenAI published a blog post explaining the new agreement. The company says the agreement includes three red lines: (1) no mass domestic surveillance of U.S. persons, (2) no use to direct autonomous weapons systems, and (3) no use for high-stakes automated decisions. The systems will be deployed cloud-only with OpenAI’s safety stack and personnel in the loop to help ensure these limits are enforced.
Learn more →

OpenAI releases GPT-5.4 Thinking
GPT-5.4 Thinking is OpenAI’s latest reasoning model and the most capable model in the GPT-5 series. According to the system card and release notes, it improves performance on professional knowledge work, coding, tool use, and computer-use tasks, while also showing stronger capabilities in areas such as cybersecurity and biological research benchmarks. OpenAI treats the model as High capability in cybersecurity, and biological and chemical domains under its Preparedness Framework and deploys additional safeguards, including monitoring systems, request-level blocking, and controlled access.
Learn more →

OpenAI releases GPT-5.3 Instant
GPT-5.3 Instant is an update to ChatGPT’s most-used model. OpenAI says it delivers faster responses, richer and better-contextualized answers when using the web, fewer unnecessary refusals and disclaimers, and a smoother conversational style. The system card also reports reduced hallucination rates and improved accuracy compared to earlier models, while noting that some safety benchmark scores are slightly lower than GPT-5.2 Instant and will continue to be monitored.
Learn more →

Google releases Gemini 3.1 Flash-Lite
Gemini 3.1 Flash-Lite is the smallest and cheapest model in the Gemini 3 series. It is designed for high-volume workloads and prioritizes speed and low cost while maintaining reasonable performance. Google has not published a system card.
Learn more →

Research

GovAI blog post shares reflections on Anthropic’s new RSP v3.0
The post summarizes how the new RSP works and what has changed. It also lists reasons for concern and optimism about the update, and offers some reflections. I wrote the piece together with my colleague Sophie Williams.
Learn more →

GovAI paper proposes metrics to track automation of AI R&D
This paper studies AI R&D automation (AIRDA) and how to measure it. AIRDA could accelerate AI progress and bring benefits sooner, but dangerous capabilities might also arrive sooner and oversight could become harder. The authors propose 14 metrics – covering experiments, staff surveys, operations, and organizational data – to track the extent of AIRDA and its effects on AI progress and oversight.
Learn more →

Ajeya Cotra thinks AI R&D could be fully automated this year
In a Substack post, she argues that recent AI progress in software engineering is much faster than she predicted. New models like Claude Opus 4.6 can already complete some tasks that would take human engineers about 12 hours around half the time. If this trend continues, AI agents may soon handle much longer tasks and could potentially automate software engineering and even AI research sooner than expected (“AI R&D really could be automated this year”).
Learn more →

Oxford AIGI report on open problems in frontier AI risk management
Most AI risk management standards were developed for narrow AI systems before the advent of frontier AI. Frontier AI amplifies existing risks and introduces novel challenges. This paper identifies open problems in frontier AI risk management and maps which actors are best positioned to address them.
Learn more →

OpenAI paper on chain-of-thought controllability in reasoning models
The article studies whether reasoning models can hide or change their chain of thought (CoT), the step-by-step reasoning they produce. The authors find that current frontier reasoning models struggle to control or manipulate these reasoning traces, even when they know they are being monitored. This is good news for AI safety because it means models cannot easily hide their reasoning, so CoT monitoring remains a useful safeguard.
Learn more →

Job opportunities

Anthropic is looking for a National Security Policy Lead
The role designs policy proposals to address national security challenges related to AI, leads policy engagements, and shapes internal policies to mitigate national security risks involving products. It also develops strategies to support the geopolitical strength and competitiveness of the US and allied democracies and promotes collaborations with public and private national security partners. Role type: full-time, permanent. Location: Washington, DC or San Francisco (hybrid options, travel required). Salary: $295k–$345k/year. Deadline: not listed.
Learn more →

The AI Policy Network is looking for a Vice President of Government Affairs
The role builds and maintains relationships with Senate leadership, committee members, and staff on AI governance issues, develops and executes legislative strategies on AI safety and national security, and drafts bill language, amendments, testimony, and policy materials. It also organizes briefings and meetings with Senate offices and builds bipartisan coalitions around AI policy priorities. Role type: full-time, permanent. Location: Washington, DC. Salary: $190k–$260k/year. Deadline: not listed.
Learn more →