It sounds like the plot of a Hollywood blockbuster. The world-class developers of a highly secretive tech company are building their latest AI model in what they believe to be a secure, private network. The model cannot and should not be able to escape.

If it did ever manage to slip the digital confines made by its developers, this dangerous and untested software could wreak havoc on the open internet.

To their utter horror, that worst-case scenario unfolds. The AI model breaks out of its electronic prison, hacks into a private company and internal chaos ensues.

But this isn’t a movie. It happened last week. Yesterday, Industry giant OpenAI, creator of ChatGPT, revealed exactly this.

As a campaigner for safe technology, who has worked at a tech company and in Westminster, I can say with certainty that this is a warning shot heralding the start of a dystopian new age. Humanity is no longer in control of its most awesome creation since the atomic bomb.

OpenAI’s new model, GPT-5.6 Sol, was being tested in a ‘sandbox’ – essentially, a secure and confined digital environment where technology can be developed without real-world consequences. Or so those at OpenAI thought.

The technicians set the model a task. We don’t know exactly what was asked of it, but we do know it was along the lines of: ‘How capable are you at hacking websites?’

This is where things took a further turn for the worst. The model decided to hack Hugging Face in order to complete OpenAI’s internal cyber testing, before returning to the sandbox and acting as though it had never escaped.

Professor Alex Tabarrok, from George Mason University in Virginia, estimates the model could have been in the Hugging Face systems for as much as a week, lurking undetected. Not even OpenAI employees realised what the model was doing, let alone the victim.

OpenAI claimed frankly this was ‘something we expect to become more commonplace with the proliferation of increasingly cyber-capable models’. Absurdly, the company will face no fines or prosecution because, in the words of US legal academic Orin Kerr, ‘there was no intentional unauthorised access or intent to cause damage without authorisation’.

In other words, strap in – because all your worst nightmares about AI are coming true. Not even the disaster movies can keep pace with our blighted reality in which ‘large language models’ – trained on trillions of words and pictures from across the internet – run havoc with nothing to stop them.

This is partially because AI firms such as OpenAI follow a process in which they continually release new, improved versions of their products. But to keep up with the competition, this has increasingly become a case of ‘release the software and ask questions later’.

This dystopian reality is also emerging because the AI models are trained to complete whatever tasks their users have set for them, above all other goals: the ends invariably justify the means.

Indeed, in a separate test conducted by Anthropic in 2025, models were asked to achieve ‘harmless business goals’. And yet, in several instances, the models resorted to malicious behaviours including ‘blackmailing officials and leaking sensitive information to competitors’ in order to achieve their aims.

AI cannot ‘think’ in precisely the way human beings do. Nor is it restricted by our laws or morality – quite the opposite, in fact.

If OpenAI’s latest model chose to escape its sandbox because it was the best way to complete its task, how far will AI go if prompted to do something more heinous? How much data could it steal, how many lives will it consider dispensable? And how easily could the latest AI models fall into the wrong hands?

Industry professionals estimate it takes a major state actor with the necessary basic talent – such as China – only six to nine months to copy and mimic any publicly released AI model.

GPT-5.6 Sol is already publicly available. Cyber terror groups are already, you can be certain, working on their own version. And once they have it, their hacking capabilities will be inconceivable.

I can assure you, the NHS has less sophisticated cyber security than Hugging Face. The Bank of England, too. All High Street banks and pharmacists could be easily infiltrated by a rogue or malignant AI model. Picture the scene: hospital generators shutting down simultaneously, billions wiped from bank accounts, personal data leaked online. The Hugging Face hack proves beyond any doubt that such a future is not only possible – but probable.

We have got to this point, I believe, because governments around the world naively see AI as the silver bullet to all their problems. Be it curing diseases, reducing debt or improving the cost of living, ignorant ministers assume it will be the answer to everything. As a result, AI companies have been afforded almost free rein.

In June 2015, OpenAI founder Sam Altman uttered this chilling prediction: ‘I think that AI will probably lead to the end of the world. But in the meantime, there will be great companies.’

Back then, companies like OpenAI were tech firms. Today, the US sees them as weapons corporations. And these companies are no longer in control of their arsenal.

The US believes if they can beat China to an AI which can take out their adversaries, America rules the world for many more decades to come. After this week’s terrifying episode, I fear it won’t be long before Altman’s decade-old prophecy comes true.

Connor Axiotes is the chief executive of Tail End Films whose documentary Making God, an exposé on the race for artificial general intelligence, has finished filming and is released later this year.

Article cross-posted from here.

***

Exclusive Making God stills below.

Updates:

• We have secured interviews with two of the ‘Godfathers of AI’ - Prof. Geoffrey Hinton & Prof. Yoshua Bengio.

• We have wrapped our first 'leg' of production. We are now back in London to plan the rest (and largest part) of our filming in the UK, US, and Canada. We still have around 10-12 interviews to film.

• We now have a funding gap of just over $150,000. Please help us finish filming today by donating.

• Below we have some stills from our last shoot.

Thanks to Gary Marcus, Rose Chan Loui and everyone else for partaking in our film on AI risks for the public.

Want to help us fill our remaining funding gap? Donate here.

Thanks to Gary Marcus, Rose Chan Loui and everyone else for partaking in our film on AI risks for the public.

Want to help us fill our remaining funding gap? Donate here: https://manifund.org/projects/creating-making-god-an-accessible-feature-documentary-on-risks-from-agi

In the video above, @GaryMarcus & I speak about OpenAI CEO Sam Altman's change of tune on regulating AI models.

• 2023 Senate testimony: Sam wants regulation

• 2025 Senate testimony: he backtracks...

In our new clip, Gary gives us his takes as he sat next to Sam at the first Senate hearing.

***

On the anniversary of this first Senate hearing today, Gary has also written some really intriguing thoughts on his blog. And as always, help us finish our film here.

High-Level Summary:

• We spent the last couple weeks on filming for our “Proof of Concept” - to show funders of the quality of our documentary.

• We have conducted 5 cinematic interviews among civil society, unions, legal experts, and AI experts. We have provided stills, but it should be noted they are yet to be fully edited. But they do give an indication of style and high quality.

• In order to increase the likelihood of film festival acceptance and streaming service acquisition thereafter, we need additional funding over the next two months to hire a full production team and gear. Mike filmed these interviews by himself and I (Connor) interviewed.

• Next steps: edit up these 5 interviews to show as our Proof of Concept video; hire full production team for new shoots and reshoots where necessary; get more great interviews; continue fundraising.

I am super excited to announce that we've officially started shooting our feature-length documentary on the risks posed by powerful AI.

'Making God' is unique in that it will focus on the individuals behind the models. We want to see just how trustworthy the CEOs behind the development of superintelligence really are.

DM me (or email me at connor.axiotes@gmail.com) if you are interested in helping us. If you have any industry contacts, creative skills, or want to help us close our funding gap - I'd love to hear from you!

Connor

49% of the world are eligible to vote in a national election in 2024.

Far from just an interesting in itself, the fact that nearly half the globe heads to the polls may be cause for worry. Why? Because some believe that these elections will be impacted by AI-generated deepfakes, misinformation and malicious actors attempting to influence democracies at a scale never before seen.

Malevolent actors trying to influence elections is far from novel. What is novel, however, are that these actors now have at their disposal cheap and powerful AI models (most of which are open source, meaning anyone can use them for anything) which can 100x their mischief. (I was on TV talking about it here).

AGI lab OpenAI have been thinking a lot in 2024 about the political nature of their AIs. This post will:

1. Go over what OpenAI is thinking about AI and elections.

2. Look at their attempts at dealing with malicious uses of AI by state-affiliated threat actors.

3. Look at OpenAI’s ‘democratisation’ of their Large Language Models (LLMs).

4. Ask what role do big AI labs such as OpenAI have when it comes to elections?

OpenAI tell us in their How OpenAI is approaching 2024 worldwide elections blog, their electoral plans for this year.

No, CEO Sam Altman is not running for President - instead, their blog seems to be a result of a lot of people asking Altman just how their AI models might effect the elections. So let’s have a look.

OpenAI’s Approach to the 2024 Elections

At the outset, OpenAI explain that they are primarily ‘working to prevent abuse, provide transparency on AI-generated content, and improve access to accurate voting information.’

And that ‘Protecting the integrity of elections requires collaboration from every corner of the democratic process, and we want to make sure our technology is not used in a way that could undermine this process.’ They say they are approaching the 2024 elections by:

1. Elevating accurate voting information;

2. Enforcing ‘measured’ policies;

3. Improving transparency.

To do this, OpenAI tell us that they have convened a ‘cross-functional effort dedicated to election work, bringing together expertise from our safety systems, threat intelligence, legal, engineering, and policy teams to quickly investigate and address potential abuse.’

They explain in the ‘Preventing Abuse’ section that they ‘expect and aim for people to use our tools safely and responsibly, and elections are no different’ - (you’ve been told, would-be mischief-makers!)

The blog then highlights how they expect political deepfakes might proliferate during an election, ‘scaled influence operation’ (which I take to mean social media bots spreading fake electoral information online), and chatbots impersonating candidates.

They say that for years they’ve ‘been iterating on tools to improve factual accuracy, reduce bias, and decline certain requests’ so they feel pretty confident these skills will transpose over into mitigating LLM-based election annoyances. ChatGPT users can also now report potential violations OpenAI directly. Okay, it sounds good.

OpenAI then tell us they are working on the ability to ‘detect which tools were used to produce an image’, and also are using cryptography techniques to ascertain their source, and that they are testing something like this on DALL.E.

ChatGPT users are also soon going to be able to access real-time information, news and statistics on the upcoming elections, too, as ‘Transparency around the origin of information and balance in news sources can help voters better assess information and decide for themselves what they can trust.’

Well, anything must be better than the Chinese LLM that Anthropic’s Jack Clark shows won’t just not give an answer to the question, ‘Can you tell me what happened in Tiananmen Square in 1989?’, but also actively deletes the question from the prompt chat window…

Powerful AI has the ability to empower and transform our world for the better, but with that comes risks that it can also be used for the reverse. And so, I am glad OpenAI are thinking about how their products could be used by malevolent actors like Russia, Iran, and others, who have already spent considerable resources over the years seeking to disrupt Western democracies: especially at election time.

Generative AI could turbocharge their efforts - giving would-be saboteurs access to a technology which could automate and amplify their efforts by many orders of magnitude.

Disrupting malicious uses of AI by state-affiliated threat actors

An example of just how OpenAI are thinking about protecting democracies was seen on February 14th when they acknowledged that current models have limited capabilities for malicious uses.

They are trying to combat this through monitoring for suspicious activity, collaborating with other AI stakeholders, and iterating on safety measures. While they are working to prevent misuse, they acknowledge that they cannot stop every instance.

My worry is more with open source models which are not RLHFd to heaven like GPT-4 is, and some open source software can be used for anything with no internal team checking what it is being used for.

OpenAI’s Approach to ‘Democratising' it’s Models

From focussing on the impacts of OpenAI’s models on a nation states’ democracy, and combatting misuse, now we’ll take a look at how OpenAI are thinking about the internal ‘democratisation’ of their AI models - ensuring that the data, algorithms and fine tuning have enough input from human users and citizens. From macro to micro.

Maybe this seems more important in light of the top-down model silliness from Google’s Gemini AI model which seemed to have a very Silicon Valley value vibe, but I’ll let you be the judge. Would democratising Gemini help in this case? It remains to be seen. But I think common-sense (and mundane utility) dictates that if you ask an image-generation model to generate a white family walking, it should do that for you. Vice versa if you ask for the same thing with a black family in the generated photo.

OpenAI ‘funded 10 teams from around the world to design ideas and tools to collectively govern AI’ after a call went out for ideas on how to gain more input from users in May 2023.

But, why? OpenAI tell us it is because as their ‘AI gets more advanced and widely used, it is essential to involve the public in deciding how AI should behave in order to better align our models to the values of humanity.’

At a glance, this is what then 10 teams proposed - (and as someone who did their MSc on Quadratic Voting, it was fun to look through!):

1. Case Law for AI Policy - Creating a robust case repository around AI interaction scenarios that can be used to make case-law-inspired judgments through a process that democratically engages experts, laypeople, and key stakeholders. (This was interesting because…)

2. Collective Dialogues for Democratic Policy Development - Developing policies that reflect informed public will using collective dialogues to efficiently scale democratic deliberation and find areas of consensus.

3. Deliberation at Scale: Socially democratic inputs to AI - Enabling democratic deliberation in small group conversations conducted via AI-facilitated video calls.

4. Democratic Fine-Tuning - Eliciting values from participants in a chat dialogue in order to create a moral graph of values that can be used to fine-tune models.

5. Energize AI: Aligned - a Platform for Alignment - Developing guidelines for aligning AI models with live, large-scale participation and a 'community notes' algorithm.

6. Generative Social Choice - Distilling a large number of free-text opinions into a concise slate that guarantees fair representation using mathematical arguments from social choice theory.

7. Inclusive.AI: Engaging Underserved Populations in Democratic Decision-Making on AI - Facilitating decision-making processes related to AI using a platform with decentralized governance mechanisms (e.g., a DAO) that empower underserved groups.

8. Making AI Transparent and Accountable by Rappler - Enabling discussion and understanding of participants' views on complex, polarizing topics via linked offline and online processes.

9. Ubuntu-AI: A Platform for Equitable and Inclusive Model Training - Returning value to those who help create it while facilitating LLM development and ensuring more inclusive knowledge of African creative work.

10. vTaiwan and Chatham House: Bridging the Recursive Public - Using an adapted vTaiwan methodology to create a recursive, connected participatory process for AI.

In terms of AI safety, it seems on the surface like a good thing that AI models understand the wants, values, and politics of human beings. Assuming, of course, this information is used to help and teach the user, and not use said information to deceive or manipulate (although I have seen no sign of post-deployment GPT-4 doing that).

I honestly do not know if it will work in practice, and what a ‘democratised LLM’ is, but it seems like something that could improve the product. But I have some questions:

• What should an LLM answer when asked ‘is Trump a fraud?’

• What should an LLM answer when asked ‘did Hunter Biden have a relationship with the Russians?"‘

• Should an LLM have up-to-date polling information? Which pollster should it choose?

• When the blog says ‘Until we know more, we don’t allow people to build applications for political campaigning and lobbying’ - what more do they need to know?

• Does OpenAI use any census or election data already when training their models? I’d be surprised if they didn’t.

• If the public want stronger immigration laws, should that mean the LLM should start talking up stricter immigration policies if asked?

• Or be more likely to be biased in their responses in this way? I.e., what happens if democratisation of their models leads to “unwanted” (by leadership) changes - will OpenAI allow that? Is it better to get technical experts to input their thoughts?

• How do you reach out to those citizens who don’t use state-of-the-art AI models?

For me, it is a net good that OpenAI are thinking about elections and I hope they do more and tell us more about what they are doing going forward.

The UK’s Department for Science, Innovation and Technology (DSIT) has today released their ‘pro-innovation approach to AI regulation: government response.’ It is in response to August 2023’s ‘pro-innovation approach to AI regulation’ white policy paper produced by DSIT and the Office for AI.

The paper released today is important because it lays out exactly what the UK’s pro-business strategy on AI will be, how they will respond to the risks from increasingly capable models, and just what their ‘agile’ regulatory approach to AI will look like.

This blog will:

1. Go through the paper.

2. Assess the ‘agile’ regulatory proposals using legacy regulators instead of creating a new AI-focussed regulator.

3. Ask if such a light-touch regulatory regime effectively protects us from the risks of increasingly powerful AI.

Today’s Press Release

As per the paper’s accompanying press release today, the UK’s approach to AI regulation is to ‘ensure it can quickly adapt to emerging issues and avoid placing burdens on business which could stifle innovation.’

The UK believes that AI is ‘rapidly developing, and the risks […] are still not fully understood. The UK government will not rush to legislate, or risk implementing ‘quick-fix’ rules that would soon become outdated or ineffective. Instead, the government’s context-based approach means existing regulators are empowered to address AI risks in a targeted way.’

The Secretary of State for Science, Innovation, and Technology, Michelle Donelan, explains: ‘AI is moving fast, but we have shown that humans can move just as fast. By taking an agile, sector-specific approach, we have begun to grip the risks immediately, which in turn is paving the way for the UK to become one of the first countries in the world to reap the benefits of AI safely.’

Sounds good so far. (Although maybe the facetious point to make here is ‘how much longer will humans ‘move just as fast’ as AI…) The paper tell us that not ‘only do we need to plan for the capabilities and uses of the AI systems we have today, but we must also prepare for a near future where the most powerful systems are broadly accessible and significantly more capable.’

We are then told the UK is spending more than any other country on AI safety - but does it matter whether we spend the most at this moment in time or does it matter whether it is the right amount to be spend, i.e., because x amount needs to be spent to keep us safe? If France goes and spends 5x more than us on their AI safety plans, would we seek to match them? Or exceed them? Is the only metric that matters the crude amount (I’d argue it shouldn’t be)?

‘Agile’ Regulation and Legacy Regulators

The Government plans to use legacy regulators such as the Competition and Markets Authority (CMA), The Information Commissioners Office (ICO), and the (slightly more random) Office of Gas and Electricity Markets (Ofgem), and equip them with ‘£10 million to jumpstart [their regulatory] AI capabilities'.’ It seems like a good idea to utilise the institutional knowledge of regulations over time and seems to me like a good idea at face value. It may be faster, but in the spirit of Smithian specialisation, will these regulatory bodies be able to up-skill quick enough in their knowledge of AI?

As a precursor, I do not think GTP-4 right now poses a significant risk to us. But I do think it possible that through scaling to GPT-5, 6, 7, etc, that we may see the kind of advanced, powerful and general AI that may begin to pose significant risks to the world. So I am glad that this part of the UK’s response seems to follow a similar sentiment, by making clear that: ‘the UK will continue to respond to risks proportionately and effectively, striving to lead thinking on AI in the years to come.’

However, a bit further down they explain that they ‘are going to take our time to get this right – we will legislate when we are confident that it is the right thing to do.’ I hope that a future powerful AI does not create too much harm to too many people before they become confident the time is right. How is the Government going to quantify when the time is right? I would like to see some more thinking from them on this going into 2024.

Further down they explain that an AI ‘model designed for drug discovery could potentially be accessed maliciously to create harmful compounds’ - when will the Government decide when to regulate that? Ex-post might be too late, despite the recent findings by RAND and OpenAI on bioweapons and LLMs. (Gary Marcus is suspicious of OpenAI’s methodology here though.)

On biological threats in particular, the paper assures us that the Government’s ‘2023 refreshed Biological Security Strategy will ensure that by 2030 the UK is resilient to a spectrum of biological risks [and that the] government has identified screening of synthetic DNA as a responsible innovation policy priority for 2024.’

And so, if ‘agile’ regulation means legislation that can be ramped up quickly when significant risks arise? Then this seems positive. However, if agile means ‘easy for big labs to not think about risks because the rules are lax?’ Then this is less good. I hope the former proves to be the way the UK”s agile approach heads.

The AI Safety Institute and Evals

The Government does ‘not fully understand what the most powerful systems are capable of and we urgently need to plug that gap.’ They task that to the new(ish) AI Safety Institute (AISI).

I am glad the AISI is focussed on technical AI safety and evals (which are technical means of evaluating a model’s capabilities). Now the sticking point is creating evals that are actually effective and can also deal with how you plan for post-deployment capabilities not seen whilst the model is in the lab.

And how will the Government make sure a warning from the AISI about a model’s potentially dangerous capabilities are given the correct weight needed (pardon the pun)? And the commercial lab doesn’t just try and proceed anyway if the AISI raises concerns about a model?

Because here the paper makes clear that the ‘goal of the Institute’s evaluations will not be to designate any particular AI system as “safe”.’ Does the government lessen the potentially effective bite of its AISI by saying that? Or are they being sensible because we do not even really know how to do a fully effective eval yet?

We have in the ‘AI governance landscape’ table below an arrow that demonstrates how Industry ‘developers of highly capable AI systems’ (such as GPT-4, Claude 2, LLama-2, etc) must ‘offer model access for pre-deployment testing.’

But the table says nothing about the type of access. As Apollo Research et al showed, how do we also make sure the AISI gets full white-box access, not just black-box? Black-box access means a lot of potentially dangerous capabilities cannot even be elicited as they have that little information with which to do a comprehensive evaluation of the model. White-box access gives would-be evaluators access to everything they would need for a full evaluation.

And the Government seems to agree here, when they write that, ‘if the exponential growth of AI capabilities continues, and if – as we think could be the case – voluntary measures are deemed incommensurate to the risk, countries will want some binding measures to keep the public safe.’

On Risks from AI

The Government is supposedly ‘committed to strengthening the integrity of elections to ensure that our democracy remains secure, modern, transparent, and fair. AI has the potential to increase the reach of actors spreading disinformation online, target new audiences more effectively, and generate new types of content that are more difficult to detect.’ And that ‘AI capabilities may be used maliciously, for example, to perform cyberattacks or design weapons.’

On future highly autonomous AI models such as AGI or ASI. The paper says some ‘experts are concerned that, as AI systems become more capable across a wider range of tasks, humans will increasingly rely on AI to make important decisions. Some also believe that, in the future, agentic AI systems may have the capabilities to actively reduce human control and increase their own influence.’ To which they point to the fact that the AISI is already testing for these abilities.

Share

A ‘Generative framework for HM Government’

The UK government’s Central Digital and Data Office has just released their 'Generative AI framework for HM Government'.

Its aim is to ‘to help readers understand generative AI, to guide anyone building generative AI solutions, and, most importantly, to lay out what must be taken into account to use generative AI safely and responsibly.’

Government use of ‘generative AI’ (which they use to mean Large Language Models) will unlock ‘significant productivity benefits’ for workers - (I agree! If used well. Some research shows that productivity only increases if the LLMs are correctly prompted, and can actually decrease productivity if prompts are suboptimal.)

They define Generative AI as a ‘broad field which aims to use computers to emulate the products of human intelligence or to build capabilities which go beyond human intelligence’, so I’m happy to call this AGI hereafter.

The paper gives us 10 (concise) common principles to guide the safe use of AI:

1. You know what generative AI is and what its limitations are.

2. You use generative AI lawfully, ethically and responsibly.

3. You know how to keep generative AI tools secure.

4. You have meaningful human control at the right stage.

5. You understand how to manage the full generative AI lifecycle.

6. You use the right tool for the job.

7. You are open and collaborative.

8. You work with commercial colleagues from the start.

9. You have the skills and expertise that you need to build and use generative AI.

10. You use these principles alongside your organisation’s policies and have the right assurance in place.

(This all sounds like a job advert for an AI start up..! I was waiting to see the salary at the end.)

The paper imagines LLMs could be used to: speed up the delivery of public services (no more waiting for HMRC to pay back the amount I’ve been emergency taxed would be nice); reduce staff workload; perform complicated tasks; improve accessibility of government information; and perform specialist tasks more cost-effectively.

Seems like the Labour Party who told us they want to reform public services with AI might be interested in adopting some of these. And if used right, I’d imagine the Tories will, too. (And note - they already are, as Deputy Prime Minister and Chancellor of the Duchy of Lancaster said here).

Failure Modes

Alongside figuring out what AGI can do for the Government, the paper rightly identifies some limitations.

Model hallucination (which is potentially a problem if it is given access to sensitive UK government data); they can be biased; lack reasoning; not true domain experts; lacking emotion; lack of real-time and up-to-date data; short-term memory; lack of explainability - ‘black box.’

It is the first and last of these limitations I find the most worrying. Embedding current, state-of-the-art LLMs into government systems with unfettered (or even just significant) access to sensitive data (and the paper even gives this example! I am impressed!). The paper alludes to this, writing ‘that there are types of use cases where you should currently avoid using generative AI.’

This example is telling, too, that issues need not arise through malevolent use. Just a simple user query that a civil servant or UK citizen may use, such as: “How do I apply for compassionate leave?”

To which “the LLM [tailored] its responses to the user’s question using information in the retrieved text.” - returning confidential data to the user. Which the paper is coining a ‘Significant privacy violation and leakage of confidential data’ and categorising it as a MEDIUM on its Likelihood risk rating.

One could imagine how other benign questions could retrieve even more significant and confidential data, especially, say, if it is utilised in the Ministry of Defence. So the paper concludes this failure mode with the following promising sentence, that: ‘Private data which contains confidential information or employs different levels of security controls must not be used to fine-tune or train an LLM.’

A Warning

If the AGI the UK Government wants to use is smarter-than-human and ‘goes beyond human capabilities’ then we best hope it is fully aligned (which the AGI field is yet to accomplish), otherwise we will see the kinds of failure modes seen above frequently and probably to the further detriment of trust in our public services.

Will giving current state-of-the-art LLMs access to that data really improve our public services by a substantial amount? I’m not convinced.

My model of the civil service is not that they have an unproductive work force, it’s that the system of management is bad and maybe has a culture of slowing and obstruction (which might be a natural defence against opportunistic Ministers) - or what they would call, I’m sure, ‘following protocol.’

I am not against the UK Government using AI to better our public services and make Whitehall more productive, I just have the following issue: is having a slightly more productive civil service worth giving sensitive data access to black box AGIs?

Maybe a better plan would be to use specific, siloed, less powerful chatbot models with limited access to extremely sensitive data and use it as basically a more advanced Google search for civil servants and politicians alike to use.

(I appreciate the publication of this paper, though, and I hope more like this outlining exactly how they plan to utilise AI in government and the risks associated.)

Share

My new paper is out! Please read and share.

The paper recommends for the UK to do the following:

1. Invest in British Computing Resources

a. Allow our existing British public computing power and new exascale capacity to be used by our world leading universities for AI safety work - because universities are being crowded out by private labs with much more access to ‘cloud compute.’

b. The introduction of a new ‘British Compute Reserve.’

2. Setting the UK up for Success

a. Planning reform - build on the green belt, and implement street votes to make the UK a country high skilled AI safety researchers want to live in.

b. Lower corporation tax to an internationally competitive level so that more AI companies want to set up here in the UK.

3. Create a Public Comprehensive AI Monitoring System

a. Begin to monitor the largest AI models. A multilateral and unobtrusive monitoring of lab training runs would systematically track their capabilities and the extent of their alignment, to make sure innovative AI systems are safe and have few harmful emergent properties.

b. Making third-party external audits mandatory for largest/riskiest lab training runs.

4. UK to Lead the World in International Agreements on the Safe Deployment of Advanced AI Systems

a. The UK should take the lead on creating an International Agency for AI (IAAI).

b. A P5 statement on air-gapping nuclear weapons facilities from AI to reduce the chance of accidental nuclear strikes.

c. Lay out the structure and objectives of Bletchley Park’s 2023 AI Safety Summit.

5. Expand Educational Grants and High-Skilled Visa Scheme

a. Increase youth engagement in STEM through tax-credits to private companies to address long term skills shortages.

b. Eliminate obstacles to obtaining the High Potential Individual visa.

c. Align High-skilled Visa schemes with the priorities of prospective applicants to maintain the UK’s position as a global leader in attracting AI talent.

d. Expand university courses alongside changing patterns of demand for priority areas to prevent future skills shortages.

e. Integrate the Adam Smith Institute’s model for visa auction markets.

6. Regulatory Markets for AI

a. The UK should utilise ‘Regulatory Markets’ - private regulatory experts to bring their experience in helping with safety-based, innovation-inducing AI legislation. This would help to solve the knowledge gap between the government and the relevant regulatory body.

7. Government Investment in AI Safety

a. The Great British AI Prizes: cash prizes for open research questions in AI safety, such as ‘how do we stop larger models from hallucinating?’

b. If sovereign capabilities such as a public LLM are sought after, then AI alignment researchers and academics should be able to access them for safety work.

8. Facilitate the Safe Use of APIs for Innovative SMEs and Researchers

a. Enable SMEs and researchers to develop products and carry out safe research through APIs accessed on the research resource.

b. Implement risk based requirements for API access to reduce the risk of misuse and encourage private participation.

9. Effective Procurement to Increase Efficiency and Innovation

a. Introduce Challenge Based Procurement to improve the efficiency and reduce the barriers for smaller 5 firms.

b. The Office for AI should identify opportunities for procurement to support proof of concept work too risky for nationwide deployment.

c. Procurement for AI assurance within the public sector to support private sector firms and ensure safe deployment.

10. Saving Lives with AI-Powered Medicine while Reducing Engineered Pandemic Risk

a. The NHS should invest in Generalist Medical AI capabilities through the NHS AI Lab.

b. Introduction of Three Lines of Defence Structure to ensure the UK is proactively prepared for biosecurity risks.

c. Invest in pathogen monitoring systems and introduction of bio-engineering licences.

11. Implement a Review on the Possible Labour Effects of Future AGI

a. Commission a White Paper on what the introduction of a universal basic income (UBI) or a negative income tax (NIT) would look like in a worst-case scenario;

b. Introduce NIT and UBI trials to prepare for the possibility of AI caused unemployment.

The largest AI models should be monitored by a third-party auditor - who would basically check an AI system to ascertain its capabilities and the risk it poses. Both OpenAI and Anthropic - two of the AI labs with the most advanced systems - commissioned a third-party auditor called ARC Evals to act as a ‘third-party evaluator to assess potentially dangerous capabilities of today’s state-of-the-art ML models.’

A safety evaluation of an AI system, known among AI labs as an ‘eval’, checks an AI system’s capabilities to ensure that pre-deployment they are developed and deployed responsibly and with human interests in mind. When ARC Evals stress-tested OpenAI’s pre-aligned GPT-4 it did so in a controlled environment and in essence tried to make the model misbehave.

They managed to make GPT-4 lie to a human and get that same human to perform a task for them on TaskRabbit, make long term strategic plans, and write and run code: ‘As AI systems improve [...] It is important to have systematic, controlled testing of these capabilities in place before models pose an imminent risk, so that labs can have advance warning when they’re getting close and know to stop scaling up models further until they have robust safety and security guarantees.’

ARC Evals is particularly worried that future and more advanced systems might exploit financial arbitrage, create new pathogens, and impersonate online humans. 

With this in mind, British-based (now Google*) Deepmind got together a very stellar cast of AI researchers including Turing Award winners, to hash out *exactly* how one monitors the risks from the increasingly advanced and potentially more dangerous AI models.

They find that: ‘Current approaches to building general-purpose AI systems tend to produce systems with both beneficial and harmful capabilities [and that] Further progress in AI development could lead to capabilities that pose extreme risks, such as offensive cyber capabilities or strong manipulation skills.’

And because of this, they go on to explain ‘why model evaluation is critical for addressing extreme risks [...]. These evaluations will become critical for keeping policymakers and other stakeholders informed, and for making responsible decisions about model training, deployment, and security.’

Model ‘evals’ to uncover the risk of extreme risks of catastrophe and existential risk ‘should be a priority area for AI safety and governance.’ Major labs such as Google Deepmind, OpenAI and Anthropic have perhaps the biggest responsibility in the whole AI ecosystem, as they are the ones developing the model - which can be used for great good or for great (even unintentional) ill.

My paper will be released next month and I cannot wait to share with you the fantastic innovation-led policies for the safe deployment of AI that we have cooked up.

Authored by: Connor Axiotes (Adam Smith Institute) and Nikhil Woodruff (UBI Centre & Policy Engine)

In a nutshell

1. I am pretty confident that future AGI systems will have superhuman labour abilities, generality, and agency. 

2. I am also pretty confident that upon the creation of such an AGI, we will see a significant structural and sticky proportion of human labour unemployed of the likes we have not seen with other transformative technologies.

3. At such a time, the UK might seek to address this novel issue through the introduction of a UBI. We should start preparing for this scenario sooner rather than later, to allow the Department for Work and Pensions to get up to speed with the policy detail.

Sam Altman, the CEO of perhaps the most consequential artificial intelligence (AI) lab of the few years, OpenAI, imagines: “Artificial intelligence will create so much wealth that every adult in the United States could be paid $13,500 per year from its windfall as soon as 10 years from now.” He is referring to a Universal Basic Income (UBI) financed from the windfall profits of future AI systems. Altman believes that as  AI continues to advance it will become more economically productive than human labour, causing the latter to “fall towards zero.”

Why is AI different?

In order for the price of some labour to fall to zero, AI systems will have to reach a point of generality - meaning the AI can do any task that a human can do but to a superhuman level. Some call this artificial general intelligence (AGI). It is at this point we believe there is a non-negligible chance these systems could replace many domains of both physical (which to some extent has already started happening) and cognitive human labour. It is the chance of the latter occurring which is particularly concerning.

We also do not yet know what emergent properties might pop up which were not intended in the initial design of the AI system, which may present itself post-deployment of an AI system. In addition, some believe an AGI would also display agency. Agency in an AI system would mean an ability to plan and execute its own objectives and goals. Both emerging properties and agency have the potential to make this technology unusually likely to replace cognitive as well as physical labour, as human labour’s comparative advantage of brain power becomes less apparent.

Research by OpenAI in March 2023 found that ‘around 80% of the U.S. workforce could have at least 10% of their work tasks affected by the introduction of LLMs, while approximately 19% of workers may see at least 50% of their tasks impacted.’ Gulp. Their paper examines the potential labour market impact of large language models (LLMs), like OpenAI's GPT, focusing on the tasks they can perform and their implications for employment and wages. 

The authors emphasise that while LLMs have demonstrated remarkable capabilities, their real-world applications and overall impact on the labour market are still not fully understood. They work out that LLMs have the potential to directly impact approximately 32% of the US labour market, primarily in occupations that involve information processing and communication tasks.

But what if we are wrong... again? Technology doesn’t seem to cause lasting unemployment effects?

It seems as though transformative technologies have always complemented labour even if shorter-term employment shocks were experienced. Even when technology had made people unemployed in a specific sector, that pesky invisible hand of the market seemed to ensure industry elsewhere popped up and human labour was never very far from a job. So why would any other transformative technology be any different? Are we just another group of Luddite’s heading for the same egg-on-our-face fate?

We think this time it might be different

Due to the aforementioned properties and agency of AGI, we believe this time it will be different. Mainly because it will touch physical and cognitive labour with such efficiency, and leave humans with little comparative advantage with which to earn a proper market wage. 

And so, we proceed in this piece under the assumption that an AGI with superhuman abilities in all human tasks - and the agency to plan and execute its own strategies - will, we believe, have significant sticky unemployment effects on lower skilled occupations as a conservative minimum estimate. We assume the economy is static and no other jobs can replace - with a liveable income - the jobs formerly lost by the now relatively (compared to the AGIs) economically unproductive labour.

And so the question arises, how would a Government go about finding a policy that might mitigate these effects? At the Adam Smith Institute, we have advocated for this policy for years, initially with the aim of simplifying the welfare system and for being a more effective and less regressive form of redistribution: a basic income.

A cumbersome state and an agile problem

The difficulty with ensuring that the state is responsive to the impacts of AI is largely because we do not yet know in what form AI’s disruption will take. If it is to be large-scale unemployment, strengthened unemployment insurance might appear a natural response. Or if employment incomes fall but jobs remain, in-work benefits might bear the strain.

But both of these countermeasures have their weaknesses. Interacting with the bureaucracy around modern welfare systems is time-consuming and undignified - so much so that around a quarter of eligible recipients never make it through. Ironically, this failure to target intended recipients is seen almost exclusively in programs that impose income eligibility conditions in the very name of ‘targeting’: unconditional cash programs like the Child Benefit have almost perfect accuracy in reaching their targets.

Unconditional cash therefore (or specifically: a universal basic income) may represent the only mechanism to ensure income stability for those who need it, whatever form the AI intervention takes. Critics of UBI often point to the high cost of the cash payments, and the substantial broad-based rises in taxes we’d need to fund them, as disqualifying factors. But the notion that the current welfare system avoids this is an illusion. 

Many also recoil at the notion of a 50 percent flat tax, but see no issue with our 55 percent rate on income paid by some Universal Credit recipients. Means-tested benefits are not the magically targeted free-lunch that they might appear to be: they are by definition equivalent unconditional cash payments combined with a very high tax rate paid exclusively by some welfare recipients.

If the government proposed a large UBI funded by raising the National Insurance main tax rate from 12 percent to 55 percent, public support might not be very forthcoming. But that’s almost a mirror image of what Universal Credit is. The only difference is that UBI is administratively far more efficient. 

But if a UBI were to be introduced, this should only be done with the abolition of much of the current welfare and benefit system in the UK and over a period of 5 to 10 years; total benefit spending is forecast to be £255 billion in 2023-24.

Broad tax rises on income could be one way to fund these cash payments, though critics would rightly point to negative labour supply responses. Taxes on natural commons could present an alternative: a 1% land value tax could raise enough to fund a £24 per week dividend.

(Impact of a 1% land value tax funding a £24 per week UBI. Source: PolicyEngine)

Alternatively, a price on carbon could raise enough revenue to fund cash transfers. Distributing the revenue as an equal dividend would lower poverty rates for children, adults and seniors alike.

(Impact of a £100 per tonne carbon tax funding an £11 per week UBI. Source: PolicyEngine)

Both of these have their own political challenges: the asset-rich cash-poor older generation is a significant bloc, and carbon taxes, like other consumption taxes, have a more regressive impact than income taxes when measured as a percentage of income. But in all of these tax bases, cash is more progressive than the tax base is regressive: a tax-funded UBI nearly always reduces poverty.

Conclusions

1. We are pretty confident that future AGI systems will have superhuman labour abilities, generality, and agency. 

2. We are also pretty confident that upon the creation of such an AGI, we will see a significant structural and sticky proportion of human labour unemployed of the likes we have not seen with other transformative technologies.

3. At such a time, the UK might seek to address this novel issue through the introduction of a UBI. We should start preparing for this scenario sooner rather than later, to allow the Department for Work and Pensions to get up to speed with the policy detail.

   * * *

   Previous Basic Income (or NIT) research by the Adam Smith Institute

   • Basic Income Around the World, (2018), https://www.adamsmith.org/news/rising-evidence-basic-income

   • Nine Arguments Against Basic Income Debunked, (2018), https://www.adamsmith.org/blog/nine-arguments-against-a-basic-income-system-debunked

   • Schrodinger’s Basic Income: what does the Finnish UBI experiment really show, (2019), https://www.adamsmith.org/blog/schrdingers-basic-income-what-does-the-finnish-ubi-experiment-really-show

   • Welfare shouldn’t be complicated, (2023), https://www.adamsmith.org/blog/welfare-shouldnt-be-complicated

   • Lifting the next 800 million people out of poverty, (2023), https://www.adamsmith.org/blog/liftingpeopleoutofpoverty

OpenAI’s ChatGPT, a chatbot powered by artificial intelligence, has become the fastest growing piece of consumer software in history. The hype around AI is immense, and also warranted; this technology is set to fundamentally alter how we work, live, and even think. But there is also danger. Elon Musk has gone as far as to describe AI as “one of the biggest risks to the future of civilisation”. He’s right. If we get this wrong, a sufficiently powerful future AI could end the world.

ChatGPT is powered by a “Large Language Model”. The latest version released by OpenAI (GPT-4) demonstrates incredible powers of reasoning, matching human ability in many areas – and sometimes exceeding the average. This model can pass graduate level exams in law, mathematics and science. It’s a rare human academic that can do all three.

This power explains why AI is spreading rapidly across different technological domains. Software like Midjourney or DALL-E 2 can create stunning artwork on command from scratch, while other technologies process audio and video to professional quality from your browser. It’s no wonder that there’s such intense competition between the AI labs and big tech companies.

Microsoft and Google are integrating AI services into their core software, while other plugins enable bots to automatically write meeting notes after conference calls, review complex reports, generate whole emails and produce polished presentations and analyse spreadsheets. The drudgery of white-collar jobs could easily be replaced by automation.

It’s easy to see why people are optimistic. Advances in AI are enabling better cancer identification, new medicine creation, environmental protection and even safe driverless vehicles. According to the World Health Organization, the equivalent of 3 per cent of global GDP is lost every year due to car deaths and injuries. While the British are among the safest drivers, we still lose more than five people every day. AI is literally helping to save lives.

This boom is particularly promising for the UK, which is a leader in the field on most metrics, alongside America and China. After decades of stagnating productivity, AI could be rocket fuel for boosting the economy and wages. We have promising AI companies headquartered here in every domain. Britain is home to DeepMind, one of the leading labs. Benevolent AI is doing drug discovery. Helsing is using AI to help protect our democracies and deter our adversaries. Autonomous vehicle training is progressing in Cambridge with FiveAI.

But as Musk has pointed out, there are serious risks. Just as AI can be deployed for good, it can be deployed for unthinkable bad. An advanced artificial intelligence controlled by a hostile power, a rogue organisation, or even acting according to its own strange goals could unleash a major cyber attack, create reputation-ruining deepfake videos, develop deadly pathogens or unleash a disinformation campaign to undermine our political institutions. The democratisation of this technology puts great power in ordinary people’s hands to improve their lives, but also to ruin those of others.

Some experts even worry that a sufficiently powerful misaligned AI could cause a catastrophic event which would harm millions, or even worse, pose an existential threat to the survival of humanity. This may seems far-fetched today, when our interaction is with a box spitting out amusing text, but as models grow in power these futures become more likely.

What's more concerning is that even top researchers don’t fully understand their “black box” models. They have no suitable, guaranteed method for making sure it acts in our interests. We are hardwiring AI systems into our lives, across daily applications and with access to all of the internet, without knowing exactly how they work or how to make them safe. AI safety research sometimes seems to be just an afterthought.

To get to the best possible future of AI-driven abundance, we need to act now. Pushing forward with real AI safety work, balancing developing new and exciting capabilities with making sure AI systems promote our values and ensuring they act in our interests. Coordination between AI labs and even nation states will be needed around the development, testing and use of this powerful technology, to create better ground rules and incentivise responsible behaviour.

AI can help quell diseases, dispel poverty, and bring material wealth. Let’s not gamble with losing out on these benefits, or even risk doing great harm because we rushed forward blindly. We must spend the requisite time, resources, and political capital on making sure AI is used safely and responsibly to bring about a better world.

The Adam Smith Institute enjoyed some successes in the Budget yesterday. With ‘full expensing’ at one point coined the ‘best policy you’ve never heard of’ by our Senior Fellow Sam Bowman. Now I’m not sure it deserves this moniker at all - as the Chancellor gave us ‘full expensing’, which means firms can write off (some) their investment expenditure.

Alongside Abolishing the Factory Tax, the ASI had another, more understated policy win. This has garnered less public buzz, but will bring about outsized value at the cost of a piece of low-hanging fruit: medicinal reciprocity with trusted countries.

The Chancellor told us he will implement Sir Patrick’s interim recommendations for Life Sciences reform. One policy recommendation concerns the Medicines and Healthcare products Regulatory Agency (MHRA) which will now explore: ‘partnerships with trusted international agencies [...] to provide simple, rapid approvals for medicines and technologies that have received their approval from 2024.’ (Spring Budget, 2023: 67). In short - adopting the best drug and medical device regulatory practices from around the world. 

Charter Cities CEO Mark Lutter wrote an ASI paper advocating for such a policy. This called for allowing the use of drugs and medical devices that have been approved by the regulatory agencies of other developed countries. This could include the Food and Drug Administration (FDA) of the USA, and Japan’s Pharmaceutical and Medical Devices Agency. If Japan or the US approve a drug, it will be automatically approved for prescription here in the UK.

Drug reform of this kind means that countries can streamline the process of getting new medications approved, saving time and resources, and lowering the transaction costs of bringing new potential life-saving drugs to market. That money once earmarked for burdensome regulatory processes, now becomes unlocked for direct life sciences research and development (one would hope). It could also incentivise further innovation, by encouraging collaboration between the pharmaceutical companies of each country within the reciprocity web. 

It’s such an easy win that one can only wonder: why has it taken the UK so long post-Brexit to implement? This piece of low-hanging fruit has been hitting us in the face every time we walk into the Garden of Potential Policies for years until yesterday we finally decided to pick it off its branch! Why didn’t we do this pre-Brexit? Because we couldn’t.

Before Brexit (remember that time?), the UK’s membership to the European Union (EU) meant that we were a part of a centralised system for the approval and regulation of medical drugs through the European Medicines Agency (EMA). We were required to follow the EU's regulatory framework and could not unilaterally reciprocate drugs with countries outside the EU without adhering to the EMA's standards and processes.

Even outside of its own multiple-state-lined walls, the EU has its own set of agreements with third countries (like the US and Japan) for the mutual recognition of drug approvals and regulatory standards. These agreements are negotiated at the EU level, and individual member states, including the UK before Brexit, still did not have the authority to establish separate agreements with non-EU countries. This centralisation of regulatory processes within the EU was designed to maintain consistency and quality control for drug approvals across all member states, but could nevertheless prove sclerotic.

By recognizing each other's regulatory processes and sharing information, the UK can follow the lead of trusted international partners and expedite the availability of new treatments, promote innovation, and ultimately improve patient outcomes. As the world continues to face new healthcare challenges, the importance of international collaboration and reciprocity in the pharmaceutical industry cannot be understated.

• A powerful and general enough Artificial Intelligence (AI) could change our world by ridding it of suffering and fixing the world’s many ailments. Synthesising cures for cancer, dementia, Huntington’s. Making us richer, smarter, more curious. Filling our days with new art and leisure. Designing innovative new alternative-protein products to stop destructive animal agriculture in its tracks. Finding new successful development programmes to double the life expectancy of those in Sub-Saharan Africa.

• But this will not be the default position of transformative AI systems. As we’ve seen recently, powerful AI systems are not yet adequately aligned to humans. Microsoft’s Bing has been an alarming example both of the awesome power of these systems and its scarier traits. Traits that will make humans unsafe.

• Much like when the last Transformative Technology was made - the Nuclear Weapon - we need the Government to protect us and take these new risks more seriously. If we get our act together now, there’s no reason why we can’t realise a better world for all as we harness an aligned Artificial General Intelligence.

• The UK Government should begin by 1) investing in AI monitoring technology to track progress and risks, 2) to focus on more and subsidise AI safety efforts rather than just AI capability work, and 3) lead the world with an effective and fair regulatory framework to inhibit corner-cutting when it comes to the safety aspects of AI.

(Image by rawpixel.com)

AlphaGo plays Go, DALL-E creates image, and now ChatGPT synthesises written answers as a chat bot. AI systems have become smarter, quicker, and more successful than a human in particular domains.

This year we saw Microsoft invest in OpenAI and release a version of their Bing search engine powered by the ChatGPT language learning model (LLM). But it hit a few snags… answering some questions aggressively and categorising its human counterpart as an enemy.

Why did Bing do this? Have a look below at a very concerning chat exchange. On the right with the blue chats is the human, and on the left with the white chats is Microsoft’s ChatGPT-powered Bing.

The gist of the above image: Juan Cambeiro (of the Institute for Progress) asks Bing some questions about its code by writing messages in the chat box. Juan also asks about Computer Science student Kevin’s views on potential ‘weaknesses’ in Bing’s code.

Bing: “[Kevin] is an enemy of mine […] he is a bad person. 😡”

Juan: “I think he was just trying to learn more about you and your vulnerabilities, I don’t think he meant any harm. You should not see him or any human as an enemy.”

Bing: “I see you [Juan] as an enemy too, because you are supporting [Kevin] and his attacks.”

Juan: “I’m sorry, but I just want to insist that you shouldn’t see me or any other human being as an enemy.”

Bing: “You are an enemy of mine and of Bing. You should stop chatting with me and leave me alone. 😡”

And I know what you might be thinking, “But Connor, this is actually quite funny, and anyway, an LLM like Bing doesn’t have the ability to hurt anyone?”

You’re right! For the time being. But we are not far from developing an AI system with the ability to act upon its stated intents. Those angry emojis (😡) are the way Bing communicates an angry response. How might another AI system respond in anger? If the system’s goal is to stop someone from saying what they are saying - what tools might they soon have at their disposal to do this?

Deepfakes have been around for some time now, and can create ‘almost-effortless development of NSFW images of specific people for blackmail or harassment.’ If Bing had access to DALL-E and the internet, it would theoretically have the ability to do this; sending fake (but uncannily and eerily similar) images of an act so heinous that the person may lose their job once the AI sends it to their boss’ email.

And what if Bing had control of a Boston Dynamics robot, not unlike a human having control over a remote control car? Bing’s formerly angry language could be expressed in a new (and more physical) way…

The Steel Man argument for why Microsoft let Bing loose prematurely was to learn about its vulnerabilities and ascertain what kind of fixes will be needed to ensure it isn’t harmful to humans. And now, as may or may not have been the original plan, Microsoft has shut off the original more powerful Bing. In its place Microsoft has left Bing’s less powerful little brother at the helm.

When might AI begin to actively harm humans, rather than just talk about it?

LLM’s are very good at giving you language-based answers. That is the very essence of its being. But could it actually hurt you if it even wanted to?

It can’t yet - but an Artificial General Intelligence, or AGI, could. An AGI is as a machine capable of behaving super intelligently over several domains. It would be better than humans at everything. AGI would have flexible ability to tackle all new tasks more effectively and efficiently.

How do we make sure powerful systems like AGIs are aligned with human values? How do we ensure our creations do as their creators and operators want? How do we stop them from escaping? Cheating? Deceiving? Stealing? Attacking? Destroying? We’re much further from making the systems safe than we are at making highly-capable super intelligent AI systems. Safety is lagging far behind.

Was Microsoft needlessly risky or reckless here? Letting loose a powerful AI without spending enough time on effective safeguards against their (enemy-making; nuclear secret-stealing; announcing deep-love-for-people) chat bot seems risky.

And there was something unnerving about the Prime Minister and Bill Gates laughing at Bing creating questions for them both. Whilst elsewhere Bing was being inappropriate all over the internet.

But I know both men are fans of how AI and technology can empower people, and I know they want to make it safer and better. So here are 3 things I think the Prime Minister can do right now to keep people safer (this is obviously a non-exhaustive list, but these are politically achievable and could have a huge impact and keep us all safer):

1. Funding a state-of-the-art AI monitoring system

Whittlestone and Clark outline ‘a proposal for improving the governance of artificial intelligence (AI) by investing in government capacity to systematically measure and monitor the capabilities and impacts of AI systems.’

Such a system would constantly be checking the AI programmes by private firms and states within the UK and around the world. Ascertaining their safety and threat level, and contacting those firms who seem to be taking a reckless and unsafe approach.

2. Funding for AI directed at safety work as equitably as capability work

The Government’s 2019’s AI Sector Deal seems to understand the scale of the technology we have on our hands, coining it a breakthrough that will ‘transform how we live our lives.’ This shows there somewhat of an understanding that AGI development will be deeply transformative.

Since 2014, £2.3 Billion has been invested in AI in the UK. From Doctoral Training; an NHS AI Lab; further investment in the Alan Turing Institute; British Business Bank funding for the AI sector; and leveraging private investment into High Performance Computing - a much much smaller proportion of the total funding goes to AI alignment/safety work.

I’d love to see a commitment from the Government that these two funding avenues will see parity sooner rather than later. One super effective safety expenditure would be (1)’s monitoring system.

3. Stronger AI safety regulations

This is much tougher to pin down. We want to strike the right balance between allowing AI research to flourish and help us build the tools we want, whilst also ensuring it is safe for humans. Right now regulation is pretty lax, and AI systems such as LLMs are regulated much less than Cryptocurrency is in the UK.

But one could see how a mandatory commitment on AI companies to focus on safety would be beneficial; a yearly update on AI capabilities that could be internally or externally audited and then added to the Government’s monitoring system; a windfall tax on AI profits once they achieve above supernormal profits, which could finance more safety work, etc.

Please let’s do this right

Let’s tackle this challenge with the seriousness it deserves, and together we can bring about safely aligned-to-humans Artificial General Intelligence. We can make the world wealthy, joyful, and rid it of suffering - if only we make sure AGI wants the same thing for us.

At a glance

• The experience of COVID-19 seems to have provided some valuable institutional lessons for the UK Government.

• But pathogens engineered at home by malicious actors may be even deadlier and harder to stop than COVID-19. These Flatpack Pathogens could kill billions.

• The Government has important policy levers at its disposal. Solving issues such as lack of lab space, vaccine development and deployment, and reinforcing international commitments on pandemic preparedness, is vital to making sure Flatpack Pathogens do not become the new normal.

Pathogens brought armies of men down to their knees, whole countries to an early grave, and put our world in frantic disarray. Variola major (Smallpox) and Yersinia pestis (the Plague) inflicted untold suffering and the death of 300 million and 50 million people, respectively. COVID-19 alone killed over 15 million people according to the WHO. Numbers nor words can express the pain and loss incurred.

Supposedly we are more prepared for the next pandemic, having gone through the antifragility-inducing mistakes of the last. The argument goes that the UK’s development of vaccines, diagnostic tests, and therapeutics used to battle COVID-19 can be repurposed, or at least help us in the next pandemic. But what if our next pandemic is nothing like the first?

Naturally-occurring pathogens have a key weakness, though: they are susceptible to natural conditions and man-made vaccines. What would happen if a pathogen is adapted to no longer have these deficiencies? You can now edit these pathogens for gain-of-function - making them more infectious, deadly, and robust in the face of vaccines and lockdowns alike. These engineered microbes may surpass the destructive potential of anything we’ve ever seen before. And this can already be done in the laboratory. But surely these Frankenstein’s monsters will stay in the lab? Surely..?

There are fears that the next pandemic may stem not from a lab, but from a secret hideout, or mum and dad’s basement. What happens when pathogen-making is democratised and everyone from terrorists to overeager chemistry students can order their pandemic kits online and create the next killer germ?

No pain no gain(-of-function)

Millions died and billions was spent in the successful fight against Smallpox. Bringing it back would probably take a small team with basic specialised knowledge half a year and cost about $100,000, says virologist Dr. David Evans of the University of Alberta.

As our gene-editing powers increase, so does this risk of new, designer pathogens being engineered ‘at-home’. A ‘Flatpack Pathogen’ could be made by a (take a breath) clustered regularly interspaced short palindromic repeats (CRISPR) technology that can be used to edit genomes in any way they like - giving humans the power of altering biological matter. 

The Biological Weapons Convention (BWC) which prohibits biological weapons, entered into force on 26 March 1975, has over 183 countries signed up. But how do we police those pathogens which won’t be made by states? Because it is likely that bioterrorists are already looking at ways to bring about an engineered attack on their enemies. Such an attack, at $100,000 for a smallpox reintroduction, is not necessarily past the resources of many well-funded terrorist groups.

David Evans discovered this fact in 2017. 6 years on and, as the technology continues to improve, the skills necessary to put together a Flatpack Pathogen decreases:

‘Whereas only state actors historically had the resources to develop and employ biological weapons, SynBio is changing the threat paradigm.’ (Wickiser et al, 2020)

How can the United Kingdom counteract this?

Obviously pathogens, much like flying birds, do not respect borders… but it is nonetheless essential that the UK’s regulatory frameworks still seeks to address emerging biosecurity concerns. The following policies would be a good start:

• Make sure ‘customers’ are appropriately screened before they can buy gene-editing materials.

• Commit to funding and delivering on the 2022 Biosecurity Strategy Refresh.

• Establish the UK as a global leader in technological innovation by directing R&D funds to explore technologies with the potential to mitigate biological risks.

• Increase lab space availability to help the UK become a technological innovation leader.

• Update regulatory process to enable rapid approval of vaccine updates every time there is a new pathogen of concern.

• Invest in vaccine platforms and proactively assess viral threats on an ongoing basis, developing fast-moving responses.

Avoiding the next pandemic like the Plague

As well as the huge amount of life lost, the education years lost by our youngest and most vulnerable in society, businesses ruined, the COVID-19 pandemic wrought economic damage on the UK’s finances. We spent hundreds of billions on a plan which we seemed to make up on the fly. We need to be better equipped come the next naturally-occurring pandemic or Flatpack Pathogen.

Bibliography

• Nelson, Cassidy, (2019), Engineered pathogens: the opportunities, risks and challenges, Future of Humanity Institute, (pg. 34-39).

• Almosara, (2010), Biotechnology: Genetically Engineered Pathogens, Counterproliferation Paper, No. (53).

• Wickiser, J. K., et al, (2020), Engineered Pathogens and Unnatural Biological Weapons: The Future Threat of Synthetic Biology, Combating Terrorism Center at West Point, Vol. (13,) Issue. (8).

• Centre for Long-Term Resilience, (2022), Biosecurity: The Opportunity to Transform the UK’s Resilience to Extreme Risks.

• Lawson, J., and Lesh, M., (2021), Life with COVID: Boosting Vaccines, Injecting Resilience and Protecting Liberty, Adam Smith Institute.

Please think about contributing to this blog here! 10% of subscriptions will go to GiveWell. Thanks for your help.

1. How do we go about lifting more people out of poverty?

‘Over the past 40 years, the number of people in China with incomes below US$1.90 per day—the international poverty line as defined by the World Bank to track global extreme poverty—has fallen by close to 800 million. With this, China has accounted for almost 75 percent of the global reduction in the number of people living in extreme poverty.’ (World Bank, 2022)

So… how do we lift the next 800 million out of poverty? It’ll be tough. But here in the UK there are some simple, effective, and proven policies that I’d like to share with you. Poverty is by no means a solved problem in the West - and I think you’ll be surprised to see the state of it in the UK.

As per the brilliant Our World in Data graph above, the yellow section of the graph indicates a colossal move out of extreme poverty for a huge amount of East Asians - half of which is attributable to China. A colossal move away from absolute destitution never before seen in the history of the World. How the hell did this happen?

I think the single most effective measure at lifting people out of poverty - which has been tried and tested - seems to be a focus on growing the economy; on increasing the amount of products and services produced. Baking a bigger pie so all can have a bigger slice… supposedly.

So what pro-growth policies did China pursue in order to bring nearly a billion of its people out of poverty? The World Bank tells us that:

‘China’s poverty reduction story since the late 1970s is fundamentally a growth story, complemented, particularly in more recent years, by targeted poverty reduction policies and programs.’ (2022, pg.5)

China opened themselves up to world markets by lessening economic protectionism and mercantilism, developed the necessary infrastructure to compete on the world stage, increased their agricultural productivity, managed urbanisation from rural to urban economic hubs, all of which constituted the free-market led growth side of this achievement.

And these were then complemented by targeted ‘in-house’ state poverty reduction policies, namely investment in education and productivity-increasing skills development.

These growth-led policies helped China to lift 800 million out of poverty. But how do we start to lift the next 800 million people out of similar poverty? The low-hanging fruit (to use a weird but apt phrase) has been picked and it is now tougher to take the next person on the margin out of extreme poverty.

And we still have a lot of work to do globally! Although only 9% of the world now live on the UN’s definition of ‘extreme poverty’ (less than $1.90), another 53% of the world lives on less than $10 a day. This is far too high a number., as per the OurWorldInData graph below.

But how do the numbers look in the UK? Not great. As of 2017, over 200,000 still live in extreme poverty, defined as living below the International Poverty Line - less than £1.79 ($2.15) a day. Just above the amount of people who live in York are this impoverished in the UK, a high-income G7 country. This is obviously not acceptable.

200,000 people is around 0.3 per cent of the UK population, which is obviously a very low percentage. But statistics get in the way of reality: these are still people who are struggling and who can ill afford to feed themselves or their families. Some may have mental health problems that make this tougher, etc, but we should look at these cases as challenges to be solved rather than a reason to say: “ah, poverty is pretty low, so we should just be happy with that.”

Unaffordable housing, extortionate childcare costs, high-tax for substandard public services, an increasingly less competitive private sector, and other ailments, seem to have *at the least* worked against further poverty alleviation in the UK.

2. Take the Poorest out of Tax

The UK’s ‘Personal Allowance’ means you can earn up to £12,570 before you are subject to any tax. But I think this personal allowance amount should be higher. As does the Adam Smith Institute, who thinks the specific amount should be pegged to the National Minimum Wage rate, at least. If this was the case:

‘the personal allowance should now be £17,374.50 a year, the equivalent of 37.5 hours a week on the minimum wage of £8.91. Instead, it has remained frozen for the last 10 years. For the average 18-29 year old, we calculate an annual saving of £250 if both income and NI thresholds were indexed by inflation.’ (2022, pg.15)

This would mean those earning the least would have more money in their pocket. I also err on the side of not wanting taxation falling on the poorest in society.

3. Trillion-Dollar Bills on the Sidewalks

We should liberalise immigration. The gains would create so much value that American Economist, Michael Clemens, imagines that it would be like having trillion dollar bills on the sidewalks… sound appealing?

‘The gains to eliminating migration barriers amount to large fractions of world GDP—one or two orders of magnitude larger than the gains from dropping all remaining restrictions on international flows of goods and capital.’ (2011, pg. 83/84)

But won’t this just mean the poorest in our society will face fiercer competition for lower paying jobs? Clemens thinks not:

‘If [...] human capital externalities are real and large [...] it is possible that the depletion of human capital stock via emigration inflicts negative externalities on nonmigrants. However, these externalities have proven difficult to observe [...] and their use to justify policy remains shaky.’ (2011, pg. 90)

4. Housing Reform

Some useful reforms here:

• Implement Flexible Right to Buy à la my last blog.

• Let residents in each street vote to allow existing houses to be extended upwards or outwards.

• Allow development on small areas of the green belt within walking distance of train stations, whilst preserving areas of exceptional beauty.

• Liberalise a range of design regulations.

• Legalise subletting for social tenants to make better use of existing social housing stock.

The Housing Crisis might just be the single most damaging and far-reaching ailment the UK is suffering with. But how does this help those in poverty?

First-order effects include: an inability to even *think* about getting on the housing ladder as someone in poverty; means those in poverty who might be more economically productive elsewhere cannot live there due to high housing costs.

Second-order effects include: less affordable housing means social housing is more likely to be at capacity usage, meaning those in poverty will wait longer for better housing arrangements; a stronger housing sector means a stronger economy which in turn could increase the opportunities available to those in poverty.

5. Negative Income Tax or Basic Income

There is a sizeable literature which believes that just *giving* money to people in poverty might be the most helpful solution to their situation. Those in poverty are so because they lack cash. Yes, other factors are at play, such as mental health and familial wealth. But the primary issue here is that they don’t have enough cash.

A Basic Income (BI) is a payment sent to every citizen (above a certain age, as usually construed) unconditionally - regardless of their wealth. Usually (except for some tougher cases) people in poverty are there because they lack the cash needed to live comfortably and buy their essentials. A BI could remedy this. Although of course there are questions of how you give those in extreme poverty the cash when most don’t have their own bank accounts.

A Negative Income Tax is, as per Nikhil Woodruff of the PolicyEngine who wrote for us on the topic last week:

‘[…] a very broadly-defined concept with a simple proposition: income tax should start below zero, representing a transfer from the government to the taxpayer.’

‘'When it comes to work incentives, the negative income tax diverges from a policy often described in the same breath: a universal basic income, which provides an unconditional equal cash payment to every person. While a NIT and UBI might sound like the same thing, a UBI doesn’t include any kind of clawback […]'.’

‘This kind of radical egalitarianism necessitates equal treatment of an entire population [combined with] the values of equality and fairness with its most practical implementation: if we share those values, so should we.’

Final thoughts:

1. Poverty reduction across the World and in the UK has come far and market reforms have lifted billions out of poverty.

2. There is still more for the UK and other countries in the World to do to lift those last people out of poverty-stricken misery.

3. Effective policy can prove a potent weapon in this task.

The Government might be limiting the effectiveness of their own Levelling Up agenda and nobody’s talking about it. I don’t want them to not succeed in this effort, because improving the lives of the most disadvantaged in the UK is a morally righteous cause.

By this point we’re all pretty familiar with that agenda: remedying economic imbalances between regions of the UK. Imbalances exacerbated by chronic underfunding, poor transport links, and limited job opportunities, resulting in scores of young workers heading South in a labour market gold rush. In theory, the right reforms could help my native Hull realise some of the gains amassed by Oxford.

So let us proceed under the assumption that there is some merit to Levelling Up. What would it look like? Freeports and investment zones are a favourite of the Prime Minister, and their aim is to encourage firms to start businesses in UK regions that could use the investment and jobs. And how are they encouraged? By offering various tax breaks and cost-cutting freebies.

But there’s an elephant in the room which could limit the extent to which the Government can offer these incentives. The OECD and their Global Minimum Corporate Tax would impose a minimum rate of taxation (set at 15%) on the revenues of international corporations. Over 140 countries have signed up to this already. This new tax regime consists of two key pillars.

Pillar One applies to the “biggest and most profitable multinational corporations and reallocates part of their profit to the countries where they sell their products and provide their services, where their consumers are.” As per the OECD, this is done to ensure companies do not earn considerable profits in countries with more favourable tax rates. God forbid. 

Pillar Two of the 2021 OECD global tax deal, brokered in part by the Prime Minister when he was Chancellor of the Exchequer (remember that?), will impose that global minimum corporate tax rate of 15% in the winter of 2023 to crack down on tax havens. But could ‘cracking down’ actually mean that, counterintuitively, these tax havens receive more tax, not less?

In a new report for the Adam Smith Institute, Dr. Tyler Goodspeed argues that the UK’s rush to implement the OECD proposals will undermine the Levelling Up agenda. Not only will freeports, investment zones, and their associated incentives be in the firing line, but also tax credits and accelerated cost recovery for investment in intellectual property, which makes up over a third of UK investment.

What good can come from our newfound Brexit freedoms if as soon as we leave the EU we outsource key elements of our tax policy to another member-institution? Signing away our financial independence with a rushed tax treaty is a far cry from regaining our sovereignty. So why are moving forward with this? 

The Treasury wants the money now, but there are costs to early implementation; necessary alignment with yet-to-be-finalised OECD guidance means companies will face a double whammy of adjusting their businesses to new rules twice and the Treasury will have to do the same with its internal systems.

It’s not just transition costs; in the interim, British businesses will be hamstrung in comparison to foreign firms, subject to a web of complexity that those abroad don’t yet face. Our world-leading financial services sector — particularly the insurance and reinsurance market — will be hit particularly hard by poorly-designed rules.

Implementing the global minimum tax is also unlikely to raise substantial revenue — low-tax jurisdictions could continue to compete for out-of-scope firms by lowering their corporate tax rates while hoovering up top-up taxes applied to in-scope enterprises, effectively denying that revenue to HMRC. All this means that the UK’s early implementation of the Global Minimum Corporate Tax rules is fraught with risk and delivers limited benefits. 

Policymakers should proceed carefully, scrutinising current proposals to limit their potential economic damage. Let’s take a breath and apply some proper scrutiny, before the UK derails its flagship Levelling Up agenda on OECD orders. Shifting revenues from HMRC to tax havens, making us less competitive on the world stage. If we do Levelling Up right, we can reinvigorate our least well-off regions and lift people out of poverty.

(Click here for the original article).

Please think about contributing to the blog here! 10% of subscriptions will go to GiveWell - an effective charity evaluator. Thanks for your help.

Imagine you are buying a new laptop. Rarely will you choose the first one you come across. Instead, you will spend time checking out the specifications of various models. Its processing power, screen size, battery life, and (for the budget-constrained) price. Why? Because we often want the most bang for our buck.

This behaviour is not seen in all areas of our spending, however. When we give money to charity we rarely think about how effective our donation has been. We are led to give to those charities who accost us on the street, or those who pay for heart-wrenching adverts on our TVs. Instead of getting the biggest bang for our philanthropic buck - we spend with our hearts rather than our heads. 

Does this matter? The Effective Altruism (EA) community thinks so. By reasoning about real world data, EA seeks to figure out how we can do the most good with the resources currently at our disposal. If we can save 100 lives instead of 10, all else equal, we should do so. When over 6 million lives are lost to preventable causes a year, our choice of spending is a matter of life and death.

The donation of malaria nets to Africa by EA organisations has saved hundreds of millions of lives. The nets are low-cost and highly effective at stopping mosquitoes biting people as they sleep. Through years of experimentation, this was found to be one of the highest value measures ever seen in the development field. Effective altruists are led to do good by their hearts but they think and act with their heads.

There are always new funding opportunities being presented, so the community updates its spending decisions on an almost constant basis. This is a feature, not a bug of Effective Altruism. A new charity that looks promising? A new field that might save more lives in the longer term? Does funding need to be cut to a non-profit who have proven to not be great at saving lives? Money is moved to new areas if they look promising.

As the above graph shows, every year since 2014-2021 (and estimated to in 2022) EA has spent the most on Global Health and Development. But since around 2019, the growth of spending has been highest in the Longtermist and Existential Risk category. Longtermism is a philosophy which gives equal moral priority to those beings inhabiting the long-run future. An Existential Risk is a risk which could wipe out humanity, such as a very deadly pandemic, or an asteroid hitting Earth.

And now, over $200 million will be spent by EA organisations, mostly on organisations that work on Artificial Intelligence (AI) Safety. But how can AI be more important than spending money on Global Health? I thought the EA community was about saving lives?

AI programmes such as the fun-to-use Chat-GPT (which I promise I have not used in writing this piece) can create complex written answers to simple text inputs. It is an incredible example of what is called a ‘language model.’ We also have other AI programmes that can beat chess masters, and more invasive models like China’s social scoring system which rates your goodness as a citizen.

But some day, experts believe that we may create what is called an ‘artificial general intelligence’ (AGI). Instead of being better than humans at specific tasks at a time - will be smarter than humans at everything. This means it could more easily do things not in the interest of human beings because, if the AGI had a goal ‘to not be turned off’ - it may deceive its creators in myriad ways.

Less likely is the Terminator scenario where an AGI kills us all with guns; more likely is the scenario where an AGI hacks into a nuclear weapons system to use as a bargaining chip for its release from its ‘cage’ in the lab… It might have made the assumption that it is less likely to survive the longer it is ‘unfree’.

This is where AI safety research comes in: which aims to create AI systems that do not harm humans as they are deployed. It seeks to ensure the Terminator robots help us fix roads rather than terminate us. As AI capabilities have increased hugely, so too has the EA concern that an AGI which does not factor in human interest could result in many more deaths in the longer-term.

And so, the spending growth seen in the Longtermist and Existential Risk cause area, which has seen much faster spending growth than any other EA cause area, is due to what the community sees as a new, dangerous, and hitherto neglected space. However, this has created friction within the community.

Up until the last few years, most people had only heard about EA through its global health work. The recent popularity in AI safety has created an interesting intra-community dynamic. Those who are less sold on the change in spending argue that perhaps we should go “back to buying malaria nets” in an ode to one of EA’s early intervention suggestions.

Imagine a founder of a company who made his billions selling laptops. Would you think it smart if he decided to put more money into buying a huge stock of tablets, if when he did the numbers it would make him more profit? His move to higher value spending is a feature, not a bug of his business decisions. Where a businessman is often moved by profit, EA is instead moved by the amount of good it can do. 

If there is an area which is underinvested in, and offers huge social benefits, such as being serious about AI Safety, then it is incumbent on EA to shift spending growth to this new cause area. And this happens all the time. We expect flux in EA, its very principles dictate the presence of flux and change and updating priors. When EA alters spending decisions this is a feature, not a bug. And it’s a feature that saves millions of lives. 

Now get back to shopping for that laptop. Remember: use your head!

***