Like it or not – you’re in good company either way – a wave of AI RCTs is on the horizon.

Recently, through their Economic Futures Research Fund, Anthropic announced they are committing $200 million to funding large-scale RCTs for ambitious, creative pilots or programme evaluations1. This inflow of funding, including funding calls from J-PAL, with more potentially on the horizon, will compound existing trends in economic research.

Leaving aside debates of whether this is the optimal allocation of resources (for some of our takes, see there is no randomising a technological revolution, or adaptation, not adoption, is king), if we accept the coming wave of trials as inevitable, how can we maximise the durability, generalisability, and policy relevance of these studies? We have a number of concrete proposals, but first we outline the current AI RCT trends in economics.

More RCTs + AI = AI RCT boom

Trial registries are a window into the future. By looking at the descriptions of planned and on-going studies, we can learn what the profession is investing its time and money in. Doing so reveals that AI looms increasingly large.

The number of trials registered in the American Economic Association’s RCT Registry (the economics profession’s largest registry of randomised controlled trials) and the Registry for International Development Impact Evaluations (RIDIE, focused on development economics) has been increasing steadily2. From a little under 1,000 pre-registered studies in 2019, the number had risen to around 1,650 in 2024, and a similar number in 2025. By the half-year mark, around 900 had been registered in 2026, suggesting that the upwards trend continues. This could simply reflect more diligent registration – because more journals now require it for publication, say – but likely also indicates a growing number of trials each year. Figure 1 shows that interventions that use AI in one form or another are on track to make up one in five trials – up from one in thirty in less than four years3.

Figure 1

In LMICs, AI studies are less common but rising just as steeply

Randomised trials are used (or registered) somewhat more in high-income countries – despite those only accounting for 17% of the world population.

Figure 2 shows the share of AI studies in the total number of trials, split by country income group, over time. The pattern and trend are clear: the share of AI studies in LMICs is lower, but the trend line is as steep.

Figure 2

Notes: Based on the 66.4% of AI-related records that show the study country.

Subscribe now

Where is AI being randomised?

Whether AI has the greatest value in the most capacity-constrained settings or whether it’s most useful in places with complementary infrastructure remains unclear. The cost-benefit analyses that emerge from these empirical studies should shed light on where productive applications lie – for now, though, we can only infer where researchers see potential by looking at the areas they study.

Figure 3, based on AEA registry categories and assigning each record to its main topic, shows that studies of behaviour, education and labour are the most common subjects overall, and account for over half of AI-related RCTs. We can also see that education, firms and labour research are more common for AI-related projects than for overall registry records, suggesting a broad focus on the productive impacts of AI. When comparing the AI records (red markers) with general development research (black markers), as compiled by Jessica Leight for 2021-2025, the top registry categories of behaviour, education and labour are substantially over-represented in these empirical studies.

Figure 3

Notes: Each registry receives a primary topic based on a review of its tags, title, abstract, intervention and outcomes.

Where to from here? We should be ambitious

In a worst-case scenario, we could see a wave of one-and-done research projects in LMICs, that are out of date before the results are public, don’t generalise, and aren’t taken up by governments or scaled. In this scenario, the wave of AI RCTs would not lead to a wave of useful insights, but a wave of funding for well-networked senior researchers4.

That need not be the case, but ensuring so requires urgently changing with the times. For those questions that can be answered by RCTs, the current model, and the standard research process around it, isn’t set up for success. There are a number of problems to solve, some old, some new. After discussing these, we outline some concrete fundable proposals to maximise our collective learning from the AI RCT wave.

Challenges

Longstanding issues

Scale: Governments = Scale. Research projects without initial government buy in or local collaborators are unlikely to be adopted, or scaled, by governments.

Generalisability: That many randomised interventions fail to repeat their performance when taken to a different setting remains a key challenge – and is one possible reason why policymakers don’t give much weight to impact evaluations.

Duplication: How do we avoid the scattered, duplicative pilots of, for example, the mobile health era? ‘Pilotisis’ is real – the situation got so bad in Uganda that, in 2012, the government put a moratorium on mHealth pilots.

General Equilibrium: Much of the coming wave of trials will not just involve testing AI interventions themselves, but new models of redistribution and other policies which may become more important in an age of transformative AI – to figure out their actual effects, economy wide, we need macro. Much of this work may appear unrelated to AI, but it will be helpful to design these macro experiments with the potential for AI transformation in mind.

Issues particularly exacerbated by AI

Iteration: An RCT, from planning over baseline to mid- and endline can sometimes take two years, or even longer. By the time an intervention has been tested, and the results published, the AI model it was based on may no longer be available at all. What lessons can we learn which outlive the RCT, when the underlying technology is improving so rapidly. Faster trials can help, but we probably need to find new ways of evaluating the type of (increasingly common) intervention that itself evolves during the trial window.

Figure 4

Cost: AI could make it far cheaper and easier to develop digitally-delivered interventions. Unless the cost of impact evaluations falls in tandem, evidence generation becomes a bigger share of budgets. To the extent that MEL funding is commonly capped at some part of the total budget, evaluation cost becomes a bigger hurdle to evidence generation than it already is.

Fidelity: With AI interventions, the technical capacity of the research team (i.e. their ability to adapt, iterate, and improve the AI solution) might be more important than the intervention being tested itself. I.e. The difference in impacts between different AI interventions might be lower than the difference in impacts between teams of varying quality delivering the same AI intervention.

Control group: What does ‘control’ mean in an era of rapidly expanding adoption? What would we have learned from randomising access to the internet in a country where overall adoption was rising rapidly, and the internet itself was rapidly evolving? This complicates our interpretation of results, as it’s possible that many trials won’t capture the impact of AI itself, but of the AI being randomly provided, against the AI being silently adopted in the background. Is the agricultural advisory tool built by a team of economists on top of an existing model really better than the next iteration of a general purpose model that becomes available during the trial window?

Solutions

Some of these issues arise from the technology being studied, some arise from the discipline doing the studying. But given the unique moment we’re in, and the changing funding landscape, we have the opportunity to be ambitious and tackle both. Here are some initial proposals (comments, feedback, or further ideas encouraged).

Leave a comment

New, or much improved, centralised research tracker for economics

Building on the registry of pre-registered trials, we need a registry of pre-paper results, where researchers upload mid-line, and end-line, results, so we don’t have to wait for the paper to see impacts.

We also need a pilot registry system. This would build on current registries of trials that are already underway, by also publicising those trials that never got underway because the pilot didn’t work. Learning lessons from the pilots that never got off the ground will be extremely valuable in directing future work, but is currently not tracked systematically, so we are missing a whole world of lessons learned. Building this database, with prizes for registering verified pilots that never went further, would help here.

This tracker could also include new variables that every AI RCT is likely to track, e.g.: models used, tokens spent, cost for specific language tokeniser, choices for fine-tuning (hyperparameters, training dataset size and shape), performance gain from base model, cost of training run in compute, and licensing and compensation for any individuals who helped in training dataset. These variables would be helpful for comparing between different interventions on dimensions like: performance gains from training, impact achieved, and cost effectiveness.

New, or much improved, selection mechanisms

It’s natural that funding does not always flow to what would have been the best projects from a policy perspective. Relationships, networks, academic incentives to publish, and seniority all play a role – we are human, after all.

This is particularly inefficient from a policy perspective, as local researchers, who are best placed to build relationships with the government, find it hard to break into these networks. It also prioritises academic novelty, which does not always overlap with policy usefulness – for example, academia does not incentivise testing existing interventions in new settings. Funders should deliberately support studies based on their policy relevance, even if typical academic review boards would choose ‘novel’ studies more likely to end up in a top journal.

For AI, this unwritten arrangement is particularly inefficient. Those at the top of the discipline, sat on review boards, or looked on particularly favourably by them, typically have strong development expertise but are less likely to be the tech savvy researchers with enough technical ‘taste’ to understand and implement the most interesting and useful applications of AI.

So we need new and improved selection mechanisms, made up of more inter-disciplinary review boards that contain economists, technologists, and policymakers, to fund projects that are economically sound, technologically useful, and policy relevant. These review boards would benefit from the improved, centralised research tracker we outlined above, in order to easily avoid duplication.

Funders should also incentivise co-creation with governments, and prioritise projects with local researchers. This could involve creating joint research agendas with specific governments, and/or investing in national research organisations. Building local capacity is an added bonus, but these projects will also just be better for the world by making scale more likely.

Unfortunately, we have a long way to go here. Across the AEA Registry, 14.1% of all RCTs involved the government, compared with 10.7% of AI-related efforts. AI-related RCTs were therefore about 3.4 percentage points less likely to involve the government.

Figure 5

An important open question, which is particularly relevant to how funding should be allocated, is how to design RCTs that are model-agnostic. Even if we speed up RCTs so that the results become available while the model being tested is still relevant, the results could become irrelevant in a matter of weeks or months when a new model emerges (on either the capability or cost frontier). We need a set of best practices for designing RCTs that can keep results generalisable across models, but this question does not receive enough attention. Some of these best practices would rely on capability growth being more-or-less predictable (cost for capability goes down at about this rate; voice tokens become available at this rate). It will be more difficult to design RCTs which are relevant after surprising capability gains (e.g. a pre-voice model RCT not taking voice into consideration, or some other paradigm shift).

Deliberate, cross-silo research teams

Funding should flow to inter-methodological, inter-disciplinary research teams. The most valuable RCTs will be run by teams including randomistas, those with experience feeding micro data into macro models, and those with the technical skills to actually adapt AI to local needs. We need to deliberately pair these different types of researchers.

Option C Thinking

This is the John List thinking of A/B/C trials – where the C arm is a version of the intervention that is not optimal, but what would be feasible at scale. This is worth more consideration – i.e. not only testing the best model, but having a C group in the trial which uses a cheaper model, or one that runs on most types of phones, or where the human in the loop is more indicative of the government official who’d be implementing the programme at scale.

Other ideas to explore

• Clinical trial networks could raise efficiency and shorten trial times.

• The use of AI for supplementary intervention aspects like training and outcome measurement could be one promising avenue.

• Greater emphasis on alternative approaches like A/B testing.

Share

We are not starting from scratch

Some studies are already making good progress. Fab AI, in partnership with Google DeepMind, and with support from Sierra Leone’s Ministry of Education, reported and tested an AI guided learning intervention. In just eight-weeks of this classroom RCT, they tested over 1,700 junior-secondary math students and found a large (+0.26 SD) gain. Whether this intervention was cheap to run is unclear, but it was both fast and more replicable than most.

For it not to be an exception that proves the rule, we need methodological innovation in both the interventions themselves, and also how we assess them. Interventions that leverage AI have the potential to generate detailed, timely and actionable data, but the evidence agenda needs to evolve if it is to become more rather than less relevant as AI applications proliferate.

Acknowledgements: Thank you to Han Sheng Chia for his comments on an earlier version.

This seems too crazy to be true. But after an afternoon digging into this based on the same problem being raised for those of us who use MailChimp (which I also do for my day job), I think it is actually the case.

Vive le pixel-trackers

Basically, Substack’s default way of tracking subscribers’ interactions with emails (which, as far as I can tell, we cannot turn off ourselves) is not compliant with French Law.

The underlying binding rule is Article 82 of the French Data Protection Act, which has been in force since 1 June 2019 and requires consent before accessing information on, or writing information to, a user’s device. Recently, though, the CNIL (Commission Nationale de l’Informatique et des Libertés) adopted a detailed recommendation which explains how Article 82 applies to tracking pixels in emails.

For those of us using Substack, therefore, we are most likely breaking the rules. In simple terms, Substack automatically puts tracking technology into normal newsletter emails so it can tell whether a particular subscriber opened the email. That tracking is typically done with a tiny invisible image, i.e. a tracking pixel, but French rules now say that you can’t use that kind of tracking for analytics unless the person has consented, or a narrow exemption applies.

But that’s not an option on Substack. You can’t simply say:

“Send this email to this French subscriber, but remove the tracking pixel because they have not consented.”

There was, for people already subscribed to your Substack before April 14th, a transitional period up to July 14th where you could notify them and let them object to being tracked. But that date has past, so currently, there are three options:

1. No longer write on Substack.

2. Purge your subscribers in France (calling all Jean’s, Pierre’s, and Jean-Pierre’s, you are not long for my mailing list).

3. Substack changes its policies and saves us all from French prison (alright, alright, we wouldn’t end up in prison, but could still be fined)

Another option, you might think, is sending out an email to your Substack subscribers explaining the new situation and asking for their permission to be tracked. Well, the tricky part is, if you need to obtain tracking consent from an existing subscriber, you can email them asking for consent, but that email should not itself contain a tracking device requiring consent. And Substack gives you no way to send that untracked email.

To clarify, I am not a lawyer. But I’ve now spent too much time carefully checking this and it all seems correct. And to be fair, this could even be a net positive for French inboxes. Not only are they free from our takes, but this will certainly be a spanner in the works for those incessant marketing emails that clutter all of our inboxes.

The increasingly general-purpose nature of AI brings capabilities to bear on a widening set of tasks, and new benchmarks are being released at a rapid pace to capture this generalisability. They increasingly focus on real-world applications: OSWorld measures an AI agent’s ability to complete tasks like filling a spreadsheet with information from photos of receipts, while OfficeQA tests grounded business-related reasoning over collections of firm documents. Yet despite generalising abilities and broadening benchmarks, Figure 1 below shows us that the capabilities recorded in labs (blue) are often not transferring directly into real-world adoption (red), let alone impact or economic returns – which is particularly true in low-income countries.

Figure 1: Theoretical capability and observed usage

Share of job tasks that LLMs could theoretically perform (blue) against Anthropic’s observed-exposure measure, derived from Claude usage data mapped to US occupations (red). The distance between blue and red is the capability-deployment gap; the red area is likely far thinner in LMICs, where adoption remains minimal. Source: Massenkoff and McCrory, Anthropic (2026)

The mismatch arises in the intermediary stages between lab and deployment, which introduce constraints that do not generalise across contexts. Overcoming them requires bottom-up experimentation to work out the context-specific adaptations each setting demands. So we need to build an ‘adaptation layer’: the shared infrastructure of evaluations, data pipelines, compute and talent that would let local developers adapt AI models rapidly to their own constraints. Philanthropic funders, for reasons set out below, are better placed than governments, businesses or NGOs to pay for it.

Ensuring capabilities extend to deployment requires adaptation

Broadly put (see Figure 2 below), adaptation can take place at the model training stage (upstream) where post-training can refine AI behaviour, such as LearnLM’s post-training that can solve benchmarks like CORE-Bench, or at the product application stage (downstream), where system integration and deployment happen.

Figure 2: The production function of AI

The production function of general-purpose AI, split between upstream model training and downstream product (Fraiberger and Kazemi, forthcoming).

This latter stage is increasingly accessible due to cheap compute and coding agents. For task-specific uses, it’s only getting easier to build wrappers, software products built on top of an existing foundation model. These wrappers are currently how much of the capability observed in frontier labs reaches the real world. But scaffolding cannot always overcome deficiencies embedded upstream in the models themselves. Which adaptations are needed depends on local constraints, and finding out requires rapid iteration between downstream and upstream. We need to identify the barriers that stop capabilities proven in ideal conditions from being deployed in context, and then remove them. So far, that iteration has been far too slow.

Integration into local context runs along three pathways, the three I’s: specialisation with information, orientation to institutions, and rightsizing into infrastructure. A fourth dimension, user experience, emerges only at deployment and is discussed separately below – the 3I+X framework.

As attention and money pour into scattered pilots of potential ‘AI for good’ wins, the low-hanging fruit, meaning the use-cases where constraints are weakest and returns can be demonstrated most quickly, will be picked first.

These will cluster in higher- and middle-income settings with existing data pipelines that can support task-specific training, leaving most low- and middle-income countries (LMICs) with a far lower rate of effective AI adoption. The incentives of the main actors all point the same way: governments need to demonstrate gains for citizens, businesses must show returns to shareholders, and non-profits must prove their cost-effectiveness to donors within grant cycles.

The experience of Tabiya, a non-profit that builds open-source software for labour markets in low- and middle-income countries, illustrates this. Tabiya’s conversational AI tool, Compass, draws out the skills in a person’s experience, from paid jobs to informal and unpaid care work, and turns them into a profile matched to specific opportunities. Everything Tabiya builds is developed by an engineering team based in East Africa, released open source, and designed to be integrated, owned, and operated by partners rather than run centrally. For instance, in South Africa, Compass reaches jobseekers through SAYouth.mobi and the national youth employment platform is run by the non-profit Harambee, in Kenya and Argentina this is done through youth organisations, while Zambia has a government-owned platform. Tabiya builds once and then adapts across contexts. This is why an organisation like Tabiya constantly meets the adaptation problem – each time they take their product to a new setting, they encounter new constraints.

For example, adaptation requires a high degree of trust and effortful data collection from local partners. To speed the turnaround, Tabiya developed a reproducible pipeline that fuses high-quality data structures from high-resource contexts with unstructured local information, removing the need to start from scratch each time. By taking the ESCO occupational taxonomy as a foundation and using an AI-driven semi-automatic pipeline to revise it systematically against messy local labour-market data, Compass becomes context-aware with minimal manual effort. Scaling similar data pipelines across other sectors could form a vital backbone for interoperability and context-awareness in AI products to boost replicability and development impact.

Share

X marks the last mile: Where model adaptation runs out

Further downstream at deployment, distinct dimensions of adaptation emerge which can only be worked out in the field.

Two hurdles that consistently show up are: designing for digital paradigms different to web-first environments in which most engineers are trained, and earning the institutional trust that unlocks data and deployment.

Tabiya’s Compass is a mobile-first web application, and because it handles personal employment data it follows modern best practice for secure software. This means accounts tied to an email address and password, with email-based verification. Those conventions are the industry default, and the tooling that makes secure authentication cheap to build depends on them. But they presume a specific form of digital literacy. Many jobseekers Tabiya’s partners serve are thoroughly digital, fluent on smartphones, messaging apps, and mobile money, but have rarely needed an email address with a complex password. This mismatch arises at the first screen, before any question of model capability arises. Fixing it requires authentication built on the phone-and-PIN conventions which users already trust, but there is no standard, secure, and reusable way to build it. SMS gateways and telecom integrations must be negotiated country by country, so every organisation deploying in these markets solves the same problem from scratch. A shared, open toolkit reconciling rigorous data protection with phone-based identity is exactly the kind of piece an adaptation layer could supply.

Another recurring bottleneck is a more mundane one. Data-residency rules and government preferences often dictate deployment architecture. In Ethiopia, where Tabiya works with the Ministry of Labor and Skills to digitalise the country’s public employment services, meeting local requirements meant swapping an API-based commercial embedding model for a locally hosted Qwen model. Because such constraints settle an application’s architecture first, organisations like Tabiya must remain model-agnostic, and at present that takes significant software engineering: decoupling an application from a specific provider’s ecosystem so that it can run on local infrastructure. An adaptation layer that made frontier and locally hosted models interchangeable, with minimal changes elsewhere in the application, would make these transitions considerably cheaper and mitigates against lock-in risks.

Gaps of this kind, small, local and decisive, are where AI’s promise for development is currently being won and lost.

Leave a comment

What does building the adaptation layer actually look like?

This is the opening for philanthropy. Funders able to take a longer view can back the adaptation layer, where investment generates a multiplier effect through much tighter feedback loops. A layer that enables experimentation and continuous improvement would give local entrepreneurs and developers the agency to adapt AI models and overcome their own constraints.

Across the three pathways highlighted above, we have outlined four fundable categories for philanthropy to consider, and what success would look like in each.

A few guiding notes on these proposals:

• Valuable work is already being done upstream at the pre-training layer to improve base model capabilities on languages, with novel data gathering methods pioneered by UNDP, the World Bank Group’s Development Data Partnership, CurrentAI’s Public Interest AI, and others.

• Our focus here is on adapting those capabilities to meet the task-specific requirements of each context: the next frontier in diffusing capabilities.

• The list is not exhaustive; it indicates a set with potential for high impact where we believe funding can be effective.

• We imagine each as a proof of concept which, if borne out, would build towards the adaptation layer: digital public goods that smooth diffusion across contexts.

• The proposals represent a viewpoint broadly supported by the authors, though not every author endorses every item.

• In the final section of this blog, we discuss how this holds up in different timelines.

Human-AI experiment mechanisms (Information and Institutions) – with incentives strong enough to bring in experts and hard-to-reach people in rural communities: the end-beneficiaries of AI products who often have limited digital connection. These mechanisms would enable tighter evaluation of downstream impact, while generating post-training data for specialisation and valuable behavioural evidence to guide orientation to institutions.

E.g. Holding constant the AI model and task, replicate an evaluation across diverse geographies to measure performance in different contexts and languages. This would surface the varying limitations of existing AI capabilities due to bottlenecks at deployment – gaps can present across axes such as skill-levels or unfamiliar contexts (like lower-resource languages). By using the results from these human-AI experiments, you can inform the feedback loop and then re-evaluate the model on the same end-users and tasks. A measurable improvement would motivate applying this mechanism onto other gaps between capabilities and real world use, building towards a new set of benchmarks relevant for developing contexts - such as a GDPval, but for LMICs.

Localised data pipelines (Information) – post-training AI models to suit unfamiliar contexts, with sophisticated synthetic data manufacturers which can work from limited samples to improve performance against specific tasks without triggering model collapse. Licensing models such as Esethu can then return proceeds from usage of the datasets to local communities who then reinvest the funds. This category is the key enabling stack for information-based adaptation, and works hand-in-hand with model- and product-level evaluations (levels 1-2 of the Generative AI Playbook) that would draw from the human-AI experiment mechanisms outlined above.

E.g. Gather post-training datasets for performance gaps at deployment. For instance, gaps occurring due to language, which do not occur for the same task in other languages, can be closed by gathering realistic task-specific instruction datasets in that language - such as ideal question-answer pairs generated by the end-users. Performance gains on the task-level outcomes of interest would motivate replicating the data pipeline for similar task-specific training in other contexts. A proof of concept showing that homework grading samples can be generated from just a few dozen examples to align a model to a local curriculum would motivate building similar pipelines in other contexts and curricula.

‘Small AI’ foundries (Infrastructure) - pooling the GPUs and engineering talent needed to conduct post-training from localised datasets, as well as distilling models to suit different device capacities and monitoring performance stability on downstream task benchmarks before more rigorous evaluations are re-conducted. This is the compute backbone underpinning adaptation.

E.g. A successful foundry run would show that (i) the computing and engineering costs came in below what the LMIC team would have paid for existing on-demand (typically cloud) resources, (ii) benchmark-level performance improved on the base model, and, as an optional goal, (iii) the inference-time costs of the model fell and/or its size shrank enough to operate on more edge devices.

Talent programmes (Information, Infrastructure, and Institutions) - equipping entrepreneurs and motivated members of the general public with the skills necessary to build the products they can see their communities need. An initiative that put coding agents in everyone’s hands could kick off innovation cycles buoyed by bottom-up adaptation; philanthropies such as Google.org have funded works in this spirit, including Apolitical’s Government AI Campus. Talent underpins all three adaptation pathways.

Empower end-users through a structured program which combines engineering skills such as ‘Learn Enough Claude Code to be Dangerous’ alongside best practices in impact evaluations. A successful programme would allow end-users to identify and measure performance gaps in their own workflows before setting up data pipelines and making use of foundries to assess improvements.

Learning from Tabiya’s deployment experience, and drawing from the concrete proposals above, a few suggestions emerge for the final experience (X) layer:

• Shared product-development standards, and working open-source components to go with them, that reconcile stringent data protection with the digital conventions users already hold: phone numbers, PINs, and messaging apps rather than email addresses and passwords. A secure, reusable phone-based login flow would be a natural first artifact; a common repository of such components relevant for different deployment contexts would spare local teams from reinventing the wheel or compromising on security.

• The experience layer has a second user: the institution that runs the system after handoff. Admin tooling, monitoring, and documentation that a ministry’s own staff can operate and maintain determine whether a deployment outlives its pilot, and they deserve the same design attention as the end-user interface.

• Talent programmes should not only reach end-users but upskill the local developers who have already earned the trust of the community and key actors, equipping them with the design-research tools needed to understand marginalised users. No engineer flown in from Silicon Valley for three months will build the trust that a long-standing local associate has nurtured over years.

• Modularity in model choice eases integration, since security and sovereignty considerations often favour locally hosted models over API access, and preferences shift quickly.

• Global agent certifications, so that organisations deploying AI agents on sensitive infrastructure have a recognised standard of assurance to rely on.

If we’re building god, is this really necessary?

A fair challenge to the case made so far is that the adaptation layer only matters if today’s constraints persist. Should frontier AI deliver overwhelming, general-purpose capability in the near future, then investment in adapting models to local conditions might look like careful engineering for a world about to be swept away.

How persistent the constraints will prove is genuinely uncertain. Two scenarios, drawn on different axes, illustrate the stakes:

1. Rapid capability take-off. If frontier capabilities develop along a rapid trajectory, the gap between frontier models and laggards could become insurmountable. Frontier models might then bridge deployment constraints autonomously, in the manner of a software intelligence explosion. In that world, avoiding being cut off from superior frontier AI becomes more important than adapting laggard AI.

2. Compute scarcity. If access to compute becomes limited, adaptation and efficiency gains become increasingly critical. We have lived through an era of cheap tokens, which spurred adoption with little attention to the cost of deployment. A compute crunch would force the supply of tokens to be matched carefully to the demand for machine intelligence, which is precisely the work of adaptation.

These scenarios are not mutually exclusive, and capability trajectories and compute supply could combine in various ways, but they usefully bracket the argument. Scenario (2) supports the case for the adaptation layer directly and needs no further defence. Scenario (1) is the pessimistic outlook for the adaptation layer, and it deserves reflection.

The International AI Safety Report sets out capability scenarios ranging from stagnation, as happened with flight speeds, to rapid take-off. Several commentators argue that in the latter case, exponential or super-exponential progress paired with recursive self-improvement would yield insurmountable advantages for frontier developers. As open-source AI is deployed for health and education across LMICs, the concern is that any advantage gained by adapting models to overcome constraints would be muted by the automatic advantages of frontier AI.

There are three responses to this.

First, capability gains may not translate into deployed value. We feel that model capabilities will continue to demonstrate jaggedness: whereby performance on economically valuable tasks is not directly following from standard scaling approaches, and progress may continue to suffer from a Goodhart’s Law dynamic, in which models improve against benchmarks faster than against the real-world tasks the benchmarks stand in for. This would blunt, even if it does not eliminate, the practical advantage of self-improving frontier AI.

Second, and more fundamentally, the adaptation layer is engine-agnostic. Its function is to translate whatever technological capabilities exist into deployment within particular institutions, languages, infrastructures, and regulatory settings. If frontier AI does deliver large capability differentials, that translation work becomes all the more valuable, since the returns to closing the gap between abstract capability and realised productivity grow with the capability itself. One should be able to change the engine without rebuilding the vehicle.

Third, open models retain structural advantages that are independent of raw capability: adaptability to local context, control over weights and data flows, and integration into institutions. Human institutions move more slowly than capabilities develop, and sovereignty and security concerns accentuate that slowness. Being able to swap a frontier model for one over which sovereignty is retained, much as one might switch from grid electricity to a backup generator, may prove the expedient geopolitical strategy even at a substantial cost in capability.

The answer to whether this is necessary, then, is yes. Even if we are building a god, its works arrive in the world only through broad, agential access, and that is what diffusion infrastructure secures, even as the underlying models are continually replaced by more capable successors. Frontier access will still be required in particular domains, security among them, but that is a complement to the adaptation layer rather than a substitute for it.

Where the money should go

So, we should fund the connective tissue rather than another pilot: experiment mechanisms that measure where capabilities fail in context, data pipelines that close the gaps, foundries that make the compute and engineering affordable, and talent programmes that put adaptation in local hands. None of these seem like moonshots, rather each contributes to infrastructure that makes many small moonshots possible. And Tabiya, AdalatAI, Viamo, and others show that the supply of AI tools is already expanding to meet government and consumer demand in LMICs, so there is no need to reinvent the wheel entirely. An adaptation layer can boost these existing initiatives and spark many more which, launched from the ground up, will ensure that AI’s capabilities will be converted into development impact.

Acknowledgements: The first schematics of the adaptation layer framework were developed by Sharif Kazemi and Samuel Fraiberger during the proceedings of the World Development Report 2026. The case study from Tabiya was elaborated clearly by Jasmin Baier, Christian Meyer-Wratil, Jonathan Stoterau, and Minaz Ranjita Singh. A few ideas were developed in discussion with attendees at the Rethink Priorities AIxGHD Strategy Forum on June 1, 2026, which followed Chatham House Rules. Further helpful comments were provided by Stephen Clare, Lucas Irwin, Niall Maher, Joseph Levine, and Philipp Zimmer.

Thankfully, though, there are an increasing number of researchers and practitioners bucking the trend. So I thought I’d compile the ten most interesting pieces I’ve recently read on AI and development. If you find this useful, I’ll do it again when I reach ten more things worth reading.

Subscribe now

Thing 1: What will be scarce

Ghosts of Electricity

What will be scarce?

Starbucks is a huge company (market cap of $112 billion) that sells one of the most standardized products in the modern economy. Making a cup of coffee or even one of the fancy specialty drinks is very easy to mechanize and reproduce. If the entire economy is soon to be automated, with labor being replaced with increasingly more sophisticated capital, S…

Read more

5 months ago · 1471 likes · 137 comments · Alex Imas

I contemplated whether it was even necessary to link to this, as I’m sure you’ve all read it. A couple of thoughts. First, this is exactly why I called for more researchers like Alex Imas writing about AI and development. Second, in the comments Alex promised a separate piece outlining what this thesis would look like for developing countries, which I am excited to read.

As I commented, my initial reaction was that the future Alex outlines is a hard one for developing countries to navigate, and I’m not sure how the preferences for a human touch will extend globally - i.e. if a society has experience with human therapists, their preference for them over AI could be different to a society with no prior access to human therapists. If our preference for humans in different roles is not a given, but is learned through experience, then it’s unclear to me that it will hold in places where AI extends access to doctors/therapists for the first time.

This is clearly a crucial research agenda more generally, i.e. doing the type of work Alex did on preferences for AI-generated art, across countries and sectors. Would love to hear about more research of this type.

Thing 2: The intelligence is plenty

Dan Björkegren: “The economies that LMICs trade with, compete with, and depend on are being reorganized around abundant intelligence. The question is not whether LMICs will be affected—they will be—but how they will engage. The analysis above suggests that AI that augments knowledge workers is likely to disproportionately benefit rich countries. It is less clear what alternative paths exist, but there are actions that could offer opportunities, particularly for middle-income countries and more advanced regions within low-income countries.”

YES.

Thing 3: On India’s AI Policy

Rohit Lamba

On India's AI Policy: Concrete the state understands, Cognition it does not.

I wrote an op-ed for the Indian Express with mentor and co-author Raghuram Rajan, the full version appears here in the post…

Read more

4 months ago · 9 likes · Rohit Lamba

Rohit Lamba and Raghuram Rajan: “India does not need to beat OpenAI at frontier reasoning today, but it needs to learn to compete: Alternative model architectures and approaches at meaningful scale, sovereign compute, and foreign deals structured around expanding Indian capability rather than only power and land. The Indian state knows how to create a gigawatt of power for a foreign data centre. It needs to learn how to create a research department, a laboratory, a generation of scientists”.

At the moment, it’s very much the US, then China, at the frontier — BIG GAP HERE — and the rest of us lagging behind. I’m not sure how feasible it will be for any country to join the US and China, and it would certainly be foolish for most to try. But this is an interesting argument for India to make that bet.

Things 4 and 5: AI in an open economy

On ‘the rest of us’, Raoul Ruparel has a great two part series on his Substack.

“Fundamentally, given ownership concentration, we could see a situation where the owners of the capital and IP are located in the US or China but the displaced workers are in countries such as the UK, India, and other major importers of AI or exporters of services.”

Raoul shares my worries.

Raoul, in Part 2, thinks through some solutions, and applies them to the UK, but I think there are many generalisable lessons/thought experiments.

Thing 6: Leapfrogging?

Rose Mutiso joined The AI Shift newsletter by the FT (John Burn-Murdoch and Sarah O’Connor) to outline why she feels countries can’t leapfrog into AI, and a whole lot more.

For those who don’t have a subscription, Rose discussed similar themes on Deena Mousa and I’s podcast.

Ideas in Development

Can AI take off in Africa?

You can listen to this podcast on Spotify, Apple Podcasts, or wherever else you get your podcasts. Or you can watch this conversation on YouTube…

Read more

7 months ago · 7 likes · 1 comment · Oliver Hanney and Deena Mousa

Thing 7 & 8: AI, other general purpose technologies, and South Africa

Our Long Walk

What I told our HoDs about AI

Yesterday I gave a talk to twenty-five of Stellenbosch University’s Heads of Department about artificial intelligence. I wanted to write it up, partly because the argument is easier to follow as an essay than as twenty slides, and partly because I suspect others might find it interesting…

Read more

4 months ago · 15 likes · Johan Fourie

“The interesting question has never been ‘will it replace us?’ It has always been ‘where does labour go, and who gets there first?’ AI fits that pattern, with one new wrinkle. It automates cognition itself, and its speed of diffusion is unusual even by the standards of earlier GPTs. That combination is what makes the present moment genuinely different…” Johan Fourie

Also from Johan, on AI in South Africa.

Our Long Walk

The preacher's assistant

Over the past two weeks, I’ve had three conversations about artificial intelligence. The first was a podcast with Chinasa T. Okolo, a computer scientist with a PhD from Cornell, a former Brookings Fellow, and one of the leading African voices on AI governance (available soon). The second was lunch with Ufuk Akcigit, a world-leading innovation economist …

Read more

5 months ago · 13 likes · 1 comment · Johan Fourie

Thing 9: Tech for good in the AI age

Han Sheng Chia: Cutting Through the Noise: Reimagining Tech for Good

“The point is not to overgeneralize harm—if well developed and scaled, AI can deliver substantial positive impact. It’s to understand under what conditions harmful uses scale so dramatically that they create “overmatch”: a situation where positive applications are dwarfed by the reach of harmful ones.”

Thing 10: Many more things

I had previously been compiling everything relevant to AI and development economics (not just the ‘big’ questions) in this VoxDev blog, and I will continue to update it periodically.

AI and development economics: Early evidence and how to keep up

Having caught up as best I can, and having also written about this recently (The development economics I’d like to see), I’ve summarised some additional thoughts I’ve had on this topic, spurred by the interesting feedback I received and some reflections on the Twitter debate. Ultimately, I think a lot of this boils down to whose questions we are trying to answer.

Subscribe now

The micro and macro caricatures

If I were to caricature what each side of this debate says about the other:

• Micro on macro: Your methods suck and can’t answer the big questions.

• Macro on micro: Your methods only answer small questions that don’t matter.

These are caricatures, so only represent the views of a few people. Naturally, I disagree with both. I think we’ve learned a lot of useful lessons about questions that do matter from micro development research. I also don’t think we should ignore, or give up, on the big questions that macro development tries to answer, even if the answers are harder to reach, and less ‘clean’.

I think this point by Jesús summarises best how I think about this debate (plain English translation below for non-academics).

To summarise, it’s not that micro development papers are somehow bad in and of themselves. It’s that we’ve allocated too much research effort and funding to answering narrower questions using the causal identification toolkit relative to macro development work on the bigger, thornier questions.

If funding, career and publication incentives had been more equally distributed in recent decades, and not so skewed to micro, we’d likely have a deeper understanding of the important growth-related questions, and better methods to answer them.

An aside: I would love to see actual data on the research funding breakdown across development economics. Anecdotally, the research I see overwhelmingly falls on the micro side, of which RCTs are actually a relatively small part. But then again, RCTs are more expensive than the typical research project. Anyway, I have a strong prior on how funding breaks down overall between micro and macro in development, but would love to see the data.

Big questions, for who? Donors vs governments

I’ve found myself using the phrase ‘the big questions’ a lot recently. It’s worth unpacking that a bit.

Ultimately, a lot of this debate boils down to whose questions we are trying to answer. You can probably trace a lot of the micro bias of recent years to the questions that donors are grappling with.

To simplify, donors had (and some still have) billion-dollar+ budgets, which are typically spent on programmes, and they wanted to know how to spend it more effectively. So, it makes sense that they funded micro development research to help them answer those questions – the tools of micro are better suited to establishing the impacts of their programmes, things like expanding access to microfinance, distributing deworming medication, or improving teaching in LMIC classrooms.

These are certainly big questions in the sense that there are billions of dollars of funding involved. So even if you can influence a small percentage of that funding to be spent more effectively, then the research will have been worth the cost.

But donors and their programmes, despite their many successes and the millions of lives they’ve saved, are actually a very small part of the most important development story of recent times – sustained economic growth in China, India and elsewhere, which is responsible for the vast majority of people moved out of poverty in recent decades.

Given that donors came to represent a large chunk of those who were funding research in development economics, this skewed the overall discipline in the direction of answering their programmatic questions, rather than the type of questions a government would be asking if they were looking to grow their economies, which is ultimately the best way of alleviating poverty at scale.

There are two things to consider here:

1. The total potential impact of research (Macro wins)

2. The likelihood that research has an impact (Micro wins)

I know, this is a gross simplification, as there is no guarantee that a given piece of macro research actually sheds light on replicating the economic success stories of recent decades, nor is there a guarantee that micro research actually shapes policy to be more effective or kills an ineffective aid programme.

But given that micro has dominated development economics, it does feel reasonable that the discipline shifts more attention to macro. Even with a lower chance of impact, the potential impact is so much bigger that it feels important to do this as well. And I imagine some of that rebalancing will come naturally, given that there is less aid funding to influence these days.

Subscribe now

The wrong questions?

This links to a separate but related point I made in my recent blog, that there is a serious mismatch between policy questions and research questions.

One point that was missing from my blog, which I think is important to highlight, is that it’s certainly possible that policymakers are asking the wrong questions.

I guess I’m still not sure how to deal with this. There is clearly an important role for research to play in helping to independently shed light on the types of questions policymakers should ask. But it’s unclear to me how feasible it is to influence these, particularly in the short to medium term.

If a policymaker comes to you and says ‘here’s what I’m doing, how do I do it well?’, good luck telling them that, actually, it would be better if they did something else entirely. Instead, the most fruitful route to impact in these circumstances is likely, regardless of the policy, making sure it is done effectively. This is the Stefan Dercon art of ‘second-best’ policymaking. But where you draw the line, that is which economic policies are so bad that you choose to play no part, even if you could improve them on the margin, is murky.

Who’s leading the charge

One glaring problem with this debate is that it’s being led by academics at US and European institutions. Ultimately, it should be led by academics at universities in Latin America, Africa, and Asia. But development economics has failed to empower them, and they are often forced to focus on the questions they can get funded, rather than the questions they know are most important for their local context.

Some progress has been made, and it’s now more common to see local academics added as a co-author to a US or Europe-led project. But a more fundamental change is needed. The ideas phase should instead be led by those closer to the questions that need answering, who can then add other academics if they have methodological expertise to bring to bear, or funding networks they can tap into.

Share

Progress is being made in macro development

I also wanted to emphasise that macro development is not starting from scratch, and progress has been underway in recent years.

As I discussed in my recent post on AI – There is no randomising a technological revolution – there are bright spots in the discipline, with tools emerging that are suited to this class of questions. The Structural Transformation and Economic Growth (STEG) programme, run by Doug Gollin and Joe Kaboski, stands out as one. STEG coordinates and funds research on macro questions in developing countries, including structural change, productivity and growth, often drawing on micro data. This goes beyond just industrial policy and looks at how a range of policies on human capital, financial markets, and agriculture, could ultimately lead to economic growth.

And this did not happen by accident, but was a result of funders pushing the growth research agenda at a time when the randomistas dominated. So, it’s worth highlighting the work of those at DFID, now FCDO, like Andy Hinsley and John Piper, who funded PEDL, and then STEG. I read a lot of development economics, and research originating from these two programmes consistently feels closer to the big questions on growth.

(It’s worth noting my conflict of interest here, as the same FCDO team funds VoxDev, and I’m based at CEPR which manages both PEDL and STEG, but I stand by the points I’ve made above).

And it’s not just STEG, Oliver Kim, for instance, has a number of relevant papers here.

And this thread also identifies a range of useful research on these types of questions.

So, overall, I’m glad this debate is being had, and I’m hopeful that we are on the way to a renewed focus on, and understanding of, the ‘big questions’ in development.

To help this effort, I think we’d benefit massively from a much more systematic attempt to collect policymakers’ questions, as it often feels like we are shooting in the dark when trying to establish what really are the most important questions – or, at least, those with the most interest from policy audiences. Funding that type of effort would seem to be a great guide for the discipline, both in terms of matching the body of evidence we already have to existing questions, and acting as a north star to guide future research agendas.

Keen to continue the conversation in the comments, or on Twitter.

“The sources of uncontrolled migration flows, the sources of illicit finance, the sources of terrorism, extremism… they come from these challenging environments”.

“If the global north is not engaged in investing in and stabilising these places, it impacts on the well being of the UK because the world is so interconnected.”

Leslie Maasdorp via The Telegraph.

Picking on this alone is a bit unfair. Donor governments and organisations have made a habit of endorsing this false promise of aid, that it will somehow help to reduce migration.

In fact, there’s an impressive consensus on this particular justification for aid, across a range of current and previous governments, of all political stripes, in the UK (Labour, Conservatives), the US, Spain, Italy, and more.

The problem is, it’s wrong.

Subscribe now

Aid could actually increase migration

The best evidence we currently have suggests that foreign aid will not reduce migration. In fact, if aid successfully spurs development (a big if), it would actually increase it.

Gröger, Heidland, Wellner, and Fuchs look directly at the impact of foreign aid on migration, combining data on the allocation of World Bank aid projects with survey data and data on migration flows. They find that while aid temporarily reduces irregular migration, in the longer term it actually results in larger regular migration to high-income countries.

At VoxDev, we have also featured research which refutes the conventional wisdom in policy circles. Using data from 99 developing countries, Clemens and Mendola show that investing in and promoting economic development would increase migration – rising incomes in lower-income countries lead more people to emigrate.

Clearly, this pushes back on people’s priors, and I can see why you might expect the opposite to be true. Clemens and Mendola’s work helps us to think through why people’s initial intuition is leading them astray. The following is taken directly from their VoxDev article – Who migrates from developing countries?

If one interviews migrants from developing countries, they frequently report lack of economic opportunity as a central motive to migrate (e.g. Garver-Affeldt 2021). How, then, could greater opportunity at home fail to reduce migration? This reasoning can be deceptive: those too poor to migrate do not become migrants, and thus are not interviewed. Rising incomes might have deterred some interviewed migrants from migrating; but rising incomes can also turn those unable to migrate today into people able to migrate tomorrow. The latter effect is ignored if we rely on evidence gathered from migrants, a textbook case of selecting on the dependent variable (here the choice to migrate).

Direct observations can also trip us up in a different way. The relationship between earning opportunities in the home country and emigration can be positive for any given group of people - or even every single individual person - but still negative in the aggregate. The reason is the well-known logical fallacy of Simpson’s Paradox.

For example, suppose that people with high school education are more likely to emigrate than people without. And suppose that within these two specific groups, the effect of income on migration is negative, such that for both people with and without high school education, higher incomes reduce the tendency to emigrate. In the aggregate, income can nevertheless raise the tendency to emigrate because higher incomes typically cause more investment in education, shifting people to a higher-income group. Even if the relationship between income and migration is negative within specific groups, rising incomes can increase migration by shifting people into different groups.

Figure 5 uses our data to provide a snapshot of Simpson’s Paradox in the real world. Across all developing countries collectively, income and emigration correlate negatively within education groups, but income also shifts people into groups with higher education and higher emigration. The net effect is that for poor countries, rising aggregate incomes go hand-in-hand with rising aggregate emigration (Clemens 2020).

A white lie, but an effective sales pitch for aid funding? No?

In the scramble to save aid, I guess it’s not surprising that this sales pitch is rearing its head. So, why bother calling this out now? Is this not just some harmless comms, a white lie, that can help build support?

Well, most importantly, if we are aspiring to be evidence-based, that needs to include our comms too. But even leaving ethics aside, if we are thinking strategically, misrepresenting the impact of aid is certainly not sustainable in the long term, as we are setting aid up to achieve a target it won’t (a target which I also disagree with, but that’s beside the point).

And even purely in the short term, despite the wide array of people making the ‘aid reduces migration’ pitch, I have seen no evidence that this argument is actually achieving their end goal of increasing support for aid.

This ties into a broader problem in development – the evidence gap on evidence use. While we do now have a good amount of evidence across a range of areas (even if we lack answers to some of the most important questions), we have very little evidence on how to effectively use that evidence, how to communicate it, how it gets used, and how to use it proactively to influence policy. In the absence of that, despite the amazing work of people like Michael Clemens generating and sharing the evidence on aid and migration, it’s still not getting through.

I’ve been working on a series of podcasts on exactly these ‘evidence use’ questions, they’ll be on Ideas in Development starting next month.

My main takeaway though, was not what I learned, or the cool people I met, but what development economics is missing. It struck me that almost all of the work, policies and experience being discussed by the policymakers, practitioners, investors and funders at the summit is absent from development economics research. We are missing a whole thriving ecosystem working on growth.

On my new VoxDev podcast, Ideas in Development, Kartik Akileswaran and I asked Piero Ghezzi whether he felt like the research he saw provided answers to the types of question he had as Minister of Production in Peru. Clearly, he’d reached the same conclusion from his time in government.

“I wasn’t too impressed in general with the development literature”

It’s vital that development economics grapples with this mismatch between research questions and policy questions, which is particularly severe when it comes to growth.

Subscribe now

Development economics is missing a whole ecosystem of growth

I read a lot of development economics research. But at the summit, I felt as though much of the research I’ve seen, commissioned and edited over the past three years did not speak to the issues being discussed, or the questions policymakers were asking.

While development economics should be a broader tent than just growth, growth, growth, it has become a sprawling discipline, covering essentially any topic that econometrics can be brought to bear on, in countries making up the majority of the world’s population (there is no single definition, but I collected some good ones here).

In a variety of areas, we have made wonderful strides in our understanding of what works for policy. Across education, health, microfinance, trade and many other areas, we are making lots of progress. I’m particularly bullish on education – I don’t think it gets the attention it deserves from funders, but that’s for another blog (maybe).

And these do all have indirect implications for growth. But if we are thinking directly about economic growth, and the type of questions a policymaker would ask on that subject – something along the lines of: how can I get growth going before the next election, or replicate the success of East Asian economies – development economists have little to say on one of the main levers they can pull, industrial policy.

Meanwhile, away from the pages of economics journals, every government, everywhere, has been doing industrial policy in some form. Some well. Some poorly. Some without realising it.

It’s a bizarre oversight by the discipline. One that was probably partly ideological, partly methodological, and partly a matter of incentives (some similar issues which I surfaced in my recent AI blog). If research questions remain divorced from policy questions, then development economists have nothing to say about this key pillar of most countries’ growth strategies.

An aside: What even is industrial policy?

This table from Ana Margarida Fernandes and Tristan Reed’s new World Bank report, Industrial Policy for Development, is a good primer on what industrial policy looks like in practice, outlining the main industrial policy tools available to governments.

This report has drawn its fair share of ire, some reasonable – I can understand the frustration of policymakers around the developing world at this about turn by the World Bank – and some lazy – courtesy of The Washington Post.

I like what I’ve read of the report (not all 270 pages FYI, as Todd Moss says, death to 100+ page policy report). And the graphic above is a nice summary of what industrial policy entails, which are some of the first ports of call for governments looking to kickstart growth. But this list alone is not earth-shattering news. Most policymakers know their options, and what they want to know which one they should choose, and crucially, how they should implement it.

Chapter 3 of the report details what we know so far on the ‘how’ for these different options, it’s certainly worth a read (I like the principles of practice), but my impression is that we have a lot more to learn.

And that makes sense, as it’s exactly these questions, on choosing between policy options, and actually getting stuff done (implementation), where development economics research has really dropped the ball.

The mismatch between research questions and policy questions

How you actually do industrial policy well was a key theme of the conference.

On the last day, I visited an industrial park with policymakers and practitioners from across Africa. Their questions were almost entirely practical, on implementation, e.g. how does power come through, what does it cost, how do you make people deliver on their contracts to build the necessary infrastructure, which government agency is really in charge, how do they coordinate the different actors, where does the money come from.

Across the conference, similar questions arose. Those who’d had success on the policy side reflected on the importance of having talented and capable bureaucrats working with them, effectively coordinating the public and private sector by creating mechanisms for useful dialogue, the nitty gritty of attracting investors, and the importance of that first anchor investor.

Evidence at the conference mostly came in the form of anecdotes, reflections, experience, diagnostics and case studies – not the type of evidence that you could publish in an economics journal, but important nonetheless, and the best we’ve got given the oversights of modern development economics.

Sure, there are lots of messy questions here. But they are extremely policy relevant ones. So, if you are looking to do impactful research, that corresponds to what policymakers actually ask, there are a wealth of important questions in this space.

Share

The real plumbers

Luckily for researchers, there is an established and growing community of organisations that have already been thinking about these issues and directly involved in the nitty gritty of implementing on the ground.

I’d love to see development economists interacting with, and who knows, maybe even partnering up with, these organisations (I am listing some of the many examples from the conference here). Development economics would be far richer for it. And I think it could wind up helping these organisations, who might struggle to quantify the exact impact they have when talking to funders.

• Growth Teams – they have built a cool Export Boom Atlas full of case studies.

• The Harvard Growth Lab – who produce growth diagnostics (and a range of other resources) for governments around the world, that attempt to answer how a Minister should decide to spend their limited budget.

• Gatsby Africa – who work on sector transformation in East Africa.

Another aside: The missing middle in development economics

As I mentioned in my communicating economics guide, I feel there is a sweet spot where evidence is most useful to policymakers, between the country-wide macro and the village-level micro, between what can be studied credibly, and the importance of the question. Research at that sector or ministry level, between the remote village and the entire country, seems to better match the level at which decisions are made.

One last aside, I promise: Expert-based policymaking

The evidence-based policymaking debate has always felt a bit odd. Countries that successfully industrialised did so without needing ‘rigorous’ (by today’s standards) development economics research. That doesn’t mean they did so without evidence, they just relied on different, less ‘rigorous’, more practical evidence.

A big theme of the conference was that learning by doing is a crucial part of the growth process, and successful industrial policy efforts. There’s a necessary process of accumulating institutional knowledge that helps countries move up the ladder. Another key form of evidence on industrial policy, it seems, comes from sending delegations to South Korea, China and other recent industrial policy success stories (delegation-based policymaking?). And experts have also played their role from China – The Cruise That Changed China – to Vietnam – Pham Chi Lan discussed the importance of different international experts and advisors on our podcast.

I have been working on a series of podcasts on the role of evidence in policymaking – subscribe to Ideas in Development to receive these, first episode with Michelle Rao coming sometime in May. One point that surfaced in a scoping discussion with Michelle is that development economics currently does a lot of ‘expert-based policymaking’. That is, development economists earn their stripes, publish in Top Five journals, get tenure, and are then unleashed on the policy community to give expert advice to senior politicians.

But my view is that often the research they earn their stripes doing does not actually make them an expert on the types of questions policymakers ask. So, more policy aligned research would not only improve the evidence base, but also the expert base, who seem to be one of the main avenues by which research currently shapes policy.

Of course, there are some experts out there who have been doing the type of work I’m calling for. One example that comes to mind, on industrial policy specifically, is Dani Rodrik. I guess my argument is that there should be ten Dani Rodrik’s, each with different perspectives and focused on different countries or regions.

This is not an original take

Many have been making this argument. Lant Pritchett has some particularly strong views, as he discussed at the conference in Morocco, and outlines on his Substack.

The problem with the move of development economics to RCTs is not that they only answer a small part of the needed knowledge to help countries develop. The problem is that they address none of it. That is, ask the question: “How would my country replicate the success in economic growth (or more broadly national development) that leads to higher levels of human wellbeing, like South Korea or Chile or Vietnam?”… The reason this research is chicken feed is that it is about feeding “chickens”—about how to do charity work, which mitigates the worst consequences of the lack of national development, better—and not about development at all.

As is probably obvious now, I agree with Lant’s overall take, and he has certainly shaped my thinking on this. Unfortunately, while I understand his frustration, I am not convinced that his combative approach is actually going to win hearts and minds in the discipline, or do much other than annoy, offend or put people off.

I realise that this is my second blog in a row ‘criticising’ development economics, which is slightly at odds with my day job at VoxDev, promoting and sharing the development economics I critique above. I’ve written many blogs highlighting the lessons we have learnt from development economics, so my intention is not to bash researchers, but highlight some of the high value work I’d like to see done.

So, what to do about this

Development economics has become all of economics as it happens in the ‘developing’ world, which itself lacks a good definition. I’d like to see the discipline recapture a central mission, growth (and/or structural transformation).

As development economics sprawled, it lost this central purpose – that is, trying to understand how countries can develop.

But over time, the line is becoming increasingly blurred between what is currently counted as development economics, and what would better be described as labour economics that just so happens to be in Kenya, or health economics that just so happens to be in Colombia.

It’s great that development economics provided a home to those academics who wanted to study questions in LMICs, but were shunned by an economics discipline focused exclusively on the US and Europe. But we are reaching the point where labour economics should take a global view, rather than implicitly referring to the labour markets of the US and Western Europe. Health economics should think about China and India, as much as the US. Public economics should investigate public policy from Madagascar to Norway. Looking at the growing geographic diversity of field-specific journals and conference programmes, this does seem to be happening naturally, if slowly.

And in that world, development economics should still exist, but in a narrower, more focused form, on the big questions about development and structural transformation. In fact, it just so happens that we are now at a funding cross-roads that could act as a final push. With universities and development budgets under pressure, as researchers scramble for new funders of their development research, the new players in town (themselves funded by newly minted tech billionaires who have committed to giving away significant portions of their wealth) will have an outsized ability to shape research, and a lot of money to do it with.

Finally, I will reiterate a point from my last blog, that I hope this period of disruption (not just in terms of funding, but AI too) can also wrest control of development economics away from universities in the US and Europe – and move the discipline from epistemic authority to context authority. I think that would also help close the gap between research and policy questions, as local researchers are closer to, and understand better, policy priorities in their context.

If you are a policymaker in a low- and middle-income country, you might be wondering how best to grow your economy, manage the fallout of a war that’s spiking energy and fertiliser prices, or plan for the AI age.

If you are a typical development economist, you likely haven’t got answers. More worryingly, you probably aren’t spending your time thinking about, or working, on these questions.

To be fair, they are hard questions. But development economics as a discipline is not set up to even attempt to answer them. Individual researchers aren’t to blame, but something needs to change.

In this blog, I’ve focused on AI as a ‘moment’ which development economics is failing to meet. I’ve been thinking about this a lot, and I recently finished a series of nine episodes with Deena Mousa for VoxDev’s new podcast – Ideas in Development – on topics relevant to the AI and development debate.

Below, I delve into why development economics needs to rapidly step up to meet this AI moment, why I’m not confident it will, and outline some of the questions we need answers to.

Uncharted, muddied waters

There is a dearth of independent advice available to governments navigating these uncharted AI waters. This seems particularly pronounced for developing countries who are thinking about their AI policies. Who should they turn to? The World Bank (who might be their lender), AI labs (who want investment and are thinking about future revenues), other governments (who are trying to navigate the China vs US geopolitics), or consultancies/private firms (who are in the habit of selling unnecessary and expensive enterprise solutions to governments).

If the correct policy advice is – actually minister, you don’t need to borrow to build a bunch of data centres, and you can wait a few years for capable and compressed open weights models that are suitable for most government applications, just make sure to sort out you data and energy infrastructure with some smart investments – who is motivated to give that advice?

This could be flawed. For the record, I feel it’s still not clear what the optimal approach to AI policy would be. But those currently offering advice have no incentive to recommend this type of approach.

So, enter development economists? This could be a chance for them to step in and be the independent, ‘evidence-based’ voice of reason. But they don’t have the evidence, and I’m not sure whether they will in time for it to matter.

And in their absence, we’ve ceded the public debate on economic impacts and AI policy to technologists. This group dominates the current headlines and is mostly made up of computer scientists on the West Coast, who have a poor grasp of economics, and of the world outside Silicon Valley.

The AI 2027 forecast last year made a lot of noise, encouraging everyone and their nan to weigh in with their take on timelines. What stood out to me were, timelines aside, the insane predictions on global poverty. There were two scenarios in this forecast. According to the positive AI 2027 forecast, by 2029:“Even in developing countries, poverty becomes a thing of the past, thanks to UBI and foreign aid”. And even in the negative scenario, where humanity is extinct by the mid-2030s, at least by 2029 there is “an end to poverty”.

Just because development economists can’t suddenly produce clean causal estimates about the effectiveness of different AI-related policies, or how AI will impact different economic outcomes, they can still do a lot better than the current standard of debate.

There is no randomising a technological revolution

While development economists have spent the past few decades of the credibility revolution and randomista movement honing their causal inference playbook, for the big questions on AI, I fear that’s not what we need.

No doubt, they’ll do a great job running RCT’s on the various applications of AI in developing countries.

I’m sure there are hundreds of RCT’s underway testing how, now out-of-date, generative AI tools, perhaps used by some type of public servant (teacher, nurse, bureaucrat), improved a health or education outcome. Run the first draft through refine.ink and I’m sure we’ll learn a useful, if very specific, lesson from the working paper – even if it’s completely irrelevant by the time the paper ends up being published five years later.

We should be running these types of experiments, but there is a huge gap on the more important, more ‘macro’ economic questions related to AI and development. And that is not a surprise. Development economics is not set up to grapple with these big picture questions.

Partly, this is an academic incentives story. But it’s also a story of the methods and questions development economists have gotten used to, which are completely unsuited to the important questions on AI, the type that rely on scenario planning, and macro modelling.

There is a growing body of researchers and thinkers in the US and Europe starting to grapple with the big economic questions of the AI age, but not yet in development economics. And this is despite some of these countries potentially being more exposed to the initial economic disruption AI is set to cause.

For sure, there are some bright spots in the discipline, with tools emerging that are suited to this class of questions. The Structural Transformation and Economic Growth (STEG) programme, run by Doug Gollin and Joe Kaboski, stands out as one. STEG coordinates and funds research on macro questions in developing countries, including structural change, productivity and growth, often drawing on micro data.

But even if the methods evolved, and everyone started doing STEG-style research, there’s still a bridge to be crossed. Given the rapid pace of technological change we’re seeing, we can’t over-index on current impacts, even if we were to measure them accurately. We desperately need speculative research, planning and thinking.

Share

How can development economics step up

We need a range of development economists involved big-picture thinking, informed predictions and careful thinking through of future scenarios, even if these methods lack the rigour needed for a top five publication.

This does not mean abandoning evidence or economic theory. But there will be a small group of people shaping economic narratives and stories in the near future that will influence the direction, and diffusion, of AI and AI-related policies. I’d rather that economists working and living in different contexts are able to carefully think through what AI might mean for their country, rather than completely ceding this space to computer scientists in the US.

This might ring some alarm bells as, for good reasons, many development economists have retreated from the big-picture grand theorising of previous generations. But if developing countries want to keep up, RCT’s are not going to cut it.

The conversations I’ve had on this topic typically end with, ok, but what about academic incentives? This is a tricky one, as effectively communicating the potential impacts of AI poses a number of problems.

For example, this more speculative work might mean putting your name to an article, paper, or idea, that is shown to be completely wrong. Clearly this wasn’t an issue for the development heavyweights of the past, cc Jeffrey Sachs. But ironically, in this moment, I would argue that we need economists willing to put their neck on the line and put ideas in the public sphere, even if they only cross the 70-80% confidence threshold, rather than the 90-95% they need to feel about their work before submitting a paper. Shying away from the big questions might stop individual researchers from being embarrassingly wrong, but it cedes the debate to people less likely to be right.

I’m not sure this will actually happen. All economists, and in my experience especially development economists, have been conditioned into extreme caution. So, we need to think about incentivising speculation, and coaxing experts to carve out time to give their two cents.

What would this look like? In the short term, we need Alex Imas and Anton Korineks for Africa, Asia and Latin America. Alex and Anton are quite a unique blend of senior enough to have tenure and therefore step away from the publishing rat-race to dedicate time to thinking about AI, but in the loop enough to have been using these tools and be relatively up to date with the AI frontier. This narrows the field of suitable candidates. We don’t really want dinosaurs who’ve outsourced their code to RA’s for decades suddenly opining on AI. And while it would be great for more PhD students and Assistant Professors to weigh in, it’s not really a reasonable expectation, given the incentives they face, to sacrifice papers for a weekly Substack.

Money is one solution, but I’m not sure where this would come from – Anthropic’s economics team only has so much funding. Ventures like the African Tech Futures Lab (founded by the excellent Rose Mutiso) also aim to start enticing Africans to contribute to this debate.

From personal experience, at VoxDev, very few people say no to writing for us, even though we don’t pay authors. Clearly, they have an intrinsic motivation to contribute to a public good while also valuing the platform we can offer their research – even if a VoxDev article is not going to add much to their academic CV. I think academics appreciate being prompted to write and being helped with the writing process. So, it’s not an impossible challenge to solve.

Telescopes to microscopes: From epistemic authority to context authority

One side note, which perhaps reflects wishful thinking on my behalf, is that this feels like an opportunity to finally wrest control of development economics away from universities in the US and Europe.

Development economics is massively desensitised to its domination by Americans and Europeans. With any objective lens, even after the limited progress of recent years, it’s obscene. Left to its own devices, the discipline has remained an enterprise for elite universities thousands of miles from the countries being studied. We literally still host development economics conferences in places which are impossible, or prohibitively expensive, for academics from the regions being studied to actually attend in person.

AI will likely be a force against localising research in the short-term, as the AI adoption gap, and AI capability gap, between economists at MIT and Makerere widens. But longer-term, as the models necessary for economic analysis diffuse, get cheaper, and easier to run, my hope is that this period of change will result in those based at institutions in Africa, South Asia and other LMIC’s on top of the discipline. Imagine that.

What will we need in five years’ time, if the barriers to learning and adopting the cutting-edge econometric methods have lessened? Local data, local context and local questions.

Maybe this is wishful thinking, but I hope that we’re moving into an era of economists putting their own economies under a microscope. We’ll learn more than using a telescope from afar.

This needs to be coordinated somehow

While academics get excited about doubling their productivity and churning out papers, we need to think about an architecture that pieces together the puzzle.

This includes platforms that screen for quality and relevance and then disseminate open access versions of reliable papers alongside accessible summaries that tease out the main takeaways. A VoxAI, if you will. (Let me know if you’re interested, or if you’d fund this – I’ve not had success in pitching it so far, which is either a reflection on the idea, or my pitching skills, it remains to be seen).

We also need living literature reviews, that look to piece together insights from the slew of new papers to draw overall reliable takeaways as they emerge and compare and update different predictive exercises.

With the state of recent job markets for economists, I imagine there will be more than a few highly talented PhDs in need of a job. With a bit of money, they could get to work on this type of project, if those who were lucky enough to get hired can no longer justify it career wise. I see a huge opportunity here to create a high impact public good that curates good quality research on the economics of AI in an accessible way for policymakers and the public, while also coordinating synthesis work.

Subscribe now

Papers I’d like to read

I realise I’ve pointed a lot of fingers, and talked at length about the big questions, the important questions, that development economics needs to answer.

So, what exactly are these questions? Well, I thought I’d list a few of my main ones here.

Speaking of misaligned incentives, there is a lot of room in the short term for rich descriptive research on AI in LMIC’s, even if that doesn’t get an economists blood pumping like a regression. We need more tracking of the adoption of AI in developing countries, the costs of AI in different settings, and the performance of AI in different languages.

We need to immediately start thinking about jobs in developing countries. We spoke in Ideas in Development about our fears that patient zero for AI induced job loss will be those reliant on Fiverr-style, or call centre-type, basic, repetitive service jobs exported from countries like Pakistan and the Philippines to wealthier economies. This is a worry based on best guesses, so we need economists tracking this, for example by matching adoption of AI in high-income countries to their outsourcing of different tasks and business processes to countries with cheaper labour.

This links to a broader question, what is the bottom rung of the development ladder in 2026, 2030, 2035? Matt Yglesias had an interesting piece in the Argument Magazine, which I broadly agreed with, exploring the future of garment factories – as automation gets cheaper and better, and we need less human labour in those types of jobs, where is the ladder?

AI may be a way off impacting manufacturing, but, as discussed above, basic service exports seem significantly and imminently more vulnerable. Thinking forward five years, or ten years, or whenever we have cheap AI models replacing basic service jobs as well as cheap robots being deployed at scale in factories, what industries will still provide employment as well as help on the path to development, particularly given other worrying trends like rising protectionism and increasing conflict across the world.

And for those people who lose their jobs, what policies make sense. I liked Deena’s thread which highlights why this is an area where development economists are needed. Social protection in countries like the US might be strong enough to act as a buffer to AI-induced unemployment. They could even be beefed up a bit by redistributing some of the enormous profits that AI companies bring into the American economy. But for the countries that capture none of this value, and still lose jobs, how do they expand social protection, which already allows many to fall through the cracks.

What have economists done that can help us think about these future AGI scenarios? Does it require a new paradigm? What happens to the existing human population, when the total labour population rapidly expands beyond it in number and skill. Do we have any examples of a group that becomes outnumbered by a far smarter and economically valuable population?

Are power dynamics set to become even more unequal? How does a country approach a negotiation on critical minerals exports with a superpower which could always overpower them militarily, if at great cost and illegal under international law, that now has access to AI models/tools that could easily turn off their power grid, rig their next election, paralyse their institutions, and precisely assassinate key leaders, for very little cost in a world without international law. (From the perspective of the developing world, I don’t trust any superpower with AGI, although a least-bad option may present itself.)

As you can probably tell by the tone of my questions, I’m not exactly feeling optimistic. There is certainly huge promise across a range of applications, like health and education, but we are not thinking enough about the economic risks.

I get it, these are uncomfortable questions. I imagine that simply by asking them I’ve put some people off, as they seem ridiculous, ludicrous, sci-fi. But any serious attempts to answer them are valuable. Here’s hoping I’m just a crank.

After outlining the current trends I see as most important (Google search, social media, people over organisations), I expand on the following principles for effectively communicating science. I see these as keys to still being relevant in 2030 (comments and thoughts welcome!):

• Curation, curation, curation

• You’re only as big as your biggest influencer

• Direct contact

• Be useful everywhere, all at once

• ‘Living’ content is king

• Focus on what is ‘AI-proof’

And after going into these points, I’ve summarised some of the things I am still unsure about, plus why I’d invest in this area if I had the funds. To finish, I think about whether any of this advice will still hold up in 2040, and link to some related interesting reading.

The purpose of this long, meandering blog is to put all of my thoughts in one place. I mostly refer to the science I know best, which is the ‘dismal’ one. But I think these lessons apply regardless of the specific subject or topic you are trying to communicate to a wider audience.

Communications does not feel like the main focus of many of the bigger-picture debates on how AI will change the world. I think it is an underrated one (e.g. check out this scary stuff on ‘malicious AI swarms’), that is compounded by other current trends.

For me, it feels like an exciting time to be a nimble, driven organisation in the business of communicating science, at least for the next five years or so. My hope is that this period will be a much-needed shake-up to the traditional ways research reaches a wider audience. One that’s less formulaic, less about playing the game, more focused on what people enjoy consuming, and ultimately better quality.

Current trends: Where we find ourselves

The traditional players: Who is in the communicating science business

Where I work, at VoxDev, our goal is to communicate the policy implications of the best economic research in low- and middle-income countries in an accessible way. There is a whole ecosystem of organisations that I would put into the same broad category of communicating science, that includes big players like Our World in Data, The Conversation, and TED. Importantly, there are also individuals who do this themselves, through their own blogs, podcasts and videos – more on this later. And then there are many organisations which bridge research and communications, with in-house researchers and communications teams that seek to amplify their work.

Of course, there are also the publishers, including both academic journals and journalists. I imagine these two groups both think of themselves as the most important part of this ecosystem. That has probably been true for a long time, but I am not sure it will continue to be. Many of their business models no longer make sense, and I think the importance of these organisations will continue to diminish. I am sure some will innovate and stay relevant, but the writing is on the wall for a lot of publishers. So, there is a lot of churn coming (I expand on these rather grandiose claims throughout the rest of this blog).

RIP ‘Google it’ 1998 – 2026.

Google search had a good run. But it is not long for this world. The web as we knew it will quickly disappear. And to be honest, I’m looking forward to it.

Google search was an incredible invention that made research more accessible and communicating science easier. It also incentivised bad writing and was still, after almost 30 years, remarkably useless for users in many parts of the world.

In the age of Google search there was a relatively standard model for doing well. Step 1, be a big organisation with a website that Google trusted. Step 2, rest assured that your content will stay at the top of Google search results and reach a much wider audience.

Nowadays, people increasingly ask their GenAI chatbot of choice, rather than going to Google to search. And even if they do Google it, they have to turn off AI mode and then scroll past the AI overview.

(I wanted to leave this next part in, but I also appreciate it is quite boring, so feel free to skip to the less niche stuff afterwards.)

For the Google Analytics-uninitiated, here’s an illustration of what I mean based on data from the VoxDev website.

The graph below shows the click-through rate to the VoxDev website last year. This is calculated by dividing clicks by impressions, which is the number of times a VoxDev link was shown after someone searched on Google (this now includes those who see your link in an AI overview).

Not so long ago, VoxDev would reliably get 1 click every 50 times we showed up in a search result. Now it’s closer to 1 click every 200 times. And this is a common story; the click-through rates for a whole host of websites are falling through the floor.

If you thought I sounded dismissive about the chances of publishers, I think this demonstrates why many of their business models are just not sustainable. After the good old days of selling print, the migration to the web left many publishers dependent on clicks, so they could sell ads. Even this model was not exactly a winner, but it kept the lights on at a lot of places. Now, the clicks they depended on are vanishing.

Again, this is not necessarily an issue for users, as GenAI should find higher quality content that’s closer to what you were after. But it poses a new challenge to those trying to communicate science to the world and track their reach.

The barriers to communicating science are falling rapidly

The death of Google search as we know it is one part of a trend that’s seen the barriers to communicating science fall dramatically.

Social media certainly played a big role in democratising this process. But since Twitter’s heyday, users, particularly in the research community, have become much more dispersed across different platforms.

Then there are new platforms, namely Substack, that make it incredibly easy to start writing yourself, particularly if you already have a big audience who will follow you there.

So far, I think all of the trends I’ve described make this a much more competitive space to be in, which is great for the public. It has never been easier to find and read interesting and accessible content about scientific research, often without even needing to pay for a subscription.

In the long run, as I discuss at the end, I imagine AI will do most of the communication about science itself. So, I see the next five years of churn and competition as a passing moment, but nevertheless one with very high potential for growth and ultimately impact.

Share

Still being relevant in 2030

Curation is king

AI is going to have a big impact on the amount of science getting churned out. There will be a lot more of everything: high-quality research, obvious slop, papers that look good at first glance but turn out to be rubbish upon further reading. An appropriate response will be to distrust everything, unless you have a signal that it’s worth your time.

There has been a lot written about the worries of ‘slop’ in particular. And sure, right now I can imagine journals are getting inundated with crappy, fully AI-generated submissions. But I think the main issue in the medium-to-long term, which is a much better problem to have, is what do we do when there is so much good science to communicate?

I’m confident that as a result of AI, we are going to see so much more high-quality research that has important implications for policy, helps us understand the world, and is ultimately worth sharing.

It’s certainly an interesting problem to solve. And focusing on curating the best research will be absolutely crucial.

Academic journals currently do this by giving relatively reliable signals about the quality of academic work. But they are already falling behind, particularly in economics, where it can take many years for papers to be published. Our experience at VoxDev has been that the key, useful insights from research are already relatively set long before that final published version of a paper comes out.

And this is already the situation now. Imagine the delays when we start to see an even bigger, and sustained, increase in submissions that pass journals’ quality thresholds.

Academics will have to figure out an entirely new system of publishing, but that won’t happen overnight. In the meantime, the public needs reliable signals of what’s good, and what’s gibberish. So those that establish a reputation of consistently curating research that’s worth reading and of high quality, without waiting for publication, will have the upper hand. Particularly if they also help to piece together this wave of knowledge, by summarising takeaways across the surge of papers.

We are all influencers now

It seems to me that people are tired of hearing what an organisation thinks, and would rather hear directly from people that they trust on specific topics they are interested in.

Influencer is often used as a pejorative. I see it more as a strategy than an indication of the value of people’s opinions. A mixture of the growth of blogging, and societies’ changing tastes for consuming information, mean that we have seen a new wave of ‘influencers’, who communicate their science in an accessible way, and have built large followings of their own. From those I personally follow, this includes people like Max Roser, Hannah Ritchie, Saloni Dattani, Sam Bowman, Ken Opalo, Richard Baldwin and Paul Johnson.

This new state of affairs is somewhat of a conundrum for organisations themselves. There are two answers, (1) hiring people with an established personal following; or (2) empowering your own staff to write, or talk, whilst helping to market them a bit. So, communications teams should move from doing the communicating, to supporting researchers to discuss their work themselves and build a following. That could involve encouraging them to write, providing training on communication skills, making sure they have time set aside for this, thinking through which parts of their work might be most interesting to share, etc.

Direct contact

The platforms we have come to rely on are desperate for users’ attention. That means they are heavy on the algorithms, and allergic to links taking users away from the platform.

Your followers on social media, or subscribers on YouTube, are increasingly likely to miss your latest release. Long gone are the days of, by default, chronological feeds from only the people you follow. This makes comms harder to predict. Sure, this can work to your benefit when one article blows up on Twitter, or a video gets going on YouTube. But that is hard to do, and if you care about consistency rather than occasionally going viral, it is not a reliable strategy anymore.

The response to this is a relatively simple one. Being able to contact people directly is increasingly vital. Email is the traditional way, so focus on those mailing lists. WhatsApp is also interesting, particularly for parts of the Global South where it is far more widely used than email. For example, The Continent is an African newspaper distributed through WhatsApp that seems to be nailing it.

Be useful everywhere, all at once

The standard model: Rank well on Google, tweets that go viral

The new model: Be everywhere, and be useful wherever people end up finding you

A point that Charlie Giattino made to me about OWID’s approach is I think particularly instructive - to be useful wherever people find you.

I have sometimes felt social media ‘engagement’ and ‘reach’ are hollow stats, but it really depends on what you are putting out there. If your post contains a really useful graph (like OWID’s do), or summarises a key new finding about the world, then a high level of reach means something completely different to the typical organisation’s robotic post with a link back to their website.

The temptation is to draw people to click on your link. But the most effective strategy is a short engaging post that leaves the people who read it smarter, even if they do keep scrolling.

Flexible

AI is going to try to replicate a lot of your favourite research and find some serious holes.

This is undoubtedly a good thing for science, and society. And organisations should embrace it rather than being slow to react. Even if you are left with some egg on your face, additional admin, or having to send an awkward email to a senior researcher whose famous thesis depended on a coding error (honest or otherwise).

Journals have never dealt with this particularly well, and when they do it’s an extremely slow process.

But in this new age, content should be ‘living’ anyways – by ‘living’ content, I mean something between Wikipedia and an academic paper. This means having a process in place to ensure timely updates, particularly with synthesis work. And organisations should be quick to react to replication failures, to clearly and obviously correct past mistakes.

This is why we believe the VoxDevLit model we have at VoxDev, where our reviews are updated as the literature on a specific topic evolves, should be replicated more widely across the sciences.

AI-proof content

There are two parts to being ‘AI-proof’. The first involves content that AI can make as well as a talented human – this is not a simple binary, as increasingly, we will all use AI in our writing processes. The second involves our preferences for human- over AI-generated content, the strength of that preference, and whether that preference endures.

Thinking about the current capabilities of AI, there is a fair amount of content that I still see as AI-proof over the next few years. Certain types of summaries will be replaced by AI in no time, but here is what I think AI won’t be able to do that well in the short term. In a sentence, it boils down to identifying the content people want and producing that to a high quality, often through multimedia. For example:

• Well-researched, high-quality, longer podcasts

• 10 minute+, documentary-style video, with real people on camera

• High-quality, long-form writing

• Beautiful, accurate data visualisation

Video seems particularly important, given the importance of YouTube and the current moves of many platforms to prioritise video-based content, including Spotify, Substack, and even LinkedIn.

It’s also useful to think in terms of what has not been written down before and therefore is not included in GenAI’s training sets. The internet feels incomprehensibly huge. But when you think about the subset of the internet that’s actually useful and open access, it starts to feel a lot smaller.

In academic research for example, there is a lot of interesting stuff that doesn’t make it into papers. Whether that’s descriptive statistics or claims that a researcher might come to believe during the research process, that they would share over a drink but might not be ‘rigorous’ enough for the actual publication. There is also a lot of tacit industry knowledge that has never been written down publicly, particularly on the messy processes of ‘getting stuff done’ or moving from research to implementing in the real world.

If your only value is making research accessible, this will be greatly diminished when AI puts evidence at everyone’s fingertips. But that doesn’t mean that people will magically start using evidence, trusting AI, or even preferring AI.

And this is one area I’d love to see research on.

How do people treat AI summaries vs human-written summaries? Do they enjoy one over the other? Trust one over the other? Is it actually about the content being different, or their mindset shift when reading people vs AI outputs? If the same text is presented as AI-generated, vs having a human author, do people care?

What I have no clue about

Polarised times

Academic research is far too slow to react to news cycles. Yet existing papers gain the most traction when a related issue is in the news and salient to a wide audience. So, do you go all in on targeting comms in response to the news cycle? Or do you operate above the day-to-day, and hope that others organically pick up your resources when they suddenly seem more relevant.

The answer is more obvious for some. If you’re a think tank that wants to have policy impact and go viral, then getting the timing right is crucial, but that draws you into constantly responding to the news cycle. I think traditionally, that has been a tightrope that certain organisations have been able to walk. But wading into the topic of the day seems harder and harder to pull off whilst maintaining a reputation for non-partisan scientific rigour.

Tracking reach and impact will be hard

To me, the whole point of communicating research is to get it out to a wider (or target) audience, and hopefully help people make more informed decisions, whether they are politicians, other scientists, or the general public.

Traditionally, tracking the reach of your communications has not been too challenging; you can still get relatively reliable data about your website and social media accounts, and track mentions across the entire web if you were happy to pay for certain software. But data from Google analytics is missing more and more users, whilst genAI is, for now, a complete black box, and social media is getting more fragmented in a way that tracking across all 10+ platforms you are on is more time-consuming.

Of course, reach is not always an end in itself, and tracking impact has been, and will continue to be, tough. It’s one thing getting your target audience to engage with your content, it’s a whole other kettle of fish figuring out whether that actually impacted decision making.

But now even understanding your actual reach will be hard. Will OpenAI ever tell you how often your research has been used for ChatGPT’s responses? I doubt it, at least for now.

Leave a comment

People should invest in communications

There is lots of potential to have an outsized impact on the ideas which shape the future.

While funding from traditional government sources is dwindling, there are more billionaires than ever with cash to splash.

“All of Anthropic’s co-founders have pledged to donate 80% of our wealth, and Anthropic’s staff have individually pledged to donate company shares worth billions at current prices—donations that the company has committed to matching.” Dario Amodei in his most recent essay.

If I were a billionaire right now looking to spend my money well, I’d be looking at the explosion of high-quality research we will see in the coming years (Biologists, Chemists, Physicists et al. forthcoming). Can we create a platform that reviews and features what is most important quickly, accurately, and engagingly? That synthesises across studies to find common lessons and recommendations? Can high-profile thinkers with the necessary expertise and an established following of their own be enticed to lead this type of project?

What does that mean in practice?

Creating more Works in Progress-style platforms, more platforms like VoxDev (naturally), and allocating more funding for existing platforms to expand the work they are doing.

I also think that now is the time to be bold, and experiment with new forms of academic journals. I’d personally love to see a new Economics of AI Journal, in between an NBER/CEPR working paper and final publication, to make up for the current failure of publishers to keep up with this fast-moving technology in a timely fashion, and provide accessible findings to the public while they are still relevant.

Communicating science in 2040: My best guess

By 2040, at which point it sounds like we will have reached AGI (Artificial General Intelligence), it’s hard to imagine that much of the above will hold.

There will be very little that a human can write, or say, that AI is not also able to. But that doesn’t mean that nothing will be ‘AI-proof’.

The preference we have for people over AI, whether that’s reading a person’s writing, listening to their thoughts, or watching them talk, will likely endure. Computers have beaten the best human chess players for decades, but no-one watches computers play one another.

So while I can’t see there being ‘need’ for an organisation or person to ‘communicate science’, as all science will be at everyone’s fingertips, filtered through an AI that knows their history of consuming content, and has inferred their preferences for subjects, content style, etc., that’s not to say these organisations won’t be crucial until we get there. And it’s also not to say that there won’t be space for organisations, no doubt using AI, to communicate through their people’s voices.

Ultimately, this feels like a positive to me, and solves many of the problems people in the space have spent years trying to eliminate.

Five other related + interesting readings

• Saloni Dattani’s guide to data visualisation.

• The Age of Academic Slop is Upon Us by Seva Gunitsky

• The Fight For Slow And Boring Research by Jolie Gan: “As federal research funding shrinks, scientists are looking to other sources of support. Can they learn to sell their work without selling out?”

• The next frontier for public access: building channels of meaning by Meagan Phelan

• What is Think Tank impact? (And what is it not?) by Todd Moss

Subscribe now