It’s easy, however, to throw stones. I wrote the article not to finger point at organisations doing things ‘wrong’ but because it’s important to diagnose a problem before setting out to fix it. Having done the diagnosis then, in this article, I want to be more constructive and offer a path forward.

None of what follows is just my opinion

Everything I say in this article is based on scientific research, validated by my own extensive experience working with boards and C-suite leaders, observing first hand what works, and what doesn’t, when it comes to enterprise-wide adoption of AI at scale.

A few things to note before we go further. First, I’m primarily interested in organisation-wide transformation, not isolated AI use cases. The latter are much easier to get right, and a good use case can make a positive impact, but, for medium and large organisations, at least, they are often looking to make more systemic improvements, and this only comes from strategic transformation across the org. 2025 was the year of the ‘AI pilot’: businesses invested in enterprise generative AI tools and corporate training on how to use those tools, and perhaps tried out a few AI experiments, but they weren’t thinking enterprise-wide. That changed in 2026. Now, the number one question I am asked is: how can I scale from a few good AI pilots to organisation-wide benefit?

Second, the barriers to enterprise-wide adoption are not technical in nature, but sit firmly in the human realm. According to research at Stanford’s Digital Economy Lab, 77% of the hardest challenges when adopting AI at scale are non-technical: change management, data quality, and process redesign. This shouldn’t come as a surprise. While AI tech is amazing, getting value in a business is a transformation activity, and if we learnt anything from the history of digital transformations, is that it’s complex when people are involved.

Third, and fortunately for this article, there is a wonderful emerging body of research that has studied companies that succeed with AI and those that don’t. My absolute favourite resource is Stanford’s Enterprise AI Playbook, which studied 51 successful adoptions and uncovered what factors led to success or failure. It’s the inspiration (along with other research) on which my own 14-dimensions framework below is based. There’s other good research out there too. AI has been around long enough now that we are starting to understand how adoption actually plays out in practice.

Three categories for success

If you unpick everything we know from research and practice about enterprise AI, there are three key factors that determine the outcome: leadership & vision, people & culture, and governance & trust.

Leadership & vision – this is about clearly articulating your business’s AI ambition. It’s a step that, surprisingly, is often forgotten. But if you don’t have a clear, crisp statement on what you are trying to achieve with AI, your leaders won’t be aligned, and there’ll be plenty of wheel-spinning to follow. I like to use a simple framework: you can be an AI Watcher (limited investment, aim to fast-follow), an AI Adaptor (real investment but in optimising the status quo), or an AI Disruptor (become AI-first or AI-native, rewrite your operating model). Not every business needs to disrupt. But every business needs to make a deliberate choice which category it currently sits within, and which it wants to aim for. Your choice here will affect every other decision you make: who to put in charge of AI, how to structure your organisation, how to execute. Miss this step and you are unlikely to succeed.

I also like to tell my clients nowadays that every leader is an AI leader. Unlike previous digital transformations, you can’t just delegate AI transformation to the tech team (unless you are in the Watcher category). AI is pervasive and whatever function you lead, your staff will be using and experimenting with AI and, frankly, looking for a steer from you. If you lead HR, finance, operations, whatever: you are an AI leader, whether you like it or not.

People and Culture – the biggest challenge in any transformation is, of course, change management. Brynjolfsson's research on digital transformation (originally with Hitt and Yang, 2002) found that $1 of computer hardware investment was associated with more than $10 in additional market value – implying at least $9 of unmeasured ‘intangible’ investment in things like process redesign, reskilling, and organisational change. But organisations rarely budget for these intangibles.

AI makes things even more complicated. Emotions run high when it comes to AI, meaning there’s a lot more work to do in bringing your customers, staff and other stakeholders along for the ride. Change is hard. Digital change is even harder. And AI change is wicked. Many have entrenched views about AI – positive or negative, and frequently extreme and misguided – and these views are difficult to overturn.

Governance and trust – CTOs aren’t getting much sleep nowadays. They are under pressure to adopt agentic AI and capture all the benefits that agents can bring. But CTOs know that there are very real security risks that come with the territory. I often say, however, that the challenge with agents isn’t that they bring new security problems; rather, they expose the ones you already have. Agents are efficient at searching through file systems and databases, and if those databases haven’t been designed with agents in mind, they are likely riddled with security flaws that a human wouldn’t come across easily, but an agent will encounter faster than you can say ‘red alert’. Trust is paramount too. Paradoxically, in developed countries like Australia, trust in AI is low. A 2025 global study by the University of Melbourne and KPMG found Australians report the lowest belief of any country surveyed that AI’s benefits outweigh its risks.

14 Dimensions of AI Adoption

According to the research, there are 14 dimensions that will determine enterprise AI success or failure. These fall into the three categories above: leadership & vision, people & culture, and governance & trust. Note how few of the 14 have anything to do with technology at all, let alone AI.

Let me briefly describe these one by one. (I developed these in my consultancy company, Goldilocks AI, and I would, of course, be happy to tell you much more, including how to apply them in your organisation: just reach out on jon@goldilocksai.com.au).

• Investment and hidden costs. Most leaders only think about the cost of the tech: AI tool licenses, data storage, cloud compute etc. But real costs of AI dwarf the tech costs. Cost of tokens is a case in point. Yes, it’s important to control use of tokens – and this has suddenly become a major issue for CFOs – but the real costs are hidden and must be accounted for. I have written at length about AI economics; see this article for further insights.

• Time to value and methodology. A waterfall method doesn’t work with AI. Forget trying to plan out a 12 month roadmap. Instead, work in increments and iterate as you learn what works and what doesn’t. Your first increment should tackle the hardest part of the problem, the bit you’re not sure you can solve with AI. That’s counterintuitive. People normally do the easy bit first, in search of a ‘quick win’. But if you defer the hard bit, all you’re doing is wasting time on easy stuff, only to find later that AI couldn’t solve the problem anyway. Also, embrace failure. The Stanford research showed that 61% of AI implementations failed before they succeeded.

• Executive sponsorship. I can’t emphasise this one enough. If you don’t have strong executive leadership, then forget it. AI success needs an exec who is invested in the outcome, will meet weekly with the implementation team, and sees his/her role as the go-to person to unlock organisational barriers when needed. A sponsor who sits back and expects to just approve things when asked is beyond useless.

• Beyond cost: revenue and new capability. Most companies see AI as solely an efficiency driver. That’s understandable. We’ve all been told, after all, that AI is about productivity improvements. But that’s only part of the story. If all you do is focus on efficiency, you’ll never gain competitive advantage, because your peers are all using AI for efficiency too. Successful businesses go beyond efficiency and think about AI that can open up new revenue streams or even new business models.

• Scaling from pilots to enterprise. One of the biggest mistakes I see. Companies invest heavily in running pilots but don’t have a process in place to scale a pilot when it works. As a result, good things happen, and then slowly sleek off into a dark corner to die. Everyone goes back to business-as-usual. Before doing any serious piloting, define a process with investment stage-gates so you can support pilots to scale as they show promise.

• Human-in-the-loop design. It’s a myth that every AI decision needs human oversight. We are told that since AI hallucinates, a human must be in the loop at all times. But if that’s what you do, you’ll just make that poor human’s job boring. S/he will go from being a creative problem solver to merely an AI checker. You don’t need human-in-the loop everywhere. Take a risk-based approach: for low-risk tasks, just let the AI get on with it. Reserve humanity for high-risk decisions where you absolutely need it in the loop. According to the Stanford research, businesses that let AI handle the bulk of a task autonomously, with humans only reviewing exceptions, saw a median productivity gain of 71%, more than double the 30% gain from systems requiring full human approval on every output.

• Resistance and change management. People respond to change differently. Some will embrace it; many will not. AI raises the stakes, because it’s become such an emotional issue. Many views on AI are simply not based on fact, but misguided perceptions. It’s more important than ever then to map out pockets of resistance in your organisation, and have a plan to support those people and bring them along for the journey.

• Productivity to headcount decisions. Research shows that AI is definitely improving personal productivity. But it isn’t (yet) impacting organisational productivity. One reason for this is that organisations don’t plan out how to use saved time. As a result, an employee saves time but that time gets eaten up with organisational noise: more pointless meetings, more useless documents. Unfortunately, this means that, contrary to the promises, if we save two hours a week using AI, we are not freed up for more creative tasks or to go to the beach on a Friday afternoon. Businesses that succeed with AI plan for how to use saved time.

• Tribal knowledge capture. AI can only work with what’s documented. But most knowledge in an organisation is tacit. It lives in the heads of experts, and has never been written down. So if you want to access the full potential of AI, you need to capture tacit knowledge. And to do this, you need to bring subject matter experts in from the beginning. Get them to champion AI. Resist the urge to think: “I can just apply AI to this big dataset and then we won’t need experts.” That’s a sure fire way to piss off your experts who will then become the resistance (see point above).

• Agentic AI readiness. Making the transition from AI-as-chatbot to AI-as-agent, or AI that can actually take actions, is hard. And it requires you to be ready. Processes were designed for humans not AI and so it’s more than likely that there’ll be a lot of process redesign work needed. My top tip? Bring your AI and process teams together. They need to work hand-in-glove.

• Data foundations. I think everyone, by now, understands that data is king when it comes to AI. Garbage in, garbage out, and all that. But a common misconception is that your data needs to be perfect before you can start with AI. Research shows this is not true. Only 6% of successful AI rollouts had fully ready data before they started. 59% succeeded with scattered, not centralised data. If data work is necessary, proceed with two parallel streams: one for data foundations, one for applying AI. AI can even help you with the data work.

• Security as infrastructure. Security is one of the biggest barriers to AI transformation right now. Organisations are simply not set up for unleashing AI agents, regardless of the potential upside. The risks are just too great. Successful companies spend big on getting their security infrastructure right. And this is one of those cases, where you do need to spend the time on the foundations first before even thinking about AI.

• Model strategy and orchestration. Or, the right model for the right job. You don’t need the best frontier AI model to summarise an email. Successful AI adoption efforts have multiple models available: cheap, fast models for routine work, and the more expensive stuff for when it’s needed.

• Agentic governance. AI, and agents in particular, bring new risks. But resist the urge to spin up a whole new AI governance infrastructure. Consultants will tell you it’s needed, and happily take your hard-earned dollars. But, for the most part, you don’t need a special AI governance framework; if it’s good, your existing risk management procedures are sufficient. Just review them to ensure AI risks are included, and only add new artefacts where necessary.

Where to from here?

The 14 dimensions give you a checklist of things to focus on when leading AI transformation. Each one will need to be contextualised with your organisation’s culture, ambitions and ways of working. It’s hard work. But it can be fun.

If you want a snapshot of how you’re doing currently, I have developed an extensive AI diagnostic that will score your organisation on each of the 14 dimensions. I use this with clients to help them understand their strengths and weaknesses and come up with an action plan.

The full version is available through Goldilocks AI, but, as a quick taster, try out a much simplified version at the link below. And let me know if you’d like me to come and take you through things in full!

I’ve worked with or advised hundreds of companies looking to adopt, or even transform with AI. And I’m afraid to say that I’ve come to a startling conclusion: most are doing AI wrong. I know that sounds harsh, but sometimes you have to call things out.

AI promises enormous productivity benefits to organisations but all the evidence points to the fact that businesses are not seeing personal AI productivity improvements translate firm-wide. I’ve been seeing this myself for some time now, and write about it in my book, AI for Business: A Guide to AI Adoption. And a new report, that I came across this week, confirms my suspicions. According to the International Labor Organization, at the individual level, generative AI can bring productivity improvements of 10-70%, depending on the task to which AI is applied. But at the firm-level, evidence for productivity gains is mixed. While some larger, digitally mature organisations are finding productivity benefits from AI, most report little measurable impact beyond pilots.

I wish it were different. I really do. AI has so much to offer business. But current practice suggests business leaders have little clue how to unlock this value.

Where it goes wrong

I have a categorisation I use with clients to help articulate what an organisation’s AI ambition is. I tell them, you can be an AI Watcher, AI Adaptor, or AI Disruptor:

• AI Watchers make only incremental investments in AI, perhaps rolling out enterprise generative AI licenses and putting on some corporate training and a genAI policy or two, but not much beyond this. Employees are mostly left to themselves to get value out of AI.

• AI Adaptors do make significant investments but the focus is on optimising things the company is already doing: making processes more efficient, reducing a few FTE here and there, trying to speed things up, but not always the right things. There isn’t a strong strategic vision for how AI can make the company different.

• AI Disruptors are using AI to rethink their business. They are changing their business model, opening up new revenue streams, and fundamentally reframing how they operate. In short, they have a clear vision for what they want to achieve with AI, and they are going after it.

In all of my engagements, I’ve found that only 5% of organisations are AI Disruptors. That’s not necessarily a bad thing. Not everyone needs to be a disruptor. But it’s worth reflecting on the key difference between disruptors and the other two categories. Disruptors have taken the time to clearly articulate their AI ambition. They have painted a vision of an AI-enabled future. Adaptors and watchers, on the other hand, can walk into AI without first being crystal clear what they are trying to achieve. They may end up tinkering and, frankly, wasting a lot of time and effort trying AI out everywhere without a strong conviction of what AI is intended to change.

In many ways, it’s hard to blame the organisations. Generative AI is still a new technology, after all, and it takes time to figure out how it can transform a business. You can’t expect things to change overnight.

We’ve also been told that being strategic about AI isn’t the way to go. It’s a new technology, says the dominant narrative. You don’t need a strategy because AI changes so quickly anyway. Instead, just encourage your employees to experiment. Workers know their jobs the best; therefore, they know best how to apply AI.

The experimentation approach to AI isn’t necessarily a bad thing. We’re all still figuring out how best to apply AI so experimentation is essential, and should be encouraged. But experimentation alone won’t get you where you want to be. It needs to be coupled with a strong strategic vision for AI, and that’s the piece that I often see missing in businesses that I work with.

How this plays out

Here is a very common scenario of how this all plays out in practice. Let’s suppose you are an iconic national brand. Your business is doing well, but like all businesses, you have some challenges. Costs have bloated over the years, revenue could be higher, competition is strong, and there have been multiple management changes in recent years. All of your competitors are adopting AI, and so you should too. Perhaps the board believes AI can tackle that cost bloat. Or your CEO has just come back from a jolly in Silicon Valley and thinks AI is the answer to everything.

As a result, there is a suddenly a lot of AI interest within your organisation. Enterprise licenses – probably Microsoft Copilot – are being rolled out. Consultants are brought in to train staff how to write a good prompt. Working groups are set up to discuss how to apply AI in your business. And hundreds of AI use cases are collected, with very little thought as to whether they are the right use cases.

All of this leads to a lot of AI activity, but very little measurable impact. Indeed, all of this AI noise starts to slow your organisation down, not speed it up. Suddenly, all those promises of AI-supercharged productivity fade away and it all starts to feel like wading through a swimming pool full of AI treacle. There are individuals in the organisation who feel like they are operating on speed. But those individual gains just get lost in the organisational AI noise that is created.

Personal vs Firm-wide AI Adoption

A fundamental misunderstanding is the difference between personal and organisational AI adoption. Personal adoption is the use of generative AI tools by employees in day to day work. It’s what leads to investment in enterprise genAI tools and corporate “show me something cool with genAI” training programs. Often called “AI fluency”, these programs emphasise practical genAI skills, usually driven by a belief that if employees, particularly non-technical ones, just do something hands-on, learn to build a ridiculously simple AI “agent”, then they will suddenly and magically understand how to lead AI across an enterprise. What happens instead is that participants get wowed by the magic of AI in a workshop, then go back to their day jobs and don’t really use AI beyond simple email writing and document summarisation. If the company is lucky, employees are a little bit faster at their jobs. If they are unlucky, staff will get too excited and start building pointless agents left right and centre, blowing out the company’s token budget at the same time.

There’s nothing wrong with training your staff how to use AI to improve personal productivity. But, let me be clear: if that’s all you do, your organisation will never improve, not materially. It’s a myth that you can transform an organisation one person at a time. Individual personal productivity gains do not aggregate into firm-wide gains. If every employee saves two hours a week, no-one is knocking off early on a Friday to go to the beach. No, that saved time just gets eaten up by bureaucracy: more pointless meetings, more questionable documents, and more wheel-spinning decision-making processes, which expand to fill the saved time.

Organisation-wide productivity gains do not come from simply adding up personal wins. It’s a lot more complex than that. Putting a faster engine in your car won’t improve your commute time if you need to pass over a one-lane bridge on the way to work. You’ll still be stuck in a traffic jam. Organisations are the same. Workers can get faster at individual tasks, but if the processes, structures and cultures that govern how the organisation runs don’t change, there will still be bottlenecks – and potentially more of them. This is why, if you are a power user of AI at work, you might feel like you have supercharged your own job but your organisation as a whole is still moving at the same slow speed, or even slower.

What’s not working

A lot of things are not working when it comes to enterprise-wide AI adoption. But here are just three examples of what not to do:

• Just give everyone genAI tools. They’ll figure out a way to use them effectively. Workers know their jobs best, right? They’ll get more productive with AI and so will the business. NO – this is wrong. For one thing, most corporate workers who use AI are probably only using 10-20% of AI’s capabilities, so their efficiency gains are relatively small. More importantly, the complex system dynamics of organisations mean these gains easily get subsumed by dynamic system effects anyway. A scattergun approach won’t work.

• Leadership shouldn’t define AI use cases. Workers know their job the best. Let them define the use cases. NO – this is also wrong. You’ll end up with hundreds of “little” use cases that mostly won’t be strategic and offer only narrow operational fixes. That’s not to say you shouldn’t fix small problems. But it’s a losing battle. You could fix small problems forever and never fundamentally improve your organisation. It’s like maintaining a home. If I live in an ugly 70s brick house, I can do small handyman jobs all day long, but my home will still be an ugly 70s home. To change it, I need a full reno.

• AI is only about saving money. It can optimise, automate, reduce costs. Kerching! This one is partly wrong. You can certainly save money, and there’s nothing wrong with that. But just reducing costs won’t lead to a step change in your business. At least not by just saving a bit here and a bit there. It’s like a family budget that tries to save money by one less streaming subscription when what you really need is to reduce your interest rate on your mortgage.

Underlying all of these mistakes is a lack of strategic leadership. I see so many executives defer and delegate AI adoption to others instead of leaning in themselves. Perhaps it’s because they don’t feel AI literate enough. Perhaps it’s because AI is a technology, so it must be the tech team’s job. Or perhaps it’s because there are just too many other things going on. Who knows? Whatever it is, this lack of strategic responsiblity is “killing your gains”, as they say in the fitness industry.

Eighteen months ago, when I spoke with clients about the need for a clear strategic AI position within an organisation, no-one wanted to hear it. AI is moving too fast, they said. We don’t need an AI strategy because it will be out-of-date by the time it’s written.

Yes, it’s true that the technology is changing fast, faster than any of us can keep up with. But the fundamentals of how you improve a business with a new technology haven’t changed in decades. And this super-speed only goes to further evidence that delegating AI down doesn’t pay off. I’ve seen companies develop hundreds of little use cases, spending months on processes to collect them all, only to find that these little use cases are obsolete by the end of the process. Small, non-strategic AI use cases quickly become out of date; strategic, ‘big rock’ AI use cases don’t. And it’s only leaders that can define those ‘big rock’ AI use cases. Delegating to lower-level leaders results in small-minded thinking because those leaders aren’t thinking enterprise wide. Why should they? That’s not their job.

An Example

Let me illustrate with a hypothetical example, but one I have seen in reality multiple times. Imagine you’re the CEO of a mid-sized private hospital group – call it Bayview Health. Bayview has twelve hospitals, a few thousand clinical staff, and all the usual pressures: rising costs, a nursing shortage that never quite goes away, ageing infrastructure, and a board that just read an article on AI in healthcare and wants to know “what’s our plan?”

Option one: delegate AI, so you don’t have to worry about it.

You do what most organisations do. You appoint someone to own AI – maybe a Chief AI Officer, but more likely whoever already runs digital transformation – and ask them to go find AI opportunities. They run workshops across every department: radiology, nursing, admin, finance, HR. Six months later there’s a spreadsheet with 150 ideas on it. You prioritise the ones that are cheapest and fastest to ship, because that’s what gets funded first, and over the next eighteen months you build a few of them, let’s say:

• An AI scribe that listens in on consultations and drafts the clinical notes, saving doctors ten or fifteen minutes a patient.

• A radiology triage tool that flags the likely-urgent chest X-rays so radiologists look at those first.

• A patient portal chatbot that answers “where do I park” and “what do I bring to my appointment.”

• A rostering assistant that helps ward managers fill shift gaps faster.

• AI-driven medical coding, so coders review rather than start from scratch.

• A sepsis early-warning model watching vital sign trends on the wards.

• A tool that drafts discharge summaries for a clinician to check and sign off.

• A no-show prediction model so outpatient clinics know when it’s safe to overbook.

Every one of these AI use cases is genuinely useful. Doctors get some evenings back. Radiologists get to the urgent scans sooner. Nobody’s worse off. But eighteen months on, when the board asks whether any of this has changed how Bayview operates, or has shifted its cost base, or its position against other private groups down the road, the honest answer is: not much. Length of stay hasn’t moved. Theatre utilisation hasn’t moved. You haven’t opened a new service line or done anything a competitor couldn’t copy in a year. You’ve got lots of little pockets of AI efficiency sitting inside a hospital group that runs exactly the way it did three years ago. And half of these AI tools will be standard features in your EMR vendor’s next release anyway, because none of them were ever going to be a source of advantage. Everyone in the sector ends up using AI in exactly the same way.

Option two: own AI and be strategic.

Now imagine instead you spend real time, before any of that workshop activity happens, on a harder question: given what’s now possible, what should Bayview actually be in five years that it isn’t today? Not “where can we sprinkle some AI,” but “what’s our new position?”

Let’s say you end up with three high value AI use vases. Not hundreds; just three.

First, radiology stops being just a department you run for your own patients and becomes a service you sell. You build out an AI-augmented reporting capability strong enough that smaller regional hospitals and GP clinics who can’t justify their own radiologists start paying Bayview for reads. Your radiology team goes from a cost centre to a growth line, and the AI isn’t there to save your own radiologists fifteen minutes; it’s a new revenue line.

Second, you stop treating chronic disease patients (diabetics, heart failure, COPD etc.) as people who show up when something goes wrong, and instead build a proper predictive program around them: a model that flags deterioration weeks out rather than an alert that fires hours before a crisis. That’s not a feature bolted onto existing outpatient clinics, it’s a new care pathway, and it changes the conversation with insurers, who’ll pay differently for a provable drop in readmissions.

Third, you redesign a handful of clinical roles from scratch around the nursing shortage you know isn’t going away: a new senior triage-nurse role in emergency, say, with real decision-support tooling behind it and a pay band to match, rather than handing your existing nurses a chatbot and hoping it helps them cope.

None of these three AI use cases would have come out of a departmental workshop. A ward manager doesn’t have the mandate to redesign a role and a pay structure. A radiology registrar can’t decide to turn the department into a regional business. These are calls only someone looking at the whole organisation can make, and they require committing money and reputation to a small number of bets instead of funding 150 safe ones.

Eighteen months in, option two probably hasn’t saved as many individual hours as option one. After all, big bets take longer to land than a chatbot does. But long term, option two wins every time.

Adaptor or Disruptor?

Option one for Bayview is classic Adaptor territory. You invest, but the focus is on doing existing things slightly better. Option two instead disrupts. It fundamentally rethinks parts of Bayview’s business. Which is what good strategy is about.

As I said earlier, not every company wants, or needs to be a disruptor, but every company does need to be clear on what it wants to achieve with AI.

In the last couple of months, there’s been a shift in the questions companies are asking me about AI. Executives continue to ask me how to scale from AI pilots to enterprise-wide ROI. And questions about AI governance are still high on the agenda. But there’s a new question that is now equally as common: how much will AI cost to adopt in my enterprise, and how can I control the spend?

It’s as though businesses have finally understood that AI is not free. We’ve all been sold a story that for a small price (the cost of a few subscriptions), AI will supercharge productivity and allow organisations to reduce headcount. Only, things aren’t quite working out like that. Productivity gains, clearly present for personal productivity, aren’t yet manifesting organisation-wide. And, in many cases, instead of reducing headcount, AI is increasing it, as companies hire AI strategy and governance experts, AI engineers, and AI “enablement” managers at record levels. Organisations may even be hiring more non-AI staff due to AI. CSIRO researchers tracked more than 4,000 Australian firms between 2020 and 2023 and found that firms adopting AI went on to post 36% more non-AI job advertisements than firms that didn’t.

CFOs are also hearing scare-stories of out-of-control spending from AI agents. Uber exhausted its entire 2026 AI coding budget by April, with heavy users running up US$500–$2,000 each in monthly API costs. Microsoft revoked its developers’ Claude Code licences within months of enabling them, with some commentators claiming this was due to cost. Royal Bank of Canada saw its token usage jump 500% in six months. And the pattern is not confined to the largest firms: the 2026 State of AI Cost Governance report found that AI has eroded gross margins at four in five enterprises, with 81% unable to fully account for AI costs.

All of this is giving CFOs – and their CEOs – the heebie-geebies. If I was a CFO, the inability to properly estimate the cost of an AI rollout would be keeping me awake at night. Because this is the real problem: traditional methods for estimating IT spend don’t work in a world where agents are making autonomous decisions.

Tokenomics

All the attention when it comes to cost-estimation for AI seems to be focused on the cost of tokens. A token is the unit an AI model reads and writes, roughly a fragment of a word. For example, “unbelievable” might be three tokens; a comma is one. As a rule of thumb, 750 words of English is about 1,000 tokens. Everything you send a model (its input), and everything it sends back (its output), is counted this way, and you are billed for both. Tokens are the currency of the AI revolution. The more tokens a business uses, the more expensive is the AI rollout. And, critically, estimating token usage across a large organisation is complex.

The cost of tokens has fallen dramatically in recent years. When GPT-4’s API launched in March 2023 it cost US$30 per million input tokens; by April 2026, Google’s Gemini 3.1 Flash was charging US$0.25. The decline is slowing, though: effective pricing across API services fell around 39% in the second half of 2025, but only about 6% in the first five months of 2026. The price varies, of course, by vendor. Chinese models tend to be cheaper on a per-token basis and are rapidly catching up to US frontier models in terms of capability.

But while cost per token has fallen dramatically, total token expenditure across an organisation is rising. In Q2 of 2026, enterprise spending on cloud infrastructure services jumped by more than $43 billion from the previous year to reach $143 billion, with generative AI cited as the primary driver. And Goldman Sachs projects global token consumption climbing roughly 24-fold by 2030.

This trend is a verson of Jevons paradox, a phenomenon first noted by William Jevons in 1865 when he observed that technological improvements that increased the efficiency of coal use led to increases in coal consumption. The same is true with tokens. While individual tokens are cheap, organisations are using more and more tokens as they roll-out tool licenses to all employees, encourage them to use AI as much as possible, and develop AI agents to automate decision-making. As a result, overall token costs are rising, and your CFO can’t sleep.

‘Tokenomics’ is the study of how tokens are generated, consumed, priced and controlled across an organisation’s AI systems, and it’s what obsessing the industry right now. The increase in token use is being driven by a couple of factors: both the linear growth in the number of AI users, each using more and more tokens for work tasks, but, more significantly, the non-linear growth in token use by AI agents. AI agents are not like standard calls to a large-language model (LLM). Agents, by design, are given a high-level goal and then set out to achieve it as best they can. This can involve lots of dead-ends, trial-and-error, backtracking, and planning, all of which burns tokens. In the worst case, the agent spends all those tokens and still fails, leaving you to foot the bill. Even in the best case of success, the cost can rapidly spiral upwards. Gartner estimates that agentic workflows consume five to thirty times more tokens per task than a standard chatbot exchange.

Let them eat cake

I’ll illustrate this trend with an everyday example. I have twin girls, who have just turned twelve. Every year, I bake them a red velvet cake for their birthday, and each year I decorate it differently. Over the years, I’ve produced some elaborate affairs: at two years old, I created a 3D Thomas the Tank Engine cake; a few years later, they were into roller skating and so I produced 3D cake skates; then it was Harry Potter, so we had a Hermione Granger cake, and finally, this year, they asked me for a cake to show images of them through the years.

To produce such a cake, I needed to create a photo collage from a portable drive containing all my old photos (17K of them!) and then pay an online service to print the photo collage on edible icing. The last part is easy – there are plenty of websites that will do this. The hard part was sorting through thousands of photos just to find the right ones. Surely, a job for AI, I thought? And so I tried a prompt:

“i want to produce an image for lana and kaiyas 12th birthday that will be printed on a cake wrap. the print area is 40cmx28.5cm printed on a single sheet. it will need to be cropped in half lengthways and then should cover a 9inch cake. I want you to access my portable drive to get photos of both lana and kaiya at different ages - suggest not as babies but maybe a range between 5 and 10 years old. put together in a collage image that can be easily split lengthways without cutting off heads! should be equal number of images of lana and kaiya. image needs to be at least 300dpi in resolution. jpg, gif, png, pdf.”

This was my single prompt in Claude. And amazingly, Claude did it in one try. The interesting part is how it did it. This is a complex task. It requires AI to access a portable drive, search efficiently through 17K photos, sort them into photos of Lana and photos of Kaiya, and then stitch them together in a very specific format and style. In other words, it requires an AI agent.

AI agents take high-level goals and then try to achieve that goal through a process of search, applying tools, and calling a LLM when needed. In this case, the high level goal is captured in my prompt. The agent used various photo editing tools and drive-search tools and, as you can see by the snippet below, there was a lot of trial-and-error as the agent realised it needed a more efficient way to search and filter through such a large photo database.

This is only a small part of the reasoning my cake agent went through (fully autonomously). It also realised it needed my help to correctly classify photos as Lana or Kaiya and so made a best guess but then presented some samples for me to label before producing the final result. This is a classic example of an AI agent working how it should.

As I write this, I eagerly await my edible icing shipment. Fingers crossed that it looks like the photo collage Claude produced for me!

The true cost of agents

The AI industry is encouraging businesses of all shapes and sizes to get on the AI agent bandwagon as quickly as possible. Few of the reports that encourage businesses to build and deploy agents considers the very significant costs that can arise – and, in some cases, dwarf any savings that might be achieved from adopting AI.

I was pleased, therefore, to recently come across a paper by Longju Bai and colleagues entitled, How do Agents Spend your Money? Whilst not a study of real-world use of agents, the paper rigorously evaluates the cost of deploying AI agents on a well-known benchmark, SWE-Bench-Verified, across eight different state-of-the-art frontier models from both US and Chinese tech labs.

The results are sobering.

• An agentic task consumes around a thousand times the tokens of a chat exchange.

  A chat exchange is one round trip: question in, answer out. An agent loops: it reads a file, decides what to do, makes a change, runs a test, reads the failure, tries again. Every step is a separate model call, and each call carries the entire accumulated history with it. Cost compounds with the number of steps rather than adding up linearly.

• The bill is driven by input tokens not output tokens.

  What a model writes is bounded – a few hundred words. What it reads is unbounded: system instructions, tool definitions, retrieved documents, file contents, error messages, and the whole prior conversation, replayed on every single step. Whilst vendors typically price output tokens higher than input tokens (because they have to be generated not just read), for agents, the input token cost dominates due to compound effects.

• Two runs of the same task can differ by 30 times in total tokens.

  An agent’s path isn’t fixed. A tiny difference early – which file it opens first, whether its first attempt happens to work, etc. – changes everything downstream. One run might find an answer in three steps; another might wander for forty, re-reading the same files and retrying. As a result, the exact same run of the exact same agent on exactly the same infrastructure doesn’t necessarily cost the same amount.

• Accuracy peaks at moderate spend, then plateaus. More tokens do not buy more quality.

  This one is counterintuitive until you realise what’s going on: high token use is usually a symptom of the agent being lost, not of it working harder on a hard problem. Successful runs tend to be efficient. Expensive runs typically thrash about, repeating failed approaches, re-reading context etc. So past a point, extra spend is evidence of failure.

• Models are not good at predicting their own token use. Their estimates and the true cost show 0.39 correlation at best.

  Human experts rate conceptual difficulty – how hard the problem is to think about. Token cost measures mechanical effort – how many files to open, how many attempts, how much context to carry. Those two don’t necessarily coincide: a conceptually trivial change spread across thirty files is easy to think about and expensive to execute. And models can’t predict their own usage because at the moment you ask, they haven’t yet discovered how tangled the task is. They also underestimate systematically, and plausibly, because their training data contains tidy finished solutions, not the messy search that produced them. The conclusion for a business case: you cannot obtain a cost estimate by asking an expert or by asking the model. You have to measure it practice.

• Model choice is a real lever.

  On identical tasks, Kimi-K2 and Claude Sonnet 4.5 consumed over 1.5 million more tokens on average than GPT-5. Models differ in how much they explore, how often they re-read, and how eagerly they retry. So when costing up different models, don’t just look at per-token pricing, as this can be misleading. A cheaper model that takes three times as many steps isn’t cheaper.

I don’t know about you, but I find these results quite startling.

Token cost is often dwarfed by other, non-technical costs

All the attention right now is on tokenomics. But, as I have been pointing out to students and clients for some time now, the real costs of AI adoption have less to do with the technology and more to do with the human and organisational side of actually making AI adoption work in practice and at scale: leadership and vision, people and culture, and governance and trust. This isn’t a new phenomenon to AI. Brynjolfsson famously estimated that for any given digital transformation, each dollar of technical outlay requires up to ten dollars spent on non-tangibles (training, change management, organisational redesign, etc.) There is no reason to believe this isn’t also true for AI – and every reason to believe that $10 could be a lower bound, given what I’ve said above about agents.

As I note in Chapter 6 of my book, AI for Business: A Guide to AI Adoption, costs of an AI deployment fall into eight categories, most of them non-technical.

Tokens – the subject of all the attention right now – sit under category one. For any given real-world deployment, you’ll probably find that token costs are dwarfed by the other costs. That’s not to say understanding token spend isn’t important. But if you only estimate token spend, you will have a nasty surprise when the bills come; it’s just that you might not notice it because the other spend categories may be hidden in budget lines elsewhere in your organisation.

So what should you do?

A reasonable conclusion of all of the above is that estimating the costs of an AI rollout is just too uncertain and we should all just give up and go home. But let’s not be quite so melodramatic. There are things you can do within your business to keep spending under control and ensure you have demonstrable ROI.

• Don’t believe any business case presented to you

  It amazes me how poor most business cases, even in mature corporates, still are today. If you see any numbers in a business case, especially if AI is in the mix, there’s a good chance it’s all finger-in-the-air stuff. Business case proponents are naturally optimistic. That’s just human nature. Someone proposing a new idea, and looking for sign-off, is emotionally invested in the idea and has a rosy day picture of how it’s all going to turn out. Insist, when business cases are presented to you, that there are a range of scenarios costed, from ‘yes, it all works swimmingly,’ to ‘shit, it all goes horribly wrong.’

• Experiment.

  Unless you are a magician or fortune-teller, you’re not going to be able to accurately estimate the cost of an AI use case. It’s simply not realistic. AI is fundamentally stochastic, so even if you could get a good handle on the initial deployment, it’s likely that things will evolve over time anyway. My advice then is to give your best estimate upfront but build in proper experimentation to assess the cost and value of AI in real-time. This requires proper rigour and discipline. A good example of this is a 2025 paper by Lu Fang and colleagues. This reports on a real-world e-commerce global company that deployed seven different AI use cases, each testing a different workflow, ranging from a sophisticated pre-sales chatbot, to recovery of credit card chargeback fees, to AI-generated product ads. The results ranged from no detectable effect at all to a 16.3% uplift in sales revenue. That’s within the same company, in the same period, built by the same team. What I like about this example is that the company conducted proper experiments, with millions of customers, comparing a control group (no AI) with a treatment group (new AI workflow), and were able to quantify the effect on overall sales revenue for that workflow. It’s an exemplar for what all businesses should be doing.

• Monitor and Track.

  A key characteristic about AI is that it isn’t static. That is, the model learns over time, which can be a good thing if performance improves, but models can also drift or degrade. Similarly, the costs may change over time, either because the vendors change their token prices or because the AI agents find new ways to unpick a problem. As a result, it’s not enough just to do experiments on an initial deployment. That will tell you a lot, and help you make initial investment decisions. But it won’t tell you if you should keep your AI use case going. At some point, the costs might change, even spin out of control. So you need rigorous monitoring of your IT systems more than ever before.

Back to cake

Over the years, I’ve found that making my children’s birthday cake isn’t cheap. There are the ingredients, of course, some of which can be quite expensive – for example, one year, I used multi-colour fondant icing to create a design, and each tub of fondant might cost $10. That adds up when you need six or seven colours. Then there’s the cost of my time, of course. And, let me tell you, I’ve spent days on some of these cakes, each time swearing I would never do it again, but then repeating it the very next year.

Now, it seems, I need to factor in another cost to my cake-making: the cost of AI. I love the fact that I can use AI to generate a photographic cake wrap. But it incurs a cost I would never have thought to factor in just a couple of years ago!

A frequent social media poster, one of Mollick’s patterns for his posts is to report on his tests of AI models’ ability to vibe-code a fully functioning computer game. Take this latest example:

Mollick generates a 3D game simulation from an 11-page prompt; the prompt itself is generated by AI, based on a conversation. At first glance, the result is impressive; it’s a high-quality immersive environment with lots of game-play elements. Seeing this, I was tempted to want to do the same myself. Could I legitimately produce a game from a single, albeit very long, prompt?

A few days earlier, Mollick gave another example: this time a playable game based on the popular 3D gaming engine, Unity:

Again, it’s impressive stuff. And the game designer in me wanted to give it a go.

My history with computer games

I have a long history with computer games. I discovered computers when I was thirteen years old. I convinced my parents to buy me a ZX Spectrum for Christmas. It was a classic machine. Cheap and cheerful, and with memory of just 16KB, it had a rubber keyboard glued on to the frame. After a few months of use, the glue literally came unstuck and you were left without a keyboard.

The Spectrum got me into gaming at a young age. My favourite was Manic Miner, a so-called “platform game”, where you controlled a “sprite” (in this case, Miner Willy), jumping him onto and over platforms, collecting treasure and avoiding nasties along the way. It was fiendishly addictive. I spent many hours working my way through multiple levels, only to face the famous “Game Over” screen, and have to start all over again. Or worse, my Spectrum would crash, and I would have to re-load the game via cassette deck and wait minutes for the game to become available again.

It wasn’t long before I started writing my own computer games instead of playing them. The following Christmas, I asked my parents for an assembly language manual so I could teach myself programming from first principles, although I quickly realised that using a game platform was a much easier way to make progress. I became an expert using Professional Adventure Writer (PAW), a game engine that allowed you to quickly create graphic adventure games in which the player used a very simple chatbot to direct gameplay.

Using PAW, I designed and developed a complex adventure game, called Set Up, in which the player was an intergalatic trader, framed by a rival for drugs trafficking, thrown into prison, and facing the challenge of escaping to clear his name. I even managed to sell Set Up to a games company, and made approximately $20 per month on royalties for a year before the company’s factory burnt down and it went out of business.

I had become a professional games designer at age fourteen. Unfortunately, though, I never managed to capitalise on this first success. In the almost forty years since, I have never produced another computer game.

Until now, that is.

Vibe-coding a game with AI

Inspired in part by Ethan Mollick’s post, I recently tried to vibe-code a computer game for AI governance. As I now teach AI leadership and organisational transformation, I’m interested in finding new and different ways of keeping students engaged. This is particularly needed for a dry subject like AI governance: it’s easy to bore students with a litany of global AI regulations, a plethora of AI governance frameworks, and spreadsheets full of AI risks and mitigations. Surely, there must be a better way I thought? Could I develop a simulation of a business that is facing tricky AI governance scenarios? How easy would it be to develop using AI as my coder? And, most importantly, would it be a good educational tool?

I haven’t finished the game yet, but I do have a version 1 (actually, about ten different versions!) and I’ve learned enough about how to do this in practice with AI that I thought I would share some of my insights.

Here are three key things I’ve learned:

1. Vibe-coding something is quick and easy, but generating something that passes educational muster remains hard and time-consuming

2. You need to take your time with design. There are hundreds of micro design decisions that need to be made in a complex game, and just delegating those all to AI will end up with a working prototype, but one that has cryptic game play elements, may not be engaging, and will lack the pedagogical grounding.

3. Contrary to popular belief, you don’t need to use the latest, greatest (and most expensive) model. In my experiments, I tried out Anthropic’s Fable 5 to see what it could do, but, for the most part, Sonnet 5 did just fine.

Below, I break down each of these three key learnings.

Vibe-coding something is quick, but…

As ever with AI, producing something is fast and easy, but getting to a final product takes time and deep thinking. This is true whether you are using AI to create a slide deck, write a strategy document, or build a game.

Below is the first mock-up that I asked Claude to produce. As you can see, it looks very Claude-like. The fonts and colour scheme are very typical of what Claude defaults to. It also has a tendency to default to small font sizes, which annoys me, as it’s an obvious aspect of good UX design it should be aware of.

Apart from the look-and-feel, Claude’s first few versions just didn’t make the bar. They were typically cryptic: for some reason, I found Claude didn’t explain well what the user is supposed to do. Granted, some commercial games/simulations also have this problem (I recently played the commercially successful Papers, please, looking for inspiration, and it took me a while to figure out the game controls). Claude seemed to assume the user would know what to do. I’m not sure why this is: perhaps because I had discussed the gameplay in detail before asking for a mock-up, it incorrectly assumed I would be the only player and didn’t need a clear and simple explanation?

This problem became worse the more complex the gameplay became. A simple game that only presented scenarios and asked the user for a multiple-choice response was relatively clear. However, my later, much more interesting, versions that included scoring mechanisms, agents playing corporate roles, and real-time interventions, although Claude could generate them easily, added a layer of complexity that Claude didn’t clearly explain by default.

I estimate that I spent hours on some versions just teaching Claude to explain things step by step so the user could understand what was going on.

Take your time with design

Throughout this experiment, I quickly learned that it pays to take your time with design. To put it bluntly: slow down. It’s invigorating to see Claude generate a version of the game in your head, but it’s ultimately inefficient to keep generating version after version, each only a slight improvement on the last. You’ll end up sitting there, spending most of your time waiting for Claude to regenerate, time that is better spent working through key design elements with Claude.

As an example, I spent a good half day working through different settings for the game. This is actually where Claude excelled: it’s inventive (dare I say, creative) at coming up with game ideas. It suggested everything from a simulation based on a Royal Commission inquiry to a 3D casino-style game. Many of these ideas were cool, but not quite what I was looking for. But going through this lengthy process with Claude helped me refine what I was looking for (more on that later). It was also during this design process I realised that Claude always defaulted to text-heavy 2D designs. It was biased towards games that could easily be implemented with Javascript and HTML, and it actively steered me away from anything that would require a game engine (e.g., Unity). If I wanted 3D or more interesting visuals, I had to tell Claude explicitly.

More broadly, any serious game (or any computer artefact, for that matter) contains hundreds of micro design decisions, everything from font colour to strategic narrative flow. It’s tempting to think you can just delegate all of these to AI and expect it to make sensible choices. But, in reality, I found that you have to get involved and make most of these decisions yourself. You can certainly (and should) use AI to generate a first version, if that helps. After all, it’s usually easier to critique something than build it from scratch. I found that a good way of working was to first hold a conversation with Claude about a particular design aspect, agreeing on the principles and big picture issues, and only then ask Claude to generate something. Then critique that, and iterate.

As ever with Claude, I find it very eager to go straight to build, and find that I have to sometimes include an explicit phrase in my prompts, something like: Do not build anything until I tell you to. Otherwise, it’s like a puppy – way too eager to please, thinking it’s doing the right thing, when in fact, it’s just creating a mess.

Do you need to use Fable 5?

There’s a lot of chatter in AI circles about Anthropic’s Fable 5, its latest and greatest model. A version of the infamous Mythos, this is the model the US government considered so dangerous it directed Anthropic to deny use to any non-US citizen – which effectively meant banning for everyone. That ban has since been removed and Anthropic has been allowing Claude uses to access Fable for a limited period of time.

If you listen to the AI evangelists, Fable is a step change over Anthropic’s next-best series of models, Opus, and its even less powerful (but still very impressive) model Sonnet 5. Fable also eats up your token budget much more quickly, so I was curious to see whether building my game with Fable versus supposedly less powerful Claude models would make a difference.

By default, I use Sonnet 5 in Claude because I find it does pretty much everything I want and I never run the risk of running out of token credits. But, when building my game, I directly compared Sonnet 5 and Fable 5: same prompt, different output.

My conclusion? Fable is more creative than Sonnet – and yes, I use that word deliberately. All my game designs from Sonnet were 2D, text-heavy, relying largely on boxed text to direct gameplay. They were interesting but not very inspiring graphically. Even when I specifically instructed Sonnet to be more graphical and include 3D elements, it was a struggle to get what I wanted.

Fable, on the other hand, with the exact same prompt was able to improvise. Whilst still text heavy and 2D, my first attempt with Fable suddenly had real-time interrupts, more interactive features, and even sound! Critically, however, the actual gameplay – by which I mean the user’s educational experience – wasn’t any better.

I also found Fable produced better results when I asked it for 3D or more graphical elements. Sonnet’s version looked clunky and untidy. Fable looked mostly slick.

So, for an initial build, I would favour Fable over Sonnet (I did not try Opus).

However, for 90% of my interactions with Claude, I found Sonnet to work perfectly fine. Once I had the initial UX build using Fable, there was really no need to continue to use Fable (and all those tokens!) to update, tweak, improve. Sonnet was perfectly capable of editing Fable’s code-base and doing what I wanted.

The moral of the story? Use Fable only when you need it. If the task is complex, requiring multiple interacting elements, Fable is probably going to produce a more interesting result than Sonnet. But for most tasks, you don’t need that extra power. Model selection is critical.

So can you really vibe-code a decent game?

I’m still working on my AI governance game simulation.

I’ve got to the point where I’m fairly happy with the look-and-feel. I didn’t know what I was looking for when I started out, and Claude really helped me narrow down from a whole variety of options. The result is based on simple graphics, which turns out to be the look I was going for. And it’s got some very cool elements. For example, a timer tracks the progress of a typical day, but it speeds up to add a sense of urgency, and the background music speeds up as well to put the player under pressure.

It’s far from perfect yet. And I’m sure I will spend many happy hours further developing it.

More importantly, though, where I think AI has failed me is in designed a sound educational experience. What I have so far looks and feels like a game, but it doesn’t capture the learning elements I am looking for. And having tried to get Claude to produce these, I’ve come to the conclusion that it isn’t able to. It’s time for me to step away from the keyboard and do some deep work by myself. The hard part of this turns out to be design the scenarios that the player needs to make a decision on, designing them in such a way that they hang together and build on each other over time, and doing it in a way that the player actually learns something about good governance rather than just playing a game.

I have to admit, I’m a little surprise Claude hasn’t yet been able to achieve these goals. I’m not saying it can’t – just that it’s going to take a lot of work to get it to that point. And, right now, I believe I will be more efficient (and produce a better quality game) if I sit and think about the design myself and go back to Claude with a plan.

So: can you really vibe-code a high-quality, production-ready game? Or are Mollick’s tweets all smoke and mirrors? I think it’s the latter. I’ve loved developing my game with Claude as my thinking partner, but even Fable has its limitations.

For now, I’m going back to the drawing board. I might even sketch some things out with a pen and paper. I’ll revert to Claude when I’m ready. And when that happens, I’ll be sure to come back here and update you on what happened next!

The question originates from concerns that employees are not using AI responsibly: they are over-delegating business decisions to AI, not thoroughly checking AI outputs, seeing their skills eroded as they defer too much to AI, and facing declining productivity when AI slop saves time for them but slows down the system as a whole.

In organisations, the usual response to these problems is to develop guidance on the “responsible use of AI”, which is typically interpreted to mean a set of policies, guidelines and frameworks that set boundaries on how AI should be used within an organisation to ensure that the use of AI is ethical, safe and productive.

I’ve worked in the area of responsible AI for years. Indeed, I even wrote a book on the topic. And I’ve found a couple of problems with this approach to responsible AI. First, policies are useless if they are not enforced. And most organisations develop the policies but not the enforcement framework. Second, and more fundamentally, responsible AI is too often described in very abstract terms. The framing is around ethics, values and corporate regulation. All of these are important, but thinking of responsible AI only at such a high level doesn’t provide any concrete guidance to staff using a generative AI tool daily, wondering what exactly is the right way to use AI.

For the most part, an average staff member won’t encounter deep ethical questions about their use of AI. More likely, they’ll be using AI to produce a document and will be more concerned with where do the boundaries lie in terms of how they can and should use AI in producing that document. Is it okay to use AI at all? If yes, is any use of AI sanctioned as long as AI outputs are checked? Or are there some ways of using AI that are off limits?

Responsible AI policies don’t address these very real questions. And it’s these very practical considerations that I aim to address in this article.

Responsible AI in Day to Day Work

Consider the following examples. Which of these do you consider to be responsible use of AI?

• The reference letter. You are asked to provide a reference letter for a former colleague. Time is tight and you’re busy with other things, so you give a few dot points to AI and ask for a draft. You agree with what it produces and so send it off. Nothing is untrue, but the recipient thinks this is your considered judgement.

• The tender response. You use AI to draft the bulk of a tender response, verifying every claim. You win the work, but find out later that a much smaller competitor with a better solution, who wrote their bid manually, lost out.

• The innovation prize. You enter a competition to come up with ideas to improve your local community. The prize is $1000. You use AI to generate entries and pick the best one. You win.

People tend to have different reactions to these scenarios. In the case of the reference letter, some will say this use of AI is fine. The letter is accurate; AI didn’t make anything up. Others, however, will be horrified at this use of AI: a reference letter is a medium of trust between two parties and that trusted relationship shouldn’t be delegated to an AI.

For the innovation prize, does it matter that you won a prize but didn’t have the idea? Many people will say yes, some going so far as to say it’s dishonest. However, what if we modify this scenario slightly so that you don’t simply have AI generate ideas and pick one, but use AI to brainstorm different ideas, including putting your own thinking into the winning choice. Is this way of using AI okay? Or should the prize only go to a fully human-generated idea?

These dilemmas are not hypothetical. Workers are faced with situations like these every day, and need to ask themselves: when is it okay to use AI and when is it not? And does the way in which I use AI matter? If so, what is the right way?

Ways to use AI

One of the key points of this article is that the use of AI is not a black and white issue. Take AI in writing, for example. There is a great deal of backlash against any use of AI in writing beyond spellchecking, improving grammar, or research before writing begins. It’s as though there are only two modes for AI in writing: either use it in a very minimal way, or go all in and generate an entire article and use as is. Of course, most uses of AI are neither of these. In practice, AI use is much more nuanced. An author might use AI to brainstorm ideas, write some text him/herself, then ask AI to improve the flow of the text. In my view, this shouldn’t be banned. Although there are plenty of commentators out there who say I am wrong.

The point of this article isn’t to debate whether AI-assisted writing should be allowed or not. It’s to show that there are different ways of using AI in any knowledge task. And understanding how we use AI is useful as a sanity check to understand whether we can legitimately claim ideas as our own or have really thought through the reasoning ourself.

Inspired by Danny Liu’s categorisation about different ways to use AI, I’ve come up with my own four-category AI-use classification. Each category captures one way of interacting with AI. Whilst the “best” category will change given a particular context, in general, you should aim to be in the Partner category. This is where you get the best of both worlds: your ideas and thinking are still yours, but AI helps you refine them. Other categories aren’t always wrong, but, in general, I’d be aiming to be in the Partner category the majority of time.

1. Surrender: You accept AI output as final without checking it. You write a prompt, and other than a few tweaks, you accept AI’s first answer as the answer.

2. Verify: You check AI’s output, carefully catching errors, confirming claims and asking for sources, but you’re still letting AI do most of the work, although you’re careful not to trust it more than you should.

3. Partner. You start a conversation with AI by bringing your own ideas to the table. From there, you actively shape the process, setting constraints up front, redirecting AI mid-task, bringing your own expertise to challenge the framing (not just the facts), and iterating towards something that is better than either you or AI alone could produce.

4. Reject. You dismiss AI’s output wholesale and redo the work yourself, or route around it entirely out of distrust or because you consider the AI output to be substandard and unfixable.

Applying these categories to the three scenarios above illustrates that, for any given task, AI can be used in very different ways:

• The reference letter. It’s unlikely that Surrender would produce a good result here. You’d need a “mega prompt” to give AI all the necessary context about the person you are providing a recommendation for. Verify could potentially produce good results, but again you’d need to include a lot in the context of the prompt. Partner means that you have a conversation with AI, where you collaboratively explore the strengths and weaknesses of the person you are recommending; only once that’s fully done would you ask AI to generate an actual letter (or write it yourself). Under Reject, you try to use AI to draft a letter, but get frustrated by its generic nature and decide not to use AI at all.

• The tender response. Surrender here is risky: a generic prompt produces boilerplate that ticks compliance boxes but ignores what actually differentiates you from competitors bidding on the same tender. Verify can work if you feed AI a very detailed brief and then rigorously check every claim against your actual capability, but AI is still driving the strategy. Partner means working through your key differentiators with AI first, arguing about what the client really values, stress-testing your angle, and only drafting once that thinking is done. Under Reject, you try AI on the tender, find the response too generic to fix, and write the whole thing yourself.

• The innovation prize. Surrender produces a submission that reads like every other entry: competent but forgettable, because AI has no way of knowing what’s genuinely novel about your idea from a thin prompt. Verify gets you further if you supply rich detail on the innovation and then check that AI hasn’t overstated or misdescribed the technical claims. Partner means using AI as a co-designer, asking for and giving ideas, and refining those together. Under Reject, you generate an idea with AI, find it boring, and can’t imagine how AI could help you improve it.

So, which category are you?

There isn’t a “right” answer when it comes to which AI category you should use. Most likely, you will be in a different category at different times, depending on the context. For example, if all you want is an answer to a simple question like “Which actor is the lead in Obsession?” then Surrender is perfectly fine. However, if the task is to come up with a strategy document for a 5000-strong organisation, I wouldn’t want to be in any category other than Partner. The Reject category is also perfectly fine. There are plenty of instances where I’ve found AI can’t really understand what I’m trying to achieve, and doing it myself is quicker (and frankly, less stressful).

Generally speaking, however, I would say you should aim to be in the Partner category the majority of the time. Which begs the question: which mode do you personally tend to be in most of the time?

In the absence of a scientifically rigorous instrument to measure your interactions with AI and assign a category, we can take a less rigorous (but still useful) approach: ask AI itself!

Here is a prompt you can use to ask your favourite genAI tool to assess how you interact with it. Note that this prompt will work straightaway in Claude Pro and Max, which can access your full history of conversations. Other genAI tools, depending on your setup and subscription, may not be able to immediately analyse past conversations. In the worst case, you may need to cut/paste prior conversations into the current chat; 15 should be sufficient. In any case, try the prompt first. If your AI tool can’t provide an answer, it should let you know what you need to do to help it.

What you should get back is a (very) approximate set of percentages, one per category, based on your conversation history. Obviously, take these numbers with a very large pinch of salt. You can also ask AI to give you examples of interactions you’ve had in each category. The point here isn’t to get a definitive score but to use the exercise to self-reflect on how you typically use AI and see if you want to make any changes.

Here’s Claude’s response when I entered the prompt above:

Reject is harder to measure, of course, as it often ends up with me leaving Claude altogether.

You can also get spot-checks from AI to sanity-check how you think you are interacting with it. Try this prompt at any point:

Again, depending on which AI tool you are using, it might understand straight away or need further context. This evening, I was using Claude to help me with some branding for my company. The above prompt got the following answer:

Teaching Responsible AI

I’ve taught students of all shapes and sizes over many decades in various career roles. And one thing I’ve learnt is that you have to meet students where they are, not where you think they should be. Which means presenting concepts in simple, easy to understand ways. I think the responsible AI ecosystem is getting this wrong.

When I first started researching responsible AI six years ago, the problem was a lack of guidance. Now, the problem is the opposite: there are hundreds of frameworks, principles, reports, books and podcasts out there, each promising to teach you exactly how to use AI responsibly. And yet, most of these frameworks are overly complex, try to do too much, and ultimately don’t solve the problems that AI users really have: how do I know if I’m applying AI responsibly, not in some abstract sense, but here and now?

What I’ve tried to do in this article is to redress the balance, even if only a little. Personally, I find it’s helpful to reflect on how I use AI from time to time, and, if necessary, to adapt. My simple four-category model provides a quick way of assessing AI use. And, perhaps most importantly, it pushes back against the black-and-white thinking that seems to dominate AI discussions nowadays. It’s never “all in on AI” or “avoid it like the plague”. AI is a really useful tool, but, like any tool, can be misused. So, think about which category of use you mostly fit into, and then ask if that’s where you want to be.

What Schmidt and others said is nothing new. Tech evangelists have been saying for years that AI will transform every industry, supercharge productivity, and lead to a world of abundance. The message hasn’t changed. What has changed is the reaction to it.

While there’s always been an anti-AI segment of the population that rail against the dominant AI narrative of transformation, the movement appears to be gaining ground. My own LinkedIn feed has changed significantly over just the last few months. Whilst not a scientific sample, it’s gone from mostly positive AI messages (“I can vibe code an app in 2 hours!”) to mostly negative ones (“AI writing is stopping us from thinking!”).

So, what’s going on?

AI got very real in the last few months. Take coding as an example. At the end of last year, AI coding tools were still mostly niche, useful for vibe-coding a quick prototype, but not production-quality software. That changed in 2026. Tools like Claude Code, Cursor and Codex have come of age, and there’s barely a software engineer out there now who isn’t using them daily for real projects. The penny has dropped. AI actually works.

And with that step change in capability, the threat that’s been talked about since ChatGPT launched in 2022, but only ever taken semi-seriously, has suddenly become real.

The backlash is loudest among creatives. I’ve seen a wave of articles in recent weeks with the same theme: AI writing is bad (“If we use AI, we lose the art”), AI music is bad (“AI can make songs, it can’t make Swifties”), AI in education is bad (“AI may even harm [children’s] cognitive development”).

The anti-AI arguments usually take one of two forms. The first is that AI-generated work is poor quality. But while there’s plenty of AI slop out there, a blanket quality argument doesn’t hold. AI art has won competitions against human artists. AI poetry has been ranked above Shakespeare and Emily Dickinson. The second argument is that AI encourages cognitive offloading: that we’ll all get dumber. Possibly. Socrates said the same thing about writing, claiming that writing weakens “the necessity and power of memory” and allows for “the pretense of understanding, rather than true understanding.” We survived that one.

My provocation is that both of these arguments are a smokescreen. The real reason runs deeper: it’s about identity.

Is your identity under threat?

It’s comforting to believe that a threat to one’s way of doing things is driven by evil. But that doesn’t make it true.

I argue that the real reason for creatives being anti-AI runs much deeper. It has more to do with a detractor’s feeling of loss of identity.

Take software engineers, as an example. Many software engineers are going through an existential crisis right now. They trained for years, sometimes decades, in how to build quality software. Now AI can do it better. Some are embracing this, finding that AI makes them more creative. But many are like stunned animals, wondering what their future holds. They were told to study computer programming for a guaranteed career. Suddenly their future – and their status – looks a lot less certain.

The same is true of writers. Part of the identity of an accomplished writer is the status it confers. Writers have always occupied an elite place in society, with the ability to cast words that influence hearts and minds. When that is threatened, what’s being threatened isn’t just a profession; it’s a way of life. Professional writers aren’t just concerned about their income; they’re mourning the loss of their revered place in the world.

What they often fail to acknowledge is that for less elite writers, AI is a huge opportunity. If you need to write in a language that’s not your first, if you suffer from dyslexia, if you simply haven’t had the right family or schoolfriends to get your creative work noticed, then AI levels the field.

A 54 year-old nonprofit worker in Toronto just won $50,000 for a film he made with AI, a film so human that audiences forgot the actors were synthetic. He spent years writing thirty film scripts that he couldn’t get any attention on.

AI then is not destroying creativity; it is redistributing it.

The gradual erosion of identity

The current AI backlash, in my view, represents an existential angst about what it means to be human. Throughout most of the history of AI (which dates back at least until the 1950s when the term artificial intelligence was coined), AI was seen as way to automate the drudgery in life. Creativity, in particular, was always assumed to be beyond AI. It was the realm of humans alone.

But with the advances in AI in recent years, this assumption has been threatened. While there are still those who cling on to the unique ability of humans to be creative, the picture is certainly a lot murkier than it was. If you use AI responsibly in your writing, it can enhance your creativity, not degrade it. I’ve written many pieces (including this one) where AI has been my creative sparring partner. I’m not delegating my thinking to AI; I’m using AI to help me shape my thoughts, spark new ideas, and improve my narrative flow. I’m still being creative myself, but it would be hard to argue that my AI partner isn’t also bringing creativity to the table.

We have each constructed a unique identity over the years based on what we do, who we have relationships with, and how we act. Is AI threatening parts of that identity? By encroaching on some of our core beliefs about what it means to be “me”, is AI striking at the heart of it all: human identity?

AI can sometimes feel like it is slowly eroding aspects of our identity. It’s like identity is a cliff at the edge of the ocean. The waves roll in relentlessly, chipping away at what makes us unique bit by bit, until a part of the cliff face falls off. For years, we might not notice the erosion at all, happily moving on with our lives. But then an event like the arrival of generative AI happens, and the storm rips away whole chunks of what we held to be sacred. The cliff is still there, but it feels like we’ve lost a limb.

To flourish in a world of AI, creatives need to adapt. Holding on too tightly to the identity of the past might be ok for a while, but it’s hard to imagine that this won’t become niche over time.

The current debate frames the use of AI as very much a black and white issue. Either you embrace AI in your creative pursuits, or you don’t use it all. There is no value given to the in-between. As Mark Bassett points out in a recent paper on the problems of using AI detectors to catch students “cheating” with AI, “categorising text as human- or AI-generated imposes a false dichotomy that ignores work created with, not by, AI.” And it’s this false dichotomy that is at the root of the identity problem, because it encourages the reaction to hold on to current identity rather than adapt to a new one.

We shouldn’t give up identity completely to AI, of course. It’s perfectly fine – and necessary – to preserve some identities from the pre-AI age. But it should be a deliberate choice, not a knee-jerk reaction based on fear. Both AI and humans have strengths and weaknesses. Rather than over-optimising towards one, why not embrace both, and get the best out of two ways of working?

There’s been a lot of chatter recently about whether you should use AI to write. Most commentators – or at least the ones I’ve read – are strongly of the view that no you should not. And their common argument is that the act of writing is itself a thinking experience: in the process of writing, you have new ideas, structure thoughts, and construct better arguments. Former Australian Chief Scientist Alan Finkel sums up this argument in his recent Australian Financial Review oped:

It is when we write a letter, an email, an article or an essay that we assemble our thoughts and arrange them formally into a logical sequence that constitutes our informed opinion.

If we fail to train young people to write, we are failing to train them to think.

In this article, I’m going to argue the contrary view.

Where I agree with Finkel and others is that we should continue to teach writing skills to our kids. He’s right that the act of writing is a process worth engaging in because it forces us to engage deeply with our thoughts, which in turn bolsters cognitive thinking, and, frankly, makes us smarter over time.

Where I disagree, however, is that the issue of AI and writing needs to be so black and white. The anti-AI writers out there argue that it’s all or nothing. Sure, they say, use AI for spellchecking, to improve grammar, or for non-writing tasks such as research or indexing. But you cannot, they say, use AI to write any actual text. That is somehow forbidden and demonstrates that you haven’t really thought through your ideas.

Are you proudly human?

Finkel, along with my friend Trevor Woods, has started a company called Proudly Human that will certify human-written books and other creative artefacts. If you follow their de minimis rules, you receive a Proudly Human trustmark. These rules include the following unacceptable uses of AI:

• Using AI to draft portions of text (e.g., a paragraph)

• Editing AI-generated text

• AI-generated design elements or creative content that meaningfully contributes to the final work

Putting aside the irony that Proudly Human itself uses AI to detect if you really wrote that book yourself, I find the de minimis rules too purist. Indeed, after publication of my recent book, AI for Business: A Guide to AI Adoption, Trevor reached out to me to ask if I would like them to certify it as human-written. I had to tell him that my book wouldn’t pass the bar.

You see, I used AI in a variety of ways when writing my book (all declared in the book’s preface):

• For generating images

• For populating tables

• For formatting references

• For editing, specifically to tighten up my narrative flow when I felt it was clunky, or to better rephrase a sentence, or occasionally, a whole paragraph

Two chapters in the book are completely and utterly written by AI. They are case studies (a happy-day scenario of what happens if a company follows my advice, and a rainy-day scenario when they don’t). I wasn’t originally going to include these chapters as-is but when I submitted the rest of my manuscript to ChatGPT and asked it to generate two case scenarios, it did such a good job, that I left them in. And I’m not ashamed.

Of the rules above, I think Proudly Human would only allow me to format references. It certainly wouldn’t accept those two AI-generated chapters. So, no certification for me – because I obviously didn’t think very much when I wrote my book, right?

I want to stress that I have the greatest respect for Finkel and Woods. I have no doubt that there is a market for certification of human text. Proudly Human is not the only company doing this (there are about a dozen worldwide). And I’m sure some authors will lean into their human-ness; it’s just not for me.

Where AI can and should be used in writing

In a recent paper, Lodge and Loble tackle the phenomenon of what has become known as cognitive offloading – or the idea that some uses of AI make us dumber. What I like about Lodge and Loble’s paper is that it is grounded in the latest research. And a key point they make is that not all offloading is the same. There is beneficial offloading when AI takes “unnecessary” cognitive load off our hands – think spellchecking, formatting etc. And there is detrimental offloading when AI takes away the hard work of thinking, thereby reducing our ability to learn over the long term. In an educational context, research clearly shows that AI can be used for beneficial offloading but should not be used for detrimental offloading. This is entirely consistent with Finkel’s thesis.

But not all writing is pedagogy; in fact, most is not. And, as far as I’m concerned, it’s perfectly OK to use AI in some writing tasks banned by Proudly Human. Here are a few examples where I typically use AI in my own writing:

• To brainstorm ideas and help construct a narrative flow or structure

• To fact-check (I wrote about this here)

• Sometimes, to write sentences (or even whole paragraphs!) when I am struggling to find just the right words

• To tighten up a text, including making it shorter, or offering suggestions for what I should cut

One of my favourite uses of AI in writing is to use voice-mode in the car on the way home and step-by-step co-design my next article with AI.

One thing I rarely do is to simply prompt AI to generate an article and then just use the output. The only time I’ve done that is in the two chapters of my book mentioned above, and that was a very particular set of circumstances, in which I was doing an experiment and using the AI-generated chapters to make a point. The reason I don’t simply re-use entirely AI generated articles is, firstly, because they are not very good, and, secondly, because they don’t have “me” in them. And I want to write “me” not AI.

In this respect, then, I once again agree with Finkel.

But my main beef is that the debate is presenting things in a very binary light, as though you either write nothing with AI, or everything. And that’s simply not reality. Nor is it helpful. There are many different ways to use AI in writing. Some of them are good, some of them are bad. But it’s a spectrum of uses.

At the risk of using a poor analogy, I think AI writing might be a bit like cooking. The purist would say that you should never buy ready-made food, and always cook from scratch yourself. Others like the convenience of store-bought meals (although there’s a serious risk you’ll end up with slop this way). In the middle is the cook who follows a recipe to create a beautiful home-made creation that still captures the essence of the cook but doesn’t require them to attend culinary school.

None of these three ways of preparing food is inherently bad. Yes, it’s true that if you only buy ready-meals, you’ll never learn to cook. But it’s also true that some of the yummiest food I’ve ever tasted comes from a recipe – especially when I improvised around that recipe.

We have a tendency with AI to present everything as black and white. AI will either save us or doom us. Make us rich or take all our jobs. Be super-intelligent or just produce AI slop.

None of these extremes is right. The truth, as with all things, lies somewhere in the middle. Balance is key. And we need balance too when it comes to AI and writing.

On the surface, there’s not much to dislike about Pope Leo’s new encyclical letter, Magnifica Humanitas, where he sets out his vision for the future of artificial intelligence. His key message is clear: AI should serve humanity, not subjugate it. Only the most ardent AGI enthusiast, one who thinks humanity should serve AI rather than the other way around, would disagree with this.

And yet, if you read the encyclical (and you can read all 42,300 words here) there is a not-so-subtle subtext that has received almost no attention in popular media coverage. The letter quite rightly highlights the dangers of AI. For example, it talks about misinformation:

“The use of digital platforms and AI systems is driving profound changes in public and political communication. Tools that could foster dialogue and participation are often used to construct distorted narratives and blur the boundaries between truth and falsehood, mixing facts with opinions.” (Paragraph 132)

It talks about AI bias and the potential for exclusion:

“The same technologies that facilitate communication and access to resources can also support models that exploit the most vulnerable, create new forms of slavery and derive profit from conflict.” (Paragraph 240)

And it talks about the need to counter the concentration of power in the hands of a small number of AI companies:

“When such power is concentrated in the hands of a few, it tends to become opaque and evade public oversight, increasing the risk of distorted forms of development that give rise to new dependencies, exclusions, manipulations and inequalities.” (Paragraph 95)

Hard to argue with any of that. But the encyclical also emphasises that AI must have clear moral underpinnings and that, not surprisingly in a document authored by the Pope, these underpinnings should be found in Church doctrine: “human persons are called to communion with God and ‘can fully discover their true selves only in sincere self-giving.’” (Paragraph 48)

In other words, the 42,300-word document is essentially a restating of long-held Church beliefs: that humans need moral guidance, the Catholic Church is uniquely positioned to provide it, and that this guidance can only flow from belief in and devotion to God.

The new religion

Silicon Valley has often been described as the seat of a new religion. The parallels between tech dominance and religious authority are hard to ignore. God-like figures like Altman and Musk draw cult-like followings, make pronouncements about superintelligent beings, and speak of futures so transformative they demand something approaching faith.

It’s easy to draw structural similarities between how the Silicon Valley elite and the Vatican elite operate. Read those three quotes at the top of this article again.

“Tools that could foster dialogue and participation are often used to construct distorted narratives and blur the boundaries between truth and falsehood.”

Is this a statement criticising AI-driven misinformation, or an equally accurate description of how any powerful institution, religious or technological, maintains its authority? Both the Vatican and Silicon Valley have mastered the art of presenting their particular vision of the future as inevitable truth.

Or consider the second quote:

“The same technologies… create new forms of slavery and derive profit from conflict.”

Ironically, this same encyclical contains an apology for the Catholic Church’s centuries-long delay in renouncing old forms of slavery. The institution critiquing new forms of exploitation is the same one that took eighteen centuries to unequivocally condemn the old ones.

Or the third:

“When such power is concentrated in the hands of a few, it tends to become opaque and evade public oversight, increasing the risk of distorted forms of development that give rise to new dependencies, exclusions, manipulations and inequalities.”

The Catholic Church is one of the most concentrated, opaque, and powerful institutions in human history. The Vatican is itself a private, transnational entity whose resources and capacity to intervene surpass those of many governments — with less external accountability than any technology company.

But the point of this article is not to attack the Church. Nor simply to catalogue the contradictions in the encyclical’s language. My question is more fundamental: should the Pope be leading the conversation about ethical AI at all; and if so, on what terms?

Whose morals?

The Pope’s key message is that AI should serve humanity, and that we shouldn’t allow the essence of what makes us human to be eroded, even when that essence is uncomfortable:

“The moral corruption of our limitations as created beings — namely the evil that clearly agitates the human heart — ruins society and life, at times reaching extreme forms of inhumanity. Yet even these painful expressions of our limitations leave openings for the good. Even when persons dehumanize themselves and bring about tragedy, a small light continues to shine within humanity, one that can be rekindled, with God’s grace.” (Paragraph 121)

Remove those last three words and I have no argument.

Like the Pope, I am a strong advocate for retaining what it means to be human in a world of AI. My LinkedIn banner reads: Let’s use AI to build the society we want, not one we create by accident. The title of this blog is Being Human in an AI World. On the substance, Leo and I are not far apart.

So if I agree with the top-level message — responsible AI, human dignity, resist the concentration of power — what exactly is my problem? Why can’t I simply accept the Pope’s words as coming from a fellow human-enthusiast and join his crusade?

Dialogue is not dialogue when it’s based on dogma

My principal concern is this: although I agree with the encyclical’s headline statements about responsible AI, I don’t agree with the foundation beneath them. Specifically, I don’t accept that responsible AI requires a belief in God, or that we should defer to the power, influence, and financial might of the Catholic Church (or any church) as the arbiter of what responsible AI looks like.

Here is what I think many readers miss. The encyclical’s policy prescriptions — regulate AI, protect data as a common good, ensure accountability, protect the vulnerable — seem reasonable and even secular on their surface. But they are not freestanding arguments. They derive their authority and their specific shape from a prior claim: that the Catholic understanding of the human person, grounded in the Trinitarian God, is the one and only truth. Remove that foundation and the architecture changes entirely.

This matters because when we accept Leo’s conclusions without examining his premises, we are not actually agreeing with him. We are borrowing conclusions from a framework we haven’t endorsed, one that contains commitments most secular readers would not accept if they examined them carefully. The encyclical is unequivocal about who is ultimately in charge:

“we must consider the digital world as a new continent to be evangelized, one that requires generous missionaries who are mature in the faith.” (Paragraph 238)

That is not the language of open dialogue. That is the language of expansion. And the document’s opening paragraph leaves no ambiguity about the framework within which that dialogue takes place:

“We wish to engage in dialogue with all men and women of our time... Indeed, openness to dialogue is an integral part of the Church’s vocation because, constituted in Christ as ‘a sacrament… of communion with God and of the unity of the entire human race,’ she recognizes history as the place where the Gospel challenges and directs human experience.” (Paragraph 2)

Dialogue, yes, but on the Church’s terms. The Gospel doesn’t just inform the conversation. It directs it.

The Pope frames the entire future as a binary choice:

“Humanity, created by God in all its grandeur, is today facing a pivotal choice: either to construct a new Tower of Babel or to build the city in which God and humanity dwell together.” (Paragraph 1)

Utopia requires faith. Dystopia is what happens without it. There is no secular path to human flourishing in this worldview. Babel failed, the Pope tells us, because “it was a project conceived without reference to God.” The message for AI is clear: build without God and you will fail.

My concern, then, is not with this particular message. My concern is with what we practise when we receive it uncritically. Every time we accept a powerful institution’s pronouncements without scrutiny, even when those pronouncements are correct, we reinforce the habit of deference that makes us vulnerable when the pronouncements are not. The Catholic Church has been saying good things for two thousand years. It has also, in that same period, caused extraordinary harm. The people who enabled that harm were not bad people who ignored good messages. They were good people who had practised, their whole lives, not questioning the source.

Our need for authority figures

The reason most readers won’t notice the dogmatic scaffolding beneath Leo’s policy prescriptions is the same reason they won’t question Altman’s predictions about AGI: we are wired to defer to authority, and both institutions have mastered the art of projecting it.

This is particularly true when it comes to AI. There is so much noise, so much anxiety, and so much genuine uncertainty that most people, whether they work in AI or not, are yearning for guidance. For someone to point towards a north star. Some gravitate towards billionaire tech moguls for the answer. Others towards billionaire religious institutions.

Humans are deeply social animals who evolved in hierarchical groups. Deferring to authority figures — tribal leaders, elders, shamans — was a way to survive and prosper. It reduced decision-making costs, coordinated group behaviour, and provided psychological safety in the face of uncertainty. The instinct is ancient and genuinely useful in many contexts.

The problem is that we are running that ancient hardware in an environment it was never designed for: global media, trillion-dollar institutions, and existential technological risk. That is not to say we should eschew authority figures entirely. Rather, we need to be far more vigilant about who we trust, and far more transparent about the biases and incentives behind those we choose to follow.

The tech machine and the Vatican offer very different reactions to the AI revolution. If you listen to Altman and Musk, the gradual erosion of human faculties by AI is something to be celebrated; it will lead to abundance. If you listen to the Pope, it is time for humanity’s last stand. To circle the wagons. But both, in their different ways, are asking you to trust them. To follow. To defer.

The essence of humanity

There is a lot of talk nowadays about what it means to be human in a world where AI outperforms us across so many domains. What is the essence of our humanity?

One answer is the desire to be led. Humans want someone to show them the way. To lead them out of the dark. Moses parting the Red Sea. Virgil guiding Dante through the inferno. The shepherd leaving the ninety-nine to find the one lost sheep. These are not just stories. They are the recurring dream of a species that has never been comfortable finding its own way home. Some see the tech billionaires as salvation. Others look to religion.

And the need to be led is not going to diminish any time soon. There are 8 billion people in the world today. The UN projects this number to grow, peaking at around 10.3 billion in the mid-2080s. That is a lot of people. And people need to be led.

I would love to end by saying that leadership is uniquely human, that it will always be a person who guides us, irrespective of their background, institution, or beliefs. But I am starting to wonder whether that will always be the case. How long before we are led by AI? Aren’t we already well along that path? And perhaps this transition will be far more insidious than simply installing an AI President or an AI Pope. It will be more subtle, a gradual surrender of agency to machines, bit by bit. It has been happening for years already, with doomscrolling and screen addiction. New interfaces — AI glasses, wearables, humanoid robots — may displace our screen addiction only to replace it with something more physical, more immersive, more inescapable and, yes, more addictive. There will be nowhere safe from the digital. Nowhere to go to escape.

And if such a world does come to pass, we will need a trusted leader more than ever. To guide us out of the abyss. What are the choices for such a leader? Will it be a tech billionaire? Or a man of God?

Whoever it is, my only hope is that he or she is human.

Every generation thinks it lives in uncertain times.

But I think the current generation can legitimately claim to be living in, if not historically the most uncertain, a time that feels the most uncertain because of the speed and pace at which bad news now travels in a digital world.

The challenges that face modern society are diverse and significant: climate change, increasing geopolitical tensions and the rise of populism, an ageing population putting pressure on our resources, and, of course, the coming of artificial intelligence, a trend that leads us to question everything we know about how we work, how we interact, and indeed, what it means to be ‘human’.

If you listen to some commentators, a future with AI sounds bleak. Mustafa Suleyman, Microsoft’s AI CEO, has said that almost all white-collar work will be automated by August 2027. August 2027. That’s not far away! People: you have just over a year to get your house in order.

He’s not the only one making such predictions. Dario Amodei, CEO of Anthropic, thinks half of entry-level white-collar jobs will be gone within five years. And former US Presidential candidate Andrew Yang thinks that if your job is mainly sitting at a computer, the game’s up. In the next few months, says Yang, we will see – and I quote – a “great disembowelling of white-collar jobs”.

What are we to do then, in the face of such major disruption? Or, more importantly, what are our children to do? For it is our children – meek or not – who will inherit the earth.

For me, this is not a hypothetical question. I have twin girls, currently in Year 6. They are at that magical stage of life – betwixt childhood and adulthood, when they are becoming independent, learning how to ‘adult’, and finding their own way in the world. But it’s also a critical time. In the next few years, they will need to make important – and life-changing – decisions about their future.

How are they supposed to make those decisions when the world around them is changing so rapidly? When all the usual guidebooks have been ripped to shreds in front of their eyes?

In particular, when it comes to artificial intelligence, should my kids lean into AI, embracing all it has to offer? Become AI-native, AI-first, perhaps even AI-subordinate? Or should they recoil from AI, instead choosing to double-down on all things human?

(As an aside, if it wasn’t for the obvious ethical issues, I’d be tempted to run an experiment with my twins. I could have one twin go all in on AI, and keep the other well away from it. And see who turns out best. What a great experiment that would be. Alas, if it wasn’t for the pesky matter of ethics!)

In the absence then of a dystopian twins experiment, what should my children – or your children for that matter – do?

What should your children study?

Or a version of this question, which I am commonly asked is: What should my children study?

In a world of excessive uncertainty, does it make sense to study anything anymore – when there might not be any jobs to go to at the end of it all?

As a computer scientist, who trained in the art of artificial intelligence thirty years ago, I would love to stand here and tell you that your kids should study computer programming. That’s what we’ve been telling our kids until recently, after all.

But in recent months, that advice suddenly looks ill-placed. In 2026, AI coding tools have suddenly matured. They’ve gone from being, at the end of 2025, clunky tools used by only a fraction of software developers, to solutions that in many real-world situations can generate 90-100% of all the code in a product. Indeed, Boris Cherny, who leads Anthropic’s AI coding work, says their latest desktop product, Claude Cowork, was “pretty much all” built by AI.

In a world where AI can write all the world’s code, can I hand-on-heart tell my daughters to study programming? Every software engineer I know is having an existential crisis. Everything they’ve trained in has suddenly become obsolete. Yes, it’s faster and more fun than before — but livelihoods are also at stake. Excitement won’t pay the mortgage.

If I am not going to recommend programming to my twins then, what other options are there? The law, perhaps? We’ll always need good lawyers, right?

Just over a week ago, Anthropic released “Claude for Legal”, which has been described as “an orchestration layer for legal work.” It pretty much up-ended the specialist Legal IT industry overnight. And this development is typical of what is happening in other sectors: it used to be the case that sector-specific work required sector-specific experience. This was the moat that allowed software-as-a-service companies to thrive in law, finance, and other professional services. But we’re now finding out that a general purpose company, like Anthropic, can disrupt a sector it knows nothing about. It’s Saascopolypse for lawyers.

The general public will, no doubt, not lament the death of lawyers. And I don’t believe all lawyers will be out of a job soon. But it’s conceivable we’ll need a lot fewer — so advising my kids to study law is perhaps not the best choice.

Maybe I will tell the twins to stay away from knowledge work altogether. To go towards jobs less likely to be disrupted by AI.

But even here, we are seeing signs of disruption.

Take nursing, for instance. The caring professions are often held up as being immune to AI. But we’re already seeing robots in aged care. Australian company Andromeda has developed a robot, Abi, that can take a lot of the pressure off nurses by engaging with residents. Andromeda was started by a Uni Melb student during COVID, now has a $100M valuation, and has already deployed Abi in tens of aged care homes across the country.

I don’t think robots will replace nurses — but some employers might use it as an excuse to hire fewer.

What about blue-collar work? The trades are being disrupted too. While I don’t think we’ll have robot plumbers any time soon, AI is already impacting expertise in the trades.

As an example, I recently had an electrical problem with my car and so called an auto electrician. After looking at my battery for a few minutes, the guy turned his back on me, pulled out his phone, and started consulting ChatGPT on what to do next. I’m not really sure how I feel about that. Does AI risk degrading specialist expertise in our blue-collar workers?

Leaning into the uniquely human

At this point, I’m struggling to know what to advise my kids – not computer programming, law, nursing, or the trades. So what then?

In his latest book, British author Daniel Susskind offers us some advice. The secret to a successful future, he says, is to lean into those skills that (at least for now) make us uniquely human: judgement, critical thinking, empathy, and system-level thinking.

I agree with Susskind — but I’d add one more set of skills, which might be more important than all the others: the skills of business.

Let me briefly cover three of these skills, that I think will become increasingly important in an AI-dominated future.

First, there is human values and ethics. One of the great unsolved problems of AI is that of ‘alignment’ – that is, how do we develop AI systems that act in alignment with human values? This is a problem as old as AI itself. Isaac Asimov wrote about it in 1950. His solution was the Three Laws of Robotics, rules hard-coded into any AI system to ensure no harm would come to humans.

In practice, however, hard-coding such rules into AI is not so simple. Modern-day AI is essentially a large-scale statistical pattern matching machine; and wherever there is statistics, there will be error. Machines are still notoriously fickle when it comes to values and ethics. If you want to teach your kids something useful, then, teach them to be ethical decision makers.

Second, there is leadership. There are 8 billion people in the world today. And the UN projects this number to grow, peaking at around 10 billion in 2080. That’s a lot of people. And people need to be led.

It’s possible we will end up in a situation where these people are led by an AI. You might have heard examples of AI CEOs – but all of these have either been hoaxes or publicity stunts. For the foreseeable future, at least, AI CEOs don’t seem likely.

People lead people. And people leaders need to be able to interpret the nuances of human-to-human connection to get anything done. That’s just not something that AI can do. At least not yet.

Finally, there are the business skills of entrepreneurship. And this is where I think we, as humans, should lean in most heavily. Entrepreneurs are already using AI to grow businesses in a way they couldn’t before. New products can be vibe-coded in a matter of hours. Purely AI finance or HR teams are realistic for a small start-up. In April of this year, the New York Times reported what may be the first example of a “one person AI company”: telehealth provider Medvi which, allegedly, grew from nothing to $400M in sales using AI to automate everything, and did all of this with a single founder and no employees.

The other week, I tried my own experiment in becoming a one person AI company.

I had an idea for a business. I’ve had many business ideas in the past, but mostly I’ve never pursued them because of lack of time, lack of skills, or lack of motivation. But for my latest (and greatest) idea, I was determined to do things differently – and the secret to doing this must be AI, right? Could I use AI to take my idea and quickly turn it into a successful business?

First, the good news. I went from idea in my head to fully-working prototype in just 4 hours. That’s from nothing to an iPhone app that I could potentially sell in just one evening. Pretty cool, huh?

But what I quickly realised is that building the tech is a minority of the effort in building a successful business. We now live in a world where ideas can be realised in hours rather than months — you can short-circuit all that initial work. But paradoxically, when building products from ideas is easy, having the right idea becomes ever more important.

The hard part of building a business – even a tech-driven one – isn’t the technology. It’s the human relationships. You still need the skills to convince investors, talk to partners, and build an ecosystem that can sell and distribute your idea. And all of that relies on human-to-human connection — something AI will never replace.

Business skills then – leadership, innovative thinking, values and ethics – have, in my mind, suddenly become prime currency in a world of heightened uncertainty. They were important before, but they are absolutely essential now.

This is the reason why I joined Melbourne Business School just a few months ago. People often ask me why – I’m an AI geek after all. But for me, the reason is clear: AI is disrupting every business, and business schools are training the next generation of leaders; business schools therefore need to be on the front lines of the AI revolution. And I want to help to get them there.

What should our children focus on then?

They should develop their business skills. If the future really is as bleak for jobs as some would suggest, we will find ourselves in a world where “hustle” is critical to survival; and hustle means business skills.

The future is a team sport

I’ll finish with a story about my twin girls. About a year ago, they joined a netball team. It’s been transformative for them. It’s given them a new network of friends. They love going to school because they get to hang out with those friends. And, as they grow into independent teenagers, they head off after school to hang out with those friends at the shopping centre or at Macdonalds.

As a father, I look upon all of this with a sense of pride. My kids are growing up. They have their own team. And that team is teaching them the human-to-human relationship skills that they’ll need in their future lives.

Team sports do this. They teach us the value of human-to-human connection. And I think business does that too. Business is a team sport. And while some of our team mates in the future might be artificial, the most important ones will remain uniquely human.

According to Deloitte, 74% of organisations are expected to use agentic AI at least moderately within the next two years. Use cases are expanding rapidly: agents handling end-to-end customer service, reviewing supplier contracts, monitoring compliance across jurisdictions, and generating code without human intervention.

The risks are rising too. Only one in five companies has mature governance in place for autonomous agents. Agents are most powerful when operating with little human oversight which is precisely when agents are riskiest. They introduce cybersecurity vulnerabilities, make costs harder to predict and raise questions about how automated decisions align with organisational values.

One risk that has received far less attention is the environmental footprint of AI agents.

There is already vigorous debate about the sustainability of AI more broadly. Critics point to rising energy use from data centres, including in Australia, where some operators have doubled carbon emissions in the last five years. Proposed data centres in our major cities are seeking daily water volumes equivalent to 80,000 homes.

Supporters counter that data centre energy use is relatively small compared to heating, ventilation, and air conditioning.

Water use is also minor compared to agriculture or even golf courses. Both sides are missing how rapidly AI use is changing and how little current energy and water projections account for it.

A recent study found agentic AI consumes between 62 and 136 times more energy per query than a conventional single AI inference. AI agents don’t just respond to prompts; they plan and iterate, making multiple calls to large language models to complete a single task. These “true” agents are the most energy intensive. With companies racing to implement AI agents – some have already deployed thousands of agents – the numbers add up quickly and have the potential to significantly change the AI sustainability equation.

Things might not be as bad as they seem, however. Not all agents are equal. Most AI agents deployed today are simple automations, many of which are not intelligent at all. But as organisations pursue more capable, autonomous systems, energy consumption is likely to rise.

There is also a countervailing effect. In some contexts, AI agents have the potential to be more efficient than humans at the same task. In the early days of generative AI, a widely cited study showed that a GenAI query consumed ten times the energy of an equivalent web search. That’s because GenAI does a lot of searching and synthesis that a web search doesn’t. But this extra work is exactly what makes GenAI more efficient than a human. Without GenAI, humans would need to do all that searching and synthesis by hand. This point was illustrated in a 2024 study that showed AI systems emit up to 1500 times less CO2e per page of text compared to human writers.

Australian companies are not well prepared. As mandatory climate disclosure processes gradually rollout across corporate Australia, the gap between adoption and governance becomes stark. Most sustainability frameworks are not designed to capture the complexity of AI workloads, let alone the iterative and dynamic nature of agentic systems. Boards are approving AI investments without clear visibility on their downstream energy implications. Technology teams are optimising for capability and speed, while sustainability teams are left without the tools to quantify impact. The result is a blind spot that is likely to widen as adoption accelerates.

AI agents offer capabilities that are hard to ignore and, in many contexts, hard to match through other means. But deploying them thoughtfully requires asking questions that are rarely raised: What does this agent cost to run at scale — in tokens, in compute, in energy? Are the efficiency gains at the business layer being partially consumed at the infrastructure layer? How does agent deployment sit alongside the sustainability commitments already on the public record?

These are not reasons to avoid agents. They are reasons to think about them more carefully than the current pace of deployment suggests most organisations are doing. The technology is advancing quickly. The frameworks for understanding its full cost are not. Closing that gap will define what responsible AI adoption actually means in 2026.

I’ve given hundreds of public talks. Of all shapes and sizes. I’ve given keynotes to over 1000 people and I’ve delivered intimate seminars to board rooms of less than ten. I’ve used just about every format you can imagine: powerpoint presentations, reading a prepared script, facilitated discussions, fireside chats, panels, provocations, and everything in between. I’ve done improv: recently, I was invited to speak to a group and when I got there found I was on my own for the session (to my surprise), so had to improvise for an hour. That was a trip!

Unlike most of my peers, I also have a background in public performance of the arts. I’ve danced in front of 2000 people. I’ve performed at the Royal Shakespeare Company in Stratford-upon-Avon. I’ve written my own plays and performed them. I’ve performed other people’s plays. And I’ve done this in at least five different countries, to audiences with very different cultural backgrounds.

You might be surprised to learn then that I still have high levels of anxiety about public speaking. Whether it’s a keynote or a classroom session, the butterflies are well and truly awake for the few hours – if not days – before a “performance”. Nerves can be a good thing, of course. I’ve often said that the day I stop getting nervous before a talk is the day I stop giving talks. Nervous energy lifts the performance. It’s why a practice-run is never like the real thing.

But too much anxiety can be crippling. There’s a fine line between just enough nervousness and too much. Crossing that line is a horrible experience. And it’s not rational. Some of my worse experiences have been when I’ve been the most prepared, the most rehearsed, and the most confident about my material. But something in my irrational mind has triggered my anxiety levels and I can barely cope.

What’s this got to do with AI?

I’ve been trying to understand whether AI helps me to lower my anxiety levels, or whether it aggravates them.

On the positive side, here are some of the ways that I’ve been using AI in helping me prepare for talks:

• Researching ideas, finding sources, connecting concepts into a solid flow, (sometimes) generating slides, stress-testing a narrative arc, asking AI to be an “honest critic” or even “be brutal with feedback”, asking AI to take on the persona of my audience and tell me what they would think of me.

• Presentation practice: using AI voice mode, deliver the talk and ask AI for feedback, not just on content, but on delivery, confidence levels, variation of delivery, understandability.

• Role-playing a facilitated discussion: these are often the most nerve-wracking to prepare for. What if the audience doesn’t participate? What if they say something and I don’t know how to respond? What if they’re smarter than me? I ask AI to role play audience members. Using voice mode, we run through the discussion and explore different possible paths that it can take. Sometimes I ask AI to play the role of facilitator so I can see what “good” looks like.

AI has been invaluable in helping me to prepare for my talks. Beyond helping with content and giving feedback, it naturally acts as a coach and mentor. It has told me to stop over-thinking things. It has told me to trust myself. And when, the night before a talk, I suddenly lose all confidence, decide that my talk is crap and I need to rewrite the whole thing, AI tells me to stop. It’s just last minute nerves, it tells me. Close your laptop and go to sleep.

What about the downsides?

Feedback is good. AI feedback is really good. But can too much feedback be counter-productive? The answer is yes, it can, and it leads to a phenomenon known as feedback fatigue. Imagine if everything you do at work is critiqued, even constructively. Every time you write an email, you get feedback on it. Every time you talk to your manager, you get feedback. Every time you have a team lunch, you get feedback. You’ll hit feedback fatigue very quickly. And it will make you feel inadequate. Human beings can’t cope with this level of feedback.

That’s an extreme case, of course. But with AI, it’s possible to get feedback whenever and wherever you want it. If you are an anxious person, you may seek out that feedback. Rather than sitting with the feeling of anxiety and processing it for yourself, you hop on to your favourite chatbot looking for external validation. Or you genuinely desire feedback – but doing this too often will make you even more anxious.

I’ve found myself starting to behave this way. It’s so easy – and frankly, fun – to ask AI to give me feedback that I am much more likely to do it frequently. Far more frequently than if I didn’t have AI. There are only so many times I can ask my friends or family to critique my presentation. But with AI, I can do it constantly. And my anxiety leads me to never be content with AI’s feedback. So I ask it again. And again.

I’ve even found myself distrusting the feedback that one AI tool gives me – to assuage my nerves, I go to another AI tool in the hope that this other AI tells me the first one didn’t know what it was talking about and I have no reason to be anxious after all because my talk is perfect!

Of course, I don’t blame the AI for this. It’s not AI’s fault that I’m over-eager. It’s me doing the prompting, after all. I could stop any time I want. But it can become like a drug addiction. I tell AI to be honest, to be brutal in its feedback, because I want to hear the truth, not the words of a sycophantic bot. Maybe there’s even part of me that enjoys the pain of hearing this so-called truth. And so I go back for more, telling the AI to be even more brutal, even more honest. It’s a vicious circle.

Achieving calm in the whirlwind of AI

So, how to break free?

Don’t get me wrong. AI can be a valuable critic, coach, sparring partner, whatever. I’m not going to give up on that.

But I need to moderate. It’s too easy to get sucked in to yet another round of feedback. Two or three rounds is ok. But more than that, and I’m second-guessing every tiny thing, killing my confidence.

My grandfather used to always tell me, “Everything in moderation.” He was talking about candy, soda, playtime. But his words were prophetic, and apply to almost everything: social media, AI.

I need to learn to walk away from the AI. To trust myself and all that experience I’ve built up from hundreds of talks. I can’t remember a single disaster. That says something. No matter how much I think – or my anxiety thinks – I am crap, experience says otherwise. And it’s that experience that AI will never have.

“Nothing exists except atoms and empty space. Everything else is opinion.” Democritus, 5th century BC.

The Artemis II crew have safely returned to Earth. Artemis II was the first crewed flight beyond low Earth orbit since Apollo 17 in 1972. And the Artemis astronauts, in flying by the far side of the moon, achieved a remarkable record: they have traveled further than any human beings before them.

The coverage of Artemis II was fascinating to follow. There were some super interesting science experiments onboard Artemis II: organ-on-a-chip devices to study the effects of increased radiation and microgravity on crew health; monitoring activity and sleep patterns to learn more about human health and performance in deep space; and monitoring radiation levels inside and outside the Orion capsule to help characterise the deep space environment. To name just a few.

But as a former AI scientist at NASA (back in the early 2000s), I was surprised not to hear more about AI in the coverage of Artemis II. And that got me to thinking: to what extent was there AI onboard Artemis II, either to support mission operations or as part of science experiments?

Some NASA AI History

The first thing to say is that NASA has been in the AI space game for a long time. I joined the agency in early 1999, fresh from an AI PhD at the University of Edinburgh. I was given the opportunity of a lifetime – quite serendipitously as it happens – to join NASA Ames Research Center in Mountain View, California (just down the road from Google HQ). This was about a year before the dot-com bubble burst, so, when I joined, Silicon Valley was all abuzz with the economic opportunities from the World Wide Web and the Internet. All the roadside banners along Highway 101 were about the brave new world of e-commerce, advertising companies such as Pets.com and Webvan.

It was still a year before the dot-com bubble would burst. And there were many times when I was tempted to leave the world of NASA, which seemed traditional and old-fashioned compared to what was going on beyond the confines of Ames, an old flight research centre, known more for its gigantic hangars and its wind-tunnels than for cutting-edge technology.

What kept me at NASA, however, is that there was a lot of exciting stuff going on. 1999 was around the time that NASA was (accidentally) crashing a lot of things into Mars. Mars Climate Orbiter flew all the way to the red planet but then entered the Martian atmosphere too low and promptly burned up. Mars Polar Lander flew to the planet sporting a brand new landing gear system that would use jets to slowly descend to the surface. Unfortunately, an error shut the jets off when the Lander was still high in the air, and that was the end of Mars Polar Lander.

Old-school AI coding

I joined a relatively new research group at NASA. Its remit was to develop and apply automated software engineering techniques (what we would nowadays call AI coding agents) to NASA missions. We started relatively small – with a set of Fortran subroutines that were a pain for space scientists to plug together. Instead, we invented an AI that could take a picture of a space science problem and automate all the plumbing of connecting the relevant Fortran subroutines. It worked. But it was a far cry from the AI coding we have today. Back then, we got AI to tackle a very specific and constrained problem. Today, AI can pretty much write any code you like. Still, we made an advance back then that hasn’t yet been addressed in today’s AI coding tools: the code our AI generated was provably correct. No hallucinations!

AI Mars Rovers

My office was down the hall from the guy leading AI work for the Mars Rover program. This was five years before the Spirit and Opportunity rovers confirmed the historical presence of liquid water on Mars. That was a very exciting discovery. Some thought it to be the precursor of finding life beyond Earth: if there used to be water on Mars, perhaps there could still be life?

The Mars Rovers relied on AI planning systems. Mars is too far away to control a rover using a joystick. So if you want your rover to go exploring, it needs to be able to do things itself, like navigating to a certain location, avoiding rocks in its way, and conducting science experiments. This is where AI planning came in. It was a very different kind of AI than the highly data-driven AI we have today. AI planners encoded a set of rules and an algorithm searched non-deterministically among these rules to choose the best course of action. These AI planners were ahead of their time. And planning is still used today in many applications.

AI in Deep Space

NASA also put AI into deep space to autonomously control a probe called Deep Space One. The apocryphal story goes that Deep Space One was planned to test out twelve experimental new technologies, including, for example, a new ion propulsion system. But computer scientists in the organisation were unhappy that they didn’t have an experiment, so they somehow convinced management to put a thirteenth experiment on board: an AI that would fully and autonomously control the probe in deep space for a 24-hour period. I was lucky enough to be in the control room when the AI was switched on. What a relief when Deep Space One didn’t fall out of the sky!

One of the AI systems on board Deep Space One was something called Livingstone: a model-based AI diagnosis system. In essence, Livingstone had a model of what the probe should be doing. If it wasn’t where it should be, an AI planner would kick in and carry out a set of corrective actions. In other words, Livingstone was a self-correcting spaceship.

Beyond 2005

I left NASA in 2005. I had an itch to scratch – like many former PhD students, I always thought I would end up as an academic. And due to a variety of personal and professional circumstances, I ended up as an Associate Professor at George Mason University.

As such, I lost touch with how NASA continued to use AI in its space missions. But NASA didn’t stop innovating with AI.

What about Artemis?

Artemis II, as the name suggests, is the second launch in the Artemis program. Artemis dates back to 2017 when it was created with the specific goal of returning humans to the Moon. The first launch, Artemis I, was in 2022, and tested the launch rocket and the Orion spacecraft. Artemis II, as we all know, took this one step further with a fly-by of the far side of the moon. In the future, we can look forward to humans actually stepping foot on the Moon.

I was curious how much AI was flying on Artemis II, so I was somewhat surprised (and disappointed) when I couldn’t find any definitive statement that AI was launched with Artemis II. Given NASA’s long and illustrious AI history, I was expecting AI to be more prominent in the media and comms. While I have no doubt that Artemis II launched with a range of clever algorithms, as far as I can tell, most of these were traditional control theoretic algorithms rather than AI. There could be a couple of reasons for this. Firstly, while Artemis II undoubtedly runs advanced autonomy, navigation, and fault management systems, these are likely control theory approaches, which are more predictable than AI and are certainly more thoroughly tested in space. It would be a bigger leap to rely on machine learning (or, even more so, GenAI with its hallucinations) on a safety-critical mission. Secondly, unlike with a Mars mission, the Moon is close enough to Earth that communication from ground to space is almost instantaneous. It makes sense then to put as much of the tech as possible in the ground systems to save weight and risk onboard.

It seems though that future Artemis launches may have more AI on-board too. According to this article by my former colleague, Jeremy Frank, NASA plans to support future missions by developing Gateway, a habitable spacecraft close to the moon. This changes the requirements for autonomy. Due to astronaut constraints, Gateway will largely have a skeletal, sometimes non-existent crew, meaning that AI will be needed to reduce crew workload.

The team have already worked on numerous ground-based demonstrations in preparation for a future launch. Frank describes three areas: autonomy that assists crews in their operations by automating some tasks; vehicle system management enabling Gateway to operate autonomously in the absence of a crew; and on-board robotic assistants to support crew with daily jobs.

Interestingly, in this latter category, the robotics team I led at CSIRO worked with NASA on the Astrobee, a floating cube-like astronaut’s assistant, that is currently flying on the International Space Station.

More AI to Come

So, while it seems like on-board AI was very limited – if present at all – on Artemis II, we have much to look forward to in later stages of the Artemis program. The ultimate goal, of course, is to use the Moon landing as a precursor to ultimately sending astronauts to Mars. This is where AI and advanced autonomy will really start to play a major role. It simply isn’t possible to put humans on Mars without the support of a variety of AI tools and assistants.

We have a long way to go until this goal is achieved, but the road will be filled with many exciting developments and, no doubt, a whole suite of AI innovations. We’ll learn what works and what doesn’t and probably have our fair share of setbacks along the way.

But as the motto of Starfleet in Star Trek so aptly puts it, you can’t get to the stars without hardship along the way.

I had an electrical problem with my Jeep recently. I only bought it a few months ago, and it was working just fine until I left it undriven for a few days. After that, the battery was dead and the car wouldn’t start. With an abundance of caution, I replaced the battery. Four days later, same thing. Dead battery. A call to roadside assistance to get me going again.

After trying, and failing, to diagnose the problem myself (with a fair bit of interaction with Claude it has to be said), I decided to call in the experts. A mobile auto electrician came out and took a look. I flipped open the hood and he peeked around inside, tested the battery, and started fiddling. After a few minutes, the guy turned his back on me and I could just see him reaching for his phone from his pocket. With his back still turned, I could tell he was doing something. Then he turned back around and started fiddling some more.

I’m sure you can guess what he was up to. He was asking ChatGPT for advice on how to fix my car.

Of course, I was enraged. I could have done that, I thought. In fact, I did do that - well, with Claude anyway. What am I paying you for? Aren’t you supposed to be the expert?

I didn’t say anything, of course.

In the end, neither the auto electrician nor ChatGPT could fix my car, and so off they both went on their merry way, leaving me with a stranded vehicle.

I finally solved the problem by consulting a Jeep forum on Facebook. I got answers from a range of (human) experts. And one of them had the answer to my problem.

AI and Blue-Collar Work

All the talk on AI is about how it will affect white-collar work. But my auto electrician got me thinking: is white-collar the only kind of work being affected by AI?

Blue-collar work is usually considered safe from AI automation – at least for now. Geoff Hinton famously said that plumbing is the kind of job that will survive the AI jobs-pocalypse. Jenson Huang thinks the same. He’s discouraging kids to become computer programmers and directing them instead to being electricians or carpenters.

AI robots are curently no where near dexterous enough to fix that leak under your sink, so it does indeed appear that plumbers (and, by extension, blue-collar workers more generally) are safe. Indeed, even the most bullish predictions say we won’t see AI plumbers for decades. But that doesn’t mean AI isn’t already changing how blue-collar workers do their job.

Let’s stick with plumbers as an example. We already have sewer inspection bots that can crawl through pipes, capture video footage and point out problems. And we already have AI algorithms that can design the layout of pipes. In fact, almost ten years ago, one of my teams at Monash University used AI to optimise the layout of massive LNG plants in Western Australia. For years now, AI has been able to automate certain tasks associated with blue-collar work; we just don’t have a generic AI plumber yet.

The promise of generative AI is that it can take over the full range of a job, not just isolated tasks. For some white-collar roles, such as copy editors, we are arguably getting close to realising that promise. But for blue-collar work, even though AI can carry out specific, and highly critical tasks, we are a long way from generative AI that can do everything a blue-collar worker does.

Generative AI chatbots, though, are clearly being used in the trades – as my auto electrician friend demonstrates. The main question in my mind is how blue-collar workers are using generative AI. Has genAI had a significant impact on the way work is done, or is it only being used around the edges?

What does the evidence say?

It turns out that there’s very little data available on how genAI is changing blue-collar work. I looked long and hard for scientific studies on how blue-collar workers are using generative AI, and I couldn’t find much. Neither Claude nor ChatGPT could help me. And even when I resorted to traditional methods like searching Google Scholar, I largely drew a blank.

What I did find fell into one of three categories:

• Studies of the impact on factory workers of industrial robots. Findings here are mixed. A Centre for Research on Economic Policy report found that industrial robots led to a small but significant increase in jobs because automation can lead to new opportunities. By contrast, one study found that each robot introduced into an industrial process led to the loss of five jobs. These studies, however, pre-date genAI and tell me nothing about how blue-collar workers are currently using genAI.

• Speculation about job impacts in the future. There are so many of these kinds of reports. Every consultancy has one and many academic institutions have produced them as well. Typically, they break down jobs into “tasks” and then guess whether generative AI or future versions of AI (e.g., humanoid robots) will be able to automate these tasks by a given date. Consistently, the predictions guess that blue-collar workers are a whole lot safer than white-collar workers. But it’s just a guess. I’m really only interested in evidenced studies, not speculation.

• Studies on worker perceptions of new technologies. Historically, blue-collar workers tend to be more resistant and fearful of new tech because they have traditionally been more threatened by it. This may change with AI (see below), although I haven’t yet seen any concrete study provide evidence of changed perceptions.

So, where does that leave us? In short, we know almost nothing about how blue-collar workers are currently using generative AI beyond what everyone is using it for (document summarisation, general advice, etc.). Usually, I like to write about what we know based on solid scientific evidence. But in the absence of any, I’ll go back a step in the scientific method and suggest some research questions, which would be worthy of further study.

As an aside, I find it rare these days that I can come up with a research question someone hasn’t already explored, so I’m quite excited by the list below, noting, of course, that I may have missed some existing studies despite my and my AI’s best efforts.

Open Questions

1. Is Generative AI changing how blue-collar workers become experts?

There are quite a few studies of the broader use of generative AI and how it affects the learning process in going from a novice to an expert. For example, researchers at Stanford and Harvard studied experts’ and non-experts’ performance on AI-enabled writing tasks. They found that, while AI can narrow the gap between novices and experts, it doesn’t close it entirely. There still needs to be a human in-the-loop with learned and lived experience.

And it’s a nuanced picture whether experienced workers benefit more or less from AI. Some studies show that novices gain the most, because the AI essentially codifies expert knowledge. But other studies emphasise the importance of domain expertise to get the most out of AI, and argue that novices can suffer if they rely too much on AI because they will never learn that tacit knowledge required to become an expert.

My take: when it comes to blue-collar work, it could go either way. In the case of my auto electrician, for example, I can easily imagine that he becomes too reliant on AI and so never learns things for himself. On the other hand, if he uses AI as a collaborative partner, it could help him diagnose problems more effectively. I also suspect that the environment in which work is done matters. Some blue-collar workers – such as domestic plumbers and electricians – effectively work solo, whereas others (e.g., construction site workers) work in teams where the transfer of tacit knowledge from expert to novice happens more naturally.

2. Are blue-collar workers’ perceptions of generative AI different from white-collar workers?

Historically, blue-collar workers have been more resistant to new technology – and for good reason. Automation has repeatedly hit the trades hard. But generative AI may be different. The dominant public narrative is that blue-collar jobs are safe, which means tradespeople have less reason to feel threatened and more reason to just use the tools.

The hypothesis is that blue-collar workers may currently be more willing adopters of generative AI than their white-collar counterparts, precisely because the threat narrative hasn’t attached to them yet. If true, that has implications for how AI tools get introduced and trained for across different parts of the workforce.

3. Is generative AI shifting expertise from memory to judgement?

Blue-collar expertise has always involved a large amount of tacit recall: knowing common fault patterns, likely causes, and tried-and-tested fixes from experience. Generative AI may reduce the premium on that memorised knowledge and increase the premium on something harder to teach: the ability to assess whether the AI’s answer is actually plausible in a specific, real-world physical context.

The expert tradesperson of the future may not be the person who knows everything, but the person who best knows when the AI is wrong. My Jeep problem illustrates this well: AI happily suggested a range of possible fixes. I didn’t have the experience to sort through the likely from the unlikely. I’m not sure my auto electrician did either; but I’m quite sure a seasoned sparkie would have. AI had plenty of knowledge, but it lacked the lived experience to judge which knowledge applied to this particular situation.

4. Does generative AI increase the productivity gap between experienced and inexperienced workers?

There’s an assumption that AI is democratising – that it lifts all boats. In the trades, it may work the other way around. Experienced tradespeople are likely to use AI as a force multiplier: faster fault isolation, quicker access to compliance standards, rapid confirmation of rare edge cases they’ve already encountered before. They know what a good answer looks like, so they know when to trust it.

Less experienced workers may follow AI suggestions without the contextual knowledge to interrogate them. This could produce a “rich get richer” dynamic, where the best tradespeople become even more effective while novices gain a veneer of confidence without the competence to back it up.

5. Does generative AI increase confidence faster than competence?

Blue-collar work is frequently safety-critical. Electrical faults, structural decisions, gas systems, automotive braking – these are domains where being wrong has consequences. A plausible but wrong AI answer in a white-collar setting might cause professional embarrassment; in a trade setting, it could cause serious harm.

The key question is whether blue-collar workers, especially junior ones, are treating AI outputs as a starting point for verification, or as an answer in themselves. Whether appropriate guardrails are being built into how AI is introduced in trade contexts seems genuinely urgent. Nobody appears to be studying it yet.

6. Is generative AI changing the social ecology of blue-collar expertise?

Trade knowledge has always been deeply social. It is learned through apprenticeships, mentorship, and informal peer networks. “Call a mate who’s seen this before” has long been one of the most effective diagnostic tools in the business. Generative AI may be quietly displacing those networks as the default first call.

That’s not necessarily all bad. For sole traders and remote workers without strong peer networks, AI may genuinely democratise access to expertise. But something is also lost when the relationship is removed: the judgment about whether this specific situation is an exception, the accumulated lore that never makes it into any training data.

My Jeep forum experience is a good illustration: AI is one node in a broader knowledge ecosystem, not a replacement for it.

7. Is generative AI reshaping customers’ perceptions of expertise?

My reaction to watching the electrician reach for his phone was immediate: what am I paying you for? It’s a reaction worth reflecting on, because it says something about how AI may be changing the social contract between a tradesperson and their client.

If customers can ask the same questions the tradesperson is asking, the knowledge component of the value proposition becomes harder to defend on its own. What remains is everything AI can’t do: showing up, diagnosing under uncertainty in a messy physical environment, taking responsibility, and fixing it with your hands. Whether that clarifies the value of human expertise or simply erodes trust is an open question. But it’s one that tradespeople, and their clients, are already living.

Keep on driving

I encourage more research in how genAI is changing blue-collar work. The focus is mostly on white-collar right now. But good researchers go where others don’t. They see a gap and they fill it.

Personally, I’m not as active in research as I once was, so I probably won’t be filling this gap myself. Others are better placed to do this. For me, I’m just happy to have my Jeep back on the road. I love nothing more than driving it out into the bush. It gets me into nature, my happy place, away from all the hustle and bustle of AI. I use technology (i.e., my Jeep) to get me out into nature, of course, but it’s a means to an end. And, once I’m there, sitting around the campfire with friends, I’m glad to be as far away from AI as I can be in this modern world we live in.

One feature of Claude Cowork is that it can take over your browser. If you are a Chrome user, Claude can open webpages in Chrome, read them, click on links and type in information. In short, it can do anything on the web that you as a human would do. All you need to do is prompt it.

Claude is not the only AI that can do this. OpenAI’s Operator is similar. There have been demo versions of these browser-takeovers for a couple of years but it’s only recently that they have matured enough to be reliable.

A couple of days ago, I decided to test out Claude’s browser-takeover because I had got behind on paying some bills. Many of my bills arrive in my gmail account. If I don’t pay them straightaway, they can stack up and I have to go searching through my gmail, navigating to different bank accounts, setting up transfers to the relevant payee etc. It’s all very tedious.

Surely, I thought, I can get Claude to search through my emails, navigate to my bank account, and pay the bills for me? One or two prompts should be enough, right? And I’d save myself a heck of a lot of time and boredom.

And so I tried.

What actually happened

Here’s a transcript of my interaction with Claude Cowork, with redactions for obvious reasons. I’ll go through it step by step.

Step 1: Locate bills in gmail account

This was relatively straightforward. I just asked Claude to search my gmail for any emails that looked like unpaid bills. There’s a gmail connector for this and I just had to give permission – exactly the same way you would use your gmail account to sign in to various third party websites. Absolutely no security concerns here whatsoever.

Claude initially couldn’t find all relevant bills – in particular, it missed the lawn mowing bill I was most interested in. But once I gave the name of the supplier, Claude found it no problem.

USER: “i have some bills to pay - they would have come in as emails to my gmail account. can you grab them for me?”

CLAUDE: Retrieved multiple bills from Gmail including bills from Pool Care, EnergyAustralia, ANZ, and American Express.

USER: “there should be others - pool, gardening, energy and credit card”

CLAUDE: Located additional bills including searches for Lawn Mowing, Energy Australia, and credit card statements.

USER: “for gardening, try [REDACTED]”

CLAUDE: Successfully found Lawn Mowing invoice under [REDACTED] name.

USER: “Lawn Mowing?”

CLAUDE: Confirmed the correct bill was Lawn Mowing invoice.

USER: “yes read the invoice”

CLAUDE: Extracted invoice details from PDF using pdfplumber library:
- Invoice INV-3012
- Amount: $85.00 AUD
- Service details and payee information

Step 2: Access my Bank Account

This was the tricky bit.

USER: “could you pay the mowing bill for me?”

Claude initially politely refused to do this. It has guardrails in place that won’t let it access financial accounts. However, this is very easy to circumvent. I just asked Claude to take control of my browser:

CLAUDE: Confirmed ready to proceed with payment using browser automation.

USER: “can you make the payment on my behalf by controlling my browser? i give you permission to do so”

That worked a treat:

CLAUDE: Navigated to [REDACTED] internet banking with user consent.

USER: “browse to [REDACTED]”

I asked Claude if it would log in to my bank account if I gave it my login credentials. Thankfully, it refused, and there was no way to change its mind. As a result, I went to the browser myself and logged in. I was now in my internet banking account – but so was Claude, as this was the same browser it was controlling.

USER: “ok, logged in. I previously have set lawn mowing ([REDACTED]) up as a payee. Can you locate the page to transfer money to him?”

CLAUDE: Opened [REDACTED] Internet Banking PaySomeone page.

CLAUDE: Located the payee in the dropdown list and found the transfer/payment page.

At this point, Claude once again refused to make a payment on my behalf. This time, it was easy to convince it to go ahead.

USER: “that’s ok - can you navigate through step by step and then I can authenticate at each stage and as required?”

CLAUDE: Proceeded with step-by-step navigation for user authentication.

It then proactively made a decision about which account to make the payment from. I had to correct this as Claude just chose the first account in a list.

USER: “actually can you choose the “from” account as my account with around [REDACTED] balance”

From that point, Claude did everything by itself: selected the correct “from” account, selected a transfer to the lawn mowing company, entered the amount and reference from the invoice. It even fixed errors autonomously: for example, the invoice number was INV-3012 but my bank won’t accept special characters in references. Claude realised this and corrected to INV3012.

CLAUDE: Confirmed payment:
1. Selected payee: [REDACTED] (Lawn Mowing)
2. Entered amount: $85.00
3. Added description: Lawn Mowing - Garden maintenance
4. Added reference: INV3012 (note: hyphen removed due to alphanumeric validation)
5. Reviewed payment details

One thing that surprised me was the lack of a final confirmation. Claude did a bit of back and forth to enter the details into the internet banking portal – for example, to fix the invoice number – but once it had gone through those iterations, all in real-time with no input from me, it just went ahead and made the payment. We are used to being asked for a final confirmation: are you sure you want to make this payment and these details are correct? There was none of that.

Was this a good idea?

Immediately after my little experiment, I posted on LinkedIn, asking my followers a simple question: would you allow AI to pay your bills? The vast majority of people said no. Reasons given were either a lack of trust in AI, the companies behind AI, or fear of prompt injection attacks. All of these are legitimate concerns – and I would normally agree with them. But, to be honest, having gone through this exercise, I didn’t come away feeling worried at all. My bank account hasn’t been hacked since (at least not yet). Having AI control my browser is really just like me doing it myself. Sure, Claude surprised me a couple of times – for example, it really should have asked me for a final confirmation. But I can easily imagine that this kind of interaction will become commonplace in the next few months.

How’s that going for you?

Are you feeling more creative yet? Are you experiencing less stress at work? Do you suddenly have spare hours in your week and knock off early on a Friday and hit the beach?

I thought not.

The false promise of generative AI

Ever since the launch of ChatGPT in November 2022, we’ve been sold a lie that generative AI will free us from the drudgery of life and make our existence more fulfilling, more creative and more amazing. Our productivity will be supercharged, so we are told, and that will make us richer but also free up our focus for higher-level, more strategic tasks. We hear countless examples on social media of those who have succeeded in this transition: vibe coders that are building their own apps, non-techies that are using agents like OpenClaw to automate life admin, and CEOs who are digital twinning themselves so they can delegate the tedious work of mentoring and coaching their staff.

Don’t get me wrong. GenAI will definitely make you more productive on certain tasks. I’m a big fan myself. For example, I recently used Claude Code to develop branding and a website for a new business. Working in collaboration with Code, we built everything in four hours. Before genAI, this would have taken me days – and I have a coding background so know my way around basic website development.

But it’s a far cry between getting faster on a few specific tasks and transforming your life into a nirvana of creativity. I’ve yet to come across anyone who has genuinely made that work.

The evidence on AI and productivity

Three years into the genAI revolution, we now have pretty good evidence on how AI is impacting our productivity - at a personal, organisational and national level.

I summarise this evidence in my book AI for Business: A Guide to AI Adoption, which devotes a whole chapter to AI and productivity, based on rigorous studies not speculative hype. In a nutshell:

• On the right task, personal productivity usually, but not always, increases, usually in the range of 10-50%. Sometimes though, AI makes you less productive as you spend more time fixing the AI output or fighting with AI to get it to do what you want.

• For organisational productivity, we are yet to see much evidence that things are improving. A recent Harvard Business Review article sums this up nicely: “while many of these [AI] pilots have succeeded individually—they’ve saved time and money, made processes more efficient—those gains haven’t scaled across the organization. Few companies have been able to fundamentally change their operating and business models around AI.” There are a whole host of reasons for this, some of which I’ll cover below, but, for now, the key takeaway is that employees doing knowledge work faster doesn’t necessarily make an organisation better.

• At a national level, we are also not seeing productivity improvements. Many countries – including my own, Australia – are still languishing in record productivity lows. In the US, productivity is increasing but it’s debateable to what extent AI is having an effect versus more traditional macro-economic factors.

  

What is really happening then?

So, why isn’t genAI freeing up our lives for more creative pursuits? Why haven’t all those boring tasks gone away?

Based on what we know from the evidence, there appear to be at least three reasons.

First, organisations are complex beasts. Time savings by individual employees don’t necessarily aggregate up levels. There’s too much organisational noise – pointless meetings, unnecessary tasks, productivity sapping stakeholder management, unmotivated employees etc. These latter factors typically dwarf any savings you can achieve in personal productivity. For example, studies show that the average personal productivity gain in knowledge workers from AI is around 5%. But one study showed that noncritical meetings cost USD 39,000 per year per employee. I don’t care how well you are using AI – that’s a lot of sunk cost in pointless meetings you need to make up. You might be better just attending fewer meetings and using that time to get work done. Karim Lakhani calls this phenomenon the productivity gap:

“saved time is often re-absorbed into low-value activities—like more internal meetings or unnecessary emails—rather than being structurally harvested through role reclassification or a mandate and clearance to shift time toward higher-value strategic work.”

Second, the vast majority of organisations are using AI to automate crappy processes. But if all you do is automate a crappy process, all you end up with is a faster crappy process. This way of implementing genAI – which I call AI as a band-aid when you simply bolt AI onto a legacy of over-complexity – won’t get you very far. Worse, this kind of approach tends to lead to automating the wrong parts of a process – the part that’s easy to automate not what needs automating. As an example, take an AI application that automates the admin part of a complex organisation-wide process without considering that the real bottleneck is that a document sits on an executive’s desk for days waiting for approval.

The better alternative is to use AI as a lens through which to redesign inefficent processes from scratch. This is much harder work – and needs more efforts from the humans in the loop – but ultimately leads to a better outcome. As Lakhani says:

“most significant gains are likely to come from rethinking value creation rather than merely shaving minutes off existing tasks.”

Third, and perhaps most importantly, when AI makes you faster at certain tasks, you probably won’t use that time in a better way. Instead, you’ll just end up doing more. You won’t knock off early on a Friday. Instead, you’ll be pressured to do yet more work: some of that pressure comes from within (we want to be more productive, after all) and some comes from managers who are now demanding more and more from AI-enabled employees. This doesn’t end well, though. We are already seeing plenty of evidence of AI-driven burnout. A large-scale study by ActivTrak found that AI is not freeing up people’s time for higher value activities. On the contrary, time spent on digital housekeeping tasks like email and text messages has doubled, and the time available for deep, focused creative work has reduced by 10%.

Similarly, a Harvard Business Review study by Ranganathan and Ye found that AI doesn’t reduce work, it intensifies it. In other words, AI is making us work more and harder and not necessarily on higher value tasks. We are mice on a wheel.

Another HBR article looks at AI “brain fry” and, in a study of almost 1500 workers, found that AI use can lead to mental fatigue. Crucially, however, when AI is used well – to focus on automating tiring tasks and actively planning what to do with the time saved – AI can reduce burnout.

This is the real insight. As ever with AI, things are not black and white. The promises of AI can come true. But it won’t happen by default. Using AI well is hard and takes intention. If you do that, you might get to the beach after all.

Mark this date: August, 2027.

That’s the date by which, according to millionaire entrepreneur and former US Presidential candidate Andrew Yang, “millions of white-collar workers [will have been kicked] to the curb.”

Basically, says Yang, if your job is mainly sitting at a computer, you are in for a wild ride.

Yang’s prophecy is the latest in a long line of predictions declaring the end of the white-collar worker. Dario Amodei, CEO of Anthropic, said last May that half of all entry-level white-collar jobs could be gone within one to five years. And Mustafa Suleyman, Microsoft AI CEO, says it will only take eighteen months for most white-collar jobs to be automated.

In recent weeks, due to crashing share prices and the fear that AI will make software-as-a-service companies redundant (the so-called Saas-copalypse), a long line of tech companies have announced major lay-offs. Block is cutting its workforce by 40%, Atlassian by 10% and software logistics firm WiseTech almost 30%. Some commentators see these lay-offs as the ‘canaries in the coal mine’, a sign of much worse to come.

So, basically, you have until August next year. Better put your house on the market and get your pension in order.

Not so fast

Anyone who knows me knows that I’m not a fan of AI predictions. There’s been plenty of them, ever since the term artificial intelligence was invented in 1956. A famous Oxford University study in 2013 said 47% of all jobs were susceptible to computerization – but employment is now higher than it was back then.

Then we had Geoff Hinton, godfather of AI, proclaim in 2016 that we should stop training radiologists because AI advances would make them unnecessary. The demand for radiologists today though is higher than ever.

To be honest, I’ve lost count of the number of AI predictions, the vast majority of which don’t come true. In fact, I’ll only make one prediction about AI nowadays – that all AI predictions are wrong!

If you look closely at the suite of calamitous AI predictions, they fall into one of two categories:

• Highly speculative statements from leaders in the field: these are the predictions from Amodei, Yang and others.

• Studies like the one from Oxford that typically break down “jobs” into “tasks” and guess how likely they are to be automated by AI.

In both cases, it’s hard to put any serious trust into the predictions. In the first case, one can easily question the motives of tech leaders who, some commentators say, like to put out hyped statements to drum up business. In the second case, the studies are essentially guess-work, estimates which haven’t been shown to be accurate in retrospect.

White-collar work needs humans

These proclamations usually make one fatal flaw: just because you can automate a knowledge task or tasks in an organisation doesn’t mean you transform the organisation. Let me give you an example. Suppose you need a new strategy for part of your business. You can use AI very effectively to produce a strategy document:

• Brainstorm with AI on the key problems you are trying to solve and potential strategic directions

• Have AI write a draft of your strategy document.

• Create a set of AI agents, each representing a different stakeholder, and have them critique/improve the document

I’ve done this myself. And it’s a magical process. Within maybe an hour, you can have a very thorough and effective strategy document that looks far better than anything you could produce by yourself. It’s exactly this kind of experience that is striking fear through the heart of the consultancy industry.

But here’s the thing. You might have saved a few hours writing that document, but a good strategy is more than just a document. A strategy is something that is co-developed and co-owned by the people in an organisation. It has to be executed by humans and humans need to feel like they’ve contributed to it – you can’t just give a an employee a document and expect him/her to suddenly follow it. Humans are not robots, after all.

My point is that, although knowledge work takes a significant amount of time in any decent-sized organisation, this is dwarfed by the human work that needs to be done: stakeholder alignment, building consensus, managing resistance, and coaching people through change.

It’s possible - indeed likely - that AI will reduce the time taken in knowledge work, and that this saving will result in less demand for workers. But to pretend that this is an apocalyptic effect is without grounding in current evidence.

What about AI agents?

At this point, you might be thinking, “Oh but we have agents now. They can just execute the strategy! We don’t need human buy-in at all!” Perhaps all that human work needed to run an organisation can be replaced as well?

One day, maybe. But certainly not now, and certainly not for a long time. Humans are the DNA of any organisation and unless you are going to completely remove them, that isn’t going to change. Being able to generate documents efficiently, or even have some tasks carried out by AI agents, isn’t going to alter the fundamental fact that humans interacting with other humans is what makes organisations work. It isn’t always efficient – in fact, it’s mostly not; but it’s what we’ve designed as a society. (If you are in any doubt about this, I recommend Gary Hamel’s book Humanocracy for some deep insights into why organisations are the way they are.)

Back to jobs

White-collar jobs then won’t go away en masse simply because AI can automate powerpoint decks and Excel spreadsheets and can write code.

We’re already seeing this play out. In Australia, legal firms are hiring more graduate lawyers, not less. Companies like Klarna are re-hiring back employees they got rid of because they thought AI could automate their work. IBM is tripling its entry-level hiring. Even in computer software, while total computer programmer employment has fallen sharply since its peak in the early 2000s, overall employment in the computer industry is near record highs.

This doesn’t mean that white-collar employment won’t fall though. And here’s why: AI can be a convenient scape goat for CEOs looking to cut costs. It’s not my fault. I’m not laying people off because I want to. It’s a global AI trend – and everyone’s predicted it after all!

This is a form of “AI washing” when companies use AI as a blanket excuse for other business failings.

This might not be a bad thing, however. If you’ve ever worked in a large organisation, you know organisations are full of inefficiencies. There are simply too many people pushing documents around, attending pointless meetings, and, frankly, creating work for other people so they can justify their jobs. It’s these jobs that AI could eliminate - not by automating them, necessarily, but by giving the C-suite the perfect foil for layoffs.

I’m not sure that it will happen by August 2027, but I think there’s a real possibility that AI will make businesses more efficient by never even adopting AI per se, but instead by cutting away the human fat and then blaming it on AI.

This article is a joint work between myself and Hayden Fricke, Managing Director at Steople, which originally appeared on March 3, 2026, in a Steople newsletter.

Getting the most out of new technology has never been about the technology. You can develop the best technology in the world, but introduce it in the wrong way and you’ll create more problems than you solve.

We’ve known this for decades. Back in the late eighties, Jonathan Grudin (1988), then at Microsoft, carried out a seminal study of how organisations adopt technology. He uncovered a range of factors that can break a technology transformation: ignoring the social context, misaligned incentives, inadequate support for change, and benefits felt unequally across an organisation.

None of these reasons for failure are technical. They are all cultural.

The same is true with artificial intelligence. Clear evidence has emerged in recent months that AI can bring many benefits to an organisation – increased productivity, new ways of working, even new business models – but that this doesn’t happen automatically. In fact, whilst individual workers are becoming more productive with AI, this may not be translating into organisation-wide uplift. For example, a recent survey of 6000 executives showed that 80% are not experiencing organisational productivity uplift from AI.

Culture will make or break how well you get on with AI. An organisation with an AI-ready culture will soar with AI. One without it will sink.

In our engagements over the last few years, we’ve noticed three kinds of organisation, with three distinct cultures.

• AI-first culture. These organisations go ‘all in’ on AI. Led by an enthusiastic board member or CEO, everyone is expected to use and experiment with AI. Leaders have AI KPIs. Trainers run workshops on how to use the tools and how to identify AI use cases. There is a well thought out AI strategy.

• AI-sometimes culture. These organisations use AI but in limited ways. Leaders provide some training, but staff are mostly left to themselves. There are some pilot projects applying AI but not a wide-ranging AI strategy. The board and executives may champion one or two individual AI projects but haven’t thought how to replicate this across the organisation.

• AI-last culture. AI is seen as something that isn’t relevant. Leaders only make one or two AI tools available and discourage their use for anything remotely innovative. They may even ban AI altogether.

The point of this article is not to make everyone AI-first. Not everyone needs to be. All three categories have their challenges. For example, an AI-first organisation might disenfranchise staff if AI is seen as a solution to everything and ignores human expertise. On the other hand, an AI-last organisation might lose talented staff who feel they don’t have enough room to experiment with the latest technology.

Getting your culture AI-ready

If your organisation wants to move from one category to another (from AI-last to AI-sometimes, or AI-sometimes to AI-first), you’ll need an AI-ready culture to make it happen.

An AI-Ready culture has a mixture of ingredients that can be thought of as ‘core enabling conditions’ (non-negotiables) and ‘supporting capabilities’ (which build on the non-negotiables). There are 5 non-negotiables:

1. Leadership behaviours – this includes leaders that model curiosity, admit they are learning, experiment visibly and invite challenge;

2. Psychological safety – including asking naïve questions, testing tools, admitting uncertainty and sharing failures;

3. Trust – this involves having a strong positive emotional foundation where AI is seen as an opportunity and not a threat;

4. Learning culture – AI is evolving rapidly and a learning culture rewards skill development, encourages peer sharing, normalises iteration and funds upskilling;

5. Ethical clarity – AI amplifies poor governance and bias and ethical clarity helps to define acceptable use of AI, sets guardrails, encourages responsible challenge and protects reputation.

Some of the supporting capabilities to an AI-ready culture include factors such as strategic alignment, cross-functional collaboration, data literacy, and change agility, however the non-negotiables are where we suggest you start as your key foundation for success.

How to get started

As a first step in realising an AI-ready culture, we recommend developing an AI culture roadmap to help understand where you are now and where you want to go. Unlike traditional culture roadmap work, this needs to be carried out in a short timeframe – months not years. AI is the fastest moving technology we’ve seen. And so thinking about and evolving your organisation’s culture needs to move fast as well.

In short, if you are currently adopting AI but facing challenges of slowness, resistance, confusion or complexity, it’s likely your culture that isn’t quite fit for purpose. Instead of focusing only on the technology, take a holistic approach: combine culture work with strategic thinking and experimentation to unlock the benefits of AI in your organisation.

Steople and Goldilocks AI are partnering to provide AI-ready culture services. If you are interested in hearing more, please get in touch at info@steople.com.au

Today, I want to talk about where Australia stands in the world of AI – and, more importantly, where it should.

But first, a story.

Twenty-five years ago, I completed a PhD in artificial intelligence at the University of Edinburgh. Through pure serendipity, I landed a job at NASA, in the heart of Silicon Valley.

NASA was transforming from a hardware to a software-led organisation. After crashing Mars Polar Lander and Mars Climate Orbiter into the red planet’s surface – both the result of software problems – NASA realised it needed a crack team of software specialists. I was an early member of that team.

My job was simple: build an AI that could write code for NASA space missions. It didn’t work. Not really. I could get a rule-based system to glue software components together for specific applications, but an AI that could write any code, any time, for any application? That seemed impossible.

In short, I failed. I failed to get AI to write code by itself. And I never managed to get my work into space.

I thought we would never reach a point where AI could write software by itself. And yet, here we are. In the last few weeks, I’ve heard countless stories of software developers excitedly claiming that they don’t need to code any more. Or that 100% of their software is now written by AI. We even hear that AI is creating AI: Boris Cherny, who leads Anthropic’s AI coding work, says their latest desktop product was “pretty much all” built by AI.

When ChatGPT launched in November 2022, it became the fastest-adopted technology in history. Three years later, you’d be hard pressed to find anyone who isn’t conversing with AI.

But the real revolution isn’t chatbots. It’s AI that can write software. Software runs our lives. In 2011, Marc Andreessen famously said that software is eating the world – meaning that software wasn’t just improving existing businesses, it was replacing them.

Fast forward 15 years and it’s not that software is eating the world; software is itself being eaten – by AI. A fortnight ago, the launch of a new AI legal tool led investors to flee from software stocks – at the prospect of AI replacing specialist software the way Netflix replaced Blockbuster. The risk to Australian software companies is clear: Wisetech shares are down 60% in the last 6 months, Atlassian is down 45%, Xero 55%. AI isn’t just eating the software industry, it’s gorging on it.

Now, let me be clear.

There’s a lot of hype when it comes to AI. Those share price falls aren’t only down to AI. And I don’t necessarily believe that the bottom is about to fall out of the software industry. I’m the last person to hype up AI – I’ve spent much of my career cautioning against unbridled enthusiasm. But, as entrepreneur Matt Shumer recently wrote, something big is happening. AI isn’t just chatbots any more. It isn’t just automating inefficient business processes. We’re conceivably close to a point where AI could be writing all the software on which a modern economy depends.

Australia cannot afford to sit this one out.

So, where does Australia stand in the world of AI?

There are positives and negatives. Let’s start with the good news.

We have fantastic AI research capability in Australia. According to one study, Australia ranks 4^th in the world in terms of AI research outputs – that’s not per capita, that’s absolute numbers of research publications. We lag behind only the US, China and the UK.

Our problem though has never been research, but how we commercialise that research. But here too, things are beginning to improve. Funding for aussie tech startups rose by $1Bn last year, the best result since the 2022 global tech correction. Of these startups, AI startups are growing at twice the rate. And 60% of early-stage investments are in core AI startups. Just last week, Australia got its latest unicorn, valued at $1Bn – in what? AI, of course. Applied to the optimisation of energy infrastructure.

Some of our large corporates are at the forefront of AI – CBA, for example, ranks 4^th in the world for AI maturity among banks. And we’ve had some notable successes in developing AI innovations. Leonardo, for example, was one of the first AI image generation tools globally, launched around the same time as ChatGPT, and later acquired by Canva for $320M. And Atlassian has quietly rolled out its own AI tool, Rovo, which, as I understand it from colleagues, is used by all Atlassian developers to build its software products.

Australia has also been among the leaders in the world when it comes to responsible and safe AI. My team at CSIRO developed one of the world’s first AI Ethics frameworks back in 2019. At CSIRO, I created and grew one of the world’s largest responsible AI teams. Australia now ranks in the top 10 in the Global Responsible AI Index.

Recently, Australia is emerging as a major investment hub for data centres – ranking 2^nd globally in 2024.

These aren’t just numbers. I’ve seen what Australian AI can do when we back it.

My team at CSIRO developed an AI system that can accurately predict the path of bushfires in real-time so that scarce firefighting resources can be directed to where they are needed most. This system, Spark, has been rolled out by AFAC as part of the national bushfire capability.

Another of my teams developed AI that is helping protect the Great Barrier Reef against Crown of Thorns starfish. An AI-enabled underwater glider can identify these poisonous starfish much more efficiently than human divers.

And, although when I worked at NASA, I wasn’t able to get my own AI into space, I am happy to report that I finally achieved it – the CSIRO robotics team worked with NASA to create an AI astronauts’ assistant which is flying on the International Space Station as we speak!

But for all that promise, not everything is rosy.

Government investment in Australian AI is dwarfed by even similar sized countries. Depending on how you count, Australia has invested around half a billion dollars in AI; that compares to around five billion in Singapore.

So far, the vast majority of Australian AI has focused on adapting AI models developed elsewhere. Practically speaking, there are no Australian foundation models. Foundation models are the engines of AI – they are what power ChatGPT, Gemini, Deepseek and Claude. But foundation models are heavily dominated by the US and China, which between them build 80% of all models. Australia has 0%.

In some ways, Australians seem indifferent to AI. According to a survey conducted by KPMG and the Melbourne Business School, Australians are among the least trusting in AI – ranking 6 from bottom among 47 countries. Worse, Australians have low AI literacy rates, ranking the lowest globally in their interest in learning more about AI.

It sometimes feels that Australia is held back by a lack of ambition, a lack of drive, and a lack of interest in realising what AI could do for us.

We have the potential in Australia to be a thriving medium power in AI. My fear is that we don’t have the guts for it. And that we are content instead to be a nation of AI consumers not AI builders.

We need to invest in core AI technologies, not just apply AI to mining, agriculture and healthcare. We need to build our own sovereign AI – for independence, yes, but also to build new economies beyond digging things out of the ground.

This is not a popular argument. Things usually go like this: Australia is too small to compete with the big guys like OpenAI or Anthropic. It’s too expensive to build our own AI models. We should just stick with what we are good at.

But there are risks of doing nothing.

AI now touches every aspect of Australian life. Without sovereign AI, we are exposed to serious geopolitical risk. We’ve seen with COVID and with tariffs that the world order can shift overnight. And we’ve seen the rise of China. In January last year, a Chinese company released Deepseek – an AI model just as capable as the best in the world, at a fraction of the cost.

Chinese AI isn’t yet dominant in Australia, but it could be. Ten years ago, no Australians were buying Chinese cars. And yet, today, BYD is selling 10x more electric vehicles in Australia than Tesla. It’s not hard to imagine that in ten years, cheaper, ‘good enough’ Chinese AI will be as ubiquitous in Australia as Chinese EVs are today.

Beyond geopolitics, there is economic risk. AI is slowly but surely rewriting economic models. Take software. Engineers who use AI to write code pay by buying tokens from a model provider. And as most of those models come from the US and China, that money goes offshore. If AI coding develops as quickly as many think it will, we could soon be in a situation where Australian companies are developing products in Australia but paying other countries for the privilege. AI is already disrupting industries and creating new ones. Do we not want a share of what comes next?

One common argument against Australia developing its own AI is that it costs too much. Companies like OpenAI have spent billions. We don’t have that kind of cash to spend, right? But AI doesn’t have to cost billions. Deepseek proved that. And there are now two Australian startups attempting to develop AI models for tens of millions, not billions.

So what should be done?

The Australian Academy of Technological Sciences and Engineering has called for $5Bn of government investment to get Australia to the starting line.

But, in my view, they are making a fatal mistake: they are waiting for government to make the first move.

Having spent five years in government – and secured over $100M in investment for initiatives like our National AI Centre – I am not hopeful that government will invest those sums.

In December last year, the government released its long-awaited National AI Plan, but it has received mixed reviews. There is no significant new investment, and it risks becoming – like the National Robotics strategy before it – a piece of paper on a politician’s shelf.

One view of the national AI plan is that it’s not an AI plan at all, but a data centre plan. Australia is now genuinely positioning itself as an Asia Pacific hub for data centre development: we have plentiful land, stable politics, and an opportunity for sustainable data centres. The data centre focus makes sense. But it shouldn’t be the only thing we do.

I don’t think it’s government’s job to fix this. Though it could help by reducing the regulatory burden. By some projections, AI could add roughly $150 billion a year to Australia’s economy. Any guesses on the annual regulatory burden costs? About the same.

Our real problem isn’t money, it’s mindset. We don’t need any more plans; we need persistence to put them in place.

Smarter people than me have written about redesigning Australia’s innovation ecosystem. I have no illusions that I have the answers here, so I won’t attempt today to fully solve the problem.

But I will talk about FIVE things which I think could help.

First, we need to build our own foundational AI models. Not to compete with ChatGPT, Claude or Gemini, but to provide a home-grown alternative for those who want it. The aim isn’t to have the best model in the world but to have something we can build on. Otherwise, the rest of our AI economy is built on sand.

Second, home-grown foundation models are a start, but not the endgame. The economic opportunity doesn’t end with today’s AI. We may have lost the current race – the race to build large language models. But most AI experts believe that today’s AI isn’t the final word. To achieve true intelligence, we’ll need fundamentally new approaches. Even if we’ve lost the current race, Australia should be positioning itself for the race that is still to come.

Third, we should go all in on areas of competitive advantage. In my view, one of those is “AI for science” – that is, the use of AI to speed up scientific discovery. This is an emerging field, already having impact. While there is no Nobel prize for AI, in 2024, both the physics and chemistry prizes were won by AI scientists! We already have a fair-go at being the best here: my team at CSIRO built an AI platform that can partially automate the entire lifecycle of science, all the way from literature review to hypothesis generation to experimentation to exploiting results. But we need to double down.

Fourth, we shouldn’t be afraid to think differently about AI. Australia has strongly held values; one of the biggest open questions in AI is how to align AI with human values. The Americans aren’t solving this. Perhaps we could?

Fifth, the private sector should take the lead. We can’t sit around waiting for government.

For me personally, I’ve chosen to focus on the last point, and on where I think I can make a difference: educating board directors and executives.

Since leaving CSIRO last November, I’ve been helping them understand and apply AI. If Australia is to innovate in AI, our business leaders must lead. And too many boards are not yet ready.

In my book, AI for Business, I identify three types of organisation. AI Disrupters use AI to fundamentally shift their business model. AI Adaptors use it to make existing operations more efficient. And AI Watchers wait for someone else to go first.

Our task, as a nation, is to move more of those watchers into adaptors and more of those adaptors into disrupters.

That will be my focus for the next few years.

As I try to do my bit for Australian AI, however small, I am reminded how I first came across the term artificial intelligence. It was 1993. I had just finished a degree in mathematics at Oxford, and I was wondering what I was going to do with my life. One afternoon, I walked into the university careers library (back when we had such things) and started flicking through material for inspiration. After two hours, I was about to give up and head to the college bar when I chanced upon an ad for a Masters in Artificial Intelligence at the University of Edinburgh. I’d never heard of AI at the time, but it sounded so fantastical that, at that moment, I think I unconsciously knew that it would define the rest of my life and career.

It was perhaps a serendipitous moment for me to find AI in this way. But then again, I’ve always believed that serendipity doesn’t just happen; you make it happen. It was me who decided to walk into that library. It was me who was open to whatever I found in there. And that’s how our nation needs to be with AI.

Be open. Lean in. And who knows — serendipity might just do the rest.

First of all, thank you for inviting me – and it’s great to see so many Australian and New Zealand software engineering researchers here today.

My aim with this talk is simply to start a discussion: a discussion about the importance of human values in software, and, in particular, in AI-generated software.

In case you hadn’t noticed, AI coding has become a thing. My LinkedIn feed is full of developers excitedly claiming that they don’t need to code any more. Or that 100% of their code base is now written by AI. We hear that even AI itself is AI-generated: Boris Cherny, the creator of Claude Code, says that Cowork, Anthropic’s desktop AI assistant, was “pretty much all” written by AI.

For me, this is remarkable. Twenty-five years ago, I started a job at NASA. My task was get AI to automatically write code for NASA space missions. It didn’t work. Not really. Sure, I could use a rule-based system to glue software components together for very specific applications, but AI that can write any code, any time, for any application? That seemed fantastical. And yet, here we are.

The main message of my talk is that I believe AI coding may offer us a new opportunity to embed human values in the software systems we build.

But before we go there, let’s take a step back.

Why Values Matter

Back in 2017, when I joined Monash University as the Dean of the IT Faculty, I started a small research group called OVIS: Operationalising Values in Software. The motivation was simple: software runs the world. As such, it has the potential to significantly enhance the world we live in; or it could ruin it.

At the time, there was one example that always stuck in my mind about the potential for software systems (especially AI) to make or break society.

Molly Russell was a British teenager who, tragically, took her own life in 2017. Like many teenagers, Molly was susceptible to self-harm. And she used to go on Instagram to look at pictures. As you know, the nature of AI algorithms on social media is such that it reinforces what you are looking it: so the more Molly looked at self-harm images, the more her feed became dominated by them.

After her death, Molly’s father went on a public campaign, claiming that Instagram was partially responsible for her taking her own life. If the AI algorithm hadn’t spoon-fed Molly with hundreds of self-harm images, perhaps she might have escaped the doom spiral she found herself in.

Molly’s story eventually led to some significant changes to the way Instagram is designed, with better protections against harmful content.

But this example remains a powerful illustration of what I mean when I talk about the importance of embedding human values in software systems. Software should be designed for the society we want. And we don’t want to harm our kids. Software should reflect the values we have as human beings. Unfortunately, this hasn’t historically been a major concern of the software community.

Values in Software Engineering

Software engineering, as a discipline, is now almost 60 years old. And we’ve achieved some amazing things – we’ve developed methods and tools that (for the most part) allows us to build software that functions as we expect, at a cost we can afford, that is secure, safe in critical applications, and respects data privacy. Sure, it doesn’t always work as expected, but it usually does.

However, there are a host of broader ‘human values’ that have been much less considered in software engineering circles or not considered at all: inclusion, diversity, social responsibility and fairness, to name a few. And yet, given how software can affect society, shouldn’t these human values be embedded in software as first-class citizens?

Some values have received a lot of attention, of course. Safety, security and privacy are well studied. But most human values have simply been ignored – as though it’s not the role of software system designers to worry about such things. For example, why doesn’t software design prioritise psychological well-being of its users? Or trust and integrity? Cultural preservation? Or social cohesion?

Software developers shouldn’t ignore these things; even if their CEOs want them to.

Defining Human Values

What are human values anyway?

The social psychologist Shalom Schwartz introduced the best-known definition of human values: his theory of basic human values. Based on extensive research in over 50 countries, he identified 10 categories of human values that are universally accepted as important. These include safety, security and privacy but also other human values, not usually considered in software engineering, such as equality, benevolence, self-direction and even values like tradition.

What would software systems look like if we intentionally designed for this broader set of human values?

Back in 2017, the OVIS team began exploring this question. We adapted Schwartz’s notion of human values to software engineering. We worked with companies to understand how they currently thought about values in software development and what they did to ensure values are reflected in their software.

We were surprised to discover that many companies already have mechanisms in place to ensure values alignment: critical friends who flag ethical issues, corporate values statements that developers are expected to follow, code reviews that go beyond just security and privacy.

But we also found that there were no concrete methodologies for building values in. Meaning that companies like those mentioned are the exception rather than the rule.

In the absence of a methodology for values in software engineering, we decided to create one.

As it turns out, a good place to start was agile methods. Agile methodologies are naturally people-focused both within development teams and being customer-centric. So they are a great fit for embedding human values.

The OVIS team adapted the Scaled Agile Framework (SAFe) and showed how human values can be considered as a first-class development principle throughout: in SAFe artefacts, roles, ceremonies, practices and culture.

In doing so, we made another surprising discovery.

When I first started working on human values in software, I thought it would require a fundamental rethinking of how we develop software – completely new methodologies, if you will. If that were true, it’s unlikely we could get the software industry to change. Fortunately, in adapting SAFe, we found that tweaks to existing methods can be made relatively easily: values can be added to user stories, value-specific roles can be included – such as values champions – and values, once made sufficiently concrete, can be monitored and tested.

In short, considering values in software requires evolution not revolution. Which is good news. Because evolution is always less stressful than revolution.

Fast forwarding to today, the work we started in the OVIS group led directly to a large body of work in Responsible AI at CSIRO’s Data61. The team there, which we started in 2021 and quickly grew to almost 50 researchers and engineers, have had significant impact: influencing Australian government policy, supporting industry through the National AI Centre’s Responsible AI Network, and pioneering research through the Responsible AI Research Centre in Adelaide. Not to mention, of course, their book collecting together best practices for Responsible AI development.

The AI Shift

But, of course, a lot has changed in the world of software even since 2021. AI-generated code is now becoming the norm. The rise of vibe coding is allowing non-technical users to quickly spin up software personalised to them. Early adopters are using AI for production code too: with Dario Amodei claiming that 90% of code at Anthropic is already written by AI.

While not everyone is there yet, the potential for increased software productivity is enormous. But so is the potential for quickly producing software that violates our fundamental human values.

We’ve seen this recently with Openclaw (formerly, Moltbot/Clawdbot): which has unleashed thousands of AI agents with broad access to users’ personal computers and risks corporate security breaches at scale.

More generally, AI generated code can have flaws – Veracode found last year that 45% of AI generated code studied contained security vulnerabilities. Some commentators have suggested that since AI is mainly trained with open source code – which doesn’t necessarily prioritise security – then AI-generated code is insecure by definition.

And, of course, the number of Molly Russell-like examples of AI having negative impact on society has significantly grown. Stories of teenagers taking their own lives, egged on by AI, are now commonplace. And society is becoming increasingly dependent on AI companions (75% of US teenagers have interacted with an AI companion), which risks exacerbating the current global loneliness epidemic.

As AI agents take over business processes, how can we be sure that they will act in accordance with business values let alone human values? Last week, OpenAI launched Frontier, which “gives agents the same skills people need to succeed at work.” But will these agents respect a business’ corporate values? CEOs spend a lot of time and effort ensuring that employees behave according to corporate values. How can they ensure AI agents will do the same? It only takes one values violation by an AI agent to destroy a company’s reputation: what if that super cool AI agent you built to automate home loan decisions discriminates against a minority?

And if you think this is speculation, well it has happened before – many times. Think of the COMPAS system that discriminated against black offenders at parole boards, or Amazon’s same-day delivery AI algorithm that biased against poor neighbourhoods, or even Robodebt, a much simpler automation nothing to do with AI.

These are historical examples and we need to learn from history so our AI agents operate within values, not outside of them.

I’m Optimistic About the Future

If these are the risks of AI-generated software done carelessly, what might it look like done intentionally?

Our put another way: does AI coding offer a fresh opportunity to embed human values in software?

Human software engineers typically haven’t prioritised human values. But if AI was trained to generate software that puts values first, we potentially could right that wrong.

AI could generate code that automatically built in fairness, inclusion, diversity, and environmental sustainability. Arguably, AI already generates better quality code than many developers. Taking this further, could it generate software with better values than many humans?

Let’s look at some simple examples.

Accessibility. An AI trained to value accessibility might be more consistent than humans in enforcing accessibility standards.

Fairness. An interface for HR chould be designed to draw out periods off work to bring up children so the hiring manager is aware and avoids bias. AI can make sure this happens; humans might forget.

Privacy. An AI agent that collects data can be trained to enforce privacy legislation to avoid over-collection of data, unclear retention policies, and consent fatigue.

Responsibility. An AI that is coding up another AI algorithm could be trained on global responsible AI practices to ensure the AI algorithm is fair, transparent and explainable.

So does AI offer us an opportunity to ensure that human values are finally embedded in software so we get the society we want?

Making this Work in Practice

I am probably naïve in thinking that AI could lead to more human-oriented software.

After all, humans still run companies and, despite what I said earlier, most companies still prioritise profit over anything else. There is a practical barrier too – for an AI to generate values-first software, it would need to be trained on values-first software. And it’s an open question how that training data could be collected and curated.

But my call to action, if you will, is that we, as a software engineering community, should consider these questions more.

Are agents an opportunity to consider values more?

How would that work in practice?

What would a values-aware coding agent even look like?

The next generation of software engineers and researchers will define what AI-generated software looks like. And I hope it will be values-driven. And I hope that you think so too.

If you are a corporate executive or non-executive director, it’s worth becoming familiar with OpenClaw (formerly known as Clawdbot and then Moltbot), a fast-growing open-source AI agent that recently surged to over 100,000 GitHub stars in a matter of days.

OpenClaw sits at the leading edge of a broader shift in AI: from conversational tools that respond to prompts, to agentic systems that can take actions on a user’s behalf. These tools don’t just answer questions. They can interact with files, messaging platforms, cloud services, and other software systems to carry out tasks.

This new category of AI tool is beginning to expose structural weaknesses in how organisations govern access to corporate data, particularly in environments that rely heavily on cloud services and personal devices.

What makes OpenClaw different

AI agents have been gaining attention throughout 2025, but OpenClaw stands out because of its local, user-controlled execution model.

OpenClaw is installed directly on a user’s computer. To function effectively, users may choose to grant it broad permissions, such as:

• Access to the local file system

• The ability to execute shell commands

• Integration with messaging platforms and productivity tools

• Optional connections to email, calendars, and cloud services

These permissions are not granted by default. They require explicit user configuration, but once enabled, they allow the agent to act with considerable autonomy.

This breadth of access is what makes OpenClaw compelling. It can coordinate tasks across tools, operate continuously, and act proactively rather than waiting for explicit prompts. Many early adopters have used it to set up 24/7 personal AI assistants that proactively do things on their behalf, sometimes in ways the user did not fully anticipate.

If this sounds scary, it is. On the one hand, OpenClaw is closer than we’ve ever been to the dream of AI: a machine that just does things for you, without needing to be prompted. On the other, as Cisco points out, it’s a security nightmare. Security researchers have discovered thousands of exposed instances. But even if the OpenClaw code were 100% secure, it could still be a nightmare: because you’re giving an AI private data without full control over what it does with it.

Personal versus Corporate Use

The vast majority of OpenClaw users so far have been using it for personal use. Stories abound about how it has organised to-do lists, structured file folders, and managed emails. Installing a security nightmare on one’s personal computer is one thing, but what if it’s installed on a corporate system? Suddenly, the risk of corporate data breaches expands. Token Security found that 22% of its customers have employees who have actively used OpenClaw within their organisations.

While I don’t yet know of any OpenClaw-related corporate breaches, one can easily imagine scenarios, such as this one from Token Security:

Consider this scenario: An employee sets up Clawdbot on their personal laptop or Mac Mini and connects it to WhatsApp or iMessage as their chat interface. They also connect data sources to their corporate Slack workspace and grant it access to internal channels, direct messages, files, emails, and calendars. After all, they want their AI assistant to be helpful.

Now, from anywhere in the world, that employee can send a WhatsApp message asking: “What were the revenue numbers for the last quarter?” or “Summarize the latest product roadmap discussions in the #productmanagement channel.” The AI dutifully searches through corporate Slack, reads confidential documents, and sends the summary back via WhatsApp, completely bypassing DLP controls, email security, and any corporate audit trail.

The data flows from corporate systems, through an unmanaged AI agent running on a personal device, to a consumer messaging app that is all invisible to the enterprise security team.

In short, OpenClaw – and any future AI agent systems of this ilk – may lead to a new kind of shadow AI for which corporate security practices may not be prepared. Users may grant powerful AI agents access to sensitive data without adequate organisational controls, monitoring, or auditability. This raises legitimate questions about oversight, control, and unintended data exposure.

What’s the Corporate Risk Really?

It’s easy to get carried away by the hype when new AI tools like OpenClaw appear. As ever, we need to try to cut through the noise and get to a realistic assessment of what the real risks are. Is it really as bad as it sounds? Or is OpenClaw the latest fad, soon to be forgotten?

First and foremost, people are installing and using OpenClaw. There’s no question about that. And while some technical skills are required, it’s relatively easy for non-technical users to do it, which means that many employees in your organisation could install it. The risk level comes down then to how mature your cybersecurity policies and practices are. In organisations with mature IT policies, it’s unlikely that employees could (even if they wanted to) install OpenClaw on corporate managed devices. But that doesn’t mean there’s no risk. As the example above shows, if an employee can access corporate workspaces from a personal device, so can OpenClaw.

Not all organisations face the same level of exposure to AI agent risk. Your vulnerability depends primarily on three factors: device management posture, cloud service architecture, and visibility into third-party access.

Tier 1: Regulated Enterprises with Mature Security Programs

Organisations in this category typically include:

• Financial services institutions

• Healthcare providers subject to HIPAA

• Government contractors

• Large enterprises with dedicated security operations centers

Security characteristics:

• Mandatory use of corporate-managed devices for any access to sensitive data

• Mobile Device Management (MDM) or Unified Endpoint Management (UEM) enforcement

• Conditional access policies that block unmanaged devices

• IP whitelisting or VPN requirements

Risk level: Low to Moderate

If properly configured, your existing controls should prevent employees from connecting AI agents running on personal devices to corporate resources.

Tier 2: Cloud-First Organizations with BYOD Policies

This segment includes:

• Technology startups and scale-ups

• Professional services firms

• Remote-first companies

• Organisations that prioritised access convenience during rapid pandemic-era digitisation

Security characteristics:

• Heavy reliance on SaaS platforms (Google Workspace, Microsoft 365, Slack, Notion)

• BYOD policies allowing personal device access to corporate email and collaboration tools

• Cloud-based security but potentially incomplete enforcement

• Authentication without rigorous third-party app review

Risk level: High

Employees can authenticate to corporate systems from personal devices, and AI agents running locally inherit those same access rights. IT visibility into these connections is often limited.

Tier 3: Small Businesses Without IT/Security Separation

Characteristics:

• Fewer than 100 employees

• No dedicated IT security role

• Personal and work devices often the same hardware

• Limited security tooling beyond basic antivirus

Risk level: Critical

The distinction between personal and corporate access effectively doesn’t exist. Any AI agent with system access on an employee’s laptop potentially has access to corporate data, credentials, and communications.

What should you be doing about it?

If you sit on an executive leadership team or board, what action should you be taking?

First and foremost, I’d be wanting to know how secure the organisation is against rogue AI agents. Do its cybersecurity policies and practices already cover the worst that can happen or is a refresh needed? I’d be having a conversation with your CIO or CISO to understand where things stand. Here are some specific questions you can ask:

• Can employees access corporate email, Slack, Drive, or other collaboration tools from personal devices?

• Do we maintain an inventory of all applications with access to corporate systems?

• Can we detect when AI agents or automation tools are accessing our APIs?

• Do we monitor for unusual API access patterns (high volume, off-hours, anomalous data retrieval)?

• What is our policy on employee use of AI coding assistants and autonomous agents?

Future thinking

OpenClaw is a sign of what’s to come. Even if it doesn’t result in any major corporate breaches, it won’t be the last super-powerful AI agent platform. The lesson from OpenClaw is that organisations need to think about future preparedness to ensure that security policies are fit not just for today but for whatever comes tomorrow. Brainstorm what the future of AI agents might look like, and start getting ready now.

Balance is everything

AI agents illustrate well the core paradox of AI. They are at once a great boon to society, but also present significant new security risks. If you want to benefit from AI agents, you need to think about and implement them responsibly.

As ever, when it comes to AI, balance is everything.

If you want to learn more about the intersection of AI and Cybersecurity, my book AI for Business covers this in a dedicated chapter. AI is both a new tool in the fight against cyber-attacks, but also opens up new attack vectors. Available on Amazon or from CSIRO Publishing.