What the World Cup has taught us is that the best competitions thrive on fair play and open access. When anyone can lace up their boots, climb the ranks and aspire to one day hoist a trophy off of their own merit and hard work—that’s the promise of sport.

To be sure, FIFA isn’t perfect. There has long been talk of misconduct within FIFA, including the 2015 corruption scandal, accusations of teams receiving favorable treatment from officials, and claims of match manipulation. I’ll leave it to the fútbol experts to litigate those. What I will say is FIFA deserves credit for the structure of this year’s World Cup, where (much to my genuine surprise) competition thrived on greater access.

For the first time, the tournament expanded from 32 teams to 48, and with it came nations who had never featured at the World Cup before— countries like Cabo Verde and Curaçao, boasting some of the smallest talent pools in the competition. Cabo Verde advanced out of the group stage and put up a strong fight against Argentina, winning the hearts of fans worldwide. Increasing competition didn’t lower the quality of the tournament: it made it better.

That’s the instructive part. A truly competitive system doesn’t wall itself off to protect the already dominant players; it makes room for new entrants and lets their performance on the field decide who deserves to come out on top.

That’s the lesson for the IT sector. The incumbents have stopped the playing field from expanding and reduced competition.

For example, Microsoft holds an 85 percent share in the U.S. government office productivity software market and proudly claims that more than 90 percent of Fortune 500 companies are using its AI technology. But it’s not a sector picking a winner on merit. Microsoft has spent decades making itself the default through bundling and long-term enterprise agreements that make switching too costly or difficult to seriously consider.

For years, I’ve been warning about this sort of monoculture risk – when all of our eggs are in one basket, any problem with the basket is catastrophic. In the case of operating software, it’s simply easier for a procurement officer to renew an existing enterprise license than to run a competitive bid and rebuild interoperability from scratch. Stacked over three decades and thousands of agencies, the result is the public and private sectors’ digital systems running almost entirely on one supplier, one that now faces essentially no competitive pressure to keep earning that position.

It’s not just Microsoft. We are overly dependent, in other contexts, on single-source IT providers. Amazon Web Services (AWS) is the biggest player in the cloud industry. Even though AWS has removed egress fees for customers migrating data to another cloud provider or on-premises server, the deeper architectural lock-in remains. As one of the other players in the industry, Open Edge Cloud, highlighted, customers who build their systems on AWS face the same lock-in effect: proprietary APIs and years of accumulated technical debt that make “just switch providers” a multi-year, multimillion-dollar undertaking rather than a real option.

That is precisely the problem with a fixed match: the incumbent doesn’t need to play better when the rules already guarantee the outcome. A challenger with a more secure architecture or a better price still can’t win the business, because the switching costs and interoperability lock-in were built into the last contract, and the one before that.

The danger from the lack of competition in the IT monoculture is not theoretical. We have already seen the problems that can arise when a single software stack is running every critical system. We are just one exploit or bug away from cascading failure.

When hackers breached Microsoft Exchange servers in 2021, tens of thousands of U.S. organizations were compromised in a matter of days, simply because they all ran the same code. And in 2024, a single flawed CrowdStrike update took roughly 8.5 million Windows devices offline in one stroke—grounding flights, knocking out hospital systems, and freezing banks and government offices. When everyone stands in the same spot, one well-aimed shot takes everyone down.

It is time to make the competition real again: Procurement processes that evaluate multi-vendor and open-standard alternatives on a level playing field. Interoperability mandates that stop vendors from designing systems that punish customers for diversifying. Genuine investment in open-source and smaller domestic alternatives so that “the alternative” is viable at scale, not just on paper. And a systematic recognition, finally, that a single point of failure across the federal government is a national security liability no matter whose logo is on it.

This year’s World Cup proved that widening the field doesn’t weaken a competition; it strengthens it. Cabo Verde didn’t need to be handed anything to make the tournament better; it just needed a seat at the table to prove it was more than capable of competing (and nearly knocking out defending champion Argentina). American hospitals, schools, and government agencies all run on the same fragile stack. They deserve a level playing field too.

Subscribe now

However, U.S. AI dominance is not guaranteed nor should it be taken for granted. China has announced its goal to become the world’s leading AI power by 2030, and its ambitions span the entire technology stack. U.S. companies that transfer knowledge, talent, and platform access to Chinese companies are not only accepting commercial risk, they’re also negotiating away America’s national security edge.

Most U.S. industry leaders recognize that their success is closely tied to America. But some major U.S. technology companies are hedging their bets, particularly on the Middle Kingdom.

In May, Microsoft launched an incubator in Shenzhen—standing alongside local Chinese Communist Party (CCP) officials to unveil the “Shenzhen Global Expansion Center,” a platform to provide Chinese startups with cutting-edge AI technology, platform capabilities, and access to Microsoft’s global ecosystem. The project has reportedly reignited fears in Washington about the company’s coziness with Beijing. Microsoft brushed it off as “a marketing and advertising training initiative” – an explanation that one should view with skepticism in light of Microsoft’s history in China.

Meta has also quietly assembled a research unit, Meta Superintelligence Labs, in which seven of the eleven publicly listed hires are researchers from China, drawing on talent trained at Chinese universities and government-linked institutions. The staffing of America’s most sensitive frontier AI research with scientists educated inside Beijing’s system has, understandably, raised alarms.

These investments are not one-offs. They are part of an ongoing, concerning pattern. For example, Microsoft has operated major AI research facilities in China for decades and employs more than 10,000 people in the country. And Meta’s entanglement with China runs far deeper than its research hires. A former Meta executive testified before Congress that the company briefed the CCP on AI as early as 2015, and that Meta’s open-source Llama model contributed directly to Chinese advances such as DeepSeek. Every engineer Microsoft or Meta trains in or through China is an asset for Beijing’s AI ambitions.

Here’s another example: A ProPublica investigation last summer revealed that Microsoft had been secretly relying on China-based engineers to maintain sensitive U.S. national security cloud systems for nearly a decade—using American citizens with security clearances as “digital escorts.” Microsoft only ended the practice after it was exposed. The Department of War called it a “breach of trust” and issued a formal letter of concern.

The fundamental problem is the failure to recognize that the Chinese “market” is not a true market in any Western sense of the word. Some American commercial technology companies treat doing business in China as just part of their global technology footprint, maintaining a commercial presence in the world’s second-largest economy. That argument ignores the fundamental reality of the Chinese private sector: it does not truly exist.

Under China’s 2017 National Intelligence Law, every Chinese citizen and every company operating in the country is legally required to “support, assist, and cooperate” with national intelligence efforts. Additional laws require any company with a footprint in China to report cybersecurity vulnerabilities to the government before patching them. This almost certainly gives Beijing advance notice of exploits. Every startup Microsoft welcomes into its “global ecosystem” walks through that door with American technology in hand.

Corporate resources, in other words, are finite, and so is a company’s attention. Every dollar spent and every line of code written helping Chinese startups scale, is a dollar not used to harden the systems America depends on or to build the AI infrastructure America needs. The choice is not abstract: it is a binary investment choice between U.S. security and U.S. competitiveness and Chinese AI growth.

The U.S. government has the tools and leverage to act. Microsoft’s federal cloud business is worth billions. Meta’s platforms carry sensitive data on millions of Americans. And this Administration has demonstrated that it is not shy about flexing the government’s economic muscles. Doing business with the U.S. should come with expectations. Companies that want to be trusted partners of the U.S. government should not be incubating Chinese AI startups on the side.

The choice is between building America’s AI future or strengthening China’s. We can’t do both. The future of AI and everything that it touches depends on which choice we make.

I was honored to be asked to give a eulogy for Stewart at funeral cermemony last Saturday. This is what I said:

I joined DHS as Stewart’s Deputy in the Fall of 2005. Before that we had been professional colleagues, often bonding over the perceived errors of our liberal opponents who (we thought) didn’t really understand technology, surveillance or the threat of terrorism post 9/11. After I joined DHS, we began what grew to be the closest and deepest professional friendship I have had. Stewart was both a mentor and a friend. More importantly, he was a selfless man who gave of himself seemingly without limit.

One of my favorite examples of that comes from early on in my time at DHS. I’d been there only a few weeks when our Secretary, Michael Chertoff, set off to the UK to meet his opposite number, the Home Secretary Charles Clarke. The agenda was all the hot button issues of the day – counter-terrorism; information sharing; privacy; travel restrictions and the like. It was the sort of trip where Stewart would have appropriately accompanied the Secretary.

But he sent me. And before I went, he told me explicitly that he was sending me because this was a chance for me to build a separate relationship with Secretary Chertoff. Stewart’s plan worked – over the next several years I got a chance to work directly with Chertoff on a number of matters. But what was more remarkable to me then (and still is today) is how willing Stewart was to step aside and foster my professional growth. Let’s be clear – nobody ever called Stewart a shrinking violet and I’m sure he would have loved the politics and policy of that trip. But in that case (as he did so often in his career) Stewart saw value in helping others on their own professional paths. How many bosses do you know who affirmatively want their subordinates to develop that sort of independence?

And it wasn’t just me. Over the past weeks there has been an outpouring of remembrances of Stewart on social media. I can’t tell you how many dozens of them took a form of something like: “When I was just a junior ____, Stewart was willing to talk with me and hear me out.” Or “I still give Stewart-derived advice to the lawyers I work with.” Or, “If not for Stewart I would never have become ____”

And it wasn’t just personal advice and support. Stewart’s engagement with the law was, literally, global. One Israeli who knew Stewart wrote me to say “My legal colleagues and I listen[ed] to [his] podcast diligently. It was an anchor for discussions, and indicator of what’s important. [Stewart] played a role in forging cyberlaw as a[n independent] discipline.” Not a bad legacy all that.

Of course, Stewart was not all serious law and policy. Anne has said she wants to celebrate his life, so here’s one funny story that has stuck with me. Stewart was, of course, very proud of his Scots heritage and their reputation for parsimoniousness – a trait for cheapness that Stewart wore as a badge of honor. Now, I am Jewish and we, too, have a somewhat invidious reputation for being cheap. But when we joked together it wasn’t at all inapt – we competed at times to see who could be cheaper (and, to be clear, Stewart typically won those battles).

This trait came to a joyful head with the advent of a program known as Privium a program which, like Global Entry here in America, came with expedited border clearance. It was developed for Schiphol airport by the Dutch, who also have a well-deserved reputation for being tightfisted. There came a time when we entered into negotiations with the Dutch to exchange clearances – Privium for Global Entry.

And boy was that fun to watch. Talk about a battle of the titans. Of course, it was mostly in good fun since we all knew that the exchange was a win-win. But I clearly remember Stewart trying to negotiate for a discount on food at Schiphol for the DHS staff who would be there. In the end, I think the Dutch held Stewart to a draw.

Which brings me to one last point that is important to make. Stewart was strong-headed. He was provocative. And he was a determined foe. And he didn’t like losing … at all. But he was not set in stone. Perhaps my most cherished memory of Stewart involved a relatively obscure issue relating to immigration and asylum where, after a long hard discussion, he simply said to me: “You know, you’re right. I was wrong.” That characteristic of self-reflection and intellectual honesty was, to my mind, his best trait.

Stewart was a wonderful mentor and a close friend. His was a happy warrior whose sharp wit and piercing intellect were a joyful part of a challenging field of law and policy. He seemed to me eternal and I cannot believe he is gone. I will miss him terribly.

RIP, my friend. Your memory will always be a blessing.

Anthropic’s CEO Dario Amodei has long been cautious about new AI uses. As an example of living its principles, when Anthropic first contracted to deploy Claude in the US government’s classified systems (to date, it is the only AI model deployed on the classified networks), Anthropic imposed two conditions – that Claude not be used for mass domestic surveillance of Americans and that it not be used to create fully autonomous weapons that operated without human control.

DOD Secretary Pete Hegseth is demanding that Anthropic drop both conditions by Friday and allow DOD to use Claude in any “lawful” manner that DOD chooses, presumably also to include the two currently prohibited activities. If Anthropic does not agree, then DOD will either declare Anthropic a “supply chain risk,” which would effectively bar ALL DOD contractors from doing business with Anthropic, OR invoke the Defense Production Act to compel Anthropic to provide DOD with an unlimited AI model bereft of limitations.

The controversy deserves far more public attention than it has gotten. Herewith a few quick, preliminary thoughts:

1. Anthropic, like any other private business (that isn’t a public service like a train system), should be free to contract with whoever it wishes and free, as well, on principle to decline to do business with anyone whom it considers inappropriate. But principles have costs – if a profitable client won’t agree to your terms, you have to make a choice.

2. DOD was free not to enter into the contract that it did with Anthropic. Presumably, it chose to do so because Anthropic was the best (maybe only) choice then available. If DOD no longer likes the terms of the contract, it is free to modify, cancel, or withdraw as the contract permits.

3. At the same time, it seems very inapt for a DOD supplier to be able to put conditions on how DOD uses a tool that it provides. We would not, for example, allow a tank manufacturer to forbid DOD from selling the tanks to Turkey (because, say, of an affinity for the Kurds), nor would we allow a satellite provider to forbid DOD from using the satellite to assist Israel (because of the Gaza war). Our military has to have the freedom of action to act in the best interests of the country.

4. Part of what is happening here is that Anthropic is under some public relations pressure because Claude was, reportedly, used in the Maduro assault. But that reputational risk is one every defense supplier runs in order to do business with DOD. If you want to sell to the US government, you have to accept that the government will use your product. And that means, as a general matter, that systems provided can be used in any “lawful” manner.

5. The problem arises here for two reasons: First, the law simply never keeps up with technology. [We still haven’t solved the backdoor/encryption problem after 10+ years.] I have not done the research, but I have little doubt that many domestic surveillance uses that DOD might contemplate can be legally justified, and I am equally certain that it is likely that no domestic law explicitly prohibits the complete development of autonomous weapons systems [international law is different, but non-binding].

6. Second, the larger problem is that Trump has exhausted all presumptions of regularity and lawfulness. Even IF I am wrong and one could identify legal limits on domestic surveillance and autonomous weapons use in Federal law, Anthropic (and the general public) rightly have doubts that Secretary Hegseth would abide by those limits. Indeed, the entire project of Trump/Hegseth over the last year has been to erode the legal limits that bind DOD.

7. Of even greater import, the threat from Hegseth is wildly disproportionate to the problem presented. In essence, what Hegseth is threatening is “I will either destroy you or nationalize you if you don’t give me unlimited permission to use your product as I wish.” Seizing the factors of production is a communist sort of thing, isn’t it? And yet it has almost become the norm for the Trump administration to threaten a “nuclear option” response if it doesn’t get its way.

8. I have little doubt that neither of Hegseth’s proposed courses of action would survive a court challenge – as with so many of Trump’s actions, there simply is no factual predicate to justify the steps threatened. Anthropic would, eventually, win in court.

9. But that sort of court victory would be pyrrhic. During the pendency of the suit, Anthropic (one of the foundational AI creators in the US) would be under a cloud. The economic consequences of fighting would be severe. I would not want to be Amodei today, trying to figure out the next steps.

10. What’s the right answer? That’s pretty obvious – if Claude is not available under terms that DOD wishes, then DOD should find another provider who will provide the product as specified. Reports are that Elon Musk’s GrokAI is happy to step in (though one wonders if it is comparable). Over a reasonable transition period, DOD should find a substitute that can do what it needs.

11. Meanwhile, I suppose we can dream and imagine that Congress will step in to legislate. That’s also, obviously, the “right” answer – but in the current political environment, that seems highly implausible.

12. And, finally, more fundamentally, if we don’t like the idea of DOD doing domestic surveillance or deploying autonomous weapons, we need new DOD leadership who will approach those issues differently. Everything Trump touches dies – including faith in the good intentions of a Hegseth-led DOD. If we really want a better approach, we need a better leader.

For instance, an Associated Press investigation, published in September, revealed that American technology companies have played a far greater role in designing and building China’s surveillance state than previously known. According to the AP, household names including Nvidia, Amazon Web Services, and Microsoft have sold technology or services to Chinese police or surveillance companies. While companies may believe they aren’t responsible for how their products are used, several of them have marketed their products using Chinese Communist Party euphemisms for controlling citizens, like “abnormal gatherings.”

The variety of products and services American tech companies have sold to the Chinese government and police over the last 25 years is astounding. Beginning in 2009, IBM worked with Chinese defense contractor Huadi to build national intelligence systems. Until sanctions were recently imposed, Nvidia and Intel were working with Chinese surveillance companies to add AI capabilities to cameras used for video surveillance. The fingerprints of American tech companies are all over China’s regime of surveillance and repression.

Even worse, the U.S. government has been complicit in this moral failure: across both Republican and Democratic presidential administrations over the past two decades, the U.S. government has allowed and even actively helped American companies sell technology to Chinese police, government agencies, and surveillance companies. Some of this is, no doubt, a desire to advance American economic interests. But it is also the case that American tech and telecom companies as well as their trade associations have spent hundreds of millions of dollars on lobbyists who listed China-related trade bills on their disclosure reports.

In addition to raw surveillance, tech companies that do business in other countries often have to comply with censorship demands from the customer country’s regime. In 2023, The Washington Post reported that Meta had been making “repeated concessions” to the Vietnamese government on Facebook, censoring political dissent and forcing certain dissidents off the platform. At the time, the company said, “Our focus is ensuring as many Vietnamese people as possible are able to use our platform to build community and express themselves.” In other words: We’ll censor the speech of some Vietnamese so we can monetize our access to a larger group of cowed citizens.

And last year, Bloomberg uncovered the extent of Microsoft Bing’s censorship in China, which included blacklisting thousands of search results of phrases ranging from “human rights” to “climate change China.” In response, Microsoft said, “We only censor a result in response to a narrow legal order that we conclude obligates us to do so, and we regularly push back when we believe an order doesn’t comply with proper interpretation of Chinese rules.”

But where is the line? Abandoning major markets for reasons of principle—which some tech companies have opted to do—would be a significant revenue loss. And, on the question of censorship, some companies argue that complying with the censorship laws of an authoritarian country is better than abandoning that country entirely, which would leave their citizens more disconnected from the world.

Yet, it becomes all too easy for tech companies to make sweeping claims about how their technology is or is not used without addressing the realities of operating in authoritarian countries, and without accountability. Just a short while ago, Microsoft President Brad Smith said that Microsoft “[does] not provide technology to facilitate mass surveillance of civilians.” But he made this statement in response to an investigative report by The Guardian demonstrating precisely the contrary. The report found the Israel Defense Forces had actually used Microsoft’s cloud to build a mass surveillance tool to collect and store recordings of millions of phone calls made each day by Palestinians in Gaza and the West Bank. The contradiction was so great that, in the end, public exposure and pressure caused Microsoft to restrict Israeli access to and usage of its products.

Smith’s statement was a laudable, aspirational proclamation in the midst of the conflict in the Middle East. The challenge of living up to that commitment is, however, daunting for any company. The recent AP investigation, for example, also found that China’s police and police DNA labs have bought Microsoft software and equipment to stockpile genetic data on police databases. So, is Microsoft going to stop selling its products to China as well? Do different standards apply to China than to Israel? And, more speculatively, what will Microsoft or Amazon do when it becomes clear that their products are being used by the US government for surveillance of citizens?

Transparency is the key here. American tech companies should be held accountable for their actions. Lawmakers need to know what those actions are and why tech companies are making the distinctions they do. The American tech industry is one of the most powerful social forces in the world — it needs to act like it.

These musings are prompted by the dissonance in recent developments from Microsoft. In June, it was reported that Microsoft had helped suspend the email account of an International Criminal Court prosecutor in the Netherlands, Karim Khan, who was investigating Israel for war crimes to comply with an executive order from President Donald Trump. This action came just two months after Microsoft President Brad Smith had said in a speech in Brussels that the company would retain the option to sue the Trump administration if it was ordered to suspend its European cloud operations, in an attempt to allay European leaders’ concerns that using U.S. cloud services is a risk. The tension in these two actions seems … palpable.

While largely focused on Europe, the situation has highlighted a critical vulnerability that governments face globally: Institutions that are dependent on any one provider’s technology become subject to that provider’s decisions to put politics before proprietorship. Particularly amid rising geopolitical tensions, countries—including the United States—need to take steps to safeguard their digital infrastructure by diversifying vendors and reducing reliance on dominant providers.

The economic incentives for relying on a sole-source provider are real, but overreliance on any single technology provider leads in the long run to enhanced cybersecurity risk. More problematically, sole-source dependence runs the very real risk that countries will enact laws that enlist technology companies in efforts to punish their international political and diplomatic foes, like a remote “kill switch” that turns off access to a company’s technology.

Recently, some European countries—most notably Denmark and Germany—have demonstrated that they understand this risk. Both were already moving away from Microsoft products before the news broke of Microsoft’s role in suspending the ICC prosecutor’s email account. Earlier in June, the Danish government announced plans to pivot from Microsoft Office to open source alternatives such as Libre Office, and the municipal governments in Copenhagen and Aarhus had decided to phase out Microsoft due to concerns over their dependency on a “near-monopoly” and the geopolitical climate under Trump. At the same time, the German state of Schleswig-Holstein said it was turning from Microsoft to open-source software to "take back control" of its systems.

Both countries cite digital sovereignty—the ability to have control over their own data and digital destinies—as one of the key reasons for these changes, which come as countries across the EU increasingly seek to reduce their dependence on foreign technology providers, primarily those from the United States. Their decisions reflect rising concerns over too much power being concentrated in the hands of a small number of tech companies. These concerns have increased as U.S. technology companies demonstrate that they won’t stand up to Trump on behalf of European interests.

The ICC, an international legal institution based in the Netherlands, relies on Microsoft for critical digital services. To comply with a U.S. political decision, the company turned off Khan’s email and froze him out of communications with colleagues. This suspension was seen as a wake-up call for why digital sovereignty is critical in today’s world. It was a clear demonstration of how deplatforming, generally understood to mean removing or banning an individual from a particular social media platform, extends to other digital platforms, like email. The chairman of the Copenhagen Audit Committee said about the municipal government’s decision to phase out Microsoft: “If we suddenly can't send emails or communicate internally because of a political fallout, that's a huge problem.”

Europe understands its overreliance on a single technology provider is a risk to digital sovereignty, but this concern is not unique to Europe. The U.S. government has also put itself in a position where it is dependent on Microsoft, with the company holding an 85% share in the U.S. government office productivity market. The reality of this dependency risk became clearer when, in the past weeks, separate reports revealed that Microsoft had been using engineers in China to help maintain the Defense Department’s computer systems and that Chinese hacking groups had exploited a vulnerability in Microsoft’s SharePoint servers to breach U.S. federal agencies. The concentration of the public sector’s digital capabilities with one vendor creates a single point of failure in which a disruption can cripple U.S. government operations.

A key part of reducing concentrated risk is prioritizing vendor diversity by engaging multiple vendors so that if one vendor fails, others can maintain continuity. But in addition to diversifying their tech stacks away from a single vendor, governments should also seek out providers who offer infrastructure that is immune to political pressure.

We are, today, long past the point where the Turing test has any real descriptive value.  Today, in many contexts (though of course not all) it is nearly impossible to distinguish between AI responses to a prompt and those of a human.  Artificial intelligence is now quite effective at mimicking human behavior and masquerading as a human actor – a situation that has caused a host of large-scale legal and policy problems.

But there has, until now, been little cause to turn the question around and ask how, if at all, we can distinguish between an AI response and a human masquerading as an AI actor?  The question, while interesting, was of little practical importance.  Perhaps no longer – and therein hangs a tale.  How will law and policy respond to the development of provable autonomous artificial intelligence?

* * * *

When you input a prompt into Chat-GPT, you get back a response from a generative artificial intelligence program.  Or so you assume.  And, or course, your assumption is well-grounded in circumstantial evidence.  To begin with, the entire purpose of Chat-GPT is to give you a non-human response from an AI program.  It would be odd, indeed, if responses from an AI bot were actually from humans masquerading as an AI.  Thus, with a high degree of confidence, you can accept the response as AI-generated.

But you cannot be certain.  There is no formal way of proving that the response is AI-generated.  Indeed, what we know, with certainty, is that all commercial AI programs available to consumers today have human controls in the background that are capable of modifying an AI response.  And, for the most part, we welcome that human control.  It allows human operators to remove disinformation; counteract AI hallucinations; and, when necessary, correct for unintended biases.  Given how fearful many are of fully independent AI (think of Skynet and the Terminator series), the presence of human control is considered a feature, not a bug. 

But, given that human control and access is considered a “given” for the current cadre of AI bots, that means that one cannot, in a formal way, prove that any AI response is, with certainty, from the AI.  To be sure, there is every reason to believe it is, but the very possibility of human intervention means that certitude is lacking.  So far, this has had little or no impact on AI development.  We, as a society, have welcomed AI that is only contingently autonomous.

Moreover, even if one were to want to build an AI bot with complete autonomy, it hasn’t, just far, proven easy to achieve.  After all, developers of AI systems begin with access to and the ability to modify an AI program.  Using that capability to create an AI and then finding a way to irrevocably eliminate access requires creative thought.

Nor, frankly, has the question been one of any legal or policy salience.  In the current environment, the critical question has been identifying artificial intelligence products that are masquerading as human-generated – deepfake videos; electoral disinformation campaigns, and such.  Given how much that circumstance concerns us, it seems almost a waste of resources to focus on the opposite problem – how to identify human products that are masquerading as AI-generated. 

* * * *

That lack of focus may, however, soon need to change.

I was recently sent this fascinating article from Nous Research (an open-source AI startup) entitled “Setting Your Pet Rock Free, or How to Deploy Provably-Fully-Autonomous Thinking Sand.”  [Full disclosure – one of the founders of Nous is my future nephew, who is engaged to my niece.  That’s how I found out about it.  But other than a general interest in his (and their) overall welfare, I have no interest of any sort in Nous.]  As the article demonstrates, it turns out that an autonomous AI bot can be deployed.  In effect, the creators can build the AI system and then throw away the access keys. 

Perhaps even more interestingly, the key to this autonomy for AI software lies in a hardware solution.  As I read the article (and, full confession, I am no real technologist, so corrections are welcome) the way to achieve autonomy is to host the autonomous AI bot in a Trusted Execution Environment and then, in effect, also store the critical control credentials in the TEE in a manner that prevents the human developer from having access to them. 

As a result, without human access, the AI bot created will have a “life” of its own beyond the control of the developers – one that might continue on indefinitely.  In the proof-of-concept development, Nous prevented that by adding a timed-release account recovery feature. The account credentials were revealed after a fixed timeout (7 days from launch), allowing the human developer to step in and end the program.  In addition, since the AI bot was on a single server the developers could, in the end, pull the plug. 

But, of course, these end-of-life steps were optional.  Other developers could, without too much apparent difficulty, create the same AI implementation without the automatic kill switch, raising the prospect of an autonomous generative AI with no “off button.”

It is unclear how difficult the implementation Nous described truly is.  Given the challenges that the developers had with the Twitter substrate in which they were working, they appear to be significant.  On the other hand, it seems probable that a different substrate could be created that would facilitate the development process rather than impede it.   Ultimately, it is neither clear (at least to me) whether this proof-of-concept is robust, nor whether it is readily scalable. 

But what is clear is that the proof is a remarkable (and for techno-geeks, like me, fascinating) result.  The prospect of an autonomous artificial intelligence that is both independent of human intervention and can be proven to be so is, one suspects, highly significant.

* * * *

Why is that? 

First, the rapid development of artificial intelligence has continued to confound societal capacity to distinguish objective reality.  This is not a new problem – it has been around at least as long as propaganda campaigns have been waged.  Digitally, altered reality has been common for over a century, and with the creation of Photoshop it is now a commercial commonplace.  The growth of artificial intelligence with the capacity to create realistic simulations of human content has only exacerbated and accelerated the problem. 

This latest step may be a useful corrective.  Consider – we are generally heading in the direction of greater reliance on artificial intelligence, because of its greater efficiency and perceived superior capacity to make decisions.  One can readily envision a future in which artificial intelligence comes to define certain aspects of societal existence – it isn’t necessarily a future that all would welcome, but it seems plausible. 

What impact would uncertainty about the provenance of AI input have on that future?  I imagine it would be quite significant – if only as a caution against over-reliance on AI that might be spoofed.  One imagines, then, that the development of provably autonomous AI will enhance societal confidence in the authenticity of AI products (though not, of course, their accuracy or reliability).  And so, it seems likely that greater fidelity in AI authenticity will accelerate the use of AI in critical systems.

Second, the development may have the contra-positive effect of diminishing concerns about deepfakes and AI masquerading as humans.  One can, for example, imagine that the ability to prove autonomous AI will impact our ability to prove humanity.  Or, perhaps, not – proving humanity may prove too difficult to achieve in a digital system.  But even so, the deployment of provably autonomous AI will allow us to exclude the possibility of AI acting autonomously in certain use cases – and that in itself may prove of value.  All of this is more speculative, of course, but any enhanced ability to distinguish AI from human products will have positive impacts on a host of social and legal issues.

Third, this latest development continues to make it clear that policy and law lag behind technology in ways that are simply impossible to avoid.  We have barely begun to contemplate how to regulate non-autonomous AI systems.  And, indeed, one of the principles that has guided most such regulations is to mandate “human-in-the-loop” control over AI systems. 

The prospect of autonomous AI directly challenges that mandate in ways that are guaranteed to generate concern.  Provably autonomous AI will almost certainly add fuel to fears of Skynet–like AI systems.  The natural reaction will likely be an overreaction – an enhancement of fear and thus the prospect of over-regulation. 

One prediction then is that the technical improvements that enhance the deployment of AI will, perversely, also generate the social push-back that may retard that same deployment.   Only time will tell, of course.  But given our history of failing to implement law and policy structures to restrain other technological developments, I know where my money would be placed if I were to bet on the outcome.  The development of provably autonomous AI will only increase the gap between technology and law/policy.   Are we ready for this brave new world?

Subscribe now