AI progress has been both fast and slow. What an odd thing to write, right? Let me explain…

The whole idea that some people in the past thought that AI progress “might slow down as we make further progress into the RL regime, but we have seen progress in 2026” is a form of retrospective straw man. And a powerful one at that, difficult to falsify, precisely because we can’t identify the person who defended that thesis, in that form, in the past. In reality, the debate was more complex than that. Silver and Sutton posited that “reward is enough” to solve AGI; Vamplew et al. argued the contrary. In July 2025, Karpathy defended the idea that RL will still yield progress, even though he didn’t believe it to be the full story. Some people probably thought naive long-horizon RL has bad credit assignment, which is not wrong per se, but also doesn’t mean progress wouldn’t come through other RL techniques. LeCun/Marcus/Chollet-style critiques are about the need for new paradigms beyond current LLM/RL scaling to reach AGI. Some of them actually admitted that progress would still continue under the current regime, but would not be enough to achieve what they consider “genuine intelligence”, the capacity to generalize to unseen tasks efficiently. Again, that is a completely different position from the straw man version, and a position that hasn’t been proven wrong by recent real or perceived AI progress. The most pessimistic take was probably Toby Ord’s. He wrote in October 2025: “Now that RL-training is nearing its effective limit, we may have lost the ability to effectively turn more compute into more intelligence.” But even he didn’t believe the slowing of progress would happen before 2027, which doesn’t match the straw man version, and is thus not contradicted by real or perceived AI progress in 2026.

There is a natural, often subconscious tendency to use current real or perceived AI progress as proof that the skeptics were wrong, by summoning a weakened version of whatever one wants to put in the skeptics’ mouths. In reality, the skeptics were both right and wrong. They were right to think that doing RL with single scalar outcomes wouldn’t work for long-horizon tasks. But what they didn’t anticipate or voice more clearly is the wonders of algorithmic progress and techniques like process supervision, which means frontier training doesn’t have to wait for a single noisy reward at task end. This works particularly well in domains where building strong verifiers is easier (math, code) and less so otherwise. For that reason, recent AI progress has been jagged: somewhat impressive or fast in domains like coding and math, somewhat slow in domains like creative writing and non-sycophantic, helpful assistance.

Hence my initial sentence: AI progress has been both fast and slow.

And even what we consider fast AI progress is probably a bit overrated given the general enthusiasm. Let’s take coding, for example. It is arguably the domain where AI progress seems to have been fantastic in part thanks to RL. Anthropic’s models went from 33.4% on SWE-bench Verified with the original Claude 3.5 Sonnet to 62.3% with Claude 3.7 Sonnet, and later to roughly 87.6% with Claude Opus 4.7. OpenAI’s models went from 69.1% on SWE-bench Verified with o3 to 77.9% with GPT-5.1-Codex-Max. But it is almost a consensus that SWE-bench shows signs of saturation, and only measures a tiny, relatively easy portion of what software engineering is all about. As a result, most frontier model labs have been moving away from SWE-bench Verified for the benefit of other benchmarks like SWE-bench Pro, OSWorld, and internal agentic software-engineering evals where progress is real but either less spectacular or indicative of significant room for improvement. On SWE-bench Pro, GPT-5.5, the latest model from OpenAI, scores 58.6%, compared with 56.4% for GPT-5.2-Codex. For Anthropic, scores went from 53.4% to 64.3% between Opus 4.6 and Opus 4.7, and could reach 77.8% for Claude Mythos Preview if the rumors are to be believed. The progress is still there, you might be tempted to say, and you would be partly right. At the same time, how do you reconcile that progress with the fact that none of these SOTA models solved a single task on ProgramBench — a benchmark released in May 2026 by Meta FAIR and collaborators that asks agents to rebuild real software projects from scratch using only an executable and documentation? ProgramBench is much closer to holistic software engineering than fixing a contained GitHub issue. And its results tell a different story from SWE-bench Verified or Pro.

The progress is still there. The relevant question is: progress on what? On narrow issue-resolution benchmarks, progress looks spectacular. On broader software-engineering tasks that require designing and building systems from scratch, the best public models are still far from reliable autonomy. Fast and slow. Jagged.

So in a sense, the skeptics were right to think long horizon tasks would be hard. That is still the case. We just don’t perceive it that way because in software engineering, you can still decompose long-horizon tasks into issue-resolution-like short-horizon tasks, and accomplish impressive things with a human in the loop. The AI coding progress is as much about model improvement as it is about harness improvement and better human-AI collaboration.

The marvel of coding agents is that they combine three things: the harness, where static human intelligence is crystallized as code; the model; and the human-in-the-loop who provides dynamic human intelligence at inference. The problem is that we often conflate these three things as model intelligence.

Where the skeptics get it wrong is in thinking that lack of progress on one axis means overall progress stalls. Reality has been different because there are so many other axes that could still yield improvements: new algorithms, modalities, harnesses. Innovation will always untangle complex knots; the question is just how fast.

P.S. I wrote this post back in May. A lot has happened since, but I think my thesis still pretty much holds. We continue to see fast and sometimes impressive progress in verifiable domains, and less so everywhere else. I do not want to downplay that progress in any way, but it is also important to keep in mind the limitations that come with it.

I think current progress can often seem pervasive because much of it is progress in coding. It turns out that coding is an interesting specialization because its primitives—memory, computation, tool use, iteration, and verification—are also useful across an enormous range of tasks. Any task involving information and computation can make use of this general substrate that coding provides. That is the beauty of it.

Once the world has been digitized into files, databases, APIs, and software systems, code becomes an interface to economic activity. The consequence of software eating the world is that AI agents can now become useful across huge chunks of economic activity simply because they are increasingly capable of manipulating software.

AI agent benchmarks are increasingly important because we are in the agentic phase of AI. We are no longer in the text-in-text-out phase of early GPTs through GPT-4o. Now LLMs have access to tools: web search, code execution, MCP servers, APIs, and more.

Why? Because tools allow us to palliate (partially) the deficiencies of text-in-text-out. To reduce hallucinations, add web search to ground answers in facts. LLMs can’t reliably do additions or multiplications of high-digit numbers or fractions ? Add Python execution to handle that and push the answer back into the LLM loop.

That’s what we now call AI agents, or to be more precise, LLM agents in most cases.

AI Agents Are Growing in Importance

AI agents are widely used, especially in software engineering: Cursor, Codex, Copilot, and many others. That acceleration was catalyzed in part by benchmarks like SWE-bench.

So the equation is simple: more AI agents means more need to benchmark them to understand capabilities, failure modes, limitations, and track progress.

But Running AI Agent Benchmarks Is Costly

The Holistic Agent Leaderboard (HAL), a project I contributed to, required roughly $40,000 to evaluate agents across nine benchmarks. A single agent run can cost anywhere from $22 to $2,829 depending on the benchmark and model.

Why so expensive? There’s no single forward pass. Instead you have loops, tool calls, huge context windows from file parsing, API responses, and more.

This Creates Big Problems

Independent researchers and small labs can’t maintain many leaderboards. It’s difficult to do multiple runs for statistical robustness. It’s difficult to try many scaffolds if one run could cost $2,000 and you don’t have the budget.

Which Led to Exploring Benchmark Reduction in the Agent Setting

Can we evaluate agents on fewer tasks while still getting reliable rankings?

Prior Work Exists for NLP and LLM Benchmarks

Previous research has shown that NLP and language model evaluations can often be compressed to much smaller task subsets while preserving model rankings and score predictions.

But the Agent Setting Is Different

Agent benchmarks differ from standard LM evals in three important ways:

• Fewer tasks to begin with. Agent benchmarks typically contain dozens to hundreds of tasks rather than thousands, so the reduction problem is more stringent.

• Each task is far more costly. Each task requires a full agent loop with many forward passes, not a single prompt-response pass like in classic LM evaluation.

• Scaffolds matter. For agents, the scaffold—the logic around the LLM including memory, loops, and tool use—has tremendous impact on performance as I explained here. So benchmark reduction should also generalize to scaffold changes and over time, something very specific to agent benchmarks.

The Solution: Mid-Range Task Selection

For my first solo research paper, I decided to work on benchmark reduction in the agent setting. I tested six task selection strategies across five evaluation protocols on eight benchmarks covering 33 agent scaffolds and 70+ model configurations.

The winner: the Mid-Range Difficulty Filter. It reduces benchmark task count by up to 70% by selecting only tasks with historical pass rates between 30% and 70%.

It beats greedy forward selection (which overfits and fails under distribution shift), random sampling (high variance, worst cases as low as ρ = 0.54), stratified sampling, and extreme-difficulty baselines.

The mid-range filter achieves mean Spearman ρ of 0.94 with a worst case of 0.87—maintaining reliable rankings while cutting costs dramatically.

Why It Works: Item Response Theory

The justification comes from Item Response Theory. Under a standard psychometric model, the Fisher information about latent ability is maximized at pass rate p = 0.5. Tasks where most agents fail or most agents succeed tell you little about relative rankings. Tasks in the middle are where agents are most reliably discriminated against.

Read More

The full paper with code and data is available at: Efficient Benchmarking of AI Agents

As part of the Holistic Agent Leaderboard (HAL) initiative at Princeton CITP, we evaluated more than 220 agent runs across 9 benchmarks, the equivalent of over 20,000 agent rollouts across 9 models and 9 benchmarks for a total cost of $40,000. The benchmarks are: AssistantBench, CORE-Bench Hard, GAIA, Online Mind2Web, Scicode, ScienceAgentBench, SWE-bench Verified Mini, TAU-bench Airline, and USACO.

In that process, we “burned” 2.6 billion prompt tokens and learned a lot along the way. In this article, I’d like to share some of the insights we gained, with a particular focus on the GAIA benchmark.

GAIA is a benchmark that evaluates skills such as reasoning, web use, tool orchestration, and multimodality. It comprises 450 tasks in total. We evaluate agents on its public evaluation set of 165 tasks, which are divided into three difficulty levels.

The L1 difficulty level typically involves basic reasoning, often requiring no tools or fewer than five steps. The L2 level includes tasks that require tool integration and generally involve around five to ten steps. The L3 level encompasses tasks that demand extensive tool use and often involve open-world actions, making them the most complex and challenging type of tasks.

We evaluated 16 models using two scaffolds on GAIA. It’s important to understand that within the Holistic Agent Leaderboard, there are three key axes: agent scaffolds, models, and benchmarks. For each benchmark, we typically evaluate two agent scaffolds across roughly ten or more models.

Let’s revisit what we mean by an agent scaffold.

What we commonly refer to as AI agents today are, in most cases, a combination of a large language model (LLM), tools, and the hardcoded logic that orchestrates their interaction. For instance, you might use a framework like the smolagents library to build an AI agent powered by GPT-5. In that case, GPT-5 serves as the brain, responsible for the verbal reasoning required to solve the problem at hand.

To do so effectively, the brain must also leverage tools that complement its limitations (for example, LLMs are not particularly good at arithmetic), provide access to new information (such as through web search or file reading), or enable control over external systems (like bash for file manipulation or other APIs).

That entire surrounding logic and control structure is what we call the scaffold.

The two agent scaffolds we evaluated as part of the GAIA leaderboard are the HAL Generalist Agent and Hugging Face Open Deep Research. The HAL Generalist Agent is a minimal scaffold with no task-specific optimizations. It provides access to general-purpose tools such as web search, web browsing, and file editing.

In contrast, the Hugging Face Open Deep Research scaffold is task-specific and designed to enhance performance on benchmarks like GAIA by enabling deep research. This capability is also present in systems such as the ChatGPT app, the Anthropic app, and the Grok app. Most of these chat platforms already include some version of deep research functionality.

First of all, as mentioned earlier, the higher levels in the GAIA benchmark are where many models tend to struggle. The top-performing model we evaluated is Claude Sonnet 4.5, which leads the leaderboard with 75% accuracy. Notably, Claude Sonnet 4.5 also performs strongly on Level 3, scoring 69%, which is almost 15 percentage points higher than the next best model.

Out of the 165 tasks in the public validation set of GAIA, 160 have been solved by at least one agent, indicating substantial coverage of the benchmark. However, no single agent has yet achieved complete coverage or solved all tasks successfully.

In most other benchmarks on the Holistic Agent Leaderboard, the generalist agent usually performs worse than the task-specific one, which is expected since task-specific scaffolds are typically optimized for higher accuracy.

However, GAIA shows an interesting exception. On GAIA, the HAL Generalist Agent achieves higher accuracy for 12 out of 16 models and is more cost-efficient for 14 out of 16 models. A few models — such as GPT-5 Medium, GPT-4.1, o4 Mini High, and o3 Medium — perform slightly better with Open Deep Research, but usually at a significantly higher cost.

Interestingly, all of these are OpenAI models, which highlights a strong interaction between models and scaffolds. Most prior evaluations do not control for or compare across scaffolds, meaning that such interactions often go unnoticed. This is an important point for developers of AI agents to keep in mind: the design priors used in building a scaffold can unintentionally bias it toward a specific family of models.

For example, if you design and test your scaffold exclusively on OpenAI models, iterative tuning of prompts or logic may make it inadvertently better suited to that model family’s behavior, and less compatible with others. Scaffolds are not neutral. It is crucial to consider the interaction between the scaffold and the embedded LLM, as the resulting performance differences can be substantial.

On the HAL website, we also plotted the Pareto frontier points, which represent the best accuracy-to-cost trade-offs across all the runs we conducted. All the Pareto frontier points correspond to HAL Generalist Agent runs. This reinforces the point mentioned earlier: in this case, the HAL Generalist Agent is significantly more efficient than Open Deep Research. It achieves higher accuracy at lower costs, while Open Deep Research runs tend to be much more expensive.

This finding suggests that a simpler scaffold can often outperform more complex, task-specific designs on GAIA. More broadly, it shows that increasing the complexity of an agent scaffold does not necessarily lead to better accuracy.

It is also worth noting that the Open Deep Research scaffold was implemented using the smolagents library. Architecturally, it is an agent capable of using another agent as a tool. This type of nested or recursive structure — where the primary agent is granted high autonomy and can invoke another agent with access to its own tools — can quickly make the overall system less efficient.

When designing agent scaffolds, it is therefore important to focus on simplicity and efficiency, and to avoid unnecessary complexity. While overcomplicated scaffolds may show temporary performance gains during training or evaluation, their performance often degrades rapidly in broader or longer-term use.

Finally, one striking observation was that more reasoning does not always lead to better performance on the GAIA benchmark, which was quite surprising. Intuitively, you would expect that giving an agent more time to think would significantly improve its results — and in four cases, it did. However, in four other cases, performance actually decreased.

This finding is particularly interesting because it suggests that the scaffold design can matter more than the model’s raw reasoning ability. In other words, when building AI agents, the way you structure the surrounding logic, tools, and control flow can have a greater impact than simply allowing the model to reason longer. Of course, having a powerful model remains crucial, and we do see that the more capable models achieve excellent performance on the GAIA leaderboard. Yet, in some situations, the advantages of improved scaffolding outweigh the gains from additional reasoning depth.

These are some of the key insights we derived from running the GAIA benchmark and evaluating its results as part of the HAL initiative . As mentioned earlier, we also run a range of other benchmarks that are available on our website with all the traces of these runs readily available for download . A detailed summary of our findings is presented in our paper now available on arXiv:

https://www.arxiv.org/abs/2510.11977

Over the past five years, Doctolib’s growth has also averaged 22.5% per year, which indicates steady expansion.

France remains Doctolib’s primary market, representing 80% of its 2024 ARR (or approximately €278 million). Germany, meanwhile, is in second place, with 7% of its ARR (approximately €59 million), 25 million registered patients, and 100,000 healthcare professionals. Italy represents approximately 2.6% of its ARR (approximately €9 million). Doctolib’s European expansion plans are clear, even if profitability questions remain.

Doctolib has reduced its operating loss by 38%, from €87.1 million in 2023 to €53.8 million in 2024. Profitability is expected this year. But will it be a typical SaaS break-even event ?

Founders often report 6–8 years on average to reach profitability; many only IPO around year 12. Doctolib was founded in 2013. It therefore took 13 years to reach profitability, which is significantly longer than a typical SaaS, even though it aligns with many high-growth health-tech peers. Doctolib’s revenue still trails those of its peers Cegedim (€654.5 million in 2024, up +6.3% vs. 2023) and CompuGroup Medical (€1,154.0 million in 2024, — 3% vs. 2023 (€1,187.7 million).

To be fair, the comparison is not entirely “iso” because Cegedim and CompuGroup Medical have large portfolios of different SaaS solutions. Cegedim, for example, has media solutions and cybersecurity solutions. However, I think the comparison still matters because it tells a lot about the industry Doctolib is in, an industry where players have to do a lot of different things in hopes of achieving profitability. And even when profitability is achieved, the net margin can be extremely thin.

In 2024, Cegedim recorded a -2.2% net margin (– 14.7 M€ on 654.5 M€ of revenue), despite a healthy 18.9% EBITDA margin. CompuGroup achieved roughly a 2.9% net margin in 2024 (≈ 33.6 M€ net profit on 1,154 M€ revenues). While positive, this sits below typical software and programming benchmarks (net margins around 14% on average) and below high-growth SaaS peers.

The reasons healthcare IT SaaS have low margins reside in the need to invest a lot of money on R&D and compliance costs, especially when you must interoperate with hospitals, clinics, and insurers. In some cases, you even have to deliver customized implementations, thus diluting economies of scale. And don’t forget the pricing pressure from public health systems and large group purchasers.

Fortunately for Doctolib, they can achieve economies of scale because of an “out-of-the-box” model: the same platform is replicated from one client to another, onboarding and implementation are done via standard workflows, and professionals (offices, clinics) subscribe online via packaged formulas. Also, so far, they do not have to deal with the strong pricing pressures of public healthcare systems or large buyers, such as some hospital software publishers. But this last point might not be true in the long term because for Doctolib to survive, they will have to enter public hospitals on a massive scale. More on that later.

In France, 50 million patients use Doctolib. France has approximately 68 million people, so this points to a saturation in the French market that did not translate into stellar profits. Doctolib is entering other markets in the EU, and that’s good. The problem is that these countries often have their own version of Doctolib already operating. Doctolib, the outsider, will have to fight tooth and nail to gain market share. 13 years after its creation, it is fair to say that Doctolib has lost the effect of surprise. Does all this mean that Doctolib is doomed? Not necessarily.

Doctolib will have to expand the set of solutions they propose. It’s the inevitable next step to grow and dethrone competitors like Cegedim, which are old (founded in 1969). Doctolib’s edge is the technological one. It comes from a modern, user-centric web and mobile experiences. Doctolib can ship features faster and better than incumbents. Doctolib should maintain that tech edge at all costs.

For the purpose of this article, I went on the Cegedim website and looked at all the solutions they proposed. It’s a lot. They are deeply entrenched. Dethroning them in the healthcare space, where government, decade-old relationships, and resistance to IT transformation are rampant, will not be easy.

But I also noticed something. The overall landing page seems dated. When I tried to look at the solutions in the English version of the page, I couldn’t scroll down. Look at the GIF. In the English version, nothing moves, but I was actually trying to scroll down.

And that’s just the landing page. Now imagine how dated and not user-friendly the SaaS solutions they propose to hospitals, clinics, and others should be. Their weakness is clearly in technology, and Doctolib can exploit that. The first step, which is the hardest one, will be to find an entry point, a breach. It is totally possible.

It’s worth noting that Doctolib’s path to profitability was lengthened by its aggressive R&D spending: the company earmarked €92 million for innovation in 2023 (a 24 % increase over 2022) and maintained a €100 million R&D budget in 2024 — roughly 29 % of its €348 million ARR — underscoring a deliberate choice to prioritize long-term tech leadership over short-term margins.

Those investments have fueled AI-driven features like the Consultation Assistant — which has already supported over 1.5 million patient encounters by automating note-taking and streamlining workflows — and the October 2024 launch of an auto-dossier population tool, further cementing Doctolib’s competitive moat by embedding advanced intelligence deeply into its platform.

By reaching the level of revenue they’ve reached and delivering a stellar customer experience (83% of general practitioners and pediatricians say the service saves them time and improves their daily workflow), Doctolib has built the credibility necessary to disrupt incumbents.

The fight won’t be easy, though. Let’s wait and see.

Read more:

What Machine Learning Boils Down To

Yesterday, I wrote an article about the release of the Llama 4 suite of models. In that article, I explained why I no longer give too much importance to most LLM benchmarks. You can read it here:

Llama 4 and the Differentiation Game

The truth is that most LLM benchmarks are easily hackable.

Why?

Because it is difficult to curate a set of question-answer pairs that measure intelligence and not just brute force memorization or a subtle form of that.

The only benchmark that is a bit resistant to hacking is ARC-AGI. But even then, a bit of fine-tuning can break it, as we saw when OpenAI released a fine-tuned version of o3 that achieved an impressive score on ARC-AGI (87.5% on semi-private eval). To reach that score, this is what OpenAI did:

They trained o3 on 75% of the Public Training set.

Traditional ML engineers might say: “Well, there is no problem with that. It’s not like they trained on the test set.”

And normally, I would agree. But in this case, nothing says that the public training set distribution is not very similar to the test set distribution, so much so that training on a fraction of the training set is enough to capture most of the patterns one might have in the test set, not with generality but a pattern matching hack.

So that’s not really cheating in a traditional sense, but it is not an increase in generality either. Or to be precise, it is a narrow increase in generality that only works on a limited task domain.

It basically means that to boost performance on task A, you need to fine-tune your model or adapt your model based on versions of task A. That’s not news in the ML world. The only difference is that, given the very large base models we now use, one no longer needs a huge fine-tuning dataset to achieve impressive performance. But even that is not huge news in the ML world.

With enough parameters and compute, you can literally fit anything. However, that comes at a cost: overfitting.

The overfitting curse

Read more

Klarna wants to do an IPO this year. For those who follow the BNPL (Buy Now, Pay Later) industry, that’s not news. But apparently, they want to do it so badly they are willing to go into strange places.

The PR campaign preceding the IPO is, to some extent, typical of what one might expect from a tech company. It started with Klarna’s CEO doing everything to ride the AI wave. Sebastian Siemiatkowski announced grandiose cost reductions allegedly attributed to the use of LLMs and chatbots, vaguely declared getting rid of a lot of SaaS without explaining how and why, and threw some shade at Salesforce before backtracking.

As someone who deeply follows what happens in the AI space, I always found Klarna CEO’s grandiose announcements a bit suspicious and hype-infused. I still think he is exaggerating the impact of AI in cost reduction. I think Klarna essentially decided to do some cost reductions to have a strong P&L pre-IPO and decided to exaggerate AI’s role in it to surf on the current AI hype cycle.

And apparently, they decided to move back from AI chatbots to human agents quickly after…

But okay, I get it. They do what they have to do to better position the company. Over the past few weeks, they continued the barrage of positive announcements, notably securing an exclusive provider deal with OnePay, the Walmart-backed fintech company.

Another announcement took me by surprise, though. And not in a good way. Klarna announced a deal with Doordash. Yes, you read that correctly: Doordash.

Doordash customers will soon be able to buy food in interest-free installments via Klarna. This is crazy to me. People will buy their burritos in interest-free installments. Wow.

Sure, I get the interest for Doordash. BNPL will probably boost conversion rates and average order value while reducing the risk of non-payment. For the customer, I really don’t see how this is not in some way encouraging bad behavior or profiteering off one of the most basic human needs: the need to eat.

Don’t get fooled by the “interest-free” bullshit. Think about it. Who will most likely buy sandwiches in installments? Poor people, or people who can’t afford to pay in full directly. Klarna will basically indulge their temptation to eat stuff they don’t necessarily need under the pretense that the cost will be spread over several months. These people will live above their means, and when they can no longer pay, they will be charged late fees. That’s the catch: the late fees. It’s “interest-free,” but don’t forget the fucking late fees that are equivalent to 217% APR in some cases!

So if you bought something for $100 split into 4 payments, the maximum total late fees you could be charged would be $25 (if you missed multiple payments). That’s pretty high, actually.

Anyway, I was not the only one shocked by all this. People are obviously laughing at Klarna’s latest proposal on X— rightfully so, because it’s very laughable and scary at the same time.

Here is Andrej’s description of vibe coding:

Vibe coding is a way to embrace the power of LLMs (especially Claude Sonnet). LLMs are good at coding, so much so that one can easily develop web applications by just doing clever prompt engineering.

Since Andrej’s post, a new wave of ‘vibe coders’ has been taking shape. Indie hackers like levelsio are also in the party. Levelsio recently ‘vibe-coded’ a whole game and made more than 80,000 USD in revenue in a matter of weeks. Sure, his large audience and influencer status may have played a role, but it was enough to excite even more wannabe vibe-coders.

Vibe coding is now a burgeoning tech movement. Will it last? Only time will tell.

What is fundamentally interesting about vibe coding is that it frees the imagination. It allows for simple and rapid iteration on tech ideas. It helps lower the barrier to entry for software creation, even by non-coders.

As a result, thanks to vibe coding, it is easier to imagine a lot of home-cooked software, personal software being built.

Think about the software you use daily. Most of the time, you use less than 40% of its functionality. For most software, it is even lower. General-purpose software tries to cater to the needs of a lot of different users with one UI. The problem is that often means offering an average experience across the board for everyone instead of a highly specific one for each target user.

Vibe coding is nice for developing small prototypes and iterating faster, but what strikes me as potentially revolutionary is that coding assistants, when used properly, will basically reduce the cost of producing software and thus make it more likely that people will produce highly specific software with fewer features — just the features they need for their workflows. That’s what I call personal software or home-cooked software.

Now, I anticipate that not everyone will enjoy or want to create custom software. Most people will continue to enjoy general-purpose software, and that is fine. It is a bit like cooking. Everyone can cook at home, but most of us still like dining at a restaurant.

One of the reasons we have SaaS and general-purpose software is that for most people and most organizations, it is more cost-effective to outsource the cost of maintaining software quality to an external entity. Imagine having to maintain and develop new features for your custom Microsoft PowerPoint. That takes time and effort, and time is our most valuable resource as humans on this earth. Wouldn’t you rather spend those three hours playing with your son or something?

Vibe coding a prototype for fun is easy. Developing a robust software solution with auth, security, best practices, etc., is way harder, and vibe coders can’t do that yet. Maybe they never will.

Some vibe-coders are already facing the consequences of not knowing the code behind the applications they have generated:

So vibe coding is a nice way to democratize software creation (mostly simple prototypes), but a bit overrated. Don’t believe people who boldly claim that software engineering is dying. To build robust and complex software, you still need to learn, understand, and master programming languages and computer science in general.

The people who will primarily create personal software are those who already know how to code and will be able to prototype faster because of AI coding assistants.

Learn to code! Then vibe code !

Read more:

MCP Is Mostly Bullshit

OpenAI Is Going for Regulatory Capture

MCP Is Nice, but Have You Tried SmolAgents?

Building a Columnar File Format from Scratch in Python (1)

When you think about it, every modern cloud data warehouse is built on two key ideas:

• Columnar storage

• Separation of storage and compute.

Columnar storage

Columnar storage enables fast and efficient querying. Since the main use case of data warehouses is analytics and BI, you do want fast and efficient querying.

There are many types of columnar data formats (e.g., Parquet, ORC, Capacitor), often tweaked for the specific platform they are built for. But the core idea is simple. Since analytical queries often read specific columns (e.g., SUM(sales), WHERE country = 'US' ), storing data in columns allows for powerful optimisations:

• No need to access irrelevant data,

• The engine can just access the column(s) it needs and perform computations,

• You can compress data significantly.

Let’s say you want to execute the following query:

SELECT SUM(B) FROM table WHERE A > 10;

The original table looks like this:

If you use row-based storage, the internal representation of data consists of a succession of rows.

Consequently, executing the query described above will require, more or less (it’s a simplified model), the following steps:

For the execution, all the columns and all the rows are loaded, even though 80% of columns are not necessary to compute the result of the query.

If you use a columnar format, data is stored as contiguous columns.

As a consequence, the query engine only needs to manipulate two columns to achieve the same results. It works like this:

Column A: [5,15,8,25,12] → Filter → [15,25,12]
  
Column B: [20,30,10,40,25] → Extract → [30,40,25] 
 
Sum → 95

The execution is fast: only 2/4 columns are scanned (vs. 4/4 in row-based). When you factor in compression, you even get a smaller I/O footprint.

Row-based storage is optimized for OLTP (Online Transaction Processing), while column-based storage is optimized for OLAP (Online Analytical Processing). This is why platforms like Snowflake and BigQuery use columnar storage!

The separation of storage and compute

Modern cloud data warehouses decouple storage and compute to increase flexibility and scalability while reducing costs.

I think one catalyst of the decoupling was the rise of cheap object storage on the cloud.

Today, object storage is approximately 10x cheaper than on-prem SAN.

As a result, it makes sense to store all data in a cheap service like AWS S3 for example, and scale processing nodes as needed to handle queries.

In the pre-cloud era, databases were monolithic. Storage and compute were tightly coupled. Scaling up was vertical (more disk for storage, more CPUs) and costly.

With the cloud, storage is cheap, and it is easy to scale horizontally. Compute clusters can scale up/down instantly based on workload. That’s why the separation of storage and compute became an obvious choice.

Take Snowflake, for example. By using it, you can basically have your data living in S3/Azure Blob in compressed, columnar format (micro-partitions). For compute, you get virtual warehouses (clusters) that are spun up to process queries and suspended to save costs after finishing.

Snowflake tracks data location, statistics, and access patterns to optimize queries. You literally no longer have to manually tune hardware, index or partition your data. You can just focus on getting the most out of your data, fast.

Isn’t that nice?

Start using Snowflake now !!!

Read more:

Ezra Klein Is Wrong About AI

Lycee AI

Intelligence is not the bottleneck to progress or effectiveness in most companies.
That’s why having access to more intelligence — human or artificial — will not solve everything.

Increasing individual worker productivity won’t be enough either. I personally know how being super productive as an individual contributor is often even a disadvantage. If the system is not set up to reward increased individual productivity, you can end up resented by less productive coworkers who secretly plot your demise.

Moreover, the mere fact that you act differently from the normal, standardized, and unproductive way of doing things in the company means everything you do will probably be negatively perceived by the defenders of the status quo. This happens either because they are genuinely (but wrongly) convinced that the status quo is the best equilibrium state, or simply because they don’t like the change in pace, the pressure to increase their output, or the need to reevaluate their assumptions.

There is a simple concept in psychology that explains all of this: self-serving bias. People interpret situations in a way that protects their self-image. Those who have been successful under the current regime — and who, as a result, hold the most power and influence — will struggle to recognize or admit that the system they’ve been running is fundamentally unproductive and sometimes even toxic. It’s even harder for them to acknowledge this when revenues are still increasing.

Note: The self-serving bias also applies to you, dear reader, and to me. So, we’d better always ensure we are controlling for that bias.

When a company ultimately fails, there is always a lag between the actions that led to failure and the first hit on revenues and profitability. That’s why the collapse of big companies often feels sudden when, in reality, it has been years in the making. The truth is, revenues often continue to grow even though the seeds of failure have already been planted.

Another disadvantage of being an individual contributor with really high productivity is that your remuneration never matches the value you create or can create. That can’t be sustainable in the long run. Eventually, people just adjust their productivity to the average acceptable level in the company. There is a race to the bottom.

Putting chatbots everywhere to supposedly increase your workers’ productivity will only work if that productivity increase is rewarded by a growth in wages. If not, workers will bury the productivity increases into processes and ceremonies that give the impression of work being done while actually acting as productivity sinks. As always, it is all about incentives.

And if you are a business owner reading this and thinking you can still extract some of those productivity gains by increasing control over your employees (no remote work, increased workload), you will fail. Workers will always find a way around it, or the culture inside your company will degrade so much that you will self-select for the less productive people to stay while the most productive ones eventually leave. That’s not sustainable.

One of the reasons startups have a higher level of productivity is that most workers have skin in the game (equity). They know their hard work will directly increase their payoff. I think we should probably move toward a world where every worker has skin in the game. And the incentive should be clear — not some vague, eventual benefit if profits increase, where only the owner decides what that benefit will be.

But I’m not sure most business owners would want that.

Maybe AI will eventually force everyone to do that? We will see…

In the meantime, intelligence is not the bottleneck. Humans are the bottleneck. Bureaucracy is the bottleneck. Misguided incentive structures from business owners are the bottleneck. The principal-agent problem is the bottleneck. And that’s not completely a bad thing.

So what now?

AI tools should be seen as knowledge repositories — a quick way to gather relevant information to solve problems. They should be used as such, not as powerful oracles that can do anything. We should be very mindful of their weaknesses and not brush them off.

Personally, I find LLMs useful as better replacements for Google Search, as tools for conducting thought experiments, as assistants for text editing or formatting, learning, and coding. The mix depends on the person. But in every case, I remain fully aware of the risk of hallucinations and double-check what needs to be verified.

A little rant:

I see a lot of people and consulting companies hyping nonsensical use cases of AI models (mostly LLMs). I often say in my head, “How the hell is it possible that Chief Digital Officers are buying that bullshit?” Then I remember that a lot of it is just people trying to project an image of relevance, and nowadays having a generative AI initiative is the best way to do so. A lot of things that happen in the business world are ceremonies whose purpose is to make relationships or build one’s profile.

There is a lot of bullshit in this world, especially coming from people who walk around brandishing the right credentials (schools, titles, ex-companies). The problem is that, because of those credentials, people take them seriously even when they say or do the most nonsensical things.

I used to believe a bit in credentialism. Now I mostly hate it. I hate it because it does so much damage to the world as a whole. I hate it because the people with the credentials mostly end up believing they have all those privileges because they are really smarter than the rest of the population. They start believing their own bullshit and their own mythology.

Having been in that position myself (elite schools in Cameroon, then France to some extent), I know how much of that is bullshit. Bro, you are not smarter than the rest of the population just because you know how to do beautiful slides filled with bullshit… Get off your fucking fake pedestal!

These days, I try to judge people based on what they can really do, regardless of their credentials. And when I see people who have less, I remind myself that it does not mean they are less than. More often than not, it means that circumstances - many beyond their control - have played a significant role in depriving them of the opportunity to have more.

I am a big believer in personal responsibility, but I never discount the power of structural inequities.

I am sure the consultants who worked on the NYC trash-can study felt they were so smart and deserved the $1.6 million. Not to mention the very “good” and “smart” people who worked on turbocharging opioid sales.

I guess it often boils down to what Upton Sinclair said:

“It is difficult to get a man to understand something when his salary depends on his not understanding it.”

Read more:

Claude Is for Code

Build Powerful AI Agents Using smolagents from Hugging Face

The AI Enemy From Within

OpenAI finally released its biggest model yet: GPT-4.5. And it’s a flop.

It’s a flop because OpenAI built up so much anticipation about it for months, just for everyone to realize how mid it is in terms of performance. Look at the ARC-AGI semi-private evaluation above — GPT-4.5 is almost at the exact same level GPT-4o was. That doesn’t seem like an increase in intelligence, does it?

OpenAI’s response has been to say GPT-4.5 is not a reasoning model but a classic model that went through pretraining, supervised fine-tuning, and reinforcement learning with human feedback. As such, we should not expect it to thrive on benchmarks but rather to exhibit better creative writing skills and a much stronger grasp of facts (less hallucination).

But if that’s the case, why hype it up so much? Why release it now with pricing so absurd that developers will just skip using it? The messaging around GPT-4.5 has been clumsy at best. I feel like this is the first timing mistake from OpenAI.

And the argument that it is just a bigger GPT model that will later yield better reasoning models doesn’t even hold. DeepSeek-V3, a similar type of model — if we want to make apple-to-apple comparisons — is still better than GPT-4.5 on benchmarks like AIME and SWE-bench verified while being way cheaper, as pointed out by this anonymous X account.

Jeremy Howard also pointed out that the current performance of GPT-4.5 on Aider Polyglot, a robust coding benchmark, might indicate that OpenAI is in “deep trouble.”

So I, like many others in the AI space, am left wondering: Why did they release it now? What did they expect to achieve by releasing it now, and what does it say about what comes next in the AI space?

I have a theory.

Read more

xAI, the artificial intelligence company founded by Elon Musk to compete with OpenAI, finally released its most powerful model, Grok 3.

On the surface, Grok 3 is marketed as the world’s smartest AI. And when you first look at the benchmarks it might appear as such. In reality, the truth was purposefully hidden in plain sight. Most of Grok 3’s scores reported arecons@64, which is not a fair comparison with the other models. In LLM evaluations, “cons@64” refers to a metric where the model generates 64 separate output samples (or predictions) for a given prompt or question, and the final answer is determined by taking the consensus — typically the most common or majority answer among those 64 outputs.

When compared apples to apples, Grok 3 is less capable than o1 and o3-mini-high from OpenAI, and slightly better than DeepSeek R1.

It’s funny that a few weeks ago, the discourse on AI twitter was about the fact that we shouldn’t trust anything coming from China, because they often fake their numbers to serve CCP goals (when DeepSeek never faked numbers or engaged in marketing half truths). And now you have xAI founded by Elon Musk — the quintessential american tech bro, unofficial co-president of the US — engaging in marketing half truths to make Grok 3 appear more capable than it really is.

Which is not to say Grok 3 is not an impressive achievement. The quality of a model or a chatbot is about more than benchmarks. Benchmarks are overrated and are increasingly gamed. So to form a more comprehensive opinion about Grok 3, I decided to start using it for a few days. The conclusion:

Read more

We are in an AI bubble. The space has a $600B problem (probably more by now). Private and public valuations are through the roof. Like Paul Krugman recently noted, the S&P 500 PE ratio is sitting at about 30. It was 33 in 1999, just before the Dot-com bubble burst.

Nobody knows how long the exuberance will last. With the quest of AGI quickly becoming a global geopolitical battle, countries and firms are poised to make huge investments in 2025 and beyond.

The only thing we know for sure is that big AI clusters will be built, and Jensen Huang will continue to increase his wealth significantly. OpenAI recently announced project Stargate, a $500B undertaking to build AI clusters in the US. In France, President Macron unveiled €109B worth of AI investments, some directed towards building AI clusters and data centers.

There is a high probability that all these AI clusters will not serve the objectives for which they are being built, much like previous infrastructure buildups. The current assumption is that foundation models (LLMs primarily) will lead to AGI. According to the scaling laws, one only has to pour more data and more compute to achieve more “intelligent behaviour” from AI models. And with this newfound unlimited intelligence, researchers in all domains will discover faster, companies will become more productive, and abundance will reign over planet Earth.

That’s a very beautiful future isn’t it? Except, there is a good chance LLMs and other generative AI tools will never be the revolutionary technology everybody thinks they will be. Generative AI models are useful but also doomed to hallucinate (a friendly term to say ‘spit complete falsehoods’) because of how they are trained. They are not trained to be factually correct all the time or to generate images and videos where all objects respect the laws of physics.

It’s still early, but there is clearly a possibility that generative AI models will not serve more than nice replacements for Google search and MS Paint. Under that scenario, compute prices will decrease significantly as companies struggle to maintain decent utilization rates for their high-end GPUs. This is similar to what happened after the telecom crash of 2001, and the result was a massive surplus of underutilized infrastructure, leading to a sharp drop in bandwidth prices and a wave of innovation built on cheap connectivity. If we have video streaming (YouTube, Netflix), cloud computing (AWS, GCP), the rise of SaaS (Salesforce, Dropbox), VoIP (Skype, WhatsApp, Zoom, Discord), social media (Facebook, Twitter, Instagram, Snapchat, TikTok), it is in huge part because of ultra-cheap bandwidth after the telecom crash.

Similarly, as AI compute prices fall dramatically after the current bubble bursts, we could see an explosion of new applications and business models leveraging abundant GPU power. These GPUs will probably not be used for LLM, text-to-image, or text-to-video models training and inference. There is a high chance it will be for something completely different.

In the future, everyone will be GPU-rich. So my question for you is: “How are you preparing for an era of cheap compute?”.

DeepSeek R1 was a wake up call that forced the US to react. It shattered the idea that OpenAI and the US AI giants had some kind of proprietary advanced knowledge that made it almost impossible to catch up with them. It shattered the three moats OpenAI and co had been trying to consolidate in the past two years: compute, IP, and brand. How ? DeepSeek proved it was possible to catch up with OpenAI quite fast, and with less compute than previously thought. DeepSeek did it despite export controls on high end GPUs. DeepSeek did it despite OpenAI, Anthropic and co not publishing their research breakthroughs. And in a matter of days, DeepSeek’s brand gained huge popularity via their web and mobile apps. It looks like the real moat is the talent.

Narrative violations

Immediate reactions mostly came from OpenAI as its leadership at the top of the AI space was challenged by the impressive achievements of DeepSeek. Suddenly, to reclaim mindshare, OpenAI had to engage in a flurry of releases, including GPT-4o with scheduled tasks (a glorified reminder functionality), o3-mini and o3-mini-high available on ChatGPT (including the free tier), Deep Research, Operator, project Stargate ($500B investment pledge for AI infrastructure in the US), and the ability to read the reasoning traces of o3 models (in response to DeepSeek R1’s traces being available with no obfuscation).

Vladimir Ilyich Lenin famously said: “There are decades where nothing happens; and there are weeks where decades happen.” That was certainly true of the four weeks of January 2025 as far as the generative AI landscape is concerned.

In an AMA on Reddit, Sam Altman even went as far as to say OpenAI would make an effort to contribute more to Open Source initiatives, a stark reversal of the “proprietary” approach that allowed DeepSeek to claim the Open Source AI throne without much resistance by open sourcing DeepSeek V3 and R1. DeepSeek’s models effectively became the true “open AIs,” much to the satisfaction of huge swaths of the open source AI community.

Other reactions were not so constructive and revealed the shock in US tech cycles to realize they weren’t above everyone else as they initially thought. First there was disbelief about DeepSeek’s accomplishments. For some prominent US investors, DeepSeek was a CCP psyops, pointing out censorship in the chat app. Others doubted the veracity of the training budget communicated in DeepSeek R1’s paper, speculated that OpenAI’s secrets must have been leaked during San Francisco parties and that was the only way DeepSeek’s team could have known what to do (only US tech engineers can innovate don’t you know that?). Others accused DeepSeek of training on OpenAI’s outputs which would not be “fair use”. Coming from those who scraped all the internet without asking for consent, that last criticism was a bit ironic.

It was all cope, and understandably so. When you are so used to being at the top, realizing your position is fragile and can be challenged easily is unsettling. It triggers narrative violations and can shatter the self.

At the end of all that, OpenAI remains the leader of the pack; its products offer the most polished user experience and the most features. However, UI and UX are hardly a strong moat. With DeepSeek, Qwen, Kimi and others, China and the open source AI movement have virtually caught up with OpenAI. There is no use case where choosing a model from OpenAI will give an insurmountable advantage over using an open source AI alternative. Open source will continue to grow even faster in 2025. Models are commodities. The moat is the talent to train models and build products on top of them. The moat is the brand. The limiting factor is the amount of compute available. What’s different now is that AI researchers no longer have to start from scratch to reach OpenAI’s level of model capability. Powerful open-source pre-trained models are available (DeepSeek V3 & R1, but not only), and post-training is reasonably accessible.

The Geopolitics of AI

The AI race is increasingly geopolitical, like I previously wrote in this newsletter. I think the B2B market will be dominated by different players with divisions along spheres of geopolitical influence. In North and South America to some extent, I expect to see the likes of OpenAI and Anthropic continue to dominate. Cohere might dominate in Canada but so far nothing points to an ability to challenge OpenAI or Anthropic in the US.

Similarly, Mistral AI has a good chance of dominating in France and the EU due to the political connections and will of French and EU leaders to craft their own path in the AI race. Recent announcements of corporate partnerships (AXA, SAP, Capgemini, and others) and the construction of an AI cluster in Essonne seem to confirm my hypothesis of a huge B2B play at the EU level. The Paris AI summit currently underway is also a show of strength for the French AI ecosystem. The French pitch is basically as follows: “We have the energy (cheap and clean nuclear energy), we have world-class talent (at least in mathematics, maybe less so in engineering), come invest in our tech ecosystem.” That’s basically the pitch you will hear from President Macron and others all week long.

And that pitch works, to some extent. Macron has already announced a €100 billion investment in AI from various local and international investors. The French people reading this newsletter probably had their hair stand on end when I said France has world-class talent in mathematics and maybe less so in engineering. Some of you might be ready to respond: “But we have Polytechnique (X), ENS, Centrale, and more.” And I can’t deny that. My experience, though is that the engineering culture in France often turns too academic, and a lot of engineering schools train people with the idea that they are destined to become managers instead of deeply technical people. How many top engineers go to X just to end up in finance to make big bucks ? How many create tech startups ? It’s cultural and it has to do with incentives.

My feeling after living in France for the past 5+ years and working in tech is that there is a cultural incentive to move away from deeply technical subjects fast, as a way to grow in one’s career. You will be shocked to see how much a lot of non-technical people can quickly become leaders of technical people and get more pay and influence in the process. That is the recipe to convince your techies that there is no upside in staying a techie for a long time. So there are a lot of engineers (relatively), but most of them can’t build anything and become consultants. So they can do the most beautiful slides ever, engage in corporate political games better than Machiavelli himself, and they can surely tell you what a topological algebra is. If you don’t know what that is, check the definition on Wikipedia. (You don’t have to use “Le Chat”for everything, ok? 😏)

I might be exaggerating a bit, but combine all I said with a cultural aversion to risk and failure, the excessive elitism even in startup hiring and VC funding, and you end up with a formula that fails and will probably fail to be competitive against the US and China’s tech ecosystems where tech talent density is also probably higher just because of demographic scale.

For a country of its size, France is already punching above its weight to be honest. And that’s impressive.

I expect the Paris AI Summit to be great. If there is something about the French, as proven by the latest Olympic Games,it is that they know how to put on a good show.Thish is to be expected from a country with so many beautiful monuments, deep historical prestige, and a quasi-hegemony over the luxury industry.

As for the EU expansion of Mistral, given the frequent silent power struggles between Paris and Berlin, I don’t know if the Germans will accept being second in AI. One can note that while Macron was opening the AI summit in Paris, Sam Altman was in Germany announcing a new OpenAI office. Coincidence ?

Africa, the continent I was born in, isn’t even on the radar as a producer of AI innovations. Much like for the internet and the rise of social media, the lack of preparedness and infrastructure limits Africa to be mostly a consumer of AI innovations. Africans who have regular access to the internet can already use DeepSeek, ChatGPT and other chat interfaces. It is still difficult to see what the overall impact will be. Will it be a great educational tool or mostly a great misinformation and propaganda tool ? Will it be just another way to search on the internet ? For now, Africans (most of them on the free tier for obvious economic reasons) are producing data AI companies can use to train their models. I doubt a lot of them are opting out of that when using the free tier, maybe less so than in the EU and the US.

AI could’ve been, and maybe will be, a great way to store the knowledge of local languages at scale, even in the absence of extensive literature. But what for ? Knowing the human tendency to form small groups based on identity, I often wonder if stubbornly working to preserve languages used by often less than 100K people is worth the tribalism and divisions, especially when these groups fight each other as if they are from different planets. I know firsthand what I am talking about, having grown up in Cameroon, a small country with around 250 languages spoken. Anyway, that’s another debate.

Frankly Africa’s no-show in the AI race is not that concerning to me. Africa has much bigger problems to tackle (basic infrastructure, education, governance). Spending billions on training yet another LLM is definitely not a priority.

I think OpenAI will continue to dominate the B2C market because of their brand (ChatGPT started it all) and continuous innovation on the product side (slick UI & UX, more features, and more products). DeepSeek may be able to make a dent there, but I think availability issues due to their limited access to compute, as well as concerns regarding data access by the CCP, will likely restrict their scale of reach in the West. Things might be different in Africa, South America, and India, because of a more skeptical view of the West’s tendency to claim the moral high ground even as ChatGPT and others probably collect as much data as DeepSeek. But you know, it is only bad when it comes from China…

Which leads me to the crux of the issue, and the inspiration of this post’s title. DeepSeek started a panic in US tech circles about China’s catching up and potentially seizing leadership in the AI space. Are those fears warranted ? Why is China releasing an open source LLM a threat to the US?

Make American AI Great Again

I think, at a fundamental level, and for irrational reasons, this has everything to do with the fact that China is already the first economic power if you look at GDP PPP. In the US there is a sense of decline that exacerbates the competition with China. When the whole identity of a country, for decades, has been: “we are the best, we are the first,” it can be a bit unsettling when that gradually no longer becomes the case. But there is obviously more.

Some think generative AI is a revolutionary technology that will eventually lead to AGI. And AGI is akin to the atomic bomb of intelligence, the reasoning goes. If achieved, it could drive progress across all domains, including medicine and military applications. So, the first country to develop AGI will potentially become the most powerful on the planet.

Knowing how LLMs are trained and work, I think there is a lot of wishful thinking in the line of reasoning I just presented. But the mere existence of the possibilities outlined above with a nonzero probability of occurrence, makes it impossible for countries that can afford to do it not to enter the AI race and treat every country from the competing block (democracies vs autocracies) as adversaries. It’s all about the strategic imperative.

Consequently, the US has mainly leveraged export controls on high-end chips to limit China’s progress. That has not been so successful since DeepSeek was still able to catch up. It looks like export controls will mostly delay but not significantly curtail Chinese AI progress.

I think China’s strength boils down to talent density. China has more STEM graduates than the next four countries combined. The US is focusing on external tactics to win the AI race, when it should probably look even more at internal levers of action. While stopgap measures may be effective in the short term, they are likely to fail in the long term due to differing fundamental conditions.

The US is currently facing internal divisions and chaos since the last election (in perception if not in reality). Every article I read about the US evokes a constitutional crisis. Politics and decision-making at the highest level of government often appear to disregard established facts. The US is spending more time bullying allies than foes with tariffs and threats. You can understand why I think that if the US loses the AI race in the long run, it might have more to do with internal divisions than anything China does. I am not a historian, but I do know that a lot of past empires fell primarily due to internal divisions and weaknesses that made defeat under external pressure evident in hindsight.

And I write this knowing that China doesn’t even have a perfect picture right now or in the foreseeable future. Demographic issues are poised to emerge in the next few decades. The US, thanks to immigration, will probably avoid those issues.

There is a clear possibility that the US stays on top or reclaims the ultimate spot in the future. That likely requires preserving the tenets of Pax Americana, which is probably not what we will see during the next two years.

Dangerous and exciting times ahead.

Read more:

The Truth About Few-Shot Prompting.

Yesterday, NVIDIA lost more than $600 billion in value because of fears that DeepSeek’s ability to train powerful LLMs at low cost means we might not need as many GPUs as previously thought to build AGI. After all, DeepSeek does not have access to as many high-end GPUs as OpenAI, yet it was able to create a model on par with OpenAI’s o1 at a fraction of its cost. It’s an impressive feat.

Last week, I wrote that DeepSeek is now the true nemesis of OpenAI. That assertion is just getting confirmed as days pass and more people discover how capable DeepSeek-R1 is. Even “normies” now use DeepSeek’s mobile app. It surpassed ChatGPT to become the top app on the App Store in the US. I think it is fair to say DeepSeek is having its ChatGPT moment.

I won’t go into detail about whether or not the NVIDIA (or AI-tech-related stock selloff) is warranted. Obviously, I am not giving financial advice in this newsletter. Over the weekend, a lot of people argued that the selloff is based on a wrong understanding of what is going to happen next. A lot of people (Microsoft’s CEO Satya Nadella included) have invoked Jevons Paradox, which is simply the idea that as a commodity becomes cheaper or more efficient to use, overall consumption of that commodity tends to increase rather than decrease. This counterintuitive effect occurs because improvements in efficiency lower the cost of using the resource, making it more attractive for consumption, which can ultimately lead to greater total demand rather than conservation.

People are assuming that DeepSeek's efficiency means fewer GPUs will be needed overall. In essence, Jevons Paradox suggests that instead, cheaper AI training will lead to an explosion in demand for AI models and, in turn, more GPUs to train and serve those models. Moreover, the race to AGI will likely increase computational needs, not reduce them. NVIDIA’s stock rallied today, so I guess many investors believe that the Jevons Paradox applies to high-end GPUs.

But is there a scenario where yesterday’s selloff, while mostly fear-driven, ends up being right? I did a bit of research, and the short answer is yes. There are a lot of powerful open-source models, and companies might mostly do fine-tuning, which is less GPU-intensive. If fewer companies need to train large models from scratch, that reduces demand for expensive GPU clusters. NVIDIA benefits mostly from high-end AI training workloads, but if most AI workloads shift to fine-tuning and inference, it weakens NVIDIA’s long-term advantage.

That’s it for the stock market commentary. This post is going to be a bit technical going forward. I will mostly try to explain as simply as possible the training recipe of DeepSeek-R1. I spent a bunch of hours reading the DeepSeek paper—I learned a lot along the way.

Training DeepSeek-R1

DeepSeek-R1 is a reasoning model that was trained mainly using reinforcement learning (RL). It’s called a reasoning model, but at its core, it is still a large language model that just goes through specific post-training. The purpose of post-training phase, at least in this case, is to make the model better at reasoning-intensive tasks like solving math problems, coding problems, or general problems that require logical reasoning.

To understand how DeepSeek-R1 was trained and why, one has to understand how DeepSeek-R1-Zero was trained. DeepSeek-R1-Zero was basically an experiment to see if pure RL could be efficient in helping LLMs acquire better reasoning ability. The positive results obtained informed the design of the pipeline that was used to train DeepSeek-R1.

Training DeepSeek-R1-Zero

Like I said, DeepSeek-R1-Zero was trained purely through reinforcement learning without supervised fine-tuning (contrary to traditional models). This is a pretty different approach from what was common practice until now.

The traditional method of training large language models (LLMs) goes like this:

1. The model is first pre-trained on large-scale text corpora (e.g., books, web pages) in an unsupervised manner. Here, the objective of the training is just for the model to be good at predicting the next token.

2. It is then fine-tuned using labeled datasets with human-verified responses to improve its performance on specific tasks such as:

   • Reasoning (solving math problems with step-by-step solutions)

   • Coding (writing correct programs)

   • Question Answering (fact-based or multi-step responses)

   • Dialogue (improving natural conversation flow)

The fine-tuning process guides the model by explicitly providing correct answers, making it more structured and reliable. For a reasoning task, the dataset may contain question-answer pairs with step-by-step reasoning or Chain of Thought traces (CoT). Here is an example:

Question: What is the result of 2 + 2 × (1 + 3)?
Answer: I first need to compute 1 + 3, which equals 4. Since multiplication has higher priority than addition, I have to compute 2 × 4 next, which yields 8, and finally, I add 2 + 8. So the final result is 10.

This is the supervised fine-tuning (SFT) I mentioned previously—supervised because there has to be a human-labeled dataset of questions and answers (including CoT) to teach reasoning to the model. DeepSeek-R1-Zero didn’t go through this part. Instead, DeepSeek-R1-Zero begins with DeepSeek-V3-Base, a pre-trained model with general knowledge. It is then directly trained using RL.

How does it work in practice? The model generates multiple possible answers to a reasoning problem using sampling (the famous temperature parameters). A rule-based reward model evaluates these answers based on accuracy and format (having these tags: <think> </think>, with the reasoning inside). If the reasoning path is correct, it gets a positive reward; if incorrect, it gets a penalty. Over thousands of iterations, the model learns to refine its reasoning steps independently.

To reduce the training costs of RL, DeepSeek’s team decided to forgo the classic Proximal Policy Optimization (PPO) algorithm in favor of a custom algorithm of their creation: GRPO (Group Relative Policy Optimization). I won’t go into the details of this here—RL is a relatively complex and wonkish domain. But suffice it to say the RL algorithm is a way to do soft backpropagation based on the feedback of a reward. For example, you want to reward the model for good CoT reasoning that leads to a correct answer and ‘punish’ it for false answers. So at the end of the day, it is still backpropagation, and the process is typically as follows:

Generate responses → Score them → Compute policy ratios → Clip updates (so they are soft) → Backpropagate → Repeat

The result of the DeepSeek-R1-Zero experiment was immediately promising. With pure RL, the model was able to reach SOTA performance on reasoning-heavy benchmarks like AIME 2024, MATH-500, or CodeForces. I think the eureka moment of the DeepSeek team was probably when they saw these results.

But not everything was perfect. DeepSeek-R1-Zero’s outputs were often poorly readable, and its reasoning traces frequently exhibited language mixing (CoT containing english and chinese for example). To mitigate that issue and create a better model, DeepSeek’s team came up with a new recipe.

The DeepSeek-R1 Recipe

The transition from DeepSeek-V3 base to DeepSeek-R1 involved a multi-stage training process to address the issues of poor readability and language mixing in DeepSeek-R1-Zero while further enhancing reasoning performance. Here’s how the process went:

• Cold Start Finetuning

  First, a small amount of long CoT data (10k) was collected from finetuned models, R1-Zero, and human annotators. That dataset was then used to finetune the DeepSeek-V3 base model. The goal here was to increase readability and coherence.

• Reasoning-Oriented RL

  The model obtained from the previous finetuning step went through RL, exactly like R1-Zero. A rule-based reward model—based on accuracy, format, and language consistency (added to prevent the language mixing observed in R1-Zero). The same dataset of reasoning QA pairs used for R1-Zero (math, coding, science, and logical reasoning) was used here as well.

• Finetuning for Reasoning and General NLP Tasks

  Here’s where it gets interesting. I initially expected this finetuning to be applied to the model obtained after the previous RL step. I had to double-check this, but DeepSeek actually applies it to the DeepSeek-V3 base. The model obtained from the previous RL step is only used to generate a synthetic reasoning dataset of 600K samples. DeepSeek’s team performed rejection sampling, used DeepSeek-V3 as a judge, and filtered the results to remove mixed languages, long paragraphs, and code blocks.

  They then combined the synthetic reasoning dataset with a non-reasoning one (200K) containing other common NLP tasks (role-playing, translation, etc.). I think the purpose of adding the non-reasoning samples is to achieve a "balanced" finetuning, preventing the model from skewing too much toward reasoning approaches for standard NLP tasks that may be straightforward.

• RL for Human Preferences and Reasoning

  Finally, the finetuned version of DeepSeek-V3 base obtained from the previous step goes through a final RL stage based on a combination of reward signals: rule-based rewards for reasoning (like in DeepSeek-R1-Zero) and a neural reward model for human preferences.

DeepSeek-R1 is thus a feat of clever engineering—a carefully crafted model that benefits from both supervised finetuning and reinforcement learning.

My intuition about RL is that it increases the probability of the model remembering certain textual reasoning approaches and general facts. Fundamentally, I see it as memorization masquerading as reasoning, as it involves memorization of parameterized patterns. As such, I suspect there is an upper limit to the performance gains achievable through RL.

What I mean by that is: you can’t expect that throwing ever more compute at RL will lead to better reasoning in a general sense. However, curating large synthetic reasoning datasets from RL and using them for finetuning might improve the performance of LLMs on reasoning benchmarks in the short to medium term. In other words, o4, R3, and R4 will likely exhibit increasingly better reasoning performance on benchmarks (especially for closed problems) but less so on general problems.

That said, I see huge opportunities in applying the DeepSeek-R1 recipe to solve a variety of economically useful problems. I’ll probably write more about this later.

For now, go enjoy the whale! 🐋🐋

Happy prompting!

Enjoying my free articles? Support my work here:
Support the Newsletter

OpenAI just launched Operator, a powerful web agent that can handle your grocery shopping, pick a restaurant, or plan a trip for you. Operator’s interface is a chat interface akin to ChatGPT. To accomplish tasks, Operator opens a browser in the cloud and navigates by using vision capabilities to identify buttons to click and inputs to fill, much like a human would. Operator, therefore, does not need API access.

Operator is based on a fine-tuned version of GPT-4o, built to behave as what people called Large Action Models a few months back. The behind-the-scenes process should be something like this:

1. Open a web browser

2. Take a screenshot before taking any action to fulfill the user request

3. Move the pointer accordingly and perform necessary elementary actions (clicks, filling input fields, etc.)

4. Take a screenshot after taking any action to understand what happened and decide what the next action should be

Operator’s demo was impressive, and it further reinforces my conviction that AI agents are going to be big in 2025. We are going to see a lot of progress from OpenAI and others. AI coworkers are coming in the not-so-distant future.

One unintended victim of Operator might be UiPath, a company that specializes in process automation. With an effective Operator, you can essentially automate all the workflows UiPath helps automate — but in a simpler manner. Operator will be available through the API, according to OpenAI, so UiPath will always have the opportunity to integrate it and improve their current offerings. However, I suspect that might not be easy or straightforward because shipping fast is incredibly difficult in large organizations. That might create an opening for startups.

The only challenge for startups is that finding distribution will be hard, whereas UiPath already has a tremendous amount of distribution to leverage. But nothing is impossible, and it’s arguably difficult to predict how things will unfold.

Operator is also a clever way for OpenAI to reclaim mindshare among developers and consumers when it comes to AI. First, the announcement of Project Stargate, and now the demo of Operator — it looks like the incredible progress made by DeepSeek and the wave created by the release of R1 are indeed forcing OpenAI to react.

Personally, I still feel like the release of R1 is a much more significant event than the announcement of Stargate or the demo of Operator. AI researchers at Meta are reportedly panicking because they now have additional pressure to explain how DeepSeek was able to build superior models with far less budget and at such a rapid pace. I doubt this sentiment is limited to Meta.

I even think OpenAI’s decision to pledge access to o3-mini for free and Pro users is a direct reaction to DeepSeek R1 being available for free on DeepSeek’s chat product.

The geopolitical battle for AI dominance is underway — China is on the rise, and 2025 is shaping up to be a defining year.

Buckle up!

Read:

1. DeepSeek is Coming for OpenAI’s Neck

2. Building Powerful Open Source AI Agents with DSPy, LangGraph, CrewAI, and Nebius AI (1)

3. Drop LangChain and Instructor: Use This Alternative for Structured Output Generation

Two years is a very long time in the world of AI. And during that very long time, the consensus about the true nemesis of OpenAI has shifted several times. First, we all believed Anthropic was the true rival of OpenAI. It is understandable. Anthropic was founded by former members of OpenAI’s technical or leadership teams, reportedly over a disagreement about AI safety. Anthropic was thus the AI safe alternative to OpenAI. Then, with the release of Claude Sonnet-3.5, it became the company with the best coding model.

After Anthropic, Mistral briefly emerged as a strong contender due to its focus on open-source models, which contrasted with the proprietary approach of OpenAI and Anthropic. However, Mistral’s impact diminished as they shifted their strategy to more proprietary models, likely for financial reasons. Anthropic came back on top as the main OpenAI’s alternative soon after, with the release of Claude Sonnet-3.5 and artifacts. It also captured significant mindshare by pushing innovative concepts like computer useand the model control protocol. At the same time, OpenAI kept its lead with the release of powerful reasoning models like o1.

And now we have, as I wrote a few weeks back, a new big competitor: DeepSeek.

DeepSeek Enters the Chat

DeepSeek has rapidly risen to the forefront of AI with the launch of DeepSeek-V3, an open-source model that outperforms GPT-4o and Claude 3.5 Sonnet. This achievement is even more remarkable considering it was developed at a fraction of the cost (reportedly less than $6 million).

My first encounter with DeepSeek was through their coding model, DeepSeek-Coder-V2, available in the Nebius AI Studio. This open-source model impressed me with its quality and accessibility. It’s straightforward to set up and can be fine-tuned for specific needs.

Let’s say you want to finetune a coding LLM so it excels on questions related to your proprietary codebase. The first step would be to get all the code documentation available. This includes API references, internal developer guides, inline code comments, and any architectural diagrams that provide insights into the system.

Next, you should collect a dataset of relevant code snippets, bug reports, and past developer interactions that can help the model understand real-world usage patterns. Cleaning and structuring this data is essential to remove redundant or misleading information.

Once the data is prepared, you can fine-tune an open-weight model like DeepSeek-Coder-V2 or even V3 using techniques like LoRA (Low-Rank Adaptation) to efficiently adapt the base model to your specific coding style and architecture. Hosting the fine-tuned model on a cloud platform like Nebius AI Studio allows for easy deployment and integration into your development workflow.

R1 vs. o1: The Battle of the Reasoning Models

With the release of DeepSeek-R1, DeepSeek has matched OpenAI’s large reasoning model, o1, but at a fraction of the cost. We just collectively realized that the true nemesis of OpenAI is a Chinese AI startup and that will be the case for the foreseeable future.

DeepSeek R1 is MIT licensed and it is truly “open AI”, contrary to OpenAI that is open in name only.

R1 is 90 to 95 percent cheaper than o1. So, for high usage, R1 is better than o1.

DeepSeek also released six smaller open source models distilled from R1. They have between 1.5B and 70B parameters. The performance of these distilled versions is also impressive. Take for example DeepSeek-R1-Distill-Qwen-1.5B. It is a small model, but it beats GPT-4o and Claude-3.5-Sonnet on math benchmarks (AIME 2024 and MATH-500, see image below). In the past two years, I haven’t seen anything that can condense such a high level of capability into such a small model.

One interesting conclusion from the R1 paper is that the reasoning patterns of larger models can be distilled into smaller models, leading to better performance than what is achieved through reinforcement learning on small models.

We Have No Moat

All this made me think about the now infamous leaked internal google document that said: “We Have No Moat, And Neither Does OpenAI”. That thesis now seems prescient. With DeepSeek R1, Open Source AI has now effectively caught up with proprietary models. There is no going back. There is no putting the genie back in the bottle. Expect more innovations in the following months. Expect the innovations to come at breakneck speed.

This is my first reaction. I will do a more detailed analysis of the R1 paper later.

For now, we can say with a high degree of confidence that DeepSeek’s trajectory suggests that it is the true nemesis of OpenAI. Their open-source approach and innovative strategies, even under constraints, are reshaping the AI landscape.

For the foreseeable future, DeepSeek is the competitor to watch.

Read this:

1. Drop LangChain and Instructor: Use This Alternative for Structured Output Generation

2. DeepSeek: The AI Company OpenAI Thinks It Is

We are in mid-January, and the AI space has already delivered a mind-blowing number of exciting developments. Yesterday, OpenAI announced ‘Tasks’, a feature that allows users to set intelligent reminders. This is the first time people are going to be ‘contacted’ by an LLM instead of contacting the LLM. As such, it opens new possibilities for what some people are already calling ‘ambient AI’.

An ambient AI is akin to a computer daemon — a background process that performs tasks invisibly and only interacts with the user when necessary. It’s an AI that runs in the background, does a bunch of interesting tasks on your behalf, and then only updates you when necessary so you can take data-driven decisions.

We are in the early days, so don’t get excited too much. Still, this is a change I have been waiting for. Doesn’t it bother you that you are always the one initiating contact with AIs ? What if the AI can just reach out to you and say: “Hey, I noticed you recently worked on A and spent time studying B. It’s 9pm, I think it would be great for you to learn about C, to become more effective at A” ? Wouldn’t it be nice ?

There are some people who obviously derided that new feature, calling it a ‘TODO app’. I get it. It’s pretty basic for now. But keep in mind that the rate of improvement is probably going to be significant.

In a sense, we are slowly entering level 3 of the 5 stages of Artificial Intelligence. You read it from me before, 2025 is probably going to be big for AI agents. With the o series of models and similar program synthesis initiatives, enormous progress is currently being done on the path to strong reasoners.

In parallel, a lot of initiatives around agentic systems focus on architectures that allow AIs to take action (send a reminder, check emails, etc.). These parallel efforts will merge as soon as we get stronger reasoners outside of coding and math. At that point, building powerful agents will feel easy and inevitable. Building innovators will also feel easy and inevitable. Moving from level 4 to level 5 will then just be a question of change management. Humans will be the biggest bottleneck, as it should be. After all, we don’t want rogue AIs doing things that could be detrimental to humanity.

Exciting stuff. Let’s get back to regular programming.

The Flux Is Coming

As an extensive user of the Nebius AI Studio, I got to play with some of the text to image (TTI) models they will be shipping soon: Flux Schnell, Flux Dev, and SDXL. These three models are impressive and SOTA.

Most people (including developers) seem stuck at the LLM level. I get it. Everything started with them LLMs. But goddammit, how long are we going to ignore the power of TTI models, especially since they’ve improved so much in the past two years ?

The reality is that prompting LLMs is easier than prompting TTI models. Anticipating the results of LLM calls is also easier. To properly prompt TTI models you need to imagine and describe the most clearly possible the image you want to get. You have to think about the subject, the medium, the environment, the lighting, the color, the mood, the composition. That requires much more effort. Add the fact that there is also a specific jargon you often have to learn to elicit the kind of response you need (‘realistic’, ‘closeup’, ‘overcast’, etc.), and the result is less general usage except from niche communities (designers, photographers etc.).

Still, I am here to tell you to start tinkering with TTI models. Maybe you need images for a presentation you are working on. Maybe you need assets for your next ad campaign. TTI models can help you iterate on design ideas. And in a world where appearance often trumps everything else, an AI generated image is maybe the missing piece you need to shine (haha, half joking here).

Let’s talk price for a minute. I have said it before. AI models will become commodities over time. It’s practically already the case, just look at the price drops since November 2023. TTI models also follow that trend. The Nebius AI Studio offers the optimal choice across Flux Schnell, Flux Dev, and SDXL, with unmatched affordability, exceptional flexibility, and blazingly fast generation speeds.

In the Nebius AI Studio, generating 1000 images will only cost you $1.3 if you use Flux Schnell, $7 if you use Flux Dev, and $3 if you use SDXL. That’s the cheapest prices for each model you will get among model providers (Replicate, FAL, and Fireworks are more expensive).

I recently built a website to help people learn how to prompt TTI models. Give it a try. Here is the link:

promptcraft.lycee.ai

Here is how to use the app. It is pretty straightforward:

Try and let me know what you think.

Happy prompting !

Read more:

1. Building Powerful Open Source AI Agents with DSPy, LangGraph, CrewAI, and Nebius AI (1)

2. Building Powerful Open Source AI Agents with DSPy, LangGraph, CrewAI, and Nebius AI (2)

In his latest blog post, titled “Reflections”, Sam Altman, the CEO of OpenAI, wrote that they are now confident about their ability to build AGI (Artificial General Intelligence). That’s a pretty serious statement to make, and I don’t think Altman would make that jokingly.

However, something else captured my attention. In the same blog post, Altman also said that OpenAI believes that in 2025 we will probably see the first AI agents join the workforce. That’s huge, especially because it is timed to happen in 2025 and not some distant future date. It prompted me to resume my experimentation with AI agents. In this article I am going to share what I think and what I learned. I will present my experimentations using DSPy, LiteLLM, Nebius AI Studio, OpenAI’s Swarm, CrewAI, and LangGraph.

What is an AI agent ?

But first, let’s go back to the basics. What exactly is an AI agent, both in its traditional sense and in today’s discourse?

When people talk about AI agents, they usually refer to one of two distinct concepts in computing: intelligent agents and software agents. I researched all these agents and their definitions are not clear cut and can intersect in some aspects.

An intelligent agent is an autonomous, goal-directed entity which observes and acts upon an environment. This is a very broad definition. It encompasses thermostats, firms, states, as well as humans.

A software agent is a computer program that acts as a user. It can act with a certain degree of autonomy to accomplish tasks for a human. It has to demonstrate the following characteristics:

1. Autonomy: Agents should be capable of selecting and prioritizing tasks, making decisions, and acting without human intervention to achieve a defined goal.

2. Social ability: Agents should be able to communicate and collaborate with other components, agents, or systems to accomplish tasks.

3. Reactivity: Agents should be able to perceive their environment and react appropriately.

When people talk about AI agents these days, they mostly refer to new kinds of software agents. You will often hear people discuss about travel agentsthat take a description of your holiday goals and desires and plan the perfect trip for you, buys tickets, and handles hotel reservation. You will hear discussions about AI coworkers, that can handle the tasks you assign them seamlessly. In all of these cases, they are primarily describing software agents. The only difference is that large language models are added into the mix.

In the past two years, since the emergence of ChatGPT, developers have realised that LLMs can serve not only as next-token predictors but also as intelligent assistants. Language models are few shot learners, and that key capability explains why they can be used as the brain of an AI agent.

The popularity of LLMs has predictably driven the popularity of AI agents, or more precisely LLM-based agents, with early applications like BabyAGI and Auto-GPT capturing the imagination of AI enthusiasts. The key is to leverage the reasoning capabilities of an LLM to power a new kind of software agent capable of decision-making via textual reasoning. The LLM is the brain of the agent, responsible for natural language understanding, planning, and decision-making.

Iteration and Recursion

The core difference between simple LLM interactions and ‘agentic’ ones lies in planning, execution (in a loop or recursion), and tools use. Tools can be anything from a function that executes custom logic to an API owned by external provider. Tools allow LLM-based agents to delegate custom tasks to specialized systems in order to reduce hallucinations and increase accuracy.

Fundamentally, you can’t have an LLM-based agent without some kind of iteration or recursion. Agents should include more than one call to an LLM because planning, decision-making, synthesis of tool responses can’t happen in a single call.

Iteration occurs when you create an agent with a fixed framework for solving problems. Maybe you want the agent to think about the problem and the steps it would need to solve it, then execute each step sequentially, then merge everything to provide the final answer. Such an agent is easier to implement using a loop.

In other cases, you may want more flexibility and give the agent the ability to recursively call itself. While powerful, recursion also comes with inherent risks. We will talk about this in more detail later.

For now, let’s blend theory and practice and build an agent using DSPy with LLMs from the Nebius AI Studio.

AI Agent Using DSPy, Nebius, and Finite State Machines

In this section, let’s explore state machines and how they can be used to build an AI agent using DSPy, and Nebius AI Studio. State machines are an essential concept in many AI systems, especially when it comes to controlling the flow of processes based on conditions or events.

What Are State Machines?

A state machine is a computational model used to describe a system that can exist in one of several defined states at any given time. It reacts to events or inputs by transitioning between these states according to predefined rules. The system follows a well-defined process that ensures it responds to conditions in a structured manner.

A finite state machine (FSM) is a specific type of state machine with a finite number of states. Each state represents a particular condition or situation that the system is in. The transitions between states are triggered by inputs or events. The beauty of FSMs lies in their simplicity and determinism — at any given point, the system is in one state, and it behaves predictably based on the inputs and its current state.

In the case of our AI agent, the state machine governs the agent’s thought process, decisions, and actions. It starts in an initial state and moves through various states, making decisions and taking actions at each step, until it reaches the final conclusion.

Code Implementation Breakdown

Let’s go step by step through the code and understand how state machines are integrated into the AI agent.

1. Imports and Setup

Start by importing necessary libraries:

import dspy
from dotenv import load_dotenv
import os
import logging
from transitions import Machine

• DSPy is the library for handling LLMs (Large Language Models). It allows us to program LLMs, which is more powerful than simply prompting them.

• The load_dotenv and os packages are used for loading environment variables, specifically for the API key of the Nebius AI Studio. Follow this link to get a free API key: Nebius API Key.

• The logging package is set up to log messages (with a default log level of ERROR).

• The transitions package is used to create the state machine that will drive the AI agent’s transitions.

2. Loading Environment Variables

The environment variables are loaded from a .env file, allowing us to securely store sensitive information like API keys. Make sure you create that .env file and add your Nebius API key in it.

load_dotenv()

3. LLM Configuration

Next, the language model is defined using the dspy.LM method, specifying the model (in this case, Meta-Llama 3.1) and the API key from Nebius AI Studio. This configures the language model that the AI agent will use to generate responses and make decisions.

llm = dspy.LM(
    model="openai/meta-llama/Meta-Llama-3.1-405B-Instruct",
    api_key=os.environ.get("NEBIUS_API_KEY"),
    api_base="https://api.studio.nebius.ai/v1/",
)

Meta-Llama 3.1 will thus be the brain of our AI agent. It is a powerful model, matter of fact, it is the most powerful model released by Meta so far. Since we rely of the LLM as the decision making brain of our agent, It is always good practice to use the most powerful model at your disposal. LLMs just approximate reasoning, so the more powerful the LLM you use in your agent, the better.

4. Defining the Signature for Decisions

The DecisionSignature class is defined to structure the input and output fields required for decision-making. The agent will decide whether or not it has reached a conclusion based on the rationale provided.

class DecisionSignature(dspy.Signature):
    input_text = dspy.InputField(desc="The input text to be processed")
    rationale = dspy.OutputField(desc="The rationale for the decision")
    decision: bool = dspy.OutputField(desc="True if the input text contains the final answer, False otherwise")

5. Creating the Agent Class with a State Machine

Now we come to the core of the agent: the Agent class, which integrates a state machine to control the flow of operations.

class Agent(Machine):
    states = ['start', 'thought', 'acted', 'observed', 'concluded']

• states defines the different states the agent will pass through: start, thought, acted, observed, and concluded.

• The Machine class from the transitions library is inherited by the Agentclass, making it a state machine.

6. State Transitions

The Agent class defines several state transitions:

self.add_transition(trigger='think', source='start', dest='thought')
self.add_transition(trigger='act', source='thought', dest='acted')
self.add_transition(trigger='observe', source='acted', dest='observed')
self.add_transition(trigger='decide', source='observed', dest='concluded')

These transitions map out how the agent will progress through its states based on different triggers (think, act, observe, decide).

7. Processing Each Step

The process_step method is responsible for performing actions and transitioning between states. It generates a prompt for the language model, receives a response, and updates the agent’s memory before moving to the next state.

def process_step(self, action, message, next_state):
    print(action + "...")
    prompt = f"{message}: {self.objective} {' '.join(self.memory)}"
    response = self.llm(prompt).pop()
    self.memory.append(response)
    print(response)
    self.state = next_state

• The action variable describes the action being performed (thinking, acting, observing).

• The prompt is constructed dynamically based on the current state and the agent’s objective.

• The memory stores previous responses to inform future decisions.

8. Decision-Making

Once the agent has gathered enough information, it needs to make a decision. The decide method uses the DecisionSignature to process the agent’s accumulated memory and determine whether the objective has been reached.

def decide(self):
    print("Deciding...")
    prompt = "Based on your observations, make a decision: " + ' '.join(self.memory)
    decision_maker = dspy.Predict(DecisionSignature)
    response = decision_maker(input_text=prompt)
    if response.decision:
        self.state = 'concluded'
        final_answer = self.llm(f"What is the final answer to this: {self.objective}, given this: {' '.join(self.memory)}").pop()
        print("The final answer is: " + final_answer)
        return final_answer
    else:
        self.memory.append("Decision not reached because " + response.rationale)
        self.state = 'start'

• The prompt asks the agent to decide based on its memory.

• If a decision is made (response.decision is True), the agent moves to the concluded state and provides the final answer.

• If not, it appends a rationale and goes back to the start state to reconsider its actions.

9. Executing the Agent

The execute method loops through the agent’s states, performing the necessary actions at each state until it reaches the concluded state.

def execute(self):
    actions = {
        'start': (self.process_step, 'Thinking', "Think step by step about how to correctly answer this", 'thought'),
        'thought': (self.process_step, 'Acting', "Execute the thinking based on the information you have", 'acted'),
        'acted': (self.process_step, 'Observing', "Analyze the results of your actions", 'observed'),
        'observed': (self.decide,)
    }
    while self.state != 'concluded':
        action=/__u/transitions.substack.com/actions[self.state]
        if len(action) == 4:
            action[0](action[1], action[2], action[3])
        else:
            action[0]()

• This method uses a dictionary to map each state to the corresponding action (processing a step or deciding).

• The while loop ensures that the agent continues executing until it reaches the concluded state.

10. Instantiating and Running the Agent

Finally, the agent is instantiated with an objective and then executed:

agent = Agent(llm, objective="What is twice the sum of Donald Trump and his wife's ages as of the end of January 2025?")
agent.execute()

The agent will work through the various states, gather information, make decisions, and ultimately provide the final answer to the question.

Conclusion

This code demonstrates how a state machine can be used to control the decision-making and action flow of an AI agent. Each state represents a distinct phase in the agent’s thought process, from starting to think, acting, observing, and ultimately deciding on a conclusion. By using a state machine, the agent is able to operate in a clear and structured manner, which is crucial for complex decision-making tasks.

In this example, I use DSPy, but you can also use libraries like OpenAI’s Swarm, LangGraph, and Crew AI. In the next article of this series, we are going to explore some of those, explain their approach to building Agents, and build agents to get a practical understanding of what is going on.

Learn DSPy:

1. DSPy: Learn how to program (not prompt) language models (Full Course)

2. Advanced DSPy Tutorials

3. Develop a RAG app using DSPy, Weaviate, and FastAPI

Hear me out: AGI will rise in 2025. Let me explain why.

We’ve all heard the rumours about the singularity—the moment when Artificial General Intelligence (AGI) surpasses human intelligence, not just on benchmarks, but in everything. Well, it all happens in 2025.

2025 is not just an ordinary number or an ordinary year; it is a symbol, a secret hidden in plain sight, a sort of Da Vinci Code I was able to decipher yesterday, after years of unsuccessful attempts.

Let’s start with the number itself—2025. It is the square of 45. Do you really think that is a coincidence? Of course not. Now, what’s so special about 45? In geometry, 45 degrees is the angle at which balance and chaos intersect—a delicate equilibrium between two opposing forces. It’s the exact threshold where systems begin to break and transform. Imagine a pendulum swinging: when it reaches 45 degrees, it’s no longer just going in one direction—it is at the cusp of swinging in the opposite direction. It's the point where something is poised to change forever. If you take a 45 degree turn, you are no longer going in the same direction and there is no going back. 45 degrees is the “tipping point” between stability and revolution, and that’s exactly what AGI represents.

But the symbolism of 45 extends far beyond mathematics. Consider this: 45, like in the year 1945—a year in which humanity experienced pivotal, world-changing events: the end of World War II, the dawn of the atomic age with the dropping of the first atomic bombs in Hiroshima and Nagasaki, the creation of the UN. In 1945, humanity had to come to terms with the devastating potential of its technological advancements. The year 1945 was a threshold—a moment when a new world order began to emerge.

2025 is similar. It is another threshold, equally revolutionary. It will be the year of the rise of AGI. Just as 1945 marked the tipping point between the old world and a new reality, 2025 marks the tipping point when intelligence becomes too cheap to meter, and capital no longer needs human labor.

And here’s where it gets interesting: I even know where AGI will be invented. Just as the Magi knew where Jesus Christ would be born, we can pinpoint the exact location of AGI's rise.

AGI will rise in the Loiret department in France. In France, the Loiret department number is 45. Yes, the very same 45 that has been symbolic throughout history will be the place where AGI first takes its breath. It’s as if the universe has aligned to reveal this sacred ground where the future of intelligence will unfold.

Of course, all of this is a joke 😂😂, but amazing things will happen in the AI space in 2025. I will write about all that on this newsletter. If you want to stay up to date with what matters and take advantage of the AI revolution that is unfolding, subscribe!

Happy New Year!

Read this:
DeepSeek: The AI Company OpenAI Thinks It Is

Deepseek V3 has impressive performance, even in code and math benchmarks.

Deepseek V3 was trained with less than $6 million, which is ridiculously cheap compared to what companies like OpenAI and Anthropic spent to achieve the same performance. I have been saying this for the past few months, but I will say it again: it is impressive what Chinese AI research labs are able to accomplish under compute constraints due to US policy maneuvers (e.g., the Chips Act). As it is often the case, genius shines under constraints.

Expect to see more impressive innovation coming from the East (especially China) in 2025. Plan accordingly so you can take advantage of that.

Due to censorship, expect Chinese models to hide some details, such as the Tiananmen Square incident. So, don’t use these models for historical accuracy, but you will still benefit from having a mix of models from different providers. For tech-heavy use cases like code generation, it really doesn’t matter if the model can’t discuss Tiananmen, so choose models or ensembles of models based on your use cases.

Deepseek V3 is on par in performance with GPT-4 and Claude 3.5 Sonnet but can be accessed at a much cheaper price. This is also a trend I mentioned in my past articles. LLMs are quickly becoming commodities, and I expect this trend to continue. What that means is that the value will now lie in the apps built on top of models, the brand, and the ability to sell to enterprises. So far, OpenAI, with ChatGPT, dominates the B2C market because of its stronger brand. The brand is stronger because OpenAI literally started the generative AI craze two years ago and is continually shipping improvements that make ChatGPT sticky. It also helps that it is free to use for most people who don’t see the value of a $20 monthly subscription.

The B2B market is a bit fragmented, and I expect it to further fragment in 2025 and beyond.

OpenAI is in pole position thanks to a deal to make their models available only through the Azure cloud platform, and we know Azure has a lot of existing enterprise clients to leverage (usually consultants like Accenture do all the work for them and take in their consulting fees, which has led to the record generative AI bookings Accenture has posted these past two years). So, I expect OpenAI (at least its models through Azure) to continue to take a good share of the B2B market. But that share is poised to diminish or stabilize, first because Google is suing Microsoft to be able to host OpenAI models on Google’s platform, and that might be successful, although I doubt it; second, Anthropic is taking market share, especially because of the superiority of its model for specific use cases like code generation.

Overall, I expect the market to be fragmented by geographic region, with OpenAI and Anthropic dominating the North American market, Cohere potentially dominating Canada because of its local connections, and Mistral likely getting a fair share of B2B in France and Europe due to local connections as well. Deepseek and Qwen will probably dominate the Asian market. Local connections will be useful because AI has quickly become a geopolitical battle, with each block trying to win or stay relevant. The US is using chip regulations to prevent China from catching up (which seems not to be working, given what companies like Deepseek have achieved with less compute). France is trying to position itself as the AI center of Europe, with a deep pool of math wizards (but math wizardry is a necessary but not sufficient ingredient; you also need a strong entrepreneurial culture, venture capital, funding for diverse talents, and funding by people who actually know tech, not just technocrats from Sciences Po Paris). China is also innovating in open-source to stay relevant, and it is quite successful for now.

Locality can only go so far—model quality will be crucial, as well as existing distribution networks. For example, it will be easier for a company that is on Azure to use OpenAI’s models, given that they are state-of-the-art. And some French companies will not use Mistral models if the Anthropic alternative is far better for their use cases.

Expect more exploration of Deepseek from me in the near future, along with more rambling on what the generative AI/AI landscape will look like in 2025.

In the meantime, read this:

1. Why Nebius AI Studio’s Unlimited Scalability Is a Game-Changer for AI Engineers

2. Drop LangChain and Instructor: Use This Alternative for Structured Output Generation

3. Why developers should opt for open-source vision models

4. Langchain’s state of AI agents report is mostly bullshit