The room falls silent. Everyone knows he is right.

Academic publishing in 2026 is that room.

The Enigma Machine Is Already Running

Generative AI has dramatically lowered the friction of producing text. Researchers now draft manuscripts in hours that previously took weeks. Submission volumes at major journals have risen sharply. Review queues have grown longer. The global pool of volunteer peer reviewers, already stretched before AI arrived, is now burning out at a rate that editors describe, in private, with language unsuitable for a scholarly journal.

The math has simply stopped working.

Consider what the system asks of its participants. Every researcher benefits from peer review because publication drives hiring, promotion, and grants. But reviewing a paper well takes four to ten hours, counts for almost nothing on a CV, and earns precisely zero on any performance dashboard. So when the review invitation arrives, the rational calculation is clear. Many researchers, especially junior ones under the most pressure to publish, decline. Or they accept and then quietly disappear, which is worse because the editor has now lost two weeks. This is especially prevalent among Indians, because in Indian culture saying “No” is rude. To be honest, I will be very happy if young Indian academicians are rude to me and learn to say “No”.

In the previous post in this series, I described this as a public goods problem: everyone benefits if enough people review, but each individual benefits slightly more by letting someone else do it. If enough people reason this way simultaneously, the system does not adapt. It collapses, slowly and then suddenly, like a journal’s special issue that takes three years to complete.

We built the publication system on professional goodwill and civic duty. Goodwill does not scale. AI does.

You Cannot Beat Machine Speed with Human Hours

The instinct of many academic institutions has been to treat AI as an adversary to be banned. Some journals now require authors to declare AI use. Some require them to certify that no AI assisted their writing. This is the scholarly equivalent of asking Turing’s team to decode Enigma faster by working longer shifts.

It will not work. The machine is already running. The question Turing asked is the question publishing now faces: do we keep decoding by hand, or do we build something that works at machine speed?

The answer, I think, is obvious. Journals, publishing houses, and academic societies need to invest seriously in what I would call Certified AI Reviewers: open-source models whose architecture, training data, evaluation criteria, and version history are publicly documented and independently audited by scientific consortia.

Not a black-box commercial system whose guardrails shift unpredictably behind a corporate API. Not a general-purpose large language model that knows a little about everything and a lot about nothing. A purpose-built, field-specific, small language model trained to do one thing extremely well: review papers in a defined scientific domain.

Why small rather than large? A frontier model tries to know everything superficially. A scientific reviewer needs to know one field extremely well. A carefully trained seven-billion-parameter statistics reviewer can outperform a general-purpose model on narrowly defined review tasks because it is specialised, reproducible, cheaper to run, and possible to audit. Science demands reproducibility. Open-source SLMs allow the research community to inspect training data, fine-tuning decisions, and evaluation rubrics. Authors and readers can understand exactly what benchmarks the system applied. Universities and researchers can host these models locally, on their own servers, ensuring that unpublished preprints never travel to a third-party commercial cloud.

This is not a concession to the machine. This is Turing’s move.

Introducing Reviewer D: Tireless, Systematic, and Deeply Uncomplicated

To avoid unnecessary resistance, one thing needs to be said clearly: AI does not decide publication. AI prepares an evidence report.

Think of it not as an editor but as Reviewer D, the fourth reviewer nobody asked for but everyone needed. Reviewer D does not have opinions about the authors. Reviewer D does not have a deadline conflict. Reviewer D does not dash off a three-sentence review at midnight and call it adequate. Reviewer D is tireless, systematic, and transparent.

What Reviewer D does: it runs and reproduces code where available, checks mathematical derivations and flags inconsistencies, inspects supplementary material, verifies datasets, identifies missing citations, and compares factual claims against the cited literature. Every one of these tasks currently consumes hours of human reviewer time, hours that could instead go toward the work that only humans can do.

The intellectual core of peer review has always been about evaluating originality, assessing scientific taste, and guiding authors toward better versions of their ideas. That is not mechanical work. That is judgment, and it improves with practice. Reviewing papers has always been one of the sharpest forms of deliberate practice available to a working researcher: it exposes you to emerging directions before publication, to methodological mistakes, to the precise standards that separate a good paper from a published one.

Reviewer D preserves all of that by eliminating the drudgery. The human reviewer arrives at a manuscript with the mechanical checks already done, the inconsistencies already flagged, and the literature already compared. What remains is the interesting part.

Where will AI Reviewers fail?

Certified AI Reviewers will sometimes miss genuine novelty. A model trained on existing literature has no template for truly unconventional methods, and will occasionally flag originality as error. Every model inherits biases from its training data, and in peer review those biases will matter: a statistics SLM trained mostly on frequentist literature will read a Bayesian paper differently than a human expert would. These are real limitations, and they argue not against using AI in peer review but against allowing these models to operate without transparent benchmarks, open-source accountability, and rigorous human oversight.

By keeping the models open, the academic community can continuously identify and correct blind spots. By keeping humans in the final decision, the community preserves the judgment that no training data fully captures.

The alternative, leaving the author side of the pipeline to AI while keeping the reviewer side entirely manual, is not a principled position. It is a losing one.

What Turing Would Have Said

Turing did not argue that machines should replace the codebreakers. He argued that the codebreakers needed a machine to do what human hands could not: work at machine scale, without fatigue, without error accumulation, without running out of hours in the day.

Peer review is one of the last great public goods in science. The academic community built it on trust, maintained it through professional obligation, and now watches it strain under a volume that no human workforce can match.

The question is no longer whether AI will participate in peer review. It already does, on the author side, every time a researcher opens a language model to draft a paragraph. The real question is whether the scholarly community will build equally transparent AI for the reviewer side, or leave that responsibility to proprietary systems it does not understand and cannot audit.

Turing built his machine, with his team, to solve a problem that would otherwise defeat them.

That seems like a reasonable model.

Thanks for reading Random Walk. If you have a better idea for saving peer review, I would genuinely like to hear it. The editors I know are running out of patience and reviewers simultaneously.

Share Random Walk

Leave a comment

Somewhere in the world right now, a paper is waiting to be reviewed. It has been waiting for six weeks. The editor has asked fourteen people. Three declined immediately. Four said yes and then went silent. Three are being sent a reminder that will probably be ignored. One is actually reading it.

This is not an exaggeration. Many editors today routinely invite ten to twenty reviewers before finding two who agree to review a paper. The system that science depends on most is the one scientists least want to participate in.

How did we get here?

The incentive structure has quietly become asymmetric. Every researcher benefits from receiving peer review because publication is the currency of the academic career: hiring, promotion, grants, and reputation. All of it flows through the publication system, and the publication system runs on peer review.

But doing the reviewing counts for almost nothing on a CV. A careful review takes four to ten hours. It appears nowhere on a performance dashboard. It impresses no hiring committee. It does not move the h-index needle. So when the review request arrives, the rational calculation is obvious, and many researchers, especially junior ones, who are under the most publication pressure, do the rational thing.

They decline. Or they accept and then quietly disappear.

Economists have a name for this: a public goods problem, or more colourfully, the free-rider problem. Everyone benefits if enough people review. But each individual benefits slightly more by letting someone else do the work. If everyone reasons this way simultaneously, the system collapses. We are not at collapse yet. But we are somewhere between “straining” and “quietly panicking.”

There is a deeper irony that I find genuinely funny in a slightly painful way.

Academics study long-term phenomena for a living. Econometricians model decade-long business cycles. Epidemiologists track disease patterns across generations. Statisticians build models of processes that unfold over years. And yet when it comes to their own careers, the same people make relentlessly short-term decisions. A paper accepted today has an immediate, measurable reward. Fifty excellent reviews over five years? The payoff is delayed, uncertain, and invisible.

The people who study patience the most are often the least patient about their own careers.

This brings me to the three types of statisticians.

• A good statistician knows exactly how to measure.

• A better statistician knows how to measure it better.

• The best statistician knows when not to measure, because measuring sometimes causes more harm than it solves.

The academic community has spent thirty years building increasingly elaborate ways to measure intellectual capital: citations, h-index, i-index, journal impact factors, altmetrics, and combinations thereof. What it has almost entirely failed to measure, or even acknowledge, are the other two forms of capital that successful academic careers are actually built on.

Social capital: the trust and relationships you build within your scholarly community. Reputational capital: what respected colleagues say about you when you are not in the room.

Peer review contributes to both of these simultaneously, and we have done a remarkably poor job of teaching this to our PhD students.

Think about what a thoughtful review actually signals.

When an Associate Editor sends you a paper, they are implicitly asking: Can I trust your judgement? If you return a careful, balanced, timely review, you have answered that question more clearly than any publication could. A well-written review reveals how you think, how you read evidence, and where you draw lines. Sometimes more transparently than a published paper, because in a review, there is no incentive to perform.

Editors remember this. Not consciously, not on a spreadsheet, but in the way that people remember who showed up when it mattered. When they need a guest editor, an editorial board member, or an Associate Editor, they think first of people who have helped them. When a senior colleague needs to write a support letter, and they have seen your judgment across twelve reviews over four years, they can write something that no amount of citations can manufacture.

This is the invisible compound interest of academic life. It does not show on any dashboard. It accrues silently. And then, usually when you least expect it, it pays out.

The Doorman Fallacy

In my earlier post, I wrote about what I called the Doorman Fallacy. A hotel replaces its doorman with an automatic door. The cost saving is immediate and visible. The cost of the missing smile, the missing judgment, the missing institutional memory, is invisible and delayed. It shows up two years later in declining occupancy rates that the CFO blames on macroeconomic headwinds.

Peer review is the doorman.

Its value is real but unmeasurable on any quarterly dashboard. The academic system keeps reducing the incentive to do it, one CV optimisation at a time. The cost will show up, is already showing up, in slower science, lower quality publications, and editors who spend more time chasing reviewers than editing journals.

Academia operates on an implicit social contract: today I review your paper, tomorrow someone reviews mine. When enough people stop contributing, the contract frays. We are watching it fray in slow motion, and the people doing the fraying are, by and large, behaving entirely rationally within the incentive structure they have been given.

That is not a criticism of individuals. It is a diagnosis of a system.

The cure is not to lecture researchers about duty, though I have just done a version of that. The cure is to take social and reputational capital seriously, to teach PhD students that a career is built on three legs, not one, and to design evaluation systems that reward the invisible work that holds the whole edifice together.

Until then, the conscientious reviewer is doing something that Mahalanobis would have recognised: building an institution, quietly, without being asked, without being measured, because the institution needs it.

That is, I think, the most underrated form of scholarship.

And if you are an editor reading this: please stop sending me review requests. I am already two weeks late on three of them.

Thanks for reading Random Walk. If you have declined a review request in the last month, I am not judging you. I am just asking you to think about who reviewed yours.

Share Random Walk

Leave a comment

The Problem: A Broken Plumbing System

When India buys oil from the UAE, both countries price the oil in dollars, pay in dollars, and route the payment through American banks in New York that connect to the SWIFT messaging system. Neither India nor the UAE prints dollars. Both are therefore permanently dependent on the USD. But most of the time it worked for both countries since the Second World War.

However, in 2022, Russia was thrown out of the SWIFT system. Overnight, Russia discovered it could not receive payment for its oil in the normal way. It sends a message to every leadership of the countries that are not part of the Western Hemisphere that if their policy is not liked by Washington, they can be thrown out of the plumbing system.

India has faced several US sanctions in past, so the threat is real for India. Particularly, when the field marshal of India’s enemy country is very good at sucking up to Washington’s knee. The question is whether a group of Asian countries can build a parallel plumbing system. The short answer is: yes, partially, and with significant difficulty. The long answer is the rest of this post.

Step One: The Bilateral Swap

The Reserve Bank of India and the UAE Central Bank sit across a table and make a simple exchange.

UAE Central Bank hands RBI 50 billion AED. At an exchange rate of roughly 22 rupees to one dirham, RBI hands back 1.1 trillion INR to the UAE central bank. Both central banks now hold the other country’s currency in a dedicated pool.

The plumbing now works like this.

An Indian oil company wants to buy crude from an Emirati supplier. Instead of buying dollars, it pays the RBI in rupees. RBI draws from its AED pool and pays the Emirati supplier in dirhams. The dollar never enters the transaction.

On the other side, a UAE food importer wants to buy rice from India. It pays the UAE Central Bank in dirhams. The UAE Central Bank draws from its rupee pool and pays the Indian rice exporter in rupees.

At the end of the agreed period, say one year, both central banks calculate the net position. If India has drawn 30 billion AED worth of oil payments, and the UAE has drawn only 15 billion AED worth of rupee payments for food, India owes the difference. That difference is settled in gold, or in a third agreed currency, or rolled into the next period at an adjusted rate.

The elegance is in the netting. Neither country needs to earn dollars first. They simply use what they already produce.

But the moment you draw this on paper, the trade imbalance problem arises. India buys far more oil than the UAE buys rice. The rupee pool at the UAE Central Bank fills up faster than it can be emptied. And here is the asymmetry that cracks the foundation: a dirham can buy things in many places. A rupee can only buy things in India. So, the UAE is accumulating a currency it cannot easily use elsewhere. This is happening because India is a systematically net-importing country and the UAE is a net exporter.

The bilateral system has a ceiling. To raise that ceiling, you need more countries.

Step Two: The European Lesson

In 1950, Europe faced almost exactly this problem, without the oil.

The Second World War had left every European economy in ruins. The Marshall Plan gave Europe some dollars, but not enough to fund all the trade that reconstruction required. And the deeper problem was structural: no European currency was convertible. A German mark could not be freely exchanged for French francs. So trade was bilateral and cumbersome. Germany could only buy as much from France as France bought from Germany, because there was no mechanism to settle imbalances multilaterally. A French surplus with Germany could not be used to offset a French deficit with Italy. This precisely the problem Europeans solved.

Seventeen countries joined the European Payments Union (EPU) in 1950, administered by the Bank for International Settlements (BIS) in Basel. Here is how it worked concretely.

Every month, each country’s central bank reported every trade transaction it had conducted with other member countries. The BIS in Basel took all these numbers, and instead of settling seventeen separate bilateral balances, it calculated one net position for each country across all its partners simultaneously.

Imagine Germany exported to France, France exported to Italy, and Italy exported to Germany, each in roughly equal amounts. Under a bilateral system, three separate settlements would be needed. Under the EPU, these positions netted to nearly zero, and settlement was minimal. Trade that would have been blocked by the lack of convertible currency now flowed freely.

Countries that ended up as net creditors, meaning they exported more than they imported across all partners, received partial payment in gold or dollars, and extended the rest as credit to the system. Countries that ended up as net debtors paid a portion in gold and received credit for the rest, up to a limit. If a country’s deficit grew too large, it had to adjust: buy less from others, sell more, or borrow from the EPU’s central pool.

This arrangement did two things quietly over eight years.

First, it allowed European trade to grow dramatically. Countries could run deficits with some partners and surpluses with others, and only the net mattered. Trade volumes roughly tripled between 1950 and 1958.

Second, as trade grew, so did European economies. As economies grew, governments accumulated foreign exchange reserves. As reserves grew, governments gained the confidence to remove the capital controls and restrictions that had made their currencies unconvertible in the first place. A currency is unconvertible when a government is afraid that if it allows free exchange, everyone will dump it for USD and the exchange rate will collapse. That fear recedes when the economy is strong and reserves are deep.

By December 1958, the major European currencies, the German mark, the French franc, the Italian lira, and the pound sterling, became convertible for current account transactions. It just achieved its purpose in just eight years, where it built the foundations that allowed European currencies to walk from weakness to strength. On that foundation, Europeans built everything that followed; the European Economic Community, the European Monetary System, and eventually the Euro. The EPU was not the destination. It was the soil.

That is the best possible outcome for any such system. Build it, use it, outgrow it.

Step Three: The Asian Version, With a Digital Layer

Now imagine not two countries but five, to keep the example manageable. India, UAE, Malaysia, Indonesia, and China.

Each central bank deposits its currency into a shared pool managed by a jointly owned International Clearing Bank; let’s call it the ICB. The ICB is not owned by any one country; it is a treaty institution, like the BIS, governed by a board where each member country has representation.

Every transaction between member countries is recorded digitally on the ICB’s system in real time. An Indian company buys oil from UAE: the system records it. A UAE company buys palm oil from Malaysia: recorded. A Malaysian company buys Indonesian coal: recorded. A Chinese textile exporter ships to the UAE: recorded.

At the end of each month, the ICB calculates net positions.

Say UAE has accumulated 500 billion rupees from oil sales to India, but only spent 100 billion rupees on Indian goods. Under a bilateral system, the UAE is stuck with 400 billion rupees it cannot use. Under the ICB system, the UAE can spend those rupees to buy Chinese textiles or Indian IT services or Malaysian electronics. The rupee flows multilaterally, not just back to India.

The net positions are settled partially in a hard reserve asset, gold or a small shared reserve fund, and partially in credit that rolls to the next period. Countries with persistent large deficits face a graduated system: first a warning, then a requirement to pay a higher proportion in hard assets, then a ceiling on how much they can draw. This creates discipline without being punitive.

The digital layer makes this faster and cheaper than the European version. Smart contracts can automatically execute settlement the moment the shipment is verified. Exchange rate adjustments can be calculated in real time. The correspondent banking fees that currently extract a percentage from every cross-border transaction disappear.

Step Four: The Dispute Resolution Framework

This is the part that sounds boring and is actually the load-bearing wall.

Three types of disputes will arise.

Commercial disputes: the rice arrived but was substandard. The oil shipment was short. The software was delivered late. These are resolved by a fast arbitration panel, modelled on international commercial arbitration, with a binding decision within sixty days. Both parties must agree to arbitration as a condition of joining the ICB system.

Currency disputes: one member country is deliberately devaluing its currency to make its exports cheaper, at the expense of other members. The ICB monitors exchange rates and flags deviations beyond an agreed confidence band. Automatic adjustments to settlement rates kick in. No human discretion required.

Trade disputes: one member imposes a sudden export ban on food during a shortage, breaking an existing supply contract. This is the hardest category. The only real enforcement mechanism is suspension from the ICB system. That threat only works if the system is valuable enough that no country wants to leave it. Which is why building the value of the system, growing the volume of trade inside it, is not just an economic goal. It is the enforcement mechanism.

The Fundamental Difficulty

All of this is architecturally sound. The EPU proved the concept. The digital layer makes it more efficient. The multilateral netting solves the bilateral imbalance problem to a great extent.

But two problems remain that no clever design fully solves.

The first is a persistent structural imbalance. If India is always a large net importer from the group because it buys oil and commodities, it will always be drawing down its reserves and accumulating debt to the system. The EPU handled this through credit limits and discipline. But the EPU was also working alongside the Marshall Plan, which gave Europe an injection of hard currency that helped the system stay liquid. An Asian system has no Marshall Plan.

The second is the convertibility problem. The EPU worked partly because all members had roughly similar motivations: rebuild, trade, grow. An Asian system spanning India, UAE, Malaysia, Indonesia, China, and potentially Iran or Russia involves countries with very different interests, different relationships with the dollar system, and different levels of trust in each other. Particularly, the trust deficit between China and India is so huge; I will be really surprised if an Asian Payment Union really works out. The rupee will not become freely convertible simply because India joins a clearing union. That requires deeper structural reform. India has to implement reform which they promised everyone (including their own citizen). But the Indian leadership (both bureaucracy and civil, left and right) truly believes in the wisdom of the turtle, so they are slower than Miss Turtle.

The honest answer is: an Asian multilateral clearing system with a digital payment layer can reduce dollar dependence at the margin, protect members from the most extreme sanctions scenarios, and grow trade between member countries meaningfully. It cannot fully replace the dollar system. Not yet. Perhaps not for a generation.

But Europe in 1950 was not trying to replace the dollar either. It was trying to survive without enough of them. It built something modest, practical, and useful.

That is not a bad model. The question is not whether Asia needs this system. The question is whether Asia can trust itself enough to build it.

Share Random Walk

Leave a comment

The image was generated using the Gemini Flash 2.5 model

After the last two posts, some of my friends asked me a version of the same question.

Okay, they said. We understand the Doorman Fallacy. We understand that quarterly capitalism is structurally blind to invisible value. We understand that the Sleeping Giants are sitting on an opportunity and appear to be missing it. But what should I do? I am twenty-three years old. I am starting my career. How do I make myself valuable in a world where AI is eating everything in sight?

I did not have a clean answer. So I did what I usually do: I sat with the question until it became a better question.

This is what I found.

The Junior Engineer Has Always Had to Reinvent Herself

A junior engineer in 1995 wrote boilerplate code. That was the job. Write the repetitive scaffolding that the senior engineers did not want to write. It was unglamorous. It was also, in retrospect, a curriculum. You learned the structure of code by writing a lot of it. You learned what broke by breaking things. You became a senior engineer because you had written enough boilerplate to understand why the boilerplate was wrong.

A junior engineer in 2005 learned to Google efficiently. The skill was not the answer; it was knowing which answer to trust and which to ignore. A new kind of judgment, built on top of the old one.

A junior engineer today should be learning to interrogate AI outputs. To find the failure modes. To test the edge cases. To understand why the model got it wrong. That is still a junior role. Why is an AI agent constantly hallucinating? It still requires someone to do it. It still builds judgment over time.

The role has not disappeared. It has transformed. The problem is that most companies have not noticed the transformation. They have seen the AI and thought: We no longer need the junior engineer. What they have actually done is eliminate the first chapter of the curriculum that produces senior engineers.

The Vicious Cycle Nobody Is Talking About

Here is the cycle that worries me more than the layoffs themselves.

Companies stop hiring junior engineers because AI can do junior work. Junior engineers never develop into senior engineers. In ten years, there will be no senior engineers who understand the domain deeply enough to know when the AI is wrong. The AI produces a confident, fluent, subtly incorrect answer. Nobody in the room has the judgment to catch it. If they don’t want to shoot on their own foot, they should build a talent pipeline that will know how to fix a hallucinating rogue agent in 2035. If we don’t do it, China will do it.

This is not a hypothetical. It is the same cycle that hollowed out American manufacturing. Not a single decision. Thousands of rational decisions, made by thousands of companies, each one sensible in isolation, collectively catastrophic.

And it is perfectly consistent with the Doorman Fallacy. The junior engineer is the doorman. Remove her to save the cost. Discover, years later, that she was also the person who knew the guests’ names.

The Paradox at the Centre

So the advice for a young person seems obvious: go deep. Develop judgment. Think from first principles rather than from shortcuts.

But here is the paradox that stops me every time I try to give that advice cleanly.

First principles thinking is not a technique. It is a capacity. And like all capacities, it is built through friction: through hitting a wall, failing to solve a problem with the easy answer, and being forced to go back to the foundations. The struggle is not the obstacle to understanding. The struggle is the method.

AI removes friction.

A young person who asks AI for every answer never hits the wall. They get a smooth, confident, well-formatted response, and they move on. The muscle that would have developed through the struggle never gets used. It atrophies. And the cruel part is that they do not notice, because the answer was right there, and it looked correct, and nothing broke immediately.

The easy path is one prompt away. Always. That is both the miracle and the danger of this moment.

Two Ancient Technologies for Blocking the Easy Path

I keep coming back to two traditions that understood this problem long before AI existed.

The first is the Socratic method. Socrates did not give answers. He asked questions, and when his interlocutor reached a comfortable answer, he asked another question that pulled it apart. The method works not because questioning is clever, but because the easy path is physically blocked. There is a person in the room who will not let you stop at the first answer. Every comfort is interrupted. The goal is not to reach an answer; it is to reach a better question, and to sit with the discomfort of genuinely not knowing.

The second is Adi Shankaracharya’s practice of purvapaksha. Before Shankaracharya could argue his own position, the tradition required him to first state the opposing position in its strongest possible form, stronger than the opponent had stated it themselves. He had to inhabit the view he was about to refute. He had to understand it from the inside before he could critique it from the outside. The practice makes intellectual laziness structurally impossible. You cannot dismiss an idea cheaply if you are required to defend it first.

Both traditions share one crucial feature. They are not techniques you apply occasionally when you remember to. They are systems you live inside. And they work because the system is enforced externally, by a person, a community, a tradition, until the day, if you are lucky, the enforcement becomes internal. Until the question becomes the reflex, not the answer.

The Problem That Neither Socrates Nor Shankaracharya Had to Solve

Here is where the advice gets hard to give.

Both traditions required an institution. The Socratic school. The gurukul. A community where shallow thinking was socially unacceptable, where someone would ask why every time you reached a comfortable answer, where the purvapaksha was not optional. The method without the institution is a gym membership you never use.

The young person today is largely on their own. There is no gurukul. There is no Socrates in the room. There is only the screen, and the prompt box, and the answer that arrives in three seconds, formatted beautifully, sounding confident.

You cannot develop first principles thinking by yourself, occasionally, with good intentions. The easy path will win. It is not a question of discipline or character. It is a question of what the environment rewards. The gurukul worked because the environment made the hard path the only path. The modern environment makes the easy path the default. Fighting defaults requires enormous energy, and most people, quite reasonably, do not have that energy in infinite supply.

The Question I Am Left With

So we arrive at the same kind of ending I seem to keep arriving at.

The individual answer is correct but insufficient: develop judgment, embrace friction, practice purvapaksha on your own ideas, find a Socrates who will not let you off the hook. Do not use AI as an answer machine; use it as a sparring partner that you interrogate rather than consult.

But the systemic answer does not yet exist. The gurukul answered this for its age. The Socratic school answered it for theirs. What is the institution for our age? What is the community, the practice, the social technology that makes the hard path the expected path, in a world where the easy path is one prompt away?

I do not know. I do not think anyone has built it yet.

The generation that does will define what human intelligence means in the age of AI.

And if you are trying to build it, I would genuinely like to hear from you.

Thanks for reading Random Walk. If this made you think of a question rather than an answer, that is probably the right response.

Share Random Walk

Leave a comment

The Doorman Fallacy is what happens when capitalism optimises for what it can measure and destroys what it cannot. I ended that piece with a question: Can capitalism govern itself when the people with the most power have the shortest time horizons?

I have been thinking about that question since. And I think the answer, at least in the technology industry, depends entirely on which tribe you belong to.

There are four tribes of technology companies in the AI era. Each has a completely different relationship with the doorman.

Tribe 1: The AI Landlords

Microsoft. Google. Meta. Amazon.

These companies own the AI. They built it, they sell it, and they use it to cut their own workforce simultaneously. Amazon is spending $125 billion on AI infrastructure while eliminating 30,000 corporate jobs. Mark Zuckerberg has said, with the casual confidence of a man who has nothing to lose, that AI will soon replace a mid-level software engineer. Andy Jassy of Amazon says the company will “need fewer people doing some of the jobs done today.”

They can say this out loud because they are the infrastructure. They are the landlords of the new economy. The workers who are displaced will eventually pay rent to them for access to the AI tools that replaced those very workers. It is an elegant business model, if you can stomach the view from inside it.

For most of my readers in India, Tribe 1 is a distant reality. We use their products. We read about their valuations. We watch their CEOs speak at Davos. But we do not build them, and we do not own them. They are American companies, and the wealth they are creating in this moment is accumulating in America. This is worth sitting with.

But even the landlords are not entirely safe from the Doorman Fallacy. When Zuckerberg replaces mid-level engineers with AI today, where do his senior engineers of 2035 come from? Senior engineers are made, not born. They are made by years of writing code, making mistakes, fixing them, and slowly accumulating the kind of judgment that no prompt can replicate. Stop hiring mid-level engineers today, and you are quietly cancelling the senior engineers of tomorrow.

Tribe 1 knows this. Their answer is simple: in 2035, they will write a large cheque and poach whoever they need from wherever that talent has been patiently nurtured. They have the cash. They will buy the doorman back when they need him.

This is not the Doorman Fallacy. It is something colder. It is the rational outsourcing of the cost of talent development to the rest of society, while keeping the private benefit for themselves. No single company is making a mistake. The system, collectively, is.

A game theorist would recognise this immediately. Each firm, acting rationally, passes the cost of training the next generation of engineers to society at large, to universities, to smaller companies, to governments. When every firm does this simultaneously, nobody trains anybody. The skill quietly dies. Not because anyone chose to kill it, but because everyone chose not to pay for it.

America has seen this film before. It did not plan to lose its manufacturing to China. No single company made that decision. Thousands of companies, each acting rationally, each outsourcing one more factory, one more supply chain, one more production line. The cumulative result was the hollowing out of an industrial base that took a century to build and a decade to lose. They kept the profit. They exported the capability. And now they cannot build a semiconductor fab or a shipyard even when they desperately want to, because the knowledge is gone; the workers who knew how retired, the apprentices who would have learned from them were never hired.

Software engineering is next. The mid-level engineer that Zuckerberg is replacing today was the senior engineer of 2035. The senior engineer of 2035 was the architect of 2045. You cannot buy back institutional knowledge with a large cheque if the knowledge no longer exists anywhere to be bought. While China is quietly building the talent pipeline! Have you seen their martial arts robot dancing at the 2026 festival gala?

Tribe 2: The Builders

OpenAI. Anthropic. Mistral.

Privately held. The talent is the product. You cannot fire the doorman here because the doorman is writing the next model. Every researcher who leaves takes with them not just skill but institutional memory; the specific knowledge of what was tried, what failed, and why. That knowledge cannot be documented in a handover note. These companies hoard talent the way previous generations hoarded land.

They are not paragons of virtue. They are simply in a situation where firing the doorman destroys the hotel. The incentive structure aligns with stewardship, not because the founders are philosophers, but because the economics demand it.

Like Tribe 1, Tribe 2 is largely an American and European story. India has brilliant researchers contributing to these organisations, often from abroad. But the companies themselves, the ownership, the upside, the strategic control; that sits elsewhere. India is a talent exporter to both Tribe 1 and Tribe 2. That is a different conversation, and a harder one.

Tribe 3: The Lean Builders

The VC-backed AI startups. One engineer with AI can now do the work of ten. These companies were never going to hire large teams. There is no doorman to fire because there was never a doorman budget to begin with. The Doorman Fallacy does not apply here; the hotel was automated from the ground floor.

Tribe 4: The Sleeping Giants

TCS. Infosys. Wipro.

Their business model was built on one elegant arbitrage: Indian engineers are talented and cost less than Western engineers, so Western companies outsource their technology work to India. For thirty years, this worked beautifully. India’s IT sector became a $280 billion engine of the economy. Millions of engineering graduates found their first job, learned their craft, and sent money home to parents in smaller cities and towns.

AI has destroyed the arbitrage. The marginal cost of an AI coding agent is now approaching the cost of electricity. An AI agent does not need a visa, does not ask for a raise, and does not take maternity leave. Indian IT stocks lost $50 billion in market capitalisation in a single month in early 2026. The bench of available engineers across the top five Indian IT firms shrank from 300,000 to 225,000 in two years.

One of these companies fired over 12,000 people in a single year, its largest layoff ever. And that company is a Tata company.

The Tata Trusts, which fund hospitals, universities, and rural development across India, own 66% of Tata Sons, which in turn is the largest shareholder of that listed IT giant. By any structural measure, it should be the most insulated large company in India from quarterly capitalism. The Tata name. The Tata philosophy. The same group that introduced the 8-hour workday in India in 1912, decades before it became law. The same group that Ratan Tata built into a symbol of responsible enterprise.

And yet. Twelve thousand people.

When even a Tata company cannot keep the doorman, the question is settled. Going public is not a risk factor for losing courageous stewardship. It is a permanent, structural transfer of authority from the founder’s conscience to the market’s quarterly mood. The minority public float wags the majority Tata tail. This is not a failure of character. It is a failure of structure.

But I want to resist the temptation to end Tribe 4’s story in tragedy. Because Tribe 4 is sitting on an opportunity that most of its leaders have not yet seen clearly.

Think about the CEO of an auto manufacturer. She is under pressure from her board to use AI across her organisation; to automate procurement, to optimise supply chains, to cut design iteration time. She opens a browser and looks at the available tools: ChatGPT, Gemini, Claude. Large, powerful, cloud-based models run by third-party technology companies.

She will not use them.

She will not type this month’s production numbers, her supplier contracts, her next model’s launch timeline, into a third-party AI model sitting on someone else’s server. She will not put her competitive advantage into a third-party company’s training data. She cannot; her lawyers will not allow it, her board will not allow it, and her instinct will not allow it.

What she needs is a private LLM, fine-tuned on her own data, running on her own infrastructure, maintained by people she trusts. She needs someone who already knows her systems, her sector, her compliance requirements. Someone who has been inside her company for fifteen years.

That someone is Tribe 4.

And here is what makes this almost poignant. The engineers who built her data pipelines, who maintain her servers, who know her systems better than her own CTO; these are the very people now sitting on the bench. The doorman. The Python girl. They are already inside the building. They already have the data access, the infrastructure contracts, and the security clearances. Fine-tuning an open-source model on the client’s own servers is not a transformation. It is a slight pivot. Tribe 4 does not need to win new clients, build new infrastructure, or negotiate new data agreements. They need to change what they do with the access they already have.

The people being replaced are the people best placed to do the replacing. If only someone would let them.

This is the time Tribe 4 should have championed “sovereign AI”. Let me be precise about what I mean by sovereign AI, because in India, the term has been kidnapped. People hear “sovereign AI” and picture a government committee building a model that arrives six months late and works sixty per cent of the time. I do not mean the government version. I mean yours, a private enterprise LLM that never leaves your servers, trained on your data, answerable only to you. No third-party cloud. No third-party training data. No lawyers losing sleep.

The market for this kind of sovereign AI, built on open-source models like LLaMA and Mistral, fine-tuned on proprietary data and deployed on private infrastructure, is going to be one of the largest enterprise technology markets of the next decade. Every bank, every hospital, every defence contractor, every mid-sized manufacturer in Germany and Japan and South Korea faces exactly the same problem as our auto CEO. They all need the same solution. And the Indian IT services companies have the one thing that no AI startup can acquire quickly: thirty years of enterprise trust.

These companies have been inside that auto manufacturer’s systems for fifteen years. They know where the data lives. They have the infrastructure contracts. The sovereign AI market is theirs to lose.

But winning it requires the opposite of what they are currently doing. It requires investing in ML engineers who understand transformer architectures, fine-tuning, and vector databases; exactly the kind of talent currently sitting on the bench being eliminated. It requires patience; the sovereign AI market will not fully mature for another three to five years. It requires the courage to say to the quarterly earnings call: “We are retraining 50,000 engineers rather than firing them, because they are the raw material for the business we are building.”

That courage requires being protected from the market.

And they are not.

Here is what troubles me. By now, at least one of these companies should have stood up and announced a Sovereign AI Services model — a clear, bold pivot: we are becoming the trusted partner for enterprises that cannot put their data on third-party servers. We are the sovereign AI layer between your data and the world. It should have been the defining announcement of the year. The CEO should have gone on stage at Davos and made the fund managers feel clever for owning the stock.

We have heard nothing.

No major announcement. No strategic pivot. No rallying cry. Just silence, a few cautious press releases about “AI-readiness,” and quarterly earnings calls explaining why margins are under pressure.

Have they seen the opportunity and decided the restructuring is too hard? Have they not seen it at all? Or have they simply frozen, caught between the enormity of what is coming and the quarterly pressure of what is already here?

We now have an answer. In December 2025, TCS, Infosys, Wipro, and Cognizant made their big AI announcement. They signed partnerships with Microsoft to collectively deploy 200,000 Microsoft Copilot licenses across their workforces.

That was it. That was their moment.

Instead of standing up and saying “we will be your sovereign AI layer, your data stays on your servers, we fine-tune the model, we own the stack,” they said “we will be Microsoft’s largest distribution channel.” They chose to become resellers of the very third-party technology companies that our hypothetical auto CEO is afraid to trust with her data.

They have made the structurally rational choice and dressed it up as a strategy. Sell someone else’s product, collect the service fee, protect the margin, and satisfy the quarterly call. It is not courageous. But before we condemn it, we should pause.

A significant portion of these companies is held not by faceless hedge funds but by ordinary Indians, the retired schoolteacher in Pune whose mutual fund holds TCS, the engineer’s parents in Nagpur whose provident fund is quietly compounding in Infosys. The same quarterly capitalism that prevents courageous stewardship is also the structure that protects middle-class India’s savings. The CEO who takes the bold sovereign AI bet and gets the timing wrong does not just lose her job. She wipes out the retirement plan of millions of people who never asked to be part of the experiment.

This is the trap’s real cruelty. There is no clean villain. The structure that makes the brave choice impossible is also the structure that makes ordinary Indian life a little more secure. Condemning one means reckoning with the other.

The sovereign AI market is theirs to lose. Whether the structure allows them to win it is a different question entirely.

The Trap

Here is the scenario that concerns me most about Tribe 4’s opportunity.

A private, founder-led sovereign AI company emerges. It trains engineers carefully. It builds enterprise trust slowly. It refuses to fire people because it understands that institutional knowledge is the product. It grows to a point where a large, listed Indian IT company notices.

The large company acquires it. The founder receives a large cheque. Within eighteen months, the culture is gone, the engineers have left, and the acquirer has a client list and an empty building.

Or the large company does not acquire it. Instead, it poaches the top twenty engineers at three times their current salary. The company hollows out. The remaining team cannot deliver. The enterprise clients quietly move their contracts to the larger firm.

The money used to acquire or poach comes partly from the salary savings of the thousands of people these companies fired last year.

The doorman was removed so the hotel could hire a consultant to figure out why the guests stopped smiling.

The Zoho Question

There is one Indian technology company that has cracked the code of courageous stewardship. It is not an IT services company; it is a software products company. But its philosophy is worth studying.

Sridhar Vembu built Zoho Schools in 2005, training students from rural backgrounds, many without formal degrees, into capable software professionals. Fifteen per cent of Zoho’s workforce now comes from there. He moved to a village in Tamil Nadu and built a hub-and-spoke network of offices in tier-2 towns and villages, employing thousands of people who were previously invisible to the technology industry. He refused VC funding and a stock market listing for three decades. He has a no-layoffs policy. And he said, with a directness that most technology CEOs would find career-threatening: “I don’t need more money. Where I come from, people need jobs.”

Zoho cannot easily be acquired, not because of a legal structure but because the culture is the product. The engineers who chose Zoho over better-paying offers did so for reasons that a large acquisition cheque would not resolve.

This is the defence against the acquisition trap. Not a clever shareholding structure, not a poison pill; culture so deep that money cannot buy it. Build the mission before the revenue. Make the company about something larger than the contract.

Tribe 4 does not need to become Zoho. But it needs a Zoho. It needs a private, founder-led IT services company that bets on sovereign AI, retrains rather than fires, builds enterprise trust over a decade, and becomes so embedded in its mission that the acquisition offer, when it comes, is simply not interesting.

The Question I Am Left With

That founder exists somewhere in India right now.

Probably in a tier-2 city. Probably being told by every VC that the business does not scale fast enough for a Series A. Probably watching the quarterly results of the Sleeping Giants and running the numbers on whether the safe move is to take the acquisition offer when it arrives.

The ecosystem around her; the investors, the clients, the government, the universities; will determine whether she gets the time and the space to build something that cannot be bought.

Or quarterly capitalism, patient as always, will simply wait for her to get tired.

And then it will make her an offer.

—

Thanks for reading Random Walk. If you know that founder, or if you are that founder, I would genuinely like to hear from you.

Share Random Walk

Leave a comment

A few weeks ago, I sat down with an LLM and asked it a question that has been bothering me for a while.

If AI takes everyone’s jobs, and it will take a lot of them, who exactly buys what AI produces? A factory with no workers, producing goods that nobody can afford to purchase. A very elegant trap.

What followed was a long, increasingly desperate debate in which the LLM proposed: Universal Basic Income, Social Participation Wages, a Nation of Freelancers, Post-Scarcity Price Deflation, a National Dividend, a Tiered Shareholder Model, Smart Contracts, and Direct Digital Voting by citizens.

In each case, I pointed out that it sounded like communism.

Because it did.

By the end, the LLM had essentially argued for communism while desperately insisting it wasn’t communism. It even used the words Animal Farm itself. I did not make it say that. It walked into it on its own.

But the most interesting thing happened after I stopped the debate. Because the real answer, or rather the real question, wasn’t anywhere in the LLM’s proposals. It was hiding in a hotel doorman.

I will get to the doorman. First, let me show you the debate.

The Debate

A note on format: my prompts are short and rude. The LLM’s responses are long and optimistic. Make of that what you will.

Me: AI will result in layoffs, that makes complete sense, most manual work will be done by robots. Perhaps this is good, because the economy will correct itself. But if unemployment goes through the roof, that will result in a huge population with no disposable income. Then the stuff that AI produces; who will buy it? You may propose UBI (Universal Basic Income), but that will be insulting to most of the population. Getting money for doing nothing. Also, the other side of the population who still has income will say, “Why should my tax go to unproductive people?” It seems like a vicious cycle. Is there any way to break it?

Me: Social Participation Wages; sounds like a socialist idea. But I am open to hear it.

Me: “Make-work bureaucracy where the government decides what is valuable” is a really bad idea. A socialist idea which does not work. Also, why would the current corporate structure give up their companies to a bunch of bureaucrats?

Me: “Nation of Freelancers” feels more viable, but with freelancing, it is very difficult to raise a family, because one will be competing against big AI tech giants.

Me: Hyper-Specialization markets are so niche and small; they will not generate enough employment.

Me: “The cost of a middle-class lifestyle drops by 90%” has a big problem. Labour will be replaced by AI-robots, so labour is in infinite supply at almost zero cost, but the material to build anything, house, road, food, infrastructure, is finite. Oil will still come from oil-producing countries. So lifestyle cost will not drop by 90%. Only labour cost is reduced. The cost of most products will reduce by only 30–40%.

[This, I think, was the moment the LLM started to sweat. If it sweats.]

Me: You are suggesting a highly socialist economy.

Me: So you think you will redistribute for “fear of revolution,” sounds like communism, and force the 1% to agree to a shareholder model under duress? That does not sound like democratic values.

Me: The “Stock Buyback in Reverse,” buying shares on the open market for a National Trust, is a good idea. But how do you decide the share of a Python coder who has been fired and the share of a taxi driver who just lost his job to autonomous Uber? Will the dividend be enough to cover a family? Including health insurance?

Me: “The dividend is typically equal for every citizen”; sounds like communism. Everybody is equal, except those in the Politburo who are more equal. Similarly, here every shareholder will be treated equal; except those on the board, who will be treated more equal.

Me: (I stopped here. Because the debate was going nowhere. And because I realised the LLM was now cornered — proposing communism via blockchain, which is somehow both the oldest and the newest bad idea simultaneously.)

The Pattern I Noticed

Every solution the LLM offered followed the same arc:

1. It proposes something.

2. I point out it is socialism.

3. It repackages the same idea with a new label.

4. Return to step 1.

UBI → insulting. Social Participation Wages → make-work bureaucracy. Nation of Freelancers → can’t raise a family. Hyper-Specialisation → too niche. Post-Scarcity Deflation → materials are finite (the 30-40% point). National Dividend → communism. Smart Contracts to manage the dividend → still a Politburo, now with a blockchain.

The LLM kept looking for a structural solution. And every structural solution eventually became socialism when you pulled the thread far enough.

But I think we were both looking in the wrong place.

The Doorman Fallacy

In a five-star hotel, you can replace the doorman with an automatic door. You cut a cost. Next quarter, your profit goes up. The CFO is happy. The fund manager is happy. The stock ticks up a little. Rory Sutherland named this idea in his 2019 book Alchemy. I borrowed it gratefully; the application here is my own.

But a doorman does not only open the door. He picks up luggage. He calls the taxi. He smiles. He is the first responder. He is the face of a luxury hotel. He remembers the name of the guest who stayed three years ago.

When the doorman is gone, and a guest is standing on the pavement struggling to hail a cab, the experience is no longer five-star. The guests leave quietly. They don’t complain; they just don’t come back. And that revenue loss does not appear anywhere on next quarter’s report. It appears, silently, two years later, in declining occupancy rates that the CFO will blame on macroeconomic headwinds.

This is the Doorman Fallacy: the cost of removing the doorman is visible and immediate; the cost of the missing smile is invisible and delayed. Capitalism, as currently practised, is structurally blind to the second kind of cost.

Now apply this to AI.

You can fire a Python coder today. Next quarter, your profit goes up. The fund manager applauds.

But if you keep that Python coder, that shy girl who has not yet found her problem, and nurture her talent, she might solve a million-dollar NP-hard problem with a few innovative prompts next year. She might build the thing that makes your competitor obsolete. She is not a cost. She is an option. An option with unknown but potentially enormous value.

Quarterly capitalism cannot price options it cannot see.

Is This a Cultural Problem or a Structural One?

I initially thought this was a cultural failure, that the people running companies had simply lost the courage to think long-term. That we needed what I called courageous stewardship: leaders willing to keep the doorman, to nurture the shy Python girl, to resist the pressure of the eleven-month fund manager.

And then I looked for examples.

Toyota has not conducted a mass layoff of full-time employees since 1948. During the 2008 financial crisis, they retrained and redeployed workers rather than cutting them. Their philosophy has a name, The Toyota Way, built on Respect for People and Continuous Improvement. The result: turnover less than a third of the industry average. Institutional knowledge that compounds for decades.

Tata Steel introduced the 8-hour workday in India in 1912, decades before it became law. The philosophy is explicit: the community is the purpose of the enterprise, not a byproduct of it.

Both are extraordinary examples of courageous stewardship. Both kept their doormen.

But here is what stopped me cold: both are family-controlled businesses. Toyota by the Toyoda family. Tata by the Tata Trusts.

They could afford courage because no activist fund manager could show up and demand they fire the doorman by Friday.

Which means what looked like a cultural problem, short-termism, cowardly CEOs, and quarterly capitalism, is actually a structural one. The incentive structure rewards the cowardly decision. The fund manager who fires the doorman gets a bonus this quarter. The one who keeps her and waits for her to solve the NP-hard problem may himself be fired before she gets there.

Courageous stewardship, it turns out, is only reliably possible when you are protected from quarterly capitalism.

The Question I Am Left With

So we arrive here. The LLM’s structural solutions: National Dividends, Tiered Shares, Smart Contracts; all quietly become socialism when examined closely.

And the cultural solution: courageous stewardship, turns out to require a structural shield to survive.

Which leaves me with a question I cannot answer, and I do not think anyone has answered it yet:

Can capitalism govern itself when the people with the most power have the shortest time horizons?

The doorman’s smile depends on it.

Thanks for reading Random Walk. If you have an answer to the question above, I would genuinely like to hear it - because I don’t have one.

Share Random Walk

Leave a comment

A few days ago, I was reviewing a data science paper. The analysis looked neat, the graphs were shiny, and the conclusions sounded just the right amount of confident. So, like any responsible reviewer, I asked for the code; you know, just to check if the magic actually works outside the author’s laptop.

That’s when things got interesting. At first, the authors politely declined; apparently, the code was “under revision.” (Translation: “it works only when the moon is in the right phase.”) After some gentle insistence, they finally sent it. I opened it with hope in my heart and coffee in my hand, and within five minutes, both hope and coffee were gone. The code looked like a ransom note written by a computer having an existential crisis.

I tried to reproduce the results. The results reproduced me, a frustrated reviewer questioning his life choices. So now I faced a dilemma: should I trust the paper and accept it, or trust my sanity and reject it?

And under these circumstances, I decided to write this post. In data science, writing a paper without sharing the code is like declaring a theorem without providing the proof. It sounds grand, even authoritative, but underneath the polished equations and neat tables, it’s just a story.

Code is the proof. It’s the place where lofty ideas crash into the real world of dirty data, missing values, and the occasional “why-is-this-NaN” panic. Equations look elegant on paper, but code tells us whether those equations survive contact with reality. Until your algorithm runs and reproduces the promised results, it’s not a method; it’s a manifesto.

Reproducibility isn’t just academic hygiene; it’s the antidote to scientific fraud. When your code is open, anyone - a student in Chennai, a data scientist in Nairobi, or a professor in Boston - should be able to run it on their laptop and see the same results light up their screen. That’s how you earn respect in this field: not by saying

“trust me,”

but by saying

“try it yourself.”

If you really believe in your model, publish your code. Let it face the world. A good idea survives scrutiny; a great one invites it.

So yes, in mathematics, a theorem without proof is just philosophy. In data science, a paper without code is just marketing.

Show the proof. Or better - show the repo.

After all, as I like to say in Random Walk - there may be a method to the madness, but in this case, the method better be on GitHub.

Share Random Walk

Leave a comment

The Generals’ Laughter: A Failure of Acceptance

This was not a failure of intelligence gathering; it was a failure of acceptance. Canaris’s agents, from spies in Mexico City to Spanish diplomats in Washington, were drowning the Abwehr in accurate data. The numbers showed that a single American factory was producing more heavy bombers than all of Germany’s aircraft factories combined.

But the Nazi leadership refused to believe the mathematics because it contradicted their mythology and core beliefs.

Reichsmarschall Hermann Göring, head of the Luftwaffe, literally laughed when first presented with figures on American production. When Canaris later showed him photographs of the mile-long assembly line at Willow Run, Göring scoffed. “Do you take me for a fool? These are obviously wooden mock-ups, props for propaganda. The Americans want us to believe they can build aircraft like sausages.” His mind could not reconcile the fact with his racial prejudices. He dismissed the consumption of 16 million pounds of aluminium monthly by asking with contempt, “Then where do they get this aluminium? From their Negro workers? From their women riveters?”

Fu¨hrer Adolf Hitler shared this blindness. He dismissed American capability, arguing a bomber required a dedication and precision that the “mongrel population” of a “weak democracy” could not possibly possess. He decreed that the speed of construction and production was “impossible.” The men who tried to present the truth, like Canaris and his deputy, Hans Oster, were eventually removed for defeatism. Their final punishment for reporting reality with data was execution at Flossenbürg concentration camp in the war’s final days.

Learning: The data and science behind it are not for everybody.

The American Miracle: Willow Run’s Triumph

What the German generals saw as impossible was simply a triumph of American industrial engineering, supply-chain logistics, and resource mobilisation.

The architect of this miracle was Ford production chief Charles Sorenson. He approached the complex 18-ton bomber, with its 1.5 million individual parts, not as a handcrafted machine but as a mass-market automobile. Sorenson redesigned the process, breaking the assembly into 60,000 simple, repeatable operations. Instead of having workers swarm over a stationary aircraft, the aircraft moved slowly past stationary workers. This systematic approach cut the man-hours required per plane from 180,000 to just over 6,000.

Learning : My data science students who curses me why they have to learn complexities of algorithms, Big-O, Small-O, etc. Now you know why understanding the complexity of a process is so important !

The speed of production was matched by the speed of mobilisation. The factory employed 42,331 workers at its peak. This workforce was composed of men, 11,000 women (Rosie the Riveters), 3,800 African Americans, and people of all backgrounds, a truly diverse workforce that the Nazis had dismissed as inherently inefficient. Rose Willil Monroe, the real-life “Rosie,” summed it up best years later: “We just went to work every day and did our job.... It’s just free people doing their best.” This collective, pragmatic effort delivered 8,685 B-24s from that single factory.

The Four Pillars of Enduring National Strength

The story of Willow Run is a profound warning that reality does not bend to ideology. The fate of the Third Reich hinged on its inability to grasp four core pillars of national strength that the United States maximised:

1. Scientific Mindset: An enduring nation must be committed to empirical truth and data. When confronted with facts, leaders must have the integrity to change their strategy. The Nazi leadership’s willful rejection of all evidence, photographs, manifests, and agent reports demonstrated a fatal intellectual rigidity.

2. Industrial Engineering: National success requires the capacity to create and constantly optimise complex, efficient systems. The American triumph was in process innovation, turning a complex product into a mass-manufactured one. Germany’s failure was an adherence to outdated methodology.

3. Banking and Finance: The US demonstrated an unparalleled ability to rapidly mobilise and invest staggering capital into wartime infrastructure. The $200 million for tooling at Willow Run alone eclipsed much of Germany’s total aircraft industry investment. Without this financial depth, industrial engineering cannot be scaled.

4. Embracing All Talent: Above all, national strength relies on utilising all available human resources. The American system, for all its flaws, was flexible enough to bring 11,000 women and thousands of African Americans into the most sophisticated manufacturing jobs. This principle, that talent has neither colour nor gender, created a labour pool exponentially larger and more resilient than the ideologically constrained workforce of the Third Reich.

The verdict of history is clear: mathematics did not yield to mythology. The German intelligence officers were correct. By early 1943, the sheer scale of American production had already determined the outcome of the war. They simply died because their leaders were too arrogant to believe it.

In a two-front, very long war, can India make Tejas in every hour?

The Tejas is far more sophisticated than a B-24. It incorporates advanced composites, radar, avionics, electronic warfare systems, and a modern jet engine sourced from abroad. The challenge of building one every hour is vastly greater. Yet the principle remains. India currently produces only a handful of Tejas aircraft each year. To reach the scale of Willow Run would require a transformation of mindset and organisation. The production model would need to shift from artisanal craftsmanship to true flow-line assembly. HAL would need to share the load with private manufacturers, turning multiple factories into nodes of a single coordinated system. Is HAL ready to give up its control and power it enjoys?

Finance would be the next hurdle. Willow Run cost two hundred million dollars in 1941, a sum that dwarfed much of Germany’s aircraft investment. India would need to commit billions in today’s terms to tooling, robotics, and supply chain contracts. Without such investment, speed will remain elusive.

Equally important is talent. If India seeks to build at scale, it cannot afford to exclude 70% of its population, which resides in village and study in third-tier colleges. Women engineers, technicians trained in ITIs, and skilled workers from smaller towns, must all be part of the effort. In 1943, it was ordinary free citizens who rose to the task at Willow Run. India must be willing to trust and mobilise its own people in the same spirit, inspite of gender, caste, relion or elite graduates or not. Is the Indian leadership, (both political leaders and bureaucrats), ready to disown the belief that everything will be done only by IIT graduates and within the campuses of ISRO, IISc, IIT, HAL and BHEL?

Finally, India must resist the temptation to dismiss data in favour of ideology. If simulations, logistics studies, or war-gaming expose weaknesses, the rational response is to adapt, not deny. The Germans laughed at mathematics and lost. A modern nation cannot afford that mistake.

So, can India build a Tejas every hour?

• In peacetime, the answer is no. The constraints of engines, electronics, and global supply chains still bind.

• But in a true existential war, history shows that it can be done, provided India commits the finance, reorganises its industry, and mobilises its entire population.

The deeper truth is not about aircraft alone. It is that national strength rests on mathematics, engineering, scientific mindset, financial depth, and inclusivity. Myth and ideology may comfort, but in the end they are powerless against the relentless logic of data and science. That will endure and test the true character of a nation.

Share Random Walk

Leave a comment

Economists love to talk about No-Arbitrage. It’s the holy grail: there is no free lunch. If you can make riskless profit with zero investment, you’ve either discovered a time machine or mis-coded your backtest. In real markets, true arbitrage opportunities vanish faster than free samosas in an academic seminar.

Then there is Market Efficiency. This grand theory says prices reflect all available information. Whether you are Warren Buffett, a PhD in stochastic calculus, or your neighbour’s cousin dabbling on his phone, you can not consistently outsmart the market. Nice idea, but then you meet reality: high-frequency traders whizzing around with algorithms faster than you can say “margin call.”

Between the textbook no-arbitrage world and the messy reality lives Statistical Arbitrage, where traders earn money by exploiting tiny, temporary mispricings. It’s not risk-free, but it’s profitable if you’re fast, rich, and have more servers than sense.

Then there are market inefficiencies. Occasional inefficiencies are fine, like traffic jams; they annoy but don’t kill the system. But what happens if inefficiency becomes systematic? Imagine a casino where the roulette wheel is rigged to always favour the house (which is always the case). Eventually, the gamblers wise up, stop playing, and the casino either shuts down or gets raided by regulators.

That, dear reader, is roughly what happened with the Bank Nifty weekly options.

Retail traders thought they were the casino. Selling out-of-the-money straddles for quick “theta income” felt like good rent money. In reality, they were shorting gamma into expiry, and every time Bank Nifty sneezed, their premiums evaporated faster than your New Year’s resolutions. On the other side? Hedge funds, HFTs, and multinational trading desks; calmly scalping gamma, hedging like machines, and ringing the profit bell week after week.

This wasn’t a market of equals. It was a systematic wealth-transfer mechanism. One side (retail) consistently lost. The other side (institutions) consistently won. That’s not market efficiency; that’s a one-way conveyor belt of money.

And here is the punchline: when inefficiency is systematic, the market ceases to exist. It has to. Either the losing side voluntarily withdraws (“thanks, but no thanks”), or the regulator shows up with a cricket bat.

Enter SEBI. With its investor-protection mandate, SEBI looked at the Bank Nifty weeklies and said:

This isn’t a market, this is a slaughterhouse.

In November 2024, the regulator pulled the plug. No more Bank Nifty weekly expiries. Only Nifty weeklys survived, and even they are now tightly restricted.

So, next time someone lectures you about “market efficiency,” remind them: efficiency is not just about no-arbitrage and martingales. It’s also about survival. A market that systematically bleeds one side dry is not efficient. It’s not even sustainable.

Perhaps what we really need is not just an efficient market, but a sustainable market, one where both sides live to trade another day, rather than one side being fed to the Bakasura every week. Because in the end, such a market doesn’t just become inefficient,
It ceases to exist.

Share Random Walk

Leave a comment

Dear Data Science Students,

You think your biggest fear is your model not converging. Wrong. Big Mistake.

Your real fear should be the government giving you free food, free housing, and free Wi-Fi (actually, no Wi-Fi) in a high-security hostel with iron bars.

Yes, a data scientist can face jail time. Here is how.

Privacy Violations: Scraping Your Way to Prison

You saw a shiny dataset behind a login wall and thought,

If I can code 
requests.get(), 
it’s public.

No. That’s like saying if you can climb into someone’s house, the fridge is yours. You can eat the rotten burger, but if the owner calls the police, you are going to jail.

Real case vibes: Cambridge Analytica. People went to court for treating Facebook like a candy store. In the EU, GDPR violations can land you in jail. In India, the new DPDP Act (Digital Personal Data Protection Act) has teeth, too.

So before you scrape 10,000 patient records for your “fun side project,” ask yourself: Do you want your side project to also be your side hustle in jail?

Insider Trading: LSTMs Meet the SEC

You’re working at a hedge fund, building a cool model on company earnings data. One day, your boss drops a “confidential” file on your desk. Your LSTM screams BUY, you scream YOLO, and next thing you know… the SEC screams “see you in court.”

Judges don’t like your time-series plots. They don’t care that your Sharpe ratio was amazing. They only care that you used information you weren’t supposed to.

And no, explaining “but my model was just doing stochastic gradient descent” will not reduce your sentence.

Fraud by Algorithm: When Models Cheat Better than You

Volkswagen engineers built an algorithm that could detect when the car was being tested for emissions and fake the results. Brilliant engineering. Also, criminal fraud.

Data scientists have power: you can manipulate credit scoring, recommendation engines, and even medical predictions. But if you knowingly ship an algorithm that lies, you’re not an ML engineer anymore, you are a co-conspirator.

Remember: prosecutors love Jupyter notebooks. They’ll project your #temporary_hack comment in court at size 48 font.

Health & Safety: When Your Model Kills People

Let’s say you’re building a hospital triage AI. You cut corners because of deadlines, accuracy drops, and suddenly, the AI tells a patient with a heart attack to drink more water. The patient dies. Now you are not facing a Kaggle leaderboard; you are facing manslaughter charges.

This is where “dharma” stops being philosophy and “karma” starts being criminal law. Judges won’t ask about precision and recall. They’ll ask why you deployed a model you knew wasn’t reliable.

The Takeaway

Data science is fun. Jail is not. “Dharma” is not an elective course. It is the invisible line between you and a very bad Netflix docu-series about your life.

So before you hit run, ask yourself:

“Would I be comfortable explaining this to a judge who doesn’t know Python, but knows the law (and has the power to take away my laptop forever)?”

If the answer is no, close the notebook. Trust me, prison Wi-Fi won’t let you pip install anything.

Random Walk - finding the method in the madness, and making sure that madness doesn’t involve an arrest warrant.

Share Random Walk

Leave a comment

Data Science is basically a detective show. Correlation is the charming suspect with perfect hair; causation is the villain we must catch in the last act. Causal inference is our forensics kit, randomised control trials, natural experiments, difference-in-differences, instrumental variables, used to answer the only question that really matters: what would have happened otherwise?

Below are two fan-favourite episodes where causal inference solved the case.

The Burger Border Experiment: Minimum Wage vs Jobs

In 1992 New Jersey raised its minimum wage while neighbouring Pennsylvania did not. Two economists, David Card and Alan Krueger, did something disarmingly simple: they compared fast-food employment on the NJ side with the PA side, before and after the change—same burgers, different law, tidy counterfactual. Contrary to textbook fear, jobs in New Jersey did not fall relative to Pennsylvania; if anything, they nudged up. This did not end the minimum-wage debate, but it did force economists to bin easy slogans and measure carefully. (davidcard.berkeley.edu, JSTOR, American Economic Association)

Why this is a causal relation: The policy changed in one place and not the other, providing a “treatment” and a “control”. The method, difference-in-differences, lets us net out common shocks (like a national burger fad). Result: a credible estimate of the policy’s effect, not just vibes.

Moving to Opportunity: Neighbourhoods Really Matter

In the 1990s, the Moving to Opportunity programme randomly offered housing vouchers to families in high-poverty areas, letting some move to lower-poverty neighbourhoods. Years later, tax and education data revealed a striking pattern: children who moved before age 13 earned more as adults and were more likely to attend university; older teens didn’t benefit and sometimes lost ground (teenage disruption is a thing). The causal arrow points squarely from place to life chances. (American Economic Association, scholar.harvard.edu, NBER, Opportunity Insights)

Why this is a causal relation: The voucher lotteries created treatment and control groups by chance, gold-standard randomisation, so differences in later outcomes can be attributed to neighbourhood exposure rather than pre-existing family traits.

What these stories teach

• Design beats debate. Arguments shrink when you can point to a clean counterfactual.

• Context matters. Causal answers are local: fast-food in NJ in 1992 is not retail in 2025; moving young is not moving at 17. Update priors, don’t worship them.

• Data grows up. Long-run administrative data turned a temporary housing policy into a lesson about intergenerational mobility.
  

A final open question

If raising wages doesn’t automatically kill jobs, and moving neighbourhoods can lift lifetime earnings, then here is today’s cliff-hanger: Does imposing a tariff make a country great again?

That’s not a slogan; it’s a research design problem. We’d need quasi-random tariff shocks (say, staggered trade rules or sudden exemptions), a credible control group, and outcomes that go beyond GDP, productivity, innovation, real wages, prices, and even diplomatic blowback. Until we can sketch that counterfactual, “tariffs = greatness” is just a catchy plot twist waiting for its data.

References:

[1] Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania, By David Card and Alan B. Krueger The American Economic Review Vol. 84, No. 4 (Sep., 1994), pp. 772-793 (22 pages); Published By: American Economic Association

[2] The Effects of Exposure to Better Neighborhoods on Children: New Evidence from the Moving to Opportunity Experiment, By Raj Chetty, Nathaniel Hendren, Lawrence F. Katz, American Economic Review vol. 106, no. 4, April 2016 (pp. 855–902)

Share Random Walk

Leave a comment

Act I: The Dot-Com Bubble (2000): The Great Internet Party

Once upon a time, in a land called Silicon Valley, everyone discovered this magical thing called the internet. Suddenly, every company had to add “.com” to its name; it was like putting “extra cheese” on a pizza. Investors gobbled it up.

Never mind that these “businesses” had no profits, no customers, and sometimes not even a business plan. Who needs revenue when you have a URL?

It ended badly, of course. The Nasdaq fell faster than my New Year’s resolution to go to the gym. Out of the rubble crawled a few champions, Amazon, Google, eBay, while the rest vanished like the pop-ups they once sold.

Act II: The Subprime Crisis (2008): When Houses Became ATMs

Fast forward a few years. The new motto was: “Everyone deserves a home.” Noble idea; except banks started giving loans to anyone who could sign their name (and sometimes to people who couldn’t).

These loans, called subprime mortgages, were bundled into shiny packages and sold around the world as “safe” investments. (That’s like wrapping rotten chicken butter masala in gold foil and calling it gourmet.)

When house prices stopped rising, the bubble popped. Lehman Brothers went belly-up, Wall Street panicked, and the world economy caught pneumonia.

Act III: The AI + Debt Bubble (2025?): Chatbots and Credit Cards

Now we arrive at today. The new darling is Artificial Intelligence. Start-ups are raising billions to build chatbots that can generate recipes, code, or poetry (though not always in the right order). Nvidia is the new Amazon; OpenAI is the new Pets.com; we just don’t know which is which yet.

Meanwhile, the US government is running the world’s biggest tab. Uncle Sam has a $34+ trillion credit card bill, and the minimum payment isn’t looking pretty. Treasury bonds are supposed to be the safest asset on Earth, just like subprime mortgages were in 2007. (Nervous laughter, please.)

Act IV: The Olive Branch or the Tariff Club

Here’s the twist. If the US decides to cooperate with the rest of the world (instead of slapping tariffs like they are hotcakes), chances are it will be offered an olive branch. And by olive branch, I mean the world will politely let America keep printing the greenback to pay its bills. Maintaining the status quo is preferred by many middle power countries, like India. It will allow America to maintain its lifestyle.

That would mean:

• No sudden market crash.

• But a long, slow time correction.

• High global inflation as the world keeps swallowing US dollars.

• A generation (maybe two) watching their wealth erode quietly, like a sandcastle under the tide.

The Alternative Ending: Gold and Trump Coin

But if President Trump keeps swinging the tariff hammer (and it really does look like we’re approaching the point of no return), things get spicy. Why should India, China, Brazil, South Africa, and friends keep holding US Treasuries if they’re getting whacked in trade? The rational decision is to dump US Treasuries, sell dollars, and move into gold.

Problem: You can’t exactly pay for container ships of semiconductors with gold bars. Enter the possible gold-backed stablecoin, issued by central banks. Imagine an alternative financial system where trade is settled in digital tokens backed by sovereign gold. That could be the biggest challenge yet to the USD’s reign.

And here’s the kicker: rumours swirl that the Trump family is investing heavily in Pakistan’s crypto scene. As “Trump Digital Coin” appears, don’t laugh; he might actually be dreaming of replacing the US dollar with his own brand of digital money. For that, Trump has to make sure nobody touches the USD. (Somewhere, Satoshi Nakamoto is either facepalming or laughing hysterically.)

The Rhymes of History

• 2000: Dot-com hype said, “The Internet changes everything.”

• 2008: Housing hype said, “Home prices never fall.”

• 2025?: AI hype says, “Generative AI will print money forever,” while debt markets whisper, “Don’t worry, Treasuries are always safe.”

Spoiler: hype always ages badly.

So What Happens Next?

Maybe nothing dramatic. Maybe the AI revolution really does transform productivity, and maybe Uncle Sam finds a way to refinance his debt forever. Or maybe, just maybe, we’re in Act III of the same old play: where the audience knows how it ends but still buys popcorn.

Until then, enjoy the show. History is rhyming again, and this time the punchline might not be funny at all.

Share Random Walk

Leave a comment

One of the oddest experiences I have had in corporate AI demos is watching someone confidently announce that their model was “fully interpretable, completely explainable, and absolutely identifiable.” It sounded like the menu description of an artisanal coffee. People nodded. I tried not to laugh.

These words, interpretability, explainability, and identifiability, sound like they belong together. They don’t. In fact, identifiability is often the neglected poor cousin at the dinner table. We rarely invite it to the conversation unless we are doing something “fancy” like causal inference or latent variable models. But it has been with us all along, hiding in plain sight, even in the simplest models.

Imagine you are running a one-way ANOVA. Three schools; A, B, and C, have all just finished an exam, and you, the statistician, are tasked with comparing their average scores. You set up the textbook model:

HereYij is the score of the j-th student in school i; μ is the grand mean, and αi is the effect of school i. The εij is the error term, which in theory represents random noise but in practice is everything from bad handwriting to the neighbour’s dog barking during study time.

You estimate the model and find μ = 70, α_A = 5 , α_B = −2, and α_C = −3. School A is clearly ahead, School B is a bit behind, School C is trailing slightly more.

Now comes the trick. Suppose I secretly add 1 to every α_i, making them 6,−1,−2, and then subtract 1 from μ, giving me μ = 69. The predictions for every student’s score remain exactly the same. In fact, you can add any constant to all the α_i’s and adjust μ in the opposite direction, and nothing changes. You have different parameter values but the same fitted model.

The conclusion? The parameters in this setup are not uniquely determined by the data. In technical terms, the model is not identifiable until we impose a constraint — something like “set α_C = 0” or “the sum of the α_i’s is zero.” Only then do the numbers stop playing musical chairs.

This is identifiability, stripped of glamour and acronyms. It is simply about whether your model’s parameters are pinned down uniquely by your data and assumptions. And here’s the kicker: if they’re not, you can still predict just fine. That’s why it can sneak past you in everyday work. Predictions don’t complain. Only parameter interpretations do.

If you’ve ever looked at regression coefficients and thought, “Oh, this one means X changes by two points when Y goes up by one unit,” you have implicitly assumed identifiability. If that assumption fails, the coefficient is more like a cardboard prop: it looks real until you try to lean on it.

This simple ANOVA trick teaches a deeper lesson. Identifiability is not just a concern for complex models with Greek letters spilling off the page. It’s part of the basic housekeeping of statistical modelling. In more complicated settings, mixture models, factor analysis, causal graphs, the same principle applies, but the furniture is harder to rearrange, so you may not notice the missing screws until the whole thing wobbles.

So next time someone tells you their model is interpretable, explainable, and identifiable, smile politely. Then ask them, “Identifiable how?” If they can answer without resorting to magic words or vague hand gestures, you may have found a rare species: the statistician who checks identifiability before printing coefficients.

And if they can’t, well, you now know what to say. Just tell them you learned it from a one-way ANOVA.

The thing our first statistics class didn’t tell us, but should have.

Leave a comment

It’s not as absurd a question as it sounds.

As teachers, have we ever asked ourselves: Is there a student in my class who is actually smarter than I am? If we’re honest, the answer could be “yes.” And that’s exactly why I feel uneasy when I hear colleagues laughing at a student’s mistakes in private.

Mistakes are not weaknesses; they are divine. They’re the raw material of learning. And sometimes, what we think is a “mistake” is simply the sign of a mind travelling a different, deeper path than we anticipated. What if the student saw a problem in a way that never even crossed our minds?

I’ve seen this happen often: a genius-level student ends up with an average score. Not because they didn’t understand, but because their brain was too quick, too divergent, or simply uninterested in playing by the exam’s narrow rules.

The Clash Between Genius and the System

I have seen many teachers penalise the students in exams if they do not write the answers in a certain way. The really smart students often think so fast and so diversely that teachers do not even realise that maybe he/she is speaking to a young Einstein. The bloated ego of the teacher and the power invested in them often makes them blind.

But if the teachers indulge these students a bit often, they will see that these weird students could connect ideas across disciplines in ways they did not expect, e.g., bringing relativity into a probability lecture, which may end up in the area of quantum probability, which is not only beyond the syllabus, but it is a completely different land. Their questions might start with “What if…?” and lead you down fascinating but unplanned tangents.

Instead of only teaching, you’d often be learning alongside them. A simple discussion might turn into collaborative research-level brainstorming sessions. They could inspire you to update or expand the curriculum to keep pace with their curiosity.

These students might spot subtle errors or oversights in standard explanations in a regular textbook. They could propose alternate proofs, models, or interpretations; sometimes more elegant than the ones in the syllabus or in the textbook. Their “why does it have to be that way?” could push you to rethink concepts you’ve taught the same way for years.

If nurtured well, they could go on to make groundbreaking contributions, and you’d have the joy of saying, “I taught them.” They might inspire a culture of deeper thinking in your class. Your own teaching could evolve into something far more creative and exploratory.

Why Standardisation Hurts the Smartest

The Indian education system just relies too much on rote memorisation. Our way of teaching is simply obsolete. It appreciates mediocrity and penalises the people who are different in the name of standardisation.

We have to realise that standardisation helps most of the students, particularly the average and above-average students. But standardisation often ends up penalising the smart students just because they are smart.

What Needs to Change

Einstein famously struggled in some subjects that didn’t interest him at school. Gifted students often pour energy into topics they love and neglect others that feel irrelevant. This creates uneven performance; brilliant in some areas, average or worse in others. Our Indian exams test memory, speed, and formatting skills; not deep reasoning or creative problem-solving. A high-IQ student might think slowly but deeply, which isn’t rewarded in a “finish in 3 hours” test. They might find multiple-choice questions too restrictive and get tripped up by careless mistakes.

Einstein himself was a late talker, disliked rote learning, and clashed with rigid school systems. He did well in subjects he cared about, but was not the top student in every class, and some teachers even considered him a troublemaker. The time has come for our test exam should test deep reasoning or creative problem-solving skills and not rote memorisation and speed.

Share Random Walk

Leave a comment

Dear Young Statistician and Data Scientists:

Hope this letter finds you in good health. I know you have recently learned Statistical Inference, Time Series Models, Machine Learning, Deep Learning, LSTM, ARIMA and whatnot. You have picked up Python and R. Some of you have even explored Julia. Now you are excited and all charged up to solve the greatest and toughest problem of all: predicting stock prices.

Please do not try to predict stock prices using daily closing data from Yahoo Finance or other open sources.

I know, I sound odd. But hear me out.

As a statistician, it is my duty to remind you that before you try to predict any time series, the first thing you must check is whether it follows a Random Walk. And that is where the Augmented Dickey-Fuller (ADF) test comes in.

This is an interesting test. It assumes the null hypothesis that the series follows a random walk (i.e., it has a unit root). The alternative is that the series is stationary. If the p-value is high, you fail to reject the null, which means the price series likely follows a random walk.

Go ahead, try it.

Take all the Nifty 50 stocks from the Indian market, or the S&P 500 stocks from the US. Use a couple of months’ worth of daily closing prices. Run the ADF test. I am confident you’ll fail to reject the null hypothesis in every case. Which means: the prices behave like random walks.

So what does this mean?

It means there are no arbitrage opportunities. No trend. Markets are efficient.

There is even a theory called the Efficient Market Hypothesis (EMH). You might want to spend five minutes on its Wikipedia page before throwing your Deep LSTM or ARIMA model at the data.

Now, I know what you might say: “A random walk is just white noise, right?”

I am sorry, but a random walk is not white noise. They are entirely different concepts.

You might say: “But there’s mean reversion.”

Again, I am sorry. A random walk does not revert to the mean.

You may argue: “My deep LSTM model will find a pattern.”

Well, by the very definition of a random walk, there is no pattern to find.

At this point, you may be thinking, maybe there is something to this.

Otherwise, every Amar, Akbar, and Anthony on the street would be running Python code and becoming millionaires overnight. But that is not happening.

Maybe, just maybe, you should study a bit of Market Economics and Quantitative Finance before running your Python code.

And if you are serious, you may wish to study the Fundamental Theorem of Asset Pricing. It says If there is no arbitrage, there must be a risk-neutral probability measure under which asset prices (adjusted for interest rates) behave like martingales. One can say all random walks are martingales (under suitable conditions). But the point is: these are not predictable processes. They are designed that way.

So what can you do?

In case you are scratching your head and if you are thinking what you should do, well, there are effectively two paths you can take, which I think may be meaningful.

Two serious directions:

1. High-Frequency Trading (HFT):

Here, predictability does exist. In fact, many HFT processes exhibit fractional geometric Brownian motion, where returns are correlated. But this is a different world. You’ll need to learn about co-location facilities, market-making algorithms, and how to shave off microseconds from your execution time.

2. Predicting Volatility (not Prices):

If you want to work with daily data from Yahoo, try predicting volatility instead of price. Because volatility is predictable. That is why Econometricians have developed GARCH models and the Stochastic Volatility Model. Please read these models.

If you can predict the movement of volatility, then you can devise Option strategies which basically trade volatility in the market, and one can make fairly good money from the stock market, because it is not gambling, it is a skill, a real skill.

But you have to really learn option pricing, option Greeks, option risk management, fire fighting when your option trade goes wrong, and above all, you should have a good amount of money to put as margin.

Now, if you still want to predict prices because you believe in your machine learning model, please go ahead. Just one request:

Do not invest your money based on that model.

Because you’ll likely lose it all. And please, do not announce it on social media.
Because the moment you do, all the experts in the investment bank would immediately know not to hire whom and who is the new rookie in the block.

Happy predicting the next step of the random walk!

Warm regards,
A fellow statistician who has already lost money trying to predict the market.

Leave a comment

A few years ago, I sat through a corporate AI showcase where a senior executive boasted that their latest fraud detection model was “100% interpretable, 95% explainable, and absolutely identifiable.” I remember scribbling those words on a piece of paper, staring at them like a puzzle. What did he really mean? Did the model reveal its internal logic like a mathematical proof? Did it allow post-hoc narratives for each decision? Could we actually recover the structure of the fraud process from the data alone?

These three words, (1) interpretability, (2) explainability, (3) identifiability, have become part of the everyday vocabulary of data science. Yet they are often used interchangeably, mashed together in presentations, model cards, grant proposals, and AI ethics documents. In reality, they refer to three very different ideas. The confusion is not innocent; it has consequences.

Interpretability

Let us begin with the simplest: interpretability. This is the property of a model that allows a human to understand how it works internally. If I give you a linear regression with three variables, you can read off the coefficients and immediately tell which way the outcome moves when one input increases. A decision tree, carefully pruned, can also be interpretable, you can trace the logic and say, “if the income is above this and age is below that, the loan is approved.” Interpretability lives in the realm of transparency. It is about seeing the skeleton of the model.

Explainability

Now contrast this with explainability. A model may be utterly uninterpretable yet still allow us to explain its predictions. Take a complex deep learning model that classifies images of skin lesions into benign and malignant. The model’s internal parameters may be a mess of convolutions and activations, but we can still use a technique like SHAP or LIME to say, “this part of the image contributed positively to the malignancy prediction.” We explain the output, even if we do not understand the internal machinery. This is explainability: the post-hoc ability to justify a prediction in human-understandable terms.

Identifiability

But then comes the tricky one: identifiability. This is neither about humans understanding the model nor about explaining the predictions. Identifiability is a property of the statistical model and the data-generating process. It answers a subtle but profound question: if I had infinite data, could I recover the true parameter values uniquely? Not approximately. Not probably. Uniquely.

Consider the classic example of a finite mixture model. Suppose you are modelling test scores as coming from a mixture of two normal distributions. One for high performers and one for low performers. Even if you fit this model and get perfect predictions, the underlying parameters may not be identifiable. The same mixture distribution can often be generated by multiple combinations of parameters. You cannot tell which is the "real" one. In that case, the model is not identifiable.

Identifiability in causal inference

In causal inference, identifiability becomes even more crucial. The famous backdoor criterion in Pearl's causal framework is a set of conditions that allows the causal effect to be identified from observational data. If those conditions are not met, say, due to unobserved confounding, then the causal effect is simply not identifiable. No amount of data, no clever machine learning trick, no brute-force computation will help.

This distinction was made precise in works such as Paul W. Holland’s foundational paper “Statistics and Causal Inference” (1986), and more recently in Judea Pearl’s “Causality” (2009). In these writings, identifiability is a question of logic and algebra, whether a quantity is formally derivable from the available data and assumptions.

When HR forces you to take time off

The danger is that people often assume that if a model gives good predictions and can be explained, then it must also be meaningful or even causally valid. That is a fallacy. Let me share an anecdote. A startup once used an XGBoost model to predict employee attrition. They found that employees who didn’t take vacations were more likely to leave. The SHAP plots showed vacation days as a top feature. HR started forcing people to take time off. Attrition did not change. Why? Because vacation was a proxy, not a cause. The model was explainable. It was not interpretable in terms of mechanism. And the causal link was unidentifiable.

This is not just an academic nitpicking. In domains like healthcare, criminal justice, finance, and climate science, these distinctions have consequences. If you deploy a model in the ICU to predict mortality risk, interpretability helps doctors understand the logic, explainability helps justify a decision to families, and identifiability ensures you are modelling the true physiological process, and not a statistical artefact.

A recent paper by Rudin et al., titled “Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead” (Nature Machine Intelligence, 2019), makes a strong case for interpretable models in critical settings. The authors argue that explanations are often fragile and misleading. You do not want a justification; you want an understanding. But this does not solve the identifiability problem. Even interpretable models can be unidentifiable.

And sometimes, the reverse is true. Consider certain Bayesian models. They may be unidentifiable in a frequentist sense; there may be multiple modes or flat ridges in the likelihood. But when combined with informative priors, we can still do sensible inference. This opens up deeper philosophical questions about what we mean by “knowing” something in the presence of uncertainty.

So where does that leave us?

Interpretability is a property of the model’s form. Explainability is a property of the model’s behaviour. Identifiability is a property of the statistical structure.

You can have models that are interpretable and identifiable but not explainable, like a complex structural equation model. You can have models that are explainable but not interpretable or identifiable, like a random forest with SHAP plots. And you can have models that are neither interpretable, nor explainable, nor identifiable, like some deep generative models trained on unlabelled data with ill-specified assumptions.

When someone asks, “Can we trust this model?”, the only honest answer is, “Trust it for what?” Prediction? Explanation? Insight? Intervention?

Let us be precise in what we demand, and careful in what we promise. Not everything that looks good on a dashboard is grounded in statistical reality. And not everything that is identifiable is necessarily interpretable.

We live in a world of increasing complexity. Precision in language is our first defence against confusion. And clarity in thinking is our only defence against misuse.

Reference:

[1] Statistics and Causal Inference Paul W. Holland Journal of the American Statistical Association, Vol. 81, No. 396 (Dec., 1986), pp. 945-960

[2] Causality By Judea Pearl (2009) https://projects.illc.uva.nl/cil/uploaded_files/inlineitem/Pearl_2009_Causality.pdf

[3] Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, by Cynthia Rudin, Nature Machine Intelligence , 1, 206-215 (2019) [https://www.nature.com/articles/s42256-019-0048-x]

Leave a comment

We tend to think of severe thunderstorms as isolated, dramatic events; hailstones smashing windshields, tornadoes carving paths through farmland, gusts toppling powerlines. But there’s another dimension to these storms that we rarely talk about: their spatial footprint. How wide an area does a severe storm environment cover when it strikes? And how does this footprint evolve over time, especially under the influence of global climate patterns like El Niño and La Niña?

A fascinating recent paper by Jonathan Koh, Erwan Koch, and Anthony Davison, published in the Journal of the American Statistical Association, tackles exactly these questions with a combination of atmospheric science and cutting-edge statistics.

Why it matters

In North America, severe thunderstorms are among the most damaging weather hazards. But counting individual tornadoes or hail reports is tricky. Those records are biased and inconsistent. Instead, researchers often look at the environments that make such storms possible:

• CAPE (Convective Available Potential Energy); how unstable the atmosphere is.

• SRH (Storm Relative Helicity); how much wind shear there is in the lower atmosphere.

• And a combined metric called PROD, which reflects both instability and shear.

These variables tell us where and when the atmosphere is primed for severe weather and, crucially, how big an area is affected when it “goes extreme.”

What they did

The authors set out to model the space-time extremes of these environmental variables over the U.S. from 1979-2015, focusing on Tornado Alley and surrounding regions. But they went a step further than most prior work. Instead of assuming extremes behave the same everywhere and every month, they allowed the spatial footprint of extremes to vary over time and with ENSO (El Niño/La Niña).

They built a flexible statistical model, based on something called a Brown-Resnick max-stable field, where the spatial range of extremes depends on:

• The month of the year (to capture seasonality)

• and the phase of ENSO

They validated their model on historical reanalysis data at 1°×1° resolution, and even developed new statistical tools for testing and model selection.

For serious minds: a peek at the math

For readers who enjoy the math behind the models, here are two key equations from the paper.

1. The PROD variable

The researchers use this composite metric as a proxy for severe thunderstorm conditions:

where:

• CAPE measures atmospheric instability.

• SRH measures vertical wind shear.

A high PROD means the atmosphere is primed for severe thunderstorms.

2. The spatial dependence model

One of the paper’s main contributions is a statistical model that allows the spatial extent of extremes to vary over time and with ENSO phase. Specifically, the spatial semivariogram (which governs how dependence decays over distance) is modelled as:

where:

• t is the month.

• A encodes anisotropy (directional effects).

• α controls the smoothness of the field. The sum in the exponential lets the range parameter flex with ENSO and seasonality, via spline basis functions

  In plain English, this equation describes how the “footprint” of severe environments breathes in and out as the seasons change and ENSO fluctuates.

What they found

Summer storms are more localised. In summer, extremes tend to cover smaller areas, reflecting weaker jet stream forcing and stronger local processes. ENSO modulates the footprint. During El Niño and La Niña events, the spatial extent of extremes increased in some regions, especially in late winter and spring.

• In the Northeast, severe environments were larger during both ENSO phases.

• CAPE and SRH showed similar regional patterns, varying seasonally.

These patterns make meteorological sense: ENSO alters the position of the subtropical jet stream, which influences wind shear and storm organisation.

Why it’s cool

Better risk assessment. Emergency planners care not just about how intense storms get, but how widespread they are. Better statistical tools. The authors developed new methods to test assumptions and select models, tools that apply to other spatial extremes problems. Climate insight. Understanding how ENSO affects not just the intensity but also the footprint of severe weather improves seasonal forecasts and climate-risk models.

My take

This paper is a great example of interdisciplinary science done right. It is rooted in meteorology but leverages advanced statistics to answer a question that is both scientifically interesting and practically important: when the atmosphere goes extreme, how far does it reach? The findings also carry an implicit warning: as climate variability and change alter patterns like ENSO, the maps of severe weather risk may shift in subtle but significant ways. That’s a challenge for researchers, and a call to action for planners.

If you want to dig deeper

Paper: Space-Time Extremes of Severe U.S. Thunderstorm Environments
Authors: Jonathan Koh, Erwan Koch & Anthony C. Davison
Journal of the American Statistical Association (2025), Volume 120, 2025 - Issue 550, Pages 591-604

Your thoughts?

Have you ever experienced a severe thunderstorm that seemed to stretch endlessly? Do you think about the footprint of extreme weather where you live? Share your stories and thoughts in the comments.

Written for Random Walk, where I explore the hidden patterns behind random events like a big thunderstorm.

Leave a comment

Get more from Sourish in the Substack app

Available for iOS and Android

Get the app

Refer a friend

Many papers proudly report a p-value below 0.05, signalling “significance” in the traditional sense. This threshold has become deeply entrenched in scientific practice. But the more closely we examine p-values, the more their limitations become apparent. They measure the probability of data as extreme as what we observed, assuming the null hypothesis is true, but they tell us nothing about how likely the data is under an alternative model. Worse, they are fragile when data collection is sequential or when the researcher stops the experiment early. In short, p-values do not behave well when used in the way modern science often works.

More troubling is that p-values do not behave well when we collect data sequentially, peek at intermediate results, or stop early. These practices are common in real-world research, including both academia and the corporate world. Under such conditions, p-values lose their advertised error guarantees and can mislead researchers about the strength of evidence.

An alternative approach has been gaining traction. The e-value offers a more direct and interpretable measure of evidence, one that aligns naturally with Bayesian reasoning while maintaining frequentist safeguards. To understand why it works, it helps to see it through two lenses: Bayesian updating and martingale behaviour.

An e-value is defined as the ratio of the likelihood of the observed data under the alternative hypothesis to that under the null:

where P0(x) is the likelihood under the null model and P1(x) is the likelihood under the alternative. Unlike the p-value, the e-value is not a probability but a measure of how much more likely the data is under the alternative than under the null. If E(x) = 50, we can say the data is fifty times more likely under the alternative than under the null, a clear, quantitative statement of evidence.

The Bayesian connection becomes apparent when we express P_1(x) as a marginal likelihood under a

The e-value then becomes:

This is exactly the Bayes factor when prior odds are set to one. In this sense, the e-value behaves like a Bayes factor with neutral prior odds, making it appealing even to researchers who prefer to avoid subjective prior probabilities on the hypotheses themselves. It provides a Bayesian-style update without requiring a commitment to prior beliefs about which hypothesis is true.

The e-value also has an important property under the null: its expected value equals one.

This ensures that, on average, the e-value does not falsely accumulate evidence against the null if the null is actually true.

The power of the e-value becomes even more apparent in sequential experiments, where data is collected over time and researchers may decide to stop early. Suppose we observe a sequence of independent outcomes

and compute the running product of likelihood ratios:

The above sequence En forms a martingale under the null hypothesis. That is, the expected value of E{n+1} given the current data equals En. This martingale behaviour protects against Type-I error inflation, even under optional stopping, providing a safeguard that p-values lack.

A simple example helps illustrate these ideas. Imagine repeatedly tossing a coin to test whether it is fair (the null hypothesis) or biased towards heads with probability 0.75 (the alternative). Each head gives a likelihood ratio of 0.75 / 0.5 = 1.5, and each tail gives 0.25 / 0.5 = 0.5. The e-value after n tosses is the product of these ratios. If the coin is fair, the expected e-value stays around one, regardless of when we stop. Conversely, if the coin is biased, the expected e-value grows exponentially with the number of tosses, providing accumulating evidence against the null.

This behaviour makes the e-value far better suited to modern research practice, where data is rarely collected all at once and researchers often monitor results as they emerge. It quantifies evidence on a meaningful scale, allows for flexible stopping rules, and speaks to both Bayesian and frequentist concerns.

Perhaps it is time to complement or even replace p-values with an approach to evidence that is more intuitive, flexible, and honest, the e-value.

Referances

[1] Vladimir Vovk and Ruodu Wang. E-values: Calibration, combination, and applications. The Annals of Statistics, 49:1736–1754, 2021.

[2] Larry Wasserman, Aaditya Ramdas, and Sivaraman Balakrishnan. Universal inference. Proceedings of the National Academy of Sciences, 117(29):16880–16890, 2020.

[3] Ian Waudby-Smith and Aaditya Ramdas. Estimating means of bounded random variables by betting. Journal of the Royal Statistical Society Series B: Statistical Methodology, 86(1):1–27, 2024. With Discussion.

Understanding E-Values using Julia:

using Random
using Plots


# Parameters
n_tosses = 1000
p_null = 0.5      # under null: fair coin
p_alt = 0.75      # under alternative: biased towards heads


# Likelihood ratios
lr_head = p_alt / p_null   # 1.5
lr_tail = (1 - p_alt) / (1 - p_null)   # 0.5


# Function to simulate one experiment
function simulate_evalue(n_tosses, p_coin)
   e_values = ones(n_tosses)
   e_current = 1.0
   for i in 1:n_tosses
       toss = rand() < p_coin ? "H" : "T"
       if toss == "H"
           e_current *= lr_head
       else
           e_current *= lr_tail
       end
       e_values[i] = e_current
   end
   return e_values
end


# Simulate
Random.seed!(321)  # for reproducibility
e_null = simulate_evalue(n_tosses, 0.5)    # fair coin
e_alt  = simulate_evalue(n_tosses, 0.75)   # biased coin


# Plot
plot(1:n_tosses, e_null, label="Fair coin (null)", lw=2, xlabel="Number of Tosses", ylabel="E-value", legend=:topleft, title="E-value Evolution in Coin Tossing", yscale=:log10)
plot!(1:n_tosses, e_alt, label="Biased coin (alternative)", lw=2)

Leave a comment

In the world of statistical inference, the p-value has long occupied centre stage. It is printed in journal articles, determines whether a result is deemed significant, and governs many of the choices researchers make when designing experiments. Yet for all its ubiquity, the p-value is increasingly under scrutiny. Not because it is incorrect, but because of how it is used, and misused.

Imagine a researcher who runs an online A/B test. The sample size is not fixed in advance. She checks the results every hour, hoping for a statistically significant effect. After 14 hours, the p-value dips just below 0.05. Delighted, she stops the test and reports the finding. It may sound innocent, but this practice, known as optional stopping, invalidates the interpretation of the p-value. It assumes a fixed sample size. By peeking, the researcher has actually inflated her chance of having a false positive.

Now consider a team analysing 40 variables, testing each one separately, and reporting only the ones where the p-value dips below 0.05. This is a form of p-hacking. The original assumptions that make p-values valid no longer hold when such data-driven decision rules are applied. What we get is a mirage of significance, not real evidence.

The criticism here is not of the mathematics behind the p-value. Rather, it is the mismatch between its assumptions and the realities of modern data analysis. In practice, we explore, we adapt, and we iterate. The p-value, as a tool, is not robust to this flexibility. The p-value works well for a fixed sample size problem. This is where the E-value enters the picture.

Unlike the p-value, which measures the probability of observing data as extreme as the one seen under the null hypothesis, the E-value quantifies the strength of evidence against the null hypothesis, and it remains valid under optional stopping.

Formally, an E-value is defined through an object called an E-variable. This is a nonnegative test statistic E satisfying:

That’s it. If this condition holds, then the observed value of E, the E-value, can be interpreted as a measure of evidence against the null. Crucially, this property is preserved even if one continues collecting data and recomputing E along the way. This is in stark contrast to the p-value, which assumes the sample size and stopping rule are fixed in advance.

To see the E-value in action, let’s visit our textbook example of testing a coin for bias.

Suppose we flip a coin 20 times and observe 15 heads. Is this enough evidence to reject the hypothesis that the coin is fair?

Under the null hypothesis that the coin is fair (p = 0.5), the probability of observing 15 or more heads out of 20 is approximately 2.07%. That gives us a one-sided p-value of 0.0207. Since this is below the 0.05 threshold, we would typically reject the null.

But now let’s compute the E-value. The logic is different. We compare how likely this observation is under the null versus the best possible alternative. That is, we compute the likelihood of seeing 15 heads under the null (with p = 0.5) and compare it to the maximum likelihood under the alternative, which occurs at the empirical success rate

Under the null:

Under the alternative:

The E-value is then the ratio:

What does this number mean? It means the observed data are 5.73 times more likely under the best-fitting alternative hypothesis than under the null hypothesis. That’s meaningful, but it falls short of the typical rejection threshold for E-values, which is

So while the p-value says “reject,” the E-value says “not enough evidence yet.”

This difference becomes even more important if we imagine peeking at the data every 5 tosses. Each time we recompute the p-value, we risk increasing the chance of a false positive. But the E-value is designed precisely to withstand this kind of behaviour. If we stop whenever the E-value exceeds 20, we are guaranteed that the overall Type I error remains below 5%, no matter how often we look.

In short, the E-value aligns with how people actually do science. It permits exploration. It supports sequential data collection. It allows for combining evidence across studies by simply multiplying E-values, often known as compounded E-values. And it brings Bayesian and frequentist ideas into a common framework, since E-values often take the form of Bayes factors with certain priors.

The p-value is not going away any time soon. But if we care about robust, transparent, and adaptive science, then the E-value deserves serious attention. It is not just a new statistic. It is a new standard for what we mean by evidence.

In a follow-up post, I will demonstrate how to simulate optional stopping with coin tosses using both p-values and E-values, and show precisely where the p-value fails and the E-value holds its ground.

Let the data speak, even when we listen more than once.

In case you want to know more about E-Value, you can consider exploring the papers below.

Referances

[1] Peter D. Grünwald. The E-Posterior. Philosophical Transaction of the Royal Society of London Series A, 2022.

[2] Steven R Howard, Aaditya Ramdas, Jon McAuliffe, and Jasjeet Sekhon. Time-uniform, nonparametric, nonasymptotic confidence sequences. The Annals of Statistics, 49(2):1055–1080, 2021.

[3] Aaditya Ramdas, Peter Grünwald, Vladimir Vovk, and Glenn Shafer. Game-theoretic statistics and safe anytime-valid inference. Statistical Science, 38(4):576–601, 2023. ISSN 0883-4237. doi: 10.1214/23-STS894.16

[4] Glenn Shafer. Testing by betting: a strategy for statistical and scientific communication (with discussion and response). Journal of the Royal Statistical Society Series A, 184(2):407–478, 2021.

[5] Vladimir Vovk and Ruodu Wang. E-values: Calibration, combination, and applications. The Annals of Statistics, 49:1736–1754, 2021.

Leave a comment

Mahalanobis and His Friend Nehru

P. C. Mahalanobis was born into the elite bhadralok Bengali society of Kolkata. Educated at Cambridge, he was a scientist, nationalist, and institution builder in the truest Gandhian sense. But more than that, he was a close friend and confidant of Pandit Jawaharlal Nehru. They were like brothers, united by a common passion for philosophy, science, public policy, and the dream of a free, self-reliant India.

Though Mahalanobis was about four years younger than Nehru, the two shared an unshakable belief that the democratic values embedded in British and American institutions, such as free speech, rational inquiry, and the rule of law, would be vital for India’s future. At the same time, they were not naïve about the contradictions within those Western societies.

When Mahalanobis travelled to the United States, first in 1946 as India’s representative to the inaugural session of the United Nations Statistical Commission at Lake Success, and later in the early 1950s to discuss agricultural statistics and sample surveys, he and Nehru were keenly aware of the racial and economic inequalities that shaped American society. They knew where Indians would be expected to sit on a bus. They recognised that American capitalism, like British imperialism, was built on the systematic exploitation of labour, land, and marginalised communities. The practice of segregation in American society left a deep scar on the human mind.

Planning a Nation

By 1935, the Indian National Congress had already begun serious internal discussions about the need for planned economic development. Leaders like Nehru and Mahalanobis clearly saw that state-led investment in infrastructure, industry, and education was essential for lifting a colonised nation out of poverty and dependence. The idea of planning was not simply imported from the Soviet Union. It arose logically from India’s own experience, informed by the realities of segregation in the United States, apartheid in South Africa, and the exploitative policies of British rule. For instance, Indians were not allowed to produce salt in their own backyards.

In the same decade, Mahalanobis conducted large-scale sample surveys on jute production in Bengal, which turned out to be more accurate than British estimates. His statistical work attracted international attention, leading to collaborations with R. A. Fisher in the UK and Jerzy Neyman in the USA. In 1938, Nehru appointed him to the National Planning Committee, assigning him the responsibility of thinking through India’s economic future using data and evidence.

By the time of independence in 1947, Mahalanobis had built the Indian Statistical Institute (ISI) into a respected centre of statistical research. His team was already designing national sample surveys, agricultural forecasts, and demographic studies. India's planning machinery had found its scientific backbone.

The Vision and the Strain

Mahalanobis’s most famous contribution was the Second Five-Year Plan, launched in 1956. Inspired partly by Soviet central planning and shaped by his own statistical thinking, he envisioned an economy where the state would build the machines that build machines. In today's language, that vision resembles a blueprint for industrial self-reliance and even echoes the structure of modern artificial intelligence ecosystems.

However, this ambitious plan encountered several difficulties. While the model worked on paper, implementation proved challenging. In a tightly controlled economy, private capital was marginalised and public sector enterprises were often inefficient and poorly managed. Coordination between public and private actors, essential for success, was lacking due to ideological rigidity. As the INC leaders were allergic to private capital, they believed that private capital would lead to widespread inequality and discrimination against the marginalised in society.

By the mid-1960s, it had become clear that central planning alone could not deliver the intended growth and equity. Yet, India continued with the Licence Raj model for another 25 years until the 1990s balance of payment crisis forced us to reform, introducing a more liberal economic approach.

Dissent Within the Vision

Today, it is fashionable to criticise Nehru and Mahalanobis. That is the easy thing to do. The more meaningful challenge is to understand the context in which they made their decisions and consider the realistic alternatives available to them at the time.

One notable dissenter was another Bengali, Syama Prasad Mukherjee. He founded the Bharatiya Jana Sangh, which later evolved into the Bharatiya Janata Party. Mukherjee was a critic of Nehruvian socialism. He believed that central planning stifled private enterprise and innovation. He advocated for a market-driven economic model rooted in nationalist values. His disagreements were not merely ideological. They represented a fundamentally different vision of India's economic future. This diversity of thought reflected the spirit of a vibrant democracy. One problem with S.P. Mukherjee’s approach was that it was more like an ideological argument, and it was not grounded in Statistics and Data, like Mahalanobis and Nehru did. Hence, S.P. Mukherjee’s argument for a market-driven economic model was less compelling.

An Open Problem for Mahalanobis

On the occasion of the 132nd birth anniversary of P. C. Mahalanobis, on 29 June 2025, I want to pose a problem that I believe would have intrigued him.

Since World War II, national income accounting systems have never included the economic contribution of homemakers. As a result, the statistical imagination of GDP has long ignored the most foundational labour of all: raising a family.

As both parents entered the formal workforce, driven by modern aspirations and public policy, the work traditionally done by mothers at home began to be outsourced to schools, daycare centres, domestic help, and other services. This shift led to higher measured productivity and faster GDP growth, especially in countries like Japan, South Korea, Singapore, Taiwan, and China. However, it came at a cost. Birth rates began falling, populations began ageing, and social systems started struggling to support the elderly.

India remains in a different phase of development. Many Indian women still choose to prioritise family care, even as the economy grows at over six per cent annually, with a stable population replacement rate. In many ways, this may be the most balanced and sustainable growth model available to a large, diverse society.

So here is the open question.

Can we design a statistical framework to measure the economic value of homemaking and caregiving labour?

If we could, we might finally give recognition to the invisible work that sustains our families, communities, and society. We might acknowledge the contribution of mothers, including the one who raised a brilliant mind like P. C. Mahalanobis. Perhaps the greatest tribute to him is not only to celebrate what he built but also to continue asking the difficult questions he did not have time to solve.

Leave a comment