When I first undertook this seminar on behalf of SIWI, I knew nothing of Indigenous knowledge or practice. It’s not an area I have any academic familiarity with, though I have attended conference sessions by Indigenous people before.

The SIWI seminars operate as follows: two SPC members are tasked with advancing the conference's goals. They recruit suitable co-convenors who are subject-matter experts. In this case, we were fortunate to have Aleena Kawe, who leads the “Restoring Balance Collaborative”. They are a consortium of Indigenous organisations mostly from Australia, New Zealand, the US and Canada. The seminar was supported by the Ohneganos, an Indigenous-led water research program in Canada, and the Environmental Defence Fund, who agreed to join as co-convenors.

Aleena was clear about her terms for engagement from the beginning. For too long, Indigenous people had stayed on the sidelines. This time, they would own the space and set the agenda. She mobilised a global consortium of Indigenous scholars and practitioners, including youth, and organised the sessions to foster rich dialogue.

Over the three sessions, I gained some deep insights. These reflections are not an objective summary of the discussions — they encapsulate my subjective insights drawing on my own life experiences and worldview.

Creating an “ethical space” is key

One of the goals of the seminar was to share examples of effective Indigenous-led water governance and stewardship practices based on a conceptual framework of ethical space. The Indigenous groups repeatedly emphasised the idea of an “ethical space”. Dialogue cannot occur between two groups that do not see themselves as equals. Western scientists often see Indigenous rituals and culture as performative, not endowed with any deep understanding of the natural world, and consequently not to be taken seriously. Safe and respectful dialogue and collaboration between two distinct world views - Western and Indigenous - therefore requires laying the ground for an ethical space first.

Quite a bit of time was spent laying the ground for what an equal, inclusive dialogue space might look like. What struck me was how much indigenous cultures are rooted in respect, asking permission, and gratitude for the welcome received. Perhaps a stark contrast with colonisers and invaders who simply came and took. It can seem drawn-out and formal, but relationship-building and trust take time.

I have seen vestiges of this in many cultures. It made me better appreciate the elaborate welcome songs and vote of thanks I have witnessed in meetings in Nigeria and India.

Indigenous people focus on the “who” not the “what”

In framing the session, one of the speakers pointed out that Indigenous people ask “who is water” not “what is water”. In Indigenous cultures, water and the natural world are viewed as living relatives: Grandfather sun, Grandmother moon and so on. And the self and clan are seen as part of that family. Thus, nature is not separate from humans. They are all interconnected in a web of life. This means any form of extraction or accumulation does not make sense. Would you steal from your mother to enrich yourself?

I understood better Aleena’s discomfort with framing “water as a resource”. If water is a beloved relative, then measuring and quantifying it would be strange. It would be like describing your favourite aunt only through her height, weight, eye colour and waist size. You know your aunt and her bond with your mother in a way that neither their respective vital statistics nor the hours they spend together can quite capture.

The analogy helped me understand how difficult it would be for Indigenous knowledge to sit alongside Western science. Indigenous knowledge is a value system and worldview, and less a scientific methodology. As Aleena put it, their relationship with the river is the worldview that guides their scientific practice; it is not the science itself.

Indigenous knowledge itself is deep, intimate and place-based — that wolf pack, this river, these trees, but so much more difficult to universalise. But recognising that Indigenous knowledge is a worldview and value system means that Western science can be consistent and work along Indigenous practices.

This was particularly evident in the approach taken by the Six Nations Polytechnic in Canada, where youth groups described their water-quality monitoring work. It was not the mixing of Indigenous and Western science; rather, their worldview remained rooted in Indigenous values and expressed in Indigenous languages, even as their methodologies and measurement techniques were Western.

After all, you can still persuade your mother to do bloodwork and be concerned about her lab test results even as you recognise that those pieces of paper do not describe her fully.

Personhood of the River Wanganui

For anyone who has worked in the water sector for long, the news of New Zealand granting legal personhood to a river would be familiar. I had always wondered how that worked. The Māori people of New Zealand, after 150 years of struggle, managed to change the legal doctrine of rivers to reflect their own imagination of respect, responsibility and relationship. The principle is to keep riverine integrity and health as sacrosanct, and everyone in the watershed is asked to think about what they can do for the river rather than the other way around.

Today, the New Zealand Government legally recognises the Whanganui River as a living entity. Under New Zealand Legislation, Whanganui Iwi (a Māori tribe) and the NZ Government each appoint someone to serve as Te Pou Tupua. The Te Pou Tupua is the “human face and voice” of the river, who speaks on behalf of the river. I had the privilege of speaking with the Te Pou Tupua Tarama, who explained how decision-making works. This means legal personhood is not just on paper but underpinned by a proper institutional structure and due process.

We have had many discussions in India, and the Uttarakhand High Court even granted legal personhood to the Ganga in 2017 (though the Supreme Court later overturned the ruling). But what I learned from the sessions was that in the case of the Whanganui, the relationships, values and principles came first. Everyone in the river was already committed to the idea; the law was merely a recognition of what was already in practice. If we hope to apply these ideas in India, we should probably start with a river and watershed that is not completely destroyed and where the people in the watershed are completely committed to protecting the river first.

The unknown laws of nature

Just because Indigenous knowledge is place-based does not mean it lacks universal truths that Western science has yet to uncover.

It occurred to me as I listened to speaker after speaker that indigenous people know truths that Western science might recognise as law-like generalisations; but we don’t (yet) have the language and framework for them.

This is, after all, true for many eastern traditions — Western science is only now discovering the gut and soil microbiomes, ideas that most Indigenous peoples have long understood. Similarly, Harvard Medical School researchers recently discovered a specific neuroanatomical pathway that shows how acupuncture can drive powerful anti-inflammatory responses.

My dad gave me a particular appreciation for this idea that there can be undiscovered universal truths. He was raised a Theosophist in Adyar, Chennai. One of the foundational objects of the Theosophical Society is to investigate unexplained laws of nature and the powers latent in man. (The others are universal brotherhood and the study of comparative religion, philosophy, and science.)

My dad, a nuclear physicist, took the goal seriously; he was one of the most open-minded people I know. One of the things he worked on in his retirement was the study of subtle energies and consciousness, even at the risk of ridicule from his Indian physicist friends. What was remarkable about theosophists was their willingness to use eastern methods (transcendental meditation) to study western questions (structure of the atom). And one of the remarkable texts that emerged from that work by Annie Besant and C.W. Leadbeater was the publication of “Occult Chemistry” in 1908, which laid out atomic weights, isotopes and the string nature of atomic particles, ideas that would only be discovered by physics till much later. People can be sceptical and claim fraud, but the fact is that the book, written in clear English, lay buried in the library in Chennai through my dad’s childhood. And the string theory of the atom did not take root for decades later. That was enough to convince me that there are things about the nature of the universe we simply do not understand. Some of these Indigenous and ancient societies knew and understood well.

This is a thin line because if one believes everything, one can fall into traps of superstition and unethical behaviour. Indigenous peoples have long-standing cultures and values to steer them. But for the rest of us, the key is to stay open-minded, open-hearted, and curious, ask questions, and remain rooted in our own sense of right and wrong.

The canoe and the ship must sail together, in parallel

If Western science (or at least some scientists) dismisses Indigenous knowledge, I did not see any disdain among the Indigenous people for Western science.

In fact, there was a great openness to it with two caveats.

First, do not force a “mixing” of Indigenous knowledge and Western knowledge. Let them sail in parallel like a ship and canoe floating along a river together. We might not yet be ready to understand fully how they blend. But like braids of an intricately woven shawl, they can still interweave without mixing.

Second, rather than have academics in universities study them, it is better that the science should be undertaken by Indigenous people. Train their youth in scientific methods, invest in their labs and let them discover and express traditional knowledge in Western terms. Any blending of Indigenous knowledge and Western science must be inside out.

This concern arose from Western academics harvesting Indigenous knowledge and stealing the credit for knowledge passed down the generations. This worry is accelerated by AI and its insatiable thirst to codify all knowledge.

In an age of AI, maybe indigenous cultures will teach us to stay human?

Recent headlines about how fast AI is moving and how little our political systems are able to rein it in have me worried that the worst science-fiction stories about a super-intelligent overlord could come true.

I imagine that in such a world, we will be forced to return to knowledge systems that rely not on thought but on consciousness, feeling, and relationships between people and the natural world. Whether we will move from the “I think therefore I am” of Western philosophies to the “we are” of Indigenous ones.

I feel almost silly writing these pieces. But then my earlier Sub-Stack on missing research design skills got 900+ views. I don’t think that people are reading these because they had nothing else to do on Sunday. Moreover, I still see intelligent researchers making these basic blunders. So here goes.

Subscribe now

The intent here is not to “graph-shame”; I know I myself sometimes make graphs in a tearing hurry, and while I would want every presentation I make to achieve "the gold standard” (described at the end of this article), the first step is knowing there is a gold standard.

I have been gifting copies of “Storytelling with Data” by Cole-Nussbaumer Knaflic whenever I can. Her Talk@Google, YouTube channel, and Mini-Workshop are all fantastic, and I think they should be required watching for all young scholars.

I am assuming in the age of AI you can produce graphs at high speed. But the judgement of whether it makes ANY sense still has to be made by humans. BTW, asking Claude to use the principles of “data storytelling by Cole Nussbaumer Knaflic” is a legitimate prompt that will produce some nice charts. I would still hope people read/watch the actual content and become familiar with the concepts.

But Knaflic's books come after people have stopped making basic quantitative blunders. This article is about first recognising, and then avoiding those.

These examples have been drawn from actual presentations or journal articles I have had to review. The data and context have been changed to maintain anonymity.

Case #1: Ordinal vs. cardinal data

Ordinal data has categories with a meaningful order or rank, but the differences between values aren’t necessarily equal or meaningful. Example: Survey responses like “Poor, Fair, Good, Excellent” — Excellent is better than Good, but the gap between Poor→Fair may not equal the gap between Good→Excellent.

Cardinal data (also called numerical or quantitative data) have values for which both the order and the exact magnitude of differences are meaningful. You can add, subtract, and meaningfully compute averages.

But you cannot use the same type of chart to represent both, just because Excel has a Line Graph feature. In the figure below, which shows something by district (population, area, rainfall, whatever), it makes no sense to have a line graph. The districts are ordered randomly and are categorical. Therefore, connecting the data points with a line graph makes no sense.

There are other basic issues with this graph — as presented by the interns —no Legend, missing Y-Axis Labels, small fonts, and unnecessary clutter. However, those you can learn to avoid from Knaflic’s book/ YouTube channel. The line graph is simply a blunder.

Case #2: Adding / Averaging Ratios

Another mistake (hopefully less common) is summing or averaging ratios. A ratio like GDP per capita looks like an ordinary number you should be able to average or sum, but it’s actually a ratio (GDP ÷ population), and ratios don’t add the way raw quantities do because ratios have different denominators.

E.g. GDP per capita for a region = (Total GDP of that region) / (Population of that region)

If you have two regions:

• Region A: GDP = ₹100, Population = 10 → per capita = ₹10

• Region B: GDP = ₹500, Population = 100 → per capita = ₹5

Firstly, adding 10 and 5 is simply not a meaningful number, and therefore adding it to complete a pie has no meaning. A pie chart is always made up of things that answer the question (“what fraction of the pie does X have”. You can say “What fraction of the World’s GDP does India have? but not “What fraction of the world’s GDP/capita does India have?”

Secondly, even a naive average of the ratios, (10+5)/2 = ₹7.5, is not meaningful.

But the actual combined per capita = (100+500)/(10+100) = 600/110 ≈ ₹5.45

These are quite different (₹7.5 vs ₹5.45)! The naive average is wrong because it treats both regions as having the same population, ignoring that Region B has 10x the population of Region A and should dominate the combined figure.

Case #3: Comparing Ratios and Absolute Figures

Another mistake I have observed is plotting ratios vs. absolute numbers. Now this is not always wrong; sometimes there is a valid reason to do this. But it is often useless. For instance, we have no reason to expect per capita electricity consumption to have a meaningful relationship with absolute GDP. Since you have large and small poor countries and large and small rich countries.

But per capita GDP consumption does have a relationship with per capita GDP. And not surprisingly, that graph looks meaningful.

Case #4: Space vs. time variations

I saw an example of this in a paper I reviewed a decade ago. The paper was trying to compare changes in industrial water use before and after a major event (think war/earthquake/political coup, whatever). The thing is, the authors used a spatial map to make the point because GIS maps look pretty.

The problem is that the industry in that country was concentrated in one region (in this fake map, the eastern region). So the spatial map obscured the temporal decline. If you look at the raw (fake) data in the temporal bar graph below, you can see a consistent 10% decline across the country. But because the water use was so spatially skewed, the individual states did not change categories in the legend. So the spatial map stayed more or less unchanged. Anyone reading it would assume that nothing changed; when in fact Industrial water use had dropped 10% across the board.

The Gold Standard: Tell a story with the data.

Cole Nussbaumer Knaflic’s Storytelling with Data lays out core principles for turning data into a message people actually understand and act on, rather than just displaying numbers. Here are the main ideas of her gold standard (I used Claude here, edited and added my own experiences).

1. Understand the context
Before creating any visual, know who your audience is, what you want them to know or do, and how you’ll communicate it. She recommends being explicit about the “who,” “what,” and “how” before opening any charting tool. Jumping straight into building a graph without this framing usually produces something unfocused.

This is equally true of writing. But the point is you cannot expect people to understand a graph unless you yourself have clarity of the story you want to tell. If you are throwing numbers on a chart hoping others will get something out of it, well, you may not be happy with what they took away!!

2. Choose an appropriate visual display
Different chart types suit different kinds of comparisons. She pushes back against defaulting to pie charts or 3D charts (which often distort perception) and advocates matching the graph type to the message.

This is the step where I would really check for the types of blunders I listed above.

3. Eliminate clutter
Borrowing from design principles (including the idea of “data-ink ratio”), she argues that every element in a chart should earn its place. Gridlines, borders, unnecessary labels, and decorative elements that don’t add meaning should be removed, since clutter competes for attention with the actual insight.

I really like Knaflic’s attention to clean, minimalist images. It’s not cluttered.

4. Focus attention where you want it
Once the clutter is gone, use pre-attentive attributes such as colour, size, contrast, and position. Her approach is to intentionally guide the viewer’s eye to what matters most. One common technique she uses is greying out most data and using a single bold colour to highlight the key point.

One thing you learn quickly is that colour contrast really really matters.

5. Think like a designer
Concepts like affordances (what’s clickable or important?), intuition (what do different colours mean?), accessibility (is it colourblind-friendly and readable in a large room?), and using alignment or contrast to create visual hierarchy all help make data visuals more intuitive and usable, not just accurate.

This is not something that most engineers think about. But I am continually surprised by papers that don’t think about the psychology of colour at all. Just the other day I saw a map where red was used to represent “cool temperatures/ wet soil”, and blue was “ warm temperatures/ dry soil”. The cognitive burden of that presentation was just intolerable; I had to keep telling my brain to switch the colours, and I am getting old!

6. Tell a story
And finally there is her main theme: data alone doesn’t persuade; narrative does. She recommends structuring data communication the way a story is structured, with a clear beginning (context/setup), middle (the “aha” insight or tension), and end (the call to action or takeaway), rather than presenting a wall of charts and letting the audience infer the point themselves.

I need to do a whole separate piece on structured writing. But the point is, having clarity on what message you want to convey. A good test is to show the chart to a friend who is unfamiliar with your work and see if they “got” the message just by looking at it for 20 seconds.

PS:

OK After I wrote this and wondered if I was being too simplistic, my team presented this graph today (to be fair to them, it was done by a partner — but one that is supposed to be in the business of communications!). But I am leaving it here now for people to play “Where’s Waldo” in terms of finding all the things that should be changed to meet the Gold Standard.

For some context, they wanted to see how farmers views changed across multiple sessions of the consequences of introducing Water User Cooperatives (WUCs). Session 1, 2 and 3 were parallel not sequential. They were done with different groups. H = Head-end, M = Mid-end, T=Tail-end, DL = Dryland and HL = Highland.

I had an epiphany this weekend. People who have PhDs in engineering from elite Indian institutions have not been taught research design. They had learned the laws of thermodynamics, hydrologic modelling, or Python programming, but not necessarily statistical analysis or study/sample design.

This post is then for me to have something to forward to young scholars when I see them making the same mistakes over and over again. There is nothing profound or philosophical in this article. So if you are a seasoned researcher, you can stop reading here.

Subscribe now

Drawing from some case studies

Here are some real-life case studies I have encountered. Some of these are by PhD students, others by early-career researchers. For me, the question it begs is: as the Indian government is pushing researchers to engage in real-world research (such as through the recent ANRF MAHA-Water grant, are we simultaneously building the new generation of research skills needed?

These things can be learned. And by no stretch of imagination am I an expert in any of this. But at least I have “Knowledge of Ignorance” — I know what I don’t know. My encounters suggest that many have “framing blind spots” or “Ignorance of Ignorance”. If you are not familiar with a concept, you probably won’t notice blunders made by students/colleagues/AI.

With the rise of AI, I am noticing blunders more and more, except they are now cloaked in slick, grammatically correct (if verbose) writing.

Case 1: Does technology A do better than technology B? The problem of sample size and sample selection.

An early career researcher presented analyses from the deployment of a treatment system in Bangalore. There were two potential technologies. The researcher wanted to know which one performed better in a real-world setting. Now it was an expensive study (say Rs 1 lakh per site). Given the limited budget, they picked a few plants (a total of 6 ) that they had access to (i.e., not randomly chosen from the population). and compared, finding that technology A was better than technology B.

The problem is that a large study by a major global foundation simultaneously sampled 5% of the plant population (at a cost of $1 million!). That study found that the technology made no difference. The performance depended on their size and other factors. Both technologies did equally well or badly overall.

This is exactly where statistically random sampling makes a difference. Let’s say the figure below represents units — the larger the icon, the larger the unit; stars represent technology A, circles represent technology B. Blue icons are functioning units, red ones are defunct.

About half of the units fall into each category, functioning and defunct. If we pick a non-random small sample, you could reach pretty much any conclusion. For instance, the specific small non-random sample below suggests that Technology A performs better than Technology B.

Statisticians routinely conduct power analyses to determine sample size and study design. The “correct” sample size and, therefore, the budget/feasibility of doing the research depend on the effect size. If we expect the performance difference to be at least 15% (based on an agreed-upon performance measure), we need a sample size of 2-3%. If the expected performance difference (“effect size”) is likely to be lower and there are 2-3 variables, we may need to sample almost 10% of the population and so on.

Case 2. “Does technology A do better than technology B?” The problem of omitted variable bias.

The second problem I keep encountering is the omission of variables. Engineers seem convinced that systems don’t work because the technology is poorly designed. But it may be just one of many factors. Some technologies may be superior, but they may need additional training for operators. Others may work only at some scales, not others.

In the same sample above, if unit size is also considered, it becomes clear that Technology A performs better at large sizes, Technology B at mid-sizes, and neither performs at small sizes. So if size is omitted as a variable, then the whole conclusion becomes skewed. In designing the research, then, one would “stratify” the sample by randomly selecting plants from large, medium, or small units. It could be that it wasn’t size but location within the city, or need for a particular skill set in operators, etc.

How would one know which variables are relevant? The only way is to read the literature, talk to people, and hypothesise. For instance, a quick qualitative survey of “Is the plant working?” may enable a large sample to be surveyed quickly and develop some hypotheses. It is very reasonable to conduct a cheaper hypothesis-generation exercise before starting the actual experiment.

Case 3: What is the optimal way to operate Technology A in a real-world setting? Choosing a suitable independent variable.

I came across a scientific poster by a student who was just finishing his PhD. The question he was asking was what the optimal design for a particular pollution removal technology was.

So the graph looked like this, and the conclusion reached was that 5.3 kg/cu m was the optimal loading. When I asked why the X-Axis numbers were so weird, he said it was due to natural flows and that he couldn’t control the influent pollution loadings, which came from a natural stream.

I asked what the point was of optimising a technology for an input value that couldn’t be controlled. The student said he could always adjust the flow rate and, therefore, the loading. I should have asked why he then did not adjust the flow rates in the experiment to obtain uniformly spaced values in the first place. And why did he pick kg/ cu m instead of kg/min if that was the case?

But I let it go and instead doggedly pursued the bigger picture question. Nothing in the graph indicated that 5.3 kg/ cu m was optimal. It just happened to be the highest number. The numbers were up and down before and after. There was no theoretical basis to expect an “optimal,” and it wasn’t clear that the natural stream quality across the samples was similar with respect to other variables such as temperature or salinity. Yet I imagine the study was quite expensive to do. Because of the number of parameters tested, including heavy metals.

For me, the only reasonable finding was that the efficacy rate was between 50-90%, for reasons that needed to be better understood. The student, a month from graduation from a prestigious department, had never been asked these questions and did not think them necessary to ask.

Case 4: How frequently does a particular event occur and why? Comparing apples and oranges.

Another student was designing a research program to measure a parameter in natural systems and wanted to know where and why something occurs (let’s say, a rain or a flood event). They wanted to run a recall-based survey of community members. But the student was not trained in survey design. So the wording of the question was not designed carefully enough to yield a useful answer.

For instance, social scientists know that asking “How many times did the water level rise higher than knee level in the last 2 years?” is different from asking “What was the highest water level observed on August 8?” One results in a generalised risk profile, the other in a specific risk related to a specific event. I have found people collecting data asking the question one way and interpreting it to mean something different.

It’s also important in surveys to ensure that the event itself is observable and significant enough for a recall survey to be reliable. For instance, most farmers will clearly remember every detail of a borewell they drilled, even if it was 10 years ago, because it was such a major (and risky) expenditure. They are unlikely to remember if it rained in August, 10 years ago. And they are unlikely to be able to estimate how many times it rained more than 25 mm last year or how many liters of water they used per when showering that morning.

Case 5. Is an Operations and Maintenance Protocol effective? Generalising the results beyond the original study context.

This final case study was from an international study team examining an operations and maintenance protocol for rural water utilities. They wanted to run a Randomised Controlled Trial (RCT) experiment to see whether training utility operators in a specific way would improve overall maintenance.

Now, RCTs have long been considered the “gold standard” in causal research. Because the treatment is assigned randomly. Presumably, all other confounding variables will then be equally likely to occur in the control and treatment groups. This particular RCT bothered me for a long time, because RCTs are expensive to do and hard to pull off. You typically need the government to be on board to be able to randomly assign a major intervention.

In this particular case, though, the research team decided to execute the RCT with a private utility operator in Africa. Since they had an MOU with the operator (which also, in part, received grants from major US foundations in addition to user fees), they would have some control over the study design. The RCT was carefully designed and executed with appropriate protocols, ensuring that the results would be robust. My problem was that they wanted the results (if positive) to inform how utilities across the developing world conducted Operations and Maintenance.

What bothered me was that the solution was tested in a very specific context — a private utility where the incentives, leadership and capacity were already primed to maintain the systems. So how the protocol performed in that context may not translate to public utilities, with all their attendant politics. The problem here was not the robustness of the research design but the generalisability of the research beyond the study's context where the findings would be applied.

So where is the gap?

All of the cases above are basic research design and interpretation problems. Nothing I have said is profound or even beyond the merest commonplace. Every economist and biologist would have been taught how to control for confounding variables. Yet I have found these study design issues to be ubiquitous in engineering studies.

In each of these cases (and dozens of others), the research design was not even a matter of discussion. The student simply assumed the design was appropriate and was unable to engage in a discussion about validity when questioned. And then there is heavy use of AI for number-crunching everywhere. It’s as if slick number-crunching can overcome terrible design.

If we want to move towards more practical, societally beneficial research in water and natural resources, we need to change how researchers at elite institutions are trained.

That does not involve more AI. It involves basic research design and philosophy of science skills.

PS:

Remember the fight between Sheldon Cooper and Amy Farrah Fowler in The Big Bang Theory? "You wouldn’t recognise a confounding variable if two of them hit you in the face!" That’s the problem in a nutshell. It’s not just an Indian institution problem. It’s just that not all science and engineering disciplines require these techniques. The problem is that PhDs who go into industry will need a broader understanding of research methods, but they aren’t getting that training.

I found these articles thought-provoking, but also slightly disconcerting. They were authored by the two biggest Indian philanthropies.

Given that WELL Labs is betting on systems change, if “systems change” philanthropies are not fully convinced of the need for scale and systems change, are we even on the right track?

The current debate on “to scale or not-to-scale” is not entirely helpful

There is often a tendency in the social sector to say that we will run a pilot as a proof of concept, then government will scale it. But the sector is notorious as a graveyard of gold-plated pilots that work only because they propped up with external funds and charismatic leaders. Thus, the phrase, “pilots never fail, pilots never scale”.

On the other hand, every big programme has to start somewhere, usually with a proof of concept. And most large philanthropic programmes cannot justify the scale of their investments without the promise of significant social transformation. If you are giving away 10 Cr per year the net impact of that cannot be 100 people.

So how do we reconcile these two? We all have to start somewhere, but how do we stop it from remaining a pilot? As Doulaye Kone from the Gates Foundation put it: “Is it a baby mouse or a baby elephant?” A baby elephant can grow into an adult elephant but a baby mouse can only grow into an adult mouse.

I found this a very helpful metaphor. But before I launch into my version of how to identify the species of mammal, I should clarify that my views are very contextual to land and water systems. It’s what I know. Its what we do at WELL Labs.

In general, scale is achieved in different ways, and “systems change” is one of ways. At least in the water (and natural resources) sector. In other words, my position is that you cannot change water resources at scale without transforming systems. In fact, there is an entire field of study on “sustainable transitions” that provides its theoretical basis for such systems change. Finally, systems change is possible through collaboration, but there are fantastic examples in India with clear lessons.

Scale is achieved in different ways and the pathways are context dependent.

Scale can be achieved in different ways. “Service delivery” programmes, such as distributing mosquito nets, running deworming programmes or vaccination camps, or handwashing campaigns, involve replicating a proven template.

These types of interventions require people to change behaviour; but they are personal behaviours. “Get your kids to wash their hands so they don’t fall sick as often”, “buy subsidised mosquito nets so you don’t get malaria” and so on. With such interventions you still need to build capacities and convince people of the benefits of the new behaviours. But you do not necessarily need systems change. If people are convinced that washing hands makes them less sick (not saying this is trivial), and water is available there is not much else that is needed.

I would argue scaling service delivery programs is fundamentally different from scaling programs that require behaviour change linked to livelihoods or production systems. This therefore applies to most behaviours related to natural resources management. If you want a farmer to grow something different, pump less groundwater, or use fewer pesticides, you cannot expect to run training programmes and expect their behaviour to change. The new behaviours still have to fulfil the primary objective. If the purpose is to make a decent living, the new behaviour has to guarantee at least as good a livelihood.

For the agriculture sector, scale is not possible without transforming systems.

There is a fundamental difference between behaviour change around production systems. Farmers aren’t growing food for their own consumption. They are growing it for the market. In fact it is very common to encounter farmers who are very careful with the land on which they grow their own food vs. land on which they grow food to sell.

Farmers are incredibly rational. They may be more risk averse or have a higher discount rate (in the sense they may not care about the land in the long-term) than the general population, but they are still rational. So, to change farmer behaviour on water, they need to perceive a direct benefit from the change. This means aligning policies and schemes to what benefits them in terms of higher income, lower cost, lower risk, ease of cultivation; or all of the above.

By definition this means changing the conditions around agriculture. The bulk of the work to benefit farmers and change behaviour occurs elsewhere. Input supply changes, market conditions, etc. In other words the production system as a whole needs to shift. Then extension services need to help farmers move in response.

The field of sustainable transitions offers a theoretical basis to understand systems change.

The field of sustainable transitions offers useful frameworks to overcoming systemic lock-ins.

Figure: Sustainable Transitions. Deleye et al., 2019

The field of sustainability transitions,( or socio-technical transition studies) is based on the idea that the systems through which we meet our basic needs (energy, water, mobility, food) are not merely technologies. Rather they are socio-technical regimes: stable configurations of technology, infrastructure, markets, regulations, institutions, skills, user practices and cultural meanings that have co-evolved and now reinforce one another. This results in systemic “lock-ins”. Once you have a lock-in, single-instrument fixes tend to fail. To “lock-out” the system, you often need a multi-pronged strategy.

Systems change is possible, and there are fantastic case studies in India.

There are great examples of such shifts even in the Indian agriculture and livelihoods context. In these case studies, systemic lock-ins were successfully overcome.

Odisha Millet Mission transformed cultivation of millets in Odisha. It is formally, the Special Programme for Promotion of Millets in Tribal Areas. It is a flagship programme of the Department of Agriculture and Farmers' Empowerment, launched in 2017 to revive millet cultivation and consumption "in farms and on plates" across Odisha's tribal districts. Today the programme covers over 250,000 tribal farmers. The mission originally began by promoting a package of climate-resilient practices and incentives for adoption. But it scaled because of the concurrent guaranteed procurement of millets through their inclusion in public schemes like the Integrated Child Development Services (ICDS) and Public Distribution System (PDS). The Mission also undertook a massive campaign to promote millets as an aspirational super food in urban areas. To make the scheme work both demand and supply side levers, along with delivery logistics had to be deployed simultaneously.

The Andhra Pradesh Community-managed Natural Farming (APCNF) policy is another example. The APCNF project was recently awarded the 2026 Food Planet Prize. The approach Andhra Pradesh chose to deliver natural farming not through the conventional agriculture department's input-subsidy machinery but through a dedicated non-profit Rythu Sadhikara Samstha (RySS), acting as the implementing agency. The programme today covers almost 2 million farmers (albeit with a generous the definition of what a natural farmer is). Over 10,000 Community Resource Persons, around 60% of them women, drive the farmer-to-farmer learning. The programme works through a model of "champion farmers". These are local trainers who are paid to drive adoption. The APCNF scheme did not involve incentives; rather the focus was on reducing barriers to adoption. The goal was to help farmers perceive the reduced input costs and health benefits.

Another example is the Pashu Aadhar scheme (I had no idea livestock in India have unique IDs — someone just told me about it recently). It is designed to function just like the Aadhaar card for people. It is a core component of the Government of India's National Digital Livestock Mission (NDLM). Livestock, like cows and buffaloes, are tagged with a 12-digit bar-coded tag, usually affixed to their ears. The ID on the tag is linked to the Bharat Pashudhan database. It logs the species, breed, pedigree, calving, and milk yields. It tracks data on artificial insemination, vaccinations, disease treatments, and deworming. Adoption is high (9.5 crore or 95 million already) because it simplifies processing of cattle insurance, subsidies, and death claims by providing verifiable proof of ownership and medical history.

But not all systems change programs involve individual behaviour change, sometime they require trust creation.

In all the three systems change success stories mentioned above, the behaviour changes were transactional not relational. Farmers gain the benefit of lower costs or risks because of a government scheme.

But what if systems change requires collaboration. An example is canal irrigation systems, where the last mile is farmer managed. Or Farmer Producer Organisations that are cooperatively owned. Here the system can only improve if groups of farmers agree on collective rules. This type of behaviour change is inherently slow and painstaking because it requires trust building, which is fundamentally hard to scale quickly. This is where the skepticism for systems change comes. In our own work, we found that even when farmers would collectively and unambiguously benefit from cooperation, there was still temptation to free ride or worse, screw someone else over. We came across a case of one farmer, who refused to cooperate because he did not want his brother to get water, even it meant he himself would also not benefit.

Now the temptation may be to argue that we should not scale. Human relations are messy. Let’s just deal with it one farmer at a time and let change take decades if needed. But in an irrigation system this is not a choice. You cannot reform an canal irrigation system or an aquifer covering 100,000 Ha, one Ha at a time. You need to still do it at the scale of the system, which is still tens of thousands of farmers.

Is there a design principle that would scale trust quickly? The closest I have come to an “Aha” moment was a Water Data Podcast episode with Henk Ovink, the Chief Commissioner on the Global Commission on the Economics of Water. Henk describes the idea of imagined redundancy to capture a programme design where you run hundreds of bottom-up trust-building processes in parallel instead of in sequence. In each, you pretend you have all the time in the world to listen deeply and engage with people’s concerns, but you still drive the process to reach consensus within a reasonable time-frame of a year. This type of trust-scaling needs “distributed negotiation skills” of the type we simply have not invested. But this is the type of scaling that is expensive and especially needs philanthropic capital.

In our own work at WELL Labs, we deal with both types of system change — the faster and slower kind.

So how do we know if a programme is a baby elephant or a baby mouse?

I think the answer is not hard but it requires absolute honesty. All of these systems change schemes had common characteristics. They created schemes that changed incentives, market conditions or perceptions of individuals coupled with a delivery/ training ecosystem to drive adoption, sometimes coupled with a digital ecosystem.

• Draw a picture of the initial (current) low level equilibrium and desired end state (high level equilibrium).

• Do a “spot the differences” and list out all the agent and biophysical conditions that need to behave differently.

• List all the necessary and sufficient conditions needed to induce systems change.

• Identify what external conditions influence each agents behaviour. Describe how these would have to change based on what we understand from the literature.

• Identify which of these are changing independently say because of technology or demographic trends.

• Identify what is changing because of ongoing programmes by other government or civil society actors. The gap is what the systems change programme is seeking to change.

• Now not all changes are equally important and urgent. We need an honest discussion about whether the programme has correctly identified the “monkey” (in the analogy of teaching standing on a pedestal reciting Shakespeare — yes, yes I know — too many species in this metaphor between the monkey, the mouse and the elephant). The monkey is the hardest or rate limiting change. Its the one which is the make or break condition. So someone needs to be tackling it and you need to be convinced based on data that its possible.

• If the most crucial component (the monkey) is trust building, then the whole programme needs to be designed to accommodate this.

Last year, WELL Labs co-organised DropTalk with Fluxgen. This year, we were happy to participate. It was a busy, exhilirating two days, with panels, a lake walk, a painting competition for kids, pitches from new start-ups, a “Hydromingle” session where start-ups mingle to find ways to collaborate, etc.

Subscribe now

As I was sitting through the sessions, it crossed my mind that there was something amazing about the water tech community in how radically collaborative it is. Ironically, private-sector organisations whose goal is to maximise shareholder profits collaborate without any hang-ups. In contrast, in the social sector, which is supposed to be collaborative, working together is so much harder.

It’s been bothering me all week.

The free market is an example of radical collaboration.

Let’s examine how the private sector “collaborates”. Companies obtain raw materials from other companies in exchange for money. The transaction is complete when the goods are delivered, and payment is made. A company supplying bricks does not fret over why they were not “acknowledged” when the building was sold. The sale already represents the exchange of value. The brick company can show “proof” of value delivered in its annual statement.

To an alien from Mars, the free market would be a form of radical collaboration. A modern Apple iPhone depends on a complex global supply chain: cobalt may be mined in the Democratic Republic of the Congo, lithium extracted in Chile, rare earths processed in China, chips designed by Apple Inc. in the USA but fabricated by Taiwan Semiconductor Manufacturing Company in Taiwan, and final assembly carried out by Foxconn in India. The finished phones are then shipped through global logistics networks to retailers and consumers in markets such as the United States. None of the people working at any of the facilities has ever met the iPhone's final owner. Yet, it all seamlessly works.

Now, of course, companies do not view their vendors or customers in the same way they see their competitors. But I found this basic attitude continued to some extent even when it came to competitors. In just the last few months, competing start-ups have gathered at our office in Basavangudi. They are interested in establishing common standards for treated wastewater reuse or in improving consumer awareness of the services market for water treatment or rainwater harvesting. When the market grows, everyone benefits. This is why industry associations are so successful at lobbying the government (albeit sometimes to the detriment of the greater public good).

Academics build on others' ideas through citations. This also enables collaboration.

We often say that researchers can see far because they stand on the shoulders of giants. Academics formally ackwowledge the work done by previous researchers by citing their work. By tracing citations back, you can trace the origins of an idea. No one complains that their idea is stolen, provided it is properly cited. Academics are rewarded based on the number of citations of their work. And the system works perfectly. The “exchange of value” occurs in the bibliography of a research paper.

The social sector, in contrast, lacks such a system. This ought to be strange.

After all, aren’t we all working towards a mostly common goal (with some minor disagreements)?

At Drop Talk, I had made a general call for collaboration on a panel. A young lady came to me afterwards and asked me how to collaborate, when NGOs are so suspicious of each other, worrying endlessly that their “credit” will be stolen. She said she was working with some other orgs in the solid waste space, but they refused to share data. This mirrored my own experience. We were working with a couple of organisations last week. They did not actually have similar expertise (or indeed an interest) in the type of work we do. Nonetheless, they were extremely reluctant to share data, perhaps worried that we would produce better, slicker external communications and thus “steal” their thunder and years of hard work. Yet, what we were proposing to do was crucial to them achieving the impact they desired. In the private sector, we would have been their “customer”.

But recently, we started working with another start-up, Krushi Hrudaya, which has installed IoT-based sensors in BWSSB wells. I started noticing that Dr Seshagiri Rao, the CEO, made a point of verbally acknowledging all government and academic partners in every discussion. He would even add this addendum to every single WhatsApp message. Initially, I was confused about what to do with the information and wondered why he felt it necessary to keep reminding us that “many had contributed intellectually to his product”. I later realised it was his way of acknowledging the value of services provided for which no payment had been made.

I did some Googling (and yes, asking Claude), but I could not find much about what the architecture of “exchange of value” would look like in public problem-solving.

Now, it is often written into our grant agreements that everything we produce must be open source. This is something we take seriously. Yet how do we get (and give credit) to other organisations for their work? Citations of reports are fine, but in the grey literature, there is no formal tracking system, so the other organisations may not even realise we have cited their work.

One colleague I discussed this with suggested more awards. But I argued that this does not solve the specific problem I am looking for a solution to. With awards, only the most visible “apex” organisation gets credit. Money follows awards. So, in fact, it makes other NGOs even less comfortable with sharing their hard-earned data or contacts if it means another will get an award and attract scarce philanthropic funding.

What we need is a way to pass credit on to organisations whose work/ relationships/ pilots we are building on. I thought I would take inspiration from the tech sector, in the hope that someone somewhere will build a “credit exchange” to enable radical collaboration in public problem-solving.

First, a Token model. Organisations formally offer “appreciation tokens” proportional to the value they derive from the work of others. Organisations proudly report their tokens in their Annual Reports.

This would need to be capped to prevent it from becoming grade-inflated (as in the Harvard grading system) and therefore meaningless. Perhaps it can be justified in terms of “hours saved” that one organisation saves by building on the work of another. In an ideal world, organisations would be incentivised at least to collaborate (even if they game the system a little bit — as academics do as well).

Second, the Patreon model. Everyone gives a small token amount to any organisation whose work they learn from (ranging from ₹100 to ₹ 10,000). This adds a small but real monetary stream. Organisations can point to the real recognition (cash) from other organisations for their work.

Third, the Data Exchange model.

Sattva launched the Water Data Exchange (WDE) as a potential solution to the current fragmentation of water data, enabling scalable, impactful solutions for better water management.

We proposed a version of this to Sattva. We compensate citizen scientists and local NGOs to collect hyperlocal water data. We then pay them in proportion to the value the data generates in the market. They own their own data forever. But the aggregate data are sold to companies, governments or universities.

At its core, WDE operates on a data token model. It recognises that in a world of AI, the only things that are valuable are context and primary data. Both of which, field-based NGOs have in spades. In such a model:

• Every validated data point earns the collector a base token (e.g. ₹0.10).

• Tokens are scaled based on the data's value, rarity, geographic importance, recency, and predictive utility.

• Collectors who demonstrate longitudinal consistency, same location, regular cadence, and high accuracy accumulate milestone bonuses, because time-series data is exponentially more valuable than one-off observations

• Monthly payouts are determined by what buyers are willing to pay for that data, creating a dynamic pricing mechanism analogous to YouTube’s ad revenue model, which rewards creators based on audience value.

This is a market model where the data collector is a value creator, not a volunteer. NGOs working in the field can monetise routine M&E data, especially the biophysical data they collect. Maybe even anonymised, aggregated socio-economic data (subject to digital data privacy laws).

But even without a formal system of credit exchange, philanthropic donors can still ask the hard question.

The problems we face — mental health, superintelligence, climate change, biodiversity loss, income inequality, creeping authoritarianism — are too big for anyone to solve alone. We can only solve these by working together. Yet, radical collaboration of the sort the market economy produces is rare.

Sometimes, it's just the friction of dealing with other human beings. It’s not even a lack of intent. So we need a system to incentivise this to make the dealing with humans worthwhile.

Even if we NEVER get to a formal system (though I like to believe someone will come up with one), it’s a question every donor should be asking. “Who are your suppliers and users?” or “How do you pass on the baton?

Subscribe now

There are pros and cons to bringing treated wastewater into lakes generally.

First, the pros.

1. Lakes recharge groundwater

2. Residents like seeing the lake full of water.

Second, the cons.

1. Lakes might get eutrophic if there are excessive nutrients

2. A full lake has less “room for the rain”.

But let’s back up a moment to understand the implications of filling lakes with treated water and what treated wastewater even is.

The outcome depends on the quality of treated wastewater — is it secondary or tertiary treated.

Wastewater is produced due to human activity in domestic settings. Water is an input for a lot of our activities such as cleaning, washing, diluting, catalysing and producing a host of other reactions that water enables. Through all of these activities, clean water gets polluted and is called wastewater. It is important to separate the water from the pollutants before we release it back into nature. This is done in Sewage Treatment Plants (STPs).

STPs may do primary, secondary or tertiary treatment. Primary treatment removes large solids and settleable particles. The effluent released is still quite polluted. Secondary treatment uses microorganisms to break down organic matter. The output is much cleaner water, but it still contains nutrients and pathogens. Tertiary treatment removes the remaining contaminants; the output water is suitable for reuse or safe discharge.

Most BWSSB STPs were traditionally designed for only for secondary biological treatment reducing organic matter. The treated water meets older discharge standards, but still contains nutrients, pathogens and fine suspended matter. Although BWSSB is upgrading many plants to meet stricter nitrogen/phosphorus norms through tertiary systems.

When secondary treated wastewater enters lakes like Sankey Tank it gets eutrophied.

When secondary treated wastewater, rich in nutrients is released into a lake, the nitrogen and phosphorus act as fertilisers. This leads to algal blooms and aquatic weed growth (usually around the edges). This process is called eutrophication.

During the day the algae photosynthesise and produce oxygen. But at night they respire and consume the oxygen dissolved in the water. So many lakes experience “hypoxia” or a drop in dissolved oxygen that leads to fish kills. No fish = no birds.

Phosphorous in particular, is a “limiting nutrient” and it never goes away.

Even low mg/L levels of phosphorus (e.g. >0.1 g/ ml) in the treated wastewater into a lake, can trigger algal blooms. In contrast, a typical secondary treatment plant releases Total Phosphorous levels of 20-50X that limit. Even tertiary treated STP effluent can have P levels that are 2X the limit.

Now sewage contains both nitrates and phosphates. Both these elements along with carbon and hydrogen are necessary for algae to bloom. But phosphorous is a bigger nuisance.

The difference is that nitrogen exists in a gaseous form (indeed 80% of the earths atmosphere is nitrogen gas), so any nitrates in lakes can be broken down by “denitrifying” bacteria and released back into the air. Phosphorus on the other hand, has no gaseous state. So the only place it can end up is in the lake sediment. But it means that it NEVER goes away. At anytime it can be released back into the lake water. In fact, once a lake accumulates phosphorus in its sediment, it becomes hard to “undo” because the lake effectively builds its own internal source of nutrients. The sediment itself becomes a large store of “legacy phosphorus”.

Even if you stop new input of phosphorous at a later date (by upgrading the STP), the system can keep feeding itself for years, even decades. Small releases of legacy phosphorous can trigger disproportionately large algal blooms.

The ecological function is determined by the “trophic” state of the lake.

Total Phosphorus (TP), usually measured in mg/L or more commonly µg/L (micrograms per litre) in lakes, is one of the main indicators used to classify the trophic state of a lake. It indicates, how nutrient-rich and biologically productive a lake is. A eutrophic or hyper-eutrophic lake is in poor condition.

100 µg/L or 0.1 mg/L is the absolute limit of TP in a lake for it to not become hypereutrophic.

This phenomenon is seen across lakes like Bellandur Lake, Varthur Lake, and even better-performing ones like Jakkur Lake. You see a spectrum of persistent algal blooms, foam/froth events (especially Bellandur, Varthur) and lakes that remain perennial but degraded.

Algal blooms, caused by eutrophication, are bad for ecological and human health reasons.

Ecological destabilisation

Algal blooms aren’t just “extra plants in the water”. They destabilise the entire lake ecosystem and create ecological, health, and economic problems. The severity depends on the type of algae, but in nutrient-rich lakes (like many urban ones), the impacts are often serious.

Source: Algal bloom in Rachenahalli lake. Photo Credit: Girish Varma

Public Health:

Some algal blooms (especially cyanobacteria / blue-green algae) produce microcystins and other toxins. It is actually unsafe to breathe the air around such lakes.

Sankey Tank is special. We should not “muck it up”.

In general, I have been supportive of bringing treated wastewater into lakes but not, in particular, at Sankey.

But in other lakes, there has been a long legacy of raw sewage. This is simply not the case with Sankey Tank.

The argument against this is three fold:

1. Sankey has always been a drinking water lake. It has never had sewage. A quick comparison of water quality in different lakes shows Total P is Below Detectable levels in only two lakes (Lalbagh, which receives tertiary treated wastewater, and Sankey Tank). All other lakes have Total Phosphorous levels well in excess of the 0.1 mg/l needed to avoid eutrophication.

Source: Primary Survey done by British Geological Survey in 2017, jointly with IISc and ATREE. I was a co-PI on this project. https://www.sciencedirect.com/science/article/pii/S2589915522000037

Once large amounts of Phosphorous rich treated sewage enters, it is near irreversible. Why would we do this to a lake that is one of the rare lakes in Bengaluru that has no legacy phosphorous?

While it is technically possible to ensure that all treated sewage is always scrubbed of phosphorous, in practice, it is rarely the case in Bengaluru. Often the O&M of the STPs gets neglected over time as political priorities shift and payments to the operators gets delayed or stops. So once a lake gets treated wastewater it will always be full but if the treated wastewater is nutrient rich or partially treated even for a short period of time, the lake will never be the same again. This process is near impossible to reverse (without fully drying out and dredging the lake sediment).

2. In the case of Sankey tank its mostly the groundwater that sustains the lake, not the other way around. Unlike peri-urban lakes like Jakkur and Kaikondrahalli where the water table is very deep, the groundwater table around Sankey Tank is quite shallow. So the benefits from recharge are modest.

3. Rainwater enters the lake as freshwater only during the rainy season. Treated sewage in contrast is year-round. So the lake will no longer have room to capture and store freshwater locally. Depending on how encroached the downstream drains are this can lead to flooding.

   Further, in the specific case of the proposed STP, I am told, it will entail cutting a lot of old growth trees.

Now against these risks, what remains is the dubious pleasure of seeing the lake full of water year around. The risk is it could easily be thick green sludge.

Additional Reading:

https://citizenmatters.in/bengaluru-bellandur-lake-fire-causes-prevention-system-maps/

https://citizenmatters.in/why-are-bengaluru-lakes-green-in-colour/)

Subscribe now

Today, it is about 15 hectares in size, fed mainly by rainfall, and, like many of Bengaluru’s lakes, functions more as an urban park and recreational space than as a water supply source.

The tank/lake (I will use the terms interchangeably) has started to dry up this year. This has given rise to many theories.

Source: Picture of Sankey Tank on 27 April 2026. Photo Credit: Mahesh Makhija

It has never been this dry in my memory, so I looked at the last 20 years of Google Earth imagery, and the panic in the Residents’ Welfare Association seems justified. The lake level has gone down before in the summer, but not as much as this.

(Caveat: I was not able to find April imagery for previous drought years, so I may be mistaken).

What’s at stake here?

If you know Sadashivnagar, you will know it is also home to politicians of every stripe — the CM, Deputy CM, Home Minister, and MLA all live around this lake. So political stakes are high, and there is plenty of finger-pointing. The debate is this. If this is a temporary situation, when can we expect relief? The local leaders argue that the channels into the lake have recently been cleaned. So the lake should receive stormwater with the first heavy rains.

Local residents are worried that a permanently dry lake will attract encroachment (and declining home values and quality of life in general). Is this a harbinger of a longer-term trend? If so, should something be done? The most likely solution is to bring treated wastewater into the lake. This would entail a large investment of tens to hundreds of crores. But is it what we should ask for?

The challenge with treated wastewater is that it’s often replete with nutrients, nitrogen and phosphorus in particular, because most Sewage Treatment Plans (STPs) in Bengaluru do not scrub these nutrients. These nutrients cause lakes to eutrophy and develop that thick green sludge. (You can watch a video on what excess nutrients do to lakes, and this podcast on the science of Bengaluru lakes to understand more.)

Thus, there is a downside to hasty action. So let us go about this systematically.

Where do Bengaluru Lakes get their water from?

First, some basic physics. Lakes were formed by building earthen dams on natural streams. So lakes are naturally at the lowest point in any given area, and therefore, all water drains toward them.

Second, lakes in Bengaluru get water from only five possible sources.

• Rainwater that falls directly on the lake. This is not much. In Bengaluru, annual rainfall is about 0.75 m (750 mm). But evaporation is about 1.6 m (1600 mm). So, if a lake were dependent on rainwater alone, it would dry up very quickly.

• Stormwater through channels (what we call rajakaluves). This brings excess runoff (and some pollutants) from surrounding catchments. In the case of Sankey, the catchment is relatively pristine, with Sadashivnagar parks, Aranya Bhavan, and the IISc campus. But by definition, stormwater is only generated during heavy rain events.

• Groundwater. If groundwater levels are shallow, groundwater seeps into the stream and the lake. This process, called baseflow, is a major reason streams like the Cauvery and the Krishna continue to flow many months after the rains have ended. There’s no mystery here. Water seeks its own level. So it will flow from the lake to the aquifer or from the aquifer to the lake, depending on which level is higher.

• Raw sewage. In many lakes, there is often an inlet that brings raw sewage or greywater from some unsewered neighbourhoods.

• Treated sewage: Many lakes in Bengaluru have 'lakeside sewage treatment plants’ that release treated wastewater into the lake, sometimes via a constructed wetland. (Sankey isn’t one of them.)

Why is Sankey Tank drying unusual?

Most lakes in South India dry up in the summer. In fact, most Bengaluru lakes were also traditionally seasonal. They harvested stormwater during the monsoon for use by cattle, for irrigation, or for drinking in the dry season.

So, before we understand why this particular lake is drying, it’s important to understand why this question itself is unusual. How come the Sankey tank did not dry up in any of the previous years?

If a lake is full year-round, it generally means it is being sustained by sewage or groundwater. Sankey Tank has historically been quite full year-round, with modest drops during the dry season. In many previous years, though, we have received copious rainfall in April and May, so often we don’t even see the modest drops.

Something must have been feeding the Sankey tank in the dry season.

Sewage is the most obvious candidate. As cities urbanise, streams start seeing trickles of sewage. Unlike rainwater, which occurs in the monsoon, sewage is generated every single day — after all, everyone has to drink, bathe and poop every day. Many city lakes, such as Ulsoor, do in fact receive sewage. But as far as I can tell, Sankey has not been receiving much sewage or sullage. Historical satellite imagery shows a small, consistent algal bloom near the swimming pool end of the lake.

Even if the tank was receiving a small amount of sewage directed towards the swimming pool (maybe from a leaking soak pit), it could not have been significant. Most lakes that receive treated sewage experience algal blooms at the inlets rather than near the outlet, as seen in this case. The lake as a whole has been largely clear, with relatively few fish kills, and algal blooms have never spread across the entire lake.

Source: Google Earth Historical Imagery

The absence of pre-monsoon showers and hotter-than-average March and April is another possibility. Maybe Bengaluru was receiving pre-monsoon showers in March and April, and the lake got replenished before the peak dry season. So we just didn’t notice the drop because levels recovered early in the summer. Indeed, in 2023, Bengaluru was flooded in April!

Source KSNDMC data, Yelahanka rain gauge.

From the figure, it is clear that although we have not received much rain in the dry season this year (2026). There have, however, been prior years with little to no rain in Jan-Apr, during which the tank water levels did not drop as much. So it could not just have been early rains replenishing the lake.

Groundwater is the third possibility. It is very possible that Sankey Tank was, to some extent, sustained by groundwater in the dry season. Although groundwater levels drop in the summer, they often don’t drop as quickly as the lake, so the aquifer sustains the lake in the dry season.

I have not personally analysed the directionality of recharge for Sankey. But analyses we have conducted for other lakes in Bengaluru have yielded mixed results. Peri-urban Lakes like Jakkuru, where groundwater levels were low (at least pre-Cauvery supply), always recharged groundwater year-round because the groundwater table was much deeper. Other lakes, like Kasavanahalli, however, exhibited a two-way exchange. The lake recharged the aquifer in one season but received groundwater in the other season.

Could groundwater theoretically sustain Sankey Tank, though?

Source CGWB data, Malleswaram

An examination of groundwater levels in the Malleswaram monitoring well suggests that groundwater levels have been quite shallow (just 2-6 m below Ground). So it is possible that groundwater historically contributed to the tank in the dry season.

As a side note, to understand why water levels are so shallow here compared to other parts of Bengaluru, we need to understand the sources of recharge. This is the old part of the city, which receives a lot of Cauvery supply (being politically connected and whatnot). So on one hand, groundwater extraction is low. On the other hand, these very Cauvery pipelines leak a LOT. It is estimated that almost 20% of the piped supply leaks physically into groundwater. In fact, between a third and a half of the city's groundwater recharge originates from leaking pipelines. (This has been independently confirmed by modelling studies by IISc and isotopic analyses by the British Geological Survey, which showed that groundwater in central Bengaluru resembled piped supply much more than rainfall. My own PhD thesis in Chennai concluded a similar result.)

In summary, in the city centre of Bengaluru, pipelines leak, recharging groundwater. Water levels are shallow, and the aquifer thus sustains the lake.

What theories can we rule out?

Now, as we dug deeper into the mystery of the drying tank, several theories were propounded by city engineers. Some of these we can dismiss right away as impossible or implausible.

1. The wells at the northern end of the lake were full of silt. This was blocking the channels into the lake.

   This is impossible. Groundwater seeps into a lake through thousands of pathways, myriad cracks, and fissures. So, a single dug well getting filled with sand cannot do much damage.

2. There are trees at the northern end of the lake sucking up all the water. Tree roots can take up a lot of water, but certainly not enough to drain an entire 15-acre lake. Besides, these trees have been there for years. So why this year?

3. There was sewage coming in previously. But that has now been diverted. This seems unlikely given the reasonable quality of the lake's water.

What theories are plausible?

So we are left with three remaining theories.

Theory #1: There is a crack in the outlet weir. An outlet weir (often called a surplus weir in Indian tank systems) is a fixed overflow structure built into a tank bund that allows excess water to safely spill out once the water level exceeds a set height. It thus maintains a maximum safe water level in the tank and automatically releases any surplus water into a downstream channel or another tank when the water level exceeds that maximum.

If the outlet weir develops a crack (as one lake engineer claims), it might leak water into the downstream channel, thereby draining the tank. This should be easily verified by examining the weir (perhaps with an underwater camera) and visually observing the downstream channel for a continuous trickle of water.

If this is the case, and the evidence is clearly established, the outlet weir needs to be fixed.

Theory #2: A stormwater drain was diverted somewhere upstream. It is plausible that Sankey did not receive its usual inflow during the last monsoon because a stormwater channel somewhere upstream was diverted.

This would require some discussion with the GBA stormwater department regarding any new projects undertaken in the neighbourhood. Perhaps crowdsourcing photographs taken by residents in Oct or Nov across various years could help establish whether the lake did not fill as much last year. I cannot tell from satellite imagery.

If this is the case, the stormwater channel connectivity should be reestablished.

Theory #3: Groundwater levels may have dropped as well this summer, reducing the sustaining flow seeping into the lake. One possibility is a sharp increase in groundwater pumping in the neighbourhood as “densification” increases, with single-family homes being replaced by high-rises. Another possibility is BWSSB leak reduction programs that might have fixed leaking pipelines.

If Theory #3 is the case, it’s less easy to fix. Because it means that the problem is a longer-term one. Lake levels might drop every summer if the rains don’t come by April. It should be noted that long-term climate trends indicate an increase in pre-monsoon rainfall, so this may not even occur every year.

In this case, the RWA has three solutions:

• Aggressively invest in recharge by diverting flood waters to “sponge parks” around the lake.

• Divert high-quality treated wastewater from a nearby wastewater treatment plant into the lake, but accept the attendant risk of completely contaminating the lake) and the surrounding groundwater.

• Accept that the lake will be drier every summer from now on.

My personal recommendation is to

1. Check the outlet weir and the downstream channel for a leak.

2. Crowdsource photographs from Sadashivnagar residents taken at different times of the year over the last 5 years to reconstruct lake levels.

3. Inspect and map the inlet channels, and then wait and watch for the rains.

4. Map the open wells around Sankey and, in cooperation with the RWA, monitor them through a citizen science program or by installing piezometers to track groundwater levels.

Donors and partners keep asking us a version of the question: “How do you keep track of whether you are making a difference?” I thought it might be worth laying out our thinking so far and where we have reached.

Subscribe now

I have also written earlier on some more philosophical thoughts on “how to measure progress on systems change if it takes a decade”. Thinking about this in the abstract is very different from operationalising it.

In arriving at a coherent philosophy, the recent slew of articles from donors has not exactly been helpful. The messaging is all over the place. Scale matters. No, it doesn’t. Addressing wicked problems needs a systems approach. Systems change can’t be done. We must build ecosystems for systems change.

Every donor has a different view, and unless someone is willing to fund us 100%, I find we can’t tailor our PoV to suit any given foundation. We have to have our own. For a hot moment, we considered hiring a dev sector consulting firm to help us, but we thought we were not ready for it internally. Our own thinking was too incomplete and muddled. We felt it would be a waste of our time and theirs.

However, funders have continued to ask us this hard question. Not always in a negative way, sometimes, just to challenge us to think.

Why should anyone give us money? After all, we do not generate a financial return on investment but rather a social one. If we do, we should be able to tell them the social return on investment, right?

First, there are funders, and there are funders.

To understand what matters to donors, first let us recognise that not all are the same. Within WELL Labs, we classify funds we get into the following categories.

1. Unrestricted funds: These funders are rare, but obviously the best of all. They write a check and say, “We trust you”. Some are explicitly OK with us using a greater portion of the money for administrative expenses.

2. Strategic programmatic funds: These funders fund outcomes. They are flexible in terms of the day-to-day of how we achieve it. They do want to see outputs and outcomes, but are flexible on the approach and how we spend the money.

3. Project grants: These funders fund outputs. They are flexible in how the outputs are produced and have some leeway on timelines, but that’s what they measure us on. Many CSR and ESG funds fall into this category.

4. Fee-for-service contracts: These are the most restrictive of all funds and are actually straight-up consultancy contracts. They have relatively rigid processes, milestones and timelines with strict deliverables. Till a year or so ago, they were subject to GST. But new rules waive non-profits from paying GST for research projects.

Each of these funding types (yes, even the restricted ones) has advantages and disadvantages. The first two tend to be funded by philanthropies; the third by CSR donors; and the last by private firms, foundations, or government. Since foreign currency grants cannot be sub-granted in India, any collaborative work whose funds originate abroad has to be #4.

CSR funds and contracts tend to be output-focused given their short-term horizon. So the onus of ensuring we test an outcome hypothesis rests with us. But the funder itself will rarely castigate you if the hypothesis doesn’t pan out. Say you are conducting training to achieve the adoption of a practice. Many CSR donors will be satisfied with the number of trainings conducted. Very few will re-engage after a few years to ask how many people are still practising. This, of course, behoves us to ensure that the programme is designed to maximise actual adoption, but it is self-directed rather than donor-mandated.

One important distinction between contracts (#4) and grants (#1-3) is that non-profits may earn a profit on contracts, which can contribute to Reserves. The only caveat is that Indian Income Tax Rules require that no more than 20% of annual revenue be earned through such contracts. For grants, salaries billed must reflect the actual CTC, and double-billing is not allowed, so people cannot be billed for more than 100% of their time. Under a contract, a person can be billed at a rate higher than their salary. This means contracts can be a useful way to add to reserves.

Different types of funders have different internal motivations and therefore require different metrics.

High-net-worth individuals

High-net-worth individuals tend to be whimsical and heterogeneous. They are giving away their personal money, so they need to feel good and feel like they are leaving a legacy for posterity. These donors need to believe in the cause and connect to it personally. They need to have faith in the theory of change and trust the leadership’s ability to deliver. Depending on the person, the story and evidence of impact are critical.

Foundations

Philanthropic foundations’ investments tend to be “catalytic”. They realise that their spending is limited compared to that of governments and CSR funds in India. As a rule of thumb, across sectors, the ratio of Foundation:CSR:Government spending is 1:10:1000.

Philanthropic unrestricted or strategic investment in WELL Labs results in one of three outcomes: Knowledge (reports, papers, products); Trust and Relationships (government, grassroots partners, coalitions, start-ups); and Pilots (actual projects that test specific hypotheses).

Ideally, all of these are strategic in the sense that we have a very specific sense of how a knowledge product will shift a narrative or conversation. And we tend to do work that is either urgent (because a policy opportunity has come up that cannot wait for a donor or project) or unfundable (it is too early-stage or too risky).

CSR donors in India

Most CSR donors are very risk- and compliance-focused. They are mandated by Indian law to spend 2% of after-tax profits on social causes, and their first and most important goal is to avoid legal trouble and reputational risk. The second is to get some positive brand recognition. Impact is a distant third.

ESG funds (Indian and foreign)

ESG funders are audit-focused. If they are claiming carbon or water offsets (e.g. Volumetric Water Benefit). So they care very much about quantitative proof.

Indicators depend on the stakeholder and the questions they have.

The measures donors use depend on their own motivations.

Compliance Indices answer the question “Is it safe to invest in WELL Labs as an organisation”?

These are just basic indicators of fiscal health and governance.

Operating Reserves: Does the organisation have at least 90 days of Reserves? Additionally, IFMR requires us to track and report on Accounts Payable every quarter.

Transparency: Does the organisation have all of its Annual Reports and Filings up to date and readily available?

Governance: Does the organisation have a diverse and independent Board?

Internal Indices answer the question “Is it worth investing in a specific WELL Labs programme?”

Some donors have asked how we know we are spending effectively.

Cost per output: How much does it take to produce a report/ an app, a policy brief? Because our timesheets are linked to Key Results, I hope we will be able to produce these starting next year.

One challenge I expect here is the lack of benchmarks. Does a peer-reviewed publication cost 5 L or 50L? I think most academics would argue this whole range is reasonable, and it depends on the complexity and novelty of the research. I think we are looking more for indicators of where things look obviously off. Did we spend 20L and just produce 1 blog? If so, we might want to reflect on why our outputs are so lean.

Funding leverage: A 10× leverage effect, every ₹1 of “catalytic funding” unlocks ₹10 of public funding. This is widely used by donors, development banks, and philanthropy, and is intuitive to understand. It is somewhat easy to track at the programme level, especially where we work directly with the government. It’s harder to arrive at when multiple partners are working in a consortium, or when the path of influence is a little less direct.

Social Return on Investment: Measuring Social Return on Investment (SROI) is less about a single formula and more about a structured way to link our activities to meaningful, monetisable outcomes. ‘

We haven’t made much progress in systematically tracking this across the board, but we did some preliminary calculations for Chintamani to estimate the social benefit of an early catalytic investment in data and pilots. Interestingly, the biggest social benefit to residents from clean lakes and treating wastewater to standard came from improved housing prices. I don’t think we will do this across WELL Labs, but we plan to run the estimates for two programmes in the coming FY.

External Indices answer the question “Has society measurably gotten better”?

Societal stakeholders (government and civil society) will ask this question — do you think you are making a difference? They give two hoots about the efficiency of funding use.

In the last 3 years, we have spent a lot of time on the question of whether we are “really truly moving a needle”.

Reflecting on this, we realised that there is a big difference between “did this intervention work?” and “did the region become water secure and water resilient?”, which relates to the scale of the problem and the intervention.

One way to think about the difference is through a more familiar health analogy. A doctor might check your temperature and prescribe a course of antibiotics. A week later, you might conduct a follow-up to determine whether the infection has subsided — this is what traditional Impact Assessment or “Monitoring, Evaluation and Learning” aims to achieve: evaluating the efficacy of a course of treatment against predetermined indicators.

Monitoring, Evaluation and Learning (MEL)

MEL is embedded in much of the work we do at WELL Labs. A key component of MEL is ensuring that measurement is embedded within any implementation of pilots or solutions that we undertake.

WELL Labs has been working on so-called “hydrologic impact evaluation” approaches. We have found that while socio-economic impact methodologies (RCTs, quasi-experimental, outcome harvesting) are well established, biophysical measurements are less common.

To this end, we have been compiling a collection of tools and approaches for data collection and for setting up control and treatment sites that account for biophysical similarities and differences. Some of these can be found in our MEL Toolbox.

An important aspect of this is the time and scale of the impact assessment. Sometimes, the interventions work in wet years, but not in dry ones. Or they work at the farm scale, but there simply is not enough adoption to make a difference at the watershed scale. So the design of MEL is quite tricky. We have seen donors expect too much, too soon. So expectation setting is crucial.

Furthermore, a key lesson from Impact Assessment is that, for water (unlike many social indicators), continuous monitoring is often crucial. It’s very difficult to conduct pre- and post-analyses if the project started and ended in the summer, but the change was expected to occur during the monsoon. But continuously collecting data requires sophisticated systems to collect, clean, analyse, and communicate insights. This has been quite a steep learning curve for us and a not insignificant investment. It’s quite a task to get RAs to collect data seasonally, clean it, analyse it and archive it properly in accordance with Research Ethics and Digital Data Privacy Act.

We have had to invest a lot in capacity and systems — you can check out our data portal at data.welllabs.org.

Water Security Indicators

MEL works very well, provided there are no unintended consequences. Going back to the health analogy, for instance, what if the concern was that certain cardiac treatment might weaken the kidneys over time? The doctor, recognising that many systems are intertwined, might recommend fundamental lifestyle changes and ask to track “overall health over time”. You might go back for a periodic checkup to track all vital signs. Here, you might notice that your heart health has improved, but your kidneys are not doing well.

A lot of the work we do on watersheds or canal systems is slow, at scale, and takes years, and systems ARE intertwined. How do we know whether the landscapes we are working in are improving over time? Importantly, water is often a “zero-sum game”. There is only so much water available. After a point, when every last drop is appropriated for human use, we could reach a situation in which one user expands water use at the expense of another. Or drip irrigation uses water more efficiently but still increases total water use over time (so-called Jevons’ paradox). Many water interventions have inherent unintended consequences that necessitate an “overall health approach”.

In developing an index to capture regional water security, we were inspired by the ASER (Annual Survey of Education Report), which focused on whether schoolchildren across India could read at grade level.

The so-called WISER Index (Water Index for Sustainability, Equity, and Resilience) addresses a core gap in a sector that tracks spending and infrastructure (inputs/outputs), but not whether water security is actually improving. It provides an outcome-focused framework to assess whether investments are “moving the needle.”

The challenge is that water security is multi-dimensional. It spans balance, access, productivity, resilience, governance, and ecosystem health. These dimensions can trade off with each other, so WISER tracks them together rather than relying on a single metric. WISER is designed for landscape-level assessment, moving beyond anecdotal or fragmented data and uses a combination of primary and secondary data.

We have now started using WISER in our own “Transformation Labs” to track improvements in water security over time.

In Summary:

What organisations track (or should track) depends largely on who is asking the question and their motivations.

Tracking anything beyond the simplest fiscal indicators requires systems in place and a nontrivial investment in data and monitoring, evaluation, and learning.

As a systems change non-profit, we can do nothing on our own; change happens out there. The only way we can move a needle at scale is through radical collaboration.

Mixing metaphors here, maybe, but in a scale game, non-profits are dot-connectors. They do not drive scale themselves. Rather, they create conditions for scale to emerge by other actors — the private sector, individual farmers, consumers, Resident Welfare Associations, Water user Cooperatives, Government Agencies, etc.

Given this, everything we do is done in partnership. If we want to succeed, we had better get good at it!

Subscribe now

Systems are stuck in low-level equilibria due to lock-ins. Changing them often requires moving multiple levers simultaneously.

Often, water systems are stuck in unacceptable equilibria due to lock-ins, in which farmers, institutions, or consumers become locked into a particular technology, practice, or resource-use pattern, even when better alternatives exist, because changing becomes too costly, risky, or politically difficult. Such lock-ins are a form of path dependence: once a system moves in that direction, it becomes self-reinforcing.

Lock-ins frequently involve many reinforcing factors that keep the system stuck in an undesirable state. For instance, farmers grow paddy because inputs, seeds, machinery, loans, even the way water flows from field to field are all oriented to paddy, making it difficult for any one farmer to switch to a different crop. And it’s too difficult for 400 farmers in a village or command area to switch simultaneously to something else without inputs or buyers in place.

Transforming the whole system requires multiple actors (who don’t even know each other) to change their behaviour in lockstep. A systems change player would need to use a combination of levers (capacity building, incentives, de-risking, policy change) to “lock out”.

Importantly, the different levers require different government agencies and completely different players (SHGs, Agricultural Credit Agencies, etc.).

To move systems, non-profit actors must operate on principles of collaboration, not competition

Once we recognise how many diverse actors are involved in systems change, it is immediately clear that no one organisation can do it alone. It might involve a dozen or so organisations, all acting together. This involves a mindset shift from competition to collaboration.

Ironically, non-profits are sometimes worse at collaborating than for-profits. For-profits “collaborate” every day with lenders, vendors and customers negotiating complex global supply chains, all using the “invisible hand” of the market.

What is the equivalent of the invisible hand in non-profits? How do we seamlessly pass the baton from one organisation to another? How do we find the courage to valorise each other, getting over petty differences and jealousy?

To be honest, we don’t have great playbooks or even examples of this. Many non-profits are still in the mode of tearing each other down.

But a few best practices are emerging, and we are trying all these modes in different contexts and programmes within WELL Labs.

1. Fundraising for a place-based ecosystem: This is where 1 anchor organisation raises funds and disburses it to a group of organisations (some could be service providers, and community-based organisations) all working in one landscape. Personally, I am not a fan of this model. Often, the anchor organisation has disproportionate power, and it undercuts the ownership and agency of the others.

2. Collective action platforms: Multiple organisations come together to work in a landscape. They present a common vision, use similar language, but each has clearly defined, distinct roles. The funding is not centralised or disbursed by an anchor organisation. Rather, each organisation fundraises for itself. Some may be for-profit, others CBOs with members. But they collaborate based on a well-defined common agenda.

3. Mutual benefit consortia: Groups of organisations that have a common interest and goal. They meet frequently to brainstorm and exchange ideas. They boost each other on social media and pass on leads or introductions. But otherwise, they operate independently, with no money exchanged.

4. Servicification: Organisations offer services to other organisations for a fee, akin to a private sector transaction. The fee may be paid by the partner org (with associated GST implications) or be paid for by a common donor.

We use an interest - influence matrix to map actors and decide how to approach other actors.

In the previous section, I described only partners who are willing participants in the coalition for change. With corporate partners or start-ups, the rules of the game are clear; they will engage when they can make money. But often the need is to move a bureaucrat who has zero interest in creating extra work for themselves.

In such a case, the only way forward is to figure out how to influence these actors or at least engage with them meaningfully. One tool to map external actors is the interest–influence matrix (also called a power–interest grid). It is a simple tool used in stakeholder analysis to map actors based on how much they care about an issue (interest) and how much ability they have to shape outcomes (influence/power). It helps you decide who to engage, how, and how intensively.

Specifically, a structured approach allows us to determine how to act with government partners.

At WELL Labs, we have mapped the key stakeholders we work with in each program. We should be updating these more frequently than we do. Realistically, it doesn’t happen more than once a year.

To persuade reluctant actors, we need to develop deep empathy, people skills, and the ability to read the room.

While it’s all very well to have a theoretical map of actors, including government agencies. In practice, engaging and consulting with diverse actors requires people skills. Often it requires understanding individual officers, their habits, motivations and aspirations as well as those of their bosses and reportees.

We have found that we need to have contacts at at least three levels within a government agency with matching contact persons within our organisation or consortium. Having the top boss's buy-in helps, but we run into resistance from staff who have ingenious ways of disappearing files.

Convincing people to try new things involves people skills. It’s crucial to secure buy-in from well-meaning staff (by ensuring they are recognised for their efforts), apathetic staff (by making their lives easier), and even corrupt staff (by ensuring they at least don’t throw a spanner in the works). If you treat agency staff with contempt, the game is over. But often, we are never taught the value or art of diplomacy. Mostly, humans are human. Complaining against them or yelling at them is rarely effective. On the whole, we need them more than they need us, so getting in their good books is critical.

When it comes to moving people, sometimes we succeed. Sometimes we fail, then we go back to the drawing board. Grit is key.

What we realised is that we need to help the team build skills in empathy, difficult conversations, negotiations and reading the room. Early-career staff (or even senior ones) often do not know how to recognise when a different approach or argument is needed and pivot gracefully. They are so consumed in their prepared talking points that they don’t know when to stop and pivot. The only way we have figured out to teach this skill is through debriefs with the team after an important meeting, to help understand how the human dynamics in a crucial meeting played out.

We need to find new allies and forge new coalitions.

One of the ways we have been most successful in effecting change (or beginning to move a needle) is by forging new coalitions of actors. Often, these are new actors who did not exist before, with influence and a direct stake in the desired change. The Bangalore Apartments Federation (BAF) and the Federation of Bengaluru Lakes (FBL) are two interest groups: the first representing 1,000+ Resident Welfare Associations in the city, and the second representing 70+ local citizen lake groups. The existence of these groups creates a platform and an influential lobby that politicians will listen to. Now, the group's interests may not fully align with our own. So these are issue-based coalitions. For instance, BAF may not care about slum dwellers. But like us, they do care about enabling decentralised wastewater reuse.

We need to learn to manage relationships beyond the transactional. This requires systems to sustain trust.

Beyond frameworks and skills, the big challenge is how to embed this in the organisational DNA.

There are several components to this. Some are cultural. How do we write up MOUs and contracts? Beyond this, how do we set expectations and check in on the health of the partnerships? How do we model respect and honesty with our partners?

The problem is (as an old Dutch proverb goes) "Trust arrives on a tortoise and leaves on horseback". In other words, trust takes a long time to build through small, consistent actions, but can be destroyed instantly by a single betrayal. So we need systems to build these small actions and correct for mistakes.

We often make mistakes. After all, we have had tremendous growth and are operating, as one WELL Labs staffer put it, “with one foot perpetually on the accelerator”. Sometimes we forget to invite someone to a meeting or give credit where it’s due. These are rarely acts of commission. Either we failed to set expectations early on, or we did not institute systems to ensure that the relationship aspects would be sustained.

On one occasion, we received a report co-funded by a foreign partner, an Indian donor (Arghyam), and an NGO partner (FES). The foreign partner operated on a Jan-Dec timeframe and the Indian donor on a April to March timeframe. Around February, the US-based partner asked us to release the report so they could show progress to the foundation that funded them. We had gone through several rounds of comments from all partners, but we had not yet received final sign-off from the two Indian partners. Our team, facing immense pressure, decided to release the report to keep our US partner happy.

I am very grateful to Manu Srivastav from Arghyam for calling me and conveying how disappointed he was. I would never have known, and an important strategic relationship would have been lost to us. Often in such cases, we never simply pick up the phone; we just fume silently. I am so grateful that he called. On such occasions, the only way is to apologise, communicate our intent and explain why the mistake happened and how we plan to fix it. We withdrew the report and reposted it after all comments had been addressed, this time in coordination with the Communications teams of all three entities.

Sometimes, it was the result of broken communication between two organisations. As an example, on one occasion with a donor, I realised 8 months into the project, we had completely different interpretations of the phrase “site-suitability”. For us, it meant, “Is the intervention (say check-dam) built at the right spot?” So “site” meant the specific physical location within a watershed. For her, it meant, “Is this even the right intervention for this watershed? She was a CSR donor with many factory locations, so for her, each watershed was a 'site’. Her concern (legitimate) was that grassroots NGOs were just checking the box, blindly doing the same thing everywhere. She wanted a justification of the whole solution approach for each of her “sites”.

Initially, we kept submitting reports justifying the locations of the check-dams, but she kept yelling at us for failing to meet the deliverables. In her case, the anger came from a deep distrust of NGOs and her belief that we were all useless. Our team was just in tears all the time because they were coming up with increasingly elaborate models to explain how check-dams, farm ponds, or soil-moisture locations were chosen. They simply could not understand the vitriol they were experiencing.

Once we understood each other, we could all calm down. I appreciated her pushing us to a design-thinking framework to justify the choice of interventions. It was a reasonable ask, but we had not budgeted for it in our original proposal. It required much deeper thinking and engagement, and we had to work much harder. The team eventually came to appreciate it as well, even forgiving her for the earlier negativity. But it would have been so much easier if we had paused earlier in the conflict to understand the root cause.

The hardest aspect of relationship management, though, is maintaining the relationships beyond the transactional. Occasional catch-ups, phone calls, or at least Diwali and New Year wishes. One suggestion that Ranjeet Ranade, who works with us as an Executive coach, suggested was to maintain a Relationship tracker. Where each staff member maps out their next interaction and timeframe for each key relationship they own.

In summary:

Almost the most important part of our job is not technical; it is relationships with people. Some like us, some are indifferent, and others would just like us to go away. Systems change is impossible without frameworks and systems to manage these.

Subscribe now

Culture fit must be assessed during hiring, onboarding, and confirmation at the end of probation.

I have already explained the centrality and importance of organisational culture at WELL Labs. Culture, of course, cannot be sustained without embedding it in all HR processes.

So this has meant our hiring process needs to check about how people manage conflict, their comfort with hybrid work, etc. Our onboarding process needs to emphasise culture and values. And, importantly, the probation-end confirmation processes need to explicitly check for culture fit. If it is clear that a person will not last in the organisation, it is better to part ways early.

We made this mistake early on, after struggling to understand why seemingly capable people were unable to deliver what we needed or work in the team structure we expected.

Source: https://mljar.com/blog/navy-seals-performance-vs-trust-and-ai/

Attrition is a part of the game in creating a high-performance, high-trust organisation.

WELL Labs is not for everybody. We have a great social media game, and our employees are generally happy. For many young people, it feels like an aspirational organisation. But the reality is actually mixed. Many people are simply a poor fit. We actually have a 20% attrition rate. About half of the people who leave us each year leave for higher studies and better opportunities. The other half we have to let go (or they resign) because we both see it’s not working.

I do not want to sugarcoat this. It is extremely hard to create a high-performance, high-trust organisation. It is easy to be high-trust by being lax on performance. And it is easy to be high-performing and toxic. Balancing the two requires a lot of tough love. It’s been hard for me. As a woman leader its easy for me to be “motherly” and nurturing, but hard to call bullshit. I still have to really work on having difficult conversations. But I have grown to see that it’s unfair to others if I let a few slack off.

First, we have very high standards of accountability. If people do not deliver, it’s unfair to expect others to carry the load. We try hard to be empathetic, but at the end of the day, delivering impact (not just outputs) is what matters. In the past, we have had employees who were completely checked out, were moonlighting, or simply lacked the knowledge or skills to perform at the level we expected. While we might have conversations with them about performance, in the end, letting people go, or even helping them find a job elsewhere, was necessary. It took me a while to see attrition as a mixed bag. When people leave, it creates a vacuum but also an opportunity to bring in fresh blood with a better cultural fit.

Employees often struggle with skills. Sometimes, they negotiate hard for a high salary, but this is a double-edged sword. In WELL Labs, salary is linked to a JD level. So asking for a higher salary = asking for more responsibility and promising a higher skill level. And sometimes people are just not ready. I truly believe that all humans have endless potential if they have a growth mindset. So sometimes the answer is “Not yet”.

But realising growth means providing mentorship to help people see where they were not levelling up. In some cases, the failure was mostly on our end. E.g., in a few cases, junior people reported directly to very senior people, who simply didn’t have the bandwidth to mentor them properly. Sometimes people were simply a poor fit for the job. In other cases, we tested people on technical skills, but we failed to realise the job required soft skills that the employee didn't yet have. We failed these staff by not pushing back earlier when they asked for high salaries.

Second, some staff struggle with our hybrid/remote work culture. It just doesn’t work for some people. They like the salary and flexibility, but they don’t know how to organise their day, set up a weekly work plan on Asana, and execute on it. Some were actually productive but hated the lack of daily human interaction. In fact, a couple of our best employees quit because they did not like hybrid/remote work. They craved the water-cooler conversations and the human face time at least a few times a week. Unfortunately, we do not have an in-person office in Delhi. We only have offices in Bangalore and Chennai. So we were unable to retain them.

Third, we expect people to have honest face-to-face conversations first with their teammates, then with managers, and finally with HR. We do not like staff to engage in backbiting. Now, banter with co-workers and gossip are part of life; humans are social creatures after all. The problem begins when teams try to resolve workplace disagreements without ever having the difficult conversation. Many staff have spent years in other organisations and have never developed the skill of looking others in the eye one-on-one, speaking honestly and listening deeply. However, this is key to developing trust, and we reinforce this principle every chance we get.

Finally, let’s be honest, there are many who are struggling with mental health issues; we live in an uncertain world with AI, war, climate change, corruption, and authoritarianism, and these impact our work directly. Heck, I can barely stay optimistic after reading the headlines every day and going, “Is there any point in even trying?” It’s OK to be open and honest about these struggles. At WELL Labs, we cover mental health costs, including medication, with appropriate anonymity. But even mental health has to be handled professionally. The staff member must be sufficiently functional to proactively notify the manager of downtime and delivery delays. Otherwise, it's just letting down the rest of the team, and in such cases, we have had to part ways.

We compensate based on value delivered, not years of experience. We reward productivity and innovation.

How do we assess how much it’s fair to pay an employee? This is relatively easy in homogeneous organisations. It’s complicated in hybrid organisations that involve both specialists with “hot skills” (e.g., Gen AI) and field workers with deep community relationships. In such cases, it is really hard to create a single “work spine” (a structured way of defining roles, responsibilities, and levels across an organisation, so that people, performance, and progression are aligned) in a way that accommodates a whole range of skills and mindsets, going from research to problem solving (see related article).

The challenge we faced was as follows: levels in the organisation map to JDs, which list managerial responsibilities. What we struggled with was the trade-off between taking on more project management responsibility vs. contributing deep technical know-how. After trying various combinations, including a separate management and technical track, we are reverting to a single spine. Each Level is still mapped to a JD, which reflects the level of management responsibilities. However, as of next year, we are only allowing the Level salary ranges to overlap significantly. So now a person at a higher level and title can get paid much less than a person at a lower Level who is a deep technical contributor.

We value contributions in three areas. And most staff need at least two, but ideally all three, as they grow with the organisation.

Technical Skills: Coding (including use of AI), modelling, video editing, graphic design, infrastructure design, research design, etc.

Domain Knowledge: Deep knowledge in a topic area, including familiarity with the literature, that can help contextualise and clarify.

Relationship Skills: Relationships you own, and soft skills to mobilise, negotiate, and drive consensus among diverse players.

Importantly, we value contributions. It doesn’t matter how long a person has worked or how many degrees a person has. Some grow faster, others slower.

Ultimately, for WELL Labs to be financially viable, we can justify annual salary increases (beyond inflation) only if the employee delivers significantly more value to society. This is true individually, at the programme level and as an organisation. So, how do we ensure an employee delivers more value over time? The only way to do this is to have the OKRs “stretch” commensurate with the pay.

This is hard for many employees to get used to. They get a promotion one year, but continue to do much the same work in the way that served them well last year, forgetting that the Key Results and expectations have gotten harder. Then they are shocked when we tell them they have not levelled up.

Ensuring employees retain a growth-mindset means also investing in upskilling

For an employee to grow within WELL Labs, they have to be upskilling and taking on leadership roles. Ideally, they have to do this proactively. We do this through the Individual Development Plan (IDP), a structured, written plan that outlines how a person will build skills, experience, and capabilities over time to meet their career goals and current role expectations. An IDP is essentially a bridge between where someone is today and where they need to be.

Does this work perfectly as we hoped? Not yet. The process exists, but we don’t yet have customised upskilling plans to match.

Transition Planning and “Damage Prevention” need to be proactive.

Early on, we made several mistakes. Unlike other new organisations that start small and grow, we started with 30 people, but without the systems and processes to manage the growth.

One major mistake I made in Year 1 was poor transition planning. In the first year of WELL Labs, we had three senior staff members have major personal transitions - two became mothers, and one lost a spouse. We did not proactively manage these transitions with their teams. To be fair, there was a LOT on our collective plates with three senior people out of commission. We simply assumed that everyone working for them would pick up where they left off, with very few guidelines on reporting, escalation, and related matters. We paid a high price for this. There were misunderstandings, hard feelings, and people quit.

After we started working with Ranjeet Ranade and Subha Chandrasekaran at ETC Partners on Executive Coaching, we instituted a few processes. The first insight was that unfortunate things happen, and rather than beat ourselves over it (as I am wont to do), the situation must be managed. But as an organisation, we cannot always be in “Damage Control” mode. To move to “Damage Prevention”, we needed to learn from our mistakes, to actually conduct a forensic analysis of what went wrong, process-wise, and then fix it.

Ranjeet pointed out that our weekly leadership meetings were about “running the organisation”. We were checking in on fundraising, discussing HR issues and setting agendas for quarterly reviews. A key process he had us institute was a monthly “change the organisation” meeting. This is to look at what needs to change or be managed proactively going forward. We did much better in our later leadership transitions by having a 3-6-month phased handover to the next in line, with in-depth 1-2-hour workshops with the team on who would do what.

Giving and receiving feedback is a necessary muscle.

We have realised that a culture of open, honest discussion should result in a culture of continuous review, so mid- and end-year Performance Reviews should not be the first time people receive feedback.

We have the usual self- and manager-review process, reviewing performance on both OKRs and culture (are people collaborating, are they proactive in asking for help, are they focused on impact, etc.). The reason for the latter is that we realise the achievement of OKRs depends heavily on external factors (political will, fertiliser prices, etc.). We also want to reward being for doing the right things, not just getting lucky with a policy window.

Norming: One aspect that our own staff are often confused about is the norming process across teams. Once the manager has completed the review, the leadership team meets to normalise the scores. It is very normal for some managers to give consistently higher scores and others to give consistently lower ones. So we need to ensure that they are normalised so that the expectation of a Level is consistent across WELL Labs.

Loosely, there is a bell curve that is normalised to the organisation's overall financial growth. There is no magic or secret sauce about the scores. In fact, I would say there is rarely disagreement. Higher performers own the outcome, take feedback positively and seriously, and are committed to delivering change. They are proactive, good communicators, collegial, intellectually curious, and constantly upskilling. Lower performers typically struggle with skills, need a lot of hand-holding, push back at constructive feedback, have constant “friction” with colleagues and partners, rarely ask for help, and do not communicate clearly. Importantly, high and low performers are not people. The same person can be high- or low-performing depending on the role and level. Many times, we have a discussion about a person, saying they are great, but they are operating at two levels below. In this sense, they perform poorly relative to the cohort at their level. Norming helps us see each employee as part of a cohort at a given level.

360-degree review: We have been conducting 360-degree reviews since the beginning. Each year, the process has gotten slightly better (hopefully).

The first year, we just did it in Excel. The feedback itself was weak because people just didn’t give useful feedback. Some comments were of the “she is a nice person with a good smile” variety, while others were almost insulting and not constructive. Importantly, people were upset that it wasn’t sufficiently anonymous and that they could be traced.

This year, we did it within Zoho Performance. Taking a cue from another social sector organisation that refused to share 360-degree feedback directly (because of the harshly worded reviews), we went overboard. We gave people pointers on making 360-degree reviews constructive, using professional language. Then we went a step further to “spare people’s feelings” and retain anonymity: we had HR synthesise the already anonymised, professionalised feedback (over 3 back-breaking days) using AI to generate a single summary for each employee.

This result was truly awful. Everyone’s review read the same. ChatGPT anaesthesised the feedback to the point where we could no longer figure out what it was. We eventually realised that the original reviews people had written were actually high quality, specific, useful and professional. Next year, we plan to trust our staff and the Zoho system to share the anonymised individual feedback as is.

Accountability begins at the top. An independent board that represents stakeholders is key.

All my preaching about culture would amount to nothing if I did not hold myself accountable. This realisation came to me on two fronts. A young person asked for advice about an organisation he had just joined. The head was prone to outbursts, publicly castigating junior staff. We realised that there was no check on the founder/CEO because the board was largely composed of his own friends.

This is common in many NGOs, where the board is populated by the founder's friends. Because non-profit board members in India cannot be compensated, many feel that serving on the board is already their contribution to society. The Boards rarely represent stakeholders/communities; they often see their role as primarily to “support the leadership”. Often, flaws in leaders (and founders in particular) go unnoticed or ignored for a long time.

At the same time, my own Steering Committee (SC) raised the issue of strengthening WELL Labs governance as we mature as a centre. The SC is the equivalent of our Board, since IFMR delegates governance over content and personnel to the Steering Committee. I am SO grateful to Shloka Nath for suggesting an SC-led (rather than leadership-led) process for appointing new members. This means I will have an SC that is not beholden to me and can ask me tough questions (and fire me if needed).

So are we a “world-class” organisation?

I have no idea. Every day I wake up and feel we could be so much more.

But what I have learned is that organisations need a growth mindset, too. And this requires investing in people and processes.

Subscribe now

An organisation’s culture is its DNA. It defines how it operates day-to-day.

An organisation’s culture is embodied in the stated and unstated rules, rituals, agreements, and processes that define how people behave even when no one is looking, what drives their work, how they come together as a team, how they collaborate, and eventually, how they achieve the organisation’s mission.

The way I see it, culture goes beyond the generic “integrity, honesty, and respect” that all organisations (and indeed laws) subscribe to. Rather, culture defines what makes a specific organisation’s DNA unique, in a way that works even as new staff join and old ones leave.

Organisational culture determines “how things actually get done,” often more than formal structures or strategies.

Organisational culture guides how people make trade-offs, handle failure, or respond to pressure. A culture that values ownership, learning, and follow-through leads to better delivery. Conversely, a culture of blame or risk-aversion produces delays, siloed thinking, and “checking the box” behaviour. Most importantly, people don’t just stay for roles; they stay for environments where they feel respected, challenged, and aligned with purpose, especially in the non-profit sector, where pay scales are not the most important criterion. Culture is often the main reason strong talent stays or leaves.

Picture Credit: https://www.surveylegend.com/research/types-of-organizational-culture/

We arrived at our values through collective deliberation. The process was facilitated by an external consultant.

Early on at WELL Labs, our leadership team (especially Anu Sridharan) began pushing me to define our culture. We had already hit bumps in the road where people disagreed, and the interpersonal issues were starting to distract us from the real work. We did not have a reference manual for mediating these.

Personally, I had little exposure to these ideas, having primarily worked in academia. So it was helpful for me to be asked what the “culture” of this new systems change organisation we were setting up was to be. Roughly speaking, we behaved like a start-up (in having goals and impact targets) but were non-profit in our structure and financing.

Anu brought in an external consultant, Rishi Gaurav Bhatnagar, who ran a series of workshops to define our cultural values. These were first run with senior leaders, then with the whole staff.

WELL Labs has seven cultural values: Deep Empathy, Impact on people and planet, Problem-first thinking, Scientific Rigour, Active Collaboration, Bias for Action, and Frugal Innovation.

You may have noticed that cultural values are not decency, integrity and honesty. All these are universal and very much expected, but they are part of the HR manual. Values are what make WELL Labs different and unique in its ability to deliver on mission.

1. Deep Empathy:

We value Deep Empathy for all actors, current and future (stakeholders in the room today and those who will be in the room tomorrow). At WELL Labs, we want to approach everyone with deep empathy, walking in each actor's shoes to understand their fears, aspirations, and motivations. Even actors we completely disagree with, we want to treat, at the very least, as fellow humans, listen deeply, and respond honestly.

2. Impact on People and the Planet

We are an impact-first organisation. We want to continually strive to improve human well-being, reduce inequality, and make the planet more livable and sustainable. We want employees to be clear about the goal and then constantly ask if we are on track to achieve it.

3. Problem First Thinking

We don’t want to design programs from a position of expertise. We want to focus on the problems from the perspectives of the community and the planet. What is needed to actually solve the problem?

A great example is offered by Astro Teller on Google X. If you are trying to teach a monkey to recite Shakespeare standing on the pedestal, 100% of the risk is in training the monkey, and therefore, it’s the first thing we should tackle, yet we often spend 90% of the effort in building the pedestal, saying we will figure out the “monkey thing” later.

4. Scientific Rigour

We want to make sure our work, whether peer-reviewed or not, meets the highest standards of scientific rigour and excellence. We expect people to be curious and excited about data and patterns, and about whether a hypothesis is panning out. We expect people to be intellectually honest and exhibit high integrity.

5. Active Collaboration

We want to work collaboratively with partners and team members. So we want to ask how we can help others and make sure we share credit and information. Obviously, we often make mistakes, especially at the speed at which we are moving. But Active Collaboration means we expect our staff to communicate openly and honestly, and ideally directly with people, rather than behind their backs. If escalation is necessary, it is handled empathically and professionally.

6. Bias for Action

We have a bias for action. Which means we want to experiment and try new ways of doing things rather than keep analysing the issue in search of the perfect solution. We want employees to be proactive and responsible for their time. If they have free time, ask for work. If they are stuck, ask for help.

We do not accept employees saying “no one gave me direction”. Everyone is responsible for their own time.

7. Frugal Innovation

We want employees to “think systems and scale,” which means considering what it would take for an idea to go beyond a pilot. We debated quite a bit about whether this value was even necessary. But we felt it was necessary.

There is often a tendency for staff to aim for perfection, when 20% of the effort might get is 80% of the way, and sometimes 80% is good enough. Given resource limitations, frugality is necessary. Frugal innovation is about tracking how efficiently we achieve targets. Are we setting up systems so that tasks can be done with less effort next time? Are we preventing the “best” from being the enemy of “better” and so on?

Culture gets built and reinforced by action against its violations.

Culture drifts under pressure, especially amid staff turnover and pressure from funders, so organisations must actively reinforce values through decisions, structures, and leadership behaviour.

Having a clearly defined culture deck allows us to have these conversations to help people see what it takes to thrive here. (You can see Netflix’s famous “Culture Deck” here.) But defining values is only the start. It’s one thing to create a pretty Culture Deck; it’s another for Culture to become DNA. Culture only gets reinforced by action against its violations.

Culture fails when staff get away with violating the values. An organisation can develop a good culture only when it has systems to reward those who uphold it and to call out people (including senior staff) when they may not be upholding its values. You cannot have a two-tier system. By this, I don’t mean people being abusive or dishonest, which, of course, should not be tolerated. I mean, smaller, seemingly benign activities that can erode. E.g., a programme head starts pursuing pet projects that are not “Problem first,” or starts creating factions within their team and subtly encouraging them to compete.

WELL Labs' cultural values are therefore central to every personnel-related decision, from hiring and onboarding to performance reviews and Rewards and Recognition. It is usually quite obvious when a staff member is not a good “culture fit” for WELL Labs. A poor culture fit is an employee who doesn’t take responsibility, constantly blames everyone else, doesn’t proactively ask for help, isn't detail-oriented where it matters, and, most importantly, isn’t serious about actually driving change at scale.

A person who does not fit in at WELL Labs is not universally a poor employee. We come across talented people who should be at an academic institution or a start-up, but not at WELL Labs. They could be an excellent fit for a different organisation: an academic may care primarily about intellectual curiosity, a teacher may care most about learning, a grassroots organiser may care about service, and an influencer may care about reach, and so on. We care about impact.

Culture is lived and organic, not a stick.

We recognise, of course, that developing a workforce that lives and breathes our Organisational Culture is aspirational. We won’t have everyone (including me) adhering to all of these 100% of the time, but we very much want to get there.

The goal of the culture deck is not to rank everyone based on culture fit or berate anyone. It’s to be used to frame conversations and diagnose and solve why we are not being as effective as we might want to.

People have asked how we manage finances within WELL Labs. WELL Labs is a “knowledge organisation” (as opposed to a grassroots non-profit), in the sense that most of our work involves synthesising science and data to drive change. Therefore, most of our spending is on people, and therefore, how we manage staff costs and incentives is a big part of this story.

The lessons herein are probably not relevant to revenue-based social enterprises or Section 8 companies. But it may be relevant to other knowledge organisations that have a 12AB and an 80G and are primarily grant-funded (Indian tax law allows up to 20% of income to be fee-for-service work that attracts GST; the rest must be strictly grant-funded).

Somehow, discussing finances in the non-profit sector seems frowned upon, akin to bringing up how much money you make at a party. But this silence is not exactly helpful. The less we speak about it, the weirder the assumptions donors (many of whom have never run a non-profit) seem to have. This is another reason I want to discuss this openly.

Source: https://www.yptc.com/nonprofit-financial-management/

Manpower

The challenge for knowledge non-profits is that people are our biggest asset and our biggest expense. So how we manage people IS the biggest determinant of fiscal health.

Billability

Many older Indian non-profits have endowments. Most newer non-profits do not. If you are a non-profit that doesn’t have access to endowment interest, then you are dependent on projects year to year. In other words, you are a “soft money” organisation.

WELL Labs is an autonomous soft-money centre within IFMR Society, which itself is a non-profit Society. So we have no choice but to build robust internal systems around “billability”. Specifically, all staff must be billable to projects. It might sound obvious, but it is quite common in the non-profit sector for staff to be billed on projects by one donor to deliver on objectives for another. Now, this is not entirely avoidable due to the end-of-financial-year arbitrary spending deadlines. But ideally, it should be time-banking (in the sense that you work more on Project A in March, bill it to Project B, and make it up in April).

Billability is enforced by maintaining timesheets of what employees actually worked on (not just what they are billed for). This is critical in the long run to ensuring that cross-subsidisation is kept under control and that effort is justifiable given the funds available. In the absence of robust billability, non-profits struggle when the “generous” donor or project that is cross-subsidising everyone in the organisation ceases.

Of course, this can quickly become bureaucratic. But our time sheets are simple and take less than 5 minutes. Each month, staff fill in what fraction of their time was spent on each project, or Fringe benefits (if taking Earned Leave (EL), Grief, or Maternity/Paternity leave), and how that time was allocated to each Key Result. Since most staff work on 2-3 projects and have 2-3 Key Results, it should be no more than 5 lines in a Google Sheet. Managers review it and submit it to Accounts to determine billing. On rare occasions, managers justify billing adjustments due to end-of-financial-year spending. (My team tells me our timesheet submission compliance is only 70%, so clearly we can do better.)

Some non-profits still operate on a cash basis, allocating what’s in the bank to meet payroll each month. But typically, these run afoul of donors, sooner or later. In general, it’s best to match budgets and spending at the project and line-item level within 10%.

On the flip side, I have learned that it is not a given that organisations with endowments have better finances. Often, they never develop formal systems or discipline for allocating endowment interest and instead treat all overheads and interest as a single pool of cash to be used at the leadership team’s discretion, often in a non-transparent, arbitrary manner.

Cash Flow

A big problem, of course, is that many donors are not very disciplined themselves. They release funds in February for a project that starts in January and ends in March, and expect to get a 100% utilisation certificate. This means the non-profit has to allow for some reserves/ working capital to start billing in January, even before the funds hit the account. So non-profits have to maintain reserves to cover their “working capital” needed to meet obligations like wages and programme expenses until funds hit the bank account. A common rule of thumb is 90 days of working capital. This also means we have to keep “Accounts Payable” (money billed for which payments have not been received) to no more than 90 days. Meaning, we stop billing projects 2 months after the fund balance reaches zero to ensure we do not experience a delay of more than 2 months in receiving funds for any given project.

A further wrinkle in India is that non-profits cannot mix domestic and foreign funds. So, the working capital reserve needs to be maintained separately in the domestic and foreign currency accounts.

Financial Projections

Financial projections are surprisingly hard in Indian non-profits, as we discovered. Domestic and foreign funds have to be strictly separate. Importantly, grant money is not fungible (if you have worked at a non-profit, you know, if you are wondering what the heck I am talking about — see this video “What if pizza places were funded like non-profits?). It has to be spent exactly on the line items for which the budget was sanctioned (although some funders allow some deviation with permission); so personnel costs cannot be billed to events and so on.

Traditional non-profits do this by hiring field staff or associates for specific positions on a project basis for 2-3 years, then letting them go. But this is not practical for a non-profit that wants to attract and retain top-flight talent. If you want people to stay on and grow in the organisation, they are typically billed across 2-3 projects. Now the challenge that every Programme manager in a non-profit faces is “How much money do I have for person X across projects A,B, and C in this Financial Year?” They want to be able to revisit this every month as new projects come in and new staff are hired.

At WELL Labs, this is even more complicated because we often have “cross-programme” projects, as we operate on a matrix structure (verticals are domains and horizontals are capabilities).

For the first two years, we managed with complicated spreadsheets. Each month, IFMR accounts would send us an Excel spreadsheet with fund balances by line item and project, which we would then feed into an increasingly complex Excel sheet with macros. After searching the market for suitable software to manage non-profit finances, we concluded it had to be customised, so we commissioned Zoho Creator to create a custom Financial Projections software to generate these reports for us. Our People system was also through Zoho, so there was some interoperability.

This is how it works: Each manager enters the budget for any new projects, with allocations to other programmes at the budget line-item level. Every month, after payroll is processed, the system pulls accounting data from IFMR’s SAP system via an API. Zoho Creator updates the amount spent and the funds available for each budget line item for each Financial Year.

We are just beginning to use this system, so fingers crossed.

Reserves

Closure Costs and Fringe Benefits

A major challenge that non-profits have in this context is managing liabilities - maternity and paternity leave, gratuity, menstrual, grief, mental health, and Earned Leave.

Some of these, such as Gratuity and Employer contributions to Medical or Provident Funds, already appear as line items in the Cost to Company (CTC). But many organisations do not properly provision for maternity and encashable Earned Leave (EL). Many instead try to pay these expenses from the overhead account, but often, the overhead allocation is sufficient. Importantly, these are government-mandated regulations and represent a “Cost to Company”; they are NOT an overhead expense. The only difference is that they are not incurred monthly. But neither is Gratuity. So they should be provisioned for as part of personnel costs, just like Gratuity is.

At WELL Labs, we provision this through a Fringe Benefit Bund. 4-7% of personnel costs are set aside to meet these obligations. This is based on our workforce composition and estimations of how many months of maternity benefits and EL encashment we can expect in a given year. In other words, what is budgeted and billed to projects is the “full CTC” including Gratuity, Medical, PF, and Other Fringe Benefits. Some donors, like the erstwhile USAID, have a separate Fringe Benefit line item, but no donor has told us that meeting government regulations is not acceptable. This also ensures that if we have to shut down, we will be able to meet our closure costs.

The Fringe Benefit account does NOT accumulate like a Reserve Fund. Instead, it is adjusted to be maintained at 6% of the Annual Income. Each year, the fringe benefit rate is adjusted to ensure we stay at that level. Ideally, fringe benefits are spent down (on average) each year, so we do need to keep accruing them.

Because we maintain strict billability, if an employee experiences a personal crisis and needs time off (e.g., one employee with an infant whose partner suffered a serious medical emergency for three months), they cannot be charged to projects. By definition, they are not working on the project. By charging the employees’ time to the Fringe Benefit fund, we can ensure that a substitute can be hired from within or outside the organisation. It ensures that we have retained flexibility to remain compassionate but without compromising our ability to deliver. If we don’t spend the funds (because, say, no one had a baby for a few years) and the fund exceeds 6%, the Fringe Benefit is spent down on other employee benefits, such as Learning and Development.

I would argue that an organisation’s “care and respect” for its employees is expressed through a Fringe Benefit Fund. I have seen many non-profit organisations that don’t maintain this, and end up billing Maternity leave to Overheads. Over time, they stop hiring women of childbearing age or, worse, force them to become contractors when they conceive. I have seen this result in a lot of heartburn and resentment.

Accumulation of Reserves

In India, non-profit law requires organisations registered under Section 12AB to apply (spend) 85% of their total income in any financial year for charitable purposes. NPOs are allowed to accumulate or set aside up to 15% of their total income without specifying the purpose or time limit. This 15% can be retained indefinitely in the reserve fund.

If an NPO wants to accumulate more than 15% of its income, it must file some additional paperwork with the Income Tax department.

What the law allows, though, is not the whole story. In practice, a non-profit organisation is not supposed to make a profit. So if the Reserves accumulate too much or too fast, donors will frown on it.

However, at WELL Labs, we think it's important to aim for 5% of our revenue to accrue to the overall Reserve Fund each year. This can only be achieved by optimising OH spending and charging projects properly for the actual work done. One way we do this is by keeping office space costs low through a partially hybrid model (though there is a trade-off). Accumulating the reserve allows us to remain strategic. Once reserves exceed 6-12 months (depending on the board’s comfort with reserve accumulation), some of them can be allocated to strategic investments.

Most non-profits aim to spend all their overhead, then complain that it is impossible to save any for a rainy day. But to achieve a 5% reserve target, we need proper classification of overheads. We need to ensure that very few staff in the organisation (it really is only Fundraising, HR and Accounts staff) are true overheads.

The key is to distinguish between Admin and Shared Services, for which a robust billability system is necessary. For instance, if an Admin staff member helps organise a workshop, that time can justifiably be billed to the project. Communications, similarly, is part of project documentation and should be billed within projects rather than as overhead. We have a rule that 10% of project personnel budgets should be allocated to Comms staff (this is comparable to other Global non-profits I have worked with).

Many Indian non-profits either underbill or overbill shared services like Communications, lumping all these into Overheads. Under-billing results in financial strain, and over-billing upsets funders. Doing it right requires robust internal billing and timesheeting systems.

Should reserves be allowed to accumulate indefinitely?

No real norms exist regarding what fraction of an organisation’s annual staff costs should be in the Reserve Fund in the long term — this includes working capital and the Fringe Benefit Fund. 6 to 12 months seems to be considered acceptable in India. I think donors frown on reserve accumulation exceeding 12 months unless specifically granted as an endowment, but some organisations allow it. On the other hand, many international NGOs are much leaner. This doesn’t mean that reserves should not continue to accumulate; it just means they should be spent down on strategic investments (the equivalent of an unrestricted grant) once this limit is reached.

Caveat on FCRA vs Domestic Reserves

We do not allow FCRA overheads to accumulate. Luckily, a large portion of our funding is domestic, and we do need to draw on overheads and fringe benefits each year to cover administrative expenses. So, by following a “first in, first out” principle for foreign currency reserves older than 3 years, we can ensure that only domestic funds accumulate in the Reserve Fund. We only need to maintain 90 days of foreign-currency working capital due to the non-mixing of foreign and domestic funds.

Unrestricted funds allocation

WELL Labs was lucky because in the first two years of our existence, about 30-50% of our funding was unrestricted (meaning not tied to specific projects). We used this cushion to set up systems for a future when we knew this would not be the case. The main system instituted was the allocation of restricted funds flexibly to unfunded hypotheses. This allowed us to challenge the leadership team to treat all unrestricted allocations as projects they could not otherwise fund. They would also have to justify why the investment was critical to 1) result in impact directly, or 2) raise additional funding that would generate impact.

It would be ideal if this number stayed at 20% of our annual revenue, remaining unrestricted going forward (in addition to OH, which funds administrative grants), but, of course, that’s a wish — it’s not easy to make this happen.

Personnel cost rules of thumb

We are often asked (by donors) how we set salaries and how we know we are not overpaying our staff.

This is an impossible question for NPOs. Non-profits, by definition, have public service mandates. Everyone understands that salaries are lower in exchange for “purpose and culture”. But how much lower? Typically, non-profit salaries (in the knowledge sector) are 10-20% lower at junior levels and 30-50% lower at senior levels than comparable jobs in the private sector (e.g., government consulting services). They tend to be slightly higher than academic salaries at the mid-career to senior level.

If we pay too much, we cannot be competitive and deliver value. If we pay too little, we cannot attract and retain talent. But which private sector salaries should be used as a reference? And how should organisations think of career progression?

One rule of thumb is the ratio of the size of projects a person manages to their salary. Based on conversations with other comparable global organisations, here are some thumb rules I have arrived at. This applies to knowledge work, not implementation work, since the latter involves many non-personnel materials, travel costs, etc., which can be highly context-dependent.

Early Career — Manage knowledge work equivalent to 1X-3X salary

Mid-level Managers — — Manage knowledge work equivalent to 4X-8X salary

Senior Managers /Directors — Manage knowledge work equivalent to 10X-20X salary

CXOs — Manager knowledge work equivalent 20X- 50X salary

This might seem like we are equating salary with scale of impact and impact with money. But it need not be so. When estimating the ratios, we consider the program's size, but the money may primarily go to partners. It could be, of course, that a particular staff member achieves a disproportionate impact with a lean team, but we have not yet encountered this situation. If that’s the case, we can just justify it and move on. These ratios are merely a guide.

Organisational Growth

Some donors have asked us why non-profits need to grow at all. The challenge is this. People want career growth. They don’t want their inflation-adjusted salaries to stay the same forever. But if the inflation adjusted salary bill increases by 5% we should be growing in revenue (in real terms) by at least 10%. Because as people move up the ladder, they need young people to mentor or partners to collaborate with / contract with to justify the increase in level. In other words, the entire increase in next year's revenue cannot be absorbed by the current staff alone. Otherwise, we will eventually have a bunch of 50-year-olds with no one to report to them.

This is the fundamental problem most non-profits face. External funding shrinks and grows. How do you build a sustainable career in such a scenario?

Many non-profit organisations struggle with an ageing workforce because this ratio of organisational growth to individual salary growth is not maintained. Over time, if all the growth is absorbed by existing staff and young blood does not enter the organisation. Over time, the organisation begins to struggle internally as people engage in food fights over scarce resources, entering a death spiral.

Endless growth is not the only alternative. Non-profits do not HAVE to grow. One way to manage individual aspirations vs. organisational growth is to plan to spin off parts of the organisation as for-profit entities. Another is to proactively help people chart out careers elsewhere. A third is to grow primarily through partners and consultants.

At WELL Labs, we set average annual increments for existing staff based on projected growth, using a rough 1:2 ratio between employee and organisational growth. Then we normalise performance on a bell curve to determine each employee’s increment.

Value delivery and social return on investment

As much as we would like to give everyone a 15% raise every year, that’s not possible. A 15% raise each year, in 6 years, an employee’s salary would go from 10 to 23 L. Now, current inflation is 3%. So if we are merely keeping up with inflation, then in 6 years an employee’s salary should go up from 10 LPA to 12 LPA. The difference between 12L and 23L means the employee has to be adding twice as much value to society within 6 years.

One cannot get paid more for doing the same job. Ultimately, to be financially viable, we can justify year-on-year salary increases beyond inflation only if we deliver significantly more value to society. This is true individually, at the program level, and as an organisation. Individually, some mid-level managers will leave for greener pastures, making way for younger people to grow.

In a for-profit, this is easily measurable in profits. In a non-profit, it’s harder. As an organisation, we are beginning to track cost-effectiveness and social return on investment. But eventually, the only way to justify manpower in a non-profit is by reasonably and honestly estimating value to society.

PS: I am not a Chartered Accountant or an expert in non-profit finance. This is merely what I have understood and learned from others.

WELL Labs is an “impact-first” non-profit within the IFMR-Krea university ecosystem. WELL Labs is transforming water systems at scale across India through research, partnerships, and collective action.

To this end, we intend to be accountable and transparent.

Subscribe now

Systems transformation needs a strategy and a plan to execute it.

Our systems transformation work at WELL Labs is structured around goals. Each goal represents a complex problem that we aim to solve. For each goal, have one or more theses or approaches to solving the problem underpinned by a theory of change of why we believe the approach will work.

Each thesis is accompanied by a roadmap. The roadmap is simply documented as a story of how the solution will unfold over the next decade. It is then broken up into a set of outcomes or milestones to be achieved along the way. Each outcome is associated with a hypothesis – how the activities and outputs we generate will result in the desired outcome or milestone. This in itself is pretty common — most organisations have well-documented theories of change that form the basis of grant proposals and 5-year plans.

In general, Outcomes are outside our control; Activities and Outputs are within our control. They are linked by hypotheses.

Here is the nub. A non-profit organisation aims to transform systems over which it has very little control. We promise this to donors. Yet we cannot guarantee that the activities we undertake will result in change (training programs will produce more capacitated people who can actually earn a better living, government engagement will shift policy, etc.).

At best, we have a solid hypothesis (and unreasonable hope). If we do activity X and produce Output Y, => It will result in Outcome Z. In the non-profit sector lingo, these hypotheses are part of the “Theory of Change”. Ideally, hypotheses are based on reliable prior evidence supporting the theory. These could be based on surveys, what’s worked in other sectors, psychological theory, whatever.

The bigger challenge is: how do these translate into internal incentive systems? In other words, how do we ensure our internal systems are robust? We are promising donors actual Impact. How do we ensure our culture, incentives, training, and internal systems are optimised to deliver Outcomes rather than Outputs?

We find that most non-profits struggle (including ourselves) with this. Unlike the private sector, where Outcomes (i.e., profits) are easily measurable, in the non-profit sector, impacts tend to take longer than the Annual Performance Review cycles. Of course, it’s also much harder to actually make impact — there is a reason government does things in a particular way it benefits some group of people, It is even harder to prove that impact has been achieved. If the impact, requires a large-scale change in how the government operates in 3-5 years, how can one be sure that we have made the “right” amount of progress in Year 1?

In the absence of robust systems, it is all too easy to become performative.

The biggest gap that many non-profits face is that the Theory of Change is never fully operationalised. It stays at the proposal stage or as a document. But it doesn’t translate to day-to-day operations. Importantly, as the organisation grows beyond 50 people, with lots of entry level staff who come from very different organisational cultures, it is incredibly hard to communicate the mission and Theory of Change down to the junior most staff member.

Here is how we are trying to overcome this.

Each year, we set “Objectives and Key Results” to operationalise our theory of change.

The Objectives and Key Results (OKR) framework (by John Doerr) is a goal-setting and performance management system used by for-profit organisations to align teams around clear priorities and measurable outcomes.

In our non-profit interpretation of OKRs, an Objective is a qualitative, ambitious statement describing what you want to achieve (the direction or goal), while Key Results are a small set of specific, quantitative indicators that measure progress toward that objective.

We then break up the team’s OKR down to individual OKRs in a massive spreadsheet. The framework helps ensure alignment and accountability.

OKRs in the non-profit context are not that different from the “ theory of change”. In fact, Goals = Impact, Objectives = Higher-level Outcomes, Key Results = Lower-level Outcomes. The only difference is the emphasis on setting up systems to achieve them.

Key Results are achieved through outputs and activities. Activities are “things we do to make outcomes happen.” Outputs are documentation of those activities.

The emphasis on “Outputs” is a key difference between the private and non-profit sectors. In the private sector, it is less important to document everything, including failures. A customer cares only about the product, not what went into making it. In the non-profit sector, the people paying for the product ( the donors) are different from the beneficiaries. Therefore, transparency becomes important.

The key here is to keep the OKRs real and not performative. Do we really believe the story? Are we thinking outside the box?

Our internal performance review systems reflect these principles. But the next challenge becomes setting the “right KRs” for each individual.

As people climb up the organisation, the expectations rise rapidly.

At the early career level (5 - 7 years post-Master’s degree), the KRs are Outputs. We are looking for high-quality outputs, with relatively little extra work to correct or direction by the manager. Here we are looking for ownership, proactiveness in asking for help, commitment to upskilling, and attention to detail.

For mid-level managers (7-15 years post-Master’s degree), the KRs are Outcomes. Here, we want to see evidence of constant experimentation to discover what moves the needle. It is not enough for an employee to say, “I produced X Output.” We want the employee to say, “I moved the needle on Y by doing this,” and we want to see that documented well. We expect them to lead a small team of early-career staff to produce the “right outputs.”

This is the group that has the hardest time. Most employees are not trained in outcome-oriented thinking. The jump from “do work right” to “do the right work” is actually a steep jump. Because we expect the employee to have agency, stay on top of what is working in the sector generally, and correct course in response to feedback on impact. We expect a proactive relationship-building with partners. For technical staff who act more as individual contributors, we expect high levels of technical progress and skill, with a seriousness about actual impact.

At Senior Leadership Levels (general >15 years post-Master’s degree), the OKRs focus on Impact. We want to see people making the right bets, forging the right strategic partnerships, mentoring a team, and raising money to support them.

We need impact-oriented, accountable, transparent non-profits. This should not be dismissed as “corporatisation”.

Over the last few months, I have heard from three separate sources some critiques of NGOs becoming “corporatised”. In my view, the term implies chasing money for its own sake.

But in fact, a systems transformation organisation is the very opposite; we raise money against impact targets. At WELL Labs at least, we do not chase projects.

We try very hard to set goals, be transparent, institute systems, communicate clearly, and hold ourselves accountable. I have often used the “starving babies” standard in internal townhalls. Philanthropic money could alternatively have gone to saving starving children in a more direct way. Asking for systems change funding means we have to hold ourselves to a high standard of accountability. And we have to strive to meet that high bar.

Ultimately, achieving impact means matching external commitments to internal systems. This is really hard.

WELL Labs is a water systems transformation organisation.

First, let’s remember what non-profits are supposed to do. A lot of people don’t understand why non-market and non-government players even exist. I argue that non-profits

1. Nonprofits solve transaction-cost problems associated with externalities, bringing diverse actors with shared interests.

2. Non-profits address information asymmetry and the capacity of vulnerable communities to approach markets and governments from a position of strength.

3. Non-profits overcome coordination problems between government agencies and across the government-business-civil society interface.

4. Non-profits de-risk innovation via pilots by bringing in philanthropic capital.

5. Non-profits fill gaps in government technical capacity in the pre-tender phase.

Now, the challenge non-profits face is that, because of the types of problems they solve and the diffuse nature of their beneficiaries, they are typically philanthropically funded.

This poses unique challenges in managing non-profits. Here are some struggles we have had to think about.

The challenge that all non-profits struggle with is that we are funded by small, specific, time-bound projects, but are expected to move mountains. It is very difficult to stay mission-oriented when funding is uncertain and intermittent.

We set big, hairy, audacious goals and roadmaps to achieve them.

Our work at WELL Labs is structured around big, hairy, audacious goals that we have carefully chosen and are based on maximizing our chance of success.

For each goal, we have one or more theses, the story we use to describe what the lock-ins are and why we think the systemic goal is achievable. Most of our theses offer a “win-win” opportunity and are underpinned by positive trends, signals, and political will, which makes us believe we can succeed. For each thesis, we articulate why we believe it has a strong likelihood of delivering meaningful impact within 10 years. This belief is grounded in prior case studies, empirical data, expert insights, and user research, which collectively inform our confidence level.

Each thesis is associated with a roadmap. This includes a set of outcomes or milestones that lay out how the goal will be achieved. We recognise that systems change involves moving many actors, many of whom are outside our sphere of control. Therefore, each outcome is associated with a hypothesis or theory of change, why we believe the activities and outputs we generate will result in the desired outcome.

Thus, WELL Labs engages in public problem-solving by democratising data and science to empower changemakers.

Each project we do tests one or more “Hypotheses” in the larger Theory of Change

Our Theory of Change focuses on mapping the relationship between what we do at WELL Labs and how our activities lead to achieving the desired goals.

For each goal (desired impact), we determine the outcomes to be achieved, the outputs to be produced, and the activities to be undertaken.

Now we realise that Activities and Outputs are within our control, but Outcomes and Impacts are not. The best we can do is hypothesise “If we do X and generate Y output, it will result in Z outcome and Z1, Z2 and Z3 outcomes (of which the latter two may be as a result of other organisations efforts will result in the desired Impact I”.

For each thesis, we work back from the desired impact to lay out what needs to occur for that impact to eventualise. We do not have 100% certainty on this causal chain – it is merely a “theory” - represented by hypotheses on why we think our actions will lead to the expected outcomes and impacts. Typically, a given hypothesis will take months to 2 years to pan out. If a hypothesis fails, we abandon it and formulate a new one.

Meanwhile, we track interim metrics – systemic outcomes in the short and medium-term that indicate that the system itself is moving in the right direction. Typically, the interim metrics will be the result of our own work, as well as of other actors, such as the government or corporations. We fully recognise that we cannot act alone.

The whole point of recognising the Theory of Change as a series of hypotheses is that it makes clear that success is not guaranteed. And if our hypothesis is not panning out, we abandon it.

It is key to match donors by risk appetite

One way, therefore, that we stay mission-oriented is by pursuing each hypothesis as a project. Each year, we raise funding for specific projects from philanthropic and CSR donors. In our internal planning, every proposal is for a specific hypothesis and forms part of a larger, more ambitious impact roadmap. In practice, we realise that some deviation from the plan to accommodate donor or grant-specific guidelines is necessary and part of the game. But we typically do not allow programmes to deviate significantly.

Importantly, we do not submit proposals for projects that are outside the Annual strategy and OKRs, and lack strong underpinning hypotheses. We do not “spray and pray” proposals vaguely related to the field of water resources. Of course, this requires us to have at least some unrestricted or flexible funding, or at least funding that is outcome rather than output oriented. But that is a topic for another blog.

In general, there are two types of donors:

CSR donors tend to fund low-risk projects. They have a low appetite for failure. For such donors, we propose projects that have a good “prior” probability of success. Such projects are not necessarily innovative, but rather, they are solid and dependable. The hypothesis should be proven; either the literature or another organisation should have successfully used a similar approach. But on the flip side, CSR donors usually care only about outputs, not long-term outcomes. While they may not like to advertise failure, they usually don’t stay engaged long enough to check if the needle moved on the ground.

Philanthropic donors tend to fund high-risk, innovative projects. They have a tolerable appetite for failure and care about outcomes. Often, they are willing to allow us to test different approaches and to have a less-constrained “log frame”. So we have to find philanthropic funding for less-tried-and-tested hypotheses.

The hypotheses and roadmap have to be revisited periodically.

By definition, hypotheses are uncertain, although some may be more uncertain than others. We cannot be certain what has worked elsewhere will work in our contexts. Once we have written out our hypotheses and Theory of change as OKRs, we have to revisit them regularly.

So each quarter, we conduct in-depth reviews of how each team is doing. It’s when we shift the focus from routine delivery of project outputs to questioning the hypotheses and theory of change. Basically, we ask: have we produced the outputs and conducted the activities they said they would? Are they moving the needle on the outcome as we expected? What makes us think that, if we continue, we will achieve the outcome milestones?

If we judge that something is not working (sometimes we have the wrong employee in the wrong job, in which case the problem is internal), then we have to either give up because it is just bad timing, a key champion in government has left, or political winds have shifted.

While we try really hard to pick big bets where we think we can succeed, sometimes we just fail. The key is to document how and why we failed and move on.

There is a fundamental difference between how researchers and problem solvers work. This poses a challenge for transdisciplinary projects.

First, for purposes of this blog, I am defining the terms broadly. A problem solver is a person who innovates and tests new ideas, methods, products, or processes that create value or bring about meaningful change. A researcher is a person who systematically investigates a subject to discover new knowledge, verify facts, and develop a deeper understanding. These two types of people have different training, work cultures, and intellectual histories. They have different role models and value different outcomes.

There is increasing interest in the donor world in transdisciplinary or “Research for Impact” projects that bring together researchers and practitioners. However, such projects inherently involve bringing people with different mindsets and incentive structures. Often, the lack of success then stems from not completely understanding these.

Expertise vs. generalists

Researchers aim to understand the problem. Problem solvers aim to solve the problem. Just because you understand why something doesn’t work, it doesn’t mean you know how to fix it. In complex socio-environmental problems, the barrier to transformation may not lie in the area where a researcher has expertise. Sometimes, the problem is the way municipal budgets are written or the incentive structures within an agency.

Ultimately, solving grand challenges involves bringing together deep experts who understand the nuances of specific parts of the problem and generalists who know how to assemble the parts. A successful collaboration requires, first and foremost, mutual respect and appreciation for each other's strengths.

Different tolerances of rigor

Careful, well-designed, rigorously done research is seen as necessary to build the edifice of knowledge. But in practice, problem solvers will not wait for researchers to generate the evidence. They are quite happy to learn and iterate based on feedback or rapid surveys.

Researchers want something to be fully vetted before it is put out. They see their reputations as experts at stake, so they have a relatively low appetite for failure. In contrast, innovators have an “iterative” mindset — rapid prototyping means putting out something “quick and dirty” to learn from it. They have a high appetite for failure.

This is a fundamental philosophical difference. Researchers think of evidence building as figuring out “what works and what doesn’t”. Problem solvers will either fix what doesn’t work or abandon it.

Researchers focus on peer-reviewed publications with a view to longevity of knowledge, and practitioners focus on impact. It is helpful to call out, early on, what success means to each person in the consortium. Understanding and communication are key here.

In practice, it may come down to how intermediate lessons are learned and communicated through research briefs and blogs, rather than waiting for a peer-reviewed publication. It may require conversations on who vets or reviews these grey-literature products. It does require academic institutions to value such contributions so researchers are not penalised for their participation.

Different time-scales

The gold standard for research is set by what it takes to publish in the most prestigious journals. This process inherently requires dotting every “i” and crossing every “t”. Process must be rigorously followed in a “waterfall approach”. There must be a clear research question, an approved research design (documented in an approved research ethics application), data collection and analysis, and, finally, publication. There is no scope to change protocols mid-way if the experiment is not working. Often, this whole process takes one to two years.

In contrast, problem solvers follow an iterative approach with rapid prototyping and pivots. Problem solvers prefer to “fail early and fast”, iterating until the problem is addressed. Typical sprints or experiments last a few months, especially if there is a technology component.

There are a few different ways to address this. First, introduce rigor into smaller iterative experiments by embedding a rigorous research design and carefully collecting data, even during rapid sprints or experiments. This requires quick, inexpensive, yet still somewhat rigorous research models. Second, both researchers and practitioners acknowledge that some research questions will take time to answer, and agree that action on the ground should always proceed with the state-of-the-art knowledge available today.

Hard vs. soft money

A related issue we encounter when we have university researchers work with NGO practitioners is the difference between hard (stable, guaranteed funding that is typically backed by government or endowments) and soft-money (time-limited, externally raised funds for specific purposes) institutions operate under.

Hard money institution staff have some flexibility to follow through on project outputs even after the project ends. Mainly, they are not accountable for every day of their time. Soft-money institutions find it difficult to allocate staff time to projects once they are complete. This makes the urgency of timely project completion and the ability to pursue long-term objectives very different.

One important principle for addressing this is to recognise and match timescales to tasks. If both institutions have long-term commitments to outcomes, they can walk together on a longer journey, building on prior work and filling gaps along the way. If one institution has a time-bound engagement and the other is committed to the long haul, having a clear start and end to the time-bound partner's commitment is helpful.

Postscript: At WELL Labs, we have given a lot of thought to how to design incentive structures that enable and accommodate both of these cultures. It’s an ongoing struggle, though, and must be constantly managed.

I could talk about all the things that have gone wrong with philanthropic investments, but I want to focus on one aspect that doesn’t get discussed much outside of IDR’s humor columns.

Most donors (very reasonably) ask for low overheads (<15%), efficiency (return on investment in terms of, e.g., for livelihood programs, it might be income/funds invested), and impact (how many lives, how much area, etc.). But what’s missing is some critical reflection on how their own choices make things worse.

I want to make a case that we need philanthropies to change how they give money. The sector needs its own efficiency metrics that take into account externalities imposed on non-profits, in addition to any impact metrics foundations already track.

My argument is that donors sometimes impose tremendous “externalities” on the non-profit sector that are not accounted for. Let me illustrate this argument through case studies.

1. The Prize

There is a foundation that gives out one large prize each year. Dozens of NGOs apply. The whole process is well managed by a highly paid consulting firm. The consulting firm makes multiple visits to the field sites and audits every applicant's accounts, which requires dozens of hours of NGO staff time. One year, an entire school was reopened a day early to accommodate the consultant’s visit. The floors were polished, and the kids prepared a special presentation, but the consultant failed to show up. When the NGO staff called the consultant, he was on vacation at a fancy beach resort. He had not discussed his vacation dates with his assistant, who had been coordinating with the NGO! The consultant didn’t face any consequences because the NGO didn’t dare complain, for fear of being blacklisted in the future.

2. The co-designed project

Then there are the three times, foundations approached us to co-design a project. The programme officers gave us a loose budget and a big picture idea they wanted to flesh out. In each case, over ten weeks were spent by multiple high-level staff working almost full-time to do the literature survey and flesh out the half-baked idea into a grant proposal.

But once the proposal was submitted, the programme officers either took it to their Board and failed to convince them or bailed in the last minute and simply ghosted. All that work explicitly solicited by the donor, which the non-profit would not have done otherwise, was not even followed up with a note saying, “Thanks for the work, here is why we could not fund it.” Since non-profits are soft money funded, staff time is usually billed to other projects. Consequently, the people ended up spending their vacation working to fulfill their regular deliverables.

3. The small grants programme

I have sat through my fair share of meetings that involve small grants. An anchor institution receives funding to administer a smaller grants programme on a subject in which they have expertise. They open up a call and get 1500 applicants. Then work day and night to whittle them down to a smaller number, and finally announce 20 finalists, each of whom will get 15 lakhs (~$20,000). They are thrilled by the response; tired but satisfied that they picked the best team after much internal wrangling. The problem is that no one calculated the opportunity cost of the hours spent by 1500 teams in developing the applications or the volunteer review team. When you run the Math, it comes to Rs 3 crores ($400K), which exceeds the total grant amount for the entire small grants programme.

Sometimes the grant can be larger. But if it is a global call involving a complex three-step application process with multiple workshops, videos, letters of support, and social media campaigns, the time investment by thousands of applicants can add up.

In other words, to get the grant, the non-profit sector must collectively spend the same amount that was given out in the first place. Who pays for this? It’s a hidden cost imposed on non-profit sector staff, who must just work longer hours. In the end, no one wins.

What’s the way forward?

I don’t think any of the foundation staff realize they ARE the problem. Throughout my years in the sector, I have found most to be sincere in their desire to create a better society. But there are clearly instances where the way grants are given is itself the problem because of the way the process is structured.

I am going to offer four concrete unsolicited suggestions to funders (with the sinking feeling that no one will ever give me money again!). Nonetheless, I hope more people will speak up and that these “do no harm” norms will become industry standards.

First, track and acknowledge total unpaid time invested by non-profit sector partners on RFPs in ANY interaction, including workshops, peer review, and grant writing. Have programme officers report the ratio of hours of unpaid to paid hours of non-profit staff time. The ratio of money given to money spent on writing the grants should be > 2. The effectiveness of impact assessments should internalize unpaid staff time.

Second, pay NGOs (not just consultants) through small grants to flesh out incomplete ideas. Budget for it explicitly and don’t externalize it onto NGO staff, who are on soft money and overextended. Think carefully about the co-design process, including how much volunteer labour “lean, low-touch” engagements entail and who is ultimately paying for them.

Third, solicit anonymous feedback from your social sector partners to prevent abuse of power by your own staff and third-party consultants. I have found the power-play/ entitled/ disrespectful behavior by some donor representatives to be appalling, something that we are all silent on. If I, as an upper-class IIT-Stanford educated professional, have faced this, then I can only imagine what field staff must regularly face.

Fourth, if your prize/small grants programme has less than a 5% chance of winning, it is probably best to join hands with another donor and boost those statistics to 10–25%. Give feedback on all proposals, if possible, even making the section-wise scoring available to each applicant.

I have heard three arguments justifying the status quo: First, the proposals are so poor in quality that no more than 5% are worth funding. I would like to counter that proposals are bad because NGOs are running like chickens with their heads cut off, trying to simultaneously submit multiple identical proposals to different donors in different formats. But if 95% of proposals are truly unworthy, perhaps what is needed is to build the sector's capacity. This also makes it even more critical to provide feedback so we can save everyone’s time next time. Second, it is too hard to get donors to collaborate. I found this one ironic, as it came from a donor who was setting up a small grants programme specifically to encourage non-profits to collaborate more. Third, small grants are meant to be top-up grants, not core, survival grants. This is fine, but only if someone, somewhere, is willing to offer core grants. If every donor makes this argument, we won’t have a non-profit sector left!

I truly believe everyone involved, even in the cases above, acted and spoke sincerely, and that even a highly paid consultant needs time off (I say this sincerely, as I am married to one!). But as they say, “the road to hell is paved with good intentions”. India’s non-profit sector is struggling under the burden of the new regulations. We need a Hippocratic Oath for philanthropies, so the non-profit sector isn’t killed by good intentions.

To be fair, many foundations do follow these best practices. Many have paid non-profits for ideation or strategising. When I have received such grants, it has been a lovely experience to spend time thinking during office hours instead of having to do it between 9 pm and midnight. A few offer core unrestricted support, and we are endlessly grateful to those!

But overall, we need much more self-reflection and transparency from the donor community if we are to weather this storm. I am not sure how that will happen if no one ever speaks out, because we are all too scared.

This article was originally published in Medium in Nov 2020

https://medium.com/@veenas-water/a-hippocratic-oath-for-philanthropy-8d745e36388d

Numbers without narratives:

Water data scarcity remains a problem in India. We simply don’t have the data we need to answer important questions. Our recent report on Commercial, Industrial, Institutional Water Use in Bangalore shows that even when the question is important, and data are carefully and painstakingly assembled, it’s difficult to tell a story because of the huge uncertainties and gaps in publicly available datasets.

But it is not just the absence of data; public data often has such large error bars or systematic biases that it is not useful. Groundwater levels in hard-rock peninsular India are a classic example of this blinkered vision. In our field studies, we have sometimes found that government monitoring well data to be much shallower than both our own observation data (from borewell scans and water level recorders) and farmer reports. A deeper investigation revealed systematic biases in the way data are collected and used. Yet, not much has changed in the scientific discourse.

Finally, many disciplines lack a culture of using numbers to tell stories. In the water sector, data are used to develop predictive models, but modellers rarely reflect on the meaning or implications of the water level or salinity trends observed and simulated. Do the water levels make sense given the changes in land use observed from space? Do they correlate with what farmers are saying? Why did the levels drop in one district but not the adjacent district? Stories from the field that contradict numerical models are dismissed as “hearsay”. So even when the data are incorrect or biased, no alarm is raised.

Narratives without numbers:

At the other end of the spectrum, equally problematic are glib stories of transformative change, where big claims of impact are made through effective storytelling accompanied by images.

Often, claims of dramatic rural transformation are common in the water sector. But this is often not sufficient to assign attribution as there are a number of confounding factors. For example, a community-based organisation engaged in building check dams might offer Google Earth images as proof of changes in greening pre- and post-construction. However, it would be difficult to tell whether the increase in greenery is due to the check dam or due to good rains in the preceding year. There is a need for storytelling, but it must be backed by sound data analysis.

Illustrated by Shilpa Swaraj

Using numbers to weave narratives:

What’s needed is data-based storytelling.

Of course, this also requires us to reflect deeply on what stories the data tell and about who. Often, data are biased, reflecting the concerns of the privileged. An inherent bias in the water sector, for instance, is the continued focus on irrigated farmers, even though almost half of India’s farmers and food grain production are rainfed. Rainfed farmers are much poorer and more vulnerable than irrigated farmers.

This requires a change in the culture of data collection and use. It requires us to ask the right questions, analyse the data correctly, formulate a narrative, triangulate with other narratives, and position the data analysis for use by stakeholders in specific use cases through effective communication.

This Article was originally published in Medium in April 2021

https://medium.com/centre-for-social-and-environmental-innovation/numbers-without-narratives-and-narratives-without-numbers-b261d2e4f771

Over the last few years. I have spoken with civil society and philanthropic organisations in our efforts to improve rural water security programmes.

The same scientific questions (vs. moral dilemmas) surface in every conversation; they are scientific in the sense that they can be addressed through empirical data analysis. E.g. How do we know if our intervention is benefitting farmers (vs exogenous factors like markets and rainfall)? Are all farmers benefitting equally? Are downstream farmers worse off? Which aquifers are being recharged? What are the reasonable boundaries of the aquifer system for participatory groundwater management to be initiated? Which trees should we plant so that we contribute to infiltration and soil conservation without drying up streams? How do soil and water conservation measures improve crop yields?

There are very few consistent answers, especially in the water sector. Where answers exist, researchers have usually drawn conclusions from models parameterised using data from Western long-term field observatories. We need increased attention to field-based research, not just to tweak model parameters for Indian conditions, but actually to discover new mechanisms and processes or even evaluate alternative model structures. But with a few exceptions of new critical zone observatories, we seem to be going in the opposite direction.

A lot of knowledge being produced in universities has no viable pathway to societal change.

To address this gap, we need a commitment to “Open Science.”

We urgently need better data and evidence to solve societal problems. But this isn’t happening fast enough. A lot of knowledge being produced in universities has no viable pathway to societal change. And when colleagues produce local data painstakingly collected over years of fieldwork, they find it hard to get published. Their research is dismissed as a “local case study”, lacking novelty. Journal editors refuse to send such work out for review.

On the flip side, even well-meaning researchers from the global north can be blind. Often, from their perspective, an “Indian farmer” is a distant abstract entity.

On the flip side, even well-meaning researchers from the global north can be blind. Often, from their perspective, an “Indian farmer” is a distant abstract entity. For instance, in one recent conversation, a well-respected and otherwise lovely American collaborator asked me to set up interviews with farmers to test some hypotheses. I asked him what I might present to the farmer as an incentive — meaning how might farmers in general (if not that specific farmer) benefit from the research, considering I would be asking him to lose a day’s wage. Because we knew the farmers in the region personally, their well-being was not an abstract concept for us. But it was clear that the farmers might have agency or motivations of their own that had never occurred to my collaborator.

Much has been written about the need to decolonise science. As a result of current structures, developing-country researchers have few choices — either to be the nth author on syntheses or meta-analysis papers by Western colleagues, or to build large-scale regional models using remotely sensed data to demonstrate scale. Others devise increasingly esoteric (to the point of absurdity) theoretical frameworks in a bid to demonstrate novelty.

None of these choices gets us any closer to either solving the problem or promoting the application of science. On the contrary, it fosters the impression that science is only done by an abstruse club of elites, not to be engaged with or trusted. Grassroots organisations (correctly) dismiss science as not being useful and focus on pleasing donors by presenting “cute pictures of women farmers with goats” as evidence of impact. Meanwhile, groundwater levels keep dropping, and farmers trapped in cycles of debt kill themselves.

How will this change? The scientific community must accept that we are part of the problem. We can only solve societal problems if we see knowledge production as a quest to make research relevant and accessible. This requires co-creating knowledge with partners from across academia, industry, public authorities, as well as citizen groups.

The European Commission has shown leadership by taking the first steps in this direction. They argue that an open science philosophy improves the quality, efficiency, and responsiveness of research, and when all relevant actors are invited to participate in the whole research and innovation process, creativity and trust in science increase.

We need a change in scientists' culture and mindset.

Grand political statements are a good start, but not enough. For change to occur, we need the scientific community at large to change how research is tracked, measured, and valued. This includes journal editors, university promotion committees, and award committees. We have to challenge the constant demand for “novelty” and create space for well-written, high-quality, defensible technical papers that can actually be used. In the past, when people still communicated through books, such work might have ended up in a book chapter. But we have limited room for hard copy books, and most content is now online. So what is the modern-day equivalent?

How do we make space for technical studies that are not necessarily novel, yet worthy of peer review? Perhaps we pay into a fund that guarantees high-quality peer review of technical studies? Maybe we need new journals, or have existing journals dedicate a section for “technical papers” that must be rigorous and yield local insight but not be novel in methodology? Or create more avenues for “data papers”?

Open science also needs new types of recognition and award structures.

It is heartbreaking that carefully done, grounded research that could actually help people has only a marginal place in our world, but our research institutions are also to blame. When I have asked scientists why their excellent field research study never saw the light of day, they typically respond by saying, “I tried five journals; they all refused to send it out for review, dismissing it as a case study. My university does not give me credit for anything other than peer-reviewed publications”.

So there is also a need to clarify how popular science writing and research reports not published in high-impact journals will be weighed in promotion decisions. The two most promising frameworks on how science might be evaluated to respond to the wicked problems we face are:

1. IDRC’s Research Quality Framework, which aims to change how research is defined, monitored, and assessed.

2. Utrecht University “open science” commitment has defined new faculty performance criteria

Both these frameworks emphasize relevance, co-creation, and positioning for use. The key takeaway is that we have to move away from paper counting (which is quick and easy) towards a more thoughtful, nuanced approach that involves actually reading and reflecting on our colleagues’ work.

Open science also requires new organisational structures.

Publishing is only the first step. After that, the research still needs to be positioned for use by decision-makers and/or communities of practice. For this, researchers must “translate their research” into messages and formats that can be consumed by wider audiences. In today’s world of slick “content”, it requires a whole team of people skilled in visual communication.

Much remains to be done if we want to create institutions that actually solve societal problems. If co-creation is to be an integral part of the research process, as the open science paradigm demands, researchers, communicators, project managers, software developers, and outreach professionals would have to work as a single team. For this, we need to figure out how academics and non-academics can work alongside in harmony and mutual respect.

Managing transdisciplinary teams, however, is a complex endeavor. One can theorise about transdisciplinary research and open science, but implementing these ideas in practice requires working through the nitty-gritty of creating career paths and fostering collaboration among people with widely different skills, pay scales, temperaments, and life goals.

This is a non-trivial problem, but there are emerging examples of how it might be done in transdisciplinary institutions like the Stockholm Resilience Centre, IHE Delft Institute for Water Education, and the Institute for Development Studies, Sussex.

I am very interested in this problem. If you have suggestions, best practices or ideas on making open science a reality, please do contact me at veena.srinivasan@atree.org.

This was delivered as the keynote lecture at the 48th IAH conference in Brussels on 10 September 2021.

https://medium.com/centre-for-social-and-environmental-innovation/a-call-for-a-commitment-to-open-science-974ff86c5948

We have proposed the ‘living labs’ approach to intervention design for cases where we cannot wait for the science to be settled because the cost of inaction is unacceptable. Sometimes, we must proceed even though we cannot anticipate unintended consequences. Indeed, in socio-environmental problem-solving, science may only be settled when we conduct research at the scale of the problem. This, in and of itself, is not a novel idea — after all, clinical trials for drugs and their subsequent extension to development / social sector interventions via Randomised Control Trials (RCTs) and quasi-experimental approaches are well accepted. It’s just less common in larger-scale ecological/environmental interventions.

Once the science is done or even while it is being done, it must be documented publicly. This article isn’t about the scientific method but about what form the documentation of the science must take to ensure widespread acceptance.

Traditional peer review is broken.

Scientific consensus is established by publishing in peer-reviewed journals. The peer review process is a system used to assess the quality of a manuscript before it is published in a journal. Independent researchers in the relevant research area (‘reviewers’) assess submitted manuscripts for originality, significance, and validity. Journal editors then determine whether a manuscript should be published in their journal.

The reward for going through peer review is citability — the paper can be found and cited by subsequent researchers using well-established citation practices. Search engines like Google Scholar and digital archives like BioRXiv are upending the process of search and citation. Nevertheless, recognition by scientific search engines like Scopus and WebofScience, which only include peer-reviewed journals, remains an important marker to distinguish between ‘scientific’ vs. ‘grey’ literature.

Read | A call for a commitment to open science

But having served on the editorial boards of reputable peer-reviewed journals, I would argue that the process of peer review results in both Type 1 and Type 2 errors. A lot of poor science makes it through peer review, and a lot of solid science gets left out.

First, the explosion of well-funded R1 universities, i.e., those that engage in high-quality research, particularly in the developing world, has expanded the ‘club’ of researchers capable of publishing in reputable journals. Further, in many universities, promotion metrics are increasingly tied to journal impact factors, a measure of how often an “average article” in a journal is cited over a given period.

Today, the number of papers being submitted far exceeds the bandwidth of journal editors and reviewers. Reviewers meet deadlines by reviewing late into the night. Consequently, the quality of peer review itself is not up to the mark. With the rise of open access pay-to-publish journals, a lot of bad science gets accepted, if not to the most prestigious journals, at least to journals that confer a paper legitimacy of being classified as ‘science’.

Read | Research needs to leave the ivory tower

Second, in a desperate bid to maintain sanity and control quality, journal editors must play the role of gatekeepers, judging what work is original or eye-catching, or of ‘widespread interest’ before even sending it out for review. As a result, much good taxpayer-funded science is never published in peer-reviewed journals. Most academics are sitting on piles of data that are pretty good but will never see the light of day.

This skews science in two ways — it forces researchers to ask “catchy” questions. In the water science world, at least, this translates to research with insights at regional or global scales instead of a local scale, to pass the significance filter. But it also discourages researchers, who must constantly chase novelty, from replicating prior research or publishing null results, which in the long run, is bad for science.

We don’t need traditional peer review to ensure the validity of science

On the whole, the system of peer review, as it is practiced today, does not serve our purpose because many of its functions are only partially relevant in an open science context, where applied researchers are engaged in producing knowledge to serve society.

By the definition above, there are three separate, distinct functions peer review serves — originality, significance, and validity. We argue that for work of wide public interest, originality is an irrelevant, arbitrary standard, and significance can only be determined by the beneficiaries of the research. The validity role, though, remains important — the research methodology must be sound, and the conclusions drawn must be supported by the evidence. The question is how to achieve validity without worrying about the other two.

We want to pay for peer review.

To ensure the validity of our conclusions, we propose paying in the range of $150 (Rs. 10,000) per paper for brutal, honest peer review that focuses strictly on validity. We are not asking for judgment on whether the work is original or significant. That is for our societal stakeholders to judge. Our logic is that if we have spent on the order of $30,000 (Rs. 20 lakhs) to produce a piece of research, paying $300 to two peer reviewers (Rs. 20,000) constitutes a mere 1% of the research expenditure and is entirely justifiable.

Academics are notoriously underpaid and exploited. Our proposal helps experts we respect, who have spent years, if not decades, of their lives becoming subject-matter experts, to earn a bit more money. In the Indian context, a retired academic or a young postdoctoral researcher could earn a decent living from this supplemental income.

We aren’t upending peer review, just modifying it.

CSEI is not an academic institution. We don’t need anyone to score us, judge us or act as a gatekeeper for our work. We are paying for peer review to establish the legitimacy of our work. We want high-quality reviews to ensure that what we are doing is backed by the best science. It also allows us to ‘show our work’ while satisfying our donors and partners with the soundness of our approach. We plan to obtain a Digital Object Identifier (DOI) or ISBN number to ensure the work is searchable and citable (note: a DOI or ISBN number is not correlated with the peer review status of the work).

This, of course, doesn’t solve the problem of external legitimacy. We still want to be cited by others, and we still need the wider community to accept the work. This is where traditional peer review comes in. We presume that once a work has been peer-reviewed, it should find a better, smoother process to getting published in traditional journals. We are thus not rejecting or upending peer review — just parsing it to ensure better science for society and channeling more money to experts in lieu of big publishing companies.

This still needs more discussion — our collaborators’ promotion reviews won’t change because we decided to change.

We recognise that paying for peer review is not a complete solution. Many high-impact journals won’t accept papers that have been published as a preprint. And there remains a wider conversation to be had about collaboration in a trans-disciplinary setting, where different people are recognised for different types of publications with different levels of rigour — quick reports accessible to the general reader vs. denser writing for a scientific audience that references prior literature and theoretical frameworks. We also recognize that we may want to co-author with others whose institutions may not accept our approach.

This article aims to start this conversation.

This article was originally published in Medium

https://medium.com/centre-for-social-and-environmental-innovation/why-we-are-paying-for-peer-review-2e096196e911

The paper listed five categories of bull-shit:
— Boring-question scholarship: 100s of studies of trivial empirical studies.
— Master’s thesis madness: Thin empirical studies by Master’s students.
— Literature reviews: Recycled summaries of papers
— Activist rants: Passionate calls for change with little empirical evidence.
— Recycled research: Rebranding prior work with fashionable buzzwords.

Now I sympathize with the author’s frustration (though I would consider this paper an activist rant myself), especially with the last two categories. But my advice would be — don’t read what you think is a waste of your time. Life is too short. But I want to make a case for the first two.

Much has been written about the “crisis of trust in science”. A PEW survey showed that the fraction of Americans who believed “science has had a mostly positive effect on society” dropped by 16 points since before the coronavirus outbreak. Although other studies have since shown that global trust in science remains high, there is a problem with scientists chasing novelty.

Over time, this generates a bias in science from two processes:

First “p-hacking”. Statistical decisions and methodology choices during research that artificially produce statistically significant results. For instance, psychologists Uri Simonsohn, Joseph Simmons, and Leif Nelson created their own ‘ridiculous’ experiment to see if they could reverse ‘p-hack’. They did this by designing a study to generate the results they desired. Their study wanted to show that listening to The Beatles could make one younger. They showed that by changing the variables they chose to measure, they could easily obtain a multitude of results.

Second, publication bias towards sensational results. Publishing only positive results and not null or inconclusive results. Statistical significance at 5% means there is a 1-in-20 chance of obtaining a sample that disproves the hypothesis purely by chance. But if only that study is published and the other 19 are not, we have a problem of overstating the result.

Science is built brick by brick, one at a time. Each brick is a piece of evidence that becomes stronger with each carefully done study that corroborates the findings. To ensure a strong foundation, we need many “boring” studies that attempt to replicate key results in new settings. These need to be published somewhere in a “boring replicate studies journal” so that they can be cited in literature reviews and meta-analyses.

The question is, who would ever agree to do boring replicate studies? No self-respecting university professor. Here is where Undergraduate and Master’s students come in. They are still not quite trained in either the discipline or the tools. They are perfect candidates to take tried-and-tested methods and replicate them exactly in a new setting. The goal for them should not be novelty; the goal should be a clear, useful research question, an appropriate research design, the collection of high-quality data, and sharp analytical insights. Novelty or brilliance is NOT the goal.

There is an opportunity to do this today under India’s new National Education Policy (NEP). The NEP encourages embedding research within undergraduate education. NEP mandates that a four-year undergraduate degree may include a rigorous research component in a student’s major field of study. It allows students to be “producers” of knowledge, not just “consumers” of knowledge. Provided we also build faculty capacity to ensure rigor of research, there is a chance that “boring student research” will contribute to genuine inquiry.

The real dividend is what can be learned by aggregating and synthesizing hundreds of boring studies. This can only occur, however, if we archive these “boring” studies and the data they collected (subject to research ethics and data privacy laws) in open data portals. Then they can be mined by LLMs and search engines, provided they are searchable and queryable. But if we do this well, we will create a stronger foundation for science.

This article was originally published on Medium in Oct 2025

https://medium.com/@veenas-water/in-defense-of-boring-science-27dd25bee5d3