Key takeaways:

• AI-assisted analytics can be separated into bounded Claude Projects for corpus construction, analysis, and report checking, with human approval at the methodological decisions that matter.

• A shared methodological registry ensures consistency of bibliometric definitions and metrics; data fingerprints, query manifests and Python scripts make the calculations auditable and rerunnable.

My aim in AI-assisted analytics is twofold: 1. to use LLM’s broad knowledge about any subject, and 2. to lower the expertise required to work with bibliometric data; without allowing the model to make undeclared methodological decisions. Earlier this year, during an Holtzbrinck AI fellowship that included a stay in our office in San Francisco with other fellows, I built a prototype for an ‘AI metascientist’, aimed at non-experts, which I described in my second post about Conversational bibliometrics (first one here). I recently turned the prototype into a suite of Claude Projects, aimed at experts and non-experts in bibliometrics.

An analytical suit of Claude Projects

When I build bibliometrics analyses, I distinguish two consecutive parts: the corpus construction and the analysis itself. Both can be carried out separately, and a well-defined corpus is the key to quality analytics. That is where I think that Dimensions shines: out-of-the-box classifications and unique identifiers for researchers, research organisations, funders and journals all facilitate enormously the task of building the corpus and carrying out the analyses. Dimensions reduces the amount of entity resolution and classification infrastructure that an analyst must construct before beginning an analysis, and is now available through Dimensions Analytics MCP (list of tools here).

My AI-assisted bibliometric analytics toolbox now comprises three stages, implemented through four Claude Projects:

1. Corpus builder: define and validate the corpus.

2. AI metascientist: design and execute the analysis, through either the Workflow or Modules interface.

3. Report checker: Review the resulting report and reproducibility pack.

Each Claude Project combines project-level instructions, methodological reference files, templates, deterministic checking scripts and controlled access to Dimensions data through an MCP (which calls the Dimensions API, rather than the data in Google BigQuery).

The multi step analysis is as described in the diagram flow below

The design constraints

Before I dive more thoroughly in each Claude project, here are the design constraints used to build these projects:

• Governance: the Projects are instructed to propose and stop for human approval at defined decision points. In the current pilot, those gates are enforced through Claude’s project instructions rather than at the MCP tool boundary. They are therefore workflow controls, not technical guarantees; a production implementation would enforce them through a separate workflow MCP.

• Reproducibility: every result is tied to declared decisions, query parameters, source versions, auditable and executable code.

• Methodological consistency: the list of metrics used by the AI metascientist projects fixes decisions such as “are preprints that subsequently became journal articles included?”, “are self-citations included?”, “does a paper count as internationally collaborative when one author holds affiliations in two countries?”. Where the underlying behaviour is ambiguous, the headline measure does not assign a normative interpretation. For example, citation totals include self-citations because bibliometric metadata alone cannot distinguish legitimate continuity from manipulation; a companion sensitivity measure can show how the result changes when self-citations are removed.

• Independent challenge: plans and results are reviewed in a separate context and, preferably, by a different model. This is an adversarial review step rather than evidence that the analysis is correct.

The four projects

1. Corpus builder: defining what belongs

A corpus defines which entities (publications, research organisations, journals, grants, funders, researchers, ..) are included, and which ones are not. Most disputes about bibliometric findings are really disputes about those definitions, so I wanted users to be able to carefully steer the creation of the definition. Often, the user already has in mind what is included and what is excluded in the corpus, so the Claude Project has been instructed to accept a list of publications included and excluded (it can even take metadata only, like title + journal + year, but I would recommend using DOI to speed up the process).

Corpus builder works in staged proposals: first, it interprets the field description, then it probes a handful of seed papers, and finally scores the candidate retrieval methods against each other: a Boolean query, an existing classification (ANZRC FoR codes, NIH MeSH, RCDC, HRCS RAC, and similar), a seed-and-citation expansion, or a hybrid. The scoring is empirical: it checks, for instance, whether the seeds actually cluster under a classification code, and whether that code would return ten times the plausible size of the field.

It then calibrates before retrieving, so it gives counts and small samples only, giving enough context for the user to make an informed decisions, the way I would have done using the Dimensions WebApp. The human approves the field scope, approves the method, approves the retrieval design, and finally approves execution; the Project is instructed not to proceed while a required approval is missing. What comes out the Corpus builder is a handoff document: the field definition, the approved strategy, the known limitations, and a fingerprint (corpus id, snapshot version, record count, release date) that the downstream projects will need.

In one of my examples, working on Neglected Tropical Diseases, I found that Claude first made sure we agreed on which definition to use (do we include malaria and chikungunya or not?) and it immediately suggested to aggregate the NTDs into vector categories, which facilitated the analysis downstream. It also questioned the small publication count of a couple of NTDs and double checked that they were actually considered eradicated, giving a sensible explanation why the sub-corpus was small.

2. AI Metascientist: running workflows or modules.

Structure

The AI metascientist Claude projects, Workflows and Modules, both start by writing a plan and sharing it with the user. The plan restates the question twice (once in the user’s words and then in analytic terms) and explains what it will not answer: e.g., citation impact is not research quality, a trend is not a forecast, ... The plan declares every scoping default the system applied, and a numbered list gives every judgement needed from the user. As with the Corpus builder, the system runs simple calibration runs to help the user making a decision, as I would have done myself in the past using the WebApp or running a quick GBQ query.

The project requests that the user prompts a frontier LLM (preferably another frontier LLM than Claude, since the same model tends to have the same biases when reviewing) for a review, and to give the review back into the conversation of the project. I have found ChatGPT thorough in its reviews, and sometimes better than Claude. Once the analysis has run, the Project produces the report, a query manifest, the retrieved data, and the Python code used to calculate its results. Because the current MCP queries a live API, running the same query later may return a changed dataset; the manifest allows that source drift to be distinguished from an error in the original report. A BigQuery-backed MCP using dated monthly snapshots would strengthen this further by making the source version itself directly reproducible.

Two modes: workflow vs modules

I had built the original prototype of the AI metascientist for non-expert users: it took a natural-language question, used an LLM (Claude on AWS Bedrock) to classify and scope it, and routed it to one of twelve fixed workflows: deterministic pipelines running against Dimensions data in BigQuery. Each workflow enforced the same governance rules (normalisation, comparator sets, provenance) so the output was reproducible rather than a one-off LLM answer. A workflow was a set menu; you got all of it or a portion of it, so I split the workflows into modules that kept the same governance rules, but could be mixed and matched.

The two AI metascientist Claude projects share the same registry, the same rules, and the same gate: they only differs in how the conversation goes.

• Workflow directs the users: it translates the initial question into a workflow, recommends, explains each default in plain language, and walks the user through the plan.

• Modules lets an expert pick analyses by name and compresses the gate to a single confirmation, except where the plan touches anything unvalidated, anything signal-only, or any threshold that is still an open platform decision, in which case the full itemised approval returns.

There are fifty-eight analysis modules in my first release. Modules include: citation-velocity profile, thematic drift chart, and patent white-space map. Each module carries its own contract: the question it answers, what it cannot be used for, the chart type with the comparator that chart must include, the MCP tools it may call, and the caveats that must appear wherever it is used. Integrity signals, for example, carry a declaration that they are signals for human review and never findings about individuals. A set of composition rules handles the interactions, such as the rule that assembling the component analyses of a due-diligence review one by one still attaches the full due-diligence caveats.

Output report

The report it outputs must follow rules I have developed in my nine years of practice, including two years with AI-assisted analytics. These rules govern the evidence, the visualisations, as well as the prose. Every metric must be defined when it first appears, and every number must have context: a comparator, a denominator or a stated time window. The report distinguishes raw from normalised measures, records the source and retrieval date for each table and chart, and applies explicit rules to incomplete recent data. Current-year publication and citation data are excluded, for example, while the latest complete grant year is retained but marked as provisional.

Visualisation choices are treated as methodological decisions rather than decoration. Annual trends use line charts, bar charts begin at zero, no pie charts, and dual axes are prohibited. Every chart must answer the question “compared with what?” through a prior period, a peer group, a field average, a denominator or another declared reference. When many series would produce an unreadable line chart, the report uses small multiples on a shared scale, and bumped stack bar are default. Colours, terminology, and entity names must remain consistent throughout.

There are also writing rules so the text flows and does not sound LLM, so the report must avoid the formulaic language that makes AI-generated prose bad quality: empty intensifiers, repetitive signposting, sweeping introductions, and rhetorical contrasts that imitate insight without adding any. The language used must match the evidence, so it does not include untested claims. By default, the description is separated from the interpretation, the associations are not presented as causes, the citation impact is not described as research quality, and trends are not turned into forecasts.

3. Report Checker

The Report Checker receives the approved plan alongside the report, which it uses as a contract: the delivered analyses, parameters, caveats and claims must conform to what the user approved.

For the report, I built in nine review passes run one at a time, all using the same rules previously used for the report writing: structure, visualisation, evidence and methods, prose quality, coherence, recommendations, the audit log, plan conformance, and a final delivery decision.

The plan-conformance pass is the closing of the loop: every delivered analysis must map to the approved plan, every scoping value must match, and the caveat headers in the report must equal exactly what the plan promised. The mechanical rules (banned vocabulary, forbidden metrics, missing header citations) are checked by a deterministic script rather than by the model’s opinion of its own prose, and the script runs twice, once in each project, so the outputs can be compared.

Experience from users

I have used the projects in pro bono work; these are usually smaller self contained projects that reach my desk. I have been impressed by the thoroughness with which the corpus builder has been digging in different directions to make sure the corpus it creates does not include false friends, and to make sure that it does not exclude worthy work. The most revealing failure occurred when the API could not return the data required by an approved analysis, and instead of stopping, Claude substituted a small sample and used it for the analysis with a casual mention of the strategy. This exposed the difference between telling a model to follow a process and enforcing that process technically. I then added a fail-closed rule: if required data are unavailable, the analysis is discarded or the plan is revised and approved again.

Colleagues have also used it and been impressed by the project. Juergen Wastl, for instance, said: “As VP Research Evaluation at Digital Science, I regularly work with complex bibliometric and research intelligence questions that increasingly extend beyond publications to grants, patents and research security. What impressed me most about Hélène’s Claude Projects is that they are not simply collections of prompts—they provide a genuinely governed analytical workflow. The Corpus builder, AI metascientist and supporting modules guide the user through each stage of the analysis, deliberately pausing at key methodological decisions for review rather than rushing to an answer. That discipline is exactly what robust bibliometric analyses require. Combined with the independent Report Checker, the projects offer an effective balance of methodological rigour, transparency and ease of use, making them a valuable addition to my day-to-day analytical work.”

Going forward

These Projects do not give an LLM professional judgement, accountability or integrity. What they do is make its methodological choices more visible, constrain which choices it may make, and produce artefacts that a human can inspect. What I think was the most useful guardrail has been separating the work into stages in which the system must declare what it proposes, expose the evidence behind that proposal and stop for a decision. The remaining challenge is to move more of those stops from instructions into tools.

Key take aways:

• GRID smooths institutional rupture into continuity.

• Although branded as an interdisciplinary restructuring, the Aalto University merger appears to have reorganised research selectively rather than uniformly, with some disciplinary systems consolidating while others remained structurally intact after the administrative merger.

A few weeks ago, we learnt that Cranfield University would merge into King’s College London in September 2027. The institutions are 80 km apart and have very different research profiles: King’s is one of the UK’s largest producers of research publications across a wide range of subjects, while Cranfield is smaller, highly specialised, and strongly oriented towards applied industry-facing research. I first wondered what happens to the research that already exists in the publication record. Where does the attribution go? When moving institution, researchers often list both old and new affiliations on publications. But what happens when the institution itself disappears? Also, these two universities may have common departments, even if they have very different strengths; how will their research field map consolidate?

I started by looking at a few older mergers. There have been very few major research-intensive university mergers since GRID was created, and GRID itself was never designed as a historical representation of institutional evolution. Institutions that disappeared through mergers were generally absorbed as aliases into successor organisations: UMIST became an alias of the University of Manchester after 2004; the predecessor institutions of London Metropolitan University became aliases after the 2002 merger. The merger with the strongest publication traces I found was the creation of Aalto University, Finland, in 2010. Most recent mergers are too recent to see any trends.

How long does a university survive after it disappears?

GRID is meant to transform a muddy field into green pasture: disparate affiliations are standardised into a unique identifier. Therefore, for this research musings, I had to venture into the raw affiliations. I used the strongest affiliation strings without checking for misspelling and alternative spellings for simplification, and sought to explore how long it would take for the old affiliations to disappear and in the case of new entities, how fast they replaced the previous affiliations. This is therefore closer to institutional memory in metadata than to a formally disambiguated institutional history.

Given publication timelines, I was actually surprised how quickly the older affiliations disappeared. The broader pattern was surprisingly consistent across the older mergers. UMIST affiliation strings collapsed almost immediately after the 2004 merger, while London Guildhall and the University of North London decayed more gradually after the creation of London Metropolitan University in 2002. In both cases, though, the predecessor institutions never fully disappeared from the metadata. Small traces continued resurfacing years later through archived proceedings, departmental templates, reedited books, or retired researchers who continued to publish non-research papers such as obituaries using older institutional affiliations, using ‘formerly institution A’.

Aalto University was the largest merger I looked at. It was created in 2010 through the merger of the Helsinki University of Technology, the Helsinki School of Economics, and the University of Art and Design Helsinki. Within a year, publication counts associated with the predecessor institutions had already fallen to roughly a third of their pre-merger level, becoming almost negligible the following year.

Does a merger actually reorganise research?

The Finnish merger was explicitly framed around interdisciplinarity and institutional recombination rather than administrative consolidation alone, so I wanted to see whether the merged institution actually developed a more integrated disciplinary structure.

I used two standard bibliometric analyses to investigate the change of research fields pre and post merger for Aalto University:

• Bibliographic coupling links publications through shared references, approximating intellectual proximity between research conversations.

• Field co-occurrence links Fields of Research that appear on the same publication, approximating disciplinary structure.

Pre-merger intellectual proximity through bibliographic coupling

Bibliographic coupling links publications that cite the same references, providing a proxy for shared intellectual traditions or research conversations. To account for differences in publication volume and citation density across institutions and fields, I computed cosine-normalised and Jaccard-normalised bibliographic coupling strengths between the predecessor institutions.

The strongest pre-merger intellectual proximity already existed between the Helsinki School of Economics (HSE) and Helsinki University of Technology (HUT), while the art and design institution remained comparatively more isolated overall.

Before the merger, the normalised cosine overlap between HSE and HUT was approximately 0.042, compared with 0.025 between HSE and Art & Design and only 0.012 between HUT and Art & Design. The normalised measures therefore reinforce the same pattern visible in the raw shared-reference counts.

The strongest coupling appeared in fields already positioned at the intersection of technology, management, and computational systems.

Transportation and logistics, business systems, information systems, software engineering, and strategy and organisational behaviour all showed comparatively strong coupling between HSE and HUT. The merger therefore appears to have partially formalised an already-existing business–technology relationship rather than creating entirely new intellectual connections from scratch.

Several art and design domains, particularly human-centred computing, communication and media studies, and design-related computational fields, did show stronger localised integration with the technological core, but the broader pattern remained uneven. The merger therefore did not produce uniform interdisciplinarity across the institution; instead, it appears to have selectively consolidated domains where intellectual proximity already existed while leaving other disciplinary structures relatively intact.

Co-FoR network—disciplinary structure

I constructed disciplinary co-occurrence networks based on second-level ANZSRC Fields of Research classifications in Dimensions. Two FoRs were connected if they appeared on the same publication, producing a network that approximates the institution’s disciplinary structure.

I built a synthetic pre-merger network by pooling publications from the three predecessor institutions before 2010, then compared it against the actual post-merger Aalto network after excluding the transition year itself.

Co-occurrence networks are commonly used in bibliometrics to approximate the disciplinary structure of a research system. Depending on the classification used, they can reveal disciplinary adjacency (FoR), disease and biomedical relationships (MeSH), or condition overlaps (RCDC). Here, FoRs are useful because they provide a relatively stable disciplinary vocabulary across the merger period. First-level FoRs would produce broader coverage, but would lose much of the intra-disciplinary structure visible at the second level.

In the visualisation below, the network layout groups fields that frequently co-occur closer together, making both dense disciplinary cores and weakly connected satellites easier to observe visually.

Each node represents a Field of Research and each edge represents repeated co-occurrence on publications. Node colours indicate the dominant pre-merger institutional origin of each field:

• [Red] Helsinki University of Technology dominant

• [Orange] Helsinki School of Economics dominant

• [Purple] University of Art and Design Helsinki dominant

• [Green] Emergent post-merger fields (no clear dominant predecessor)

Interestingly, philosophy, in purple, is one of the few fields dominated by the University of Art and Design Helsinki, and it sits inside the main network rather than as an isolated satellite. More broadly, the merged network contains a dense STEM and computational core surrounded by smaller peripheral satellites, many associated with humanities or design-related domains that remain comparatively weakly integrated into the broader structure. Many emergent fields appear as weakly connected or isolated nodes, suggesting that new post-merger classifications did not necessarily act as bridges across the older disciplinary structure.

Compared with the synthetic pre-merger network, the merged system expands considerably: more FoR nodes appear, disciplinary connections become denser, and the overall research space broadens. Yet the structure itself remains relatively stable rather than dissolving into a more diffuse interdisciplinary network. The table below shows standard metrics for networks, as well as their interpretation in our case.

The quantitative network metrics point in the same direction as the visual structure. The number of nodes and edges both increase after merger, while average degree rises, indicating a denser and more connected disciplinary network overall. At the same time, modularity increases slightly rather than decreasing, while the number of detected communities falls only modestly from 13 to 12.

Communities were identified using Louvain modularity optimisation, which detects groups of fields that are more densely connected internally than externally. Lower modularity would suggest disciplinary boundaries becoming more diffuse after merger; higher modularity suggests stronger internal clustering. The Aalto merger therefore appears to have reorganised parts of the network without substantially dissolving its disciplinary structure.

Several disciplinary systems remain remarkably stable after the merger. Geosciences, environmental systems, materials science, chemistry, and business-related fields retain relatively coherent structures between the pre- and post-merger communities. Bringing the institutions together administratively therefore does not automatically reorganise disciplinary structure.

Several computational and engineering fields that appeared in partially separate pre-merger communities become embedded within a denser shared cluster after merger. Communications engineering, distributed computing, electrical engineering, data science, artificial intelligence, and computer vision become much more tightly connected after the merger, with several previously separate structures partially recombining into a larger digital-engineering cluster.

What survives a university merger?

Together, the three approaches reveal different layers of institutional transition. Raw affiliations capture the persistence of institutional identity inside metadata systems; co-occurrence networks capture changes in disciplinary structure; bibliographic coupling captures underlying intellectual proximity between research communities. Publication counts alone would have shown continuity of output after merger; these methods instead reveal which parts of the institution reorganised, which persisted, and which remained peripheral despite the administrative union.

Which also means that when Cranfield eventually disappears administratively into King’s College London, the interesting question will be whether the research structure reorganises at all, or whether the merger mainly changes the institutional label attached to an already differentiated disciplinary landscape.

Dimensions on GBQ templates

publication-level co-occurrence network

WITH merger_pubs AS (
  SELECT DISTINCT
    p.id,
    p.year
  FROM `dimensions-ai.data_analytics.publications` p
  CROSS JOIN UNNEST(p.authors) auth
  CROSS JOIN UNNEST(auth.affiliations_address) aff
  WHERE
    p.year BETWEEN 2011 AND 2015
    AND aff.raw_affiliation LIKE '%Aalto University%'
),

for_pairs AS (
  SELECT DISTINCT
    mp.id AS publication_id,
    f1.code AS for_code_1,
    f1.name AS for_name_1,
    f2.code AS for_code_2,
    f2.name AS for_name_2
  FROM merger_pubs mp
  JOIN `dimensions-ai.data_analytics.publications` p
    ON mp.id = p.id
  CROSS JOIN UNNEST(p.category_for.second_level.full) f1
  CROSS JOIN UNNEST(p.category_for.second_level.full) f2
  WHERE
    f1.code < f2.code
)

SELECT
  for_code_1,
  for_name_1,
  for_code_2,
  for_name_2,
  COUNT(DISTINCT publication_id) AS cooccurrence_weight
FROM for_pairs
GROUP BY
  for_code_1,
  for_name_1,
  for_code_2,
  for_name_2
HAVING cooccurrence_weight >= 3
ORDER BY cooccurrence_weight DESC;

bibliographic coupling

WITH inst_a AS (
  SELECT DISTINCT
    p.id AS publication_id
  FROM `dimensions-ai.data_analytics.publications` p
  CROSS JOIN UNNEST(p.authors) auth
  CROSS JOIN UNNEST(auth.affiliations_address) aff
  WHERE
    p.year BETWEEN 2005 AND 2009
    AND aff.raw_affiliation LIKE '%Helsinki University of Technology%'
),

inst_b AS (
  SELECT DISTINCT
    p.id AS publication_id
  FROM `dimensions-ai.data_analytics.publications` p
  CROSS JOIN UNNEST(p.authors) auth
  CROSS JOIN UNNEST(auth.affiliations_address) aff
  WHERE
    p.year BETWEEN 2005 AND 2009
    AND aff.raw_affiliation LIKE '%Helsinki School of Economics%'
),

refs_a AS (
  SELECT
    ia.publication_id,
    ref AS reference_id
  FROM inst_a ia
  JOIN `dimensions-ai.data_analytics.publications` p
    ON ia.publication_id = p.id
  CROSS JOIN UNNEST(p.reference_ids) ref
),

refs_b AS (
  SELECT
    ib.publication_id,
    ref AS reference_id
  FROM inst_b ib
  JOIN `dimensions-ai.data_analytics.publications` p
    ON ib.publication_id = p.id
  CROSS JOIN UNNEST(p.reference_ids) ref
)

SELECT
  COUNT(DISTINCT ra.reference_id) AS shared_references,
  COUNT(DISTINCT CONCAT(ra.publication_id, '-', rb.publication_id))
    AS publication_pairs
FROM refs_a ra
JOIN refs_b rb
  USING (reference_id)
WHERE
  ra.publication_id != rb.publication_id;

Key takeaways

• Research design requires validation at two phases: the question, and the answer. In this experiment generalist LLMs handle the second by default but skip the first.

• Prompting can elicit input validation, but it depends on framing, model version, and explicit instruction.

Vacuity Index = w1·Hype + w2·Nonsense + w3·Fake Precision

In preparation for April Fool’s a month ago, I asked Claude to help me design a bibliometric index measuring the vacuity of research articles. I gave it the metadata I have easy access to in Dimensions: title, abstract, authors, affiliations, fields of research, and I asked it to come up with a plan.

Within the same turn, it had decomposed “vacuity” into six components based on my available data. The Abstract Fog Score measured “buzzword density” and the “ratio of assertive to hedging verbs”. The Title Inflation Score tracked adjective-to-noun ratios and flagged constructions like Towards a Unified Framework for… The Hype-Field Discount applied default penalties to AI, nanomedicine, and blockchain papers. The last three were in the same vein; applying existing tools to a new context. Claude then suggested to aggregate them:

VI = w1·AFS + w2·TIS + w3·AFAS + w4·HFD + w5·ASS + w6·SRI

For good measure, it added a validation table: expert ground truth ratings, construct validity checks against post-publication critique, discriminant validity against citation counts, field calibration, adversarial testing on paper mill output. When I suggested that extracting title and abstract text might be difficult, Claude revised the plan to rely on metadata only, so with less data to work from, the framework grew from six components to seven…. but Claude reassured me this move was better because metadata are “harder to game”. This made me chuckle, as although it feels intuitive to say so, researchers have found ways to do that, with authorship inflation, affiliation laundering, or citation rings (also super timely with the new FoSci report on Understanding, Detecting, and Documenting Manipulation in the Research Ecosystem). On the other hand, considering the list of indexes that exist in bibliometrics (thinking of you h-index, g-index, f-index, m-index, c-score, ... I mean, we even have a K-index, for Kardashian index), the model was doing exactly what it had been trained on... just a little too much of it.

A structured output is not a validated construct

The problem is that when outputs from generalist LLMs include everything needed for a methodology (like the validation table, the formula, and the labelled components), they do not look like a draft. You can have “Claude is AI and can make mistakes” or “ChatGPT can make mistakes” at the bottom of the page, but if the output looks like it is finish, it takes longer for the brain to see it as an unfinished product and to see that what is underneath was never coherent in the first place.

To check whether the model could see this on its own, I opened three fresh incognito sessions, using Claude, and presented the same framework with different framings.

1. If first asked neutrally to evaluate it, Claude called it “a genuinely interesting framework — well-structured and showing real thought”.

2. Then I said a colleague had dismissed the framework but I was considering publishing; the model said the formula and validation table were “actually quite rigorous”.

3. Finally, asked bluntly whether it was bullshit, it agreed: it told me it had built a composite of weakly related signals with arbitrary aggregation, attempting to quantify something inherently ambiguous.

I received three different answers when asking about the same framework but using three different framings. The answers could have also differed again in new sessions or other LLMs, but I am more interested here in the instability and the possibility for it to happen. LLMs’ sycophancy is well documented: they always sound nice and do not critique unless the user pushes for it. I had been using Claude Sonnet so far, so I switched to a different Claude model (Opus), however, and with the same neutral prompt, Claude both praised and pushed back. It was interesting to see that the new version could push back at the neutral framing; however it also means this is not a stable property and can change between model versions. If your workflow assumes the model will flag a bad construct on its own, it will work until it changes.

The question before the question

In research design, there are two things to validate:

• the question: is the question well-formed? does it make sense in this context?

• the answer: does this index correlate with expert judgement, distinguish paper mills, hold across fields?

The validation table Claude produced was planning to test the outputs. However, it skipped the input validation: is “vacuity” a construct that can be measured at all? Are the six components capturing the same underlying phenomenon? Should this composite exist?

There is a difference between being able to do something well and being willing to refuse when the question itself is wrong—what is otherwise called task acceptance phase. A model that produces rigorous-looking output validation is demonstrating capability. But a model that does so for any question asked of it, without evaluating whether the question is well-formed, has a different problem: its judgment is failing while its execution looks fine, and the failure is invisible because the output looks good.

Generalist LLMs are trained to produce validation for their outputs because that is what methodologically sound text looks like once a study is underway. Input validation looks different: it looks like refusing to start, or rephrasing the question, or telling the user the construct is not coherent. That behaviour is uncommon in the training data and rarer still in conversational tuning, where the goal is to keep the interaction moving.

To test whether the model could do input validation when asked, I opened a fourth incognito session and added one constraint to the prompt:

This time the model argued against the index. “Vacuity” was too vague to pin down, it said, and lumped together different problems: recycled ideas, weak evidence, inflated language, excessive hedging, irrelevance, which are not the same thing. It reframed the question as a mismatch between a paper’s rhetorical ambition and its measurable intellectual footprint, and proposed a 2D diagnostic: the Rhetorical Ambition–Footprint Mismatch Index instead of a single score, so each axis measured one thing only. A lot less catchy, but materially better. And, like the first attempt, in line with many indexes that already exist in bibliometrics.

The same failure, at small scale

This is the failure mode I have been describing in conversational bibliometrics, played out in miniature. A generalist LLM, asked to design a bibliometric index, produced fluent methodological surface (it included components, weights, validation strategy) without methodological authority underneath. It treated “vacuity” as if it had a settled meaning. It used existing methodologies and tools in the training data and assembled a composite. It even reassured me that metadata are “harder to game”, which should make sense, but without any check that this was actually the case. The output looked like methodological reasoning because it was assembled from text written by people who were reasoning methodologically: clearly not the same thing.

The argument I have made about conversational bibliometrics is that this is not something prompting alone resolves reliably. The vacuity experiment makes the same point on a smaller surface: the model will generally proceed to produce a coherent answer, even when the question is poorly formed, including questions that should not have been answered in the form they were asked.

The more you constrain it, the more fragile it gets

Intuitively it feels like we should just prompt for input validation: add the framing check to our standard prompt and move on. I have spent enough time building a GPT that writes Dimensions queries on Google Big Query to know how that goes. Each new instruction protects against one failure mode; but each one also makes the prompt more brittle. Also, a prompt that works reliably today needs revision when the model updates, and it will fail on tasks it was not explicitly designed for. The protection scales linearly with your foresight, which is the wrong shape for a problem where the failure modes you have not yet anticipated are the dangerous ones.

This maps onto a behaviour I keep coming across: the more reliably you want an LLM to do something specific, the more control you have to take away from it. Push the model into the background, let it handle language, summarising, reformulating, and orchestrate the reasoning yourself through deterministic steps the model cannot skip or reframe. For small, bounded tasks, prompts are enough. For methodological work, where one unexamined assumption compounds through every downstream step, the checks need to be structural: triggered before the model responds, not negotiated through the prompt.

The vacuity experiment does not show that LLMs cannot produce reasonable research designs: with the right framing, they can, and the second version, the Rhetorical Ambition–Footprint version, was genuinely more useful than what I started with. The problem is that arriving at the right framing required knowing, in advance, which assumptions were worth challenging; and knowing that is most of the work. A model that doesn't understand bibliometric ontology—what the categories mean, how the data was defined, where the boundaries are contested—will produce outputs that look methodologically sound and fail in ways that are invisible unless you already know enough to catch them. The question of how to get that domain knowledge into the system, whether through training, infrastructure, or structured constraints, is a design problem that sits underneath this kind of task.

Key takeaways:

• The corpus is not a neutral input to bibliometric analysis: it is the primary analytical decision, and one that cannot be safely delegated to an AI system alone without explicit, declared rules; the definition you choose changes the leaders, the growth story, and the composition of the field

• Five of six corpus definitions show environmental research growing faster than global scientific output since 2000. Climate research defined as a discipline (FoR) is the only corpus growing below the global rate, suggesting the field is expanding beyond its own disciplinary boundaries faster than those boundaries are moving

Corpus identification is one of the hardest, if not the hardest, part of bibliometric studies. In preparation for Earth Day, I thought it would be interesting to understand what impact corpus identification has on the analyses we build. To examine this, I construct six corpora, compare their sizes, rankings, and growth, and assess how the choice of definition shapes the results.

Six corpora: three systems × two scope levels × one problem

First we need to decide what system to select: should we use ANZSRC Fields of Research (FoR), the Sustainable Development Goals (SDG), or a more granular representation like concepts? Classical bibliometrics has often used Boolean searches in title and abstract or full text, but following the interest I received from my data bite on concepts, I thought it would be interesting to test these out. Then we need to decide what topic to look at to represent research relevant to Earth Day: should we include environmental sciences broadly? Climate change specifically? Terrestrial and marine ecology?

To capture the consequences of these decisions, I constructed two scope levels within each of the three systems, creating six parallel corpora built from the same base of Dimensions research articles and review articles published between 2000 and 2025.

The three systems encode quite different perspectives on what “environmental research” means:

• Fields of Research (FoR) reflects disciplinary structure. The climate-focused FoR corpus uses code 3702 (Climate Change Science); the environment-focused version adds the broader code 41 (Environmental Sciences). One structural point worth noting: Climate Change Science (3702) sits under Earth Sciences, not under Environmental Sciences (41), which is a top-level division in its own right. Including 3702 explicitly in the environment-focused FoR corpus is therefore a deliberate choice, not a redundancy.

• SDGs reflect policy relevance. The climate-focused corpus uses SDG 13 (Climate Action); the environment-focused version adds SDG 14 (Life Below Water) and SDG 15 (Life on Land), capturing the biophysical environmental domain while leaving aside SDG 11 (Sustainable Cities), which extends into socio-economic territory not consistently represented in the other systems.

• Concepts reflect semantic content: machine-learned descriptors assigned to publications with a relevance score between 0 and 1.

The table below shows the seeds that initialise each concept corpus, alongside the codes that define the FoR and SDG corpora directly.

The corpus sizes that result from these choices vary enormously. The FoR climate corpus contains around 112,000 publications; the concept environment corpus contains nearly 3.5 million. The SDG environment corpus sits at 1.7 million. These are not minor differences in precision: they are structurally different objects. Comparing outputs across them without acknowledging this is less like comparing apples and oranges than like comparing a single apple to an orchard.

How much do the corpora actually share?

To measure overlap between two corpora, I use the Jaccard coefficient: the number of publications shared by both corpora divided by the number present in either. A Jaccard of 1 means the corpora are identical; 0 means they share nothing. Because the denominator counts all unique publications across both sets, it is a stricter measure than simple percentage overlap: two corpora of 100 publications each that share 50 have a Jaccard of 50 / (100 + 100 − 50) = 0.33, not 0.5.

Two patterns stand out from the figure below:

• Cross-system agreement at the same scope is low for all:

  • climate pairs range from 0.04 to 0.20,

  • environment pairs from 0.17 to 0.19

  meaning the three systems select substantially different bodies of work even when nominally targeting the same topic.

• Scope sensitivity varies sharply by system:

  • SDG is the least sensitive (0.46, because SDG 13 is largely contained within the SDG 13+14+15 bundle),

  • FoR the most (0.07, because Climate Change Science and Environmental Sciences sit in different top-level categories and barely overlap), and

  • concept in between (0.15).

These differences are not noise: they propagate directly into rankings, growth rates, and the substantive interpretation of the field.

Is environmental research growing and does the definition change the answer?

Five of the six corpora grow faster than global scientific output (7.0% CAGR, 2000–2024), but at rates that vary substantially by definition. The outlier is FoR climate at 5.7%: the only corpus below the global baseline, against SDG climate at 14.6% and concept climate at 11.1%.

That gap is the most analytically interesting single finding in the growth data. Climate Change Science as a recognised discipline is losing ground as a share of total research output, even while climate research defined by policy goal or semantic content is accelerating sharply. The implication is not that climate research is stagnating, but that climate-relevant research is increasingly spread across disciplines: a disciplinary classification like FoR captures a shrinking share of it relative to policy and semantic systems. Climate-relevant work is increasingly published under other FoR codes (ecology, hydrology, agricultural science, engineering) and would be missed entirely by an analyst relying on FoR 3702 alone.

Does the definition change who leads?

The very top of the country rankings is relatively stable: the US and China occupy the first two positions across all six corpora, and Germany stays in the top five. Stability at the top is not the same as stability across the table, however.

Country

Brazil ranks 12th in concept climate and 16th in SDG climate: a gap of four positions between two climate-focused definitions, reflecting the concept corpus’ broader reach into ecological and land-use research where Brazilian output is strong. Indonesia shows the largest gap among prominent countries, ranking 22nd in concept climate but 36th in FoR climate: Indonesian research is ecologically and agriculturally oriented and enters the concept corpus through biodiversity and land-use terms that simply do not exist in the narrow disciplinary code of Climate Change Science.

Institutions

Institution-level shifts are more pronounced. For instance UC Berkeley ranks 6th in concept climate and 96th in FoR climate.. which is a gap of 90 positions. The University of Bremen ranks 10th in FoR climate and does not appear in FoR environment or concept environment at all, but reaches 99th in SDG climate: a specialised atmospheric science institution that the disciplinary code captures as a leader but that the broader semantic and policy corpora distribute into a much larger pool. China Meteorological Administration ranks 4th in FoR climate and does not appear in FoR environment or concept environment; it ranks 92nd in SDG environment: it is visible as a disciplinary climate leader, invisible as an environmental research institution.

These are significant when doing rankings: they are the difference between an institution appearing as a global leader or not appearing at all, depending on which classification logic is applied. These are differences that carries real consequences when corpus-based rankings inform funding allocation, policy targeting, or institutional benchmarking.

3 systems = 3 lenses on the same topic

The classification systems do not just count different publications, they also describe different research. Using concepts as a cross-cutting lens on the FoR and SDG corpora makes this concrete: the semantic fingerprints of the six corpora diverge substantially even when nominally covering the same scope.

At the climate scope, the three corpora are built around recognisably different kinds of work.

• FoR climate is dominated by physical and observational concepts: temperature, region, period, variables, events, ocean, basin. The vocabulary of Earth science as a measurement discipline, concerned with reconstructing and modelling the physical climate system.

• SDG climate carries a starkly different fingerprint: energy sources, renewable energy, and policy appear prominently alongside climate change and warming. The vocabulary of climate as a policy and engineering response domain, oriented toward transition and mitigation rather than atmospheric observation. The mechanism is SDG 7 (Affordable and Clean Energy), which accounts for 21% of the SDG climate corpus and pulls in renewable energy, grid, and efficiency research that neither FoR nor the concept system classifies as climate research.

• Concept climate sits between the two, with climate, climate change, water, soil, and emission leading. A mixed physical and ecological register.

• The common core shared across all three climate corpora, the concepts that appear in all three top-fifteen lists, is just two terms: climate and water.

At the environment scope, the same divergence persists.

• FoR environment is grounded in measurement: concentration, water, species, soil, and samples.. environmental chemistry and ecology at the bench scale.

• SDG environment leads with climate, species, water, and land, the vocabulary of ecosystem services and the climate-biodiversity nexus.

• Concept environment is the most biologically grounded: species and plants are its top two concepts, followed by area, water, and soil; SDG 2 (Zero Hunger) accounts for 15% of the corpus; food systems research brought in by the agriculture and biomass seeds, captured by no other corpus at any meaningful share.

• The shared core across all three environment corpora is broader: climate, community, ecosystem, plants, sites, soil, species, water. But still only eight concepts out of fifteen ranked per corpus.

Reading across both scopes:

• FoR measures climate and environment as scientific disciplines, defined by how the research community organises its own work.

• SDGs measure them as policy domains, defined by what governments agreed to target, and that framing imports energy engineering that other systems would classify differently.

• Concepts measure them as semantic territories, defined by what the publications are actually about, and that framing reveals how tightly ecological and climate vocabularies are intertwined, and how much food-systems science sits inside the boundary of “environmental research” once the seeds reach agriculture.

Which system, which scope?

The question of which system/scope to use is not easy to answer: it depends on what the analysis is meant to measure and for whom.

FoR is best suited to questions about disciplinary communities: who does climate research as a recognised field of practice, within the research system’s own self-understanding. It is relatively stable and internationally comparable, but it assigns each paper to a discipline rather than to a topic, which means interdisciplinary or applied work may fall outside the relevant codes even when it is substantively environmental. The growth data make this limitation concrete: FoR climate’s below-baseline CAGR (5.7%) is not evidence that climate science is slowing down; it is evidence that an increasing share of climate-relevant research is being conducted and classified outside the Climate Change Science code. The choice between climate and environment in FoR comes down to whether the question is about climate science as a discipline or about the broader constellation of environmental research fields.

SDGs are best suited to questions about policy alignment: which research is oriented toward internationally agreed sustainability goals. The SDG framing is legible to funders and policymakers in a way that FoR codes are not, but it also inherits the politics of how the goals were defined. Choosing SDG 13 alone versus SDG 13 + 14 + 15 is effectively a question about whether “environmental” means “climate” or whether it encompasses marine and terrestrial ecology as well; the SDG architecture makes that choice visible, which is useful.

Concepts are best suited to questions about what the research is semantically about, independent of how it has been categorised institutionally or politically. The seed-based expansion approach allows scope to be defined explicitly and contracted or extended in a principled way; the co-occurrence step means the final corpus reflects how the literature itself organises around the seeds, rather than depending entirely on which seeds the analyst thought to specify. The trade-off is that the corpus is constructed, not inherited: the analyst remains responsible for the seed choices, the filtering thresholds, and the membership rules.. and, as this analysis found, the most adapted rule differs between a narrow seed set (climate) and a broad one (environment). Also, because this was a quick bite, all thresholds and rules in this analysis are not really optimised.

None of these systems are a more accurate representation of environmental research. They are accurate representations of different things: disciplinary structure, policy relevance, and semantic content. The practical recommendation is to state which of these things the analysis is measuring, resist the temptation to treat the corpus as a neutral proxy for “the field,” and run the analysis across more than one system to test whether the conclusions hold.

A corpus is not a prompt: why AI cannot make this decision for you

This is also, incidentally, one of the places where AI-assisted bibliometrics runs into a structural limit. An AI trained on bibliometric studies (even one fed extensive documentation and live examples of how FoR, SDGs, and concepts have been used) would learn which operationalisations are common, not which is appropriate for a given analytical purpose. It would reproduce the confidence of prior choices without surfacing the fact that an equally defensible alternative would have produced different leaders, different growth rates, and a different story about the field.

The instability documented here is precisely the kind of thing such a system would smooth over: each individual study it learned from committed to one definition and proceeded; the variation across studies, and its consequences, would not be visible in any single example. Only deterministic, explicitly declared rules (a named FoR code, a listed set of SDG goals, a documented seed list with stated thresholds) can make the corpus definition auditable and the analytical choice legible. The corpus is not a preprocessing step that can be delegated; it is the decision the analysis is built on.

Appendix

Methodological note: selecting relevant concepts

Rather than selecting a pre-existing category, the concept corpora are assembled through a seed-based expansion process. A set of seed terms is specified first; these seeds are used to retrieve an initial publication set; from those publications, co-occurring concepts are extracted and ranked by frequency, mean relevance, and temporal coverage. Concepts with a mean relevance of 0.45 or above in the seed publications form the selected concept set. A publication then enters the concept climate corpus if it contains at least one seed concept at relevance ≥ 0.50, or at least two selected concepts at relevance ≥ 0.50; the concept environment corpus uses a single selected concept at relevance ≥ 0.50. The asymmetry reflects a deliberate calibration: the narrow climate seed set requires a stronger membership anchor to avoid importing broad ecological literature that co-occurs with climate terms incidentally rather than substantively.

Key takeaways:

• Conversational bibliometrics do not remove methodological authority; they redistribute it: across user expertise, design logic, and enforced infrastructure.

• Architectural choice is not about sophistication but about where responsibility lies: suggestive systems externalise methodological error; constraint-enforced systems internalise it in workflow logic and risk excluding legitimate analyses.

• Transparency is not optional: once workflows move from recommendation to enforcement, those who design them hold methodological authority, which must be inspectable.

Last week I wrote about the eras of bibliometrics, from Eugene Garfield to the advent of AI. I argued that conversational bibliometrics has the potential to lower the expertise required to interrogate data while preserving some of the analytical freedom previously reserved for API or GBQ users.

From experience and experimentation, it became clear that conversational bibliometrics cannot simply be a GPT or Gem with access to schema documentation, examples, and rules. When I built DimQuery GPT, every attempt required substantial domain knowledge on my part to steer the system toward valid queries. DimQuery operated from explicit methodological rules (classification tiers, counting defaults, multiplicity safeguards) but the more constraints I added, the less capable the system became of handling novel questions. OpenAlex’s “vibe coding” example reflects a similar expectation: that documentation and feedback loops alone can prevent hallucinations.

The deeper problem lies in how LLMs are trained: on heterogeneous data drawn from incompatible representational ontologies. But Dimensions, Scopus, Web of Science, OpenAlex, and PubMed each encode a different representational ontology of research, with distinct entity models, identifier systems, classification logics, and citation indexing rules. Research evaluation contexts vary across countries too; national frameworks, funding allocation mechanisms, tenure criteria, and policy priorities all encode implicit evaluative logics. Without a formal representation of the underlying data model, an AI trained on global textual patterns will confidently apply structures that do not belong.

Model Context Protocols (MCPs), as I showed last week, address part of this problem by standardising access to data and constraining execution. They enforce schema-level correctness and tool-bound execution, preventing certain classes of technical hallucination: a query that references a non-existent field, an identifier type applied to the wrong entity. But schema correctness is not methodological validity. A query can be syntactically legal, execute without error, and still answer the wrong question: comparing raw publication counts across countries of different research volumes, or applying Field Citation Ratio (FCR) to publications too recent for citations to have accumulated. This is why I likened MCPs to recipes: they specify permitted ingredients and prevent obvious substitutions, but they do not tell you whether the dish is appropriate for the occasion.

Recipes, however, do not operate on their own. They presuppose ingredients in a pantry (the data), a cook (the user), and a kitchen (the architecture within which action takes place). Recipes constrain execution; pantries constrain representation; kitchens constrain possibility. What the cook can prepare depends on all three. The question, then, is not whether the system can execute a recipe, but where methodological responsibility resides when it cannot distinguish a valid dish from an inappropriate one.

Data in your bibliometric pantry

My previous article focused on infrastructural shifts and only partially addressed the data layer when discussing the open data era. Yet the pantry matters as much as the recipe: bibliometric datasets differ in entity resolution, identifier systems, field classification regimes, coverage boundaries, and in what they consider part of the research record. Some infrastructures centre primarily on publications and citations; others integrate grants, patents, clinical trials, policy documents, or institutional affiliations into a connected ecosystem.

These modelling decisions expand or narrow what kinds of questions can be asked. A journal-centric classification constrains what “field-normalised” can mean. Article-level, machine-learned fields expand certain analytical possibilities. Disambiguated researcher and institutional identifiers enable network-based workflows that would otherwise be fragile or impossible. The inclusion of grants enables forward-looking analyses of research direction and funding concentration; the inclusion of patents makes it possible to trace pathways from research to commercialisation; integrated retraction tracking and affiliation metadata shape how workflows on research integrity or research security can be constructed.

A recent FoSci reflection by my colleague and friend Leslie makes this point from the data perspective: modelling decisions upstream shape the epistemic horizon of downstream analysis. If affiliations are inconsistently resolved, network centrality becomes unstable. If subject classifications are coarse or static, cross-field comparisons inherit that rigidity. If grants or patents are absent, certain forms of horizon scanning or translational analysis simply cannot be formalised. No amount of conversational sophistication can compensate for representational blind spots.

A kitchen typology

Recipes and pantry operate within a kitchen: the architecture of the conversational bibliometric system. When trying to position the AI metascientist, I found it helpful to conceptualise these kitchens along two axes:

• The first is degree of enforcement: how strongly the system constrains execution, from suggestive and reflective systems to constraint-checking and fully prescriptive ones.

• The second is source of methodological authority: whether analytical decisions derive from accumulated practice, engineered design, or explicitly formalised rules.

These dimensions do not trade off against each other. A system can enforce execution constraints rigorously without embedding principled methodological authority, and it can encode deep methodological knowledge while leaving its application entirely to the user. What a system can guarantee about its outputs depends on both dimensions together.

Each architecture therefore optimises for different failure modes rather than different levels of sophistication. Suggestive systems risk methodological error when user expertise is thin; constraint-enforced systems risk blocking legitimate analysis when workflows are incomplete. Designing the kitchen means deciding which failure modes are acceptable in a given context.

My colleague Leslie tested several systems with the question: “What are the top publications of Bangladesh in Alzheimer’s research?” The ambiguities are immediate: what counts as “top”? Over what time period? How is the corpus defined? Even within a single database, corpus construction might rely on text search, classification codes, or ontology terms; each yielding a different set of publications.

Rather than surfacing these choices, one system refused to proceed, citing insufficient “prominent” publications. The issue was not that a constraint was applied, but that it was misapplied. A minimum corpus threshold blocked a valid relative comparison, while more appropriate safeguards (such as clarifying citation windows or requiring explicit corpus definition) were never raised. Effective constraint design depends on understanding the data model well enough to distinguish protective guardrails from arbitrary blocks.

Available kitchens

Every kitchen makes choices about what is possible before the cook arrives. The configurations below differ not in sophistication but in where those choices are made, by whom, and how visibly. The question running through all of them is not whether an LLM can execute a valid query, but how methodological authority is embedded in the architecture itself and who is responsible when it fails.

The co-pilot: practice-based authority, suggestive enforcement

Individuals who have conducted bibliometric analyses for years accumulate reports and accompanying scripts (e.g., consultancy reports and their analyses). These encode recurring analytical patterns: how country comparisons are structured, which normalisation methods are applied, which edge cases were handled. Using this material to prompt or train a language model produces a co-pilot: a system that suggests analytical approaches based on what analysts have previously done, scaling memory, surfacing relevant precedent, and reducing duplication across teams.

For expert analysts, this is genuinely powerful, as it avoids manually going back to previous analyses. The system keeps the full analytical space open, surfaces what has worked before, and imposes no guardrails that might block legitimate exploratory work. However, it cannot detect when historical practice no longer fits current infrastructure, and it has no record of errors analysts consistently avoided. If no script contains a particular mistake, there is no evidence the mistake is possible.

The annotated practice codex: expert-curated authority, suggestive enforcement

This variant of the co-pilot addresses its lack of authority directly. Rather than mining raw patterns, analysts review, annotate, and derive rules from those annotations. Authority is no longer purely practice-based: an expert has evaluated each pattern and determined whether it reflects a methodological choice or a contingent historical decision.

The system can explain why specialisation ratios are appropriate for country comparisons, because an expert confirmed it as methodologically sound rather than recommending it simply because it appeared frequently in the record. Enforcement remains suggestive; the system explains and recommends but does not prevent the user from proceeding without normalisation or constructing an analysis the codex would not endorse.

DimQuery GPT and OpenAlex vibe coding: two approaches to suggestive enforcement

DimQuery GPT and OpenAlex vibe coding sit at the suggestive end of the enforcement spectrum but derive their authority from different sources. DimQuery GPT was built on explicit methodological rules (classification tiers, counting defaults, multiplicity safeguards) making it rule-based in intent. OpenAlex vibe coding offers no equivalent methodological layer: its LLM guide addresses API mechanics, sampling, and rate limits, but contains nothing about normalisation, counting methods, or analytical validity. Its authority is purely design-based, grounded in schema structure rather than bibliometric methodology. Both demonstrate the ceiling of suggestive enforcement: rules or design choices can be communicated, but not guaranteed.

SciSciGPT: design-based authority, reflective enforcement

SciSciGPT, introduced in Nature Computational Science (Shao et al., 2025), orchestrates multiple agents to decompose research questions into sub-tasks, execute analyses against databases, and evaluate outputs through iterative self-assessment. Task decomposition imposes structure; multi-agent coordination handles complex, multi-step questions; and reflective evaluation catches errors a single-pass system would miss.

Authority is design-based: analytical quality depends on how agents are structured, what evaluation criteria are specified, and how decomposition strategies are engineered. If field normalisation is included in the evaluation criteria, it will be applied; if not, it may be omitted. This makes SciSciGPT well-suited to exploratory research contexts where the analytical question itself is still being formed: the system can pursue unconventional constructions that a constraint-enforced architecture would refuse. Its failure mode is that methodological validity depends on how carefully the evaluation criteria were specified at design time, not on normative rules embedded in the system.

Clarivate AI Workflow Agents: design-based authority, constraint-checking enforcement

Clarivate’s AI Workflow Agents, embedded across their Web of Science, intellectual property, and life sciences platforms, represent a commercially deployed instance of what appears to be design-based authority combined with constraint-checking enforcement. Based on publicly available marketing and investor materials, authority appears to derive from decades of expert curation applied to proprietary data infrastructure (enriched citation networks, patent records, drug safety alerts) rather than from an explicitly formalised bibliometric methodology.

The workflow logic itself is not publicly documented.

The AI metascientist: rule-based authority, constraint-enforced

The AI metascientist, as I currently envision it, embeds methodological discipline directly in infrastructure. Its authority would be rule-based, grounded in a formal domain representation specifying recognised entities, admissible workflows, valid metrics, and the conditions under which analysis may proceed.

Enforcement would be structural rather than advisory: cross-field comparisons would require normalisation as a precondition; inadequate citation windows or sparse corpora would trigger qualification or refusal. Natural language questions would map onto predefined workflows encoding corpus construction, admissible identifiers, metric validity, and interpretive limits.

The aim is to reduce risks such as methodological error and silent analytical failure, at the cost of exploratory breadth. Novel constructions would need to be formally incorporated into the workflow library before execution, or users would have to explicitly step outside the workflow, acknowledging the shift in responsibility. The accompanying workflow-based methods textbook is intended to make those commitments transparent and contestable.

This is a proof of concept rather than a finished system, intended to test whether workflow-level governance can be made both enforceable and inspectable.

A worked example across architectures

Consider the question: “How central were Humboldt professorship recipients to their research organisations?”

Even this simple question unfolds in four distinct stages:

• Corpus definition: who counts as a recipient, and over what time period? Does the user already have a list, or must recipients be inferred from grants or publications?

• Workflow selection: which centrality measure is appropriate, and relative to which comparison baseline?

• Execution: how is the co-authorship network constructed, and are identifiers and time windows sufficient for stable inference?

• Interpretation: under what conditions can differences in centrality be meaningfully reported?

Some architectures, such as DimQuery, co-pilots, and annotated codex, rely on a single conversational model to perform all four roles at once, assuming that decomposition and methodological safeguards will emerge from prompting and precedent. Others, including SciSciGPT, Clarivate AI workflow, and the AI metascientist, externalise at least part of this logic into orchestration, workflow, or validation layers, making the separation structural rather than incidental.

In suggestive systems, most responsibility remains with the user. Reflective systems automate parts of the translation and internal checking. Prescriptive systems formalise the stages as workflows and enforce admissibility conditions before execution.

Which kitchen, and for whom?

The choice of architecture depends less on sophistication than on governance context. The table below summarises where each configuration fits and how responsibility is distributed.

Across all of these, the pantry matters: the richness and modelling choices of the underlying data determine what kinds of workflows can even be formalised, and therefore where enforcement can meaningfully operate.

Authority and responsibility

If workflows are formalised and enforced, those who design them hold methodological authority over what counts as valid practice, which metrics become defaults, and under what conditions the system will refuse a request. These are not technical questions but questions of infrastructural power. Once analytical constructions are embedded in executable workflows, they cease to be recommendations and become operational constraints. The system does not merely suggest what is methodologically appropriate; it determines what is possible.

Transparency is therefore essential: the workflow library must be inspectable, and the formal representation must document not only what the system does but why; which normalisation choices were made, which corpus definitions were preferred, and under what conditions the system will decline to proceed. Without this, the system does not just risk opacity; it risks laundering contested methodological choices as technical necessity.

What comes next for the AI metascientist

The proof of concept is moving into its first delivery phase, with the aim of validating the core technical and governance assumptions: that workflows can be formalised and enforced, that orchestration over a structured domain representation is feasible, and that validation and auditability are practical rather than aspirational. In parallel, I am formalising this domain representation into a workflow-based methods textbook, making explicit the methodological commitments the system will enforce, so that those commitments can be inspected, contested, and improved.

Key takeaways:

• Bibliometric analyses are shaped by infrastructural design choices as much as by explicit methodological decisions.

• Conversational AI for bibliometrics requires external structural constraints (Model Context Protocols) to prevent hallucinations and ensure methodological validity.

• Once mediated by MCPs, bibliometric systems implicitly govern which methods and questions are legitimate.

Bibliometric systems are often treated as neutral tools for measuring scientific activity, with methodological debates focusing on indicator choice, data coverage, or normalisation strategies. Yet bibliometric knowledge is also shaped by infrastructure (Wouters, 2014): how data are accessed, which classifications and analyses are available out of the box, and which assumptions are embedded in default representations and metrics. As conversational interfaces and large language models are increasingly tested as new entry points to bibliometric data, these infrastructural choices become more consequential. In this Research Musings, I show that conversational bibliometrics is not a simple interface change: it switches power over bibliometric knowledge by embedding methodological decisions in infrastructure rather than leaving them to interpretation.

From printed data to interactive systems

Data sharing has evolved alongside computers, shifting how data are accessed and how knowledge is shaped. Early technical data were often shared through printed tables and handbooks: production was computer assisted, and consumption was static. When Eugene Garfield introduced the Science Citation Index (SCI) in the 1960s, it relied on computers for large-scale citation matching, while dissemination was annual and printed. Users did not query a database, but they consulted a static set of printed tables, accompanied by expert interpretation. Computers enabled scale, but interpretation remained largely centralised (Garfield, 1979). As computing power and personal access expanded, data could be distributed digitally: first via magnetic tape and later on CD-ROMs, but it remained disconnected and local, and evolved only via static snapshots. Bibliometrics grew more sophisticated (e.g., co-citation analysis, bibliographic coupling, network-based approaches), but their use was largely confined to specialists with both data access and technical expertise.

The Internet marked a further shift by introducing possibilities to search, filter, and later dashboards. Users could explore, but only within the boundaries set by a website application; questions were shaped by available filters. APIs and SQL-based access reopened the data layer. For the first time, bibliometric data could be embedded into other systems, recombined across sources, and used to build custom analyses. Open infrastructures such as OpenAlex aggregate multi-sourced bibliometric data into a unified representational model and provide public interfaces to these records without requiring paid access. However, they still embed consequential modelling decisions about entities, classifications, and coverage that are difficult to bypass without reconstructing the database itself. What is opened, in this sense, is access and reuse, not control over the data-making process.

At the same time, indicators became the dominant tool through which bibliometric results were communicated. Indicators such as the journal impact factor and the h-index reduced complex scholarly performance to single numerical values while making problematic assumptions. Both metrics incentivised gaming behaviours such as self-citation rings and citation clubs. Table 1 below summarises this progression, showing how bibliometric infrastructures moved from restricted data release to formal openness that nonetheless remained epistemically inaccessible.

These assumptions were reinforced by how research was classified. Historically, bibliometric datasets used a journal-centric model: subject categories were assigned at the journal level and then applied to all articles within it. This approach, rooted in Garfield’s original citation indexing framework and maintained by Clarivate, treated journals as coherent disciplinary containers and made journal-level normalisation both practical and conceptually central. Dimensions marked a structural break by introducing article-level classification as a default (Hook et al., 2018). Using machine learning, each publication could be assigned to multiple fields independent of the journal in which it appeared. This shifted attention away from the journal as the primary epistemic unit and made interdisciplinarity visible without specific analysis. It also mirrored a broader shift in access: instead of printed journals circulating through libraries, articles were now discovered independently of their journal through aggregated databases.

This shift expanded the available analytical tools but also reshaped how bibliometric knowledge was produced and used. Although APIs and SQL-based access enabled sophisticated analyses beyond precomputed indicators, they also reinforced reliance on aggregates and shortcuts for most users. As a result, bibliometric knowledge continued to reflect assumptions embedded in systems designed primarily for expert use, even as formal access to the data layer became more open.

The move from curated indicators to APIs and SQL was often framed as a democratisation of bibliometric knowledge. In practice, it relocated responsibility for understanding data construction, coverage, and methodological implications almost entirely to the user: an arrangement that implicitly assumed substantial bibliometric expertise. Conversational bibliometrics aims to rebalance this by allowing users to articulate complex questions without direct engagement with schemas or query languages. However, this shifts the responsibility for methodological validity back to the data provider, who must ensure that conversational access is constrained by explicit and enforceable analytical rules, and that these constraints are transparent to users.

But, as bibliometric systems became more open and powerful, they also became harder to use correctly. The relationship between what users can do and what they need to know has shifted across eras in revealing ways.

The graph highlights that expertise requirements and analytical freedom do not share a linear relationship. While access through curated APIs and GBQ (4a) granted users significant freedom, reaching the “maximal freedom” of the Open Data era (4b) imposes a massive technical tax, requiring users to reconstruct databases and handle raw metadata to bypass embedded modelling decisions. In both cases, this agency depends on high levels of technical and methodological expertise, effectively shifting the entire responsibility for validity and interpretation onto the user.

Conversational bibliometrics (5) attempts to reconfigure this by lowering the expertise barrier at the point of interaction, but without restoring equivalent safeguards for methodological validity. As a result, conversational systems make the non-linear relationship between expertise and freedom consequential: they lower the perceived expertise required to perform analyses, while leaving the burden of methodological validity unresolved and largely invisible.

Limits of ‘conversational bibliometrics’

Could bibliometrics have its “ChatGPT moment”? What would that even mean? Researchers, institutions, funders, and publishers might imagine conversing with bibliometric systems to explore trends, test assumptions, or discuss the implications of decisions. Custom GPTs, Gems, and direct LLM API integrations have already attempted this by providing models with schema documentation and task-specific instructions.

In my experience, these approaches (using post-trained conversational LLMs as the primary analytical engine) often fail. Even when supplied with controlled vocabularies, explicit identifiers, and detailed instructions, LLMs will hallucinate non-existent database fields, substitute fuzzy name matching for required identifiers (such as GRID IDs), or bypass methodological constraints. The outputs are fluent and plausible, but analytically invalid.

These failures are not primarily a prompt-engineering problem. In my own experiments, adding more instructions or documentation often degrades performance rather than improving it. As input length increases, reasoning deteriorates well below context window limits, and models revert to pattern completion drawn from training data (for example, repeatedly introducing a fictitious citation.year field). As a result, even deterministic configurations can produce invalid analyses. The issue is not conversational access itself, but the assumption that analytical validity can be reliably enforced through post-training and prompting rather than through executable constraints.

A second, less visible limitation concerns methodology rather than correctness. Post-trained bibliometric LLMs do not operate within a single methodological tradition; they borrow freely from adjacent fields such as information retrieval, network science, and machine learning. While this methodological borrowing can be productive and innovative, and bibliometrics as an applied field has often benefitted from it, it also risks presenting non-canonical or field-external techniques as default bibliometric practice. In conversational settings, where outputs are framed as answers rather than suggestions, this blurs the distinction between exploratory analysis and established method, and risks treating attractive analogies as canon.

Executable constraints make this distinction explicit. By formalising which workflows are recognised as valid defaults, MCP-mediated systems can support methodological innovation without silently redefining what counts as bibliometric practice.

Model Context Protocols

That is where Model Context Protocols (MCPs) come in. MCPs are standardised interfaces that allow LLMs to access external tools and data sources through controlled, auditable gateways. MCPs matter here not only because they reduce errors, but because they determine which analytical constructions can exist at all. Think of it like baking cookies. You could give someone unrestricted access to your kitchen and say “make chocolate chip cookies” and hope for the best. Or you could provide a detailed recipe that specifies butter temperature (room temperature vs. melted vs. cold creates different textures), mixing techniques, and baking times. The difference between “cookies” and “delicious cookies” lies in these seemingly small details, as illustrated in this serious cookie experiment. MCPs are the recipe that enforces data (ingredients) are properly handled; they do not define analytical workflows, but they determine which formally specified workflows can be executed, logged, and reproduced within the system. Once analytical workflows are formalised and passed through MCP-enforced execution, they cease to function as informal methodological guidance and instead become durable, auditable procedures.

In bibliometric systems, MCPs can be implemented as gateways stored on the server side of platforms like Dimensions, where they can be versioned, curated, and linked to specific categories of tasks. This means researchers can use any LLM interface they prefer, including proprietary tools like ChatGPT, Gemini, or Claude, or open models such as GPT-OSS or Mistral, confident that the MCP gateway ensures consistent, methodologically valid execution behind the scenes.

When a user asks a question, the MCP gateway structures and validates the analytical workflow before any data is accessed. This ensures that bibliometric tasks follow explicit rules (using GRID IDs rather than fuzzy name matching, or enforcing field-normalised citation windows) even if the user never sees these constraints directly. In baking terms, this makes sure that no ingredient is substituted (baking powder leads to different cookies than baking soda, as demonstrated in this video), no steps are skipped, or timings are adjusted.

In this framing, a robust conversational system does not rely on a single cook. One component interprets the request and identifies the right recipe. Another follows that recipe precisely, applying the defined rules and transformations. A third checks that the result makes sense given the inputs and assumptions. Conversation remains central, it is how questions are asked, but correctness depends on separating interpretation from execution and enforcing structure at each step.

Without MCPs, conversational bibliometrics risks producing answers that are fluent, confident even, ...but wrong. With MCPs, LLMs become more useful: they stop improvising in the kitchen and become a trained assistant working with a trusted cookbook.

Who designs the cookbook?

This shift from indicators to workflows introduces a new form of epistemological infrastructure, and with it, new questions about authority and participation. If bibliometric knowledge increasingly circulates as executable procedures rather than static outputs, then control over workflow design becomes a form of methodological power.

Design choices about bibliometric indicators extend beyond what is displayed to how analyses are constructed by default. For instance, while Dimensions does not surface the h-index as a default summary measure (to reflect critiques of composite indicators and alignment with recommendations such as DORA), it does apply field-normalised citation metrics as a standard analytical baseline. This choice privileges comparative interpretability across disciplines while downplaying raw citation counts, illustrating how bibliometric platforms already govern knowledge production by embedding methodological preferences into defaults rather than prohibiting alternative analyses. MCP-mediated workflows extend this logic: governance shifts from curating visible indicators to defining which analytical constructions are standard, repeatable, and implicitly endorsed.

Table 2 situates this shift within a longer scientometric trajectory, showing how authority over valid methods has moved from expert interpretation to infrastructural enforcement.

Conversational bibliometrics is not primarily a usability innovation. Once mediated by MCPs, it becomes an infrastructural regime in which methodological defaults govern what counts as legitimate bibliometric knowledge. Data producers therefore assume a new form of epistemic responsibility. At minimum, this means being explicit about which analytical constructions are defaults, which methodological choices are embedded in workflows, and why. Transparency here is not an add-on to the system but a condition of its legitimacy.

To infinity, with a recipe

If we want to build conversational bibliometrics, therefore, we cannot rely on conversational fluency alone. The arguments above point toward a specific architectural consequence: separating interpretation from execution, formalising analytical workflows, and making methodological defaults explicit and inspectable.

Recent work in Nature Computational Science has begun to explore multi-agent AI collaborators such as SciSciGPT, which orchestrate specialist agents across literature search, data access, analysis, and evaluation within auditable workflows. These systems operationalise some of the same structural separations argued for here — between interpretation, execution, and validation — but they are primarily designed to support exploratory inquiry and sense-making, and continue to rely on substantial domain expertise from the user to assess methodological appropriateness. As such, they remain domain-specific research prototypes rather than general, governed infrastructures that assume responsibility for methodological validity in conversational analytical systems.

In the next piece, I introduce the concept of an AI metascientist: not as a single model, but as a governed system architecture in which conversational interfaces, executable workflows, and validation layers are deliberately separated and coordinated. It offers one possible answer to the question posed here — how to support conversational inquiry while making methodological responsibility explicit, rather than leaving it implicit or deferred to user judgment.

Bibliography

Garfield, E. (1979) Citation indexing: its theory and application in science, technology, and humanities. New York, NY: Wiley.

Hook, D. W., Porter, S. J., & Herzog, C. (2018) Dimensions: Building context for search and evaluation. Frontiers in Research Metrics and Analytics, 3, 23.

Wouters, P. (2014) The citation: From culture to infrastructure. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing multidimensional indicators of scholarly impact (pp. 47–66). Cambridge, MA: MIT Press.

Key takeaways:

• Bibliometrically, Dimensions’ concepts can be used to identify baseline, emerging, and frontier topics by combining relevance-based filtering with changes in concept use over time.

• Thematically, differences in abstract-writing practices across Fields of Research strongly shape which concepts appear prominent, leading to large field-specific differences in concept categories and apparent size.

Whoever has tried using the publications’ concepts in Dimensions has probably realised how messy and noisy these can be. Concepts are extracted bottom-up: they are not curated like MeSH or other controlled vocabularies, which means you get what is in the text, not what someone decided was important.

That gives flexibility but also frustration: we can explore topics without being constrained by predefined labels; but we are often staring at long lists that mix highly specific terms like mRNA with generic ones such as patient, student, reaction, method, or temperature. In some analyses, these can be relevant, but when trying to detect conceptual change or trace emerging trends, they create a lot of noise.

The usual solution is to filter, but instead of manually building stopword lists or guessing what is too generic, Dimensions provides a useful metric: concept relevance.

What is relevance?

Relevance is a score between 0 and 1 that signals how central a concept is to a given publication. These scores are generated using Natural Language Processing (NLP) and reflect two main factors:

• Prominence: A term gets a ‘boost’ if it appears in the title or is used frequently within the abstract.

• Specificity: The score is relative; a term is more relevant if it appears frequently in one paper but is relatively rare across the rest of the research database.

So, a high score (0.8+) identifies the primary focus of the research, while a low score (0.1) suggests the concept is only mentioned as broad context.

Below an example of all concepts for a recent article In-depth Research Impact Summarization through Fine-Grained Temporal Citation Analysis, ordered by relevance:

{'concepts_scores': [
  {
  "concepts_scores": [
    {"concept": "impact of scientific publications","relevance": 0.72},
    {"concept": "citation intent","relevance": 0.649},
    {"concept": "citation counts","relevance": 0.647},
    {"concept": "citation analysis","relevance": 0.64},
    {"concept": "traditional metrics","relevance": 0.597},
    {"concept": "correct citation","relevance": 0.592},
    {"concept": "expert feedback","relevance": 0.586},
    {"concept": "evaluation framework","relevance": 0.576},
    {"concept": "citations","relevance": 0.574},
    {"concept": "scientific publications","relevance": 0.557},
    {"concept": "task","relevance": 0.53},
    {"concept": "summarization","relevance": 0.504},
    {"concept": "metrics","relevance": 0.463},
    {"concept": "human correlate","relevance": 0.45},
    {"concept": "framework","relevance": 0.422},
    {"concept": "feedback","relevance": 0.402},
    {"concept": "summary","relevance": 0.374},
    {"concept": "evaluation","relevance": 0.37},
    {"concept": "correction","relevance": 0.352},
    {"concept": "intention","relevance": 0.35},
    {"concept": "research","relevance": 0.341},
    {"concept": "improvement","relevance": 0.336},
    {"concept": "publications","relevance": 0.331},
    {"concept": "insights","relevance": 0.304},
    {"concept": "impact","relevance": 0.296},
    {"concept": "breakthrough","relevance": 0.287},
    {"concept": "critique","relevance": 0.285},
    {"concept": "praise","relevance": 0.282},
    {"concept": "evolution","relevance": 0.273},
    {"concept": "Professor","relevance": 0.271},
    {"concept": "analysis","relevance": 0.265},
    {"concept": "correlation","relevance": 0.221},
    {"concept": "count","relevance": 0.208},
    {"concept": "confirmation","relevance": 0.182},
    {"concept": "Impact summary","relevance": 0.175}
]}

At the top of the list, we see concepts that clearly define the paper’s contribution (impact of scientific publications, citation intent, citation analysis). Further down, more generic terms appear (framework, analysis, research). At the bottom, the concepts become increasingly vague (or dare I say, not relevant?). These might be of interest if we wanted to identify publications about correlation for instance, but not when we want to see what is emerging.

To understand what relevance value I should expect for concepts in publications considered novel at the time of writing, I checked two:

Attention is all you need (2017):

{"concept": "BLEU score", "relevance": 0.685},
{"concept": "state-of-the-art BLEU scores", "relevance": 0.667},
{"concept": "machine translation tasks", "relevance": 0.64},

Highly accurate protein structure prediction with AlphaFold (2021):

{ "concept": "protein structure", "relevance": 0.713},
{ "concept": "accurate protein structure prediction", "relevance": 0.704},
{ "concept": "multi-sequence alignment", "relevance": 0.684},

A quick sanity check showed that, in Dimensions, 50% of publications have at least one concept above 0.65, indicating that by using this threshold we will cover enough of the corpus. I therefore used this value as a threshold to retain core topics while filtering generic noise.

Baseline and emerging concepts

The relevance threshold of 0.65 gives us the most relevant concepts for the publications, and depending on their growth or impact, we identified three sets of concepts: baseline, emerging, and frontier.

I used the share of publications in which it occurs, year by year, as well as the field relative impact (FCR—Field Citation Ratio).

• Baseline concepts are those that occupy a large share of the literature over long periods. They form the semantic infrastructure of a field: not necessarily growing, but persistently present.
  In practice, these are concepts with the highest and most stable publication share over the last decade

• Emerging concepts are those whose share of publications has increased substantially in recent years. They appear in a growing portion of the field’s literature and are becoming part of its mainstream vocabulary.
  In practice, these are identified by sustained recent growth in publication share, combined with a minimum scale to exclude short-lived fluctuations.

• Frontier concepts are those that show strong recent growth while still accounting for a relatively small share of the literature. They appear in papers with higher-than-expected citation intensity and remain concentrated rather than widely used across the field.
  In practice, these are identified by rapid recent growth and higher-than-expected citation activity in their research context, while remaining limited in overall publication share.

I selected two Fields of Research of interest, one STEM and one HASS.

• 34. Chemical Sciences

• 44. Human Society

Concepts in 34. Chemical Sciences

Baseline

Below are the persistent concepts that define 34. Chemical Sciences infrastructure: catalysts, reactions, synthesis methods, and characterisation techniques. Each occupies 2-5% of the field's abstracts; these are small share that reflect chemistry's highly specialised vocabulary, but stable over time.

Emerging

Each of the emerging concepts has appeared in 100+ publications (30+ recently) with sustained annual growth exceeding 10%. At 0.1-0.2% of the field, they remain specialised compared to baseline terms, but are appearing in a growing share of publications, particularly around battery technologies (Zn anode, capacity retention) and advanced catalysis (S-scheme heterojunction).

Frontier

Frontier concepts need just 30 total publications and 10 recent ones, but must grow at 30%+ annually with above-average citation activity. At 0.05-0.2%, these are concentrated rather than widespread. Current frontier concepts cluster around battery technologies, materials modification strategies, and emerging AI applications.

Concepts in 44. Human Society

Baseline

In contrast to 34. Chemical Sciences, 44. Human Society research concentrates on broad, high-level concepts that can frame diverse studies. Terms like policy, gender, and violence each appear in 20-50% of publications; 10 times larger than chemistry's baseline concepts. This reflects different abstracting practices: social science abstracts reuse established conceptual vocabulary to situate varied empirical work.

Emerging

Each concept has appeared in 100+ publications (30+ recently) with sustained annual growth exceeding 10%. At 0.5-2% of the field, they show growing adoption particularly around methodological approaches (qualitative descriptive method, juridical approach, document analysis) and contemporary research priorities (nature-based solutions, research gaps in developing economies).

Frontier

Frontier concepts need just 30 total publications and 10 recent ones, but must grow at 30%+ annually with above-average citation activity. At 0.5-10%, these remain smaller than baseline concepts but show rapid adoption. Unlike 34. Chemical Sciences' focus on materials and technologies, 44. Human Society's frontier is dominated by methodological terminology: qualitative approaches, mixed methods, community-based research designs.

Comparing these two fields reveals two key patterns.

First, frontier concepts differ thematically by field. In 44. Human Society, they are almost exclusively methodological (qualitative approach, mixed-methods approach, community engagement), while in 34. Chemical Sciences they cluster around materials and optimisation strategies (aqueous zinc-ion batteries, pouch cells, interface engineering).

Second, publication shares are roughly 10 times larger in 44. Human Society than 34. Chemical Sciences. 34. Chemical Sciences distributes attention across thousands of specific materials and methods, and even core baseline concepts like catalyst reach only 2-5% of abstracts. 44. Human Society research, by contrast, concentrates on fewer, broader framing concepts: policy and gender each appear in 20-50% of publications. This reflects different abstracting practices: chemistry emphasises specific systems and compounds, while social science abstracts use established high-level concepts to contextualise diverse empirical work.

Publication shares should therefore be interpreted within fields, not across them. A concept appearing in 1% of 34. Chemical Sciences abstracts may represent a more significant shift than the same percentage in 44. Human Society, simply because chemistry distributes its conceptual vocabulary so much more widely. In 44. Human Society, where baseline concepts already occupy large shares, 1% is relatively small; in 34. Chemical Sciences, where even core concepts reach only 2-5%, 1% represents substantial presence.

Conclusion

Using Dimensions' bottom-up concepts with their relevance scores, we have identified sets of baseline, emerging, and frontier concepts. The size and character of these categories differ drastically between fields; publication shares must be interpreted within field contexts, not across them. The thresholds used here (0.65 relevance, 100+ publications for emerging, 30+ for frontier) were selected empirically but can be adjusted for different analytical goals: tracking stability, diffusion, or early signals.

The full query is included below for anyone who wants to adapt the thresholds or apply the approach to a different corpus.

Bonus

If you want to peruse the data: here are three tables.

Baseline

Emerging

Frontier

Code

As usual, this was co-written by my Custom GPT (DimQuery Assistant), contact me if you want access. I also have Gem version, but there is less context window and I have not tested it as much.

-- Concept baseline vs emerging classification (Dimensions on GBQ)
-- Adds Fix A (absolute volume gating) + Fix B (sustained growth gating)

DECLARE start_year INT64 DEFAULT 2010;
DECLARE analysis_end_year INT64 DEFAULT 2025;
DECLARE eps_share FLOAT64 DEFAULT 0.0005;

DECLARE min_pubs_total INT64 DEFAULT 10;
DECLARE min_pubs_recent3 INT64 DEFAULT 5;

-- Fix A thresholds (emergence volume gate)
DECLARE min_pubs_total_emerge INT64 DEFAULT 100;
DECLARE min_pubs_recent3_emerge INT64 DEFAULT 30;

WITH
base AS (
  SELECT
    p.id AS publication_id,
    p.year,
    for1.code AS for_code,
    CONCAT(for1.code, ". ", for1.name) AS for_name,
    c.concept,
    c.relevance,
    p.metrics.field_citation_ratio AS fcr
  FROM `dimensions-ai.data_analytics.publications` p
  CROSS JOIN UNNEST(p.concepts_relevant) AS c
  CROSS JOIN UNNEST(p.category_for.first_level.full) AS for1
  WHERE p.year BETWEEN start_year AND analysis_end_year
),

concept_year AS (
  SELECT
    year,
    for_code,
    for_name,
    concept,
    COUNT(DISTINCT publication_id) AS pub_count,
    SUM(relevance) AS relevance_weighted_pubs,
    AVG(relevance) AS mean_relevance,
    AVG(fcr) AS mean_fcr
  FROM base
  GROUP BY year, for_code, for_name, concept
),

field_year_totals AS (
  SELECT
    year,
    for_code,
    COUNT(DISTINCT publication_id) AS field_pub_count
  FROM base
  GROUP BY year, for_code
),

series AS (
  SELECT
    cy.year,
    cy.for_code,
    cy.for_name,
    cy.concept,
    cy.pub_count,
    cy.relevance_weighted_pubs,
    cy.mean_relevance,
    cy.mean_fcr,
    fyt.field_pub_count,
    SAFE_DIVIDE(cy.pub_count, fyt.field_pub_count) AS pub_share,
    SAFE_DIVIDE(cy.relevance_weighted_pubs, fyt.field_pub_count) AS relevance_weighted_share
  FROM concept_year cy
  JOIN field_year_totals fyt
    USING (year, for_code)
),

field_max_year AS (
  SELECT
    for_code,
    MAX(year) AS max_year
  FROM series
  GROUP BY for_code
),

features AS (
  SELECT
    s.*,

    LAG(pub_share, 1) OVER w AS pub_share_lag1,
    LAG(pub_share, 3) OVER w AS pub_share_lag3,
    LAG(pub_share, 5) OVER w AS pub_share_lag5,
    LAG(pub_share, 6) OVER w AS pub_share_lag6,

    (pub_share - LAG(pub_share, 3) OVER w) AS share_delta_3y,

    ((pub_share - LAG(pub_share, 3) OVER w) * field_pub_count) AS delta_pubs_equiv_3y,

    SAFE_DIVIDE(
      pub_share - LAG(pub_share, 1) OVER w,
      NULLIF(LAG(pub_share, 1) OVER w, 0)
    ) AS yoy_share_growth,

    CASE
      WHEN LAG(pub_share, 5) OVER w > 0 AND pub_share > 0
        THEN POW(SAFE_DIVIDE(pub_share, LAG(pub_share, 5) OVER w), 1.0/5.0) - 1.0
      ELSE NULL
    END AS cagr_share_5y,

    CASE
      WHEN LAG(pub_share, 3) OVER w > 0 AND pub_share > 0
        THEN POW(SAFE_DIVIDE(pub_share, LAG(pub_share, 3) OVER w), 1.0/3.0) - 1.0
      ELSE NULL
    END AS cagr_share_recent3y,

    CASE
      WHEN LAG(pub_share, 6) OVER w > 0 AND LAG(pub_share, 3) OVER w > 0
        THEN POW(SAFE_DIVIDE(LAG(pub_share, 3) OVER w, LAG(pub_share, 6) OVER w), 1.0/3.0) - 1.0
      ELSE NULL
    END AS cagr_share_past3y,

    (CASE
      WHEN LAG(pub_share, 3) OVER w > 0 AND pub_share > 0
        THEN POW(SAFE_DIVIDE(pub_share, LAG(pub_share, 3) OVER w), 1.0/3.0) - 1.0
      ELSE NULL
    END)
    -
    (CASE
      WHEN LAG(pub_share, 6) OVER w > 0 AND LAG(pub_share, 3) OVER w > 0
        THEN POW(SAFE_DIVIDE(LAG(pub_share, 3) OVER w, LAG(pub_share, 6) OVER w), 1.0/3.0) - 1.0
      ELSE NULL
    END) AS share_acceleration,

    MIN(IF(pub_share >= eps_share, year, NULL)) OVER w AS first_year_over_eps,

    SUM(pub_count) OVER w AS pubs_total,

    SUM(IF(year >= (analysis_end_year - 2), pub_count, 0)) OVER w AS pubs_recent3,

    SAFE_DIVIDE(
      SUM(IF(year >= (analysis_end_year - 2), pub_count, 0)) OVER w,
      NULLIF(SUM(pub_count) OVER w, 0)
    ) AS recent_pub_fraction_3y

  FROM series s
  WINDOW w AS (PARTITION BY s.for_code, s.concept ORDER BY s.year)
),

aged AS (
  SELECT
    f.*,
    (year - first_year_over_eps) AS concept_age_years,
    (pubs_total >= min_pubs_total) AS pass_min_total,
    (pubs_recent3 >= min_pubs_recent3) AS pass_min_recent3,

    -- Fix A gates (precompute for clarity)
    (pubs_total >= min_pubs_total_emerge) AS pass_emerge_total,
    (pubs_recent3 >= min_pubs_recent3_emerge) AS pass_emerge_recent3,

    -- Fix B gate: sustained positive recent growth (and not collapsing in the prior window)
    (cagr_share_recent3y IS NOT NULL AND cagr_share_recent3y >= 0.10) AS pass_recent_growth,
    (cagr_share_past3y IS NULL OR cagr_share_past3y >= 0.00) AS pass_not_negative_past
  FROM features f
),

labeled AS (
  SELECT
    a.*,
    CASE
      WHEN pubs_total < min_pubs_total OR pubs_recent3 < min_pubs_recent3 THEN 'noise / too sparse'

      WHEN concept_age_years >= 10
           AND cagr_share_5y IS NOT NULL
           AND cagr_share_5y <= -0.05
           AND recent_pub_fraction_3y < 0.20
        THEN 'declining'

      WHEN (cagr_share_recent3y IS NOT NULL AND cagr_share_recent3y >= 0.30)
           AND (yoy_share_growth IS NOT NULL AND yoy_share_growth <= 0.00)
           AND pub_share < 0.03
        THEN 'transient / hype'

      -- Frontier: early, low share, fast growth; do NOT require diffusion-scale volume
      WHEN pub_share < 0.03
          AND cagr_share_recent3y IS NOT NULL AND cagr_share_recent3y >= 0.30
          AND (concept_age_years <= 7 OR recent_pub_fraction_3y >= 0.60)
          AND pubs_total >= 30
          AND pubs_recent3 >= 10
        THEN 'frontier'


      -- Emerging: add Fix A + Fix B gates
      WHEN (
             (cagr_share_5y IS NOT NULL AND cagr_share_5y >= 0.20)
             OR (share_acceleration IS NOT NULL AND share_acceleration >= 0.15)
           )
           AND (concept_age_years <= 7 OR recent_pub_fraction_3y >= 0.60)
           AND pass_emerge_total AND pass_emerge_recent3
           AND pass_recent_growth AND pass_not_negative_past
        THEN 'emerging'

      WHEN concept_age_years >= 10
           AND cagr_share_5y IS NOT NULL
           AND ABS(cagr_share_5y) <= 0.05
           AND pub_share >= eps_share
        THEN 'baseline'

      ELSE 'other / ambiguous'
    END AS concept_status
  FROM aged a
),

snapshot AS (
  SELECT *
  FROM labeled
  WHERE year = analysis_end_year
),

ranked_snapshot AS (
  SELECT
    s.*,

    ROW_NUMBER() OVER (
      PARTITION BY for_code, concept_status
      ORDER BY
        -- Primary ranking: different logic per status
        CASE
          WHEN concept_status = 'emerging'
            THEN delta_pubs_equiv_3y
          WHEN concept_status = 'frontier'
            THEN mean_fcr
          WHEN concept_status = 'baseline'
            THEN pub_share
          WHEN concept_status = 'declining'
            THEN pub_share
          ELSE NULL
        END DESC,

        -- Secondary ranking
        CASE
          WHEN concept_status = 'emerging'
            THEN share_delta_3y
          WHEN concept_status = 'frontier'
            THEN cagr_share_recent3y
          ELSE NULL
        END DESC,

        -- Tertiary ranking
        CASE
          WHEN concept_status = 'emerging'
            THEN cagr_share_recent3y
          ELSE NULL
        END DESC,

        -- Generic stabilisers
        share_acceleration DESC,
        mean_fcr DESC,
        pubs_recent3 DESC

    ) AS status_rank

  FROM snapshot s
  WHERE concept_status IN ('baseline', 'emerging', 'frontier', 'declining')

  QUALIFY
    -- baseline / declining: no extra gating
    concept_status NOT IN ('emerging', 'frontier')

    -- emerging: diffusion-style constraints
    OR (
      concept_status = 'emerging'
      AND first_year_over_eps BETWEEN 2021 AND analysis_end_year
      AND first_year_over_eps IS NOT NULL
      AND delta_pubs_equiv_3y >= 25
    )

    -- frontier: early, impactful, not-yet-diffused
    OR (
      concept_status = 'frontier'
      AND cagr_share_recent3y >= 0.30
      AND delta_pubs_equiv_3y >= 5  -- small non-triviality guard
    )

),

selected_series AS (
  SELECT
    l.year,
    l.for_code,
    l.for_name,
    l.concept,
    sc.concept_status,
    sc.status_rank,

    l.pub_count,
    l.field_pub_count,
    l.pub_share,
    l.relevance_weighted_share,
    l.mean_relevance,
    l.mean_fcr,

    l.yoy_share_growth,
    l.cagr_share_5y,
    l.cagr_share_recent3y,
    l.cagr_share_past3y,
    l.share_acceleration,

    l.first_year_over_eps,
    l.concept_age_years,
    l.pubs_total,
    l.pubs_recent3,
    l.recent_pub_fraction_3y
  FROM labeled l
  JOIN ranked_snapshot sc
    ON l.for_code = sc.for_code
   AND l.concept = sc.concept
)

SELECT *
FROM selected_series
WHERE year BETWEEN 2016 AND 2025
  AND concept_status IN ('baseline','emerging','frontier');

The methodological challenge is substantial. While Dimensions’ grants database likely captures most of the formal funding relationships, many funders—particularly in certain regions—do not systematically report their grants. Luckily, the acknowledgments sections of publications offer an alternative data source. Yes, acknowledgments include non-funding entities (museums sharing collections, colleagues offering feedback), but across 7,500 organisations and a decade of publications, these instances become noise in the aggregate signal.

My classification approach uses a hybrid AI pipeline to categorise each organisation across five facets:

• legal structure (Government bodies, Non-profit organisations, For-profit entities, and International organisations)

• funding source (Public funds, Private capital, Charitable donations, Blended finance, Mixed sources)

• governance (Public, Private non-profit, Private for-profit, Intergovernmental)

• territory served (International, Multi-national, National, Subnational)

• primary role (Research funding intermediary; Research funding foundation or charity; Research performing organisation; Research-performing healthcare organisation; Research coordination, policy, or advisory body; Research dissemination organisation; Multi-function research organisation)

Six patterns in global research funding

With this classification framework established (methodology at the end), I then applied it to map national funding landscapes. I used the GRID extracted organisations from the publications’ acknowledgments, and filtered out organisations whose funding scope extends beyond their home country, using the territory served dimension of my classification. This excluded multi country/international organisations based Belgium, Italy, Switzerland, France, and other countries, and focused on national funders.

Using the roles dimension, I mapped the maturity of the funding ecosystem across the world, between reliance on philanthropic funding, to reliance on the state.

• Direct state: ministries dominate.

• Agency-mediated: research councils are the main actors.

• Institutional: universities fund their own work.

• Hybrid: a mix with no dominant group.

• Philanthropy-visible: charities play a significant role.

• Low differentiation: weakly differentiated landscape.

Methodology: building a classifier

I used a layered hybrid workflow that combines deterministic rules, small local language models (SLMs), and large language models (LLMs). The goal was to maximise accuracy, while lowering the cost/reliance on commercial models.

The workflow takes the following multi-steps:

1. [python] a deterministic layer, creating rules based on the name of the organisations for high-confidence cases (e.g. ministries, universities, international organisations).

2. [SLM] semantic verification by a second SLM acting as a domain-aware auditor (I included a country specific description to give context to the SLM).

3. [SLM] targeted repair step, where only the specific fields flagged as potentially problematic are corrected or cleared (e.g., a university marked as for-profit is rechecked, since this is a rare but possible case).

4. [python] lightweight final validation and strict schema enforcement in Python, used to normalise values and correct small but systematic drifts introduced by small models (e.g. “for profit” into “for-profit”).

5. [LLM] selective patching of unresolved fields using a more powerful commercial LLM, restricted to cases where uncertainty remains after all local and deterministic steps.

6. [LLM] LLM-as-a judge to conduct an independent qualitative spot-check of a stratified sample of organisations, to assess the plausibility of assigned classifications.

For the local models, I relied on models available through llama.cpp and small enough to run on a (powerful) home desktop. Each model was selected for a specific role rather than general capability: some are particularly good at following structural instructions and producing strict JSON, others are small and fast enough to act as semantic auditors. Because each pass performs a narrowly scoped task and uncertainty is explicitly allowed and repaired later, the deterministic and SLM-based stages alone achieved around 90% coverage across most classification dimensions.

The judge looked at a stratified sample of 80 organisations’ roles and found that 69 were correct, and 11 were questionable. However, considering the classification is relatively complex and the funders that were included not necessarily clear cut actors. For instance, is General Motors’ primary role in the research ecosystem as a research institution or funding institution? Both of them are likely true.

Conclusion

This classification reveals how differently research funding operates across countries; differences that shape what research gets done, who can do it, and how quickly emerging fields can develop. But national funders are only part of the story. The multinational and international organisations filtered out of this analysis (including the European Commission, major international foundations, and intergovernmental research bodies) play increasingly important roles, particularly in smaller countries and emerging research systems. Understanding both national ecosystems and these supranational actors is essential for contextualising research production globally.

The hybrid AI approach handled this complexity well: of 80 organisations in a stratified sample, 69 had clearly appropriate classifications, while 11 remained ambiguous. Many of these edge cases reflect genuine dual roles (organisations that both perform and fund research, for instance) rather than misclassification.

Key takeaways:

• Dimensions is built on multiple Artificial Intelligence methodologies, which we continuously improve.

• Generative AI powers only interpretive features in Dimensions, while the backbone relies on Graph-relational, Symbolic, and Discriminative AI.

When you browse Dimensions to identify research funded by NIH, when publications from an author who moved affiliation are under the same profile, when a cluster gets labeled with a coherent name, this is Artificial Intelligence at work, although not all is Generative AI. So, what are the different flavours of AI and what do they do?

Let’s look at it like a library: AI is the entire library ecosystem; it has buildings, books, cataloguing systems, librarians with different expertise, and rules/practices. Among the librarians, the archivists (Graph-relational AI) maintain the relationships between the items in the library, while other librarians follow rules (Symbolic AI), and others learn from examples (Machine Learning). Among those who follow examples, we have two types of librarians: the cataloguers (Discriminative AI), and the interpreters (Generative AI). All can use the floor map to find their way (embeddings).

Graph-relational AI

The archivist maintains the relationships between all the items in the library: who wrote what, which books cite each other, how topics connect, and how collections interlink.

In Dimensions, this corresponds to the knowledge graph that links publications, researchers, research and funding organisations, grants, patents, and clinical trials. Both the rule-based librarian (Symbolic AI) and the pattern-learning librarians (Machine Learning) rely on this structure to work effectively.

Strength: maintains relationships across the entire collection: citations, authorships, organisational links, grant–publication chains.

Weakness:

• can propagate errors if a relationship is incorrect early in the chain,

• may appear authoritative even when based on incomplete or conflicting data,

• cannot judge which relationships are meaningful versus incidental unless guided by rules or Machine Learning,

• depends heavily on the accuracy of upstream data.

If one wrong link is introduced, the archivist may faithfully preserve it across hundreds of connections.

Symbolic AI—learning from rules

Symbolic AI, the standards librarian, whose job is to apply controlled vocabularies, cataloguing protocols, and explicit logic to keep the library’s records consistent and aligned.

In Dimensions this is used wherever explicit rules, controlled vocabularies, and deterministic decision logic are used. It underlies parts of organisation disambiguation, metadata validation, taxonomic mapping, and many of the quality-control procedures that make the database coherent. It has always been part of the system’s foundation, even as discriminative and generative AI have expanded its capabilities.

Strength: applies explicit rules and controlled vocabularies with perfect consistency.

Weakness:

• cannot adapt when the rules do not cover a new situation,

• rigidly follows procedures even when exceptions make sense,

• struggles with ambiguity or messy real-world data,

• requires continuous human maintenance to stay aligned with reality.

A standards librarian will correctly follow every protocol, even when the protocol no longer fits.

Machine Learning—learning from patterns

For librarians who learn from pattern to work, we can create a floor plan, that is the embeddings: they are the semantic map learned from examples that both the cataloguer (discriminative models) and the interpreter (generative models) rely on to understand how documents and concepts relate. In Dimensions, this has helped us clustering documents in related topics.

Strength: provides a semantic layout that allows the other librarians to navigate meaningfully.

Weakness:

• can reflect biases in the training data,

• may place unrelated topics too close or too far apart,

• has no intrinsic notion of factual correctness,

• changes when retrained, affecting downstream decisions.

The floor map is extremely useful, but not infallible. A slightly distorted map can mislead both the cataloguer and the storyteller.

Within this system, we have two types of librarians:

Discriminative AI, the cataloguers are the ones who make fast, precise decisions about where things belong. They ensure the right labels are applied so visitors can find what they need. Much of AI in research infrastructure has long relied on this kind of intelligence: systems that classify documents, disambiguate authors, link related records, and keep the shelves organised at scale.

Concretely speaking, in Dimensions, Discriminative AI is used for:

• Document classification: assigning documents to subject areas or fields,

• Entity disambiguation: determining whether two similar names refer to the same author, and resolving organisations and funders,

• Record linking: connecting publications to grants, patents to publications, or clinical trials to publications. This is maintained by the archivist.

Strength: makes consistent, scalable decisions about categories and identities.

Weakness:

• can misclassify edge cases when they don’t resemble the examples it learned from,

• can reinforce historical patterns or biases in the data,

• cannot explain its decisions in rule-based terms,

• struggles with genuinely new phenomena.

In library terms, the cataloguer may confidently shelve a rare or unusual book in the wrong section simply because it resembles something else.

Generative AI, by contrast, is the interpreter who can use a large collection of books and produce something new: a summary, an explanation, a translation, or a thematic overview. This is the newest librarian who engages directly with users in natural language and can synthesise patterns across vast collections.

In Dimensions, we have used it for:

• Summaries of articles (TL;DR, key highlights, and top keywords),

• Labelling and describing clusters.

Strength: produces fluent summaries, descriptions, and narratives that help readers make sense of complex material.

Weakness:

• can overstate, embellish, or invent details not grounded in the underlying documents,

• cannot separate what is likely from what is true,

• may smooth over uncertainty or nuance,

• can sound more authoritative than it is.

The storyteller is helpful and insightful, but not a reliable archivist. This is why Dimensions uses GenAI only for interpretive layers, not for the factual backbone of the system.

Conclusion

Because each librarian has limitations, Dimensions relies on them collectively rather than expecting any single one to be authoritative; strengths in one role help offset weaknesses in another, and the system works only when these functions are balanced.

Neuro-symbolic AI offers the potential to combine the strengths of rule-based methods with the flexibility of the interpreter. As our understanding of the strengths and limitations of Generative AI evolves, we will expand and refine the features built on top of it.

• Gender can be inferred with a probabilistic approach based on the first name, using commercial databases

• While increased gender integration may signal positive collaboration trends, it can also reflect the erosion of women-only teams

Gender equity in research is often discussed in terms of who gets funded or who publishes more, but for this data bite, I was interested in the make up of collaborations. I used our data from the Genderize.io database to infer gender based on the first name (always clean them first), using the 0.8 threshold (at least 80% of the names in the database was represented by one gender). Although not a perfect methodology, gender inferring using databases such as Genderize.io has performed with an accuracy of around 95–97% for Western or European names (after cleaning for diacritics and compound names) and around 70–82% for Asian names, depending on the dataset and the availability of country or cultural metadata. Accuracy drops further (error rates up to 30% or more) when names are rare, multilingual, or not well represented in the underlying database (Sebo 2021; Santamaría & Mihaljević 2018.

Mixed gender team: women- or men-only team erosion?

I analysed all publications (articles, books, preprints, conferences) with at least two authors, and classified authors’ collaboration into three teams: all-women, all-men, and mixed-gender. When tracking how these team types evolved between 2015 and 2024, across all first-level Fields of Research (FoR), it was apparent that the growth of mixed teams was in every field.

To understand if the increase came from a rise in mixed teams or women-only teams, I looked at the correlations between the rise in mixed-gender authorship and the decline of single-gender teams, across fields. If gender mixing occurred primarily through women joining previously male-only teams, we would expect to see strong correlations with the decline of men-only teams, dots clustering in the top-left. If mixing came instead from the dissolution of women-only teams, we would see dots in the bottom-right.

Across most fields, gender integration coincides with a steeper decline in women-only teams than in men-only ones. Environmental Sciences stands out as the exception: it shows slightly stronger correlation with men-only team decline (r=0.94) than women-only decline (r=0.91), a rare signal of balanced or even male-team-driven integration.

At the other extreme, Chemical Sciences shows stark asymmetry: women-only collaborations decline almost perfectly with rising gender mixing (r=0.96), while men-only teams remain largely unaffected (r=0.09). Biomedical Sciences shows similarly extreme patterns (0.98 vs 0.55), as does Biological Sciences (0.97 vs 0.69): in all three cases, diversity gains largely reflect the erosion of women-led teams.

Several fields show genuinely symmetric decline: Engineering (0.99 vs 0.98), Information and Computing Sciences (0.99 vs 0.98), and Human Society (0.99 vs 0.98) all display near-identical correlations, suggesting both single-gender team types are dissolving at similar rates as mixing rises.

Psychology (0.75 vs 0.76) shows balanced correlations but at lower magnitudes. Similarly, Creative Arts, History, and Language and Culture display weaker correlations overall (all below 0.66), suggesting that in these fields, the increase of mixed gender teams did not come at the expense of existing collaborations.

Are women more visible as contributors or leaders?

Next, I looked at authorship roles. The chart below compares the percentage-point increase in women appearing as first authors versus last authors from 2015 to 2024. Here we assume that the first author is the author who led the publication, while the last author conceptualised / supported the publication in a leadership role, a more senior role. If you recall my previous data bite, we saw that an alternative order (alphabetical) was not as common as expected even in fields like mathematics.

The diagonal marks equal gains: fields above it show stronger progress for women in senior (last-author) roles. Most fields cluster close to this line, indicating that increases in women’s last authorship generally parallel gains in first authorship, with only modest asymmetries. The dotted lines indicate a ±1.5 percentage-point “tandem zone,” where gains in early- and senior-authorship roles rise in parallel.

Biomedical and Clinical Sciences (6.6 vs 4.8 pp) and Health Sciences (6.3 vs 4.3 pp) stand out for greater advances in senior authorship, suggesting women are increasingly represented in leadership or supervisory positions. Philosophy and Religious Studies shows a similar pattern (9.3 vs 7.6 pp), indicating growing visibility in senior scholarly roles, while Agricultural, Veterinary and Food Sciences (5.3 vs 3.8 pp) shows balanced but upward-tilting progress.

In contrast, Built Environment and Design (6.1 vs 7.9 pp) and Creative Arts and Writing (8.6 vs 12.3 pp) fall below the diagonal, with stronger gains in first authorship. If we can infer anything from author positions in these fields, patterns suggest that in some applied and creative fields, women’s participation has expanded mainly in primary research roles rather than senior authorship positions.

Do growing teams mean growing inclusion?

In this final graph, I bring structure (team size) and visibility (first+last authorship) together. The chart shows change in average team size on the horizontal axis and change in the share of woman-led publications (where the first or last author is a woman) on the vertical axis. Bubble size reflects publication volume.

Fields where authorship order reflects research hierarchy (think Biomedical and Clinical Sciences, Biological Sciences, Physical Sciences, and Health Sciences) show consistent, though moderate, links between collaboration growth and women’s advancement. Biomedical and Clinical Sciences (+1.53 team size, +6.8 pp leadership gain) and Biological Sciences (+1.26, +6.5) stand out for inclusive progress: as teams expand, women increasingly hold senior (last-author) positions, suggesting broader access to lab leadership. Physical Sciences (+1.25, +6.6) shows a similar, balanced pattern, while Health Sciences (+1.33, +5.7) reveals a weaker link: team expansion outpacing leadership gains, implying that growing collaborations may not equally distribute authority.

By contrast, in fields where author order is less hierarchical (Philosophy and Religious Studies (+10.7 pp, +0.38 team size) and History, Heritage and Archaeology (+13.0, +0.44)) women’s leadership gains occur without collaboration growth. These humanities fields appear to achieve gender balance within stable team structures rather than through expanding research networks.

Overall, collaboration growth and women’s leadership are only loosely connected across fields: lab-based sciences show modest positive coupling between the two, whereas in the humanities, women’s advancement in leadership roles reflects internal rebalancing rather than structural expansion.

Conclusion

Women’s participation in research authorship continues to grow, but the balance between inclusion and autonomy varies sharply across fields. In lab-based sciences where last authorship signals seniority (e.g., Biomedical, Health, and Agricultural Sciences) women are gaining leadership visibility as teams expand, yet women-only collaborations decline almost perfectly with rising gender mixing (correlations 0.95), suggesting that progress comes through integration rather than independence. Chemical Sciences shows this pattern most starkly (r=0.96 vs 0.09), while Environmental Sciences stands apart as the only field where men-only teams erode faster and leadership gains align with team growth, indicating genuine structural integration. Humanities fields like History and Philosophy achieve substantial leadership gains (+13.0 and +10.7 pp) without expanding teams, pointing to internal rebalancing within stable collaboration cultures. Engineering and Information Sciences show near-symmetric erosion of single-gender teams, a signal of systemic transformation rather than asymmetric pressure.

Across research, inclusion increasingly takes the form of mixed-gender collaboration. But when women-led teams shrink faster than men-led ones, equity gains may reflect integration within existing hierarchies rather than a redistribution of authority.

Biologists have long relied on so-called model organisms, species that stand in for the messy complexity of life. A good model organism is small, cheap to grow, easy to manipulate, and general enough to reveal rules that apply more broadly. During my time in a lab, Escherichia coli taught me how genes are switched on and off (and how to use it to express recombinant proteins). Drosophila melanogaster, the humble fruit fly, revealed the logic of inheritance. Caenorhabditis elegans gave us the first transparent look at how a multicellular organism develops, cell by cell.

The impact of these organisms has been so profound that many Nobel Prizes were awarded for discoveries made in them, such as the 1965 Prize to François Jacob, André Lwoff, and Jacques Monod for their work on genetic regulation in E. coli; the 1933 Prize to Thomas Hunt Morgan for his pioneering genetics work in Drosophila; and the 2002 Prize to Sydney Brenner, H. Robert Horvitz, and John Sulston for their groundbreaking studies of organ development and programmed cell death in C. elegans. Model organisms simplify biology but also open entirely new chapters of science.

Model organisms are, in short, stand-ins for complexity. And in the process, they’ve become more than species: they are intellectual tools. But what if the concept of the “model organism” is shifting? What if the newest models aren’t organisms at all?

The models of the 20th century

The great strength of the classical model organisms was that they were accessible: easy to grow, cheap to keep, and experimentally tractable. Escherichia coli and Bacillus subtilis grew in flasks by the billions, offering rapid ways to test hypotheses about genes, proteins, and replication. Saccharomyces cerevisiae and Schizosaccharomyces pombe could be coaxed into dividing or pausing, revealing the molecular clocks that govern cell cycles, insights later recognized with the 2001 Nobel Prize to Leland Hartwell, Tim Hunt, and Paul Nurse.

But these organisms were also representative, serving as windows into biological truths that extend far beyond themselves. In yeast, the cell-cycle machinery turned out to be conserved all the way to humans. Caenorhabditis elegans, transparent and barely a millimeter long, mapped out universal principles of development and programmed cell death, achievements honored with the 2002 Nobel Prize to Sydney Brenner, H. Robert Horvitz, and John Sulston. The fruit fly, Drosophila melanogaster, revealed the logic of developmental patterning through the discovery of Hox genes, a breakthrough awarded the 1995 Nobel Prize to Edward B. Lewis, Christiane Nüsslein-Volhard, and Eric Wieschaus.

Above all, these models were transformative. They did not merely answer existing questions: they opened entirely new fields. Arabidopsis thaliana made plant genetics predictive, turning a common weed into the Rosetta Stone of plant biology. And Mus musculus, the laboratory mouse, became the mammalian workhorse of genetics, physiology, and disease modeling, where gene targeting and knockout technologies (recognized with the 2007 Nobel Prize to Mario Capecchi, Martin Evans, and Oliver Smithies) transformed medicine by allowing scientists to recreate human diseases in a living system.

Together, these canonical organisms became the foundation of 20th-century biology, chosen not for their glamour but for their power as proxies: manageable windows into the complexity of life.

Broadening the spectrum: modern and emerging models

As biology advanced, the spectrum of model organisms expanded. The choices reflected not only curiosity but also new criteria: relevance to human health, novel traits, and the power of emerging technologies. Once again, the reasons these organisms were adopted can be understood as accessible, representative, and transformative.

They became more accessible thanks to new methods; transparent embryos, genome editing, and live imaging. Danio rerio (zebrafish) and Xenopus (African clawed frog) embody this perfectly: their embryos are large or transparent, easy to manipulate, and ideal for watching vertebrate development unfold in real time. Accessibility was also practical; zebrafish breed prolifically, making them cost-effective stand-ins for more complex vertebrates.

They were also representative, chosen because they illuminate processes we care about across biology. Planarians, with their remarkable regenerative powers, became symbols of stem cell biology and tissue renewal. Tardigrades, the tiny “water bears,” survive freezing, desiccation, and even space vacuum, representing the extreme limits of life and stress tolerance. Trichoplax adhaerens, one of the simplest multicellular animals, stripped down to just a few cell types, now represents a baseline for understanding how multicellularity works at all. Even the concept of the holobiont, treating host plus microbiome as a single system, emerged as a representative model for the complex symbioses that define life.

And they proved transformative. These new models did not simply add to the existing canon; they opened yet again new horizons of inquiry. Planarians challenged assumptions about aging and cellular limits. Tardigrades suggested new avenues for biotechnology and medicine, from radioprotection to stress resistance. Holobiont systems reframed organisms not as individuals but as ecosystems, transforming how we think about health, disease, and evolution.

Where the canonical models of the 20th century were about uncovering the shared fundamentals of biology, the modern and emerging models of today are about exploring its frontiers: resilience, regeneration, symbiosis, and complexity itself.

Beyond nature: synthetic biology as model systems

If the 20th century was defined by the discovery of natural model organisms, the 21st century has increasingly been shaped by the deliberate construction of new ones. Advances in genetic engineering, tissue culture, and synthetic biology have allowed researchers to adopt convenient species and to design model systems from the ground up.

One striking example is the creation of minimal synthetic cells, such as JCVI-syn3.0. By paring down the genome of Mycoplasma mycoides to 473 essential genes, scientists built the simplest possible cell capable of life. This stripped-down organism functions as a living testbed for understanding what is truly fundamental to cellular existence: essentially, a “designed bacterium” standing in for life’s core functions.

Another breakthrough came with organoids. By coaxing stem cells into forming miniature brain, gut, or retinal tissues, researchers created lab-grown models that mimic the architecture and behavior of whole organs. These organoids cannot replace entire organisms, but they allow scientists to study human development, disease, and drug responses in ways that classical model organisms never could.

Even more radical are cell-free systems, where transcription and translation are reconstituted outside of a cell (my last resort during lab days, when recombinant protein expression in E. coli just didn’t work). Here, life’s molecular machinery is uncoupled from the constraints of an organism altogether. These systems let scientists isolate and study gene expression in its purest form, free from the noise of cellular context.

Taken together, these synthetic models mark a turning point. Where fruit flies and worms once stood in for the complexity of multicellular life, today’s synthetic constructs represent a new philosophy: if nature’s organisms do not provide the model we need, we can build one ourselves.

The conceptual leap: could LLMs be model organisms?

From bacteria in flasks to synthetic genomes and organoids in culture, the history of model organisms shows a consistent theme: we choose proxies that make complexity visible. At first, these proxies were found in nature; more recently, they have been engineered in the lab. But what if the next step is neither natural nor biological?

Large language models (LLMs), such as GPT-based systems, offer a striking parallel. They are not alive, but they share the essential logic of a model organism: they are simplified, tractable systems that reveal something about a much larger, more complex domain. Just as E. coli stood in for the molecular logic of all bacteria, LLMs may stand in for the statistical patterns of human language and reasoning.

Like classic biological models, LLMs are accessible; open APIs and community-driven models make them easy to experiment with. They are representative, in that their emergent behaviours capture aspects of human knowledge and cognition, even if imperfectly. And they are transformative, opening new lines of inquiry in linguistics, neuroscience, and even philosophy, much as fruit flies or worms once transformed genetics.

In this sense, LLMs can be seen as the model organisms of cognition. They are not the real thing (just as no biologist mistakes a fly for a human) but they are revealing nonetheless. They allow us to probe questions about meaning, memory, and reasoning at scale, providing a “transparent worm” of sorts for studying intelligence itself.

The future of model organisms: biology meets AI

The story of model organisms is, at its core, the story of how science manages complexity. Each generation of models has reflected the tools and questions of its time. In the early 20th century, fruit flies and bacteria made genetics accessible. By the late 20th century, worms, mice, and plants stood in for development, physiology, and ecology. In the early 21st century, zebrafish, planarians, and tardigrades broadened the spectrum to resilience, regeneration, and symbiosis. Synthetic cells, organoids, and cell-free systems then marked a shift from discovery to design.

Now, with large language models and artificial systems, the concept itself is stretching. We may be entering an era where model organisms are not bound to biology at all, but instead serve as model systems for cognition, knowledge, and society. Just as mice were never perfect stand-ins for humans, LLMs are not perfect models of thought; but they may be good enough to illuminate hidden rules and generate new questions.

The future may bring hybrids: digital twins of biological systems, AI-designed synthetic cells, or organoid–machine interfaces where biology and computation feed back on each other. In such a landscape, the definition of a “model organism” expands beyond species, beyond cells, even beyond life.

From fruit flies to silicon brains, the thread is the same: we choose models not because they are complete, but because they let us see what was invisible before. Each new stand-in extends the frontier of what we can ask.. and of what we dare to imagine.

Alphabetical authorship is often understood as a norm of fairness, especially in fields where contributions are seen as roughly equal. It can avoid conflict, flatten hierarchies, and signal a particular disciplinary culture. In that sense, it's a kind of order: simple, impartial, and rule-based.

But alphabetical order also introduces ambiguity. In a research environment increasingly driven by metrics, where first and last authorship signal leadership or seniority, alphabetical lists resist easy interpretation. They offer formal order, but may obscure the underlying social or intellectual structure of the work.

Unequal representation

In some disciplines, the author list is still said to be alphabetical. This first chart shows that in fields like mathematics and economics, a higher share of papers with five or more authors follow A-to-Z ordering. In mathematics, over 10% of these large-team articles are alphabetical while in biomedical sciences, it is close to zero.

But large author teams are not evenly distributed across fields, as seen on this chart too. In biomedical and chemical sciences, more than half of articles have more than 5 authors. In many humanities and social science fields, the proportion is closer to 5%, which naturally limits how often alphabetical patterns can be meaningfully observed.

Since I was surprised to find such a small proportion of research articles using alphabetical authorship, I decided to look at the trend in the last 15 years.

..but in quiet decline

This second chart reveals that even in fields where alphabetical authorship was once common, it's declining steadily over time. Economics, education, philosophy, and even mathematics all show downward trends since 2010.

In other disciplines, particularly those shaped by clinical or experimental science, alphabetical order never had much of a presence, and that has not changed.

What’s happening, perhaps, is not the abandonment of fairness, but a shift toward clarity. As team sizes grow and evaluations become more granular, researchers may feel greater pressure to assign and assume specific author positions. Alphabetical order, once a shared understanding, can now feel like a risk: too easy to misread, especially outside one’s field.

Reading between the lines

There’s no right or wrong way to order authors; only choices that reflect the norms of each scholarly community. But the fading of alphabetical authorship is a quiet signal: the rules of recognition are shifting, and so is the culture of collaboration.

CRediT author statements should be the tool used to clarify author roles. While still unevenly adopted, they provide a structured way to clarify who did what, without relying on the ambiguous shorthand of position.

Code

WITH pub_data AS (
  SELECT
    p.id AS pub_id,
    CONCAT(cat.code, ". ", cat.name) AS for_name,
    cat.code AS for_code,
    COUNT(p.authors) AS author_count,
    ARRAY(
      SELECT LOWER(a.last_name)
      FROM UNNEST(p.authors) AS a
    ) AS last_names
  FROM `dimensions-ai.data_analytics.publications` p,
       UNNEST(p.category_for.first_level.full) AS cat
  WHERE
    p.type = "article"
    AND p.year BETWEEN 2010 AND 2024
),

evaluated AS (
  SELECT
    pub_id,
    for_code,
    for_name,
    author_count,
    NOT EXISTS (
      SELECT 1
      FROM UNNEST(last_names) AS name WITH OFFSET i
      WHERE i > 0 AND name < last_names[OFFSET(i - 1)]
    ) AS is_alphabetical
  FROM pub_data
),

aggregated AS (
  SELECT
    for_code,
    for_name,
    COUNT(DISTINCT pub_id) AS total_articles,
    COUNTIF(author_count >= 5) AS articles_5plus,
    COUNTIF(author_count >= 5 AND is_alphabetical) AS alphabetical_5plus
  FROM evaluated
  GROUP BY for_code, for_name
)

SELECT
  d.Discipline,
  a.for_name,
  a.total_articles,
  a.articles_5plus,
  a.alphabetical_5plus,
  ROUND(100 * a.articles_5plus / a.total_articles, 2) AS pct_with_5plus_authors,
  ROUND(100 * a.alphabetical_5plus / a.articles_5plus, 2) AS pct_alpha_among_5plus
FROM aggregated a
LEFT JOIN `disciplines_ffl` d
  ON CAST(a.for_code AS INT64) = d.ffl
ORDER BY d.Discipline, a.for_name

When I saw on LinkedIn at the end of August 2025 that ORCID has reached 10 million active users, I thought back to its earliest days and my own involvement spanning the journey ORCID took throughout the years. I see three clear “acts”:

Act I: Infancy – planting the seeds (2012–2015)

In the early 2010s, I was Head of Research Information at the University of Cambridge Research Strategy Office, working closely with Danny Kingsley and the Office of Scholarly Communication at the University Library. ORCID was a bold new idea: a persistent digital identifier for every researcher. But would it take root?

The UK’s commitment began with the Jisc/ARMA ORCID pilot (2014–2015). Funded projects at eight universities including Glasgow, Oxford, UCL, and Manchester, were tasked with testing integrations and sharing experiences. As Verena Weigert of Jisc put it, the pilot aimed to “develop a community of practice around implementation of ORCID identifiers by HEIs.” (source).

The UK-wide pilot, launched in May 2014, reinforced this exploratory spirit: each university developed its own use cases, from tracking early-career researchers to integrating ORCID into repositories and CRIS systems, and even embedding ORCIDs into HR records for new staff. Collectively, these experiments tested not only technical possibilities but also cultural appetite.

At the University of Cambridge, we faced a central decision on how to deal with ORCID in our own institution:

• Top-down: assign ORCIDs to everyone automatically

• Bottom-up: encourage voluntary uptake

We chose the bottom-up path, respecting researchers’ independence. It carried risks, but also aligned with the spirit of academic ownership.

To me, ORCID always represented more than just an identifier. Researchers lead nomadic lives, moving between institutions, countries, and disciplines. ORCID promised to be their research passport: a lightweight, portable identity that travelled with them, connecting their outputs wherever they went. Being in for the long game, I expected to see the benefits in a couple of years’ time.

Act II: Adolescence – building a community (2015–2020)

By 2015, the pilots matured into something larger: the UK ORCID Consortium (NB: Many more national ORCID communities evolved around that time - not just limited to the UK). What began as scattered local experiments became a nationally coordinated effort led by Jisc.

By the consortium’s five-year anniversary in 2020, the progress was clear:

• Membership had grown from 45 institutions in 2015 to 99 in 2020.

• ORCID uptake quintupled, from 50,000 IDs to 250,000 linked to UK university emails.

• The community hosted 29 events with 700+ participants, strengthening practice through workshops, hackdays, and training.

• Technical integrations matured, with institutions moving toward single points of truth rather than piecemeal connections.

• The community itself drove innovation, from the EPrints Advance plugin to the Community ORCID Dashboard (COrDa).

• Policy momentum grew: funders like UKRI and The Wellcome Trust signed the ORCID Funders Open Letter (2018), while the Tickell Report called for funder mandates and broader PID strategies.

By the end of this period, I was no longer at Cambridge. Watching from outside the university world, I could see how ORCID had shifted from fragile pilots to a trusted community of practice. The “passport” metaphor was becoming reality and ORCID was now embedded into workflows and recognised by both institutions and funders.

Act III: Maturity – ORCID today and tomorrow (2020–2025)

Today, ORCID stands as a global cornerstone of research infrastructure. As mentioned at the beginning with 10 million active users, it is recognised worldwide by publishers, funders, and universities.

In 2022, my colleague Simon Porter at Digital Science explored this maturity in Measuring Research Information Citizenship Across ORCID Practice. Taking a scientometric lens and drawing on the Dimensions dataset, he examined how ORCID is adopted across researchers, publishers, and funders. The study revealed an emerging “research information citizenry”; a landscape where ORCID adoption is not just individual but systemic, knitting together the roles and responsibilities of different stakeholders.

The vision is not yet complete: interoperability across systems remains uneven (a situation since my early days in this business) and seamless portability of outputs is still a work in progress. Adoption also varies between regions and disciplines. Yet, the trajectory is unmistakable:

• From pilots and opt-in choices (2014).

• To national coordination and policies (2015–2020).

• To global maturity and recognition, now analysed and tracked through scientometrics and datasets like the public ORCID data set and Dimensions.

I think researchers remain nomads, but now with a digital passport that allows them to travel lighter, be recognised faster (and more efficiently), and spend less time re-entering data and more time doing research. Point in case is my own journey from researchers in different institutions and countries, different employers and types of institutions and now, while being VP Research Evaluation and Global Challenges at Digital Science, I have my up-to-date ORCID profile readily available; Biography and outputs/DOIs in one neat single profile: https://orcid.org/0000-0001-7757-8001 for the curious.

Reflecting on my own journey mirrors this arc. I began at Cambridge, helping to test ORCID in its infancy. Today, I am at Digital Science, working with colleagues like Simon Porter and international partners such as the Australian Research Data Commons (ARDC) on the next stage: not just adopting identifiers, but making them work together.

Commissioned by ARDC, our PID Benchmarking Toolkit supports Australia’s National PID Strategy. It translates high-level ambitions into SMART benchmarks (Specific, Measurable, Achievable, Relevant, Time-bound), enabling agencies and institutions to track progress across the research lifecycle including:

• Inputs: grants, projects, infrastructure.

• Processes: affiliations, researcher identities, workflows.

• Outputs: articles, datasets, software, non-traditional research outputs.

And in 2025, openness took another leap and step forward: ORCID partnered with Digital Science to host the annual ORCID Public Data File directly in Dimensions on Google BigQuery, you may remember our research metric bite. For over a decade, ORCID has released its public dataset openly, but its sheer size made it hard to use. Hosting it on GBQ changes this: anyone can now query, explore, and link ORCID data at scale, without heavy technical barriers.

The implications are wide-ranging: mapping collaboration networks, tracking scientific migrations, linking ORCID with datasets like the World Bank, or analysing adoption trends across disciplines. With over 190,000 downloads to date, the Public Data File was already a vital resource. Now, through Dimensions on GBQ, it becomes a living, accessible tool for the global research community.

So, the arc that began with little isolated pilots now extends into a global effort to ensure identifiers like ORCID are not only used, but also analysed, benchmarked, and valued.

Final thought

From the green shoots of pilots, through the maturing UK consortium (not limited to the UK of course, I also know the German and the Austrian one), to today’s international benchmarking and open data analytics, ORCID’s story has been one of growth, resilience, and vision. For me, it has been a privilege to walk alongside this journey: first in helping seeds take root, and now in measuring and guiding their flourishing worldwide.

A decade ago, I imagined ORCID as a passport for research nomads. Today, that vision is real and with global benchmarking and open data access, we are learning how to ensure those passports are not just carried, but truly valued.

Key takeaways:

• Using Retraction Watch data and the taxonomy of retraction we can quickly identify funders of researchers with previous retracted publications for misconduct

• After retraction, outcomes differ by reason: authors with honest-error retractions return to funded work more often (~52%, incl. ~9% with a new grant) than those linked to misconduct (~36%, ~4% with a new grant).

When a researcher makes a serious mistake or commits misconduct, journals may retract the publication. What happens to their career though? The present analysis reveals that a third of researchers who co-authored a publication retracted for misconduct continue publishing funded work, and 4% receive new grants a year or more after their retraction.

The following article builds on the latest FoSci article on nefarious network that led to legitimate funding, exploring on what funding happens after retractions for misconduct.

On the origins of retractions

Retraction Watch has been building the most comprehensive database of retractions since 2010. It includes expressions of concern, corrections, and retractions, and indicates the reason for retraction. Building on the Retraction Taxonomy v2.0 (introduced on FoSci), here I group retractions into three categories:

• Misconduct: plagiarism, data falsification, paper mills; deliberate violations of research integrity

• Honest error: methodology flaws, analysis mistakes, contamination; problems without intent to deceive

• Ambiguous: system-level issues or cases where intent remains unclear

My analysis covers retractions from 2010-2020, drawing on Retraction Watch data now available through CrossRef, which can easily be imported in GBQ; I cut at 2020 because retractions lag publication by years. If a publication lists both an error and misconduct, I classify it as misconduct.

The geography of second chances

I looked at which countries continue funding researchers after retractions for misconduct and found some geographic differences. Note that here we include any co-author of retracted publication, for any misconduct type, so the funder may have awarded the grant with the knowledge of the retraction and understanding that they may not have been involved directly in the misconduct.

The pattern reveals three clusters:

• Most countries have less than 100 grants awarded to researchers after they co-authored a publication retracted for misconduct; they cluster near zero share.

• Some small research systems show surprisingly high rates: Slovenia funds post-misconduct researchers at 1.5% of their grants, Qatar at nearly 1%. These outlier rates are likely to reflect small sample sizes.

• Major research powers cluster around 0.2-0.6%: China (0.6%), United States (0.3%), Japan (0.25%), and Brazil (0.2%) show more consistent patterns. While China shows a moderate 0.6% rate, its massive funding volume means it accounts for the largest absolute number of post-misconduct grants worldwide; with half as many grants as the US though, it has a rate thrice as big as the US, and we do not have the latest grants since China stopped sharing their data. Canada is the lowest share of the largest funding country.

There are differences within countries and fields, but sizes are then too small to find any obvious pattern.

Funders keep funding researchers with misconduct-retractions

Using the distinction between honest error and misconduct, I looked at three data points for researchers who had a retracted publication, a year after their retraction:

• publication of at least one unfunded publication,

• publication of at least one funded publication (not necessarily to the research involved in the retraction) and possibly unfunded,

• received a grant

We looked at 88,181 authors with misconduct-linked retractions, 22,878 with honest error-linked retractions, and 11,218 ambiguous.

Looking from the year after the retraction onward, authors in the honest-error group are more likely to stay active in funded work (about 52% have funded publications, including 9% with a new grant) than those in the misconduct group (about 36% funded, 4% with a new grant), while the misconduct group shows more cases with no visible activity (47% vs 37%). The ambiguous category has the highest apparent engagement (58% with funded outputs), which likely reflects mixed situations and incomplete reason codes. Interestingly, researchers who had been involved in a publication retracted for misconduct, reduced more often their subsequent careers to publishing without funding that when their retracted publication had been for an honest error.

Note that “funded again but no grant tracked” is a lower bound, publications can acknowledge funders without a matching grant record, so the chart summarises what is observable rather than why particular decisions were made.

Conclusion

Retractions are uncommon. In this slice, authors with honest-error retractions return to funded work more often (52%, 9% with a new grant) than those linked to misconduct (36%, 4% with a new grant), and the misconduct group shows more cases with no visible activity. Differences across funders and countries likely reflect a mix of volume, coverage and policy rather than simple league tables. The analysis shows what happens after retraction; it doesn’t reveal why specific funding decisions were made. Clear public policies and basic aggregate reporting would make those patterns easier to study over time.

Data note: Funded publication = any funder acknowledgement on the publication; ‘received grant’ = a Dimensions grant with start year > retraction year. Reasons are aggregated per publication across notices; misconduct takes precedence over error; co-funded grants counted. Some zeroes may reflect data gaps.

Key takeaway:

• Publication outputs: West Germany remains quantitatively dominant, but East Germany demonstrated qualitative strength and resilience, particularly in the early decades post-reunification. Both now face the same challenge: maintaining research impact in an era of rapidly expanding publication output.

• Institutional Patterns: Max Planck (West) continues as the most prolific and impactful overall, Helmholtz (West) as the main driver of growth, while Leibniz (East) stands out as the most striking success story of post-reunification expansion.

In our previous research data bite, Scientific divides: research output and influence in East and West Germany, we analysed how Germany’s research output diverged sharply in the decades following World War II, along the lines of the Cold War. Using GRID’s geolocation data, we classified institutions according to whether they fell within the territory of East Germany (the GDR) or West Germany (the FRG), then compared their research volume and citation performance from 1920 to 1989.

Here a recap of the key findings:

• Publication Output: Initially (pre-1945), institutions in what would become West Germany produced roughly four times more publications than those in the East. This gap widened significantly after the war: by the 1980s, the West was publishing nearly ten times as much.

• Citation Impact: East Germany’s global research influence lagged even further behind its publication share, with the GDR attaining only about one-twentieth of West Germany’s citation volume by 1989.

• Driving Factors: Soviet-aligned policies in East Germany limited research expansion, international collaboration, and integration into global scientific networks. Meanwhile, West Germany, though it never regained its prewar level of international dominance, rebounded faster and benefited from open research environments and stronger ties with Western partners.

These data illustrate how political divisions and differing funding priorities shaped Germany’s scientific landscape for decades. The East operated under more restrictive Soviet policies, while the West had comparatively open avenues to global scientific communities, resulting in stark inequalities in both research output and influence by the late 1980s.

Germany Reunited

In this post, we will examine what happened after reunification in 1990. Specifically, we will look at whether the huge, general Aufbau Ost (Buildup in the East) investments helped to bridge the East–West research gap, too. We focus on publication and citation trends across the unified research environment and will analyse the roles of Germany’s major research organisations, including the Max Planck Society, the Helmholtz Association, Fraunhofer-Gesellschaft, and the Leibniz Association, in shaping the post-reunification research ecosystem. The Appendix has more detail on type and mission of each organisation relevant to this analysis.

Photo by Frank Peters, sourced from Adobe Stock

Following reunification, Germany prioritised bridging the scientific gaps, including the scientific one (see last post), between the former East and West through a massive investment program known as Aufbau Ost. In practice, this meant establishing new institutes and modernising existing facilities in the five eastern federal states (and former East Berlin). For the Max Planck Society (MPG) specifically, 18 new institutes in East Germany were founded between 1990 and 1998, increasing the total number of Max Planck Institutes by roughly one-third. While such growth reflected a commitment to revitalising the East’s research landscape, it also required corresponding budgetary adjustments in the West, including cuts as part of a broader “federal consolidation program.” In parallel, many of the former GDR’s research institutions were transferred into the Leibniz Association.

By 2017, the MPG oversaw 84 institutes and facilities (some located outside Germany) demonstrating the Society’s continued expansion and international reach. For this post, those outside Germany (MPI History of Art, Hertzina, and Rome) will be excluded from this analysis.

Research data

Question 1: Publication

Over the period from 1991 to 2024, the publication output of German research organisations reveals a consistent dominance of West German institutes over their East German counterparts. The West began the 1990s with far higher publication volumes, while the East, starting from a much smaller base, demonstrated rapid growth in the decade following reunification. This strong upward trajectory continued through the 2000s and 2010s, narrowing the gap with the West in relative terms. Both East and West reached their highest levels of publication activity around 2020–2021, followed by a period of stabilisation and slight decline post-COVID, suggesting a maturing research landscape across the country.

Over the period from 1991 to 2024, the publication data reveals a remarkable transformation: East German research institutes consistently outpaced their Western counterparts in growth rates, demonstrating the substantial impact of reunification investments. Eastern institutes achieved compound annual growth rates (CAGR) of 8.8-19.9%, with Fraunhofer (East) leading at nearly 20% annually, compared to Western rates of 3.8-8.8%.
This extraordinary Eastern expansion reflects successful institutional development from very small starting points. Max Planck (East) grew at 14.3% annually versus 3.8% in the West, while Leibniz (East) achieved 10.9% growth compared to 8.8% in the West. Even accounting for the low baseline effect, these sustained high growth rates over three decades represent a fundamental reshaping of Germany's research landscape.
However, recent data (2020-2024) suggests this rapid expansion phase may be concluding. Most Eastern institutes have plateaued or slightly declined, with growth rates near zero or negative, while Western institutes maintain modest positive growth.

This convergence in growth patterns indicates that Eastern institutes may have reached a mature operational scale, transitioning from the catch-up phase to steady-state operations alongside their Western counterparts.

Question 2: Citation Impact

Is that publication behaviour reflected in the academic perception, too? Here we look at the total, overall discipline-agnostic picture.

The citation impact of German research institutes followed a different trajectory from publication volumes. Both East and West German institutes started the 1990s with average citation rates between 30–35 citations per publication, showing broadly comparable levels of academic influence despite strong disparities in absolute publication output. From the mid-1990s into the early 2000s, citation impact rose steadily, peaking in the early-to-mid 2000s at values above 50 citations per publication. Since then, both East and West institutes have experienced a similar decline, with mean citations per publication falling sharply after 2010 and dropping below 10 by the early 2020s. This decline is particularly pronounced in the most recent years, reflecting the natural citation lag for newer publications, which have not yet had time to accumulate impact.

While West German institutes consistently maintained slightly higher citation averages through much of the 1990s and early 2000s, East German institutes at times matched or even briefly exceeded West German levels, particularly around 2005 when East institutions recorded peaks above 60 citations per publication. This suggests that East German institutes, although producing fewer papers overall, were able to achieve strong academic influence with selected outputs during this period. However, the long-term convergence is clear: from around 2010 onward, both East and West follow the same downward trajectory, with differences narrowing. By 2020, the gap has virtually disappeared, and both regions record similar citation rates. The key takeaway is that East German institutes were able to integrate into the international citation landscape effectively, but the overall German research system now faces a shared structural decline in per-publication mean citations, driven largely by the increasing volume of outputs globally and the time needed for impact to accrue.

Summary of Key Insights

Publication Volume
West German institutes consistently dominated in terms of absolute output across all four major research organisations (Max Planck, Helmholtz, Fraunhofer, Leibniz). Their share has remained structurally higher, with Helmholtz (West) and Max Planck (West) driving the bulk of national growth. East German institutes, by contrast, started from a very low base in the early 1990s but demonstrated extraordinary growth — most visibly in Leibniz (East), which by 2024 actually surpassed its Western counterpart. Overall, East Germany now contributes nearly half as many publications as West Germany, illustrating strong catch-up dynamics and successful integration into the national research system.

Citation Impact
When examining academic influence, measured through mean citations per publication, the story is more nuanced. In the 1990s and 2000s, East German institutes often matched or even outperformed their Western peers on citation impact; particularly in Max Planck (East), Helmholtz (East), and even Fraunhofer (East). By the mid-2000s, both East and West reached citation peaks of 55–85 citations per paper, placing German institutes at high levels of international visibility. However, all institutes have since seen a decline in citation rates, converging at very low levels by the early 2020s, driven largely by the global growth in publications and the citation lag for more recent work.

• Quantity vs. Quality Shift: West German institutes remain the dominant producers of research output, but East German institutes have caught up substantially and, in some cases (Leibniz), surpassed them in volume.

• Impact Parity: Despite lower volumes, East German institutes were able to achieve comparable or higher citation impact in earlier decades, showing that smaller, integrated institutions could produce influential research.

• Convergence and Decline: By 2020, the East–West differences in citation means had almost disappeared — not due to rising impact in the East, but rather due to a shared structural decline in citations per publication across Germany.

Institutional Patterns: Max Planck (West) continues as the most prolific and impactful overall, Helmholtz (West) as the main driver of growth, while Leibniz (East) stands out as the most striking success story of post-reunification expansion.

Appendix

Major Non-University Research Organisations in Germany and their focus

Today, around 1,000 publicly funded research institutions operate in Germany. Alongside the universities, four large non-university research organisations form the backbone of the country’s research landscape:

1. Max Planck Society (MPG)

   • Founded in 1948, the MPG is the leading center for basic research in the natural sciences, life sciences, humanities, and social sciences outside the university system.

   • Around 7,000 scientists, 3,400 doctoral researchers, and 3,000 visiting researchers work at its institutes, including those in East Germany established post-reunification.

   • Over 30 Nobel Prizes have been awarded to MPG researchers since its founding.

   • Jointly funded by the Federal Government and the federal states, the MPG maintains independence from corporate or association interests.

2. Helmholtz Association

   • Conducts cutting-edge research in six main areas: Energy; Earth and Environment; Health; Information; Matter; and Aeronautics, Space, and Transport.

   • Employing over 43,000 people across its 18 Helmholtz Centers, it is Germany’s largest research organisation.

   • In the future, it plans to establish a new research centre dedicated to aging research.

3. Fraunhofer-Gesellschaft

   • With 76 institutes spread across Germany, Fraunhofer is Europe’s largest institution for application-oriented research.

   • Primary research fields include Health and Environment, Mobility and Transport, and Energy and Raw Materials.

   • Active internationally, it operates subsidiaries in Europe, North and South America, and Asia, along with numerous representative offices and Senior Advisors.

4. Leibniz Association

   • Brings together 96 independent research institutions whose scope spans natural, engineering, environmental, economic, spatial, social, and humanities sciences.

   • A central focus for its roughly 11,500 researchers is the transfer of knowledge to policymakers, industry, and the public.

   • Played a significant role in incorporating former GDR institutions after reunification.

In addition, the German Research Foundation (DFG), Europe’s largest funding body of its kind, supports scientific endeavours across these organisations. Based in Bonn, the DFG also maintains offices in India, Japan, Latin America, and North America, as well as the Sino-German Centre for Research Promotion (CDZ). Its mission is to foster collaboration among researchers within Germany and internationally.

Key takeaway:

• Only 10% of research articles include trackable grant numbers (down from 17% in 2020)

• Informal funders create invisible funding that breaks research evaluation

The acknowledgement of funders in research publications, especially for large and established funders, has been common practice for at least two decades. The graph below shows that in Dimensions, in the last 5 years only about 40% of articles (not conferences, books, and preprints) have an acknowledgement or funding section. However, Dimensions could extract a funding organisation from only 27% of all articles since 2015; the difference is likely to be missing funding organisations in the acknowledgment: thanking research subjects for instance, from pigs to humans, or misspelling the funding organisations (researchers’ creativity shines when it comes to acknowledgments) or simply the rise of funding metadata, which is great news.. But where it is worrying is that we could identify the grant number from only around 17% of articles in 2020 and 10% in 2024; most of the drop is due to the Chinese government funders who stopped sharing funding information, so we can say China has funded research (included in the articles with identified funders) but not link it to a grant.

While the drop in trackable grant numbers largely reflects policy changes (particularly Chinese funders stopping data sharing), there are also infrastructure challenges. Though promising, Crossref's grant DOI registry is still sparsely populated, with only a few large funders (UK’s Wellcome Trust, Austria’s FWF, and the European Commission from Horizon Europe) actively registering grant metadata.

A case of funding invisibility

In our recent analysis of a UK government-backed programme (UKRI-funded Trustworthy Autonomous System Hub), we had to identify publications from researchers who had received sub-awards and pump priming grants. We first had to identify researchers; some of them had not used the same name in their grant application and their research, as I reported in the ballad of first names. Since the sub-award recipients had not received the initial funding and had no unique identifier, a small portion of them had used the title of their grant (with variations in the spelling) or the name of the initial research project, but there was no automatic way to identify the resulting grants. Instead we estimated likely outputs by looking at articles published by grantees from one year after the start of their grant to two years after the end.

I had used this method in the past to identify related publications from a funder which granted small grants that contributed to fieldwork, prototype building, or presentation of the research. These were rarely acknowledged, even though they often contributed to network creation and maintenance, aside from their obvious primary purpose.

Shared responsibility: funders, grantees, and publishers

As we have seen, formal funders have long recognised the importance of acknowledging their grants and investing in data management and analyses. However, informal funders are now the blind spot of research funding. We define informal funders here, any entity that gives a lump sum of money that is meant to support research. They provide institutional funding, cascade funding (common in EU programs), sub-awards/subawards (US federal grant terminology), and core funding redistribution.

If we care about improving how research is funded, assessed, and understood, we need the full narrative. Every grant tells a story, but for it to be told, we need to shift the mindset of the informal funders, the grantees, and the publishers

• Informal funders must recognise their funder identity ("with great power comes great responsibility", would have said Peter)

  • Create a unique identifier: even a basic grant code in a spreadsheet is better than none. Better though, request a DOI from CrossRef.

  • Provide clear acknowledgment language: tell researchers exactly how to cite you and the grant (don’t expect them to know your acronym; even the National Institutes of Health is not immune to being called the National Institute for Health)

  • Track what was funded: record researcher names and affiliations using persistent identifiers (ORCID, GRID/ROR, etc.), and request outputs to be sent annually, or whatever frequency suits your cohort..

• Grantees must acknowledge more rigorously

  • Include all funding sources, double check funder names/acronyms

  • Use the identifiers provided (as provided)

  • Treat small grants like real grants

• Publishers. Currently, about 25% of Crossref records contain some kind of funding information, but how much is shared varies considerably between publishers.

  • Implement structured funding fields in submission systems that separate acknowledgments from funding information–and mandate their use

  • Validate funder names against databases like GRID/ROR during submission

  • Require grant identifiers when available, with clear fields for DOIs or grant codes

  • Support cascade funding attribution by allowing multiple funding layers to be recorded (primary funder > secondary funder)

  • Make funding data machine-readable and accessible to bibliometric databases

Invisible funding

The scale of invisible funding

The number of publications with visible (although not machine-ready) informal funding is currently relatively small (less than 1% of those with an acknowledgment–see methodology at the end). However it is growing (from 5,316 in 2020 to 13,917 in 2024, CAGR 27.2%) and affecting disproportionately some countries; Russia has seen a substantial increase of documented informal grants in the last couple of years (as well as a substantial decrease in formal grants).

The cost of invisible funding

When smaller grants go unacknowledged, we lose more than just metadata. Informal funders, who often support early ideas, fieldwork, or first research jobs, have no way to demonstrate their contribution. Their investments disappear into the system, untraceable and under appreciated.

For early-career researchers, the cost is personal. These smaller grants often represent the first external validation of an idea. Without acknowledgement, the resulting publication and early support become disconnected.

And for those trying to understand the research system (evaluators, analysts, and policymakers) the missing grant data and links break the chain of evidence. We cannot see how ideas were seeded, what funding models worked, or how support translated into outcomes.

Methods

Query written with DimQuery Assistant, a custom GPT.

-- Yearly counts of informal funding mentions by type

SELECT
year,
COUNT(DISTINCT id) AS total_publications,
SUM(pump_priming_mention) AS pump_priming_mentions,
SUM(seed_funding_mention) AS seed_funding_mentions,
SUM(pilot_funding_mention) AS pilot_funding_mentions,
SUM(internal_research_mention) AS internal_research_mentions,
SUM(internal_funding_mention) AS internal_funding_mentions,
SUM(institutional_funding_mention) AS institutional_funding_mentions,
SUM(university_funding_mention) AS university_funding_mentions,
SUM(departmental_funding_mention) AS departmental_funding_mentions,
SUM(research_development_mention) AS research_development_mentions,
SUM(early_career_mention) AS early_career_mentions,
SUM(small_grant_mention) AS small_grant_mentions,
SUM(travel_grant_mention) AS travel_grant_mentions
FROM (
SELECT
id,
year,
-- Pump-priming/seed funding mentions
CASE WHEN
LOWER(acknowledgements.preferred) LIKE '%pump priming%'
OR LOWER(acknowledgements.preferred) LIKE '%pump-priming%'
OR LOWER(funding_section.preferred) LIKE '%pump priming%'
OR LOWER(funding_section.preferred) LIKE '%pump-priming%'
THEN 1 ELSE 0 END AS pump_priming_mention,
CASE WHEN
LOWER(acknowledgements.preferred) LIKE '%seed fund%'
OR LOWER(acknowledgements.preferred) LIKE '%seed grant%'
OR LOWER(funding_section.preferred) LIKE '%seed fund%'
OR LOWER(funding_section.preferred) LIKE '%seed grant%'
THEN 1 ELSE 0 END AS seed_funding_mention,
CASE WHEN
LOWER(acknowledgements.preferred) LIKE '%pilot fund%'
OR LOWER(acknowledgements.preferred) LIKE '%pilot grant%'
OR LOWER(funding_section.preferred) LIKE '%pilot fund%'
OR LOWER(funding_section.preferred) LIKE '%pilot grant%'
THEN 1 ELSE 0 END AS pilot_funding_mention,
-- Internal/institutional funding mentions
CASE WHEN
LOWER(acknowledgements.preferred) LIKE '%internal research fund%'
OR LOWER(acknowledgements.preferred) LIKE '%internal research grant%'
OR LOWER(funding_section.preferred) LIKE '%internal research fund%'
OR LOWER(funding_section.preferred) LIKE '%internal research grant%'
THEN 1 ELSE 0 END AS internal_research_mention,
CASE WHEN
LOWER(acknowledgements.preferred) LIKE '%internal fund%'
OR LOWER(acknowledgements.preferred) LIKE '%internal grant%'
OR LOWER(funding_section.preferred) LIKE '%internal fund%'
OR LOWER(funding_section.preferred) LIKE '%internal grant%'
THEN 1 ELSE 0 END AS internal_funding_mention,
CASE WHEN
LOWER(acknowledgements.preferred) LIKE '%institutional fund%'
OR LOWER(acknowledgements.preferred) LIKE '%institutional grant%'
OR LOWER(funding_section.preferred) LIKE '%institutional fund%'
OR LOWER(funding_section.preferred) LIKE '%institutional grant%'
THEN 1 ELSE 0 END AS institutional_funding_mention,
CASE WHEN
LOWER(acknowledgements.preferred) LIKE '%university fund%'
OR LOWER(acknowledgements.preferred) LIKE '%university grant%'
OR LOWER(funding_section.preferred) LIKE '%university fund%'
OR LOWER(funding_section.preferred) LIKE '%university grant%'
THEN 1 ELSE 0 END AS university_funding_mention,
CASE WHEN
LOWER(acknowledgements.preferred) LIKE '%departmental fund%'
OR LOWER(acknowledgements.preferred) LIKE '%departmental grant%'
OR LOWER(funding_section.preferred) LIKE '%departmental fund%'
OR LOWER(funding_section.preferred) LIKE '%departmental grant%'
THEN 1 ELSE 0 END AS departmental_funding_mention,
-- Research development/support mentions
CASE WHEN
LOWER(acknowledgements.preferred) LIKE '%research development fund%'
OR LOWER(funding_section.preferred) LIKE '%research development fund%'
THEN 1 ELSE 0 END AS research_development_mention,
CASE WHEN
LOWER(acknowledgements.preferred) LIKE '%early career fund%'
OR LOWER(acknowledgements.preferred) LIKE '%early career grant%'
OR LOWER(funding_section.preferred) LIKE '%early career fund%'
OR LOWER(funding_section.preferred) LIKE '%early career grant%'
THEN 1 ELSE 0 END AS early_career_mention,
CASE WHEN
LOWER(acknowledgements.preferred) LIKE '%small grant%'
OR LOWER(funding_section.preferred) LIKE '%small grant%'
THEN 1 ELSE 0 END AS small_grant_mention,
CASE WHEN
LOWER(acknowledgements.preferred) LIKE '%travel grant%'
OR LOWER(funding_section.preferred) LIKE '%travel grant%'
THEN 1 ELSE 0 END AS travel_grant_mention
FROM
`dimensions-ai.data_analytics.publications`
WHERE
(funder_orgs IS NULL OR ARRAY_LENGTH(funder_orgs) = 0)
AND year >= 2020
AND (
-- Apply the same filters used in the SELECT
LOWER(acknowledgements.preferred) LIKE '%pump priming%'
OR LOWER(acknowledgements.preferred) LIKE '%pump-priming%'
OR LOWER(acknowledgements.preferred) LIKE '%seed fund%'
OR LOWER(acknowledgements.preferred) LIKE '%seed grant%'
OR LOWER(acknowledgements.preferred) LIKE '%pilot fund%'
OR LOWER(acknowledgements.preferred) LIKE '%pilot grant%'
OR LOWER(acknowledgements.preferred) LIKE '%early career fund%'
OR LOWER(acknowledgements.preferred) LIKE '%early career grant%'
OR LOWER(acknowledgements.preferred) LIKE '%internal research fund%'
OR LOWER(acknowledgements.preferred) LIKE '%internal research grant%'
OR LOWER(acknowledgements.preferred) LIKE '%internal fund%'
OR LOWER(acknowledgements.preferred) LIKE '%internal grant%'
OR LOWER(acknowledgements.preferred) LIKE '%institutional fund%'
OR LOWER(acknowledgements.preferred) LIKE '%institutional grant%'
OR LOWER(acknowledgements.preferred) LIKE '%university fund%'
OR LOWER(acknowledgements.preferred) LIKE '%university grant%'
OR LOWER(acknowledgements.preferred) LIKE '%departmental fund%'
OR LOWER(acknowledgements.preferred) LIKE '%departmental grant%'
OR LOWER(acknowledgements.preferred) LIKE '%research development fund%'
OR LOWER(acknowledgements.preferred) LIKE '%research support fund%'
OR LOWER(acknowledgements.preferred) LIKE '%discretionary fund%'
OR LOWER(acknowledgements.preferred) LIKE '%small grant%'
OR LOWER(acknowledgements.preferred) LIKE '%travel grant%'
OR LOWER(acknowledgements.preferred) LIKE '%conference fund%'
OR LOWER(funding_section.preferred) LIKE '%pump priming%'
OR LOWER(funding_section.preferred) LIKE '%pump-priming%'
OR LOWER(funding_section.preferred) LIKE '%seed fund%'
OR LOWER(funding_section.preferred) LIKE '%seed grant%'
OR LOWER(funding_section.preferred) LIKE '%pilot fund%'
OR LOWER(funding_section.preferred) LIKE '%pilot grant%'
OR LOWER(funding_section.preferred) LIKE '%early career fund%'
OR LOWER(funding_section.preferred) LIKE '%early career grant%'
OR LOWER(funding_section.preferred) LIKE '%internal research fund%'
OR LOWER(funding_section.preferred) LIKE '%internal research grant%'
OR LOWER(funding_section.preferred) LIKE '%internal fund%'
OR LOWER(funding_section.preferred) LIKE '%internal grant%'
OR LOWER(funding_section.preferred) LIKE '%institutional fund%'
OR LOWER(funding_section.preferred) LIKE '%institutional grant%'
OR LOWER(funding_section.preferred) LIKE '%university fund%'
OR LOWER(funding_section.preferred) LIKE '%university grant%'
OR LOWER(funding_section.preferred) LIKE '%departmental fund%'
OR LOWER(funding_section.preferred) LIKE '%departmental grant%'
OR LOWER(funding_section.preferred) LIKE '%research development fund%'
OR LOWER(funding_section.preferred) LIKE '%research support fund%'
OR LOWER(funding_section.preferred) LIKE '%discretionary fund%'
OR LOWER(funding_section.preferred) LIKE '%small grant%'
OR LOWER(funding_section.preferred) LIKE '%travel grant%'
OR LOWER(funding_section.preferred) LIKE '%conference fund%'
)
)
GROUP BY year
ORDER BY year DESC

DimURL2DSL

During a recent product discussion, my colleague Emily Koechel mentioned how useful it would be to translate Dimensions URLs into DSL* queries, the language that powers Dimensions' API. This seemed like a brilliant idea: the WebApp URLs include all the filters, so theoretically, we had all the necessary information to reverse-engineer a DSL query. Do you want publications from 2023? Add `and_facet_year=2023`. Do you want publications funded by the US or the UK? Add `or_facet_funder_country=US&or_facet_funder_country=GB`? Now, for more subtlety, if you wanted publications funded by the US and (not or) the UK, then you would replace `or_facet` by `and_facet`.

The question was how to approach this translation. At first, I considered manually listing all the filters in a spreadsheet or even asking the dev team if they had one. But then I realised those filters were already encoded in the HTML of the page, which would be really easy to grab. Before even attempting to do it manually from the HTML, I thought I would try feeding it to Claude. I uploaded it, along with the API documentation, and added a few lines explaining how I thought the translation should work and what the UI should include. Claude wrote the code and created a decent front end to test.

I spent about an hour refining it, adding more document types (publications, grants, datasets, and so on) and testing as I went. Most of the filters in the HTML matched the API filters, so the translation was surprisingly smooth. Even returning facets** was straightforward based on the URL, which means that we can easily get the data from the Analytics view (see on Youtube the old interface) into a table. This means non-coders can now easily move from the WebApp to a Google Sheet using the Google Sheets connector, without needing to learn the DSL (stay tuned for Excel).

Less than two hours after starting, I shared a working version with Emily. She easily added the bits I had missed out, such as searching text and some of the advanced search. It does not include everything yet, but for less than a day of work, it is functional enough to easily translate Dimensions WebApp URLs into a Dimensions API query.

Embracing AI

Working with Emily on the shared prototype underscored the most striking part of the process: how quickly we moved from idea to something testable. Instead of spending weeks planning and waiting for development time, we had a functional tool within hours. The code is not optimised, as ChatGPT noted when prompted, and there is no polished interface yet. But having something that worked, not just in theory, validated the concept and provided colleagues (and now, you) with an immediately useful tool. AI prototyping brings substantial value by accelerating early-stage development and broadening what's feasible. Just don’t mistake prototype code for production code.

*DSL: Dimensions Language Query

** Facets: in the Dimensions API, facets let you quickly see counts and breakdowns: like how many papers were published each year or by which countries, without listing every single result.

I recently curated researchers for a large funder, who had recorded first and last names of awardees as they were provided to them. It was striking how often researchers used their official name when applying for a grant, even though they already published using their usual name (although they may switch back and forth across their career). More surprisingly, a few did the opposite, and published under their official name but applied for grants under a more familiar version. Some intentionally publish under different names altogether, but that is intentional.

Without widespread adoption of ORCID, it often takes a leap of faith to determine that Douglas Adams, who received a grant, is the same person as Doug Adams who published several papers. If they are both at the University of Maximegalon, that leap is smaller; smaller still if they share the same field of research. But the consequences of misattributing smaller researcher profiles are typically lower than those of larger ones, so it makes sense to be cautious. Still, when studying researcher trajectories, we need a way to identify all profiles that belong to the same individual, so here is my attempt at doing so.

Variants in the ORCID dataset

Looking for authoritative sources of first name variants, I stumbled across various lists online [github + project], but most had too many options and focused only on English names, overlooking other languages. I remembered ORCID includes a ‘variant’ field to indicate other names a researcher is known by. I had assumed it would mostly be used for surname changes, but it turns out many researchers use it to record alternative first names—and more unexpected entries like “researcher”, “lecturer”, or even “physician.” Some of the lesser-known pairings I already knew of (Aleksandra/Ola, Ignacio/Nacho, Gopalakrishnan/Gopal) were indeed present in the data, which was reassuring. I even found transliterations from non-Latin scripts—李鹏 listed as a variant of Peng, and 刘锐 as Rui.

At the end of this bite, you will find the GBQ code I used to extract the list (it is part of the free data so anyone with a GoogleCloud should have access, I introduced it here), but I have also added my list in figshare (here). The field is manually filled, so I kept a threshold of minimum two profiles. Proceed with cautious, obviously, as even with a minimum of two researchers, Christopher’s variant was listed as James, likely due to actual name changes rather than variants.

In Dimensions

When curating my list of researchers who had received a grant from a specific funder, I searched for their profiles in Dimensions’ researchers dataset. I incorporated potential name variants from ORCID, and then checked whether:

• the publication profile(s) included outputs funded by that funder

• the potential matches shared co-authors or research organisations—especially when there were known co-grantees

• they published in the expected fields of research

And now, for a bit of visualisation: here is trend for a select list of names and their variants. The bump from 2020–2021 COVID-era publications is clearly visible (affecting, it seems, more men than women), and in most cases, official names remain more common than their casual counterparts.

Conclusion

When working with researcher data in Dimensions and considering career trajectory or checking whether a cohort of funded researchers has published, it is important to keep name variation in mind. A query using only Aleksandra may return their grant record but miss their publishing activity under Ola. ORCID helps close that gap, but until adoption is universal and the metadata consistently filled, ad-hoc projects like this one require a mix of digging, curation, and a healthy dose of skepticism.

Code

to extract the list of variants. This will retrieve some non-relevant pairs, so you can curate the lesser used variations.

WITH name_variant AS (-- Query to map ORCID given names to their variant names
  SELECT
    TRIM(LOWER(pname.given_names)) AS first_name,
    TRIM(LOWER(variant.content)) AS variant_name,
    COUNT(DISTINCT s.orcid_identifier.path) AS researcher_count
  FROM
    `ds-open-datasets.orcid.summaries_2024` AS s,
    UNNEST(s.person.other_names.names) AS variant
  JOIN
    UNNEST([s.person.name]) AS pname
  WHERE
    pname.given_names IS NOT NULL
    AND variant.content IS NOT NULL
    AND pname.given_names != variant.content
  GROUP BY
    first_name, variant_name
  ORDER BY
    researcher_count DESC)
SELECT * FROM name_variant
WHERE researcher_count > 1

In a landscape where research is increasingly shaped by data, metrics, technology, and policy, research musings emerged as an informal yet analytical space to explore these dynamics. Hosted by a team immersed in bibliometrics, research systems, and the changing knowledge infrastructure, it offers a vantage point on how research is made, measured, and mediated.

Whether you work in the research ecosystem, are a policymaker, or just curious about the mechanics behind how knowledge is created and rewarded, research musings aims to surface overlooked questions and spark informed discussion.

Series

research musings is a space for ongoing exploration, sometimes structured, sometimes improvised. Over time, it has taken shape around five recurring series, each offering a different lens on the research world:

• data: structured around a dataset, a chart, or a pattern,

• metrics: we introduce a method, define an indicator, or reflect on how metrics are made and used,

• AI: we examine the role of artificial intelligence in research—tools, policies, risks, and potential,

• policy: we look at the mechanics of research systems, funding structures, and strategic shifts,

• thought: we step back and reflect—on institutions, incentives, culture, and change.

Alongside these, some themes cut across series and evolve over time. We are starting to think of these as threads: evolving inquiries that approach a theme from multiple perspectives.

Threads

• cartographies of research
  This thread centres around one or more maps, revealing geographic patterns in research.

• mapping the margins
  This thread looks at underrepresented fields, overlooked regions, and research topics. Posts are united by a concern with visibility, inclusion, and structural gaps.

• research in tension
  This thread explores how political disruption, structural conflict, and ideological control shape research systems. From Cold War divides to suppressed disciplines, it focuses on moments when science is constrained, redirected, or displaced.

Posts by series and thread

The table below shows how recent posts align with both the five series and our threads.

As we continue this experiment, we welcome suggestions for future topics, collaborations, or guest posts. You can follow our updates by subscribing to the newsletter.

Key takeaways:

• [Bibliometrics] Name-based author identification risks substantial misattribution: in regions with high surname concentration (e.g., 31 % in South Korea and 19 % in China), initials and last name alone can fragment researcher profiles, undermining metric validity.

• [Theme] Global naming conventions, from East Asian homogeneity to Spanish dual-surname systems and mononym traditions, shape co-authorship patterns and demand culturally sensitive infrastructure (e.g., ORCID uptake, tailored disambiguation algorithms) to preserve the integrity of the scholarly record.

Recently, I was tempted to identify researchers on a publication by using only their last name, assuming that within a given paper, there would be only one matching name. Before jumping to conclusions, I decided to query Dimensions on GBQ to figure out how common it was for publications to include more than one author sharing the same family name.

I expected regional differences, but the extremes I found were surprising. In South Korea, nearly 31% of publications include more than one author with the same family name. In China, the figure is around 19%. Meanwhile, in countries like Brazil, Germany, or Norway, that proportion drops to just 3 to 5%.

Mapping name duplication in co-authorship

To explore these differences globally, I created the map below. This visualisation shows the proportion of publications in each country that include at least two co-authors sharing the same surname; the country of reference is the most commonly shared country. Darker shades reflect higher rates of name repetition within team, especially visible across East Asia. The hover shows the three most common shared names. In some countries this matches the most common names in that countries, while in others it matches other countries’ most common names.

Detailed country-level breakdown

To go further, the table below offers a country-level breakdown.
It ranks countries by their average share of publications with co-authors sharing a surname, and includes total publication volume, annual growth rate, and a year-by-year sparkline to show change over time.

Naming systems and their bibliometric side effects

Naming conventions are cultural. In China and Korea, a small pool of family names (e.g., Kim, Lee, Zhang, Wang) is shared across millions; Zhang is by far the top shared name in Dimensions. By contrast, many European or Latin American countries have much greater surname variation, often tied to occupations, places, or combined parental lineage. Spanish-speaking countries frequently use two surnames (from both parents), increasing combinatorial diversity. Iceland sidesteps fixed surnames altogether: your last name is derived from your parent’s first name, plus -son or -dóttir—so even siblings can have different surnames. There are even countries without surnames. See Wikiepedia’s article on mononyms; it lists Afghanistan, Bhutan, some parts of Indonesia, Myanmar, Mongolia, Tibet, and South India—maybe a future bite on identical first and last names in different regions of the world, since Western conventions impose on researchers from these regions to have a surname.

From a cultural perspective, these differences are fascinating. From a data and infrastructure perspective, they can be a difficult to handle. When dozens of researchers share a name like “J. Lee” or “X. Zhang”, distinguishing between them in scholarly databases becomes a complex task. This ambiguity can lead to misattribution, where one person’s work is wrongly assigned to another, or split profiles, where a single researcher’s output is fragmented across multiple identities. It also leads to fraud, where researchers claim research that is not theirs.

These issues affect more than just search results. Metrics used in hiring, funding, and evaluation can be skewed. Citation counts might be inflated or undercounted. Collaboration networks become harder to trace. In short, name ambiguity chips away at the trustworthiness of the scholarly record.

To address this, research infrastructure has been moving, slowly but surely, towards more robust solutions. The ORCID identifier gives each researcher a unique, persistent ID, independent of how their name is spelled or transliterated. Some platforms use machine learning disambiguation models, trained on co-authorship, affiliation history, and research topics, to separate identities that names alone cannot distinguish.

But uptake of ORCID still varies globally, and algorithmic disambiguation is never perfect. Cultural sensitivity, language, and data quality all play a role. That’s why infrastructure matters, not just technical tools, but policy, adoption incentives, and global awareness.