According to information theory, none of that processing should help me much. The chain runs one way: lesion (any abnormality of interest), then image, then processed image. If a processing step sees only the acquired image, what it produces can carry no more information about the lesion than that image already carried. Information theorists call this the data-processing inequality. Processing can preserve information or destroy it, but it cannot create it. In their new publication, Delfino and colleagues make the same argument: AI reconstruction and post-processing “cannot add patient-specific diagnostic information.”

And, yet, the processed image looks better and is easier to read. A denoising step that cannot add a single new fact about the patient in front of me improves every image quality metric, but that same step helps least on the cases where a clear image is most needed. Those two facts have one cause.

Where the improvement comes from

If the improvement cannot come from this patient, it comes from patients in general.

An MRI or CT image is not a photograph but a computation from measurements. Some features of the patient’s anatomy are pinned down by those measurements. Other features fall in the scanner’s blind spots, so no amount of careful reading recovers them. The set of features that cannot be measured is called the null space. Noise is a measurement that came out wrong; a null-space feature is one that no measurement addressed.

Something still has to appear in the image where those null-space features would be. A classical reconstruction leaves that part of the image blank, which is part of why classical images look soft or streaked. A learned reconstruction fills the blank space with whatever its training images suggest belongs there. The finished image does not mark which part came from the measurements and which from the guesswork.

The guesses can affect how the whole image looks, sometimes with serious consequences. Bhadra and colleagues traced hallucinated structures in tomographic reconstruction, plausible-looking features with no support in the data, to the null space. These hallucinated features can present as a lesion that was never there, or as the quiet smoothing away of one that was.

So, a denoiser trades missing information that would have been specific to this patient for information typical of the population. For most patients, that is an excellent trade.

Who pays

The trade-off between accuracy for the individual and accuracy on average has been a known statistical phenomenon since Charles Stein’s 1956 work on estimating several quantities at once. Pulling estimates toward the population average lowers error across the population while raising it for anyone who sits far enough from the average.

To find where the trade-off stops being worthwhile, I built a stripped-down computational model. In a low-noise scenario, once a simulated patient’s anatomy sits beyond roughly 1.5 standard deviations from typical, a processing step adds more error than it removes. That threshold moves outward as the raw image gets noisier, because a noisier measurement has more to gain from borrowing information from the patient’s population.

Beyond those thresholds, a step tuned on a population pulls every image toward what most patients look like, and the finding that makes a study abnormal is precisely what most patients do not have. Denoising can therefore make disease systematically harder to detect for unusual patients; the effect is not random. Unfortunately, this means that the enhancement provided by processing is at its best on the studies that would have been read correctly without it.

Why the metrics cannot see it

Peak signal-to-noise ratio, structural similarity and mean squared error, the three scores in almost every reconstruction paper, all measure how closely a processed image matches a reference. Each averages over every pixel and every case in the test set, and none is tied to a diagnostic task. An average will never report what happened to the unusual patients.

Some studies have looked at image quality metrics alongside clinical performance. For example, Jensen and colleagues measured image quality and lesion detection in one study of abdominal CT for liver metastases. Radiologists rated the reduced-dose deep-learning reconstruction higher for subjective image quality than standard-dose filtered back projection, the classical reconstruction. But, detection was worse. Readers missed several lesions in the “better” images. While the missed lesions cannot be attributed to the reconstruction alone because the reconstructed images were collected with a reduced dose, this result is still concerning. Indeed, as Delfino and colleagues point out, reduced dose and deep-learning reconstruction are marketed and used together.

Denoising is still useful

Denoising is not a scam and should not be switched off. Denoising works even though the math says it shouldn’t. The reason it works anyway is that humans are not “ideal observers,” where an ideal observer represents the hypothetical reader who extracts every bit of information an image contains. Real readers see more in a quieter image, even when that image holds the same information as the noisy one, or less. Still, we must be careful. What the math insists on is that a “better” image is better for a specific clinical task.

Trained on one pipeline, used on another

Developers of image processing pipelines have an intended use in mind, but that intention is rarely recorded, and it need not match the clinician’s task. A model trained on images from one pipeline and run on images from another is not looking at the same object, even when scanner, protocol and anatomy match. Domain shift is the name for a model meeting new data unlike the data it learned on.

Domain shift of this kind arrives through a door nobody guards. The processing chain is not reliably in the DICOM header, nor in the methods section, nor in the procurement contract, so a model can change what it is looking at without anyone being told.

Where do we go from here?

I cannot change what a scanner does to a study before it reaches me. What I can do is consider what the scanner did. A radiologist who knows a study was heavily denoised can weigh a subtle finding differently in an unusual patient; a researcher who knows which chain produced the training images can check whether the deployment images came through the same one. To allow clinicians to make these informed decisions, we need better reporting. Every imaging AI paper should state, in one sentence, the reconstruction and post-processing chain of its training images and of its deployment images, and whether the two are the same.

Clemente García-Hidalgo, MD, MSc, is a radiology trainee at Hospital Morales Meseguer (Murcia, Spain), a PhD student at the Translational Imaging Biomarkers group of Bellvitge (Barcelona), and Yves Menu Fellow (AI) at European Radiology. Main interests: DSC perfusion, AI, radiomics, and synthetic data. @TorkitorYT

Further reading

• Antun and colleagues showed that tiny, deliberately chosen changes to the measurements can make deep-learning reconstructions invent or erase structure (PNAS 2020;117:30088-30095, https://doi.org/10.1073/pnas.1907377117).

• Bhadra, Kelkar, Brooks and Anastasio traced those invented structures to the null space and gave them a formal definition (IEEE Transactions on Medical Imaging 2021;40:3249-3260, https://doi.org/10.1109/tmi.2021.3077857).

• Barrett, Myers and colleagues set out the task-based alternative to the pixel-averaging metrics: judge an image by how well a stated diagnostic task can be performed on it (Physics in Medicine and Biology 2015;60:R1-R75, https://doi.org/10.1088/0031-9155/60/2/R1).

• Barrett, Yao, Rolland and Myers compare the model observers used to score images, mathematical stand-ins for a reader, with the “ideal observer” of this post as the ceiling none of them beats (PNAS 1993;90:9758-9765, https://doi.org/10.1073/pnas.90.21.9758).

So, what is Venture Capital and what does a Venture Capitalist really do?

A Venture Capital fund is an investment class where a group of Limited Partners (LPs) contribute capital into the fund. The fund deploys capital into a targeted number of startups. The fund is led by the General Partner (GP) who makes overall final decisions on startup investments and strategic decisions for the fund and the Management Company (Manco). Funds are organized around a fund “thesis” which is their main area of investment. Funds also typically target companies at different stages of development ranging from pre-seed, seed, series A, and growth.

Funds make their returns through exit events or initial public offerings of the startups they invest in. Similar to other investment vehicles, such as real estate, stocks/bonds, or commodities, the ultimate goal is to make returns on the LPs’ investments into the fund. LPs invest into funds that fit their overall appetite for risk and their investment interests.

The Venture Capitalist (VC; the GP and the other investors employed by the GP) is ultimately a steward of their investor’s capital, aiming to get good returns. They seek companies that will return the fund with large addressable markets (generally greater than $1 billion total addressable markets). For context, markets are broken down into:

· Total Addressable Market (TAM): The total global market demand.

· Serviceable Addressable Market (SAM): The segment that can be realistically targeted.

· Serviceable Obtainable Market (SOM): The exact share that can be captured.

Venture capital funds typically invest in 15–25 startups to diversify their portfolio and minimize risk. Knowing that the majority of startups fail (due to lack of product market fit, running out of cash, etc.), funds target the “Power Law”. That is, 20% of the portfolio can represent 80% of returns. It only takes 3–4 of the startup investments succeeding to return the entire fund. Still, the VC must make a number of considerations before investing in a startup to ensure they have the highest chance of getting good returns on their investor’s capital.

What does a startup need to attract venture capital funds?

A general rule that many investors, including myself, use is the 3 T’s: Team, Timing, and Traction/Technology.

At the earliest stages investors are really betting on the team. Some common questions a VC would ask include:

● Are these founders the right ones to get the job done?

● Do they have the technical or domain expertise, demonstrated prior success, and/or network to succeed?

● Do they have the ability to draw and retain the talent they are missing if they have a technical gap?

● Are they coachable?

This last point is critical because it is not uncommon for business plans to change, requiring a pivot. The ability to receive feedback and change the initial plan if it isn’t working is crucial.

Timing is another key element. History is rife with examples of innovative ideas that were ahead of their time but failed because the infrastructure or general consumer culture was not in place to support them. Not only does the idea need to be innovative, but it also has to have the necessary elements for it to work at large scale. “Why now?” is a common question. And, “is this solution going to be something that solves a universal pain point?”

Traction and/or technology are emphasized more or less depending on the stage of the startup. At the earliest stages of investment, there may be little to no traction other than a functioning prototype and initial seed funding and early adopter testimonials. In those cases, the technology, specifically the defensibility of the technology, is paramount. At later stages, traction is the key metric to evaluate. In brief, is someone willing to pay you money in exchange for your product? And, is that someone going to be interested in the full product after a free or discounted trial has ended? Additional questions include:

● Is your product being used on a site outside your home institution?

● How “sticky” is your product? Can you show metrics of how often users use your product and how often they come back to it?

Having hard numbers rather than qualitative measures will make a strong impression on funding sources. Additional must-haves include:

● Defensible intellectual property that is clearly assigned to the company (especially critical for university-based startups)

● A solid understanding of any regulatory pathway

● A general idea of a strategy for going to market (referred to as Go-To-Market strategy in the startup world). How will you scale your product? Are there channel partners that you can leverage?

How to get started?

Before I launched my own startup, I took what was previously called the UCSF Startup 101 Course. It was very useful in giving me the foundations for learning about the general process of moving from an academic with an idea to a founder ready to move a startup beyond the walls of the university. While the UCSF course has since been modified, many programs exist. Programs like the NSF i-corps program are also excellent. NSF provides a structured program that includes a comprehensive customer discovery element which is highly valuable.

As a related aside, there is sometimes a fear that someone may “steal your idea”; however, it’s important to vet your idea, even if just conceptually, and to learn to speak non-confidentially about it. VCs see and vet numerous ideas, many of which are similar. It is industry standard among VCs to not sign an NDA as it limits the overall freedom to operate. Brad Feld, a well-known technology investor takes a deeper dive into this topic on his blog.

When is the right time to reach out to VCs?

Founders often reach out to funding sources when they feel their work would benefit from the capital, expertise, and network of an institutional venture capital firm. Unlike more consumer-based products, products in Radiology, especially those involving Artificial Intelligence, are capital intensive. It is difficult to self-fund a company through revenue generation, and the need for outside funding is typically greater. Accordingly, waiting to reach out to VCs can be a mistake. It is generally felt that securing as much non-dilutive capital as possible at the beginning stages of a startup allows for the idea to be developed without giving away equity (and degree of influence) away too early. Outreach to VCs early to establish a relationship is reasonable and mentioning where you are in your journey upfront is beneficial.

In some cases, the VC may find you! Specialty-focused VCs will keep an eye on newsfeeds and LinkedIn to see what new innovations are happening. If you have the pieces in place mentioned above (team, timing, traction, defensible IP, etc.…), initial outreach with VCs who have previously invested in your general space and understand it is a great starting point.

This post is just the beginning – there is always more to learn about venture capital. What excites me about the space is the ability to fuel talented innovators in radiology, the field I spent the last decade and a half practicing in. I believe AI will bring about revolutionary impacts in our field, with radiologists at the forefront of this innovation and VCs providing the financial support they need.

Dr. Jesse Courtier is Founder and Managing Director of Radiologue Ventures. He is also an Adjunct Clinical Professor of Pediatric Radiology at the Stanford University School of Medicine.

Despite the growth of AI research in medical imaging, few models have made the transition from promising research results to the clinical practice. This gap raises a question that deserves more attention: are we always developing AI because the clinical problem requires it?

When the availability or popularity of a technology begins to define the research question, innovation risks becoming driven by the technology itself rather than by the problems it is meant to solve. Long before choosing an architecture, defining a loss function, or tuning hyperparameters, clinical AI researchers must ask a more fundamental question:

How should a clinical problem be translated into the computational design of an AI system?

Insights from research

I did not arrive at this philosophy by reading about a new AI architecture. It emerged while I was developing a model for coronary plaque characterization.

From a technical standpoint, the obvious solution to the problem of coronary plaque characterization was to develop a multiclass segmentation model capable of identifying every plaque type simultaneously. This approach was technically sound and consistent with the contemporary AI literature. But something about it felt unnatural to me.

I couldn’t articulate why until I learned about how an interventional cardiologist examines a coronary image, and I realized that clinical reasoning unfolds differently. A cardiologist first identifies what belongs to the plaque and only then characterizes its composition. So, instead of asking one network to solve the whole problem in one step, I designed two consecutive models: one to distinguish plaque from non-plaque and another to characterize plaque type. The new, clinically grounded approach outperformed the obvious technical solution.

The improvement did not come from using two models. It came from asking the right questions in the right order, with a clear goal: to improve patient outcomes.

Designing intelligence

Between identifying a clinical need and expressing its solution through mathematical models and network architectures lies a critical step: designing the intelligence. It is the step in which we decide not how the model should be built, but how it should approach the problem and where its learning should be focused.

There is an interesting parallel in art. In a painting, not every part of the canvas needs to carry the same level of detail. While a camera captures as much detail as possible, an artist selectively renders the features that are essential to the subject while leaving other regions less resolved. In the painting shown below, for example, the feeling of connection and the journey toward peace are emphasized through the peace symbol and the imagery in the clothing, while other elements receive less visual emphasis. What matters is not simply how much detail is created, but where that detail is needed to communicate what the artist wants us to see. In other words, artists exercise judgment in how they apply their intelligence to rendering an image.

Perhaps we should think about AI in a similar way. A model does not necessarily need to learn every distinction, from every part of the input, at the same time. We can design the learning process so that one decision narrows the problem for the next. Each step directs the model toward increasingly relevant information, making the learning task more focused and potentially reducing ambiguity and error. The strongest AI systems are not necessarily those built with the most sophisticated architectures. They are often those whose computational design faithfully reflects the structure of the problem they were created to solve.

A culture that rewards visibility over value

Perhaps part of the problem lies in what academic culture has learned to recognize and reward. I have noticed two very distinct cultures within academia, each with their strengths and flaws:

In technical journals, novelty tends to be associated with computational complexity. New architectures, sophisticated mathematical formulations, additional network modules, and elaborate loss functions frequently become the center of attention. These contributions can represent scientific advances when they are driven by necessity. But complexity itself is not innovation. Mathematics should explain an idea, not become the idea.

On the other hand, in clinical journals, the AI model is treated almost as a black box. Once an architecture has been selected, relatively little discussion is devoted to the numerous technical design choices that form the final model. Instead, the focus is on whether the clinical question is compelling, whether the dataset is large enough and whether data has been collected from multiple centers. Clinical relevance is not innovation. The clinical problem should define the intelligence we build, not validate it.

Neither perspective fully captures what makes AI scientifically meaningful. One emphasizes computational novelty without always asking whether the added complexity supports clinical practice or just nominally improves a performance metric. The other emphasizes clinical relevance without questioning whether the computational design mirrors the clinical reasoning it seeks to support.

The most significant innovations emerge somewhere between these two extremes, where neither computational novelty nor clinical relevance is treated as sufficient on its own. This is the space where the clinical problem, scientific reasoning, and computational design meet. This is where the purpose of AI returns to where it should have started: solving a problem worth solving!

Looking beyond architecture

Artificial intelligence will continue to evolve. Today’s architectures will eventually be replaced by better ones. The equations we admire today may one day become historical footnotes. The ability to think deeply about a problem, however, will never become obsolete.

Every meaningful AI project should begin with a clinical need. If an algorithm does not address a real clinical challenge, no amount of computational sophistication can justify its existence. But identifying the clinical problem is only the first step. The most impactful AI solutions emerge from understanding a clinical problem so deeply that the computational strategy becomes a natural extension of the clinical reasoning itself. To me, this is the art of designing intelligence.

Designing AI is an exercise in connecting disciplines. Medicine, mathematics, engineering, physics, philosophy, and even art each illuminate different dimensions of the same problem. Clinical medicine defines the question. Physics explains the mechanisms. Mathematics provides the language. Engineering transforms ideas into systems. Philosophy shapes critical thinking, and art cultivates creativity. Artificial intelligence becomes the bridge that unites these perspectives into meaningful intelligence.

I believe the future of AI will belong to those who learn to think before they build. Meaningful innovation is rarely the product of competition or complexity for the sake of complexity. It is the outcome of thoughtful minds that value understanding over recognition, curiosity over certainty, and purpose over prestige.

Atefeh Abdolmanafi is a researcher with a master’s degree in physics and a Ph.D. in computer science, specializing in medical image analysis, which she obtained from Université du Québec, Montreal, QC, Canada, in 2018. She has made advancements in cardiovascular imaging, with a focus on Intravascular Optical Coherence Tomography. Currently based at the University of Michigan’s Department of Radiology in Ann Arbor, MI, USA, Dr. Abdolmanafi specializes in color flow ultrasound technologies, driving innovation in medical imaging. Committed to interdisciplinary excellence, she blends art and science to inspire creativity and enhance healthcare solutions.

Radiology: Artificial Intelligence is launching two new article series: Foundations and Frontiers.

The Foundations series will focus on the conduct and reporting of high-quality AI research.

Articles will cover study design, methodology, performance metrics, standards, terminology, statistical analysis, reporting, and other topics that affect the validity, transparency, and usefulness of AI research. The emphasis will be on clear, practical guidance: what works, what doesn’t, and how common mistakes can be avoided.

The goal is simple: to help authors, reviewers, editors, and readers understand how to do AI research—and report it—the right way.

The Frontiers series will look ahead.

These articles will introduce emerging topics that are not yet part of routine radiology practice but may soon influence research or clinical care. Articles might include quantum AI, robotics, autonomous systems, embodied AI, new computing architectures, and evolving forms of collaboration between people and machines.

Frontiers articles should be forward-looking but grounded in science. They should help readers understand what is coming, why it may matter, and what challenges will need to be addressed before these technologies become widely used.

We Welcome Proposals

Prospective authors are invited to submit a proposal for either series to the journal’s editorial office (rad-ai@rsna.org).

Proposals (500 words or less, please) should identify the intended article series, describe the topic in two or three sentences, explain why the topic is timely and relevant, outline the proposed scope, and describe the authors’ expertise. In general, manuscripts will be no longer than Special Reports; brevity is preferred.

The Foundations articles will strengthen today’s AI science. We hope that the Frontiers series will spark thoughtful conversations about what may come next.

Charles E. Kahn, Jr., MD, MS is professor of radiology at the University of Pennsylvania and serves as editor of Radiology: Artificial Intelligence.

Radiomics is the practice of extracting large numbers of quantitative measurements from medical images (e.g., texture, shape, intensity distribution) and searching them for statistical associations with a clinical outcome. Its appeal is obvious: the images have already been acquired, the features are cheap to compute, and the software is free. The difficulty is equally inherent. A typical study has a few hundred patients and more than a thousand measurements per patient, and the analyst therefore faces dozens of decisions about how to reduce the one to the other. But what is the outcome of perfectly defensible, standard decisions when there is nothing there to find?

A dataset with nothing in it

I constructed a dataset in which no predictive relationship could exist. I began with 150 synthetic patients for training: 1200 random numbers per patient, generated to behave like real radiomic features (including their tendency to correlate with one another), and one of two group labels assigned by a coin flip. A second cohort of 100 patients, for testing, was generated the same way (with a small shift in scale, to mimic a different scanner). Neither cohort contains a biomarker (i.e., a measurement that genuinely tracks which group a patient belongs to). There is nothing in this dataset for a radiomics analysis to find.

Then, I analyzed this fake dataset using the standard radiomics pipeline: four combinations of input feature blocks, three preprocessing methods (z-score, min-max, quantile), five classifiers, three testing strategies (a single train/test split, five-fold cross-validation, and evaluation on the second cohort), three candidate endpoints, and three cohort stratifications (all patients, by scanner field strength, by age). In total, this amounted to 1,620 analysis configurations.

Finding false meaning in the noise

Across all 1,620 configurations the median AUC for predicting the category of each patient was, appropriately, 0.49 (an AUC of 0.50 reflects a coin flip guess of category). But, research is the search for the best model. The maximum AUC across configurations was 0.77, and fifteen configurations exceeded an AUC of 0.70. This would be considered performance worthy of a publication.

What went wrong?

A reviewer looking at a result like this knows what questions to ask:

Data leakage: did anything about the test patients reach the model while it was being built?

Multiple comparisons: were so many variants tried that one was bound to look good by chance — the practice usually called p-hacking?

Sample size: was the result computed on so few patients that it could swing on a handful of cases?

These are the standard failure modes, and they are the first three questions any competent referee asks.

Only the first does not apply. Selection and scaling were confined strictly to the training folds, so nothing about the test patients could influence which features were chosen; there was no leakage at any point. The second applies in full: 1,620 variants clearly constitute excessive multiple comparisons. The difference between a paper that reports the 0.77 AUC as a resounding success and the story I am telling in this post is that I am reporting all of the comparisons. The third question deserves a straight answer: the best-performing configuration was a post hoc subgroup with only twenty-one patients in the test set. That is a small sample size. But it is not an overt mistake; it is one of the 1,620 branches an analyst could legitimately take, and subgroup analyses of that size appear in the published literature routinely.

The inflation of my results, from a sensible median result of 0.49 to an overstated best of 0.77, is a property of the search itself: when 1,620 analyses are conducted and one is reported, the reported value is the maximum of a distribution rather than a reliable estimate of performance.

Statistics measures the risk

A p value of 0.05 is a familiar threshold: if there is genuinely nothing to find, an analysis will find something about one time in twenty by chance alone. Run twenty analyses on empty data and, on average, one will look significant. Run 1,620 and the usual arithmetic says at least one spurious success is effectively guaranteed.

That arithmetic is oversimplified, and it is worth saying so rather than leaning on a frightening number. It assumes the 1,620 analyses are independent, and they are not: they share the same patients, and two pipelines that differ by a single preprocessing step tend to succeed and fail together. Measured across twenty independently generated noise datasets, the 1,620 configurations behave like about thirty genuinely independent attempts. The honest probability of at least one spurious success is therefore around 0.78 or 78% — high, but not the near-certainty the simple calculation promises.

Ultimately, a probability of 78% is still not reassuring. It means that more than three times in four, a study built like my simulation will seem to reveal something worth writing about when there is nothing there. My measured result was worse than the arithmetic predicted: I conducted twenty replicate studies, and every one produced at least one configuration above an AUC of 0.70. The weakest of the twenty still reached 0.75, and the average was 0.81. 20 out of 20. Analyses that overlap reduce the size of the problem; they cannot change its direction, because the best of a set of related attempts is still the best of a set.

The importance of external validation

One defense against a result that exists only in a single dataset is to test the model somewhere else: on patients from a different hospital, scanned on different equipment. If a finding is an artifact of one cohort, it should disappear when the cohort changes. Five hundred forty of my 1,620 configurations were tested this way, on the second synthetic institution.

The best of those 540 external validation configurations reached 0.64. That is lower than the 0.77 the search found on held-out patients from the original cohort, so testing elsewhere did buy something real. But the result is still not 0.50, which is what a dataset with nothing in it should produce. External validation narrows the advantage a large search confers; it does not remove it.

Where do we go from here?

No individual choice in my search would be considered overtly improper, and each could be defended in a methods section; screening features in a high-dimensional setting is not a vice but a mathematical necessity. The problem is not that the search happened. It is that nothing in the published record shows it happened.

Several checklists already exist to help researchers avoid common pitfalls (CLAIM, CLEAR, and TRIPOD+AI among them), and their systematic adoption would itself be considerable progress. However, none of them yet asks how many configurations were evaluated. A single sentence in the methods would close the gap: the number of analyses run, and the basis on which the reported one was chosen. That one line is what separates a prespecified result from the best of 1,620.

This blog takes its name from Glendower’s claim, in Henry IV, that he can summon spirits from the vasty deep. Hotspur replies that anyone may summon them; the question is whether they come. A sufficiently large analytic search will always return a biomarker. Whether any biomarker was ever in the dataset is a separate question, and one the published record is not currently structured to answer.

Clemente García-Hidalgo, MD, MSc, is a radiology trainee at Hospital Morales Meseguer (Murcia, Spain), a PhD student at the Translational Imaging Biomarkers group of Bellvitge (Barcelona), and Yves Menu Fellow (AI) at European Radiology. Main interests: DSC perfusion, AI, radiomics, and synthetic data. @TorkitorYT

I am thrilled to welcome the 2026-2028 cohort to the journal’s Trainee Editorial Board. This group brings together residents, fellows, and PhD students from across the world, united by a shared curiosity in artificial intelligence. Their efforts are already helping to reshape medical imaging.

Please join me in welcoming:

• Farzana Z. Ali, MD, PhD, MPH — Focuses on integrating AI-driven analysis with PET biomarkers in theranostics, bridging computational methods and molecular imaging.

• Mohammadreza Chavoshi, MD — Works on developing, evaluating, and clinically integrating AI models in radiology, with an eye toward real-world translation.

• Clemente García-Hidalgo, MD, MSc — Brings interest in dynamic susceptibility contrast (DSC) perfusion imaging, AI, radiomics, and synthetic data to explore how these tools can enhance quantitative imaging.

• Jinkui Hao, PhD — Researches the intersection of AI and cardiac imaging, working to bring machine learning methods into cardiovascular applications.

• Navid Hasani, MD — Explores multimodal AI systems, the use of AI for report analysis, and the responsible integration of AI in medical imaging, drawing on collaborations with leading AI industry organizations.

• Zelong Liu — Investigates generative AI, 3D vision-language models, and large-scale data pipelines for imaging research.

• Alexander Lu — Focuses on motion compensation in interventional imaging, combining physics and deep learning to build robust imaging algorithms and evaluation methods.

• Matthew Miller, MD — Applies a background in machine learning and computer vision to interests in procedural innovation, device development, and Bayesian statistical methods.

• Michael Olufawo, MD — Works across neuroimaging, machine learning, and health economics, applying a bioinformatics background to imaging research.

• Joshua Rothwell — Focuses on the evaluation and clinical translation of AI for breast cancer screening.

• Mahan Salehi, MBBS — Works in imaging AI research, development, and clinical evaluation, with a current focus on translating in-house AI tools into validated, commercially viable technologies.

• Mustafa Ege Seker, MD — Pursues research in medical image segmentation, large language models, workflow and screening optimization, and AI in interventional radiology.

• Sonali Sharma —Integrates reasoning, visual attention, and biomedical data to develop and evaluate multimodal AI models for radiology.

• Matthaios Triantafyllou, MD — Focuses on radiomics, artificial intelligence, and musculoskeletal imaging.

• Yuli Wang, PhD — Investigates AI in medical imaging, particularly vision-language foundation models and clinically oriented AI decision-support tools.

• Mingming Wu, MD, PhD — Works on interventional MRI, body composition profiling, quantitative MRI, and motion correction, with broader interests in k-space methods and AI for image reconstruction.

This group brings energy, expertise, and fresh perspectives to the journal, and I’m personally looking forward to working with them. They join an extraordinary community of current and former TEB members. Welcome!

Charles E. Kahn, Jr., MD, MS is professor of radiology at the University of Pennsylvania and serves as editor of Radiology: Artificial Intelligence.

But, the cloud has never been weightless. And it is not small, either.

From Episodic AI to Persistent Agents

The next phase of AI will not be limited to passive chatbots or isolated prediction tools, applications with lesser infrastructure demands. The field is moving toward agentic AI.

These agentic systems promise relief from administrative burden and cognitive overload. Yet, they cast a shadow: continuous computation that depends on an AI infrastructural backbone.

That backbone is already visible. In the United States alone estimates suggest there are already more than 4,500 operational data centers, with many more in development. Public maps from DataCenterMap and the Brockovich Data Center Map make this geography visible.

Data Centers Become a National Infrastructure Question

The United States has begun to treat AI infrastructure as a national priority. In January 2025, Executive Order 14141, “Advancing United States Leadership in AI Infrastructure,” addressed federal support for AI infrastructure, including data centers, energy resources, permitting, and reporting requirements. In July 2025, a subsequent executive order, “Accelerating Federal Permitting of Data Center Infrastructure,” emphasized faster permitting, financial support for qualifying data-center projects, and environmental-review streamlining. The policy trajectory is clear: AI data centers have become a bipartisan federal infrastructure issue.

But making data centers a national priority raises the practical question of what actions should be taken to address the issue. A U.S. Department of Energy report estimated that data centers consumed approximately 4.4% of total U.S. electricity in 2023 and could consume 6.7% to 12% by 2028. As new data centers are built, meeting that demand will require the development of a resource-intensive infrastructure to sustain them.

Water use and heat rejection are part of that infrastructure. Data centers must remove heat, and they depend on cooling strategies that typically use large amounts of water resources, a serious concern for the many planned AI data centers that will be located in drought-affected regions in the US. Water access challenges have motivated companies to explore more unusual approaches. Some facilities use evaporative cooling; others use air cooling, liquid cooling, closed-loop systems, or hybrid designs. Microsoft’s Project Natick tested whether data centers could be deployed underwater near coastal populations, where cooling might be more efficient and latency lower. The idea has continued elsewhere. In 2026, an offshore wind-powered underwater data center off Shanghai’s Lingang Special Area was reported to have entered operation, using seawater for cooling and electricity from offshore wind.

Big Tech Is Buying the Energy Stack

Large technology companies appear to understand that the next bottleneck for AI may not only be chips or models, but also the infrastructure required to sustain it. Amazon and Meta have both entered long-term agreements with energy suppliers, including nuclear-power arrangements, to support the anticipated growth of AI and cloud computing. These examples show that leading AI companies are no longer simply buying electricity. They are not only securing firm power, but also shaping the energy infrastructure needed to run AI systems at scale.

Why Medicine Should Care

Healthcare already depends on invisible infrastructure, and radiology is one of the clearest examples. PACS downtime, cloud archive interruptions, and network failures reveal how much imaging care is mediated by systems that radiologists do not control and often cannot see. Agentic AI could deepen this dependency by embedding persistent computational agents into reporting, protocoling, worklist prioritization, imaging follow-up, and care coordination.

Healthcare therefore needs AI infrastructure literacy, and radiology could help lead it. Health systems should ask where AI systems run, what infrastructure they depend on, and what happens when that infrastructure fails. Knowing these things allows radiology departments to build contingencies, preserve meaningful backup workflows, and understand the infrastructure on which imaging care increasingly depends.

The cloud has never been weightless. To reap the benefits of AI, radiology must also bear the weight of the cloud in megawatts, cooling water, land, transmission lines, backup generators, nuclear power contracts, and, increasingly, steel modules placed beneath the sea.

Dr. Jawed Nawabi is a neuroradiology specialist and assistant professor at Charité – Universitätsmedizin Berlin, where he serves as campus lead for one of Charité’s three major clinical campuses. At Charité, he heads the Neuroradiology AI Imaging Lab as well as the institute’s digital transformation division, where he oversees the implementation and governance of AI systems in clinical practice. His research focuses on the development, evaluation, and clinical integration of large language models and deep learning–based imaging models in radiology. He holds a Master’s degree in AI in Healthcare and is an alumnus of the Digital Health Clinician Scientist Program. In his current work, he leads projects on digital twins for neuro-oncological tumor boards, exploring how AI-driven models can support multidisciplinary clinical decision-making. He is a current member of the Trainee Editorial Board for Radiology: Artificial Intelligence.

Enjoyed this perspective? Check out Dr. Nawabi’s previous post:

I was working on a multicenter radiogenomics collaboration when our team encountered an unexpected problem. Two experienced institutions were studying the same disease using high-quality MRI data and asking the same scientific question. Yet our initial analyses produced very different results.

During a Zoom meeting, the discussion quickly turned to equations, statistical models, and computational methods. Everyone was trying to understand why our findings did not agree.

Years of working alongside clinicians and computational scientists had taught me that we often frame AI problems as algorithm problems. When results don’t match, our instinct is to question the model. Yet many of the most consequential decisions are made long before an algorithm is ever trained.

So instead of asking about the mathematics, I started asking different questions.

How did you segment the enhancing tumor?

How did you define normal-appearing white matter?

Which images did you use?

Which software generated the segmentations?

Those questions changed everything.

Neither group had made a mistake. We had simply built different workflows. Small differences in preprocessing and image analysis—each entirely reasonable on their own—had fundamentally altered the downstream results. Once those differences were harmonized, our findings aligned.

Behind the Algorithm

Many of the greatest challenges in medical AI arise before an algorithm ever sees the data.

Behind every published AI model are months of work that rarely receive attention: identifying the right imaging studies, anonymizing data, navigating security requirements, transferring files across institutions, harmonizing preprocessing pipelines, validating annotations, and ensuring that different teams are performing the same analysis.

Preparing imaging data may take months—or even more than a year—before meaningful scientific work can begin. Moving hundreds of MRI examinations across systems involves far more than copying files. Sequence names differ between scanners. Outside studies can follow different conventions. Transfers fail. New analyses often require repeating much of the process. Every obstacle requires another discussion between clinicians, engineers, IT specialists, and researchers.

None of that appears in the final figure of a manuscript.

Yet it often determines whether the manuscript can be written in the first place.

The Human Interface

The same lesson surfaced again during a recent panel discussion on AI in radiology. Much of the conversation focused on foundation models and emerging technologies. Then someone asked a simple question:

“If technology could solve one problem for you tomorrow, what would it be?”

My answer was immediate.

“Help us organize, anonymize, and move our data.”

The conversation shifted almost immediately. Instead of discussing the next breakthrough algorithm, people began sharing the practical challenges they face every day—moving data across institutions, reproducing results, harmonizing preprocessing pipelines, and implementing promising models outside highly controlled research environments.

It turned out that many of us, despite working at different institutions and in different disciplines, were facing the same invisible challenges.

Those conversations rarely make conference headlines, but they often determine whether AI reaches patients.

Every discipline approaches AI differently

Working on these projects also taught me that every discipline approaches AI from a different perspective. Radiologists focus on clinical relevance. Engineers optimize algorithms. Computational scientists emphasize methodological rigor. IT specialists prioritize security, governance, and interoperability.

No one is wrong. Everyone is solving a different part of the same problem.

Looking back, I realized that my role in these multidisciplinary collaborations was rarely developing the algorithm itself. More often, it was bridging disciplines—asking clinical questions that influenced computational decisions while explaining technical constraints that affected clinical interpretation.

Artificial intelligence may be built on code, but successful medical AI depends on people learning to understand one another.

Looking Beyond the Algorithm

As AI continues to advance, I believe our focus should expand beyond building better models. We should invest equally in reproducibility, shared standards, interoperable infrastructure, and multidisciplinary collaboration. These efforts may not produce the most exciting conference slides, but they determine whether innovation can be translated into clinical care.

The future of medical AI will certainly require better algorithms. But it will depend just as much on something we rarely celebrate—the people, partnerships, and shared understanding that make those algorithms clinically meaningful.

That is the AI nobody sees.

Maguy Farhat, MD, is a diagnostic radiology resident at the University of Maryland Medical Center. Her research interests include artificial intelligence, radiomics, radiogenomics, neuro-oncology, and quantitative imaging. She is passionate about developing clinically meaningful AI through multidisciplinary collaboration and translating computational research into improved patient care.

But I have often found myself uncomfortable with a common question:

“Will AI replace radiologists?”

For years, I could not explain why this question felt incomplete to me. Then, one year ago, I encountered radiology from a very different perspective, one that changed my understanding of the fundamental role of AI. I realized that the deeper question is not whether machines can become more capable, but whether capability and judgment are the same thing. Answering that question requires looking beyond technology into anthropology, philosophy, and the nature of human expertise itself.

The Space Between Evidence and Action

After receiving a cancer diagnosis, I instinctively tried to approach the reports, pathology findings, and imaging assessments exactly as I would approach a scientific paper. As a researcher, I have been trained to make sense of complex problems by turning to evidence, studying the details, and trusting that a deeper understanding of the data will bring greater clarity. So, I read the medical evidence and examined the report terminology, hoping it would lead to a sense of certainty. Finding none, I assumed that what I needed was more information, perhaps the kind of information an AI model could discover from some nuance in the data.

However, the lack of information was not the problem. The abnormality had already been detected, the pathology had already been reported, and all the data were available. What remained uncertain was what the data meant and what should happen next. How confident should my doctor and I have been in the imaging findings? Did the imaging and pathology tell a consistent story? Was additional imaging necessary? Did the risk of missing disease outweigh the risk of overtreatment?

In speaking with my radiologist, I was seeking a way to make sense of the abnormality that had already been detected and characterized. The radiologist’s value lay in interpreting what the available evidence meant, how much confidence should be placed in it, what uncertainties remained, and how those uncertainties should influence the next decision.

That experience taught me something I had not fully appreciated as a researcher: the most important question in medicine is often not, “What is likely true?” but rather, “What should be done?”

An AI system can help answer the first question by estimating probabilities, identifying patterns, and quantifying risk. However, it takes a radiologist to answer the second question. Doing so requires judgment, responsibility, and accountability. This distinction is particularly important in radiology because radiologists operate at the intersection of evidence and action. They do more than detecting abnormalities. They assume responsibility for interpreting uncertainty, communicating significance, and guiding decisions that carry real consequences for patients.

Wisdom Beyond Prediction

The distinction between data collection and judgment is not a new concern. More than two thousand years ago, long before the first radiologists, let alone the first AI models, Aristotle distinguished between techne and phronesis in Nicomachean Ethics. Techne refers to technical capability—the ability to calculate, classify, predict, and execute. Phronesis refers to practical wisdom—the ability to make sound judgements under conditions of uncertainty. AI is rapidly becoming extraordinary at techne. It can identify patterns, estimate probabilities, quantify risk, and generate recommendations. But AI cannot achieve phronesis.

As long as human patients need care, there must be human radiologists. The anthropologist Clifford Geertz argued in The Interpretation of Cultures that humans are “animals suspended in webs of significance” that they themselves have spun. We do not simply collect information; we create meaning from it. A lesion is not only a collection of pixels. For a patient, it may represent fear, uncertainty, hope, survival, or a life-altering decision. For a clinician, it may shape a course of action. For a scientist, it may become a question to investigate. The image remains the same, but its meaning changes with the human context in which it is interpreted. Understanding emerges not from the image alone, but from the radiologist interpreting that image within a broader human context.

The Human Art of Making Tools

Anthropologists have long argued that humans are distinguished not by strength or speed, but by their ability to create tools (e.g., Gesture and Speech by Leroi-Gourhan). The history of civilization includes many examples of tools that have extended human capabilities. Just as the telescope extended vision, the microscope extended observation, and the computer extended computation, AI extends our capacity for pattern recognition. Viewed through this lens, AI is the latest expression of a deeply human tendency: building tools that amplify our capabilities.

I believe AI will become indispensable to radiology, not because it is a radiologist, but because it is a tool. AI will improve detection, efficiency, and scientific discovery. The radiologist of the future may spend less time searching for findings and more time exercising their judgement, communicating meaning, and helping patients navigate uncertainty.

The next time someone asks, “Will AI replace radiologists?”, I hope you will reassure them that this is not the core question. Not because AI is incapable of extraordinary things, but because the question misunderstands both AI and radiology and underestimates their power together. The future of radiology will be a partnership between knowledge and wisdom. And that partnership may ultimately be more powerful than either alone.

Atefeh Abdolmanafi is a researcher with a master’s degree in physics and a Ph.D. in computer science, specializing in medical image analysis, which she obtained from Université du Québec, Montreal, QC, Canada, in 2018. She has made advancements in cardiovascular imaging, with a focus on Intravascular Optical Coherence Tomography. Currently based at the University of Michigan’s Department of Radiology in Ann Arbor, MI, USA, Dr. Abdolmanafi specializes in color flow ultrasound technologies, driving innovation in medical imaging. Committed to interdisciplinary excellence, she blends art and science to inspire creativity and enhance healthcare solutions.

Last month, RSNA staff and editorial board members gathered at the RSNA offices in Oak Brook, IL for our yearly Radiology: Artificial Intelligence Deputy Editor Retreat.

A retreat from the day-to-day

With an editorial board that spans time zones and subspecialties, most journal operations rely on emails and zoom calls scheduled in a narrow band of overlapping work hours. Journal responsibilities also compete with the research and clinical commitments of our Deputy Editors and Trainee Editorial Board members. So, for 51 weeks of the year, we focus on the essential: making sure that authors get fair decisions as quickly as possible.

However, this leaves little time for the questions that determine the long-term success of the journal. How do we stay relevant? What challenges are we facing? Are there opportunities we’re missing? Our hybrid, day-long meeting offered the necessary fuel to answer these questions: a return to the ideals that originally inspired the journal, time to brainstorm and explore ideas, and, more practically, an ample supply of coffee, tea, and snacks for in-person attendees.

What challenges is the journal facing?

Prioritizing the most important content

All readers want high-quality, useful content, but what exactly does that mean for Radiology: Artificial Intelligence? Beyond citations and impact factor, we’re looking at the papers that get the most views and downloads, often reviews, data resource papers, and practical guidelines. Understanding these trends will help us prioritize the most important submissions for publication.

Taking a more active leadership role

Academic publishing often moves slowly by design: careful deliberation is what allows us to ensure that we are only publishing the best science. However, excessive deliberation can drive a journal into obscurity. We hope to learn from and collaborate with industry leaders.

Keeping up with new technologies

AI is a rapidly developing field. As a journal, we’re trying to find the balance between seizing opportunity and losing sight of our core principles. Is there a way for LLMs to make the peer review process fairer and faster without replacing human reviewers? We think so, and we’re excited to explore options in the coming months.

Promoting our work

Finally, we were joined by members of RSNA’s marketing team to learn how they promote content from all of the RSNA journals. They shared their excitement about our LinkedIn page, which we began as a result of discussions at last year’s retreat, and taught us some social media tips. We expect to expand our LinkedIn content in the coming months by creating more human-focused content in addition to our posts about published articles.

Now, what?

Since the meeting, attendees have been digesting the ideas we generated. Specific plans are being organized; further data is being collected. The work will continue as long as we remember what it’s all about: being an active participant in important conversations in the field, serving authors, and ultimately improving patient care.

Madeline (Maddy) Cappelloni earned her PhD in Biomedical Engineering from the University of Rochester. She has worked as a Developmental Editor, working directly with scientists to improve the quality of their academic manuscripts, and an undergraduate writing instructor. She now serves as a Scientific Deputy Editor for Radiology: Artificial Intelligence.

Most people are familiar with “deepfakes,” AI-generated, synthetic images that blur the line between reality and fiction. While these deepfakes are often viewed as internet gimmicks or sources of misinformation, the same technology is also being applied for useful applications in healthcare. In fact, synthetic medical images are being increasingly used for clinical AI development.

However, to become a standard part of the AI development toolbox, we have to understand when and how synthetic imaging data should be used.

Problems with Real World Data

Datasets are often shaped by how the data was collected. Importantly, since not all patients are eligible to participate in research studies, there can be gaps in the dataset. For example, if a study is recruiting individuals using strict inclusion criteria, this may result in a very limited dataset that does not include a variety of patient demographics or disease outcomes. A well-known example is the National Lung Screening Trial (NLST), which enrolled approximately 53,000 participants across 33 medical centers in the United States between 2002 and 2004. Even at this scale, the NLST cohort is limited by its strict inclusion criteria. Specifically, NLST is largely composed of individuals with a heavy smoking history and those who identify as White (91%), as well as having low rates of malignancy (3.7%). Consequently, this dataset alone does not fully capture the range of lung cancer presentations seen in routine clinical practice.

Other challenges with real world imaging data include not having access to ground truths for difficult and time-consuming clinical tasks and the limited availability of certain medical imaging types, such as specific types of MRI. More broadly, rare disease and severe cases are often underrepresented in real-world imaging datasets because they occur infrequently or are less likely to be represented by inclusion criteria, which tend to exclude patients with complex medical conditions or advanced disease. While combining multiple datasets can improve variability, underlying recruitment biases may persist, leaving critical gaps in real data availability.

The Promise of Synthetic Medical Images

Recent work has begun to explore applications of synthetic imaging data. One study used synthetic medical images to create ground truth images by generating synthetic lung images to improve outlining mucus plugs, which is extremely time-consuming and challenging to segment. In another study, authors generated synthetic brain MRI images from CT images to improve radiation therapy set-up. Additional applications of synthetic imaging data include increasing dataset size, which is important for advanced data-hungry AI models, and filling in the gaps of real-world images by generating synthetic images for rare diseases and severe disease stages.

Yet, as synthetic data becomes more widely adopted, some important questions remain on their role in clinical AI: when should it be used, and are there limits to how far it can replace real-world imaging data?

As a way to determine the reliability of synthetic medical images, one study assessed the ability of radiologists to differentiate real world from synthetic medical images. This study highlighted the risk of synthetic images, which need to be recognized to allow for successful inclusion in AI clinical models.

Concerns of Synthetic Medical Images

Although synthetic data offers clear advantages for clinical AI applications, there are some areas of caution. First, depending on how the synthetic medical images are generated, they can incorporate false or exaggerated disease representations. Another potential problem is hallucinations, where the synthetic medical images may contain impossible anatomy or unrealistic disease representations.

The ultimate goal for clinical AI models is to accurately detect disease and predict outcomes to improve patient care. Synthetic imaging data can help to strengthen real-world imaging data, but real-world imaging data should remain the essential test to confirm clinical utility. These synthetic images should also be evaluated to assess the realism and overall trustworthiness.

Take this quiz developed by Tordjman et al to find out if you can tell which medical images are real and which are AI-generated: https://noneedanick.github.io/DeepFakeXRay/

Kalysta Makimoto PhD is a Postdoctoral Research Fellow in the Department of Radiology at Brigham and Women’s Hospital and Harvard Medical School. She is currently a member of the Trainee Editorial Board for Radiology: Artificial Intelligence. She earned her PhD in Biomedical Physics at Toronto Metropolitan University. Her research interests include machine learning and deep learning applications of chest CT images for lung disease.

Axel Masquelin is a NCI K00 Postdoctoral Research Fellow at Brigham and Women’s Hospital and Harvard Medical School. He earned his PhD in Bioengineering from the University of Vermont where he focused on applied machine learning for lung cancer detection and data-scarce environment. His research interests include machine learning, deep learning, and quantitative computed tomography for lung disease.

Ask anyone building a clinical language model how to make it safer, and the answers come back fast and familiar. Use a bigger model. Give it a longer context window. Add retrieval mechanisms so it can ground its answers in real sources. Let it reason in multiple smaller steps instead of one shot. Have it generate several answers and vote on the best one. Each of these is a way of scaling something, and each carries the same unspoken promise: a more accurate system is a safer one.

It’s an intuitive idea. It’s also the kind of idea worth distrusting precisely because it is so intuitive. However, in radiology, how often a tool is right and what happens when it is wrong are different questions. Nothing guarantees that a generally more accurate system doesn’t make worse mistakes than a less accurate system.

So, my colleagues and I checked. We evaluated 34 language models across six deployment conditions, the usual ways a system gets scaled in practice. The models were tested on a 200-question radiology benchmark where every possible answer carried a clinician-assigned safety label. This evaluation let us consider accuracy and safety as two separate dials.

What “safety” even means here

Once you stop treating every wrong answer the same way, you have to say what makes one wrong answer more or less safe than another. In our study, we used three error classifications, each with their own safety implications. A high-risk error is a wrong answer whose alternative could plausibly cause harm if acted upon, the kind of mistake that changes management. A contradiction is a wrong answer that directly disagrees with evidence supplied to the model in the same prompt: the system was given good information and chose poorly. Dangerous overconfidence is the combination that worries clinicians most: a high-risk wrong answer asserted with high confidence, indistinguishable in tone from a correct one. These labels were not mutually exclusive, and we did not collapse them into a single safety score so that each failure mode stayed visible.

The safety and accuracy dials are not connected

When we measured different error types separately, accuracy and safety clearly came apart. The clearest example in our study was the difference between a model using standard retrieval and the same model using a more novel, multi-step, agentic retrieval. While agentic retrieval improved accuracy, it also raised dangerous overconfidence. If you were measuring only accuracy, you would have logged the agentic system as progress and moved on without taking cautionary measures.

Other tools to improve accuracy fared no better in our safety assessment. Letting models generate many answers and vote on the best barely improved safety. In fact, ensembles of strong models brought a new failure mode: synchronized failure, where every model lands on the same wrong answer. A unanimous wrong answer is more persuasive than a single model’s mistake, not less, making ensemble errors a major safety concern.

Even model size, the most basic scaling lever, mostly bought a better starting point. In our analysis, deployment condition mattered much more than model family or parameter count; if you don’t get the deployment right, a bigger model won’t help.

Is it even possible to improve safety?

One thing did move the safety dial, and it moved it hard. When models were given concise, clinician-written supporting context for each question, accuracy rose from 73% to 94%, and every safety measure improved alongside it: high-risk errors fell from 12% to 3%, dangerous overconfidence from 8% to 2%. All 34 models improved. Nothing else we tried came close.

Why did this work? Clinician-written supporting context helped not only because of what it added but because of what it removed: noise, partial relevance, competing claims, and off-target passages. Even though clinicians and retrieval systems produce seemingly similar text, retrieval systems cannot easily determine what context is relevant and what is distracting.

To see how much the removal of irrelevant information mattered, we tested a version where just one mildly distracting sentence was added to the otherwise correct clinical explanation. Safety degraded measurably. The high-risk error rate creeping up while the headline number, accuracy, barely moved. The model was not catastrophically misled. It was just slightly less safe.

The implications of this central finding are uncomfortable. The lever that actually moves safety is the quality of the supporting context supplied to the model, and it is the thing we do not yet have easy tools to scale.

Reading the right dial

Our findings encourage more thorough pre-deployment evaluation. Accuracy on a benchmark is both necessary and nowhere near sufficient. A clinical language model should also be measured on how dangerous its residual errors are, whether it contradicts its own supplied evidence, how confidently it asserts the answers it gets wrong, and, for any system that stacks models together, whether its failures are correlated.

We also need to start questioning the use of model confidence as a safety filter. While suppressing low-confidence outputs and trusting the rest seems like a reasonable assumption, that assumption does not guarantee safety. In our study, models were almost as confident on their high-risk wrong answers as on their correct ones.

Finally, evaluating safety requires testing systems under the evidence they will actually encounter. A model that appears to be safe on carefully curated input data has presented its best case, not its deployment case. The gap between ideal performance and real performance is where patient harm can happen.

The reflex

None of this is an argument against scaling. Bigger models, retrieval, and agentic reasoning are genuinely useful. However, we don’t automatically get safer models when we improve accuracy. Safety is a separate, independent property, and it has to be measured on its own terms.

The question was never only whether the system is right more often. It is whether the answers it gets wrong are ones a radiologist can live with.

Soroosh Tayebi Arasteh is an AI researcher and lecturer at RWTH Aachen University. He holds two PhDs in computer science and theoretical medicine and was previously a postdoctoral scholar at Stanford University and FAU Erlangen-Nürnberg. His research focuses on grounded, privacy-aware, and multimodal learning systems for medicine. He serves on the editorial boards of Communications Medicine and European Radiology Experimental and the 2025–2027 trainee editorial board of Radiology: Artificial Intelligence.

X: @starasteh

LinkedIn: https://www.linkedin.com/in/tayebiarasteh/

BlueSky: @tayebiarasteh.bsky.social

Artificial intelligence in healthcare is beginning to move from single-task tools toward systems that can orchestrate multistep workflows. In this context, agentic AI systems can work toward a defined goal by retrieving information, using external tools, maintaining task context, and adapting subsequent steps based on intermediate results.

A conventional radiology AI tool might detect a pulmonary nodule, segment a lesion, prioritize a case, or draft report text. An agentic system might retrieve prior CT examinations, check follow-up guidance, draft a recommendation, prepare a message to the referring clinician, and verify whether follow-up imaging has been scheduled. These steps may seem like a natural progression from a technical perspective, but they are clinically distinct acts, each with their own consequences. It’s not enough to evaluate what agentic AI can do, we also have to decide what it should be allowed to do.

Radiology as a Focus in a Broader Conversation

While many fields are reckoning with how much power we should really give agents, radiology is a particularly natural setting for this discussion. The success of radiology depends on coordinated information workflows; agents could be immensely helpful coordinators. While images are central, they rarely stand alone. Prior examinations, clinical history, laboratory values, operative notes, pathology results, treatment timelines, and follow-up recommendations all shape interpretation and downstream care.

The early value of agentic AI in radiology may be in supporting, rather than automating diagnosis. A system that retrieves relevant priors, organizes clinical context, checks consistency between findings and impressions, or prepares follow-up communication could support radiologists in making their own diagnoses. While early efforts have shown the feasibility of such systems, implementation in routine radiology practice is still not a reality.

Defining Levels of Agency

A useful way to decide how agentic AI can safely help radiologists is to ask how much agency the system is granted. Agency can be categorized into four levels (below): observing information, recommending actions, drafting outputs, and executing actions. Risk and oversight requirements increase at each step, so the evidence and governance required for an agentic system should match the level of authority it is has.

A system with minimal agency can observe by retrieving prior imaging reports, summarizing relevant history, or listing laboratory values. Even this level requires caution. Using an agent-generated summary may be efficient but unsafe if the summary is inaccurate. Users should be able to inspect the source of any clinical statement.

At the next level, an agentic system can recommend. It might suggest follow-up imaging, flag an inconsistency between findings and impressions, or propose a protocol based on the clinical indication. While the clinician makes the final decision, recommendations can easily influence human judgment, especially in high-volume settings. They should therefore be transparent, specific, and easy to accept, edit, or reject.

A system can draft a follow-up message, report impression, tumor board summary, or protocol note if granted even more agency. Drafting can improve consistency and reduce administrative burden, but it can also constrain the final human decision. AI-generated drafts should be labeled, editable, and unsigned until reviewed by an accountable clinician.

A system given full agency can execute an action. Here, the system actually acts on a decision by sending a message, scheduling an examination, placing an order, updating documentation, or changing worklist priority. This is the point at which agentic AI enters the clinical record or care pathway. Execution should require the strongest safeguards, including role-based permissions, audit logs, institutional policy, escalation pathways, and human authorization for higher-risk actions.

Where Early Agents Can Safely Help

The safest uses are likely to be as observers and recommenders, where their contributions are narrow, auditable, and reversible, especially as current evidence supporting the use of agents remains limited. A recent scoping review of agentic AI in healthcare identified only a small number of relevant studies across emergency medicine, oncology, radiology, and rehabilitation, with most remaining exploratory or limited in clinical validation.

Despite limited evidence, there are many possible applications. In follow-up management, an agentic system could identify radiology reports containing follow-up recommendations and prepare cases for human review. In protocoling, it could gather the indication, renal function, allergy history, pregnancy status when relevant, and local guidance before suggesting a protocol. In tumor board preparation, it could assemble imaging timelines, prior measurements, pathology results, and treatment dates. In report quality assurance, it could flag laterality mismatches, missing comparison statements, or inconsistencies between the findings and impressions.

These applications address real friction in radiology practice while keeping the radiologist or clinical team in full control of the final action. They are not risk-free, but they are more appropriate starting points than independent diagnostic or treatment decisions.

Where Agents Should Not Yet Act Independently

Some boundaries should be clear. Agentic AI systems should not independently sign radiology reports, communicate a new cancer diagnosis to a patient, place follow-up imaging orders without approval, modify the medical record without attribution, or silently override radiologist judgment. These tasks all carry significant clinical consequences and thus require equally strong evidence and oversight.

Importantly, agentic systems can fail differently from traditional prediction models. A classification model may be wrong at a single point. An agentic system can create a cascade of errors. For example, if an agent misses a prior breast MRI showing interval growth of an enhancing lesion, it may incorrectly summarize the finding as stable, generate an inappropriate follow-up recommendation, and carry the error into a draft report or message to the referring clinician. The rigorous evaluation needed to support more advanced agents should therefore examine the full workflow, including retrieval, tool selection, intermediate outputs, source attribution, human review, and downstream consequences.

Governance Before Convenience

As radiology departments face rising imaging volumes, staffing constraints, and communication demands, agentic AI systems will become increasingly attractive for automating routine work. Yet convenience should not determine permission, and institutions should proactively define what these systems may access, recommend, draft, or execute, with clear requirements for responsibility, human approval, logging, evidence inspection, error reporting, and post-deployment monitoring. Autonomous AI agents may require more adaptive oversight than static device-centered regulatory models, a distinction that is especially relevant in radiology when systems move from flagging findings to retrieving data, selecting tools, drafting communication, and initiating follow-up.

The Role of Trainees

Radiology trainees will likely use agentic AI in their careers, whether by choice or necessity. They should learn not only how these systems work but also how they fail. They should understand the difference between a useful summary and a reliable conclusion, between a draft and a signed report, and between a recommendation and an accountable clinical decision.

This isn’t just a technical skill. It’s a professional skill. Trainees will need to know when to follow an agent’s output, when to question it, and when to ignore it. Working with agentic AI also shouldn’t weaken core interpretive skills. Ideally, it should make verification, source checking, and awareness of workflow consequences more explicit. The goal should not be blind trust or reflexive skepticism; the goal is calibrated use.

Conclusion

Agentic AI’s near-term value in radiology may lie less in autonomous diagnosis than in coordinating complex, information-rich workflows. Yet, this value is inseparable from risk. Once a system can coordinate actions across clinical systems, its errors can propagate. Radiology should therefore grant agency to AI gradually, with evidence, oversight, and accountability that match the consequences of each action. The question is no longer simply whether agentic AI should enter the reading room, but what other doors it should be allowed to open.

Dr. Tianyu Zhang is a researcher at the Netherlands Cancer Institute, scientific staff at Radboud University Medical Center, a senior researcher at Maastro Clinic, an Editorial Board Member for BMC Medicine and European Radiology Experimental, and a Trainee Editorial Board Member for Radiology: Artificial Intelligence. Since 2024, he has served for three consecutive years (2024–2026) as Chair of the MICCAI International Deep-Breath Workshop Series. His research focuses on medical image analysis and natural language processing, with a particular emphasis on multimodal AI model development and its integration into clinical workflows.

Interventional radiology is the branch of radiology in which physicians use imaging guidance to perform minimally invasive, targeted diagnostic procedures and treatments, i.e. threading catheters, needles, and other instruments through the body’s tissues. These procedures predominately take place in an imaging-equipped procedure room that interventional radiologists, by long habit, call the angio suite, even though the work done there now reaches well beyond blood-vessel imaging (angiography) that the term originally described.

What sets IR apart from the conventional diagnostic reading room is that here, nearly every image is paired with direct, real-time clinical action. Consider a patient with liver cancer treated by embolization: the interventional radiologist uses live imaging to judge whether the patient is a good candidate, to choose which branch of the hepatic artery to catheterize, and to decide when the tumor’s blood supply has been adequately blocked — and then, weeks later, uses follow-up imaging to determine whether it worked. Image, decision, action, and measurable outcome are bound together in one loop. That loop is what makes IR such fertile ground for AI, and it also creates challenges that diagnostic AI never has to confront. Three domains of IR are seeing the most active AI development, and they map neatly onto the arc of a procedure: before, during, and after.

Before the Procedure: Planning

Arguably the most mature applications of AI in IR do their work before the patient is ever on the table. At first glance this can look like ordinary diagnostic imaging. The inputs are generally cross-sectional scans, and they could in principle be analyzed in a reading room. The key difference lies in the question being asked. Diagnostic radiology asks, “what is this?”. Pre-procedural IR analysis goes further and asks, “how should we treat it, what is the best approach, and is it likely to respond?”, questions that only make sense in the context of a specific planned intervention.

A good example is radioembolization with yttrium-90, or Y-90, a treatment in which millions of microscopic radioactive beads are delivered through a catheter directly into the blood vessels feeding a liver tumor, irradiating it from the inside while largely sparing healthy tissue. Planning a Y-90 treatment well means predicting, in advance, where those beads will travel and how the tumor will respond. This is where AI has made some inroads. Researchers have used deep learning to forecast, from a pre-treatment scan, where the radioactive dose will ultimately settle; to predict which parts of a tumor will respond; and to estimate, from routine pre-treatment MRI alone, how likely a tumor is to respond at all. The common thread is practical: better patient selection, and a quantitative preview of the expected result, before any irreversible decision is made.

During the Procedure: Real-Time Assistance

Once the patient is on the table, the role of AI changes. This is the domain where IR diverges most sharply from diagnostic radiology, because the imaging, and any AIs using the imaging, must keep pace with a procedure unfolding in real time, something a reading-room tool never has to do. Three distinct opportunities stand out: making the live images clearer, helping the operator navigate, and smoothing the logistics around the procedure.

Sharper images.

Certain imaging modes used heavily in IR, namely fluoroscopy and digital subtraction angiography (DSA) are especially prone to visual noise and patient movement. Both fluoroscopy and DSA are forms of moving X-ray techniques that let an operator watch as catheters and other tools of IR move. Image quality with these X-ray techniques is most commonly improved by increasing the radiation dose. Thus, cleaning up the image with AI instead is a direct way to protect the patient from radiation. Luo and colleagues built a denoising network that sharpens low-dose fluoroscopy without blurring the catheter edges operators depend on. Tang and colleagues took a different route, training a model to reconstruct the in-between frames of a video so the X-ray can be pulsed less often; in reader studies, the AI-filled sequences were difficult to tell apart from the originals.

Better navigation.

Guiding an instrument to exactly the right spot is harder than it sounds: anatomy varies from patient to patient, targets can be small, and the structures an operator most wants to avoid are not always obvious. Needle and catheter tracking, automatic labeling of vascular landmarks, and the fusion of detailed pre-procedure scans onto live fluoroscopy are increasingly treated as machine-learning problems rather than purely geometric ones. Sato and colleagues, for instance, paired an AI path-planner with a robotic biopsy system and found it could reliably locate targets and plan trajectories comparable to those of experienced operators. Established image-guidance platforms are now widely used for ablations and biopsies in the liver, lung, and kidney, with vendors steadily layering AI onto the underlying software. Most of these tools remain early in their clinical maturity, and the regulatory path for anything approaching closed-loop guidance, that is, a system that acts on its own conclusions, as opposed to an open-loop system that only advises a human who stays in control, is appropriately cautious.

Smoother logistics.

IR is unusually hard to schedule. Unlike a clinic that books its patients well in advance, or diagnostic radiology scanners where high-quality, whole-body scans can often be done in seconds, an IR service must absorb a constant collision of planned outpatient cases, inpatient add-ons, and genuine emergencies all competing for the same rooms, staff, and equipment. Constraint-based scheduling tools, with interactive overrides so a human always has the final say, have shown quiet value here, helping departments avoid delays and treat more patients without ever touching a clinical decision. This is a category likely to grow, precisely because it lets AI add value while staying clear of medical judgment. Plus, similar tools are already running in diagnostic radiology operations.

After the Procedure: Did It Work?

Following intervention, the central clinical question becomes whether it achieved its intended effect. Much of the research here focuses on answering that question as early as possible. Conventional response criteria were designed for human readers comparing scans taken weeks or months apart. A more sensitive approach is radiomics: the practice of extracting large numbers of quantitative features and measures of texture, shape, and intensity pattern from a medical image, many of which are too subtle for the eye to register. These features can surface signs of response, or of progression, earlier than visual interpretation alone.

One of the most-studied examples is the prediction of response to TACE, transarterial chemoembolization, a common liver-cancer treatment that delivers chemotherapy directly into a tumor’s blood supply and then blocks that supply to trap the drug in place. Several groups have shown that a scan taken before TACE already carries signals that predict how well it will work. Peng and colleagues combined CT radiomics with deep learning across three centers and reported strong accuracy in predicting which patients would respond. Liu and colleagues showed that a similar predictive signal can be drawn from contrast-enhanced ultrasound, a more widely available imaging modality. And. Bartnik and colleagues found something genuinely surprising: features taken from organs other than the tumor such as the spleen, the pancreas, and the inferior vena cava were more predictive of long-term outcome than features of the tumor itself, a result that productively complicates the usual tumor-centered view of radiomics.

A newer direction is delta radiomics, in which the same features are measured on two scans, before and after treatment, and it is the change between them that serves as the predictive signal. Jin and colleagues applied this idea to radiation therapy for liver lesions, using the change in features across treatment to predict local control. Carrying the approach into the embolization and ablation literature is an active area of work, built on a simple intuition: the difference between two images of the same patient may carry information that neither image conveys on its own.

The objective here is not to supplant the radiologist’s interpretive role, but to provide clinicians with a quantitative complementary assessment in the early post-procedural window.

Why AI for IR Has Lagged

If the opportunity is this clear, why does AI in IR generally trail AI in diagnostic radiology? The answer is largely structural. Any given institution performs far fewer interventional procedures than it acquires chest X-rays or mammograms, so IR datasets are smaller to begin with, and they are generally concentrated at a handful of high-volume centers with unique case mixes that may not generalize elsewhere. The procedures themselves vary from operator to operator, introducing a kind of label noise that is rarely modeled explicitly. IR imaging is also far less standardized than diagnostic imaging, which makes pooling data across sites difficult even when the data exists. And, prospective validation, testing a model on future patients rather than on historical scans, has lagged behind the steady output of predictive-modeling papers, with the consequence that many of the impressive accuracy figures in the literature are likely to look more modest once tested in the real world.

The Real Frontier

To summarize the current state of AI in IR, I borrow a framing previously articulated by our editor: interventional radiologists need not become machine-learning engineers, but they must understand enough about these systems to oversee the tools entering angio suites and to push back substantively when they fail to deliver clinical value. The opportunity is real. The clinical loop in IR of imaging to intervention to observed outcome is precisely the type of feedback structure that machine learning methods are well positioned to exploit. The work required to translate this opportunity into validated tools that meaningfully alter patient outcomes is not glamorous, and progress is unevenly distributed across the field, but it is occurring. In my own training, the most meaningful progress of AI in IR I have seen has come not from any single clever model, but from rooms both physical and virtual where interventional radiologists and computer scientists sit together and argue productively about the same problem. That, more than any individual algorithm, is the quiet frontier worth watching: not a new model, but a new way of working together.

Dr. Tej Mehta is a PGY-6 Interventional Radiology-Integrated resident at The Johns Hopkins Hospital. He is the Chief Resident of Interventional Radiology and the Residency Research Director. His research primarily focuses on developing machine learning models for the prediction of chemoembolization outcomes in hepatocellular carcinoma; he additionally has research interests focused on developing machine learning models for translational outcomes of other locoregional therapies. His work is supported by multiple grants and awards from The Johns Hopkins Hospital, ARRS, SIR, and Applied Radiology. He is a current member of the Trainee Editorial Board for Radiology: Artificial Intelligence.

While I was reading images on call last Tuesday, the large vessel occlusion (LVO) detection algorithm flagged a study on the worklist. I spent an extra five minutes looking for what it saw. I never found it. I moved on with a nagging uncertainty, wondering whether I had missed an occlusion or whether the algorithm had overcalled one. But this is not a story about a bad tool. This is the story about why the study disappeared from my worklist but not from my mind.

What I felt at the workstation has a name: cognitive load. Cognitive load determines whether a tool improves or worsens the workday and is not always captured by more quantifiable metrics like accuracy and speed. It is not directly observable, and it depends on the case mix, the radiologist’s experience, the workstation, the time of day, and the rest of the worklist.

Cognitive Load in Clinical Practice

The LVO tool is a useful place to start. At my institution, the algorithm flags a study without showing me what it saw — no slice number, no vessel territory, and no overlay due to current regulatory constraints, despite the vendor being able to localize the LVO. The algorithm labels the study as “abnormal”, but it’s my job to figure out why. When I can’t, I am left unsatisfied and drained because there are only two possibilities: either I missed something I should’ve caught or the algorithm wasted my time. In the former case, I worry I’ve hurt a patient and doubt my own skills. In the latter, the algorithm is assigning me extra work during an already busy shift. I can’t convince myself of either possibility, so I feel the weight of both.

The inverse case (of a tool that lessens cognitive load) is just as revealing. A generative tool that drafts the impression section of reports may or may not make a radiologist meaningfully faster according to the existing literature. However, summarizing a study, anticipating the referrer’s questions, and precisely framing uncertainty into a helpful impression are tasks that seem to carry the most cognitive load in my experience. Impression generation tools absorb some of the load. They help ensure I haven’t forgotten to mention a critical finding or inadvertently included a dictation error that changes what I mean. These tools may not necessarily make me faster, but I know I feel less depleted at five o’clock.

The tools that have become comfortably integrated into my workflow are the ones that subtract cognitive load, sometimes without improving efficiency. The tools that I find myself resistant to are the ones that add cognitive load even if they move raw efficiency metrics in the right direction. Cognitive load is a tax, and many modern radiology AI tools inadvertently pay it because our procurement frameworks were not designed to measure it.

Measuring the Cognitive Tax

None of this means accuracy metrics are dispensable, or that subjective experience should replace diagnostic performance or efficiency. A tool that is wrong but has an excellent user experience is still a bad tool. Rather, measuring efficiency without human-factors measurement fails to capture the actual value of a tool that inherently depends on a human using it.

There is encouraging movement here. Multi-society guidance already recognizes that AI evaluation should extend beyond clinical accuracy and efficiency, and frameworks such as DECIDE-AI ask us to report human factors, learning curves, and errors in use. Adjacent fields are further along; for example, ambient clinical scribes are showing that objective time savings may be modest while the burden of documentation experienced by clinicians changes dramatically. Imperfect but validated instruments exist, including NASA-TLX, eye-tracking, and longitudinal use data, but they require deliberate study design and a willingness to treat human factors as a primary endpoint rather than an afterthought. It’s easy to count seconds saved per study, so we count seconds.

Cognitive load is hard to measure, but it is not unmeasurable. To assess new AI tools, we can start with a simple question: six months after deployment, do you still want to use it?

Satyam Ghodasara, MD (@_Satyam_) is associate staff of neuroradiology in the Diagnostics Institute at the Cleveland Clinic Foundation and serves on the Radiology: Artificial Intelligence Trainee Editorial Board. His research applies informatics and machine learning to optimize clinical workflows while supporting safe, practical deployment of AI in routine care and operational innovations that enhance patient-centered radiology services.

Radiology AI begins with scanners, contrast agents, semiconductors, data centers, and electricity. All of these depend on systems outside the hospital. Some are industrial, others logistical, and many are increasingly shaped by geopolitics. Claiming that geopolitical shocks have broken radiology AI would go too far. Nonetheless, radiology in general has already shown that it is vulnerable to global disruption and AI only adds further dependencies. As AI becomes more integrated into the practice of radiology, awareness of these underlying dependencies may become just as important as advances in the models themselves.

Global Disruptions Reach the Radiology Reading Room

Radiology has experienced how external shocks can shape clinical practice. During the global semiconductor shortage from 2020 to 2023, disruptions in East Asia and rising demand in the United States and Europe slowed the production of imaging systems. Vendors such as Siemens Healthineers and GE HealthCare reported delays in CT and MRI equipment delivery, which meant that hospitals had to keep older machines in service for longer and postpone planned upgrades that could improve imaging throughput and diagnostic workflows. In some cases, older systems may also limit compatibility with newer applications, including AI-based tools, while becoming more vulnerable to technical failure and maintenance challenges over time. In May 2021, a ransomware attack on the Irish Health Service Executive forced a nationwide shutdown of IT systems. Radiology networks were affected, imaging appointments were canceled, and staff had to revert to manual workflows. A year later, the 2022 energy crisis in Europe followed the Russia-Ukraine War and introduced new uncertainty about energy supply and costs for hospitals and data centers, threatening to have downstream effects on clinical practice.

AI Adds New Layers of Dependence

Radiology depends on infrastructure it does not control. AI extends these dependencies by adding new layers rather than removing old ones. Many of the supply infrastructures that support AI are highly concentrated, with only a small number of production sites or suppliers. This increases the risk that a single disruption can have global effects. Recent developments make this vulnerability even clearer.

In early 2026, conflict in the Middle East disrupted gas exports from Qatar, one of the world’s key suppliers of helium. The initial concern was industrial rather than medical: helium is required for semiconductor manufacturing and for cooling superconducting systems. However, both semiconductors and cooling are essential for the GPUs and high-performance computing infrastructure that support modern AI systems. MRI scanners also rely on helium, illustrating how disruptions in industrial supply chains can affect both the computational systems behind radiology AI and imaging infrastructure. Nowhere is the dependence on helium from Qatar more acute than in South Korea, home to some of the world’s leading Dynamic Random Access Memory and high-bandwidth memory producers. In 2025, South Korea imported approximately 64.7% of its helium from Qatar, the highest dependency among major chip-making nations. Semiconductor production, data center operations, and other industrial uses compete with medical demand.

Supply chain interruptions create a chain of effects. If helium supply falls, chip production can slow. If chip production slows, access to GPUs becomes more limited. If GPUs are limited, hospitals and vendors cannot easily expand or maintain AI systems. At the same time, MRI systems become more difficult and more expensive to operate. A single upstream disruption therefore can affect both computation and imaging equipment.

Who Gets Access to AI Infrastructure When Technology Becomes Constrained?

The broader context reinforces this risk. Artificial intelligence is no longer developing in a fully global system. Competition over chips, data centers, and data sovereignty is fragmenting the landscape. Compute infrastructure is increasingly treated as a strategic asset, which means access is no longer guaranteed. In a constrained situation, resources are prioritized, but radiology AI is unlikely to be at the top of that hierarchy.

Hospitals rely on local servers, vendor platforms, and cloud systems to run AI tools, and many of these require continuous access to high performance computing. If compute becomes scarce, resources may be redirected toward core hospital systems and essential clinical infrastructure, while research environments, experimental deployments, and less critical AI applications may receive reduced access. Issues also extend beyond the hospital administration itself. Export controls and sanctions can restrict access to the processors required to run modern data centers. Since 2022, advanced AI chips have been subject to export controls, which has limited where and how they can be deployed. A data center is therefore not defined only by its design, but by what equipment it can obtain. When access to components becomes political, access to compute follows the same pattern. As a result, the systems that depend on that compute, including radiology AI, inherit those constraints.

AI Infrastructure Constraints Can Impact Clinical Practice

Radiologists are increasingly used to working with AI-supported workflows. These include triage systems, automated measurements, and detection support. Such tools do not replace expertise, but they do shape it. They influence how images are read and how time is managed. If these systems are suddenly unavailable, the transition is not seamless.

The concern is not that radiologists lose their core skills. The concern is that training and daily practice shift toward AI-supported environments, which may reduce exposure to fully manual workflows. In a constrained situation, departments may need to rely more on traditional methods, even when the workforce has become less accustomed to them.

Adaptation Remains Possible

There is still hope. Radiology has adapted to constraints before. It has managed shortages, reorganized workflows, and maintained clinical function under pressure.

AI is one part of this system. In stable conditions, AI appears scalable and widely available. Under stress, access may become uneven, with some systems being able to benefit from the newest AI tools and others not. While AI’s expanding capabilities will continue to matter, its impact for real patients will also be shaped by the systems that support radiology AI. Those systems must also remain adaptable to new constraints.

Dr. Jawed Nawabi is a neuroradiology specialist and assistant professor at Charité – Universitätsmedizin Berlin, where he serves as campus lead for one of Charité’s three major clinical campuses. At Charité, he heads the Neuroradiology AI Imaging Lab as well as the institute’s digital transformation division, where he oversees the implementation and governance of AI systems in clinical practice. His research focuses on the development, evaluation, and clinical integration of large language models and deep learning–based imaging models in radiology. He holds a Master’s degree in AI in Healthcare and is an alumnus of the Digital Health Clinician Scientist Program. In his current work, he leads projects on digital twins for neuro-oncological tumor boards, exploring how AI-driven models can support multidisciplinary clinical decision-making. He is a current member of the Trainee Editorial Board for Radiology: Artificial Intelligence.

Have you ever walked up to a door, confidently pushed it, and bounced right off? And then, slightly embarrassed, pulled it open? If so, congratulations: you’ve met a Norman door.

Named after cognitive scientist Don Norman (author of The Design of Everyday Things), a Norman door is any door that lies to you about how it works. A vertical handle signals “pull,” even when the door only opens by pushing. A flat metal plate screams “push,” even when you need to pull. The design misleads you, and then you feel stupid.

Here’s the thing: it’s not you. It’s bad design.

From Doors to Diagnostics

Design failures are funny when the stakes are low. Less so when they sit between a radiologist and a patient’s diagnosis. Anyone who has ever worked in a hospital knows that clinical IT is full of design flaws: electronic health record (EHR) systems that bury critical labs three clicks deep; PACS software that completely abandons standard keyboard conventions like ‘Ctrl+Z,’ turning every minor misclick into a major detour; voice recognition software that loses window focus the second you touch an image, translating your dictation into chaotic keystrokes across your imaging monitors.

Design is more than a matter of aesthetics or convenience. Decades of experience with EHRs - built primarily as billing engines rather than clinical tools - have proven that software detached from the clinical workflow burdens clinicians and degrades care. By prioritizing administrative requirements over clinical usability, these systems have stifled efficiency, fueled clinician burnout, and introduced risks that directly endanger patient lives.

If we are not careful, radiology AI will head down the same road. For more than a decade, the radiology AI community has been obsessed with features of the model: bigger datasets, novel architectures, higher accuracy, stricter external validation. These things matter. But why have we overlooked the humans who actually have to use these models? A scoping review found only six papers on interface design in radiology AI in a decade (Gill et al. 2025), against more than 500 radiology AI papers in only five years (Kelly et al. 2022). The numbers speak for themselves.

To understand what’s missing, it helps to separate two terms. The user interface (UI) is what the radiologist sees and touches: the bounding box, the segmentation overlay, the alert message. The user experience (UX) is the overall journey: how the tool fits into the workflow, how AI findings are surfaced, and whether the system feels like an asset or an obstacle. You can have a beautiful UI and a miserable UX, or vice versa.

Building for Someone You’ve Never Watched

Before starting my radiology training, I spent several years as a product manager for a startup that built radiology reporting software. Visionary leaders, smart engineers, a roadmap full of features. Then, one day, we organized a visit to a hospital for the product and development team to see radiologists in action. The team saw the multi-monitor setup, the keypad phones from the 2000s that rang every few minutes, the referring clinician walking in to “just look at one case.” Radiologists skipping lunch to fight a worklist that never got shorter. Chest x-rays being signed off in less than a minute.

Our crew was shocked. We had been building a product for radiologists assuming they worked in a quiet office with plenty of time to think. We had thought a few extra clicks for a structured report would be fine. Now we realized that a click that costs two seconds, performed two hundred times a day, is not a minor inconvenience but a nightmare. All the prior conversations with in-house radiologists explaining their workflow hadn’t come close to the effect of watching them work.

The experience didn’t miraculously fix every usability issue. But observing real users instilled a lasting appreciation for user-centered design.

The Model Is Only Half the Product

Picture a chest CT on the worklist: behind the scenes, the AI has flagged a possible pulmonary embolism. What happens next depends almost entirely on design decisions.

When is the AI finding shown? Are the AI annotations shown by default, or does the radiologist control when to engage via a toggle? Showing the annotations automatically risks exposing the radiologist to anchoring bias and satisfaction of search but could still be sensible in acute findings where the cost of missing a true finding is high.

How is the AI finding presented? A thick red bounding box commands attention; a subtle overlay merely hints. Confidence displayed as “85%” implies a precision that “high confidence” does not.

How do AI findings reach the report? Some reporting systems now auto-populate the radiology report with AI findings. While this implementation may improve efficiency, it shifts the radiologist’s role from active author to passive editor, potentially nudging radiologists to adoption of AI outputs without critical review.

These design decisions are independent of the model itself, yet fundamentally alter how radiologists detect, interpret and report imaging findings. The same holds true for emerging applications of large language models and vision-language models, such as automated report impressions and report drafting. In each case, the model is only half the product. The rest is design.

Designing With, Not Just For, Radiologists

User-centered design does what it says: it puts end-users at the center of every decision, from first sketch to final product. The following phases define the process.

First, research before design. The goal is to design for an exhausted resident woken up at 3 AM to rule out a brain bleed, not just a hypothetical radiologist. That means interviews, surveys, and ultimately observing real users in real reading rooms (contextual inquiries). No amount of second-hand description replaces watching the work.

Second, prototype before you build. Wireframes, mock-ups, and clickable prototypes let you test a design concept before a single line of code is written. This is something worth investing in: changing a sketch takes minutes, whereas changing a shipped product takes months, at best.

Third, test and iterate. A prototype gets put in front of real users, refined based on what you learn, and tested again. Do users understand what a button does without being told? Do they complete tasks efficiently, or get lost? Metrics like task completion time and completion rate as well as instruments like System Usability Scale turn subjective impressions into measurable data.

A caveat: Putting the user at the center does not mean building whatever users say they want. Radiologists who have spent years working around broken systems often can’t picture anything different. They will ask for a slightly better version of what they have, not the tool they actually need. Good design listens carefully and then goes further. An outside perspective, unconstrained by decade-old conventions, is often what makes the leap possible. This is where designers, product managers, and developers truly prove their worth.

Conclusion

Radiology AI risks becoming a field of Norman doors: tools built by people who have never watched a resident dictate through a shift and never seen a worklist refuse to shrink. The models are becoming remarkable, but the doors are still wrong. Fixing them doesn’t require better algorithms. It requires designers who put radiologists at the center.

Su Hwan Kim (@shk03.bsky.social) is a radiology resident at the Technical University of Munich (TUM) with a MSc degree in Health Informatics from University College London (UCL). Previously, he served as Product Manager at Smart Reporting (now Jacobian), a start-up company developing radiology reporting software. His research focuses on uncovering the key factors that enable effective human-AI interaction in radiology, operating at the dynamic intersection of AI, cognitive psychology, and UX design. He is currently a member of the Trainee Editorial Board for Radiology: Artificial Intelligence.

Like the machine-being taking on human features in Fritz Lang’s Metropolis, the new generation of AI is increasingly dressing itself in a human grammar. But the drive for these systems to maintain the illusion of certainty, combined with their lack of alignment with the constraints and habits of medicine and radiology, question how they will be included in the decision-making process.

The grammar of doubt

For at least a decade, radiologists have lived alongside machines that quantify their own uncertainty. Bounding boxes, probability scores, saliency maps: the visual grammar of computer-aided detection has trained a generation of radiologists to consider not only what an algorithm sees, but how confidently it sees it. A nodule flagged with 0.62 confidence is not the same object as a nodule flagged with 0.95, and radiologists know it. Explainability, however imperfect, has been part of our daily practice.

A new robot has entered the reading room

A new generation of AI is now arriving in the reading rooms, and it does not use the traditional grammar of explainability. Generative models are not designed to output a calibrated probability about a single finding. They are designed to produce an answer to any question with uniform composure. The aplomb is the same whether the model is right, wrong, or operating entirely outside its training distribution. Even for radiologists adept at negotiating with a probabilistic partner, this is a different kind of interlocutor, one that has been trained to dazzle.

This is not an accident of design but a consequence of how these models are evaluated. Benchmarks and multiple-choice question banks reward the confident right answer and penalize abstention. Hesitation, ambiguity, and the explicit acknowledgment of not knowing carry no value in this framework and may even be scored as failure. Yet these are precisely the marks of responsible radiological practice.

The value of doubt in radiology

Imaging accesses reality only partially; an image is the visible trace of a disease at a single moment in time. In thoracic imaging, a constellation of nodules may correspond to cancer, infection, or granulomatosis, and no imaging feature alone will arbitrate, however likely each possibility is. In neuro-oncology, the situation is starker still; for example, after radiotherapy, two diametrically opposed diagnoses, tumor progression and radionecrosis, can wear exactly the same imaging clothes.

In these situations, the radiological question is not only epistemic but decisional and asymmetric. What do we risk by being wrong in one direction or the other? Even when a malignant cause seems unlikely, the nodule must be monitored after appropriate treatment. Even when radionecrosis seems most plausible, the patient must be followed more closely, because the consequences of missing progression are catastrophic. Doubt, here, is not a limit of the radiologist’s knowledge, but an integrative part of their reasoning and expertise.

Communicating this doubt is frustrating. We would like to decide. We would like to give the referring physician a clear diagnosis. And more often than we admit, we would like to reassure rather than worry, which is human and normal.

The ideal assistant, a more experienced colleague or a resident who has worked the case carefully, is not there to tip the scales. That would make no sense. The good second reader integrates our doubt into the context of the question being asked. They do not arbitrate in our place; they enrich our uncertainty by considering it within the clinical picture. This is perhaps the most useful definition we have of what radiological AI should aspire to be, and it is precisely the opposite of what a fluent, sycophantic generative model does by default.

Knowing when we don’t know

There is a rare and dangerous scenario, one that the literature misguidedly celebrates, that deserves more suspicion than it gets. Occasionally the assistant, human or machine, surfaces a rare diagnosis that had not been considered. Academic literature and news outlets alike love this story: AI as the tireless reader who catches the zebra.

Consider the epistemic position of the radiologist at that moment. If the diagnosis had not been considered, the radiologist is, by definition, not equipped to critique the suggestion. This is exactly where we are most vulnerable to the machine’s proposal, and where both reader and machine are vulnerable to confirmation bias. Somewhere in the literature there is a similar case report to reassure us that zebras do exist. Somewhere there is a similar image that could corroborate the interpretation.

In a recent Nature Medicine World View, Leo Anthony Celi argues that medical AI must be redesigned around two virtues: curiosity, so that systems recognize when a case falls outside their training, and humility, so that they defer to a human rather than answer with false confidence. His central worry is the widening gap between confidence and competence, between systems that sound authoritative and systems that actually make sound judgments.

This is precisely what we need in those rare and dangerous moments: a system curious enough to recognize that it is operating beyond its training, humble enough to defer to the human rather than answer, and transparent enough to say so explicitly. We need someone, or something, to pull the alarm, to signal that our knowledge has been outpaced, that we are no longer in a position to formulate a conclusion, and above all, that no blindly confident machine should be allowed to tip the scales.

Dismantling the illusion

Radiologists have a particular responsibility in the current debate on humble AI. We have lived with calibrated uncertainty long enough to recognize, immediately, when it is missing. The new generative tools entering our field are powerful, and many will be useful. But the question is not whether they reach the right answer more often than human radiologists do. The question is whether they make us more thoughtful when the answer is genuinely uncertain, and whether they have the architectural humility to step back when neither clinician nor AI should be deciding.

Bastien Le Guellec is a neuroradiology fellow at Lille University Hospital, France. He graduated from Ecole normale supérieure Paris with a MSc in neurobiology, where he first engaged with science and computer programming. Now a radiologist, his research interests include human-AI interaction, Large Language Models and data reuse, with a special interest on multilingual and open source projects. He is a member of the Radiology: Artificial Intelligence Trainee Editorial Board.

But, while the technology keeps advancing, reporting has not kept pace. In medical AI, strong performance is not enough on its own. If a model is not described clearly and evaluated in a way that others can interpret, compare, and reproduce, it becomes much harder to trust in clinical practice.

Traditional AI models differ fundamentally from foundation models (FMs) and large language models (LLMs). Earlier AI models were generally developed for narrow, predefined tasks such as classification, detection, or segmentation, and they often operated in a largely deterministic manner, with the same input yielding the same output under fixed conditions. By contrast, FMs/LLMs are more flexible and general-purpose, but also more context-sensitive and stochastic in nature. Their behavior can change depending on the prompt, the context they receive, their generation settings, and even the way a user interacts with them. This distinction between traditional AI models and generative AI systems underscores the need for a dedicated reporting checklist. That is exactly where the REporting checklist for FoundatIon and large laNguagE models in medical research (REFINE) comes in.

What is REFINE and why is it important?

REFINE is a reporting checklist designed for FM and LLM studies in medical research, including but not limited to imaging-focused studies. Its goal is straightforward: to make these studies more transparent, more comparable, and more reproducible in a field where older reporting frameworks do not fully capture the realities of generative AI.

REFINE was developed through a prespecified, publicly archived protocol and a modified Delphi process. The development group included 57 contributors from 17 countries, with expertise spanning clinical imaging, machine learning, medical informatics, and editorial work.

The checklist is also available as a web tool where users can easily use and generate their scores along with publication-ready figures.

What makes REFINE different?

REFINE is not meant to replace earlier AI reporting frameworks. It is designed to complement them. Established guidelines such as CLAIM, CONSORT-AI, TRIPOD-AI, and STARD-AI remain important for study design, participant selection, reference standards, and performance reporting. However, these frameworks were developed before the widespread adoption of generative AI, so they do not fully address issues such as prompt engineering and stochasticity that now sit at the center of FM and LLM research.

Although recent initiatives have made important progress in addressing the reporting gap, each addresses a more specific aspect of the challenge. TRIPOD-LLM extends TRIPOD-AI to LLM-based diagnostic and prognostic prediction studies, MI-CLEAR-LLM defines minimum reporting items for healthcare accuracy evaluations, and DEAL distinguishes between reporting considerations for advanced model development and the use of pre-existing models. By contrast, REFINE is grounded in the same commitment to transparency, but adopts a broader perspective across the full study lifecycle, encompassing workflow integration, safety, privacy, and governance.

Additionally, REFINE was built around the actual failure points and reporting gaps of generative systems. It recognizes that transparency depends not only on what a model outputs, but also on how it was built, prompted, evaluated, and positioned for clinical use. In that sense, REFINE connects technical reproducibility with clinical reliability in a way that earlier frameworks were not designed to do.

REFINE is organized into six practical domains: model specification, prompt design, stochasticity control, dataset integrity, output evaluation, and implementation. Together, these domains form a 44-item checklist built around the real sources of variability in FM and LLM research.

Model specification

This domain asks a basic but essential question: What exactly was used? That question covers the model, version, capabilities, access pathway, and required resources. In a landscape where similarly named models can behave very differently, detailed characterization is not a luxury. It is foundational.

Prompt design

REFINE gives prompt design the attention it deserves. It asks authors to describe how prompts were created, what context was provided, how interactions were structured, and how outputs were handled. This kind of reporting is critical in radiology AI, where small wording changes can meaningfully alter results.

Stochasticity control

Generative models are not always deterministic and REFINE treats stochasticity as a reporting issue rather than a footnote. The stochasticity control domain covers generation parameters, model operators, and how final outputs were selected. Put simply, it addresses one of the central challenges in reproducibility.

Dataset integrity

The dataset integrity domain focuses on the data behind the study: its origin, ethics, prior use, preprocessing, missingness, representational bias, and separation between development and evaluation data. REFINE also emphasizes contamination risk, which is especially important for FMs and LLMs.

Output evaluation

REFINE does not stop at asking whether a model performed well; it asks how performance was assessed, by whom, under which metrics, across which subgroups, and against which benchmarks. That creates a fuller picture of whether the reported output is actually meaningful in practice.

Implementation

The final domain, implementation, brings the discussion into the clinical world. It covers intended use, workflow integration, clinical utility, limitations, safety, privacy, and governance. This is the point at which reporting comes closest to real-world readiness.

How to use REFINE in practice

Beyond its use as a reporting tool, REFINE is also useful during the planning of the study. At the start of a project, it can help teams document model details, track prompts and generation settings, record dataset provenance, and define evaluation and safety checks before those decisions become difficult to revisit. Because REFINE is available as an online tool with built-in guidance, section summaries, and export options, it can function as a working document throughout development, review, and submission.

Its value also extends beyond authors. For reviewers and editors, REFINE offers a structured way to assess methodological transparency and identify gaps that may limit interpretability or reproducibility. More broadly, use of the REFINE checklist can be reinforced through journal instructions, editorial policies, conferences, and professional society endorsement.

As radiology AI moves into the era of foundation models, multimodal systems, and emerging agents, the field needs more than impressive results. It needs reporting standards that make those results understandable, reproducible, and clinically meaningful. By addressing how these systems are built, prompted, evaluated, and implemented, REFINE brings greater clarity to a fast-changing area of research. In that sense, it is not just a checklist for publication, but a framework for building more trustworthy radiology AI.

For more information, the full publication and online tool are freely available.

Ismail Mese, MD, is a radiologist at the Ministry of Health Uskudar State Hospital in Istanbul, Turkiye. His work centers on artificial intelligence in radiology, particularly large language models, foundational models, and their translation into research, education and clinical practice. He serves as a Junior Editor for the European Journal of Radiology Artificial Intelligence and is a member of the steering committee of REFINE.

Tugba Akinci D’Antonoli, MD, is a neuroradiology fellow at the University Hospital Basel in Switzerland. She serves as a board member and Chair of the Education Committee of EuSoMII. Additionally, she is a member of the editorial boards of European Radiology and Diagnostic and Interventional Radiology for the Imaging Informatics and AI section, and is an alumna of the TEB and associate editor of the Radiology: Artificial Intelligence journal. She is a member of the steering committee of REFINE.

Burak Kocak, MD, is a radiologist at Basaksehir Cam and Sakura City Hospital, Türkiye. His academic work focuses on artificial intelligence in radiology, particularly methodological rigor, evaluation frameworks, and the responsible integration of emerging AI technologies into research and clinical practice. He currently serves as a Section Editor for the European Journal of Radiology and as an Editor for the European Journal of Radiology Artificial Intelligence. He is a member of the steering committee of REFINE.

Walking through the exhibition floor at a popular radiology conference, you encounter AI devices from several different companies for the detection of fractures on x-rays. You think that fracture detection is a good use case for AI; indeed, such an AI tool could help to improve patient triage in your emergency department. Each company provides a slick pitch of their device, complete with polished visuals, a variety of testimonials, and impressive performance claims. Each device does something slightly different – one uses bounding boxes to annotate images, another uses heatmaps – but they all exist for the same purpose and promise similar impact.

How do you choose the best device for your service? You recognise that there are a variety of logistical factors at play, including financial costs and technical integration requirements with your picture archiving and communication system. But, as a staunch believer in evidence-based medicine, you primarily want to ensure that you choose the device with the best possible diagnostic performance. All the sensitivity and specificity values presented by the companies are very similar, but can you assume that the devices will perform equally well in your department, in your patient population, in images from your scanners?

Choices have consequences

Selecting the wrong AI device from different available options can cause harm. For clinical deployments, this may range in severity from financial waste on a device that adds little value to implementing a device that leads to direct patient harm or adversely impacts clinical pathways. For example, choosing a device that has lower diagnostic accuracy in patients from ethnic minority groups may exacerbate healthcare inequalities, while choosing a device with a higher rate of false positive results may lead to additional unnecessary investigations that hinder, rather than improve, the efficiency of healthcare service delivery.

In trials and other research studies, choosing the wrong device to evaluate can also have negative consequences, such as research waste and inaccurate interpretation of AI as a technology. Several large studies of single AI devices - including a recent large prospective randomised controlled trial - have failed to show any benefit to clinical pathways. Is this because AI as a technology has no benefit, or simply because the wrong AI device was chosen for evaluation? If a device with inferior performance compared to its competitors is chosen for evaluation and no benefit is found, there is a risk of unfairly condemning the technology rather than the individual device. This could discourage the evaluation and implementation of competitor devices that may genuinely improve clinical care.

Avoiding assumptions

It is worth emphasising that AI devices for clinical decision support are not just software but medical devices that carry risks. As such, decisions about which, if any, device to use must be based on sound critical appraisal of evidence about their performance. Although those looking to procure and implement AI devices typically recognise the need for evidence, there may be various pitfalls in interpretation that risk poor decisions on device selection.

Firstly, it cannot be assumed that an AI device will perform as advertised when used in clinical practice. In fact, a growing body of evidence suggests that this is rarely the case. The performance metrics quoted by manufacturers often reflect ‘best case’ scenarios based on internal testing or small-scale studies. The datasets used for these evaluations are often narrow, encompassing limited patient samples and image acquisition protocols, and frequently demonstrate class imbalance - in other words, they do not reflect the diversity, distributions and complexities of real-world imaging across different healthcare settings. This is a problem of generalisability. Accurately assessing the real utility of a device relies on understanding how the patients and images used for testing differ from those in the intended clinical setting.

Secondly, commonly reported performance metrics may be misleading. Sensitivity and specificity values are undoubtedly useful, but can misrepresent performance when disease prevalence is low, which is a common scenario when imaging is performed for undifferentiated patients from primary care or emergency departments. Sensitivity and specificity values are also only relevant to a particular threshold, which can be varied for devices, and therefore do not provide a complete view of device performance in all settings. Equally, presenting area under the curve (AUC) values in isolation can offer little meaningful insight into how a device is likely to perform under specific clinical conditions. Ultimately, performance metrics are complementary, and the most informative performance metrics depend heavily on the clinical context. Will the device be used in a screening setting where prevalence is low or further along a diagnostic pathway where prevalence is higher?

Thirdly, it cannot be assumed that devices intended for the same purpose will perform equally well. Intuition clearly tells us that devices are likely to exhibit fundamental differences due to variability in their development. Manufacturers use different machine learning architectures, obtain training dataset images from different patient populations using different image acquisition methods, annotate their datasets in different manners, and present device results in different styles. There will be devices that perform better than others for a given clinical setting. Unfortunately, the field to date has tended to treat AI devices as a homogeneous entity (“AI for haemorrhage detection on CT” rather than “Device X for haemorrhage detection on CT”). This is akin to assuming that all antihypertensive drugs are equally as effective. If the medical field won’t accept this assumption of homogeneity about drug efficacy, why should it accept it about AI devices?

Going head-to-head

The challenge of variable performance among AI devices for the same use case can be addressed in two broad ways. The first option is to appraise the evidence for each device separately and compare the pooled evidence between the devices. This is problematic, as single-device studies are themselves heterogeneous. Variability in their design, data sources, choice of performance metrics, and risk of bias introduce confounding factors that make it difficult to draw meaningful conclusions about observed differences in performance.

Undertaking comparative benchmarking studies is a more robust approach. The process of comparative benchmarking involves direct head-to-head evaluation of multiple AI devices using the same dataset, under the same conditions, with the same performance metrics. This helps to eliminate the confounding variables present across single-device studies. Comparative benchmarking studies offer value at different stages of the AI lifecycle; they indicate clearly which devices perform best under given conditions to inform decisions about clinical deployment and further research, they provide regulatory bodies with high quality evidence to determine device equivalence, and they can drive improvements in devices through competition between manufacturers. Benchmarking competing models against one another using open-source datasets has long been fundamental to driving progress in computer science, but the field must now extend this practice to devices in real-world clinical settings.

Although the importance of rigorous comparative benchmarking of AI devices is increasingly being recognised, such studies remain scarce in the literature. This gap can be explained by several factors. Head-to-head evaluations require contracts with multiple device manufacturers, which may be challenging to coordinate. Manufacturers may also be reluctant to participate in studies where their device could underperform that of a competitor, and there is currently no requirement for participation in such studies prior to regulatory approval to encourage participation. In addition, while the models underpinning AI devices may perform similar functions, the results of the devices may be presented in different ways, which can pose a barrier for direct comparison. For example, different devices for detecting lung cancer on chest radiographs may present their outputs using different image annotations, such as segmentation masks, bounding boxes, or heatmaps. Device outputs may require standardisation before they can be compared statistically, which must be handled carefully to avoid introducing bias.

Moving forwards

Establishing comparative benchmarking as core evidence for AI in radiology will require changes within the field. First and foremost, there needs to be a shift in mindset, with broader recognition that independent, robust head-to-head evaluations are a requirement for responsible and effective AI trials and deployments. To facilitate this shift, AI researchers and international organisations should collaborate to develop frameworks for conducting head-to-head evaluations specific to AI for radiology, including strategies for standardising heterogeneous device outputs to enable direct comparison between them. Regulatory bodies should mandate evidence from comparative benchmarking studies as part of approval pathways for devices entering markets where devices for the same use case already exist, which will encourage cooperation and engagement by device manufacturers. Similarly, those looking to take devices into prospective clinical trials or deploy them in routine clinical practice should demand results from comparative benchmarking studies. It is essential that those undertaking head-to-head studies ensure that they are conducted independently – free from any influence from device manufacturers over study design, data selection, or interpretation of results – and reported transparently to maintain their value. There will still be a place for research studies evaluating individual devices, but these should report clearly how such devices were selected and how their performance is expected to differ from any alternative device options.

Final thoughts

Choosing the right AI device from different options represents a significant challenge for clinical trials and deployments. Incorrect choices may waste resources, cause harm to patients and clinical workflows, and prompt undue scepticism of AI as a technology. As the number of available devices continues to grow, direct head-to-head evaluations must become central to the evidence base. The field must move away from asking “does this individual AI device work?” and towards asking “which AI device works best?”.

Dr. Ahmed Maiter is a neuroradiology resident in Sheffield, UK, whose research focuses on the evaluation of commercial AI devices across different radiology subspecialties. He is a current member of the Trainee Editorial Board of Radiology: Artificial Intelligence. As an appointed member of the AI Faculty for the Royal College of Radiologists, he contributes to developing and delivering AI education for radiologists within the UK. Dr. Maiter was awarded the inaugural British Institute of Radiology AI Fellowship in 2024.