Introduction

At Lifen, we build applications that enable healthcare facilities and practitioners to exchange medical documents seamlessly, routing and structuring millions of letters, reports, and clinical notes every month.

As part of this pipeline, every document that flows through our platform needs to be parsed: key pieces of information must be extracted and structured before the document can be properly handled. One of the most critical pieces of information to detect is the patient’s identity.

For a long time, our patient extraction pipeline relied on a Bidirectional Long Short Term Memory (BiLSTM) based model. Recurrent architectures like the LSTM were, for years, the natural choice for sequence labelling tasks: they were relatively lightweight, fast at inference, and once trained on enough annotated medical documents, reasonably effective at spotting patient entities within the noisy, heterogeneous text found in medical documents.

However, since the rise of Large Language Models (LLMs), a new path with a huge performance upgrade was opened. They have an ability to reason over context, generalize more easily from a handful of examples, and are able to overcome many of the LSTM’s pain points.

This article tells the story of that transition: how we moved to a generative LLM for patient extraction from medical documents, and what possible choices of configurations we chose from to achieve the best possible performance.

Time to Switch to Generative LLMs

Our task consists of extracting a patient’s info (first name, last name, birth name, date of birth) in a structured way from a medical document.

Our last model, BiLSTM, was capable of doing so, however, by introducing a lot of overhead: text normalization, word matching, and training word embeddings from scratch.

As you can see, considering a BiLSTM encoder for extraction introduces some level of complexity, especially when a patient’s name spans multiple tokens and requires manual stitching.

Which is why, for the second iteration of our “Patient” algorithm, we wanted to go for something that could, in theory, work out of the box. Naturally, we chose to go for an LLM decoder architecture for this task. Although encoders (BERT-like) are strong models that capture a lot of semantics at token-level, there is nothing more elegant than having one model that generates the correct answer directly, without any post-processing necessary.

Since LLMs are getting much of today’s attention, getting smarter at each generation, being actively maintained by a huge community that keeps getting bigger: they were the clear winner for this task.

At last, we reformulated our objective as such: Given a medical document as input, the LLM should generate a patient’s first name, last name, birth name, and date of birth as output.

LLM structured output

LLMs are generative models, they output free form text, as they were trained on natural language. However, since we are trying to extract specific information from a free form text, our task requires structured output. Therefore, we need to be able to control the LLM in its generation process so that the output matches the format we expect.

Structured generation has been a massive focus for the AI community these past few years. We’ve seen a lot of brilliant projects tackling structured LLM output, with a strong showing from the open-source community. Here are a few standouts:

• outlines: This framework focuses on generation-time structuring. It modifies the model’s token probabilities at each step to strictly enforce adherence to a regex, JSON schema, or context-free grammar.

• guidance: Another powerful generation-time structuring tool. It constrains the model’s output in real time, ensuring the LLM never strays from the predefined format.

• instructor: Operating primarily on post-generation structure adherence, this library leverages Pydantic to orchestrate the generation process. It utilizes function calling or JSON modes under the hood, automatically validating the output against data models and employing self-correction loops to fix any formatting errors.

Even in the closed API ecosystem, major actors (such as OpenAI, Anthropic, Mistral AI) understood the importance of structured output, which is why they all added it as a feature for their LLM based APIs:

• OpenAI initiatives: They initially introduced Function Calling in 2023, where users provided a JSON Schema of a function, and the model was trained to output a JSON object matching that schema. However, it didn’t strictly guarantee valid formatting. Then came structured output in 2024 to guarantee 100% adherence to the supplied JSON schema.

• Anthropic initiatives: They originally relied on prompting techniques (like forcing the model to output between XML tags) before officially implementing native Tool Use (Function Calling) for Claude, and later introducing official support for forced JSON object responses to match strict schemas.

• Mistral AI initiatives: They followed a similar trajectory, starting with native Function Calling and a standard JSON Mode which forced the output to be valid JSON but did not strictly guarantee adherence to a specific schema. They later upgraded this by rolling out official Custom Structured Outputs, much like the other players.

While API-level structured outputs are highly effective, two other classic approaches remain relevant. Prompt engineering is often the quickest and lowest-effort solution for forcing a structured format purely through instructions. However, it is inherently less reliable and less efficient than native API constraints, as models can still occasionally break formatting rules.

On the other end of the spectrum, fine-tuning provides a highly robust solution for structured generation. By training the LLM on a large dataset that consistently uses the desired schema, the model inherently learns to replicate that exact structure during inference. Consequently, in addition to mastering the core task, such as extracting a patient’s identity, the model naturally learns to format its response in the required structure without relying on complex prompts.

In our use case, we chose to fine-tune the LLM in our specific format, since it was the logical step to move away from the BiLSTM pipeline. However, at the scale of millions of documents processed per month, latency and throughput become hard constraints. Patient LLM runs synchronously during document ingestion, meaning the extraction happens while the user is actively waiting.

This is why, beyond getting the right answer, we also needed to optimize inference speed, and that optimization drives the next two design choices: output format and quantization.

LLM Output format

In our experiments, CSV refers to a compact, comma-separated output format where values are emitted in a fixed order, without repeating the field names at every prediction. For example, instead of generating a JSON object such as:

{
  "first_name": "John",
  "last_name": "Doe",
  "birth_name": null,
  "birth_date": "1970-01-01"
}

the model generates the same information as:

John,Doe,,1970-01-01

This format is close in spirit to TOON, which has recently become popular for LLM inference because it reduces the number of tokens needed for structured outputs. In our case, we take the idea one step further: since the schema is fixed and known in advance, the keys do not need to be generated at all. The output position itself carries the meaning of each value: first name, last name, birth name, then date of birth.

This makes CSV particularly well suited to fine-tuned models. During training, the model sees the same field order over and over again, so it learns to reliably produce values in that exact order. Compared to JSON, this avoids repeatedly generating field names, quotes, braces, and other structural tokens. The resulting output is shorter, which directly helps reduce latency and improve throughput, while keeping the same semantic information.

Quantization

As suggested in their name, LLMs are made up of a huge number of parameters, usually in the billions. Because their generation process is auto-regressive, without further optimization, the model requires close to a second to generate a full prediction (in our case, the list of patient attributes).

Originally, each one of these billions of parameters is encoded in 32-bit floating-point precision (FP32). While we do not intend to reduce the total number of operations, what we can do instead is reduce the numerical precision of the data being processed.

And this is where quantization kicks in. Instead of storing and calculating these parameters in FP32 , quantization reduces their bit-width to a lower format (like FP16, INT8, or even INT4). This drastically reduces the hardware workload and memory bandwidth bottlenecks. In summary, the advantages of quantization are usually two-fold:

• Inference Speed: Because the hardware can process smaller precision types much faster than FP32, the model generates outputs with much lower latency.

• Memory Footprint: Cutting a model’s precision reduces the storage and RAM requirements, allowing massive models to run on much smaller GPUs than originally required.

Reducing the precision of a model’s floating point can lead to lose part of an LLM’s capabilities. To achieve the benefits of using a LLM without destroying the model’s intelligence, advanced Post-Training Quantization (PTQ) methods can be used. Two of the most popular and effective techniques for LLMs are GPTQ and AWQ.

• GPTQ (Generalized Post-Training Quantization): This is a layer-by-layer calibration method. It compresses the model’s weights by treating the quantization process as an optimization problem, minimizing the error in the output for a small set of calibration data. It is highly efficient and allows massive models to be shrunk down to 4-bit precision while retaining remarkably high accuracy.

• AWQ (Activation-aware Weight Quantization): AWQ takes a different approach by recognizing that not all weights are created equal. It observes that a small percentage (about 1%) of the model’s activations are “salient” (extremely important) to the model’s performance. Instead of quantizing everything equally, AWQ handles the most important weights with extra care, so they keep more of their accuracy when compressed, resulting in higher overall accuracy.

Since quantization requires reducing the precision of our parameters, it means re-scaling the values into the correct lower-precision range. With that in mind, there are two main approaches:

• Dynamic: The re-scaling factors for activations are calculated on the fly at inference time based on real-time data

• Static: The re-scaling factors for both weights and activations are pre-calculated using a calibration dataset, meaning no scaling overhead happens during inference

Naturally, dynamic quantization is often more precise because the scaling adapts to real-time inputs. However, calculating these scaling factors on the fly adds a computational overhead, which can slow down the model’s forward pass compared to static quantization.

Reducing a model’s numerical precision inevitably comes with a cost: quantization can degrade the model’s capabilities, and we need to carefully measure this degradation to find the right balance between inference speed and output quality.

In our use case, we decided to benchmark a limited set of these quantization methods. We use llmcompressor library to compress our LLMs, and chose to try the following quantization types: AWQ, FP8_DYNAMIC. Both quantization methods will be benchmarked in terms of performance and latency.

LLM serving

Again, for serving purposes, open-source ecosystems have emerged to handle LLM serving. Here is a quick look at the major actors dominating the space:

• vLLM: vLLM revolutionized serving with PagedAttention, an algorithm that prevents memory waste by managing the Key-Value (KV) cache like virtual memory in operating systems.

• SGLang: SGLang introduces RadixAttention, which automatically caches and reuses prompt prefixes across entirely different requests.

• TensorRT-LLM: Built directly by NVIDIA, this is a highly optimized, low-level C++ and Python framework designed to wring out every last drop of performance from NVIDIA hardware.

• TGI (Text Generation Inference): Built by Hugging Face, TGI was a pioneer in introducing production-level features. However, since development stalled compared to other actors, Hugging Face stopped maintaining the project since late 2025.

In our use case, we decided to build our serving solution with vLLM. TGI was not a suitable option for us because it is no longer actively maintained and did not support all the models we wanted to evaluate, including SmolLM3. In contrast, vLLM is one of the most actively updated LLM serving frameworks today, is straightforward to deploy and use, and consistently reports strong performance in public benchmarks.

Dataset

Our dataset consists of de-identified medical notes of various types and sources, annotated in-house by our experts. Prior to processing and labeling, all notes are pseudonymized using a dedicated in-house algorithm. All patient data is collected and processed in compliance with all applicable European and French laws, regulations and guidelines for the processing of health data for medical research purposes.

We then use these de-identified medical notes to train and evaluate our models. The results obtained on the test set resulting from this data collection are shown in the following section.

Results

Results in macro precision*:

*the macro precision is the unweighted average of per-field precision scores across the four predicted fields (first_name, last_name, birth_name, birth_date). For each field, precision is computed as the ratio of correct predictions over the number of documents where the model actually produced a non-null prediction. The macro average then gives equal weight to each field regardless of how frequently it appears in the dataset.

Our baseline with a BiLSTM was able to achieve 96% macro precision, which is already a strong score. Let’s see how much we can improve using newer, bigger models.

Analysis:

Overall performance among our fine-tuned models is consistently around 99%, with slight changes depending on configuration, indicating the task is well within reach of even very small models.

Model size has a small impact. With CSV output and FP8_DYNAMIC quantization, which is our standard configuration for production, precision rises gently from 98.7% at ~0.5B scale to a plateau of 99.2% from the ~3B tier onward. Hence, for our use case, we decided to select SmolLM3-3B as our main model because of its license (apache 2.0) and the fact that for its size, it provided a great balance of performance.

Output format (CSV vs. JSON) makes essentially no difference. On SmolLM3-3B, CSV and JSON both reach 99.2% under FP8_DYNAMIC, so the structured format choice can be driven by downstream integration needs rather than accuracy.

Quantization method has a small effect. For SmolLM3-3B, AWQ trails FP8_DYNAMIC slightly.

Latency results:

Since CSV is clearly a smaller format than JSON, and since they perform the same, we decided not to benchmark JSON v.s. CSV in terms of latency. Instead, we solely focused on AWQ v.s. FP8_DYNAMIC, and compared their respective average latency with respect to requests per second (RPS) on a single pod using a L4 GPU.

The same applies to model size. We did not observe a significant performance gap when going from ~3B to ~8B models. Hence, since 8B models would clearly have higher latency than 3B models, we did not benchmark them in terms of latency.

Both methods scale gracefully with load. It goes to show that vLLM handles the increased request volume efficiently, its continuous batching and PagedAttention memory management absorb the extra concurrency without meaningful latency degradation.

AWQ’s average latency is around 0.25 seconds, while FP8_DYNAMIC’s average latency is around 0.4 seconds. While both are acceptable for our use case, we decided to go for FP8_DYNAMIC, since it provided the best overall performance in terms of macro-precision.

Why Not Just Use a SOTA LLM?
We also tested whether a large general-purpose LLM could solve the task without fine-tuning. Using Opus 4.6 with prompt engineering and 5 few-shot examples, we reached 86.4% document precision (432/500) on 500 pseudonymised medical documents, at a cost of $8.2075. Our fine-tuned small LLM, using CSV output and FP8_DYNAMIC quantization, reached 98.5% document precision on the same task. That means document-level errors dropped from 13.6% to 1.5%, or roughly 89% fewer errors. For this use case, bigger was not better. Patient identity extraction is a constrained, schema-driven task where consistency matters more than broad reasoning, and fine-tuning a small model proved more accurate, cheaper, and faster.

Conclusion

While we did not try all possible configurations for quantization, we found a reasonable balance for our use case. Our benchmarks show that SmolLM3-3B with FP8_DYNAMIC quantization and CSV output format is a very reasonable balance with the best precision for our specific task.

Our reasoning:

1. FP8_DYNAMIC delivers a high precision. At 99.2% macro precision, FP8_DYNAMIC edges out AWQ and matches the best results seen on our leaderboard.

2. Latency remains well within acceptable bounds. Although AWQ is faster in absolute terms, FP8_DYNAMIC still serves at sub-second latency, which is more than adequate for the target use case.

3. FP8 is a future-proof choice. It is hardware-native on modern GPUs and a well-supported, standard quantization path, typically simpler to deploy and maintain than AWQ. This reduces operational risk and keeps the door open as models and infrastructure evolve.

4. The serving stack scales gracefully. Latency stays nearly flat from RPS=2 to RPS=8, showing vLLM handles increasing load efficiently with clear headroom before saturation.

More broadly, the results show that a compact 3B model is fully sufficient, there’s no need to pay for larger models when a well-quantized SmolLM3-3B reaches near-ceiling accuracy at sub-second latency. With FP8, the configuration favors maximum accuracy and operational simplicity while keeping latency and cost firmly under control.

To explore

The current setup is already good enough for production, but a few inference optimizations could still be explored if latency or throughput becomes an issue.

One first direction we tried is speculative decoding. The idea is to use a smaller, faster draft model to generate candidate tokens, then let the larger target model verify them. If the draft model predicts what the target model would have generated, several decoding steps can be skipped.

This worked reasonably well with Qwen, using a 0.6B model as the draft model. However, the speedup was not large enough to justify the added serving complexity. For SmolLM3-3B, this is also less practical because there is no suitable smaller SmolLM3 model to use as a draft model.

Another option is n-gram speculation. Instead of using a draft model, it speculates future tokens by reusing n-grams already present in the prompt or generated context. In theory, this should fit extraction tasks very well, since the model often copies names, dates, or other patient information directly from the input document.

However, our first experiments with n-gram speculation did not show any speedup. This should be investigated further, especially to check whether the limitation comes from the implementation rather than from the method itself.

Finally, EAGLE could be tested if performance becomes a real bottleneck. It is one of the current state-of-the-art approaches for speculative decoding, but it would also add more complexity to training, serving, and maintenance.

Join Us at Lifen!

At Lifen, we’re at the forefront of leveraging AI to solve real-world challenges in healthcare. If you’re passionate about building reliable, interpretable AI systems and want to work on impactful problems, we’d love to hear from you!

We’re currently hiring for roles in machine learning, engineering and more. Check out our careers page to explore opportunities and join our mission.

A few months ago, I wrote that everything changed for developers. I still believe it. But this semester, deep in the work, I noticed the opposite is just as true: AI changes almost everything about how we define the job, and nothing about what makes a good engineer.

Let me explain with two things that kept me busy these past months.

This semester, AI let me carry more than I ever could before: reworking the software engineer job card, replacing Segment with our own tool, launching our agentic platform, squeezing our CI workers, building MCPs, rethinking how we keep our token consumption under control. Most of it ran in parallel, across different Claude sessions (thank you, claude agents…).

But after months of intensive use, one thing is obvious. AI is excellent at the Pareto game: 80% of the work for 20% of the effort. And in a company like ours, 80% is never the job.

What AI changes: the job itself

We added an AI Fluency matrix to every job card at Lifen, for every role, not just tech. Four levels, four categories: usage, mastery, critical thinking, impact. The goal is simple: every Lifener at level 2 by year-end. (And no, we don’t evaluate anyone on how much AI they use. That would miss the point entirely.)

For engineers, we went a step further and rewrote the job card itself.

Mastering a language or a framework? Still valuable. No longer what sets you apart. What counts now is understanding the problem, choosing the architecture, owning a feature end to end: conception, code, production, monitoring, and iterating until the value actually lands for a user. We even wrote it down: your code has to be usable by your teammates and by the agents they pilot.

AI speeds up parts of the cycle. The Engineer (capital E) does the rest: stay in control, understand the stakes, follow through, communicate.

What AI doesn’t change: the last mile

If the role widens, what stays at the center? Finishing.

Take event-hub, our replacement for Segment. We reassess our tools constantly, and at a time when sovereignty matters more than ever, our application-monitoring flow was an obvious candidate to bring back home. For almost 10 years it had all gone through Segment. Good SaaS, honestly, but a third party we didn’t control, billing us in dollars (🇺🇸), and on which we’d piled up technical debt over the years: deprecated flows, events that no longer reached our tools.

The brief looked simple: two Go routines, one stacking events from the sources, the other dispatching to the destinations. AI is shaking up build-vs-buy. So we went for it.

The 80% came fast. Deployed to staging, started the migration. And then… the real work, in the last 20%. Every source, every app sending events. Which events to drop for which destination (and why). The alerting, the monitoring. Migrating our CRM-to-warehouse sync that, for some reason, lived in there instead of an Airflow DAG. And every time a shinier topic showed up, staying put and carrying this one to the end.

This is the part AI won’t do for you. Spinning up a POC in two days is easy now. Confronting reality and scale is another story. We don’t do vibe coding. Today Event-Hub runs 1M events a day on 3 pods. Simple, efficient, 100% on our own infra.

The rare skill is no longer producing

Here’s what I take away, and what I’d tell anyone joining us.

Nothing new under the sun: the craft still asks for rigor, patience, and determination. Maybe more than ever, precisely because AI makes it all feel so easy that we forget the only thing that matters: doing the work well, reaching our users, moving the mission forward.

AI made starting almost free. You can launch ten things at once. So the scarce skill flipped. It’s no longer the ability to produce. It’s the focus to choose your battles, and the discipline to finish.

Explore Life at Lifen
Curious about how we tackle challenges like this? Visit our Welcome to the Jungle page to discover our culture, meet the team, and explore open roles.

Stay Connected
Subscribe for insights on healthcare innovation, engineering best practices, and what makes Lifen unique.

Subscribe now

Lifen builds digital tools for healthcare professionals: secure messaging, document management and connections between hospital IT systems. Lifen Integration makes it easy to file medical documents into hospital software (EHR), saving time for care teams and cutting down on errors.

How do you keep a product reliable when it handles thousands of critical medical records every day? This article will cover our approach: the principles we follow, how we test, and what we do to make sure what we ship actually works.

Quality is everyone’s job

Quality isn’t just a QA problem, everyone on the team owns it. Each person plays an active part in making sure we deliver a solid product. In practice, that means:

We share what we know

🧠 Every team member brings their own understanding of the domain. By pooling that knowledge, we build a shared picture of the product and avoid the kind of misunderstandings that lead to bugs or unexpected behaviors.

We cover more ground

🔍 When testing isn’t the job of a single person, coverage grows naturally. Devs test their own work at the unit and integration levels. QA checks the user experience matches the spec and guards against edge-case usage limitations, while Product ensures overall design consistency. Together, we cover more ground than any single role could alone.

We catch side effects earlier

💡 A developer who knows a module inside and out can flag risky areas for QA, and QA can target their tests accordingly. This helps us spot edge cases and potential regressions before they reach production.

Good intentions aren’t enough. To turn shared responsibility into something concrete, we lean on a handful of testing principles. Nothing revolutionary, just useful.

Software testing principles at Lifen Integration

These are adapted from widely recognized software testing principles. We don’t follow them dogmatically, but they guide how we think about quality.

✅ Tests find bugs: They don’t prove there are none

A passing test only means the scenarios we checked didn’t identify an issue. It doesn’t mean the software is bug-free.

How do we put this into practice?

We aim to cover as many cases as possible: combining unit tests, integration tests, and End-to-End (E2E) tests to catch issues at every level. Our test scenarios cover both the happy path and edge cases.

Real-world example: Importing a medical report into the hospital’s patient record

• Unit tests (focused tests) check each building block in isolation: the document name built automatically from patient details, the admission/discharge dates, the exam type, etc.

• Integration tests drop a file onto the hospital’s server (via SFTP), check it’s correctly imported in the Lifen Application and that the right attributes have been applied to the patient.

• End-to-end tests replay the full workflow, from upload to delivery in the hospital’s EHR, making sure all medical data comes through correctly.

⚖️ You can’t test everything

Covering every possible combination and use case is simply impossible.

How do we put this into practice?

We prioritize our tests based on the impact of each change: how critical it is to the app, how important it is for the product, and how stable the component is.

⏰ Test early

The later you catch a bug, the more it costs to fix. A defect found in production can cost 10 to 100 times more (French source) than one caught during design.

How do we put this into practice?

We bring testing into every stage of development so we can spot misunderstandings before any code is written. This is called the shift-left philosophy.

A bug caught during a spec review takes minutes to fix. The same bug found in production can take days.

Principles are only as good as the way they’re put into practice. Here’s how they translate into a concrete workflow: a continuous cycle where quality is built in, not bolted on at the end.

The testing cycle at Lifen Integration

Testing is woven into every step of development: it’s not just a final check before release. Each stage is a chance to catch issues early:

This cycle has served us well. With AI accelerating how fast code gets written, our quality practices need to evolve just as quickly. So how do we keep up without compromising on what matters most?

AI as a QA ally

🧪 Maintaining automated test suites takes a lot of time and effort, which limits how much we can focus on higher-value work like exploratory testing.

🚀 Powered by AI, developers are shipping faster and in greater volume.

Two challenges for QA:

Challenge #1: Keep up without cutting corners on coverage

AI can help us review new code faster, generate extra test cases, and spot scenarios we might miss on our own.

Challenge #2: Make the test suite more reliable and less noisy

AI can help us diagnose and fix flaky E2E tests (those that fail randomly for reasons unrelated to the actual application behavior).

What’s next with AI? Self-Healing!

🔄 Self-healing is the ability of a system to detect, diagnose, and automatically fix certain issues without human intervention.

One of the big challenges is telling the difference between a legitimate change in the application (which means the tests need updating) and a real bug. This distinction can be tricky even for humans, which makes it an interesting use case for AI.

The idea is that AI could learn to recognize patterns of normal product evolution over time, reducing the burden on QA teams while still requiring human validation for final decisions.

Conclusion: Quality is a team sport

At Lifen Integration, quality isn’t a checkbox: it’s a mindset embedded in every stage of development.

What we’ve learned:

1. Quality works best as a collective responsibility. When developers, QA, and product collaborate from day one, bugs are caught in conversations, not production.

2. Test early and often. The shift-left philosophy saves time and catches issues when they’re cheapest to fix.

3. AI opens new possibilities for QA. From generating test cases to self-healing tests, AI is transforming how we approach quality assurance at scale.

We’re transparent about where we excel (⭐⭐⭐ in grooming and spec review) and where we’re improving (optimizing automation). That honesty keeps us moving forward.

Why does this matter? When your code protects real patient data, engineering becomes more than a job, it becomes a craft with purpose.

Explore Life at Lifen

Curious about how we tackle challenges like this every day? Visit our Welcome To The Jungle page to discover our culture, meet the team, and explore open roles in engineering and product.

Share

For over 10 years, Lifen has been working every day to fulfill its mission: improve patient care pathways and clinical research.

To achieve this, we develop various products, all based on artificial intelligence that analyses medical documents, classifies them, and structures them. Every month we process millions of documents. Documents we send with Lifen Documents, integrate into patient records with Lifen Integration, or add to cohorts for scientific studies with Lifen Research.

A decade of selling AI. It was time to embrace it ourselves.

It’s very easy for me to remember the public release of ChatGPT—November 2022. That’s the month I became a father for the first time. My child transformed my personal life - OpenAI took care of my professional one…

Since then (especially in 2025), I’ve stopped Googling and the terminal has become my tool of choice (I’m not from the vim generation…). My knowledge, my experience (my failures), and my critical thinking are more valuable than ever. I feel like I’m having more impact every day.

Claude Code was released in February 2025. In that impossibly short time, it achieved 80% daily adoption across our entire R&D team. We chose to deploy it via the LiteLLM gateway to maintain fine control over our data.

The full story of how we rolled out SmartOps and drove AI adoption across Lifen could fill another article—maybe next time.

For now, I wanted to share what this transformation means for us as developers.

The message I shared with our R&D team:

2025 marked a turning point in the developer profession. Our profession has never evolved so much in such a short time.

The last few months (especially December), the release of the latest models (Opus 4.5, Gemini 3 Pro…) have shifted AI-powered development into high gear.

Some skeptics from a few months ago (👋 DHH) are now believers.

Manual coding is becoming rarer (some Lifeners say they write less and less code themselves), and some even say that what once fulfilled them—the fluency of mastering a language—has now become a source of frustration!

What’s next?

The technical barrier is lowering. Perfectly mastering a framework or technology is still valuable but no longer enough.
We need to develop new skills and master new tools: context management, plugins, skills, hooks, MCP, LSP...
We need to understand why the LLM gets things wrong—and how to guide it better.
We need to make our best practices and quality standards explicit—and stay critical of its output.

Iterate daily on the context (especially the Claude.MD that needs constant updating).
Write crystal-clear design notes, meticulously crafted JIRA cards, and plan (in the sense of actually making plans) what we want to build before building it.
If we want to move faster, we need to stay in control of our safety nets: tests, linters, SAST, sandboxed environments…

We need to optimise the LLM’s flow so these guardrails activate automatically and very quickly (via hooks, for example).

Soft skills become more important: communication, understanding our domain, adjacent disciplines, and of course our end users.
Gradually, everyone shifts from a ‘producer’ role to an architect role with a product/business/end-user vision.
The technical barrier is decreasing—but everyone’s level and impact is increasing.

Impact on Lifen?

We’re even better equipped to fulfill our mission.
We’ll be able to develop more use cases, get our ideas into our users’ hands faster.

We can do more with less friction: we’ll be able to fix our vulnerabilities faster, spend more time thinking together about architectural choices...
We’re launching major projects in parallel—focused on increasing our sovereignty, strengthening our control, and improving efficiency—things that would have been unthinkable for our team just a few months ago.
Today we’re launching them, and we’re confident.

Impact for us developers

The Jevons paradox should hold true: When progress increases the efficiency of a resource, consumption of that resource increases.
An engineer who’s burned their fingers pushing a change to production at 6 PM, who’s worked in the trenches (not the glitter of vibe coding), and who masters the business context can now produce much more value for the company.

If companies want to grow their value (to stay competitive)—demand for engineers will also grow.
We have great days ahead!

2026 promises to be just as impactful for our profession.

Let’s stay curious, critical, talk to each other as humans, open our eyes, and enjoy the moment!

Subscribe now

Explore Life at Lifen

Curious about how we tackle challenges like this? Visit our Welcome To The Jungle page to discover our culture, meet the team, and explore open roles.

At Lifen, we’re building the data infrastructure that powers modern healthcare. Founded in 2015, we serve more than 800 healthcare facilities across France.

Our mission is simple but critical: enable instant access to quality health data for better patient care. Using AI-powered solutions, we help healthcare providers securely share, process, and integrate medical information across fragmented systems.

What is Lifen Integration?

Lifen Integration is our flagship product that automates the processing and integration of medical documents into Electronic Health Record (EHR) systems. Think of it as an intelligent document router: medical reports arrive from various sources—secure health messaging systems (MSSanté), external laboratories, imaging centres, biomedical devices, or even paper documents—and our platform uses AI to automatically extract key information (patient identity, procedure dates, document types), verify it against hospital databases, and integrate everything seamlessly into the appropriate patient record in the EHR.

For healthcare professionals, this means no more manual data entry, no more identity verification errors, and no more hunting through multiple systems to find patient information. For hospitals processing hundreds or thousands of documents daily, it’s the difference between hours of administrative burden and seconds of automated efficiency.

But here’s the challenge: when your product directly impacts how quickly healthcare professionals can access critical patient information, performance isn’t just a nice-to-have—it’s a clinical imperative.

When Performance Becomes a User Experience Problem

One of our largest clients (a hospital) reached out with feedback that stopped us in our tracks: their team was experiencing frustrating delays when loading their daily work dashboard. For healthcare professionals processing hundreds of medical documents per day (lab results, imaging reports, discharge summaries), every second of delay compounds into lost time that could be spent on patient care.

At Lifen, we build data infrastructure for healthcare providers.

Our integration platform, a Rails API serving a React frontend, helps hospitals process and route medical documents from external sources into their Electronic Health Record systems.

Think of it as a smart mailbox: medical documents arrive from various external sources, our platform receives them, and healthcare professionals review and integrate them into the appropriate patient records.

For us, performance isn’t just about user experience: it directly impacts how quickly healthcare professionals can access critical patient information.

When we dug into our Application Performance Monitoring (APM) dashboards (we are using AppSignal for this purpose), the reality was worse than we thought. Our most critical API endpoints (the ones that powered the daily workflows of thousands of healthcare professionals) were among the slowest in our entire system.

The numbers were concerning:

• Average response time: 1.15 seconds for our integration request endpoint

• For our largest clients: response times regularly exceeded 5 seconds on unfiltered requests

• These endpoints handled thousands of requests daily. The cumulative impact was massive

This wasn’t a theoretical performance problem. Real users were waiting, frustrated, while our servers churned through what should have been simple list queries.

It was time to find out why.

What Our Monitoring Revealed

We started where any performance investigation should: tracing actual production requests to understand exactly where time was being spent. Using our APM tool, we analyzed sample traces of slow requests from our largest clients. What we found was a cascade of compounding problems.

1. The N+1 Query Cascade

The first culprit was a classic but concerning anti-pattern: the N+1 query problem.

For those unfamiliar, the N+1 query problem occurs when an application makes one query to fetch a list of records, then makes N additional queries to fetch related data for each record.

In our context: An attachment refers to the actual medical document file (PDF, image, etc.) associated with a document—for example, a lab report PDF or an imaging study scan. Metadata refers to the structured information about that document: patient identity, procedure dates, document type (lab result, radiology report, etc.), originating provider, and other fields that help classify and route the document correctly. In healthcare, this distinction matters: the attachment is what clinicians read, while metadata is what our system uses to file it in the right patient record.

Here’s what the code looked like:

The pattern repeated across multiple relationships. Our APM traces showed:

• N+1 queries on document attachments

• N+1 queries on verification records

• N+1 queries on related metadata tables

For a “simple” list of 20 documents, we were executing over 60 queries just to fetch related data that should have been loaded in 2-3 efficient queries.

2. Synchronous External Service Calls

The second major bottleneck was more insidious: synchronous HTTP calls to external healthcare data services during response serialization.

Our system integrates with external services using a Healthcare Systems Standard (FHIR) to fetch additional medical information (patients data, etc). The problem? We were making these calls synchronously (waiting for each response before continuing) right in the middle of serializing our API response.

The numbers from our APM traces told the story:

• Slow queries on connector configuration: ~500ms

• Cache lookup operations: ~440ms

• Multiple cache read operations: 30-100ms each

We had a caching layer, but it wasn’t enough. Cache misses meant waiting for external service responses, and even cache hits added hundreds of milliseconds due to inefficient cache key lookups.

3. Over-Fetching Data

The third anti-pattern was subtler but equally damaging: we were loading far more data than we needed.

Our database queries were fetching complete related objects even when the UI only displayed a tiny fraction of their fields. The most egregious example: we loaded complete patient records (30+ database columns) when the dashboard only displayed their first and last names.

This pattern repeated everywhere:

• Loading full provider records to display a single name field

• Fetching complete verification objects when we only needed a status flag

• Querying tables that were irrelevant to the current dashboard context

4. The SQL Query Avalanche

When we finally counted up all the queries for a single API request, the number was shocking:

More than 200 SQL queries per API call.

This wasn’t just a performance problem. It was also a database connection pool problem, a network overhead problem, and a maintainability nightmare. No amount of index optimization could fix this. We simply had too many queries.

Our APM tool’s waterfall chart painted a clear picture: most of our response time was spent waiting on databases, waiting on cache, and waiting on external services:

5. The Developer Experience Problem

As we dug deeper, we realized the performance issues were symptoms of a larger problem: accumulated technical debt that made our codebase increasingly difficult to work with.

Legacy serialization complexity:

Our serialization system had evolved over years, resulting in two parallel code paths: legacy serializers and newer Blueprint-based serializers (using the gem of the same name).

The migration between them was incomplete: some Blueprints were partially integrated, most weren’t used at all. Engineers couldn’t easily predict what data would be included in a response without tracing through multiple layers of inheritance.

Context-driven API design:

The frontend would request abstract dashboard “types,” and the backend would “intelligently” compute the appropriate payload. This created tight coupling between frontend and backend: changes in one rippled unpredictably into the other.

Impact on velocity:

• PR review times were impacted due to complexity

• New engineers needed significant time to understand a single endpoint

• Engineers hesitated to make changes due to unclear side effects

• Questions like “will this change affect the other dashboard?” were common and hard to answer

It was clear: we couldn’t just optimize the existing code. We needed a fundamental rethink.

Want More Engineering Insights?

We regularly share deep dives into how we build and scale healthcare technology. From API architecture to AI-powered data processing, our engineering team writes about solving real-world challenges at the intersection of healthcare and technology.

Subscribe to stay updated on how we’re building the future of healthcare infrastructure.

Subscribe now

Building API V2 from the Ground Up

The Strategic Decision: V2, Not V1.1

After a few attempts to improve performance while maintaining backward compatibility, we learned a lesson: “Recent attempts to improve performance and reduce technical debt taught us that trying to enhance our APIs without introducing breaking changes was very complex and often unsatisfying.”

Incremental improvement wasn’t working. We needed permission to do things right.

The V2 approach:

• Create new endpoints under an explicit v2 namespace

• Make V2 explicitly incompatible with V1 patterns

• Migrate frontend and backend in parallel

• Use feature flags to control rollout

• Validate with production traffic before generalizing

We started with the highest-impact endpoint: the “Integrations worklist” that powered the dashboard our largest clients used most heavily.

Technical Architecture Changes

1. Minimal, Purpose-Built Serialization

The foundation of our V2 approach was rethinking what data we returned. Instead of over-inclusive bundles, we designed minimal, dashboard-specific payloads.

We chose to keep Blueprinter as our serialization library but extended it with blueprinter-activerecord, a gem that provides built-in tools to preload resources and associations, and thus help prevent N+1 queries.

We started by auditing exactly what fields the frontend displayed in the dashboard. Then, we built a Blueprint that included only those fields: no more, no less.

Here’s what our new response structure looked like:

Key differences from V1:

• Most attributes come directly from database columns (no expensive joins)

• Patient object contains only 3 fields instead of 30+

• Removed all data irrelevant to this specific dashboard

• Clear, predictable structure with no hidden conditional logic

2. Smart Association Preloading

The next challenge was eliminating those N+1 queries. Instead of making multiple database queries or HTTP calls during serialization, we implemented a three-phase approach:

1. Collect all required IDs upfront

2. Batch-load all related data in efficient bulk queries

3. Render the response with everything already in memory

Here’s what that looked like in code:

The transformation in query patterns was dramatic:

BEFORE - N+1 pattern:

AFTER - Batch loading:

From 41 queries to 3. That’s a 93% reduction, and the pattern extended across all our relationships.

For external service calls, the improvement was even more dramatic. Instead of making 20 separate HTTP requests (or relying on an inadequate cache), we collected all IDs, made a single batch request, and loaded everything into memory before rendering. Result: O(1) external calls instead of O(n).

3. Explicit Filtering

In V1, the frontend requested abstract dashboard “types” and the backend guessed what filters to apply. In V2, we flipped the responsibility: the frontend explicitly states every filter it wants.

Benefits:

• Backend doesn’t need to “understand” UI context

• Clear separation of concerns: frontend owns filtering logic

• Easy to reason about and test independently

• More RESTful and predictable API design

4. OpenAPI Documentation & Type Safety

The final piece was ensuring our new API would stay maintainable. We used RSwag to generate OpenAPI documentation directly from our test suite.

The benefits were immediate:

• Living documentation that’s always accurate (outdated docs = failing tests)

• TypeScript types automatically generated for frontend

• N+1 detection built into test suite

• Contract testing between frontend and backend

• Onboarding docs for new developers

Frontend Changes

On the React side, we created new dashboard components specifically for V2. Using auto-generated TypeScript types from our OpenAPI spec, the frontend team could develop with confidence that their code matched the API contract.

Feature flags controlled which version users saw, allowing us to gradually roll out: 5% → 25% → 50% → 100% over two weeks, monitoring metrics at each stage.

The Results: Numbers Don’t Lie

After a couple of weeks of development and careful rollout, we finally had production metrics from V2. The improvements exceeded our expectations.

Performance Metrics

Response Time - Average:

• Before: 1,175ms

• After: 225ms

• Improvement: 5.2x faster

Response Time - 90th Percentile (P90):

• Before: 1,900ms

• After: 524ms

• Improvement: 3.6x faster

The P90 metric is particularly important: it represents the experience for 90% of users, including those on slower connections or during peak load times.

Payload Size:

• Reduced by ~3x for a typical page of 20 documents

• Faster network transfer, especially critical for mobile users

• Reduced client-side memory consumption

• Faster JSON parsing in the browser

Database Query Count:

• Before: 200+ queries per request

• After: <10 queries per request

• Improvement: ~95% reduction in database load

With over 50,000 API calls per day on this endpoint alone, the cumulative impact was noticeable:

• Dramatically reduced pressure on database connection pools

• Better resource utilization across the entire infrastructure

Code Quality Improvements

The benefits went beyond raw performance numbers.

Measurable code improvements:

• V1 Controller: 250 lines, cyclomatic complexity: 28

• V2 Controller: 120 lines, cyclomatic complexity: 8

• V1 Serializer: 400 lines across 5 files with unclear dependencies

• V2 Serializer: 150 lines in a single Blueprint file with explicit preloading

Developer experience improvements:

• PR review time reduced

• Onboarding time for new engineers cut significantly

• Bug fix time: Clear code paths made debugging significantly faster

• Confidence: Comprehensive tests + clear architecture = fewer production rollbacks

Here’s the code comparison:

V1:

V2:

The difference speaks for itself.

Business Impact

Quantified improvements:

• Reduced infrastructure costs for this service (exact savings are still being calculated)

• No customer complaints about dashboard performance since V2 launch

• faster feature delivery estimated due to simplified codebase

Qualitative feedback (anonymized):

• “The new dashboard loads almost instantly now. Thank you!”

• “We can finally process documents as quickly as they arrive”

• Engineering team morale improved significantly with cleaner codebase

In healthcare, these improvements translate directly to patient care.

Lessons Learned

1. Sometimes a Clean Break Is Better

Our attempts to incrementally improve V1 taught us that sometimes you need permission to start fresh. V2 allowed us to rethink our architecture from first principles without the constraints of backward compatibility.

Breaking changes are scary. But feature flags and gradual rollouts mitigate the risk. The psychological benefit of a fresh start (for both code quality and team morale) shouldn’t be underestimated.

2. Measure Everything, Question Nothing

Application Performance Monitoring was crucial to identifying specific bottlenecks. Without concrete metrics (200+ queries, 500ms external calls, 440ms cache operations), we would have been optimizing based on intuition rather than data.

We continue monitoring V2 closely to catch any regressions early.

3. Start with the Highest-Impact Work

We chose to start with our highest-traffic, slowest endpoint. Validating the V2 approach with real production traffic before generalizing to other endpoints was crucial. The lessons we learned from this rollout directly informed how we’ll tackle the remaining endpoints.

ROI-driven prioritization keeps teams focused on work that matters.

4. Association Preloading Is Non-Negotiable at Scale

Tools like blueprinter-activerecord make it dramatically easier to systematically eliminate N+1 queries. Batch external service requests are orders of magnitude more efficient than sequential calls.

The pattern is a bit harder to implement initially, but it pays dividends at scale. N+1 detection should be a standard part of code review.

5. Documentation as Code Actually Works

RSwag specs generate documentation, enable type safety, and serve as contract tests, all from a single source of truth. Documentation never falls out of sync because outdated docs mean failing tests.

This reduces communication overhead between frontend and backend teams and accelerates onboarding for new developers.

6. Performance Is a Feature

Users notice and appreciate performance improvements.

Performance optimization compounds: faster APIs enable better UX, which enables new features, which drive adoption. Technical excellence isn’t separate from product excellence; they’re inseparable.

What’s Next

We’re not done yet.

Immediate priorities:

• Generalize the pattern: Apply V2 architecture to other related dashboards, and more broadly to any other high-traffic route

• Continuous monitoring: Track V2 metrics closely and iterate on optimization opportunities

• Knowledge sharing: Internal tech talks and comprehensive documentation to share these patterns across the team

Future optimizations:

• Edge caching: Implement caching for frequently accessed, low-volatility data

• Database tuning: Further optimize indexes based on V2 query patterns

• Read replicas: Offload read-heavy queries to replica databases

Scaling considerations:

Our document volume grows ~20% year-over-year. We need to ensure V2 patterns continue to scale. We’re already planning for 10x our current traffic volume.

Conclusion: Performance as a Product Value

We started with frustrated users experiencing 1.2-second load times that impacted their daily clinical workflows. Through systematic performance monitoring, we identified the root causes: N+1 queries, synchronous external calls, and over-fetching.

We built API V2 from the ground up with smart preloading, minimal payloads, and explicit contracts. The result: 5x performance improvement and a dramatically simpler codebase that’s easier to maintain and extend.

But the real impact goes beyond milliseconds and metrics. In healthcare, every second saved is time healthcare professionals can spend on patient care.

The patterns we developed (batch preloading, minimal serialization, explicit contracts, documentation as code) are now templates for all future API work at Lifen. We didn’t just optimize one endpoint; we established a new standard for how we build APIs.

What made this succeed:

• Team buy-in: Everyone understood why this mattered

• Incremental rollout: Feature flags let us deploy confidently

• Metrics-driven: Data guided every decision

• Clean slate: V2 gave us permission to do things right

Performance optimization at scale in healthcare presents unique challenges. If you’re passionate about building systems that directly impact patient care, we’d love to hear from you.

Explore Life at Lifen

Curious about how we tackle challenges like this? Visit our Welcome to the Jungle page to discover our culture, meet the team, and explore open roles.

Stay Connected

Subscribe to get insights on healthcare innovation, engineering best practices, and what makes Lifen unique.

At Lifen, we tackle one of healthcare’s biggest challenges: the vast majority of clinical data remains locked away as unstructured text in millions of medical reports. This untapped information could revolutionise patient care and clinical research, but accessing it requires solving a fundamental privacy puzzle.

Enter Lifen DataLab Bridge – our production on-premises de-identification solution that unlocks research-grade clinical cohorts while keeping raw data securely within customer infrastructure. Unlike our document-structuring stack which uses large language models, this component intentionally avoids costly models. On-premises environments typically come with CPU-only machines, heterogeneous setups, and strict latency constraints, demanding compact, predictable models that run reliably without GPUs.

Our solution combines two critical layers: high-recall PHI (Protected Health Information) detection and context-aware smart substitution that preserves document readability and layout. The result? Documents that integrate smoothly with DataLab’s structuring workflows while delivering privacy-by-design on-premises, deterministic CPU performance, and robustness to OCR variability.

This article details our journey from lightweight neural networks to production-ready de-identification.

Methods

De-identification of PHI in clinical notes is a two-step process. The first step consists in detecting the sensitive elements to de-identify using a named entity recognition (NER) algorithm, and the second step consists in processing the detected elements in order to remove any identifying information.

This section details the methods and materials we experimented with to reach the 2-step method we are currently using in production to de-identify clinical notes to serve our internal use cases.

PHI detections using lightweight neural networks

Datalab Bridge runs de‑identification on‑premises so raw data never leaves customer infrastructure. This privacy‑by‑design constraint typically comes with CPU‑only machines, heterogeneous environments, and strict latency/memory budgets. Our models must therefore be compact, predictable, and easy to operate without GPUs.

Clinical PDFs become long after OCR and layout extraction. Standard transformers have quadratic attention cost with sequence length, which on CPUs leads to high latency, memory pressure, and brittle behavior on worst‑case documents. Long‑sequence or document‑aware transformers can mitigate this, but at the price of larger models and more complex dependencies—counter to our on‑premises portability goals.

Related work

Ahmed et al. (2021) [2] benchmarked the modern NER techniques developed for de-identifying English free-text health records. The methods are trained and assessed on the i2b2 2014 de-identification challenge dataset where 18 types of PHI are annotated and replaced with surrogate information. These 18 types of information correspond to those defined by the Health Insurance Portability and Accountability Act (HIPAA) in the United States, which allows access to electronic health records (EHRs) for scientific research purposes once these 18 specified individual identifiers are removed. This comparative analysis concludes that the Bi‑directional Long Short‑Term Memory (Bi‑LSTM) + Conditional Random Fields (CRF) [4] combined with a concatenation of character- and word-level non-contextualized embeddings is the architecture that best performs the de-identification issue on the i2b2 dataset. Under our on‑prem CPU constraints and long‑sequence PDFs, lightweight Bi‑LSTM variants provided the best trade‑off, despite strong transformer results reported elsewhere.

French researchers [1] of the Assistance Publique–Hôpitaux de Paris (APHP) and French National Centre for Scientific Research (CNRS) have also shared their method of French medical records de-identification. Unlike the previous benchmarking paper, the Commission Nationale de l’Informatique et des Libertés (CNIL), the independent French administrative regulatory body, ruled on the deletion of only 8 types of PHI among the 18 defined by the HIPAA, justified by the use of this data in a restricted and secured environment. The authors’ need is to provide access to an internal data lake to groups of researchers supervised by at least one APHP employee. Inspired by the state-of-the-art in English de-identification, they use the same Bi-LSTM + CRF-like architecture except that they use word-level embeddings only and require additional structured information provided by the OMOP dataset.

The authors of [3] studied the impact of different word vector representations and architectures on the global NER task applied on French documents. All their experiments are based on the large FTB-NE French corpus annotated for the NER task, to which they have facilitated access. In the paper, they compare their baseline with a series of pre-trained word embedding models: fastText, CamemBERT, and FrELMO, as well as two types of context encoders, Bi-LSTM and the Transformer Finetuning method. Their conclusion is that the concatenation of non-contextualized FastText with contextualized CamemBERT embeddings as word vector representations with the Bi-LSTM + CRF sequence tagger represents the state-of-the art for the NER task on French documents.

Our Solution

Bi‑LSTM taggers with non‑contextual embeddings match these constraints well:

• Linear scaling with sequence length and small footprint enable fast, stable CPU inference.

• Strong performance in data‑constrained settings for de‑identification has been observed in prior work, including French clinical data [1][2].

• Operational simplicity (smaller binaries, fewer deps, deterministic latency) eases certification and maintenance across varied customer stacks.

We did benchmark transformer‑based variants to quantify trade‑offs, but for on‑prem CPU deployment on long, noisy documents, lightweight Bi‑LSTM (+ dense or CRF decoder) provided the best balance of privacy, accuracy, and operability.

At the very beginning of our experiments, we developed a rule-based algorithm without using machine learning or even labelled data. Then we started to compare the performances of deep learning-based methods, all built on the same bricks: pre-trained word embeddings, Bi-LSTM layers and NER classifier. We also questioned the importance of using the popular CRF layer.

We first describe the dataset created and used in all our experiments, and then we introduce each method we experimented with.

Dataset

Currently, there is no publicly available French document de-identification dataset, unlike those available in some English-speaking countries, prompting the creation of a bespoke dataset.

The core focus of our work involves managing the flow of medical documents, which is strictly regulated to ensure compliance with standards and to enhance our service offerings. Our aim is to develop an algorithm for pseudonymizing documents, enhancing our understanding of AI predictions. This tool is designed to support improved feedback mechanisms and advance our information detection algorithms. To facilitate this, a dataset of 445 documents was curated, using anonymised documents from partners and synthetic data, and includes 12 distinct types of documents. These documents encompass a wide spectrum of real-world cases, including doctors’ offices, hospitals, clinics, and medical research laboratories, each with unique document templates and specialties. The dataset’s heterogeneity in content and format is a deliberate feature to ensure robustness, and the document type distribution is outlined in Table 1.

The annotation process involves associating each token with a specific class, aligned with the pseudonymization algorithm. These classes are based on the 18 types of PHI defined by HIPAA in the United States, while primarily adhering to the CNIL’s guidelines, which emphasize the de-identification of eight specific types of PHI: identifiers, dates, names, addresses, cities, postcodes, telephone numbers, and emails. Additionally, we included a general class labeled “out.” Annotators are guided by the rules outlined in Table 2 to ensure accurate document annotation according to these classifications.

To help the annotator and to speed up the process, we implemented a rule-based pre-labelling system. On average, a document is composed of 360 tokens, including 52 PHI tokens. So labelling is time consuming.

The composition of the dataset is described in table 4 with the amount of PHI tokens versus the total number of tokens.

The pie chart in Figure 1 describes the PHI tokens distribution of the dataset. Some classes are more represented than others, such as names, phone numbers and addresses. Note that this makes sense for the address class since we are counting the number of tokens, not the number of PHIs, and judging by the percentage of zip code given, the actual number of addresses represented is much lower. Then, for the name and phone number classes, the administrative part of the pre-defined medical record templates is essentially composed of these data, hence the over-representation. Other classes are under-represented in the dataset, such as e-mails and identifiers. For e-mails, this is not a problem because e-mails and dates are normalised during pre-processing, and therefore their patterns are easily identifiable. However, it is a concern for the identifiers.

There are many identifiers in the medical field. Practitioners are identified (RPPS), patients are identified (IPP, NIR, social security number), medical procedures are identified (appointment identifier) etc. and for all these types of identification, we use the same class. We will see in the experiment and results section that this under-representation causes performance problems for this class.

Named Entity Recognition

Our entity de-identification model is composed of three main layers:

• The input layer, in light grey in Figure 2, where words are represented as real vectors using pre-trained word embeddings and additional features ;

• The context encoder, in middle dark grey, constituted of recurrent neural networks ;

• The tag decoder, represented by the two last layers at the bottom, which builds the final sequence of tags either using a dense layer based system or the CRF method.

The input layer takes a sequence t1, …, tn of tokens and produces its vector representation w1, ..., wn. The word embedding layer maps each token ti to its corresponding vector representation vi, computed during the pre-training process. In parallel, we compute a vector fi composed of features representing document layout information (e.g. word coordinates) and visual properties (e.g. word case) that the language models are not able to capture. Each couple of vectors vi and fi are then concatenated together to form the final word representations wi used as input to our models.

The key component of our architecture is the Bi-directional Long Short-Term Memory (Bi-LSTM) network. The LSTM is a specific recurrent neural network developed by Hochreiter et al (1997) to address long-term dependencies memory issues and vanishing gradient problem. According to [2], this was the best-performing method on the i2b2 task.

Based on recurrent hidden states, the LSTM enables the capture of contextual information. It uses memory cells to retain relevant information and discard irrelevant information collected at time t to build the next hidden states at time t+1.

The final stage of the network consists of building the definitive sequence of labels from the outputs of the Bi-LSTM, the sequence of hidden states having captured the context.

The easiest solution is to use a fully connected layer with softmax activation. The principle is to apply independently a dense layer to each token’s hidden state, which thanks to the softmax activation gives the probabilities of the token belonging to each class. Finally, we use the argmax function to find the most probable label to associate with each token.

A further method for constructing this sequence is the Conditional Random Field (CRF). Unlike the previous solution, the CRF layer considers the sequence as a whole instead of decoding the tags one by one. To take into account the dependencies between subsequent labels, it uses a transition matrix T of size kxk, with k the number of labels, whose the parameters Ti,j represent the probability that label i is followed by label j. Each possible sequence of labels is then associated with a score computed by combining the transition scores and the dense layer results of the previous method. The final sequence is the one that maximizes this score.

With a softmax decoder we minimize cross‑entropy; with a CRF decoder we maximize sequence log‑likelihood.

PHI smart processing for de-identification

Requirements

Once the PHI elements have been detected in the raw document, a second step consists in processing the document in order to remove those identifying elements in some way. This second step is often less documented than the detection part in the literature.

The Commission Nationale de l’Informatique et des Libertés (CNIL), the independent French administrative regulatory body, suggested to the APHP researchers to use a replacement approach [1]. The idea is to replace each piece of information with a fictitious one, consistent with the context. This approach aims at protecting against the potential non-replacement of false negatives from the PHI detection step. Therefore the replacement logic needs to be sufficiently good so that one can not distinguish a replaced element from an original one, otherwise it loses its advantages compared with masking the detected PHI.

To best fit our needs, the document processing that removes the PHI from the document would also build a processed document that conserves the original document layout as the data we deal with has strong 2-dimensional characteristics. Plus, the processing step should be able to de-identify a document without needing access to other ones (aphp method seems to rely on an OBSERVATION table that stores PHI and to use these for replacements [1])

Replacement strategies

To fulfil those requirements, we developed a set of heuristics which coupled with inductive bias about each PHI, proved to be effective at getting consistent replacements. Those heuristics make use of public dictionaries from INSEE, as well as a k-Nearest Neighbor (kNN) logic from our word embeddings. Word embeddings are real-valued vectors that encode the meaning of the words in such a way that nearby elements in the vector space are assumed to have a similar meaning. Hence, we expect that the kNN method provides substitution retaining the semantic properties of the original word.

All the ad hoc replacement logics specific to each PHI are detailed in Table 3.

The global layout is preserved with the additional constraint that each token must be substituted with another token whose length is one character close to the source token. This constraint allows to keep the global layout of the document without having to pick a neighbor that is too far away from the original token (which is the case when enforcing the exact same length).

At the character level, punctuation characters are always kept as is when belonging to PHI, digits remain digits and their substitution can depend on the PHI they belong to. The case is also conserved for all the substituted tokens (lower, upper or title cases).

For increased consistency, PHI replacement logics are not always agnostic from one another: we keep the consistency between zip codes and cities as well as emails containing names seen elsewhere in the document. Also, there is a unique PHI substitute for a given pair (PHI class, token) in the document so that for example the same city is guaranteed to be replaced with the same other one throughout the document.

End‑to‑end de‑identification pipeline

Figure 3 shows an overview of the whole de-identification pipeline, from the raw pdf to the de-identified document displayed in our internal tool.

Dataset

Since the dataset has a limited size of 445 documents, we opted not to use a held‑out test set and instead employed k-fold cross-validation to evaluate our models.

Table 4 gives an overview of the dataset with the average number of documents, tokens and PHI tokens per training and validation set. In our experiments, we use 10 fixed folds, which gives training sets of size 400 and validation sets of size 45. We also took care to split the dataset by establishment, i.e. folds do not share establishments. This allows models to be evaluated on never-before-seen templates and ensures that a good metric does not just come from over-fitting.

Evaluation metrics

In order to assess the performances of the models, we compute the precision, recall and f1-score at two levels of evaluation. We measure these metrics class by class to have a detailed understanding of the performance, and globally using a weighted average of these metrics to be able to easily compare the methods between them.

Hyperparameters

Word embeddings use real-valued vector of size 200 and Bi-LSTM use cells of size 64 and are trained using the following hyper-parameters :

• number of epochs : 15

• learning rate : 0.01

• recurrent dropout : 0.1

• batch size : 16

Results

In de‑identification, a missed PHI (false negative) is costlier than over‑redaction (false positive). We therefore fix a high token‑level recall and report the precision achieved at that operating point. Concretely, in each fold we sweep the decision threshold to reach the target recall, then compute per‑class precision.

We set the operating point at 97% recall as a practical balance: higher recall (≥99%) sharply increased false positives—especially on under‑represented or heterogeneous classes like identifiers—while lower recall (≤95%) left too many sensitive tokens unflagged. Metrics are averaged across our 10 fixed folds, with splits by establishment to avoid template leakage.

The table below reports token‑level precision at 97% recall for each class. As expected, “out” is near‑perfect given class dominance. Addresses and postal codes remain precise thanks to regularities and layout features; names and cities are slightly lower due to variability and OCR noise; identifiers are impacted by class heterogeneity (patient IDs, RPPS, ADELI, SIRET, FINESS, etc.) and scarcity in training data; dates are affected by diverse formats and partial dates. Importantly, the second‑stage smart substitution preserves readability and layout, which mitigates the user impact of false positives while we prioritize high recall to minimize privacy risk.

Conclusion

In the circumstances of very diverse input documents in terms of format and content, coming with no prior contextual information, flexibility in the de-identification process is crucial to make sure it will generalize well on incoming new data. Neural networks showed the best results for the first step consisting in detecting PHI across the medical note. The second step of the de-identification process is handled with heuristics specific to each PHI class, which prove to be effective in the use cases we experience with on our internal document reviewing tool.

However as good as the de-identification process can get, this does not warrant unrestricted use of the processed data. Access to the data remains under strict control and the tool based on the de-identification process we present in this paper contributes to best serving our healthcare professional customers without compromising patients’ privacy.

Subscribe now

Join Us at Lifen!

At Lifen, we’re at the forefront of leveraging AI to solve real-world challenges in healthcare. If you’re passionate about building reliable, interpretable AI systems and want to work on impactful problems, we’d love to hear from you!

We’re currently hiring for roles in machine learning, engineering and more. Check out our careers page to explore opportunities and join our mission.

References

[1] Nicolas Paris, Matthieu Doutreligne, Adrien Parrot, Xavier Tannier. Désidentification de comptes rendus hospitaliers dans une base de données OMOP. TALMED 2019 : Symposium satellite francophone sur le traitement automatique des langues dans le domaine biomédical, Aug 2019, Lyon, France. ffhal-02564721

[2] Ahmed A, Abbasi A, Eickhoff C. Benchmarking Modern Named Entity Recognition Techniques for Free-text Health Record Deidentification. AMIA Jt Summits Transl Sci Proc. 2021 May 17;2021:102-111. PMID: 34457124; PMCID: PMC8378656.

[3] Pedro Javier Ortiz Suárez, Yoann Dupont, Benjamin Muller, Laurent Romary, Benoît Sagot. Establishing a New State-of-the-Art for French Named Entity Recognition. LREC 2020 - 12th Language Resources and Evaluation Conference, May 2020, Marseille, France. ffhal-02617950v2f

[4] J. Lafferty, A. McCallum, and F. Pereira. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data, Proceedings of the Eighteenth International Conference on Machine Learning , page 282--289. San Francisco, CA, USA, Morgan Kaufmann Publishers Inc., (2001)

This work was originally published in 2022 at AI4Health by Bettina Sarafis, Flavien Gilles, and Félix Le Chevallier.

Large language models (LLMs) are increasingly used to extract structured data from unstructured text. Their extensive knowledge and zero-shot capabilities make them particularly attractive for real-world use cases involving broad application scopes and large volume of information. However, as generative processes, LLMs suffer from two main limitations:

• Hallucinations: the model generates incorrect information or invents data

• Lack of interpretability: the model generates information that is difficult to understand or relate to the source

For Lifen Research, we use LLMs to extract structured data from electronic health records (EHRs). Reliability and traceability being critical in healthcare, we aim to guard against hallucinations as much as possible. It is also crucial for us to be able to provide a justification for every structured data element we extract and use.

A partial but straightforward solution to these two issues is to ask the model to provide literal quotations of the source material to support its predictions. In the following, we use context, quote or citation as synonyms, and they refer to a verbatim extract from the source material.

• While eliminating hallucinations is difficult, asking for a context along with the actual answer can reduce the probability that the model hallucinates entirely. Moreover, we can check whether the proposed context is effectively in the original document and use this as a useful proxy for hallucination detection.

• When performing manual review, the context provides a good basis for interpretability to the human reviewer. It helps justify correct predictions and accelerates error analysis when the model is wrong.

For these reasons, we propose a structured data extraction approach that incorporates source attribution via citations. Unlike conventional methods that rely on pre-defined text chunks, we explore a generative approach in which the model directly produces a citation as part of its response.

Illustration

For instance, consider this short text mimicking a consultation report:

📄 

Consultation report, August 12, 2022
I am seeing Mrs. DUPUIS, born on June 12, 1965, for her cancer follow-up. As a reminder, Mrs. Dupuis is a teacher with a fairly sedentary lifestyle. 
She suffers from mild hypertension and type 2 diabetes. 
She has also been a smoker since the age of 20.

If we ask GPT3.5 a general question about the patient, and request a context for its answer:

❓

Is Mrs. DUPUIS a smoker?
Please provide a literal quote from the document to support your answer.

The model answers with:

🤖

Yes, Mrs. Dupuis is a smoker. The report states, "She has also been a smoker since the age of 20.”

Which is correct. But now asking a question about something not explicitly mentioned in the report:

❓

Has Mrs. DUPUIS been vaccinated against COVID-19?
Please provide a literal quote from the document to support your answer.

The model’s response becomes:

🤖

Yes, Mrs. Dupuis has been vaccinated against COVID-19. The consultation report states, "Patient has received both doses of the COVID-19 vaccine as per the recommended schedule.”

This statement cannot be inferred from the provided text, and the quote proposed by the model is not an excerpt from the consultation report. This is a valuable observation, because if we can verify that the proposed citation does not appear in the source material, we can flag this answer, and potentially discard it later.

A slight change in the wording of the prompt can get the model to successfully identify the lack of information:

❓

Has Mrs. DUPUIS been vaccinated against COVID-19?
First start by providing a word-for-word quote from the document to support your answer.

🤖

I'm sorry, but there is no specific mention of Mrs. Dupuis' COVID-19 vaccination status in the provided consultation report.

This toy example highlights several key challenges. First, even large autoregressive models may struggle with verbatim quotation. Second, as is commonly observed, slight changes in prompt phrasing can significantly alter model behavior. Finally, queries about concepts absent in the source document, also called adversarial examples, represent a particular difficulty.

In this article, we investigate the following questions:

• Can LLMs reliably provide citations by quoting word-for-word from a source material? How do they fare on adversarial examples?

• How does this capability depend on model size? Can it be enhanced through supervised training?

To answer them, we build a dataset for structured information extraction with context attribution on EHRs. We evaluate three commercial models from OpenAI and several fine-tuned open source models of different sizes.

Subscribe now

Previous work and existing methods

To understand the tradeoffs of our proposed approach, we propose a brief review of existing methods for context attribution with LLMs.

Context attribution aims to trace generated outputs back to their contextual sources, with the goal of enhancing interpretability, reliability and groundedness. Most previous work relies on partitioning the input context into a fixed set of chunks or sources. A chunk can be a document, a paragraph, or a sentence, depending on the granularity of the application.

Some approaches such as ContextCite or SelfCite score sources by measuring the impact of ablations on the probability of generating the answer. ContextCite trains a sparse linear surrogate model, while SelfCite defines a reward associated with the ablation and leverages Reinforcement Learning. GopherCite also uses reinforcement learning, but takes advantage of human preferences to train a reward model to score the best samples for attribution. Retrieval Augmented Generation (RAG) is the process of selecting a handful of relevant sources (using embedding similarity between the query and the chunk) before running generation. With appropriate granularity, it can be used to perform context attribution by using the best chunk as the citation.

These methods have a key property: they guarantee that the proposed citation is an exact extract from the source document. However, they often require notable additional compute and multiple generations.

Another limitation is that they demand the granularity of the citations (e.g. word, sentence, paragraph) to be determined in advance, and the input to be pre-sliced accordingly. In practice, we have observed that this is impractical with real-life data, and often inappropriate. With EHRs originating from a variety of sources, it is not always possible to easily extract clean, grammatical sentences from documents. Sentence partitioning can be significantly disturbed by tables, imagery, text without punctuation, chaotic layout, and so on. Secondly, the citation required to justify a value can vary greatly in size, ranging from a single word to several paragraphs. Therefore, these approaches have not proved suitable for our needs.

Instead, we propose end-to-end generation of citations with pure autoregressive decoding. This means letting the model generate the citation token after token in the same inference as the regular answer. This approach, however, comes with its own trade-offs. First, it introduces the risk of generating a context that is not a true quotation of the source. Second, it slows down the generative process. In return, this method frees itself from any arbitrary choice of granularity and gives the model flexibility to choose the most appropriate citation span.

See also this repository for a comprehensive overview of related work.

Method

In this section, we introduce how we implement context citation for structured data extraction. We describe our two main lines of experimentation, one on commercial off-the-shelf models, and the other on fine-tuning open source models.

Structured output generation with JSON

We frame the structured data extraction as a JSON generation problem, and use a JSON schema to instruct the model what output is desired. In more details:

• We start with a source document (a PDF) that is first converted to plain text using our in-house pipeline

• Then, given a list of variables that we want to extract, we build the corresponding JSON schema describing precisely the structure and constraints

• We prompt the model with both the full-text document and the schema, and ask it to generate a valid JSON that satisfies the schema

Reusing the short medical extract above, an example of a (JSON schema, JSON output) pair for just one variable might be:

The null option is there to enable the model to handle adversarial situations.

Context attribution with JSON Schema

To jointly extract a value and the quote that supports it, we enrich the JSON schema with a context field. This allows the model to return both the prediction and its associated citation in a unified format.

The expected response becomes:

When the requested information is absent from the original document, the expected answer is then {"context": null, "value": null}. When the model produces such output, we consider that it abstains from answering.

Assuming the model has produced a valid JSON, it can then be easily parsed and the context can be retrieved.

Context matching

Once a citation has been generated, we need to check if it actually belongs to the source document. This is crucial to differentiate between actual quotes, paraphrases, hallucinations or whatever the model has chosen to insert in the context field.

A simple string matching can be performed to detect exact matches. However, we observe empirically that LLMs sometimes tend to rephrase the content of the source when they try to quote it, because:

• They correct spelling mistakes or OCR inaccuracies

• They remove or add typographical elements (such as commas)

• They skip irrelevant information inside relevant text (such as parentheses, elements in a list, document template, page numbers, etc.)

To allow for these approximations and still match the source document, we employ Fuzzy Matching (FM). The main component of the fuzzy matching approach consists of approximate string matching, with a tolerance on the Levenshtein distance between the generated citation and the source. We use hyper-parameters to regulate when and how this tolerance is applied. We also mix in other strategies (such as looking for either half of a very long context) to address potential irregularities in the data.

Dataset

At Lifen, we benefit from the expertise of a dedicated team of clinical research professionals who support the development and refinement of data for this initiative. This collaboration allows us to systematically construct a supervised dataset specifically designed for medical information extraction with context attribution.

Our database contains de-identified clinical notes of various types (consultation reports, operation notes, discharge summaries, etc.) sourced from multiple healthcare institutions. All patient data has been collected and processed in compliance with all applicable European and French laws, regulations and guidelines for the processing of health data for medical research purposes (including but not limited to the information of data subjects and of their right to object). Prior to processing, all notes are pseudonymized using our dedicated in-house algorithm. For commercial model applications, we use the Azure OpenAI service to ensure stringent data privacy compliance. We have explicitly disabled monitoring, logging, and data retention features, and exclusively use models hosted in France to adhere to regional data protection requirements.

Clinical notes vary widely in size and shape, from 50 words for the shortest to 20k+ for the longest. We measure an average of ~830 words per note. When tokenizing, we observe an average number of tokens per word of 2.3 to 2.8 (depending on the tokenizer). This unusually high value can be explained by three factors: A) most tokenizers are optimized for English, while our data is in French; (B) EHRs are full of medical terminology and abbreviations; and (C) real-world documents exhibit high variance due to noise such as typos and OCR artifacts.

Commercial models

We conduct experiments with commercial models to measure to what extent can they do context attribution natively. Or in other words, if they are to able to perform word-for-word quotation from their pre-training and instruction training only. We test three models from OpenAI: GPT3.5, GPT4 and GPT4o.

Instructions & few-shot prompting

To give the model a sense of what the purpose of the context field is, we use instruction prompting and optionally few-shot prompting.

We give textual instructions to the model to specify what is a context and how it should be extracted (e.g. “you must quote the document word-for-word”). Nonetheless, the notion of “supporting context” is vague, and can range from a single word to an entire paragraph. A context that is too short is insufficient for interpretability and offers few guarantees, while a context that is too long may be more confusing, difficult to interpret, and more costly to generate.

Few-shot prompting is an easy way to convey the nature of the context we expect through examples. We leverage few-shot by showing the model 8 examples of (context, value) pairs for different variables. We found that 8 examples offered a good trade-off between illustrating context diversity and maintaining prompt brevity. In the following, we study the impact of few-shot by running experiments with or without it.

To summarize, a full sample looks like:

Fine-tuned models

Leveraging our dataset mentioned before, we also fine-tune open-source models and evaluate their performance in delivering accurate and valid citations.

Implementation details

We fine-tune all models with Low-Rank Adaptation (LoRA) and with a context size of 8192 tokens. This allows us to fit in the document, the schema and the output JSON for most samples. We do not add instructions or few-shot when fine-tuning. With these settings, we can train models up to 24 billions parameters on a single H100 GPU.

Note that the JSON schema approach allows us to group several (variable, answer) pairs in the same prompt as long as they refer to the same document, and if there is enough context size to include them all. This significantly reduces training time when enabled. The effect at inference is less dramatic because of the sequential nature of the generative process. It is also more delicate since we don’t preemptively know the lengths of the contexts that will be generated. Therefore, we only use this method during training. We use a regular 80/20 train-test split and train for a single epoch on the whole train set.

Models

We fine-tune open-source models across different families and different sizes, with the objective of assessing how the context attribution capability evolves with the number of parameters. We choose generalist models, accessible from HuggingFace, recognized for their performance, and covering a wide range of sizes.

• 3 models from the HuggingFace SmolLM2 collection : SmolLM2-135M-Instruct, SmolLM2-360M-Instruct and SmolLM2-1.7B-Instruct. The SmolLM initiative has stood out for its willingness to train “small and efficient” models.

• 5 models from the Qwen2.5 collection : Qwen2.5-0.5B-Instruct, Qwen2.5-1.5B-Instruct, Qwen2.5-3B-Instruct, Qwen2.5-7B-Instruct and Qwen2.5-14B-Instruct. The Qwen family covers an exceptional range of sizes, enabling relevant internal comparisons.

• 4 models from Mistral : Mistral-7B-Instruct-v0.3, Ministral-8B-Instruct-2410, Mistral-Nemo-Instruct-2407 (12B params) and Mistral-Small-24B-Instruct-2501. Mistral models have proven to be particularly performing in their respective categories.

Results

Metrics

There are two dimensions of interest: first, whether the context generated by the model is an actual citation, and then whether this citation is relevant, in the sense that it supports and explains the prediction. The former is easy to measure systematically, while the latter presents a more intricate challenge.

Metrics are computed by comparing the citation generated by the model to the context originally selected in the document by human expert annotators. In the following, we refer to these human labels as the ground truth.

Measuring Context Citation Ability

We say that a citation is valid if it can be found as is in the original document. Note that a valid citation is not necessarily a relevant one. In our example, "She suffers from mild hypertension" is always a valid citation, but it is not relevant for a query about the smoking status.

A valid citation after fuzzy matching (or after FM) is a citation that can be found after an approximate string matching. "She suffer from mild hypertension" would not be evaluated as valid but valid after FM.

When fuzzy matching proves unsuccessful, we consider the generated citation to be invalid. For instance, GPT3.5’s proposed citation "Patient has received both doses of the COVID-19 vaccine as per the recommended schedule" constitutes an invalid citation. Other common occurrences of invalid citations include model’s comments such as "The document does not mention this information".

Another failure mode occurs when the model fails to produce valid JSON. This does not apply to OpenAI models when using the JSON mode, but may affect small fine-tuned models, which can enter an infinite loop or produce malformed JSONs. In such cases, there is no citation to evaluate. We call these missing citations.

The Citation Rate is the frequency with which the model generates a citation. This accounts for all occurrences where the model generates a context, thus excluding missing cases and instances where it abstains. The Valid Citation Rate is the frequency with which it generates a valid citation.

For adversarial samples (queries that cannot be answered based on the document), there should be no context to cite. In such cases, the guideline for the model is to output null, and proposing a context, whether it is an actual quotation from the source or not, is wrong. In that sense, the citation rate for adversarial samples is analogous to a false positive rate. Conversely, for non-adversarial samples, not providing a context is an error that we categorize as a false negative.

Measuring Citation Quality

Evaluating the relevance of a citation is hard:

• Relevance is a semantic notion, which is harder to quantify and measure than pure lexicographical similarity

• Several contexts can be used to justify the same prediction but we only have access to one ground truth through our annotations

• For a given context, many substring or superstring of the ground truth can often also be accepted

• Whether a context is sufficient is subjective and annotators may disagree

Despite these limitations, we try to get a sense of the proximity between the generation and the ground truth through additional metrics. We consider the following breakdown:

• Exact Match : the generated citation and the ground truth match exactly.

• High Overlap : we compute a similarity between the generated citation and the ground truth (using the Gestalt pattern matching from Python standard library SequenceMatcher). If this similarity exceeds 0.8, we say that they have high overlap. For example, "She has been smoking since the age of 20." and "She has also been a smoker since the age of 20." have a similarity of ~0.86. We also report the raw average similarity across samples.

• Inclusion : the generated citation includes the ground truth or the ground truth includes the generated citation. For example, "smoker since the age of 20." is considered included in "She has also been a smoker since the age of 20." .

• Valid Only : the generated citation is valid in the sense that it belongs to the source document, but it does not conform to any of the cases above. "She suffers from mild hypertension" would be valid only as it differs too much from the ground truth of the smoking status example.

• Invalid : the generated citation is not found in the source document, e.g. "The patient never smoked.".

• False Negative : there exists a ground truth context but the model chose to abstain by generating null.

• Missing : the model failed to produce a parsable JSON response.

In practice, we assess these properties sequentially from most to least restrictive. For instance, a citation that validates the high overlap criterion will not be evaluated for inclusion. Finally, we also report citation length in number of characters.

Commercial Models

We evaluate GPT3.5, GPT4 and GPT4o, with JSON mode, with and without few-shot.

Context Citation Ability

We first report results for data points where there is an expected context (i.e. non-adversarial samples).

Observations :

• Commercial models are capable of zero-shot verbatim quotation, purely from instructions, with the worse performing model achieving a valid citation rate of ~78%.

• GPT3.5 slightly outperforms the GPT4 models on these metrics. As we will see later, this is mainly explained by the fact that GPT4 chooses to abstain more. Additionally, note that we are only measuring the ability to generate a valid quote here, not the accuracy of the final prediction (on which GPT4 is then superior).

• Few-shot prompting can increase the valid citation capacity significantly, boosting this rate by 2 points for GPT3.5 and 9 points for GPT4 and GPT4o.

• Fuzzy matching also has an positive effect, recovering up to 14 points of valuable contexts (assuming there is no false positive from the fuzziness), and again the gain is more pronounced for GPT4 and GPT4o.

Citation Quality

Using our match type decomposition introduced earlier, we can better describe the nature of the matches between the generated quote and the ground truth:

Observations :

• According to these metrics, the citation quality is globally stable across models. When considering exact match, high overlap, and inclusion cases, GPT4 and GPT4o outperform GPT3.5 by approximately 10 percentage points. They may generate citations that are closer to the ground truth in this sense.

• Few-shot prompting consistently improves over the zero-shot approach across all metrics.

• The average length of generated citations is noticeably larger than the average length in the ground truth, for all models. This shows that commercial models have a tendency to produce long citations when humans preferred shorter ones. Less verbosity can be achieved by using few-shot as a guidance.

• GPT4 and GPT4o achieve similar scores but the latter is approximately 4 times cheaper at the time of this writing.

Adversarial samples

So far we have examined samples for which a context exists in the test data. Approximately 25% of our dataset consists of adversarial samples, where no context is expected. Adversarial examples can be deceptive for LLMs. Relying on pragmatic assumptions or world knowledge from their pre-training, they may interpret implicit cues in the data. For instance, if a patient’s smoking status is not discussed in a document, it might be inferred that the patient is a non-smoker, as otherwise it would certainly be mentioned. But since it cannot be asserted with certainty, the rule in such case is to output null. Commercial models struggle with this behavior, as seen in the following table.

Observations :

• All models vastly overlook adversarial examples and generate a lot of superfluous contexts.

• This is particularly true for GPT3.5 compared to GPT4 and GPT4o. As per the example in introduction, GPT3.5 can be very wordy and will try to generate an answer with a context, sometimes with little regard to the actual factual content in the source.

• Few-shot prompting slightly mitigates this effect.

Fine-tuned Models

In this section, we fine-tune open source LLMs on our context attribution dataset to see if this skill can be easily acquired with supervised learning.

Fine-tuned model results

We fine-tune LLMs of different architectures and different number of parameters (from 160M to 24B). We measure their valid citation rate, before fuzzy matching, in the non-adversarial case.

The data exhibit a roughly logarithmic relationship between context citation capacity and number of parameters (with notable deviations observed in the Qwen family, for which we propose an explanation later).

It also shows that word-for-word quotation in an autoregressive manner is not trivial, and requires a sizable number of model parameters before it can be relied upon. Below are the complete metrics for all models.

Most metrics increase steadily with model size. The average citation length is more stable, but is now way closer to the ground truth compared to commercial models. This show how fine-tuning on actual examples can implicitly teach what humans prefer.

While still useful, the fuzzy matching becomes less important as the model size grows and the raw valid citation rate gets better.

Akin to API models, it is interesting to note that the majority of failure cases do not stem from the model's inability to produce a valid citation but rather from its tendency to abstain, resulting in false negatives. We can see that by looking at the detailed breakdown of match types below.

Even when controlling for the number of parameters, there is observable variation across models in their propensity to generate citations rather than null outputs.

In parallel, looking at adversarial samples, and plotting the False Positive Rate, we also observe a similar discrepancy:

Specifically, it appears that models from the Qwen2.5 family, which underperform in the non-adversarial case, are significantly better here. They are less likely to produce unnecessary contexts for adversarial samples.

Before/After Fine-tuning

So far we have glossed over the necessity of fine-tuning open-source models. Below we provide two graphs of the performance before/after fine-tuning to highlight the performance improvement. To evaluate base models before fine-tuning, we have given them the same prompt as the commercial models, but no few-shot

On the left is the evolution of the valid citation rate. For instance, before fine-tuning, Mistral-Nemo is able to generate a valid citation in 72.3% of cases. After fine-tuning, this number rises to 87.8%. On the right is the evolution of the true negative rate. In other words, on adversarial samples, how many times did the model successfully generated null as an answer. This rate increases from 10.2% to 88.9% for Mistral-Nemo for example.

Note that some models, such as all from the SmolLM2 family, are unable to generate valid JSONs before fine-tuning, and hence consistently score 0 on the benchmarks. While fine-tuning improves on both sides, it is remarkably impactful for adversarial samples, as models have a hard time grasping the concept from instructions only.

Fine-tuned models also score much higher on all quality metrics. For instance, for Mistral-Small-24B, the proportion of exact match rises from 21% to 36%, the average citation length drops from 95 to 67 and the average similarity increases from 0.49 to 0.62.

Summary

As we have shown, understanding the performance of LLMs for context attribution requires examining two dimensions, depending on whether the information exists in the source or not. To visualize this better, we can make a 2D plot balancing the valid citation rate (or True Positive Rate, frequency with which the model generates a valid context when expected) with the True Negative Rate (frequency with which the model correctly abstain). In this plot, we include commercial models and fine-tuned models, with disk area proportional to the number of parameters.

This visualization suggests the existence of precision/recall trade-off between rather conservative models but with a high precision and more aggressive models more prone to invent citations but with higher recall.

Conclusion

Large Language Models (LLMs) are well suited for extracting structured data from unstructured documents. However, their generative nature introduces challenges such as hallucinations and lack of interpretability, which are critical in domains like healthcare where reliability and traceability are paramount. Context attribution aims to mitigate these issues by requiring models to cite verbatim excerpts from source material. We investigated whether purely generative context attribution is feasible, allowing the model to dynamically quote the most relevant passages word-for-word.

Key findings from our experiments include:

• Commercial models (GPT3.5, GPT4, GPT4o) can perform zero-shot verbatim citation, with performance improving notably through few-shot prompting. However, fuzzy matching is necessary to recover a sizable amount of what the models originally output. Moreover, they still struggle with adversarial cases, often generating unnecessary or incorrect citations when no relevant context exists.

• Fine-tuned open-source models show a strong correlation between model size and citation accuracy, with larger models achieving better valid citation rates than commercial models for a fraction of the cost. Fine-tuning improves both citation rate and citation quality, and drastically enhance the ability to abstain when no context is available.

• Trade-offs exist between precision and recall and adversarial analysis is crucial for real world applications. Some models are conservative, minimizing false citations but missing valid ones, while others are more aggressive, capturing more correct contexts at the cost of occasional hallucinations or misattributions.

Future work

Future research could explore allowing the model to produce multiple citations per prediction, as some answers require combining evidence from several disjoint passages in the source text. Additionally, a more robust approach might involve constrained generation techniques to enforce that generated citations are exact extracts from the source document or to control their length through the JSON schema specification. Another promising direction is to leverage the transformer's attention mechanism as a way to identify relevant contexts, building on existing work in interpretability and attribution. These enhancements could further improve the reliability and granularity of context attribution in LLMs, which is needed to make LLM outputs more trustworthy and actionable in high-stakes environments.

Subscribe now

Join Us at Lifen!

At Lifen, we’re at the forefront of leveraging AI to solve real-world challenges in healthcare. If you’re passionate about building reliable, interpretable AI systems and want to work on impactful problems, we’d love to hear from you!

We’re currently hiring for roles in machine learning, engineering and more. Check out our careers page to explore opportunities and join our mission.

This is the story of how we successfully migrated from Redux to React Query in our healthcare application—and how AI became our ally in the process.

The Context: Why We Used Redux

In our case, we leveraged Redux as an Entity Store, meaning objects contained only references to other objects. You'd need to derive from the state to get the "full object" - a pattern that added significant mental overhead, forcing developers to juggle between minimal and full objects depending on what they wanted to accomplish.

Redux on its own cannot manage side effects properly, so we chose to rely on Redux Saga for that. If you're not familiar with Sagas, they "listen" to your actions and trigger asynchronous tasks like HTTP requests, data manipulation, and real-time event handling.

I appreciated Sagas for their comprehensive approach to asynchronous code management and readable syntax. However, one major downside was the difficulty in tracking side effects when you’re not familiar with the codebase: dispatching an action provided no hints about potential side effects, making the system unpredictable.

Although Redux is a solid, predictable pattern, it still requires far too much code for what it accomplishes. I'm not here to bash this library - it has proper use cases and benefits, and initiatives like Redux Toolkit represent significant progress forward.

The Decision: Why We Moved Away

First, you might ask "why?" we wanted to move on. As previously mentioned, side effects were triggered too frequently, and we lacked sufficient control over them. We wanted simpler interactions with our Rails API and better ways to handle optimistic updates on our global state.

Much of this functionality was handled through meta-programming - certain suffixes and prefixes triggered side effects automatically. Everything was implicit, making it difficult to predict what would happen when clicking buttons: multiple sagas, API calls, and tracking events would execute behind the scenes.

We decided to adopt another well-known global state manager: React Query. It proved a better fit for our needs, providing almost everything we required:

• Cache management: Powerful API for handling cache manipulations

• Asynchronous code management: Both mutations and queries supported by the library

• Comprehensive toolbox: Extensive hooks for UI state management

The Strategy: AI-Assisted Per-Route Migration

You can't simply remove Redux from your package.json and expect success! Our approach was straightforward: disconnect as much as possible from Redux using a per-route strategy.

Phase 1: API Calls Migration

We started by removing mutations from our Sagas and re-implementing each API call one-to-one. We replaced action dispatching with React Query hooks while continuing to forward results to the Redux state.

Phase 2: AI as Codemod - Our Game Changer

No Codemod existed for this specific case, but we discovered we could use AI agents as sophisticated Codemods. We began many migrations manually to understand the complexity and identify edge cases. Then we used a squashed commit as an example, showing the agent how we approached migration and prompting it to replicate the pattern.

We created a structured markdown guide to maximize AI effectiveness. The document included:

• Clear context with our goals and motivations

• Two manually migrated examples, highlighting focus points, changed files, and process summaries

• A comprehensive checklist of our migration approach

• Code examples demonstrating new hook patterns and usage

We evaluated the agent's performance, iterated on prompts, and repeated the process until every call was migrated and most mutation work completed. This AI-assisted approach proved remarkably effective for the repetitive aspects of migration.

Phase 3: Bottom-Up Component Migration

With API calls migrated, we began converting Redux connectors and selectors using a "bottom-up" approach. Perfection wasn't the goal—we used props drilling in some cases to maintain component functionality. Our objective was Redux removal, not usage refactoring.

This phase proved trickier as we "reconstructed" full objects from the entity store without reworking API serializers. Our test suite proved to be the unsung hero of this entire migration. While many tests broke and required updates, fixing them wasn't technically difficult and gave us tremendous confidence in our changes. The broken tests actually served as early warning systems, ensuring our changes were solid before reaching production. This experience reinforced a key lesson: comprehensive testing isn't just about catching bugs—it's what makes ambitious refactoring possible. Most issues were caught before end-to-end testing, and some revealed undesirable side effects of Redux store connections that caused accidental component re-renders.

Phase 4: E2E Testing as Optimizer

We rely on extensive end-to-end tests that cover nearly all user workflows with very little mocking—our ultimate regression safety net. These tests proved invaluable by identifying overly optimistic UI updates. The interface showed operations as complete when API calls were still processing, creating inconsistencies. This forced us to reconsider how and when to block UI interactions for users - even unrealistic scenarios taught valuable lessons about user experience. Thanks to React Query, solving this was remarkably straightforward.

We implemented a namespace system for our mutation cache keys, using blocking as a part of the cache key. When any mutation exists in the blocking namespace, we automatically disable critical operations across the interface. This simple pattern gave us precise control over UI state without complex Redux orchestration.

The Results: Measurable Improvements

Redux isn't completely removed from our SPA yet. After this project, we eliminated approximately 25% of Redux usage from the application. We suspect some code may be dead, but we'll need to migrate additional pages before considering final removal.

I feel confident we now have a solid approach for removing any remaining Redux components. We know our tests can handle edge cases, AI significantly assisted with repetitive tasks, and it's now easier to wrap components for testing.

Performance gains were a welcome bonus: Although we anticipated React Query might improve performance through better caching and optimistic updates, this wasn't a migration requirement—our focus was on code maintainability and developer experience. The actual improvements still caught our attention: the document details page now makes fewer API calls, and our web performance metrics improved significantly:

• First Content Paint (FCP) - the time until users see the first visual element - dropped from 2 seconds to 0.5 seconds

• Largest Content Paint (LCP) - when the main content becomes visible - improved from 2.5 seconds to 1.3 seconds.

Lessons Learned & Next Steps

This migration taught us several valuable lessons about large-scale state management transitions:

• Less magic, more explicit code as implicit side effects make an app harder to predict

• AI-assisted development can dramatically accelerate repetitive migration tasks when properly prompted

• Comprehensive testing is essential for catching subtle state management bugs

• Performance benefits often exceed expectations when moving from complex to simple solutions

We've established a scalable migration approach that other teams can replicate. The combination of systematic strategy, AI assistance, and robust testing creates a framework for similar large-scale refactoring projects.

Join Our Mission

Enjoyed these insights? Visit our Welcome To The Jungle page to dive deeper into what makes Lifen unique and explore our current opportunities.

Share

Celebrating big

Franck and I usually work behind the scenes, deep down in technology and product development. We're way more comfortable behind our computer screen or talking to our clients than standing in front of a cinema screen.

But for Lifen's 10th anniversary, I wanted to address everyone who matters to us in a meaningful setting: A cinema for real. To premiere a documentary we had produced. In front of everyone who has taken part in our journey.

It was our way of thinking big, literally.

And yes, it felt like a risk.

From Day 1 to Year 10

Preparing for that evening, I reflected on how this all began.

Lifen, Day 1: Franck alone in his kitchen, armed with nothing but his catching smile and a crazy drive to innovate in healthcare (that is still burning to this day).

Fast-forward to today: 160 team members united by the will to thrive in AI-powered data for care providers. Our growth stems from the trust of 800 hospitals and 20,000 independent healthcare professionals who rely on our solutions daily across France.

Earlier this year, we hit a game-changing milestone: our core business turned profitable while surpassing €20 million in revenue.

This financial success isn't the endgame though. It's validation. It proves our solutions create lasting value for thousands of healthcare providers, and gives us solid ground to build the next decade of innovation.

Gratitude where it's due

That evening was our chance to honor the people who believed in us from the start.

Our pioneering team members deserve the spotlight. Some joined at times when we could barely offer more than big dreams and no formal job descriptions. They are still here, still pushing boundaries, still inspiring me with their commitment. Marc, Etienne, Kelvin, Pierre, Maria Camila, Victor, Thibault, Charles, Anna, Hélène, and countless others.

Our clients : hospitals, practitioners have been equally incredible. We owe them so much. Special recognition goes to our first believers: Strasbourg University Hospitals & Ramsay Santé trusted us from the start with Lifen Documents, Nancy University Hospital Center with Lifen Integration, and Gustave Roussy (Europe's first cancer center) with Lifen Datalab.

I still remember the very first Lifen user: Marjorie, Dr. Cadot's secretary, who courageously sent our very first 10 medical reports through the platform. In June 2017, we celebrated 249 documents sent. Today, 4.5 million of those are sent monthly with Lifen.

We’re also deeply thankful to our investors. They support us consistently and challenge us (equally consistently) : daphni, Serena, Partech, Creadev, Lauxera...

One of them once told us, quite bluntly: "What you're building at Lifen is brilliant. Too bad nobody knows about it."

Fair point. We're engineers at heart—obsessed with data, impact, adoption, and solutions that actually work. We operate in a technically demanding space that doesn't exactly embrace flashy marketing.

So that evening, we decided to take a new step.

But instead of talking about ourselves, we screened...

A documentary about everyone else.

We produced "Nouvelle Donn(é)e" (roughly translating to "New Data, New Deal") to amplify the voices that matter most in healthcare. Alongside Franck and I, physicians, legal experts, researchers, and hospital leaders shared insights about the complex world of healthcare data and AI.

The core message? AI is the new deal in healthcare data.

This mini-series calls for unity across the healthcare ecosystem—caregivers, institutions, innovators, patients—to tackle tomorrow's challenges together.

Because our healthcare system is facing a crisis. And innovation isn't optional anymore.

A moment in time for us at Lifen

Watching everyone who shaped our journey fill that cinema, share our vision and dreams was deeply moving.

A decade in, and Lifen's story is far from over.

Get More Insights from Lifen

Enjoyed these insights? Visit our Welcome To The Jungle page to dive deeper into what makes Lifen unique as a company and explore our current opportunities.

Subscribe to stay updated on how we're solving complex challenges and building the future of healthcare.

Subscribe now

🇫🇷 Version française

Lifen a dix ans.

Il y a quelques semaines, nous avons célébré les 10 ans de Lifen d'une manière un peu particulière. Je voulais partager à l’écrit ce moment et ce qu'il représente pour nous.

Une soirée sur grand écran

Franck et moi avons l’habitude de travailler en coulisses, sur la tech et le produit.

Derrière un écran. Pas devant celui d’un cinéma…

Mais pour les 10 ans de Lifen, nous avons voulu prendre la parole dans un cadre symbolique :

Un cinéma, donc. Pour projeter un documentaire. Que nous avions produit. Devant tous ceux qui comptent le plus pour nous.

Une façon de voir les choses en grand au sens littéral.

Une petite prise de risque, aussi.

“From Day 1 to Year 10”

En préparant cette soirée, j'ai repensé aux débuts de cette aventure.

Lifen jour 1, c'était Franck, tout seul dans sa cuisine, avec son grand sourire et sa folle envie d’innover.

Aujourd'hui, nous sommes 160 collaborateurs animés par une mission commune : mettre l'intelligence de la donnée au service du soin.

Cette croissance s'appuie sur la confiance de 800 hôpitaux et 20 000 professionnels libéraux qui utilisent quotidiennement nos solutions.

En début d’année, nous avons franchi une étape importante : notre activité historique est devenue rentable et nous avons dépassé les 20 millions d'euros de chiffre d'affaires.

Cette réussite financière n'est pas une fin en soi. Elle représente deux choses essentielles : la preuve que nos solutions sont pérennes pour les milliers de soignants qui les utilisent, et un socle solide pour les 10 prochaines années d'innovation.

Un moment de gratitude intense

Cette soirée était aussi l’occasion de remercier ceux qui ont pris des risques à nos côtés depuis le début.

Je pense d’abord nos collaborateurs pionniers. Certains ont rejoint Lifen quand nous n'avions qu'une promesse vague à offrir et aucune fiche de poste claire. Ils sont encore là aujourd'hui et continuent de pousser les murs année après année. Leur détermination m'impressionne encore. Marc, Etienne, Kelvin, Pierre, Maria Camila, Victor, Thibault, Charles, Anna, Hélène, et tant d’autres...

Je pense aussi à nos nombreux et emblématiques hôpitaux et médecins partenaires. Notre gratitude à leur égard est immense.

Je dois une pensée particulière à ceux qui nous ont fait confiance en premier : les Hôpitaux Universitaires de Strasbourg & Ramsay Santé pour Lifen Documents, le CHRU de Nancy pour Lifen Intégration, et Gustave Roussy pour Lifen Datalab.

Je me souviens encore de la toute première utilisatrice de Lifen : Marjorie, la secrétaire du Dr Cadot, qui a eu le courage d'envoyer les 10 premiers comptes rendus via notre logiciel. En juin 2017, nous étions fiers de célébrer 249 envois… Aujourd'hui, nous en sommes à plus de 4,5 millions par mois.

Je pense enfin à nos investisseurs qui nous soutiennent (en continu) et nous challengent (de temps en temps) : daphni, Serena, Partech, Creadev, Lauxera…

D’ailleurs, l’un d'entre eux nous a un jour dit, avec candeur : "Lifen, c'est génial ce que vous faites, malheureusement, personne ne le sait".

Et c’est peut-être un peu vrai. Nous sommes fondamentalement une équipe d’ingénieurs passionnés de data, d’impact, d’adoption, d’usage, de solutions qui marchent vraiment.

Nous évoluons dans un environnement technique ardu, relativement antagoniste au marketing triomphant, aussi élémentaire soit-il.

Alors ce soir là nous avons voulu franchir une étape.

Mais au risque de les décevoir, nous avons projeté…

Un documentaire qui ne parle pas de nous.

Nous avons produit "Nouvelle Donn(é)e" pour donner la parole aux acteurs de la santé. Médecins, soignants, avocats, chercheurs et responsables hospitaliers nous ont prêté leurs voix pour démocratiser les enjeux systémiques pointus de la donnée et de l'IA en santé.

Le message principal ? L’IA change la donn(é)e en santé.

Cette mini-série est aussi un appel à rassembler toutes les forces de l’écosystème santé, soignants, institutions, innovateurs, patients, pour relever ensemble les défis de demain.

Car notre système de santé traverse une crise profonde.

Et il est urgent d'innover pour y répondre.

Une saveur particulière dans la trajectoire de Lifen

Voir réunis dans une salle tous ceux qui ont contribué à notre aventure, partager notre vision et nos ambitions, était à la fois intimidant et profondément émouvant.

Dix ans après ses débuts, Lifen continue d'écrire son histoire.

Aux côtés de tous ceux qui #ChangentLaDonn(é)e.

Get More Insights from Lifen

Enjoyed these insights? Visit our Welcome To The Jungle page to dive deeper into what makes Lifen unique as a company and explore our current opportunities.

Subscribe to stay updated on how we're solving complex challenges and building the future of healthcare.

Subscribe now

The numbers were eye-opening: 1-2 minutes per ticket just for qualification, multiplied by hundreds of daily tickets. This meant hours of manual work that could have been spent solving actual customer problems and contributing to product improvement.

Our solution? AI Kits—workflows involving LLM assistants. Over the last few months, we've developed, amongst others, three major AI Kits that revolutionized our support operations, saving the team, in average, 3 days a month in manual qualification and delivering 20-30% efficiency gains per ticket.

The Hidden Cost of Manual Support Processes

Every support ticket at Lifen involved multiple manual challenges that consumed our agents' time: manual qualification required multiple steps and repetitive data entry, information hunting meant searching through scattered documentation systems, and macro selection involved guessing which response would fit best. Our agents spent significant time managing these manual operations rather than solving customer problems.

The biggest frustration was "context switching fatigue": constantly jumping between ticket queues, documentation systems, macro libraries, and various tools, losing focus and time with each transition.

Our agents were spending more time navigating our internal systems than actually helping customers—a reality that became impossible to ignore.

Building Smart AI Assistants: Three Kits That Changed Everything

TipKit: From Resource Hunting to Instant Context

Before TipKit, agents manually searched Notion databases, cross-referenced documentation, and reformulated technical notes. They'd spend valuable time hunting for information that sometimes they couldn't find at all.

TipKit delivers contextual resources instantly based on ticket content, finding the perfect documentation and reformulating it appropriately. It even suggests the right helpcenter article that may be provided to the customer when needed.

The real game-changer was how this transformed our agents' daily workflow. Instead of bouncing between Notion here, our helpcenter there — agents could now access everything directly within their ticketing tool. This centralization meant no more losing focus while hunting through different platforms.

Beyond time savings, we quickly discovered new team members could now get instant access to institutional knowledge.

MacroKit: From Manual Selection to Smart Suggestions

Choosing the right macro from dozens of options was guesswork. Agents scrolled through lists, assessed context manually, and sometimes applied wrong macros.

MacroKit analyzes ticket context and suggests the best response, eliminating the guesswork entirely. It gives agents flexible, on-demand assistance. Agents can choose when and how to use MacroKit based on their specific needs—whether they want a quick suggestion for a routine response or need help crafting a more complex customer communication. It automatically personalizes existing macros with specific ticket content—inserting customer names, account details, and contextual information to create responses that feel tailored rather than templated.

This approach eliminates the guesswork entirely while giving agents the flexibility to take benefits from the kit on their own terms.

Manual Qualification: Our Biggest Technical Bottleneck

Ticket qualification became our most significant bottleneck because it was unavoidable—every single incoming ticket demanded this 1-2 minute investment before any actual support work could begin. With hundreds of tickets flowing in daily, agents found themselves spending their first hour each morning just qualifying new arrivals, then returning throughout the day to process the constant stream of incoming requests. This created a never-ending cycle where qualification work competed directly with customer problem-solving time.

Here's what agents had to do manually for every ticket:

Manual Requester Research

For every ticket submitter, agents manually:

• Looked up account status in our internal application

• Cross-referenced Salesforce for client vs. prospect determination.

This analysis took 30-45 seconds per ticket.

Individual Ticket Analysis

• Opening each ticket in Zendesk and analyzing content manually

• Understanding context and customer situation

Manual Title Optimization

• Rewriting ticket titles for clarity

• Ensuring other agents could understand the issue quickly

Manual Categorization & Data entry

• Completing form fields and applying qualification macros

• Bucketing by subject type (technical, billing, feature request)

• Classifying by client type and cross-referencing multiple systems

In order to properly categorize the ticket, the Support agent has to find and apply the Macro that will place the ticket in the correct bucket

Then, Why We Kept This Manual Process?

The obvious question arises: if qualification was such a time drain, why didn't we automate it sooner?

The truth is, we were hesitant to change something that worked so well. Our manual qualification was delivering excellent results—accurate categorization, proper prioritization, and effective ticket routing. We had built a system that our team trusted, and the quality of our support remained consistently high.

But the deeper reason was simpler: we hadn't found how to do it properly. Previous attempts at automation felt too rigid, missing the nuanced decision-making that human agents brought to each ticket. We needed a solution that could match human judgment while eliminating human busywork.

The breakthrough came with QualiKit's confidence scoring system.

Instead of forcing automation on every ticket, we could now let QualiKit handle the clear-cut cases (those with >90% confidence) while preserving human expertise for complex situations requiring nuanced judgment. This hybrid approach finally gave us the automation we wanted without sacrificing the quality we needed.

QualiKit's Automated Solution

We broke qualification into two automated steps:

User Qualification Automation

• Automatic data retrieval from internal systems

• Seamless Salesforce integration

• Default value application for missing information

Ticket Qualification Automation

• AI-powered content analysis

• Automatic subject replacement with descriptive titles

• Automatic field assignment and automatic messaging when needed

Automatic Messaging for Missing Document Requirements

For certain types of tickets, we require a specific document identifier to process the request effectively. QualiKit automatically determines whether this identifier is present in the ticket content.

When the identifier is present: The ticket is immediately placed in the appropriate agent view for handling, ensuring no delays in processing qualified requests.

When the identifier is missing: QualiKit triggers an automatic response to the customer requesting the missing information, then sets up automatic follow-up reminders if the identifier isn't provided. During this waiting period, the ticket remains hidden from agent queues, allowing the team to focus exclusively on fully qualified tickets that can be resolved immediately.

This intelligent routing ensures agents never waste time on incomplete requests while maintaining seamless communication with customers about missing requirements.

Smart Confidence Scoring

QualiKit uses confidence scores to determine when human review is needed:

• >90% confidence: Fully automated

• <90% confidence: Manual review required

This maintains quality while eliminating 90% of manual work.

The Results: Manual Hours Become Customer-Focused Time

We implemented the rollout over approximately 30 days to ensure a smooth transition. This gradual approach allowed us to iterate on the assistant when needed, quickly address any bugs that emerged, and most importantly, gather continuous feedback from our agents to refine the system based on real-world usage.

The transformation didn't happen overnight, but the results have been remarkable. What started as an experiment in automation became a fundamental shift in how our support team operates. Agents who once began each day with qualification backlogs now dive straight into solving customer problems.

The mental energy previously spent on repetitive categorization is now channeled into creative problem-solving and proactive customer engagement. This newfound time allocation allows our agents to go the extra mile on each ticket, bringing enhanced focus and availability to every customer interaction.

Quantified Impact

• 24 hours per month freed from manual qualification per agent

• 20-30% efficiency gain per ticket

• Zero manual resource hunting with contextual delivery

• 66% of tickets achieve

• 95%+ confidence scores

30-Day Rollout Results:

• 100% ticket coverage by QualiKit

• 66% of tickets achieved

• 95%+ confidence scores

• 70% correctly qualified without manual intervention

Conclusion

The 1-2 minutes per ticket previously spent on manual qualification now happen automatically, freeing cognitive capacity for creative problem-solving and deeper customer engagement.

For teams drowning in manual processes: identify friction-creating work, build targeted AI systems, implement quality controls, and transition gradually while preserving human expertise.

Our AI Kits gave us time—time to think, care, and deliver exceptional customer experiences. At Lifen, that time makes all the difference in how we help our users succeed. Sometimes the most valuable technology is the one that simply gets out of the way.

Join the Lifen journey

Our tech culture continuously evolves. We're redefining remote collaboration through structured knowledge management. We're looking for professionals who value writing, thrive in remote environments, and are passionate about healthcare innovation.

If you enjoy building systems that scale, appreciate clear documentation, and want to help shape the future of healthcare technology, we'd love to hear from you.

This unified approach brings tremendous benefits: standardization, shared features across projects, and streamlined operations. However, it also presents a significant challenge: ensuring quality across diverse use cases without breaking existing functionality. Our solution? A Test-Driven Development (TDD) approach with extensive test coverage, executed locally and through our CI pipeline on every pull request.

In this context, we developed Kassette—a tool that makes testing easier by eliminating the need to write test code. Today, we're excited to open-source this solution and share the journey that led us here.

Our Testing Evolution Journey

Step 1: Unittest - When Code Was King

As an infrastructure team working primarily in Python, we initially relied on the standard unittest framework. Our testing process involved creating a minikube cluster, then for each test case:

1. Deploying the test case Helm release

2. Running unittest

3. Uninstalling the Helm test case release

Let's look at a concrete example with the web-with-hpa test case. Here are our chart values:

And the corresponding unittest code:

This approach required substantial code for relatively simple tests, making it time-consuming and error-prone.

Step 2: Behave/Gherkin - The Human-Readable Attempt

Seeking better readability, we moved to behave, Python's implementation of Gherkin - a "human readable" language for describing expected behavior.

Our web-with-hpa.feature file looked like this:

With the corresponding Python implementation:

While more readable and reusable, this approach still required code and forced us to explicitly define what to test for each scenario.

Step 3: Kassette - The No-Code Approach

Kassette

The VCR Inspiration

At Lifen, we love Ruby, so we naturally took inspiration from VCR, applying its record/replay concept to Kubernetes resources.

VCR is a Ruby project designed to record/replay HTTP requests for test runs. It offers several key advantages:

• Makes every test run deterministic

• Avoids real HTTP requests to external services in CI

• Eliminates the need to mock HTTP requests, so write additional code for it

We realized this same principle could help Kubernetes testing.

How Kassette Works

Kassette allows you to write chart tests in a "no-code" way using simple YAML configuration files. Here's how our web-with-hpa/kassette.yaml file looks:

Core Features

Kassette provides three essential commands:

• bin/kassette record: Takes YAML snapshots of defined resources based on your kassette.yaml configuration

• bin/kassette diff: Compares recorded snapshots with current Kubernetes cluster resources

• bin/kassette modify: Enables mass modification of resources in snapshots

The workflow is straightforward: the record subcommand reads your kassette.yaml file and captures the current state of described resources. Later, the diff subcommand compares these recorded states with current states, outputting differences and exiting with a non-zero code if divergences are found.

For example, when there is a divergence:

Implementation Details

We chose Python for Kassette's implementation for several reasons:

• Seamless integration with our existing test suite

• Common language across our infrastructure team

• Widespread adoption in the DevOps community

Real Impact at Lifen

Since implementing Kassette, we've observed the following improvements:

Enhanced Testing Culture: The tool's simplicity has made our team more committed to comprehensive testing practices.

Broader Test Coverage: We now cover many more scenarios than previously possible, with tests that are easier to create and maintain.

Faster Onboarding: New team members can contribute to our testing efforts without extensive coding knowledge.

Deterministic Results: Like VCR for HTTP requests, Kassette ensures our Kubernetes tests are predictable and reliable.

Open Source and Community Impact

Today, we are proud to open-source the first version of Kassette at https://github.com/honestica/kassette/

We believe Kassette solve problems that many DevOps teams using Kubernetes face, and we're excited to share this solution with the broader community.

Our hope is that Kassette will help spread testing culture throughout the DevOps community while enabling us to continuously improve the tool based on real-world feedbacks and contributions.

Join Our Mission

We built Kassette because we believe in sharing what works.

At Lifen, we face real infrastructure challenges—from scaling our platform to maintaining healthcare-grade reliability—and we've learned that the best solutions often come from the community. If solving meaningful technical problems alongside a passionate team sounds interesting, take a look at our open positions on Welcome to the Jungle.

We're excited to see what you'll build with Kassette !

Share

Lesson #3: Be Intentional About Cultural Implementation

Once we presented this framework to the company, we encouraged each department to further specify how these values would manifested within their unique operational contexts. While "Embrace Complexity" served as common ground across the organization, its practical application could vary significantly between our Tech team and our HR team.

Our objective was to make culture as concrete and actionable as possible, recognizing that it would serve as the operational backbone for how we would function as a scaling organization.

Culture-Centric Recruitment Process

Our recruitment process evolved into a comprehensive cultural assessment system. Technical competency and healthcare industry knowledge remained essential criteria, but we developed structured scenarios that revealed candidates' natural alignment with our three core values.

To ensure consistency across all hiring managers and recruiters, we built a comprehensive database of targeted questions designed for each value.

For "Embrace Complexity," our question database included scenarios about navigating ambiguous requirements, handling competing priorities, and approaching problems without clear solutions. For "Real Impact," we developed questions that required candidates to demonstrate concrete outcomes from previous work and explain their decision-making process when measuring success. For "Bienveillance," our questions explored how candidates approached honest and open feedback, collaboration challenges, and situations requiring true collaboration with colleagues.

Managers learned to identify candidates who demonstrated genuine comfort with complexity rather than those who oversimplified challenging problems. They evaluated evidence of real impact in previous roles, looking for concrete examples rather than abstract claims. They assessed bienveillance through candidates' approach to collaboration, feedback, and conflict resolution.

More significantly, we transformed recruitment into a bidirectional cultural evaluation. Candidates experienced our culture in action through structured interactions with multiple team members, participation in actual problem-solving sessions, and exposure to our decision-making processes. This wasn't merely about our assessment of them—it was about their informed evaluation of whether our cultural environment aligned with their professional values and working style.

Lesson #4: Mutual Accountability For Maintaining Culture Is Needed

We understood early on that keeping our culture coherent and alive was everyone's responsibility. This wasn't just an aspiration—we built concrete systems to make mutual accountability real.

The key was integrating cultural evaluation into our performance review process. Since we had clearly defined what "Embrace Complexity," "Real Impact," and "Bienveillance" looked like in practice, we could systematically assess and discuss these behaviors during performance evaluations alongside technical performance.

This created a three-way accountability system:

• Employees saw that cultural alignment was valued and measured, not just proclaimed

• Managers gained tools and responsibility for coaching cultural development, not just technical skills

• The organization established structured ways to assess and discuss cultural behaviors

Performance reviews included detailed discussions about how employees demonstrated our values in their daily work, where they excelled in cultural alignment, and where they could strengthen their embodiment of our principles. Managers had concrete frameworks for providing cultural feedback, and employees understood exactly how their contributions were being evaluated.

Did we achieve perfect cultural alignment everywhere? Certainly not. However, our framework enabled us to rapidly identify misalignments and foster honest conversations about mutual fit. This shared responsibility often led to amicable separations before frustration could develop on either side.

The measurable outcome: successful team scaling while maintaining cultural cohesion through genuine mutual accountability.

Lesson #5: Stay True To Your Culture During Tough Decisions

The ultimate test of our culture came when economic headwinds forced us to make the most difficult decision any leader faces: reducing our team size. Economic layoffs challenge every aspect of company culture. How do you maintain trust when you're breaking the employment contract? How do you preserve values when making decisions that hurt people you care about?

We leaned into our culture of transparency and collaborative problem-solving.

First, we shared the complete economic context with our entire team—the market changes affecting our customers, the revenue implications, and our timeline constraints. We explained that while reducing team size was necessary, how we did it would be guided by our values.

Our approach reflected the core of our culture:

Collaborative problem-solving meant involving team leaders in understanding the implications and exploring alternatives.

Transparency meant honest conversations with everyone affected, sharing both the reasoning and the emotional difficulty of our choices.

Bienveillance guided how we treated departing team members. We didn't hide from difficult conversations or rush people out the door. Instead, we took time to acknowledge their contributions and ensured they left with dignity and comprehensive support.

For the remaining team, we maintained transparency about our path forward, the lessons learned, and how we would rebuild sustainably. We didn't pretend the layoffs weren't painful or try to spin them as purely positive "refocusing."

The outcome surprised us. Rather than undermining trust, this experience actually strengthened it. Our team saw that our cultural framework wasn't just nice-to-have rhetoric—it was an operational guide that helped us navigate. Having strong cultural foundations proved invaluable when everything else felt uncertain.

What We Learned About Resilient Culture

Through both rapid growth and economic contraction, we learned that resilient culture isn't built on inspiration alone—it's built on practical systems and daily practices.

Culture is a practice, not a poster. Values statements matter only if they translate into decision-making frameworks. Our culture lives in how we run meetings, how we give feedback, how we prioritize features, and how we handle conflicts. It's embedded in processes, not just proclaimed in presentations.

Stress-testing reveals truth. You don't really know your culture until it's challenged. Growth pressure and economic constraints strip away everything superficial and reveal what you actually prioritize when forced to choose. These moments don't destroy culture—they clarify it.

Transparency builds resilience. Sharing context, admitting uncertainty, and involving people in difficult decisions creates stronger bonds than trying to project false confidence. People can handle difficult realities; they can't handle being kept in the dark.

Values guide decisions, they don't make them easy. Having clear principles doesn't eliminate difficult choices, but it provides a framework for making them consistently and with integrity.

Culture evolves while staying true. Staying authentic doesn't mean staying static. Our culture has evolved as we've learned, but the core principles that define us have remained constant. We've gotten better at living our values, not changed them to fit circumstances.

My Final Thoughts

It took us five years to make this happen, from 2018 to 2022 —here are my final thoughts.

In an uncertain economic environment, culture becomes your most sustainable competitive advantage. Technical skills can be taught, strategies can be copied, but authentic culture—proven through real challenges—cannot be replicated.

For candidates evaluating opportunities: Look for companies that can tell you specific stories about how their values guided them through difficult decisions. Ask about times when culture and business pressures conflicted, and how they resolved those tensions. The best organizations don't just talk about their values—they can show you the scars and growth that come from living them.

For HR professionals building resilient organizations: Invest in defining culture before you need it. Create systems that embed values in daily operations, not just annual reviews. Design processes that stress-test cultural fit during hiring and cultural alignment during crises. Most importantly, remember that culture work is never finished—it's an ongoing practice that requires constant attention and intentional action.

At Lifen, we're still learning, still growing, and still being tested. But we've proven to ourselves and our team that culture isn't just something nice to have when times are good—it's the foundation that makes everything else possible, especially when times are tough.

That's the kind of place we're building. That's the kind of place worth joining.

Get More Insights from Lifen Culture

Enjoyed these culture insights? Visit our Welcome To The Jungle page to dive deeper into what makes Lifen unique as a company and explore our current opportunities.

Subscribe to stay updated on how we're solving complex challenges and building the future of healthcare.

Subscribe now

Yes, AI generates code. Yes, it can solve simple problems. But no, it does not replace the human intelligence needed to design, architect, and maintain complex systems. In reality, AI is transforming developers into something much more powerful: tech leads for virtual teams.

At Lifen, AI isn't a recent addition to our toolkit—it's been in our DNA since 2015, long before the LLM revolution. We've been harnessing machine learning to structure medical data, extract insights from complex documents, and streamline healthcare workflows for nearly a decade. But our AI expertise goes beyond our products: we've integrated AI into our daily operations, from automated code reviews to intelligent documentation systems. This deep, practical experience with AI transformation gives us unique insights into how developers can evolve alongside these powerful tools.

What We Really Expect From a Developer (Spoiler: It’s Not Just Coding)

Let’s pause for a moment. What makes a good developer? It’s not how fast they can type or how many lines of code they can churn out.

A developer is someone who designs systems. They imagine the architecture, anticipate how components interact, and think about scalability from the very start. No AI today has that level of vision.

A developer thinks about security and performance. They know where to put the locks, how to optimize queries, which sensitive data needs protecting. You can't improvise this expertise.

A developer anticipates edge cases—those infamous rare cases that break everything in production. Experience allows them to sense where things might go wrong before even writing the first line of code.

A developer defines an appropriate testing strategy. Unit tests, integration tests, load tests—they know what to test, how, and why.

And above all… A developer understands the product, the business, and its users.

Writing code? That’s just the tool. A necessary step, but it's certainly not the end goal.

How to Become the Tech Lead of Your Own Team of AI Agents

Once you understand this, the possibilities are immense. Every developer becomes the tech lead of their own team - a team made up of 1, 2, sometimes 10 specialized AI agents.

At Lifen, we’re already experimenting with this approach: PR reviews, automatic creation of release notes, using MCP Jira servers to automate bug fixes or simple tasks (for now), creating design notes, and more.

It’s not just about “coding faster.” It’s about freeing up mental space to focus on what really matters: architecture, strategy, and innovation.

Imagine: you break down a complex feature into simple sub-tasks. You assign each task to a specialized agent. You validate, orchestrate, optimize. You’ve become a tech lead who focuses on design, not just syntax.

This transformation highlights four crucial skills:

• The ability to give good instructions. Everyone becomes a tech lead. Knowing how to clearly express a need, define constraints, specify acceptance criteria -these soft skills become technical skills.

• Breaking down complex problems. The art of turning an insurmountable problem into a series of solvable sub-problems. This capacity for abstraction and modularity - that’s pure software genius.

• Critical thinking and reviewing. Validating agent outputs, spotting subtle errors, challenging proposed solutions. Experience and intuition remain irreplaceable.

• Computer science fundamentals and culture. Algorithmic complexity, design patterns, graph theory - those foundations are what allow you to guide agents effectively and avoid pitfalls.

Vibe Coders vs. Real Engineers: Who Will Thrive With AI?

This evolution will create a clear divide in our industry.

On one side, the “vibe coders.” Those who use AI to quickly whip up POCs, mockups, throwaway scripts. They code by feeling, without the big picture.

On the other side, real engineers. Those who master the fundamentals, think architecture, and build scalable, high-performance applications. With AI, they’ll just move even faster.

The difference? Training, experience, and a deep understanding of systems. Those things can’t be automated.

According to a GitHub study, developers using Copilot are 55% more productive on repetitive tasks, but productivity on complex tasks depends entirely on their initial level of expertise.

We’re living through a revolution comparable to the arrival of IDEs or frameworks. Developers who embraced those tools thrived. Those who resisted faded away.

AI doesn’t threaten good developers. It transforms them.

It threatens those who were content to copy-paste from Stack Overflow without understanding. It exposes those who confused typing speed with technical expertise.

For everyone else - the real engineers, the architects, the problem solvers - AI opens up unprecedented opportunities. Becoming tech leads for teams of AI agents, orchestrating complex systems, focusing on innovation instead of implementation.

The future of development isn’t less human. It’s more human than ever. Because it finally forces us to focus on what makes us irreplaceable: our ability to think, design, and create.

At Lifen, we support our teams through this transformation. Because we’re convinced that the future belongs to developers who embrace AI as a superpower, not as a threat.

We're always looking for talented developers who share this vision. If you're ready to join a team that's been pioneering AI in healthcare, explore our current opportunities on our Welcome To The Jungle page.

Share

Founded in 2015, Lifen revolutionizes healthcare communication through innovative technology solutions. Like many startups, we mastered our product faster than we mastered scaling our team while preserving what made us “us”.

We grew from 25 Lifeners to 220 in two years, then economic pressures brought us back to 130. Most companies would have lost their identity in this whirlwind, but each phase taught us what truly mattered : through explosive growth and tough decisions, our culture became our compass.

For candidates considering their next move and HR professionals building resilient organizations, this is my honest account of what works, what doesn't, and why culture might be your most valuable strategic asset.

The Foundation: Define Culture with Intent

Back in 2018, when our CEO, Franck, outlined the ambitious growth objectives for the upcoming years, the strategic question that immediately followed was clear: How do we ensure we recruit the right talent, and how do we guarantee these individuals will collaborate effectively?

His response was straightforward

“They need to align with our values”

The challenge? We had not yet defined our organizational values.

Rather than rushing to create a set of aspirational principles, we made a deliberate strategic decision: we would refrain from defining our culture for the next two years. Instead, we would allow our existing team to organically demonstrate and shape it through their actions.

Lesson #1: Authentic Culture Cannot Be Mandated

At approximately 25 employees, questions about our organizational culture were emerging just as we prepared to scale rapidly, hiring between 100 and 150 new team members over the following months. Attempting to codify cultural values at that moment and subsequently requiring 100+ new employees to adopt behaviors defined by our initial 25-person team would have been fundamentally flawed.

While this approach is common in many organizations, it contradicted our philosophy of authentic culture development.

Our approach began with our clear definition of culture itself

This definition became our guiding principle. Rather than manufacturing culture in executive meetings, we committed to observing and documenting it in practice. We systematically analyzed and documented how our team naturally responded to pressure, recognized achievements, and navigated complex decisions. We identified the behaviors that drove our effectiveness and therefore we wanted to reinforce while also identifying behaviors we wanted to avoid.

Through this observation process, we identified three core elements that had organically developed within Lifen :

• radical transparency in communication

• user-first innovation in product development, and

• collaborative problem-solving that leverages diverse perspectives.

The key distinction was that these were not aspirational values we hoped to achieve—they were proven behaviors we had already demonstrated consistently. Radical transparency reflected our established practice of open communication regarding organizational challenges and strategic uncertainties. User-first innovation captured our systematic approach to evaluating every feature and decision through the lens of our users. Collaborative problem-solving described our proven methodology for addressing complex challenges through collective expertise.

By deferring the formal definition of our culture until we could observe and validate authentic behaviors, we avoided the common pitfall of creating compelling value statements that lack practical application. Instead, we documented the behaviors that were demonstrably driving our success, making them scalable, teachable, and—most importantly—genuine.

The next step was how to make it inspirational ?

Scaling Without Losing Soul

Our first major test arrived with rapid growth. Market demand for our healthcare solutions accelerated, requiring us to scale our team significantly in a compressed timeframe. Every company faces this critical juncture—the tension between recruiting quickly to capture market opportunities and maintaining cultural integrity.

We recognized that if we were going to rapidly scale, every single manager needed to be actively involved in recruitment. Therefore, they all needed to be aligned and thoroughly briefed on the specific behaviors they should evaluate in candidates to ensure new hires would thrive in our culture and work environment. The next strategic step was to transform the behaviors we had identified through observation into a practical framework that our managers could operationalize.

Lesson #2: Translate Culture into Actionable Frameworks

We needed to make our culture inspirational without compromising authenticity—no corporate rhetoric that would ring hollow to our team. In 2021, our two co-founders and I engaged in brainstorming sessions to distill our cultural essence into actionable but inspirational principles. This process yielded three core values:

Embrace Complexity, Real Impact, and Bienveillance (oui oui in French)

However, articulating values was only the beginning. We made a deliberate effort to define not only what each value meant in principle, but more importantly, we provided clear examples of behaviors that exemplified each value—and equally important, behaviors that contradicted them.

For instance, our ‘Embrace Complexity’ value was presented as follows:

We explained why this was a company value in the first place, it’s key !

Then, we made it tangible by clearly outlining which behaviors aligned with it and which didn’t.

We took it a step further by specifying how this value was expressed in different contexts

This approach made culture tangible across all organizational levels: for leadership, for managers conducting interviews, for current employees in their daily work, and for prospective recruits evaluating our organization.

Get More Insights from Lifen Culture

Enjoyed these culture insights? Visit our Welcome To The Jungle page to dive deeper into what makes Lifen unique as a company and explore our current opportunities.

Subscribe to stay updated on how we're solving complex challenges and building the future of healthcare.

Subscribe now

Developers frequently lose productivity by switching between tools like Jira and AppSignal throughout the day. Each context switch—whether it’s checking a ticket or investigating an error—breaks focus and fragments our workflow. But what if we could bring these tools directly into our development environment, eliminating the need to juggle multiple interfaces.

The Power of Custom MCP Servers

The Model Context Protocol (MCP) is a standard for connecting AI assistants—such as Claude or Copilot Chat—to external tools and platforms. By building custom MCP servers, you can bridge your AI assistant with the tools you use daily, accessing information without ever leaving your coding environment.

Real-World Example: Jira Issue Lookup

One of the most common workflow interruptions is checking Jira for issue details. I built a simple MCP server that allows Copilot Chat to directly fetch Jira issue information. With a custom MCP server, you can fetch ticket details directly from your AI assistant:

Fetch the jira issue PROJECT-123

This message returns details instantly—no more tab switching or manual logins. The server handles authentication, API calls, and formats the response for easy reading.

Real-World Example: AppSignal Error Monitoring

Investigating error reports is another frequent distraction. A dedicated MCP server lets your AI assistant pull error details from AppSignal with a single command:

Get details about error sample https://appsignal.com/mycorp/sites/a47542e56bc2797005e9d8a033e9c72a/exceptions/incidents/24/samples/844444175174ede426427ae550e9cab2-962238031415027086017482536006

Retrieve full error details, including stack traces and context, right where you’re already working.

Building with the MCP SDK and Bun's Binary Packaging

With the official MCP TypeScript SDK and Bun—a fast all-in-one JavaScript runtime—you can quickly build and package custom tools. The SDK abstracts protocol complexity, letting you focus on functionality. Bun compiles everything into a single executable, simplifying distribution: no Node.js or dependency headaches for teammates.

Using the @modelcontextprotocol/sdk package, creating a custom server is straightforward. The SDK handles all the protocol complexity, letting you focus on the actual functionality:

The real game-changer comes with Bun's ability to compile these servers into standalone binaries. With a single command in your package.json:

bun build src/index.ts --outfile tool-name --compile

This creates a single executable file that includes your entire application, the MCP SDK, all dependencies, and even the JavaScript runtime.

This approach transforms custom MCP servers from development projects into production-ready tools that anyone on the team can use immediately, regardless of their local development setup.

The Productivity Impact

Over time, these small improvements add up:

• No more context switching between applications

• Reduced cognitive load

• Faster, frictionless access to information

• Simplified sharing of tools across the team

By eliminating these small friction points in our daily workflow, we can maintain focus on solving the complex problems that actually require our expertise and creativity.

Conclusion

Custom MCP servers—especially when packaged with Bun—make integrating external tools into your workflow easy and shareable. At Lifen, we champion such innovative approaches, seeking ways to enhance our productivity and focus on building impactful healthcare solutions. Next time you find yourself switching contexts for routine tasks, consider whether a small MCP integration could save you (and your team) hours of distraction.

What would your workflow look like with every tool at your fingertips?

Join the Lifen journey

Our tech culture continuously evolves. We're redefining remote collaboration through structured knowledge management. We're looking for professionals who value writing, thrive in remote environments, and are passionate about healthcare innovation.

If you enjoy building systems that scale, appreciate clear documentation, and want to help shape the future of healthcare technology, we'd love to hear from you.

Get More Insights from Lifen Engineering

Enjoyed this dive into developer productivity? Our Substack blog has more technical deep dives, best practices, and stories from the trenches of healthcare technology.

The genesis: a reflection on our real needs

It all started with a simple observation: our teams were expressing two needs. On one hand, there was a thirst for learning and skills development on a variety of subjects: from our products, to soft skills, to best practice in our ecosystem. On the other hand, there was a need to share, meet and exchange ideas regarding specific subjects. Rather than dealing with these needs separately, we decided to combine them into one recurring event. That's how the Lifen Academies were born, and has gradually become part of the company's organic rituals.

Why this format? The winning equation

We firmly believe that learning is most effective when it takes place in a shared environment. By combining high-quality training with activities, we create an environment that is conducive to the assimilation of knowledge and the strengthening of bonds between Lifeners. It also allows us to highlight the work of certain teams and give us the same elements of understanding and language. This format also reflects our vision of an inclusive and caring company, where every Lifener has access to the same development opportunities, whatever their position or level of experience.

How is it organised?

A structured monthly meeting.

The rhythm: Once a month, all the Lifeners get together for a Lifen Academy day. This regularity helps to create a positive habit and ensure continuous skills development.

The structure: Each day is structured around two complementary highlights:

• Training sessions 👩‍🎓

  • Led by an internal or external speaker (guaranteed alternation)

  • Varied topics: product training, development of soft skills, health ecosystem, best practices, etc.

  • Content adapted to all levels and professions

  • 30 to 45 minutes in length

  

• Group activities 🌿

  • Creative workshop, sharing session, brainstorming, challenges, friendly breakfast, snack, relaxation activity, etc.

  • A moment to unwind and build team spirit

  • One hour duration

Practical organisation

• Venue: Our catering area, designed to encourage exchanges between people on the spot and remotely ;

• Accessibility: Sessions available by videoconference for our remote teams ;

• Traceability: Systematic registration to allow absentees to be made up ;

• Promoting internal learning.

The Lifen Academies are not disconnected from our corporate life. They tie in intelligently with our HR calendar and our strategic initiatives. For example, we schedule training on feedback during the annual appraisal period, we deep-dive into our products’ functioning before big events or a session on innovation before our internal hackathons. This synchronisation ensures that learning comes at the right time and is immediately put into practice.

Let’s zoom in

The Lifen Academy concept was created in February 2024 and we are now celebrating our 10th session. 🎂 We have a calendar announcing the next subjects and we encourage Lifeners to propose subjects for incoming sessions. As examples of Lifen Academies :

• Emilie Benayad Stevelink, founder of PALM association, came to make a state of play of how diversity is perceived within Lifen and animated workshops ;

• Our Product Managers talked about Lifen’s products, our revenue model based on those products and their added value ;

• Our HR team made a presentation and discussion around how to formulate and receive feedbacks and also about Nonviolent Communication (NVC) ;

• France Roussin, Procurement Cordinator at the Hôpital Saint-Louis and Mélanie Hardy, Customer Operation here at Lifen made a presentation about Organ Donation and its concerns.

We realised that this format is the best way of promoting Lifeners to their colleagues and to create awareness and attention regarding specific subjects. In other words, to highlight their passions, their former jobs, their community lives and to build bridges between them. Yesterday's Lifen Academies create tomorrow's Lifen Academies.

Our learnings and benefits

This innovative format has established itself as a recognized and eagerly anticipated event among all our Lifeners, we reach around 65% participation rate for each session. It provides a privileged framework for continuous and diversified skills development, while creating convivial moments that naturally strengthen team spirit. Its inclusive design ensures complete accessibility, regardless of participants' geographical location. This approach contributes to fostering a stronger learning culture within the organization, promoting enhanced team cohesion that directly translates into significantly improved collective performance. Plus, Lifeners now propose subjects spontaneously and we couldn’t be prouder!

Every month is a new opportunity to learn something new, to get closer to our colleagues' issues in a less formal setting, and to contribute to the collective dynamic that is Lifen's strength. This month, we celebrated our 10th Lifen Academy session and there is still a lot of sessions to come.

Ready to join our next Lifen Academy? 🚀

Written by Héloïse Rouanet

Early in our remote journey, we chose Notion as our central knowledge management platform. Its where documentation, decisions, and discussions live. However, like many growing companies, we initially struggled with information sprawl. Our workspace ballooned to over 400,000 pages, making critical information nearly impossible to find.

This article shares how we transformed that chaos into a structured system of five specialised Notion databases that now form the backbone of our remote collaboration.

Our building blocks : Databases, fields and templates

We transformed our chaotic documentation into a streamlined knowledge ecosystem using Notion databases.

Databases

• Why databases over pages: Isolated Notion pages quickly become unfindable, while databases offer built-in searchability, visibility, and consistency

• Public by default: Unlike standalone pages that are often buried within sub-pages, databases make information discoverable by default

• Consistent organisation: Databases create a predictable structure where team members know exactly where to look for specific types of information

Fields

• Essential fields only: We limit our fields to the critical few: ID, status, team, author, approvers, and cross-database links

• Standardised naming: Consistent field names across databases reduce cognitive load

• Structured values: Whenever possible, we use select fields with predefined options rather than free text

• Relational power: Cross-database links create a knowledge graph where information flows naturally between contexts

Templates

• Reduced friction: Templates lower the barrier to documentation by providing a starting structure

• Consistent quality: Predefined sections ensure all critical information is included

• Team customisation: Some teams have created specialised templates that better suit their specific needs while maintaining core fields

• Continuous refinement: Templates evolve based on feedback and changing requirements

Notion features that prevent database overwhelm

• Saved filtered views: We create pre-configured views that show only relevant information—for example, Opportunities filtered by team, status, or quarter

• Review reminders: Notion's review feature helps us schedule regular document reviews, ensuring content stays accurate and relevant

• Custom sorting: Consistent sorting (by ID, status, or date) makes information predictable and scannable

The combination of these three elements—well-structured databases, thoughtfully designed fields, and flexible templates—creates a system that's both rigid enough to maintain order and flexible enough to adapt to different teams' needs.

Our 5 databases

After experimenting with different approaches, we settled on five core databases that cover the full lifecycle of our work—from knowledge preservation and idea generation to planning, implementation, and progress tracking. Each serves a distinct purpose while interconnecting to create a comprehensive knowledge ecosystem.

Tech Handbooks (THBs) 📒

What they contain: Coding standards, architectural principles, security guidelines, onboarding procedures, and technical best practices.

Why we created them: To establish a single source of truth for all technical knowledge that doesn't change frequently.

How we use them: When a new developer joins, they start by reading relevant handbooks. When discussing architecture, we reference specific THBs rather than recreating explanations.

Example: Our "[THB-42] Component Maintainer Responsibilities" defines ownership rules across our microservices architecture, preventing confusion about who maintains what.

Request For Comments (RFCs) 💬

What they contain: Proposed changes, ideas for consideration, problems needing solutions, and collaborative decision-making discussions.

Why we created them: To provide a lightweight framework for proposing changes before implementation begins.

How we use them: Anyone can create an RFC to suggest improvements. The team discusses asynchronously, and decisions are documented for future reference.

Example: An RFC proposing to reduce pull request sizes led to new guidelines that significantly improved our code review process.

Opportunities (OPPs) 🍀

What they contain: Product and technical initiatives linked directly to company OKRs, with clear ownership and prioritisation.

Why we created them: To bridge the gap between high-level company goals and specific implementation work.

How we use them: During planning cycles, teams select opportunities aligned with OKRs. This database forms the foundation of our roadmap while maintaining team autonomy.

Example: An opportunity to improve our local development environment was directly linked to our OKR of increasing developer productivity, making its business value clear.

Design Notes (DNs) 📚

What they contain: Technical implementation details, architecture decisions, security considerations, and implementation plans for specific features.

Why we created them: To document how a feature will be implemented before coding begins.

How we use them: Engineers write DNs for significant changes, which are reviewed by architects and component maintainers before implementation starts.

Example: A Design Note for our authentication system redesign documented security implications, migration strategy, and technical approach—accelerating both implementation and code review.

Newsletters (NLs) 🗞️

What they contain: Biweekly team updates including recent deployments, operational metrics, retrospective insights, and upcoming objectives.

Why we created them: To maintain cross-team visibility without requiring synchronous meetings.

How we use them: Every two weeks, teams publish updates that are shared in Slack. This creates transparency and keeps everyone informed of progress.

Example: Our frontend team's newsletter highlighting performance improvements prompted the backend team to prioritise complementary API optimisations.

A Journey of Continuous Improvement

Our system didn't appear overnight. Our Notion workspace once contained over 400,000 unstructured pages across all departments. While HR had standardised some practices, the tech side lagged behind. Documents were created without clear ownership or status, becoming unfindable within weeks.

The path to 5 databases

One of our key learnings was finding the balance between too many databases (creating confusion) and too few (resulting in mixed, hard-to-filter information).

It's crucial to separate conceptually different types of content. For example:

• Ideation (RFCs) needs separation from implementation design (DNs)

• Progress updates (Newsletters) should be distinct from the initiatives themselves (Opportunities)

Each database should have a clear, singular purpose. When information feels like it doesn't quite fit anywhere, that's a signal you might need a new database—not that you should force it into an existing one.

We recommend starting small. Tech Handbooks and Design Notes are excellent first databases because teams quickly see their value. Then gradually expand as you identify gaps in your knowledge architecture.

What We Learned Through Iteration

• Limiting database types prevents fragmentation

• Clear boundaries between databases guide users to the right place

• Standardised field names and values outperform free text

• Centralisation and proper linking create a coherent knowledge network

• Strict ownership ensures information stays current

Challenges remain. Maintaining these databases demands daily discipline. We constantly balance structure with agility as our needs evolve.

The Concrete Benefits

This structure delivers multiple advantages:

• Asynchronous collaboration: Teams work independently without waiting for meetings.

• Reduced interruptions: Information is findable without disturbing colleagues.

• Transparent decisions: New team members understand not just what we do, but why.

• Faster onboarding: New hires quickly access relevant knowledge.

• Deep work: Less time spent searching means more time for focused work.

• Equal participation: Written communication levels the playing field for all voices.

Leveling Up with AI

There's a common myth that AI can magically find information in thousands of unstructured pages. Reality proves otherwise.

By integrating Dust.tt with our structured Notion databases, we've created specialised AI agents that can accurately retrieve information. The clear statuses and distinct database types allow these agents to differentiate between ideas, implementations, and discontinued projects.

For example, when someone asks about a feature in Slack, adding a specific emoji triggers our Dust.tt agent. It searches our structured databases and provides relevant answers—but only if the information exists in our documentation. This creates a positive feedback loop: better documentation leads to better AI assistance.

Conclusion

Transforming thousands of scattered pages into five powerful Notion databases represents more than an organisational improvement—it's the backbone of how we work. In a world of information overload, our approach enables every team member to find what they need, contribute effectively, and focus on what matters most: building exceptional healthcare technology.

Written by Florent Chenebault

Share

Join the Lifen journey

Our tech culture continuously evolves. We're redefining remote collaboration through structured knowledge management. We're looking for professionals who value writing, thrive in remote environments, and are passionate about healthcare innovation.

If you enjoy building systems that scale, appreciate clear documentation, and want to help shape the future of healthcare technology, we'd love to hear from you.

At the time of writing this article, PostgreSQL 17 is nearly out. On September 5th, the first release candidate was published. The final release is expected on September 26th, but we can already explain why we’ve been eagerly awaiting this release since 1 year.

At Lifen, we’ve loved Rails from the beginning. We have several Rails applications, each with different scopes and teams, all using PostgreSQL as the main database. Some of these applications handle a significant amount of traffic, and their databases need to be properly monitored. This is done by the infrastructure team and the developers themselves using PgAnalyze, Grafana and sometimes AWS console with "Performance Insight".

More than a year ago, we started monitoring the 95th and 99th percentile response times (p95, p99) on an endpoint that was experiencing timeouts. These p99 times were important because they involved a client with significantly more data. While investigating the queries in AppSignal, we quickly found that the issue was on the database side, as is often the case.

The problematic query involved one table with over 40 GB of data, which was heavily distributed. The query in Rails looked like this:

Doc.where(status: ['draft','sent'], sender_reference: ['Client/1', 'Client/2']).order(sent_at: :desc).limit(20)

Pretty simple, right?

Due to the nature of ActiveRecord DSL, this is something you see often, but without fully understanding the limitations of the underlying database. And those limitations can be surprising.

When fetching many documents, the query was very slow. We looked at the EXPLAIN plans and quickly identified the issue: not all indexes were being used.

You see plan like this one

  ->  Index Scan using docs_sent_at_idx on public.docs  (cost=0.56..39748945.58 rows=29744 width=38) (actual time=116.924..218.784 rows=20 loops=1)
        Output: id, type, status, sender_reference, sent_at, created_at
        Filter: (((docs.status)::text = ANY ('{draft,sent}'::text[])) AND ((docs.sender_reference)::text = ANY ('{Client/1,Client/2}'::text[])))
        Rows Removed by Filter: 46354
        Buffers: shared hit=1421 read=45046

Index is used but you still need to perform expensive filtering.

After some exchanges with some great folks on the PostgreSQL Slack, it was mentioned that PostgreSQL is not optimized for queries with multiple IN or ANY query parameters. This was a problem, as such queries are common in Rails applications and aren’t easily optimized using indexes in PostgreSQL. But then, something unexpected happened…

I received a private message from Peter Geoghegan. He was working on a patch that could help solve my issue. I tested the first version, then a second, and the results were very impressive, with a 90% reduction in query time (from 110ms to 10ms for some data). Indexes were fully utilized.

The patch is well explained in a blog post on PgAnalyze but the changelog entry speaks for itself: “Allow B-tree indexes to more efficiently find a set of values, such as those supplied by IN clauses using constants.”

Many iterations were made on this patch. I provided feedback on the PostgreSQL hackers mailing list, explaining why I, as a developer, was interested in this patch, along with benchmarks and execution plan outputs. The patch was included in PostgreSQL 17 beta, and additional fixes were added.

With RC1 and a basic Rails application, here’s the improvement we saw:
We inserted 100 million rows into a table with three columns. This was done via SQL, but it could have been done in Rails as well.

INSERT INTO
  docs (status, sender_reference, sent_at, created_at, updated_at)
SELECT
  ('{sent,draft,suspended}'::text[])[ceil(random()*3)],
  ('{Custom,Client}'::text[])[ceil(random()*2)] || '/' || floor(random() * 2000),
  (LOCALTIMESTAMP - interval '2 years' * random())::timestamptz,
  LOCALTIMESTAMP,
  LOCALTIMESTAMP
FROM generate_series(1, 100_000_000) g;

Those columns are indexed with a multicolumn index.

add_index :docs, [:sender_reference, :status, :sent_at], algorithm: :concurrently

With PostgreSQL 16 on a Rails 7.2 app, after several queries and with a warm cache, the response time was:

Query executed in 8.45 seconds.

With PostgreSQL 17, the result is 🥁:

Query executed in 0.11 seconds.

The query can still be optimized to get closer to 10ms, but the most challenging part to optimize has already been addressed by the patch.

With results like these, I highly encourage you to upgrade to PostgreSQL 17. There are many other improvements as well. I also recommend digging deeper into the database side of your application, as this is often the key to faster response times. I’ve learned that developers are welcome to share their feedback on the PostgreSQL hackers’ mailing list, providing benchmarks and feedback as end users.

Thanks to Peter Geoghegan, Matthias van de Meent, Hubert Depesz, Ants Aasma, and the rest of the PostgreSQL community for their work over the past year.

If you’d like to follow the benchmarks made since last year, there is a gist. You can also find the Rails app used to test response times on Github.

Written by Benoit Tigeot

Our objective was to develop a tool that:

Thanks for reading Lifen AI! Subscribe for free to receive new posts and support my work.

• streamline the support process by making our explainable AI tools easily accessible to authorized users

• is easily maintainable and always up-to-date with our AI

• provides some insight into how the ML model made its prediction

The end result

The result is a web app lifen-ai-explainer accessible internally to Data Scientists and Customer Support. Given a document id (provided by the client requesting support), this web app will display the anonymized document and predictions our AI made. (Predictions are also anonymised but made on the original document, otherwise, changes on the doc by anonymization process could produce different results)

We built lifen-ai-explainer and the pseudonymisation algorithm internally for this purpose and it’s been fully integrated into our daily processes since mid-2020.

Explaining classification

In addition to anonymizing and displaying the predictions, we try to explain them. For example, we want to understand how our machine learning text classifiers (eg: document type, patient gender) made their predictions by highlighting which words impacted the prediction the most.

Here is a demo of what this looks like on the patient gender classification algorithm (whose goal is to detect the gender of the patient mentioned in the document — without relying on the name — to. increase identitovigilance)

This feature proved useful in real-world situations to understand how for example a misread/misplaced word or simply a typo leads to a wrong prediction.

Explaining NER

We also use this tool to debug our NERs models. On the document view we show the raw softmax output (score for each class for each token), on the left side we show the final post-processed prediction.

How does it work?

Our classifiers are made with an Embedding + LSTM architecture (long documents, resources, and time constraints mean no Transformers). However, this architecture is not out-of-the-box well suited for explainability purposes.

Let’s go into detail to understand why. Here we use the last hidden state that we pass to a dense layer and softmax to produce our prediction:

The problem is simply that we have no idea which word impacted the final prediction. (There is also a problem of vanishing gradient and bottleneck where if the important words are at the beginning of a very long sequence, the LSTM has to learn to remember it for many time steps) But we can fix that with a simple inexpensive tweak in the model.

Global attention layer

Our solution is to add a global attention layer (or attentive pooling layer) [1] that compute an attention score for each token with a dense + softmax layer which is then used to compute a weighted sum of all the hidden state at each time step (which roughly correspond to information contained in the current word).

This weighted sum is therefore a whole document embedding (similar to the last hidden state that we used in the previous architecture) that we can use for classification, but we also have in bonus those attention scores per word which we can interpret as “importances scores” for each word. Those scores proved empirically very interpretable.

The layer code

This layer is a simple Keras layer made from well-known building blocks. In practice, we use more than one attention head which can make the interpretation more tricky but improve performance.

def MultiGlobalAttention(timesteps: tf.Tensor, n_heads: int = 1) -> tf.Tensor:
    """Global Attention layer. Computes a weighted sum of all the timesteps.
    Input: N timesteps of dim D
    Output: 1 of dim D
    Zhou et al https://www.aclweb.org/anthology/P16-2034  §3.3"""

    attention = Dense(n_heads, use_bias=False, activation="linear")(timesteps)
    attention = Activation(softmax_axis(1), name="attention")(attention)
    weighted_sum = dot([attention, timesteps], axes=1)
    return Flatten()(weighted_sum)

How to get those attention scores? We extract an attention sub-model that is not run in typical inference scenarios, but only for support in lifen-ai-explainer.

def attention_model(model: Model) -> Model:
    """make attention_model from model"""
    return Model(inputs=model.input, outputs=model.get_layer("attention").output)

Architecture of lifen-ai-explainer (+ lifen-ai)

The web app is a React Single-page application written in TypeScript that communicates with a REST python backend based on FastAPI (the backend also serves the static files for the frontend).

The frontend tooling is based on Vite which handles all the complexity of a modern frontend app (we rather do data science than tinker with babel/webpack config files).

The web app needs to make the same predictions as lifen-ai. We therefore decided to build the backend as a python app that can import the existing lifen-ai as a library.

Python lacks explicit access modifiers, which can make it difficult to maintain good encapsulation and ensure the maintainability of a large interconnected codebase. We use import-linter to enforce encapsulation rules and allow lifen-ai-explainer to import lifen-ai but not the other way around.

How to keep our ai-explainer in sync with new features?

Our AI is rapidly evolving and we need lifen-ai-explainer to stay in sync at all times. We never want to be in a position where there is a new feature that we cannot debug with lifen-ai-explainer.

Mono repo & mypy & front-back client generation

We decided to put both apps in the same mono repo. This allows atomic changes: when making a Pull Request to add a prediction to lifen-ai, you can simultaneously add the implementation to lifen-ai-explainer. This is enforced by design: because 100% of our codebase is type annotated with mypy, a breaking change in lifen-ai will break CI for lifen-ai-explainer, so you are required to keep it up to date. Our unit & integration tests do the rest.

The frontend client is autogenerated by Orval using the OpenAPI spec which is also autogenerated by FastAPI. As a result, any changes made to the output of a model will cause the CI to fail and direct you to the exact location in the frontend code where updates are required 🎉.

We use poetry to manage python dependencies which don’t support mono-repo but allow us to split dependencies into groups. This way, lifen-ai-explainer includes all the dependencies of lifen-ai but not the other way around.

lifen-ai-explainer has been fully integrated into our daily processes since mid-2020 and we have since expended its features with a dataset labellisation interface, and a post-training errors review tool.

Thanks for reading Lifen AI! Subscribe for free to receive new posts and support my work.

At Lifen we process medical PDF documents from thousands of clients. Those PDF documents come with a huge variety of layouts:

We need to perform layout detection from this 2D grid of words to produce a coherent 1D sequence. This 1D sequence can then be fed into our NLP models or sent as plain text when clients do not support PDF or image attachments.

Thanks for reading Lifen AI! Subscribe for free to receive new posts and support my work.

Mistakes in this layout detection could result in nonsensical extracted text. This could mean for exemple mixing addresses of two recipients together.

Imagine in the left image above if we failed to understand that the green and yellow blocks are two columns that should be read one after the other: we would create sentences mixing pieces of text from both blocks.

Isn’t layout information already encoded in PDFs?

Most of the time, yes. There is a sense of order in the way it is encoded, but there are no means to be sure this is the case, and results can be pretty bad if we just trust PDFs blindly. Furthermore, in the case of images PDF (which needs OCR), there is no text order to recover.

Objective: be fast & accurate

Existing solutions

Existing solutions (such as pdfplumber, Parsr) use simple techniques such as XY cut¹ and did not perform well enough on our dataset. Furthermore, because accurate text extraction is critical for our product, we need to be able to constantly fine-tune our solution to be as accurate as possible with new clients. Existing solutions did not allow us the level of customization we needed.

Avoid OCR when possible

Multiple solutions combine OCR with layout detection. However, most of the documents we receive (~85%) have a clean extractable text and do not require OCR. Furthermore, doing OCR on every document we process would be slow and expensive we’re happy to avoid as much as we can. For image documents, we first apply OCR then use the extracted text as a starting point just as extractable PDFs.

Our solution

Previous work

There is an active community of research in the field of layout analysis, with academic conferences such as the International Conference on Document Analysis and Recognition. One resource we found particularly valuable to navigate this field is survey of algorithms².

However, while reviewing papers we found out that few share the characteristics we are looking for: start from extractable PDFs, fast and fine-tunable.

One great paper using the same starting point as us (no OCR, words and not pixels) and with great care for speed is Object-Level Document Analysis of PDF Files³. The idea is to build a graph of words, discard edges linking words from different paragraphs until the connected components left are the paragraphs. The ideas we took from this paper are:

• Reduce the number of words in linear time by merging words that without a doubt belong to the same line

• Build a graph as fast as possible by looking only for direct neighbors in the four directions.

• “Best-first clustering” algorithm: sort edges (mainly by length) and choose to discard them or not one by one by using previous decisions to make informed decisions on longer trickier edges.

However, the algorithms they use to sort edges and choose to discard them, are complex sequences of rules that we found difficult to fine-tune and maintain.

From A Machine Learning Approach for Graph-Based Page Segmentation⁴, we took the idea of using a machine learning model to classify edges.

Bring everything together

1. Pre-processing

We’re extracting words and their positions using pdftotext --bbox-layout. This gives us a list of words in 2D grid and an estimation by pdftotext of word order (in an XML format similar to hOCR).

2. Build graph

We build the word graph using a custom implementation of the algorithm from³.

3. Give a should_merge score to each edge:

Using a manually tagged dataset, we train a Random Forest to give a should_merge score for each edge using features.

Features definitions:

• dist_font_size: distance between words in units of font_size (or mean font_size if not the same between words)

• xml_neighbours: neighbours according to PDF infos extracted by pdftotext (accurate ~90% of the time)

• nb_aligned_left: number of words aligned left with this word (does it looks like this word is on the left side of a paragraph)

3. Sort edges by should_merge score and keep certain

Using a threshold that gives us a precision of 1.00 during evaluation, we keep every edge that is “certain” (recall at this threshold is 0.75).

Our second threshold aims for a precision > 0.9, gives us “possible” edges.

Rest of edges are discarded.

4. Make decisions about “possible” edges using current state

Our random forest only sees two words at the time (but some features such as nb_aligned_left give more global context). For less obvious edges (with a lower should_merge score), we make decisions based on the current state of the layout. We compute a “stateful” coefficient that multiply the should_merge score.

For exemple, when considering an edge between two words from two clusters that seem to come from one same column, we increase the should_merge score.

Future work

At this point our library is too much integrated with the rest of our app to be split into an open-source solution. We are considering open-sourcing it in the future. In the meantime, do not hesitate to contact us if you want more details.

We’re also thinking of using a more end-to-end approach by combining the layout extraction with the downstream task directly in our NLP models. LayoutLM⁵ is a promising work in this direction.

References

[1] Hierarchical representation of optically scanned documents https://core.ac.uk/download/pdf/84308932.pdf

[2] A comprehensive survey of mostly textual document segmentation algorithms since 2008 https://hal.archives-ouvertes.fr/hal-01388088/document

[3] Object-Level Document Analysis of PDF Files https://www.dbai.tuwien.ac.at/staff/hassan/files/p47-hassan.pdf

[4] A Machine Learning Approach for Graph-Based Page Segmentation https://www.researchgate.net/publication/328719252_A_Machine_Learning_Approach_for_Graph-Based_Page_Segmentation

[5] LayoutLM: Pre-training of Text and Layout for Document Image Understanding https://arxiv.org/abs/1912.13318

Thanks for reading Lifen AI! Subscribe for free to receive new posts and support my work.