🧪 Introducing Pipette, the benchmark built for on-device intelligence

Benchmarks are a critical tool for making technology decisions, but only if what they’re measuring reflects real-world use. For edge models, traditional benchmarks fall short, measuring performance on a cloud server with limitless resources, often on a single task.

That’s why we’ve partnered with Artificial Analysis to develop Pipette, a new, open-source benchmarking platform specifically for foundation models on edge devices. Pipette is built around a simple idea: a model’s on-device performance depends not just on the model, but the entire deployed system, including quantization, runtime, hardware, and context.

Our first release includes several features aimed at delivering metrics that reflect real-world, on-device performance:

• 1000+ tested configurations across 30+ models, multiple quantization formats, runtimes, devices, and context lengths

• Five on-device performance metrics, including throughput, latency, context scaling, and memory use

• Open-source benchmark clients for macOS, Windows, iOS, and Android, including native mobile apps that run benchmarks directly on target devices

• An interactive dashboard that connects measured device performance with model quality, making trade-offs between speed, memory, latency, and capability clear and inspectable

Pipette also captures the sometimes dramatic impact of context on edge model performance, in ways that standard benchmarks don’t. Several good examples show up in the launch data:

• One model maintains its speed and performance as the conversation lengthens, while another nearly identical-looking one degrades rapidly.

• Sparse models with quick response times end up demanding more memory than most smartphones can spare.

• Different models “win” an evaluation depending on the kind of task being performed, even when their performance numbers are comparable.

Typical benchmarks miss these trade-offs, collapsing such varied behaviors into a single “best model” score.

Pipette’s benchmark methodology has been independently verified by Artificial Analysis, but be sure to check it out for yourself → Explore the Pipette leaderboard

IN THE NEWS

📢 Liquid co-founder Daniela Rus included in the TIME100 AI list

Daniela Rus was recognized as one of the world's most influential people in artificial intelligence by TIME this year, alongside other AI luminaries like Fei-Fei Li, Dario Amodei, Sam Altman, and more. Besides highlighting her career leading MIT’s pioneering Computer Science and AI Lab (CSAIL) and her decades of robotics work, the profile celebrates the uniquely “nature-inspired, energy-efficient models” Rus co-developed at CSAIL, leading to the spin out of Liquid AI.

→ Read the profile

ENGINEERING

⚡ LFM2.5 Q4_0: Four-bit models that don’t sacrifice quality

Quantization makes models small and fast enough to run on constrained hardware, but often at the cost of reduced quality. But the trade-off isn’t linear, and doesn’t always have to cripple performance.

Case in point: our recently released, updated 4-bit checkpoints for LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B. These models are trained with Quantization-Aware Distillation (QAD), which distills a high-precision teacher directly into a quantized student, rather than quantizing after training is done.

In practice, this means models with the low memory footprint and high throughput of Q4_0 GGUFs, but without the dramatic accuracy loss that quantization often forces. Across all four models, the new checkpoints retain roughly 97% of their BF16 baseline performance, and suffer 73.4% less quality loss than a standard post-training Q4_0 quantization.

All of this happens under the same format, tensor layout, and runtime path so there’s no loss of throughput compared with native Q4_0. The upshot: smaller, more edge-friendly models that leave a lot less performance on the table.

→ Read the release

→ Download the models on Hugging Face: LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B.

🌐 Deploy everywhere: Putting agents on any device with the new LFM2.5-2.6B model

What’s small enough to run on a phone, fast enough to stay responsive on a CPU, and capable enough to power agentic workflows? Edge AI agents have been improving rapidly in recent months, but most still involve painful trade-offs between speed, size, compute, and cloud LLM usage.

In many cases, escalating a task to the cloud solves whatever immediate problem an edge agent is facing, but it also brings drawbacks. There’s the cost—in time, tokens, and privacy—which defeats at least part of the purpose of putting agents on-device in the first place.

But more important is the missed opportunity for proactive agentic work. Agents that run entirely on-device can use local, periodic inference over device context to identify useful tasks before a user explicitly asks for them. Keeping that loop local can reduce network latency, preserve privacy, work through connectivity gaps, and make frequent inference more practical.

Our newly released LFM2.5-2.6B model is optimized for just that, opening up the possibility of an edge agent that does useful work before users realize it needs to be done. Its size, speed, and capabilities are ideally suited to high-volume agentic workloads, and its training was designed specifically to complete chained agentic tasks while staying on-device.

Learn more about our agentic, on-device model below and contact our team for a demo.

→ Read the blog
→ Download the model

PARTNERSHIPS

💻 MacPaw partners with Liquid AI to bring on-device AI to millions of Mac users

Liquid AI and MacPaw have announced a strategic, long-term partnership to co-develop the technology stack for local AI on the Mac, with the goal of bringing efficient, private, on-device LFMs to millions of Mac users.

LFMs will be fine-tuned specifically for macOS AI tasks, running locally on Apple silicon through MacPaw’s Elix inference engine, while MacPaw’s Mnemos memory layer lets the assistant retain context and get more useful over time. Together, that’s on-device intelligence, persistent memory, and native task execution in one product — something that hasn’t existed on the Mac before.

→ Read the announcement

IN THE NEWS

📢 Recent coverage featuring Liquid AI, from post-transformer architecture to agents on a Raspberry Pi

MIT Technology Review on what comes after the transformer. Ramin Hasani explains our hybrid architecture and why the brain’s 20 watts is the efficiency target.

VentureBeat covers the LFM2.5-2.6B launch. Maxime Labonne on building for agentic harnesses rather than chatbots and running real agents on a Raspberry Pi.

TechCrunch on our MacPaw partnership and how we select an architecture that is different and tailored to the hardware first.

McKinsey convenes us with AMD, Dell, and Mercedes-Benz to discuss where AI is headed, from pushing intelligence onto edge devices to what it takes to actually scale AI across an enterprise.

The Boston Business Journal on how Liquid AI builds models its own way and why our path differs from the frontier labs.

ENGINEERING

🧩 Tokenizer Expansion: a bigger vocabulary without retraining from scratch

On-device, decoding is memory-bandwidth limited, and the LM-head reads the entire vocabulary at every step. Cloud models absorb a large vocabulary easily - the embedding and output matrices are a small fraction of their parameters - but edge models can’t, so they ship compact vocabularies and live with fragmentation outside their priority languages. LFM2’s original 65K tokenizer left almost no budget for Hindi, Vietnamese, or Thai. Here’s how we doubled it to 128K on a checkpoint we’d already trained:

1. Extend, don’t replace: we froze the original BPE merges and built new tokens on top, so most of the original 65K carried over unchanged and every new token decomposes exactly into old ones.

2. Initialization comes for free: new embeddings start as the mean of their sub-tokens. Nothing random, no cross-tokenizer alignment problem to solve.

3. Two-stage adaptation, because training everything at once degraded what already worked:

   • Stage 1: new embeddings alone, 600B tokens

   • Stage 2: the full model, 400B tokens

4. The results:

   • Thai needs 4.0× fewer tokens, Vietnamese 2.6×, Hindi 2.4×

   • That’s roughly 2.2–3.7× faster decoding on-device for these languages

   • Quality on previously-supported languages holds steady, so there’s no tradeoff

If you own your tokenizer and can continue its original merges, this recovers the latency that fragmentation was costing your under-served languages without throwing away the pre-training you already paid for. LFM2.5-8B-A1B and the expanded tokenizer are both live on Hugging Face with open weights, and the full method, benchmarks.

→ Read the blog • Read the technical report

COMPANY NEWS

🚀 40 million downloads. Thank you.

Liquid Foundation Models (LFMs) have officially crossed 40 million downloads from the community - a milestone we couldn’t have hit without you.

As proud contributors to open source and firm believers in open-weight AI and American AI leadership, we’re just getting started. The next generation of lightweight, powerful LFMs is coming, and we’re accelerating their open-weight release to the world.

Let’s go. We can’t wait to see what you build.

→ Liquid foundation models • Read open weights letter we signed

WHAT IF

🛡️ What if your coding agent is reading more than you think?

ShieldFlow is Liquid’s on-device privacy layer for AI, redacting sensitive data locally before it ever reaches a cloud agent. Felipe Benavides traces what it caught during an ordinary vibe coding session:

• 77 sensitive entities flagged in session traffic: names, usernames, emails, device identifiers

• None of it prompted. The source turned out to be forgotten files in a downloads folder

• Redacted locally, so the frontier agent never saw any of it

Join our mailing list to get the latest on model releases, research, and deployments from Liquid AI.

Subscribe now

Liquid Foundation Models are free to use for hobbyists and organizations with less than $10m in ARR. If you work in enterprise and have a use case for LFMs, feel free to contact us.

Contact us

ENGINEERING

🔁 Antidoom: killing the "doom loop" in reasoning models

Small reasoning models have a nasty habit: on hard math and coding prompts, they emit a span, then repeat it over and over until the context window runs out. We call it the doom loop, and the usual fixes (repetition penalties, RL) are either band-aids or expensive. Antidoom is our open-source method that removes it at the source.

• Surgical, not scorched-earth: the loop almost always starts on a single overtrained token (”Wait,” “So,” “Alternatively”). Antidoom retrains just that token and leaves the rest of the distribution intact.

• Powered by FTPO (Final Token Preference Optimization): trains only the trailing token mid-generation, spreads probability across multiple coherent alternatives, and uses a KL-like loss in logit space to avoid disturbing the rest of the vocabulary.

• The results speak for themselves:

  • Early LFM2.5-2.6B checkpoint: doom-loop rate fell from 10.2% → 1.4%

  • Qwen3.5-4B: fell from 22.9% → 1% under greedy sampling

  • Eval scores rose across the board in both cases — entirely from eliminating loops, not teaching the model anything new.

• Fast and cheap: the whole pipeline runs in a few hours (about two hours to generate the training set on 8 H100s, one to two hours to train on a single H100).

If your small reasoning models loop on hard prompts, Antidoom recovers the accuracy those loops were costing you. Generation, detection, and the FTPO trainer are all open source.

→ Read the blog · Get the code on GitHub

📐 IFStruct: measuring whether models actually follow the schema

Constrained generation can guarantee valid JSON, and still hand you the wrong JSON. As our CTO Mathias Lechner puts it: no more “syntactically correct garbage.” A model can nail the syntax and still get a type wrong, invent a field, or pick an enum value that doesn’t exist. IFStruct is our new open-source benchmark for the thing that actually matters: output that satisfies the schema.

• Tests requests the way real users write them: chat requests, bullet lists with explicit paths, raw JSON Schema, annotated JSON/YAML, and ASCII tables. Half are rewritten into natural prose.

• Binary scoring: every field, type, enum, bound, and count has to be right, with no invented keys. Frontier models score near 100%.

• The task is highly learnable: its generative design makes training data easy to produce, and the same yes/no check that scores the benchmark can serve as the RL reward signal.

• Which is why a 350M model beats models 10x its size: LFM2.5-350M jumps from 21.10% → 44.90% after training, ahead of Qwen3.5-4B (36.25%) and granite-4.0-h-tiny (38.75%).

→ Read the blog · Download the test set on Hugging Face

COMPANY NEWS

🛍️ 1 billion requests on Shopify

Liquid AI powers Shopify's search and recommendations with sub-20ms foundation models, now surpassing 1 billion requests processed after just six months!

Read more about our multi-year partnership.

WHAT IF

🧬 What if a language model could design biology?

Liquid builds foundation models over amino acid tokens, enabling promptable protein design from sequence alone. In this discussion, CTO Mathias Lechner and ML scientist Kaeli Kaymak-Loveless dig into:

• Designing proteins from sequence alone

• Engineering an efficient plastic-degrading enzyme

• How the same LFM could point toward new cancer therapeutics

Join our mailing list to get the latest on model releases, research, and deployments from Liquid AI.

Subscribe now

Liquid Foundation Models are free to use for hobbyists and organizations with less than $10m in ARR. If you work in enterprise and have a use case for LFMs, feel free to contact us.

Contact us

Tell us in the comments below:

What are you building or studying? Shipping products, publishing research, hacking on side projects, we'd love to know.