Read more

A common pattern to use to implement the memory concept we have in chat-based LLMs into an audio system by using the conversation’s transcript. But this approach is sub-optimal because the transcript only contains a record of what is said - it doesn’t tell an application …

Read more

Read more

Muse Glimmer 30B is a new 30-billion-parameter causal language model from the Meta Superintelligence Lab, distilled directly from Muse Spark. Released under the permissive Apache 2.0 license, it is purpose-built to execute complex, autonomous agentic workflows entirely on consumer hardware without relying on cloud infrastructure.

You can read more about …

Read more

More info is available on AImodels.fyi here!

Key …

Read more

Read more

Laguna S 2.1 is a new 118B parameter Mixture-of-Experts (MoE) model purpose-built for agentic software engineering and long-horizon tasks. Sitting comfortably between the XS and M.1 models in the Laguna family, it achieves high efficiency by activating just 8B parameters per token.

More info is available on AImodels.fyi here!

Highlights

• Massive 1M context …

Read more

Read more

This tension sits at the heart of test-time computation scaling. The question isn’t whether models benefit from extra thinking, but how to let them think without breaking latency. Parallel Loop Transformers (PLT) were designed to solve exactly this problem: instead of looping sequentially, run all loops simultaneously on different hardware. Use position offsets to tell the model which iteration it’s in, and add gating mechanisms so the model can decide whether to use fresh computation or rely on what it already learned. In theory, this lets loop count become a design choice rather than something imposed by speed constraints. You could loop 10 times if it helped.

But should you? That question led researchers to train LoopCoder-v2, a family of 7-billion-parameter code models with loop counts ranging from one to five. What they found was counterintuitive enough to demand explanation: two loops was optimal. Three loops got worse. Not incrementally worse, as diminishing returns would suggest, but genuinely regressed. The model produced worse code despite having more refinement opportunity.

How parallel looping actually works

The standard approach to test-time scaling is straightforward: take a transformer layer and apply it repeatedly to the hidden states. Each pass refines the computation, pushing the model’s hidden states through the same learned operations again and again. But if you do this sequentially, you’re blocked. Loop two can’t start until loop one finishes, so latency scales linearly with loop count. The KV-cache, which stores attention patterns from each position, grows proportionally too.

Parallel Loop Transformers invert this constraint by running all loops in parallel. The trick is telling the model which loop it’s in. This is where cross-loop position offsets (CLP) come in: instead of using the same position indices for every loop, shift them. Loop one uses positions 0, 1, 2... Loop two uses positions N, N+1, N+2... The model learns that different position ranges correspond to different refinement stages. Because loops run simultaneously, latency stays roughly constant regardless of how many you add. Memory cost scales too, but far more gently than sequential looping.

The second component is shared-KV gated sliding-window attention (G-SWA). Instead of recomputing everything from scratch in each loop, position-by-position gates decide whether to use fresh computation from the current loop or rely on what was cached from loop one. This is subtle but crucial: it gives the model fine-grained control over when it refines versus when it reuses.

Together, these mechanisms solve the engineering problem: loop count is no longer constrained by latency. But solving the engineering problem creates a new scientific one. If you can loop 10 times at nearly the same cost as looping twice, why don’t you? That’s the question that defines the rest of the paper.

Empirical surprise

LoopCoder-v2 was trained from scratch on 18 trillion tokens across a family of models with one, two, three, four, and five loops. All models were the same size (7 billion parameters), trained on the same data, and evaluated on the same benchmarks: code generation, code reasoning, and agentic software engineering tasks including SWE-bench Verified and Multi-SWE.

Read more

This is roughly how current AI agents approach autonomous research tasks. When Claude or Codex are asked to optimize a machine learning model, improve a data pipeline, or engineer an algorithm, each interaction operates in isolation. The agent calls the model, gets code, runs it, observes results, and then starts the next iteration from scratch. If something fails, there’s no structured way to remember why, what assumptions broke, or what that teaches about the problem space. The agent might try a similar failing approach again, or swing to a completely different strategy with no intermediate learning.

Over long horizons, this becomes profoundly wasteful. Current systems explore locally but their exploration is essentially memory-less. The agent can’t say “we learned that approach A doesn’t work because of property X, so approach B, which relies on property X, probably won’t work either.” They plateau because they never develop a coherent theory of the problem space.

This is the inefficiency that a new framework called Arbor addresses. The framework treats autonomous research not as a sequence of disconnected trials, but as a cumulative process where strategy, execution, and evidence compound over time.

Reframing research as a process of knowledge accumulation

Instead of asking “how can we execute experiments faster,” ask “how can we help an AI agent actually think like a researcher.” A human researcher doesn’t run experiments randomly. They maintain a mental model of the problem, track hypotheses and whether they’ve held up under evidence, and use that accumulated understanding to choose the next experiment.

Arbor materializes this mental model as a hypothesis tree, a persistent data structure that links hypotheses, artifacts, evidence, and distilled insights across time. Unlike a linear notebook, this tree structure lets information propagate sideways. A lesson learned in one branch can inform decisions in another. A hypothesis might be refined into sub-hypotheses as evidence accumulates. A branch might be pruned if results rule it out.

This transforms the research process. Instead of executing experiments in isolation, each result updates not just “the best solution found so far” but “what we know about this problem.” That knowledge becomes the basis for the next decision.

The hypothesis tree as persistent memory

The architecture has three components. The coordinator is a language model that reads the hypothesis tree, interprets accumulated evidence, and decides which hypotheses to test next. The executors are isolated processes that implement and run specific experiments. And the hypothesis tree itself is the persistent record that connects them, storing every hypothesis, its experimental evidence, the interpretation of that evidence, and lessons that generalize.

Let’s consider a concrete task: improving a model’s accuracy by tuning training procedures.

The tree might start with a single hypothesis at the root: “we can improve performance by adjusting hyperparameters.” An executor tests this by running an experiment. Results come back showing that learning rate matters but batch size doesn’t. The tree records this. The coordinator reads the evidence and branches the tree further: “learning rate has a sweet spot around 0.001; let’s explore how it interacts with warmup schedules” versus “batch size doesn’t matter, so let’s explore data augmentation.”

As the tree grows, the coordinator’s decisions become more informed. It no longer makes random choices about what to try. It works from the map that the tree builds. It recognizes when two branches are testing the same underlying assumption and can merge findings across them. It knows which areas have been thoroughly explored and which deserve more investigation.

Coordination as strategic research direction

Read more

When an agent successfully resolves a bug, that success comes from at least three distinct capabilities working together. The agent had to understand the repository structure well enough to find relevant files. It had to pinpoint the exact lines within those files that mattered. It had to diagnose what was wrong and write a correct fix. A binary resolved/unresolved label tells us these three things worked in concert, but not which ones are weak links. Maybe an agent fails 90% of the time because it explores repositories poorly, not because it can’t write patches. Or maybe the opposite is true. Current benchmarks can’t tell us.

This is the fundamental measurement problem that SWE-Explore sets out to solve. Rather than evaluate the entire pipeline, the benchmark isolates one critical phase: repository exploration. This seemingly small shift in focus reveals something important about how coding agents actually work and where the real bottlenecks lie.

Decomposing a complex skill

The insight underlying SWE-Explore is that a complex problem can be understood by breaking it into measurable parts. Current benchmarks treat coding task completion as a holistic prediction problem. An issue either gets resolved or it doesn’t. But this masks what’s actually happening underneath.

Look at that visualization and the abstraction becomes clear. Three overlapping capabilities get compressed into a single number. You lose the ability to diagnose whether an agent’s failures stem from poor repository understanding, inaccurate line-level localization, or weak repair logic. This is like assessing a doctor by only checking recovery rates, without examining whether they correctly diagnosed the disease or ordered the right tests.

By isolating exploration as a standalone evaluation target, SWE-Explore makes it possible to measure something more granular: given a repository and an issue description, can the agent return a ranked list of relevant code regions efficiently? This single question opens up a much clearer picture of what modern coding agents are actually good at.

Defining exploration precisely

Exploration, in this framing, means the ranked list of code regions an agent thinks are worth examining before attempting any repair. It’s the pre-reading phase of problem-solving, the phase where a developer orients themselves to understand the landscape: what files are involved, what functions call what, where do error messages originate.

The benchmark defines this as a retrieval problem with specific constraints. An explorer gets a fixed line budget, like a developer with limited time to read code before diving into fixes. Within that budget, the explorer returns a ranked list of lines it considers relevant. The question is fundamentally empirical: which lines would someone actually need to read to understand and fix this bug?

This differs from traditional code search because it operates at line granularity rather than file level, ranking matters (finding critical code early beats finding it eventually), and relevance is specific to the bug rather than generic. The framing reflects reality: developers don’t examine entire repositories uniformly. They prioritize based on what might matter.

Deriving ground truth from successful paths

The clever part is figuring out what correct exploration actually looks like without requiring humans to manually annotate every instance. Instead, the researchers extracted ground truth from agents that successfully solved issues. When an agent fixes a bug, it leaves a trail: which files did it open, which line ranges did it examine?

Read more

This assumption is wrong, not because we lack compute or data, but because it misunderstands what a figure actually is. A scientific figure is not a monolithic prediction task. It is a structured composition of discrete semantic components, where errors occur locally. A bar chart fails when its y-axis label is misplaced, not because the entire visualization is fundamentally flawed. A phylogenetic tree fails when a branch angle is off by five degrees. A molecule diagram fails when a bond is the wrong color. These are not problems that scale to solve with a bigger backbone model. They are problems that demand intelligent coordination among specialists.

This is the core insight behind Crafter, a multi-agent harness for scientific figure generation that achieves something existing systems cannot: it generalizes across completely different figure types and input conditions without architectural changes. Rather than training one model harder, the system deploys multiple specialized agents that debate and refine specific components until they converge on a good figure.

When monolithic models meet diverse problems

Researchers need to generate bar charts from captions, phylogenetic trees from sketch inputs, molecule diagrams from reference images, and dozens of other figure types under widely varying input conditions. Existing systems each carve out a narrow slice of this problem space. SciFig targets bar charts from text. AutoFigure-Edit handles figure editing but requires raster inputs. Pixels-Paths works with multi-agent frameworks but for different structured outputs.

Each system optimizes for one task type and one input modality.

When you task a single model with solving all of these problems simultaneously, it learns to average. It produces mediocre compromises that work reasonably well across all cases but excellently for none. This is an architectural problem. The system is being asked to compress entirely different reasoning patterns into a single bottleneck.

The real issue surfaces when you examine failure modes. They are almost never global catastrophes. A generated figure usually gets most things right. Instead, failures cluster in specific locations: a misplaced element, a wrong styling choice, a label in the wrong position. These are localized problems that benefit from localized solutions, not wholesale regeneration.

Rethinking generation as coordinated problem-solving

Crafter reframes figure generation as a multi-agent conversation rather than a single neural network’s dream. The architecture consists of four specialized roles that iterate until convergence.

The intent reasoner begins the process. It does not generate a figure. Instead, it reads whatever input the user provides, whether caption, sketch, reference image, or combination, and produces a semantic representation of what success looks like. This semantic language becomes the common currency that all downstream agents use to evaluate proposals and feedback. By decoupling intent interpretation from rendering, the system can handle any input modality without retraining.

The plan generator does not produce one figure. It proposes K candidate plans, each representing a different approach to satisfying the intent. This matters because committing to the wrong approach early is expensive, but filtering bad approaches before rendering is cheap. By generating alternatives upfront, the system explores a broader space than greedy decoding ever would. Each plan is a structured specification of what elements should appear, where, and with what properties.

Read more

Compare this to how we train neural networks. With weights, we have a clear loss signal, bounded update steps, validation gates, and reproducible improvement. We can inspect learning curves. We can measure generalization. We know whether we’re making progress or just fitting noise. With skills, we’ve been winging it. Someone writes a prompt, maybe tweaks it based on a few examples, and ships it. If it doesn’t work well enough, the process starts again, but there’s no systematic way to improve.

As agents become more capable and deployed at scale, the skill becomes the bottleneck. A frozen model can’t improve its behavior without retraining, which is expensive. Self-revision is unreliable, and hand-crafting doesn’t scale. We need a way to improve skills the way we improve models: systematically, with reproducible results, with validation gates that prevent chasing noise.

Treating skills as trainable parameters

The core insight behind a new SkillOpt paper is simple: a skill document is just external state that modifies how a model behaves. It’s not fundamentally different from internal weights, except it lives outside the model and can be edited without retraining. What if we treated it exactly like a neural network parameter, just in text space instead of number space?

The key move is to freeze the model completely and optimize only the skill document itself. This is backward from how we usually think about improving agents—fine-tune the model, scale it up, use a better architecture. But it’s actually more aligned with how optimization works in practice. The model becomes a fixed function. The skill becomes the variable we’re training.

Once skills are framed as parameters, we can apply real optimization techniques. We get reproducibility. We get validation gates that prevent accepting false improvements. We get learning curves that show actual progress instead of random wandering. The skill becomes a learnable object, no different in principle from training a neural network weight.

How SkillOpt optimizes skills systematically

The machinery works like a very disciplined form of skill editing. Run the target model many times with the current skill, collecting successes and failures. Feed those rollouts to a separate optimizer model, asking it to identify what went wrong and propose targeted edits. The optimizer suggests changes: add this guideline, remove that constraint, replace vague language with specific examples. But crucially, each proposed edit gets tested on held-out validation data first. If it improves the validation score, keep it. If not, reject it. Only confirmed improvements stick.

This validation gating is the crucial difference from self-revision. You’re not letting the main agent tinker with its own skill unsupervised. Instead, there’s a referee (validation data) and a thoughtful editor (the optimizer model) checking every change before it lands.

The full pipeline cycles across epochs:

• Rollout and collection starts each epoch. Run the target model many times with the current skill, recording trajectories, successes, and failures.

• Optimizer reflection comes next. The optimizer model analyzes the rollout batch, identifying patterns in what succeeded and what failed. It then proposes bounded edits to the skill document. Crucially, the edits are constrained: add/delete/replace single statements rather than wholesale rewrites. A textual learning-rate budget caps how much the skill can change per epoch, keeping updates stable and preventing wild swings.

• Validation gating tests each proposed edit on held-out validation data. An edit is accepted only if it strictly improves the validation score. Rejected edits go into a buffer so the optimizer doesn’t propose the same failing changes repeatedly.

• Meta-updates and scheduling across epochs keep optimization stable and avoid overfitting to individual rollouts. The system uses slow updates and epoch-wise adjustments inspired by meta-learning.

A subtle but important detail: the optimized skill is just text. At inference time, you pass it to the model. No extra models running, no additional latency overhead. The entire optimization happens offline.

The target model executes tasks with a current skill, the optimizer model analyzes trajectories and proposes bounded edits, and a validation gate accepts only edits that improve held-out performance

Each epoch: the frozen target model executes rollouts with the current skill, the optimizer model reflects on successes and failures, proposes bounded edits, merges candidates, and only accepts edits that improve validation performance

Evidence of improvement across diverse models and benchmarks

Read more

The obvious solution is masking. Replace specific details with generic placeholders. Diabetes becomes [MEDICAL_CONDITION]. $200 monthly becomes [FINANCIAL_METRIC]. The cloud never sees the actual values, so privacy is protected.

Read more

The practical stakes are high. Search agents power research tools, web-based reasoning systems, and complex information retrieval. But most breakthroughs happen inside companies with unlimited budgets. Academic researchers hit a wall: the techniques that work are proprietary, the datasets are private, and the computational resources required seem astronomical. This creates a frustrating bottleneck where innovation clusters around industrial research labs, leaving the broader research community unable to experiment, iterate, or contribute meaningfully to the field.

Why industrial pipelines felt inevitable

The prevailing wisdom emerged naturally from how major AI labs approached agent training. They borrowed techniques from large language model development: start with massive pre-training to build foundational knowledge, apply continuous pre-training to adapt that foundation to new domains, fine-tune on supervised examples to teach specific behaviors, then polish everything with reinforcement learning to optimize against reward signals. Each stage supposedly unlocks something the previous stage couldn’t reach.

The logic seemed bulletproof. If you want frontier-level capabilities, you need frontier-level methods and resources. Pre-training builds knowledge. Continuous pre-training specializes it. Supervised fine-tuning teaches specific skills. Reinforcement learning optimizes for actual performance. Remove any link in this chain and you’d expect degradation.

This assumption led to a clear conclusion: building state-of-the-art search agents required industrial-scale infrastructure. Tongyi DeepResearch, for example, achieved strong performance through exactly this pipeline, spending enormous computational resources across all four optimization stages. For any academic team or resource-constrained organization, this seemed like an insurmountable barrier.

The dataset design revolution

Then came a simpler observation: what if the bottleneck wasn’t the algorithm, but what data you fed it?

The researchers behind OpenSeeker-v2 noticed something crucial. Most work on agent training focused on optimization techniques, assuming the data was a fixed quantity. But what if the data itself could be fundamentally restructured? What if you could take the same training paradigm (simple supervised fine-tuning) and make it exponentially more powerful just by changing which trajectories you used as examples?

Read more

Xiaomi released MiMo-V2.5-Pro under an MIT license a few days ago, and the response has been quietly enthusiastic on r/LocalLLaMA but barely registered on other places like Hacker News. The phone-manufacturer-makes-LLM angle keeps tripping people up. MiMo-V2.5-Pro is a Mixture-of-Experts model with 1.02 trillion total parameters and 42 billion active per token, and it landed at 54 on the Artificial Analysis Intelligence Index - squarely in frontier territory. On reddit, u/lendo93 reported that in their benchmark suite the model averages higher than Opus 4.6 on coding reasoning, agentic work, and decision making.

About the model

The architecture is built around two ideas...

• First, hybrid attention: 60 of 70 layers use sliding-window attention with a window of 128 tokens, while only 10 layers run global attention, in a 6:1 SWA-to-GA ratio. This cuts KV-cache storage by roughly 7x compared to a standard transformer, and it’s how Xiaomi gets a usable 1M-token context window without the cache exploding.

• Second, multi-token prediction. There are three lightweight MTP modules with dense FFNs that predict ahead of the main token stream, and Xiaomi reports this triples inference output speed. The MTP modules are trained natively rather than bolted on as speculative decoding, so the speedup compounds with the long-context handling.

Read more

Read more

Google DeepMind released Gemma 4 on Easter weekend, and the local AI community responded like it was Christmas. The family spans four sizes - E2B, E4B, 26B A4B (MoE), and 31B dense - with the 31B landing on Hugging Face under an Apache 2.0 license. That licensing change matters: previous Gemma releases used a custom Google license with usage restrictions. Apache 2.0 removes that friction for commercial deployment.

The benchmark numbers are good. The 31B scores 89.2% on AIME 2026 without tools, 80% on LiveCodeBench v6, and a Codeforces ELO of 2150. For comparison, Gemma 3 27B scored 110 on that same Codeforces benchmark. The smaller E2B model - which has only 2.3 billion effective parameters - outperforms Gemma 3 27B on MMLU Pro (60% vs 67.6%), GPQA Diamond (43.4% vs 42.4%), and LiveCodeBench (44% vs 29.1%). Some users called it “insane” - a fair reaction.

What the 31B actually does

The 31B is a dense model with 30.7B parameters, a 256K token context window, and a hybrid attention mechanism that interleaves local sliding window attention (1024-token window) with global attention layers. The final layer is always global. For long-context tasks, global layers use unified Keys and Values with Proportional RoPE (p-RoPE), which is how Google gets memory efficiency at scale without completely tanking reasoning quality.

Multimodal support covers text and images, with a 550M-parameter vision encoder. The model can process images at variable resolutions using a configurable token budget (70 to 1120 tokens per image) - lower budgets for speed on classification tasks, higher budgets for OCR and document parsing where fine-grained detail matters. The smaller E2B and E4B models additionally support audio input for up to 30 seconds, enabling single-model pipelines for voice applications.

Thinking mode is built in and configurable. Include <|think|> in the system prompt to activate it; remove it to disable. The model outputs its reasoning trace in <|channel>thought\n[reasoning]<channel|> blocks before the final answer. In multi-turn conversations, you strip the thinking content from history before the next user turn - thinking traces don’t get passed back.

Coding is a clear strength. The 31B’s Codeforces ELO of 2150 is a significant jump from anything in the open-weight space at this size. On r/LocalLLaMA, u/DigiDecode_ posted a screenshot showing the 31B ranking above GLM-5 on LMSys, which landed with some force given GLM-5’s reputation.

How to run it

The model is available on Hugging Face and loads through the standard Transformers interface. For text and image inputs:

pip install -U transformers torch accelerate

from transformers import AutoProcessor, AutoModelForCausalLM

processor = AutoProcessor.from_pretrained("google/gemma-4-31B-it")
model = AutoModelForCausalLM.from_pretrained("google/gemma-4-31B-it", dtype="auto", device_map="auto")

Use AutoModelForMultimodalLM instead if you’re working with images or video (or audio on the E2B/E4B variants).

Recommended sampling parameters from Google:

• temperature=1.0

• top_p=0.95

• top_k=64.

For thinking mode, pass enable_thinking=True to apply_chat_template and use processor.parse_response() to separate the thinking trace from the final answer.

GGUF quantizations are available via Unsloth. NVIDIA also offers a free API endpoint at build.nvidia.com at 40 requests per minute, which is useful for evaluation before committing to local deployment.

For local inference, Google’s recommended config for llama.cpp: --flash-attn on, --temp 1.0, --top-p 0.95, --top-k 64, --jinja. You’ll want KV quantization unless you have unusual amounts of VRAM available.

The KV cache problem

This is where the reception gets complicated. The 31B has a massive KV cache footprint - a consequence of its multimodal architecture. On reddit, users reported that on a 40GB VRAM card, the Q8 quantization (35GB) can’t fit even a 2K context without also quantizing the KV cache to Q4. Qwen3.5-27B, by comparison, fits at full context without KV quantization on the same hardware. A llama.cpp update since release improved this by properly implementing Sliding Window Attention, which reduces the fixed KV allocation significantly - but you need to re-download the Unsloth quants if you grabbed them at launch.

Read more

Muse Spark shipped Wednesday. It’s the first model out of Meta Superintelligence Labs, built over nine months under Alexandr Wang after Zuckerberg spent $14.3 billion on a 49% stake in Scale AI and brought Wang in as Meta’s first chief AI officer. It accepts voice, text, and image inputs. It produces text-only output. It has a fast mode and reasoning mo…

Read more

Read more