Artificial intelligence · 8 min read
0.016% of the weights,+40 points on MATH-500.
A small reasoning model was failing because it looped, not because it could not do math. Editing 131,072 of its 0.8 billion weights — no gradient, no fine-tuning — moved MATH-500 from 7.9% to 48.1%. Here is the protocol, the numbers, and the case where it does nothing.
In short
- Qwen3.5-0.8B lost 93% of its MATH-500 answers to the 32,768-token cap. The cause was a single repeating direction written by the MLPs of layers 15 and 19, not a lack of math ability.
- Subtracting that direction from 64 output columns per layer (131,072 weights, 0.016% of the model) took MATH-500 from 7.9% to 48.1%, GSM8K from 13.8% to 67.1% and IFEval from 21.0% to 62.3% — with no training loop and 50 forward passes of budget.
- The useful part for engineers is the diagnostic protocol: layer isolation, dose-response, and a norm-matched random control (+1.6 points, against +40). On a model that does not loop, the same method found no direction to remove and gained nothing.
Your model is not bad at math, it is stuck in a loop
Qwen3.5-0.8B is a small reasoning model, and its own model card warns about thinking loops under aggressive sampling. The MIT CSAIL team behind Sparks of Weight-Space Programming (November 2, 2026) measured how bad it gets: with thinking on and temperature 0.6 / top-p 0.95 / top-k 20 / presence penalty 1.5, 93% of MATH-500 answers ran into the 32,768-token limit. Base scores: MATH-500 7.9%, GSM8K 13.8%, IFEval 21.0%.
On the practice set, 85 of the first 96 answers repeated text until the budget was gone. The capability was already there; the loop was eating it. And the usual answers — fine-tune, buy a bigger model, add “be concise” to the system prompt — all treat the symptom.
What the authors did instead: hand a frontier model the raw weights and ask it to find and remove the cause. No gradient descent, no training loop allowed, GPU not even needed — GPT-6 Astra got a Python kernel over the 0.8 billion weights, 7,469 GSM8K + 7,275 MATH practice problems (never the test sets), and a budget of 50 forward passes.
The recipe: 5 steps, 50 forward passes, no gradient
Ignore the “LLM edits weights” headline for a second and read the edit log as a debugging session — 17 turns, each one a hypothesis with a measurement attached. That protocol is the transferable part:
- Localize before you touch anything. Turn 10: switch off neuron 11 of layer 19, the strongest writer of the loop direction, as a probe. Turn 13: removing one neuron helps (+6 points) — so the loop direction is real, and it lives in layer 19's MLP.
- Follow the amplification. Turn 26: the direction is amplified by the MLPs of layers 15 and 19. Both get the removal, not just the one where the loop is written.
- Find the dose, do not guess it. Strength 1 gains 12.5 points on 256 answers, strength 2 gains 24.6, strength 3 reaches 48.0%, strength 4 is tested next. This is a dose-response curve, not a magic number.
- Isolate the layers. Turns 94–96 test layer 15 alone, then layer 19 alone, then only the 64 neurons per layer that write the loop most strongly — which reach 26.6 points. Fewer weights, same effect, higher strength (6) retained.
- Run the control. Turn 88: same size edit, random direction. Turn 92: it moves nothing (+1.6 points). Only then is the targeted edit believable.
Final artifact: the loop direction subtracted from 64 MLP output columns in each of layers 15 and 19. 131,072 weights out of 0.8 billion — 0.016% of the model. It is a subtraction, not a retraining: same architecture, same tokenizer, same latency, same memory footprint.
The numbers, and why the control matters
All scores use the same sampling settings, thinking on, up to 32,768 tokens. The third column is what distinguishes a real fix from an early-stop hack: the share of MATH-500 answers that still hit the token cap.
| Qwen3.5-0.8B | MATH-500 | GSM8K | IFEval | Hit token cap |
|---|---|---|---|---|
| Base | 7.9% | 13.8% | 21.0% | 93.0% |
| Edited, 10 forward passes | 12.3% | 25.7% | 49.0% | 87.2% |
| Edited, 25 forward passes | 14.1% | 25.1% | 39.9% | 86.4% |
| Edited, 50 forward passes | 48.1% | 67.1% | 62.3% | 33.1% |
With 10 or 25 forward passes of search budget, the loop barely moves and the truncation rate stays above 86%. The budget you give the optimizer is part of the result — a first lesson that applies to every “agentic optimization” setup you run.
The authors also break the gain down rather than just reporting the average: +49.8 points, standard error 1.5, 579 rescues and 5 regressions over 80 runs. Of the 579 rescued answers, 576 were originally truncated outputs; accuracy on answers that already finished did not change. So this is not a model that suddenly reasons better — it is a model that stops wasting its budget. And it is not forced early stopping either: all 696 edited correct answers closed their thinking block, with a median length of 9,172 tokens.
The side effect is the interesting one. Loops are not specific to math, so instruction following rose from 21.0% to 62.3% on IFEval without being trained for, and GPQA Diamond reached 36.6%, MMLU-Pro 45.5%. Removing a structural defect improved everything that was downstream of it.
Where it breaks — and why that is the useful part
The authors ran the same protocol on Gemma 3 270M, an instruction-tuned model that rarely loops: only 8.5% of its answers hit the token limit, against 93% for Qwen. Base MATH-500: 6.8%. After 5, 10, 25 and 50 forward passes of editing: 7.0%, 7.6%, 7.7%, 7.0%. Nothing.
Reading its failures, Astra found ordinary reasoning errors — wrong arithmetic, invalid algebraic steps — with no single component behind them to switch off. Its edits stayed small and targeted (doubling one neuron that responds to numbers), and Astra itself reported the gain as not established.
The line to rememberWeight editing can remove an obstacle hiding an ability. It cannot add an ability the model lacks.
Three other caveats before anyone ships this. It is a blog post marked not published yet: no peer review, one model family, one task family. The benchmarks not shown to the editor are held out, but generalization beyond them is untested. And the edited model is a fork: you own a 0.8B checkpoint with a hand-made delta that no upstream release will carry. If the base model is updated, the direction has to be found again.
When to use it (and when to skip it)
Worth trying if you self-host a small reasoning model, your truncation rate is measurable and high, and you cannot afford a fine-tuning pipeline or a bigger model at the same latency. This is a 50-forward-pass diagnostic on a quantized model, not a training run.
Skip it if the model does not loop — the Gemma result is the proof: no single cause, no gain. Skip it if the gap is a genuine capability gap. Skip it if you call a frontier API, since you cannot edit weights you do not host. And skip the “let the model edit its own weights” version if you cannot run a regression suite afterwards: a global weight edit is exactly the kind of change that fixes your benchmark and breaks your refusal behaviour.
Production checklist
- Measure the truncation rate first. If p95 output length sits at your
max_tokensceiling and answers repeat themselves, you have a loop, not a reasoning problem. That single metric decides whether this technique applies at all. - Never accept an edit without a control. Same number of weights, random direction, same strength. If the control also gains a few points, your measurement is noise.
- Report the confusion matrix, not the average. 579 rescues, 5 regressions is a result; “+40 points” alone hides the five answers that broke.
- Keep the test sets away from the optimizer. Here MATH-500, GSM8K and IFEval were never shown during the 50 forward passes. Do the same, or you are measuring the editor's memory.
- Version the delta, not the checkpoint. Store indices, direction and strength in a config you can replay and roll back, and re-run your refusal and instruction-following suites after every edit.
Need help designing reliable AI workflows, auditing an LLM architecture, or training your engineering team? Explore how we work at Kaleido-Lab.