Tree-Ising repairs looping LLMs via bit-level Ising edits at $0.90 per fix
pmttyji · reddit · 2026-11-11
Tree-Ising casts LLM weight-space repair as sparse discrete optimization over BF16 weights: forward statistics define an objective (weaken a loop direction, strengthen a concluding direction, preserve productive outputs), exactly decomposed into per-row QUBO/Ising problems solved via GPU simulated annealing or simulated bifurcation, using sign flips and exponent shifts that preserve group exponents/mantissas.
Key results vs the MIT GPT-6 Astra agent repair of Qwen3.5-0.8B (which cost 158 turns, 89.7M tokens, 15.6h, $312/edit):
- MATH-500: 54.6% vs agent's 51.7% (base 7.9%); bifurcation solver hits 55.1%
- 13–17% fewer weights modified, 2.2–2.3× fewer BF16 bit flips
- Only 4 forward-pass requests, zero LLM tokens, $0.90 GPU time, 31 minutes
- Transfers to Qwen3.5-2B: greedy GSM8K 21.6% → 69.4% with layers/dose chosen from base model only
- Proof-of-concept eight-variable subproblem solved with QAOA on an IBM Heron processor
Related event: Tree-Ising: Gradient-Free Bit-Level Editing Fixes LLM Reasoning Loops(2 posts)→
More from Research
- ARLoop releases fully open looped transformer recipe; 1.2B model beats Huginn-3.5B — Lianhuiq · 2026-11-11
- UniPath (NeurIPS 2026): adaptively coordinating understanding and generation in one model — jindong_wang92 · 2026-11-11
- Scale AI's 1,200-run study: PPO wins at small batches, GRPO catches up at scale — heghbalz · 2026-11-11
- Open-source effort designs personalized pancreatic cancer vaccine for 34-year-old patient — jajoosam · 2026-11-11
- Ross Girshick posts CVPR slides recounting 1.5 decades of object detection and its link to the LLM era — CatAstro_Piyush · 2026-11-11
- KAIST's WikiTrace: evidence-linked memory fixes agent "memory-induced degradation" — dair_ai · 2026-11-11