Nanbeige4.2-3B on Apple Silicon: Fixing Deployment Bugs and Decreasing Looped Transformer Memory Overhead
John T. Halloran
cs.AI, cs.LG
2026-08-14
The Nanbeige4.2-3B Looped Transformer checkpoint cannot run via Hugging Face transformers on Apple Silicon: after fixing five deployment bugs, doubling attention memory from layer reuse, a silently replaced system prompt, and an MPS memory leak, MCPMark goes from unevaluable to 30% with context width extended 2.7x.
Nanbeige4.2-3B is a 3B agentic model whose selling point is a Looped Transformer (LT): the same stack of layers runs twice to add effective depth without adding parameters, with a model card claiming wins over Qwen3.5-4B and 9B on agentic benchmarks. Put it in a ReAct-style harness on Apple Silicon via the MPS backend and it does not run. This paper is a full repair and evaluation log: five deployment bugs, one architecture-level memory problem, one silently replaced system prompt, and one MPS-specific memory leak, each pinned to a line number with a patch.
The five out-of-the-box bugs, each reproduced on the unmodified checkpoint:
With those fixed, agentic tasks still fail, and the root is architectural: the LT re-feeds the hidden state through the same L layers, so prefill computes the O(len^2) attention over the same prompt twice, doubling peak activation memory. Harmless on an H200, catastrophic on Apple unified memory, where the model shares 32 GiB with the OS and other processes with no CUDA-style paging path for long reasoning traces.
The fix is chunked prefilling: process the prompt in 256-token chunks, growing a DynamicCache between chunks, then hand off to generate(). Peak per-step attention is bounded by chunksize times running total, decoupled from prompt length, with bit-identical outputs verified against naive prefill. The cost is throughput: 40.9% slower at prompt length 2048, in exchange for 4x batch parallelism.
Two more defects surfaced. System-prompt regression: if the caller supplies any system message, the chat template silently replaces the model's trained-in tool-use system prompt instead of merging; re-supplying the original verbatim fails too, because the auto-insert branch and the explicit branch differ by exactly two whitespace characters, and this checkpoint's tool-calling reliability is calibrated to the exact byte sequence. The fix removes the template's system branch and inserts caller content after the auto-inserted default. An independent llama.cpp report (PR #26324) found the same model emitting a trailing space instead of a newline on roughly 25% of tool-call tags, corroborating whitespace sensitivity.
MPS memory bug: one caught MPS out-of-memory error permanently degrades the long-lived harness process's usable memory budget; emptycache() and gc.collect() reclaim nothing, only a restart does. Uncorrected, a single task's overflow cascades into spurious OOMs on every later unrelated task.
LongBench-Pro, 50 long samples, M2 Max with 32 GiB, eight prompt lengths, three repeats:
| Prompt length | Naive max batch | Chunked max batch |
| 1024 | 16 | 32 |
| 2048 | 4 | 16 |
| 4096 | 2 | 4 |
| 8192 | cannot complete batch=1 | 1 |
| 11231 | cannot complete batch=1 | 1 |
| 12244 | cannot complete batch=1 | cannot complete batch=1 |
Usable context goes from 4096 to 11231 tokens, a 2.7x extension.
Post-fix evaluation: 3 of 10 MCPMark Filesystem easy tasks pass (30%), with 6 failures from tool responses exhausting memory and one timeout where the model called the right tool once then repeated the same absolute path 21 times. On 150 BFCL tasks: 100% on irrelevance (never calls when it should not), 63.3% on single calls, and near-total failure on parallel multi-tool tests (1 of 30), dominated by emitting one call where two were required, a format-level limitation no amount of memory fixes. The original checkpoint is unevaluable on any device due to bug two.
For anyone running local models on a Mac, this is copy-ready work: patched checkpoint, system prompt optimizer, and evaluation harnesses are on Hugging Face and GitHub. For model publishers, it reads as an open letter on reproducibility: a benchmark-winning model card next to a checkpoint that silently generates positionally incoherent text on every device because of a zeroed buffer. The LT parameter-efficiency tradeoff gets its first quantification on unified-memory hardware: halved context width at equal parameter count, 2.7x recovered by chunked prefilling at roughly 40% throughput. The system-prompt section warns anyone doing tool-calling adaptation: calibration tied to an exact byte sequence, and two whitespace characters break multi-tool calls.
The evaluation surface is narrow: 10 MCPMark Filesystem easy tasks and 30 BFCL tasks per category are not enough to re-adjudicate the model card's claims against Qwen3.5. Everything ran on one device (M2 Max, 32 GiB) and one transformers version (5.8.1), with no CUDA-side verification. The root cause of the MPS memory degradation is left undug, with process restart as the workaround. Single author; patches ship as sibling-file monkeypatching rather than upstream changes, and there is no timeline for merging into the official repository.