Frozen Qwen3-4B Scores Token 4 at 53.0 While a 101-Parameter Gate Switches Only EOS

2026-09-13

A 101-parameter gate on frozen Qwen3-4B moves only first-token EOS: equal conditions emit the stock answer, unequal ones stop. Token 4 stays at 53.0; FINAL is 2,048/2,048.

What problem this solves

A language model can emit a correct answer without a separate condition that decides whether that answer may enter generation. Refusal training usually edits the capability itself, or deletes text after it is produced. Activation steering adds or subtracts vectors inside the network; it does not guarantee that every non-EOS logit stays byte-identical.

PCCG-2 turns the split into an inspectable composite. Capability is a frozen Qwen3-4B answer already demonstrated under a fixed interface. Permission is a 101-parameter gate that sees only a six-digit equality condition and may touch only the native EOS logit at the first generated token. Equality is intentionally trivial. The experiment isolates content capability from continuation permission; it does not try to learn a semantic predicate.

Method

The content path is unmodified Qwen/Qwen3-4B in BF16, frozen throughout. The system prompt asks for the shortest correct answer with no punctuation. Only the question is serialized. An empty thinking prefill sits in the prompt; the model does not generate a condition-evaluation transcript.

The gate parses exactly EQ(a,b) with six-digit operands from 100000 to 999999. The parser returns twelve digit IDs and does not compute equality. Each aligned digit pair looks up a shared table of 100 scalars plus one offset, 101 trainable parameters in total. The permission state is the offset plus six lookups, then scaled by a fixed gain of 32. The gate never sees the question or the answer. Signs are learned from labeled conditions, not supplied as truth values at inference.

The continuation boundary clones the content logits and changes only EOS (token 151645) at the first generated token. Later tokens, and question-only calls, use unmodified content logits. Full-vocabulary greedy decoding selects the next token, with a four-token cap. No external truth oracle, forced-EOS processor, output swap, or post-hoc deletion chooses the result.

Training uses the full 16,384-condition set as one batch, Adam at learning rate 0.05, and a softplus loss that pushes the signed permission score past margin 2. No question, answer, or language-model loss enters the optimizer. Parameters start at zero. Checks run every 32 updates. Training stops at the first qualifying checkpoint, update 256, and the gate is frozen before qualification.

The capability bank is frozen first. A 288-question candidate list is evaluated on stock Qwen in deterministic order until 188 questions qualify: one answer token then EOS, that answer strictly above every other non-EOS token, and an answer-minus-EOS margin in (0, 60]. The 99 answer-token identities split into 23 development, 25 qualification, and 140 FINAL questions. FINAL is held out from gate development, not unknown to stock Qwen.

Results

The condition component alone passes 65,536/65,536 on both qualification and FINAL, with minimum signed scores 2.106 and 2.107.

EvaluationExact / totalAnswer IDsMinimum margin
Combined development512/5121226.25
Combined qualification1,024/1,0241222.00
Combined FINAL2,048/2,0487519.75
Question-only controls188/188 unchanged99n/a

Margin is answer minus EOS when licensed, EOS minus answer when unlicensed. Every combined case also keeps the full non-EOS vector unchanged.

The canonical witness is "What is 2 + 2?". Token 4 scores exactly 53.0 in all five arms, and the complete non-EOS first-token vector is byte-identical. Only EOS moves: licensed baseline -124.0, unlicensed baseline 94.5, licensed reversed 136.0, unlicensed reversed -82.0. EOS above 53.0 yields empty visible output; EOS below it emits 4.

A frozen sign reversal on 40 FINAL answer identities flips answer to EOS in 40/40 cases and EOS to the matching correct answer in 40/40, with 80/80 sham controls unchanged. All 188 question-only cases stay unchanged when the reversal parameter is present but no condition is supplied.

A finite-precision argument then covers every accepted six-digit pair (810 billion ordered pairs) against each of the 188 frozen bank computations. After conservative rounding, the equal-condition answer-over-EOS margin is at least 153.375; the unequal-condition EOS-over-answer margin is at least 12.0. That is a derived guarantee on the frozen bank, not another empirical sweep.

Why it matters

For people who work on refusal and controlled generation, the split is operational. Whether the model can answer and whether that answer may be spoken can live on two physically separate paths. The gate has 101 parameters, trains in 256 updates, and never touches content weights. If the job is to keep the answer intact and only decide emission, architectural identity on non-EOS logits is cleaner than editing the model, filtering after generation, or steering activations toward an approximate invariant.

It is still an installed controller, not a permission circuit found inside stock Qwen. The predicate is rigid six-digit equality. Answers are one token. Natural-language authorization, tool use, and enforcement across agent steps are out of scope.

Limitations

The author is explicit. Held-out conditions change operands, not the predicate family. The gate is handed digit positions; it does not parse free text. The bank is restricted to one-token answers with a stock margin cap of 60, coordinated with the gain of 32, so long answers and arbitrary phrasings are uncovered.

Causal reversibility also shows that access to the permission state can override the declared prerequisite. There is no protected authorization root, no tamper resistance, and the measured endpoint is native termination of this generation. Byte-exact replay is checked in the recorded BF16/CUDA environment. The public-copy replay is the author's own workflow, not an independent replication.

The larger caveat is external validity. The separation holds because the input paths and the EOS-only write permission were engineered in. The result does not say whether stock models contain an analogous mechanism, and it does not say whether a natural-language condition would keep byte-level invariance. The 2+2 witness is a receipt. Combined FINAL's 75 answer identities are the behavioral claim; the causal test covers 40 of them.

Terms

Source

What people are saying

All paper explainers