Debate Over Per-Token Noise Training to Bypass Forward-Pass Costs

willcb and @9thdev debated a per-token noise perturbation training method claimed to trade layer width for parameter count and run faster than weight-space evolution strategies. willcb clarified it does not work for FFNs, only hiding SNR terms in the context via aggregation dimensions in sequence models.

2026-10-20 ~ 2026-10-20 · 2 related posts

• Twitter debate: per-token noise training exploits transformer parallel passes, still ~10^4 per update — willcb · 2026-10-20
• willcb: per-token noise training won't work for FFNs, relies on sequence aggregation axis — willcb · 2026-10-20