Change the Product, Keep the Parameters: Associative Algebra Layers for Transformers
Ilya Koziev, Ivan Oseledets
cs.LG, cs.PF
2026-09-27
Replace FFN GEMMs with an associative-algebra product at the same parameter count. At 110M and 12.3B tokens, generation is 6.2–7.8% faster; GSM8K, IFEval, and MBPP all fall.
A Transformer spends its linear maps on Y = XW. Split the feature axis into groups and that map becomes a pile of rectangular GEMMs (general matrix multiplies). The row-column rule decides which activation block meets which weight block; training only fills in the numbers. Fast matrix multiplication, from Strassen onward, treats that rule as fixed and hunts for a cheaper way to evaluate the same product.
Hidden states and neighboring layers are learned together, so the intermediate product does not have to be ordinary matrix multiplication. This construction keeps every learned weight block and installs a cheaper, still-composable sparse table.
Tensor Train and Monarch shrink storage. MatMul-free models drop dense multiplies. Here the parameter count stays din × dout; only the allowed block pairings change.
The table comes from a directed graph. Vertices are diagonal feature slots, edges are kept off-diagonal slots, and edge-edge products are zero. On the complete graph with q feature groups the product Qq is: a diagonal output multiplies matching diagonal blocks; an off-diagonal output is source-diagonal times edge plus edge times target-diagonal.
For q = 2, ordinary 2×2 block multiplication uses eight GEMMs; this table uses six. The Alder-Strassen theorem lower-bounds the bilinear rank of a finite-dimensional associative unital algebra (a product with a two-sided unit and (ab)c = a(bc)): the least number of scalar multiplies in an exact bilinear algorithm. Six meets the bound, so this law has no cheaper exact bilinear algorithm. In general R(Qq) = 2q² − q and the dense-relative arithmetic is ρq = (2q − 1)/q². Larger q means fewer live block products and narrower cross-group maps at each position.
With weights held fixed, ΦW(X) = X ⋆ W satisfies ΦW2 ∘ ΦW1 = Φ(W1 ⋆ W2), so stacked maps of this form stay inside the same family. A two-sided unit means every weight block can affect some output.
In the Transformer, each token is assigned a row type from its absolute position and evaluates only that row of the table. Training runs all types in parallel. Autoregressive decode evaluates the new position's row and keeps a standard KV cache. Causality follows from row-local projections plus the usual triangular mask.
The pretraining pilot uses the tensor square Q⊗22 of the two-group law: sixteen coordinates, 36 explicit block products, FFN projection MAC fraction 9/16 at equal group widths. It is not isomorphic to the direct four-group algebra Q4. Attention, embeddings, and the LM head stay dense on both sides. Hold the physical GEMM block size b fixed and let q grow with matrix order, and square-product arithmetic falls to Θ(n²). That is a complexity claim. Whether quality tracks it is unmeasured.
Kernel timings on an H200 NVL use synthetic BF16 tensors, no activation fusion, against the faster calibrated cuBLAS/cuBLASLt path. q is the best warm-median in the tested grid:
| Shape | M | q | Warm / flushed |
| Qwen3-1.7B | 2048 | 32 | 3.04× / 2.84× |
| Qwen3-8B | 2048 | 32 | 5.04× / 4.52× |
| Qwen3-32B | 32768 | 256 | 11.51× / 11.70× |
| Qwen3-30B-A3B expert | 2048 | 16 | 1.57× / 1.30× |
Square products at fixed block size go further: n = 32768, b = 128 hits 39.85× warm. These are isolated projection kernels, not full Transformers. A fused SwiGLU forward on Qwen3-8B at M = 2048, q = 32 still shows 4.17×. Forward-plus-gradient on the Qwen3-30B-A3B expert drops to 0.65× and 0.57×, slower than dense.
The language-model test is a matched pair of decoder-only models, 110,631,168 parameters each: 12 layers, width 768, 32 heads, context 1536, GPT-2 tokenizer, SwiGLU inner width 1536. Same recipe, same public instruction mix, 12.3B tokens each, two H100s. The only difference is the FFN product, dense versus Q⊗22. Attention stays dense.
End-to-end generation throughput includes tokenization and sampling, 100 prompts per domain:
| Domain | Dense tok/s | Algebraic tok/s | Gain |
| Code | 987.8 | 1054.5 | 6.8% |
| Math | 991.4 | 1060.6 | 7.0% |
| QA | 958.6 | 1033.1 | 7.8% |
| WMT ru–en | 985.9 | 1047.1 | 6.2% |
Unweighted mean of the four ratios is 1.070. Downstream scores all fall:
| Task | Metric | Dense | Algebraic |
| GSM8K (5-shot) | EM, flexible | 3.18 | 2.20 |
| IFEval (0-shot) | Instance, loose | 34.5 | 31.1 |
| MBPP (0-shot) | pass@1 | 6.60 | 3.80 |
Gaps are 0.98, 3.4, and 2.8 points. Absolute scores at 110M and 12.3B tokens are low on both sides. The direction is consistent: faster, and weaker. The paper frames this as a small-scale trainability check.
The multiplication table is now an architectural knob. Strassen-style work squeezes constants out of ordinary matmul; this line is allowed to change the product, and Alder-Strassen certifies that the graph law is rank-optimal. Parameter count does not drop. Sparsity is type-dependent and fixed, not a content-routed MoE.
Kernel speedups on large shapes hold up: about 5× on Qwen3-8B-sized expansions, up to 11× on Qwen3-32B at large M. End-to-end generation keeps only 6–8%, because attention, the LM head, sampling, tokenization, and memory traffic dilute the FFN's 9/16 arithmetic cut. In the tested config the decoder-block projection share is 59/80. Decode is matrix-vector and needs its own kernel.
Same parameters, same data, same step count: extra speed, lower scores on every reported metric. 110M is a weak model, and the gap has not been rechecked at larger scale or with multiple seeds. Algebraic attention scores and cache rules live in the appendix and were not used. Against TT and Monarch, this family keeps the dense parameter count and cuts the block multiplies that actually run.
The authors are explicit: one recursive FFN law, one scale, one training run per model. Larger algebras, algebraic attention, larger models, and repeated seeds are untested. Projection benchmarks use synthetic tensors. Evaluation standard errors capture item sampling, not run-to-run training noise. Generated lengths differ (mean code replies 195.5 vs 182.4 tokens), so the throughput gap mixes a layer effect with a workload effect; isolating the layer needs fixed-length full-model timings.
A few numbers sit poorly. Forward-plus-gradient on the narrow Qwen3-30B-A3B expert FFN (D = 2048, F = 768) is slower than dense, so arithmetic savings can lose to small leaf GEMMs. GSM8K standard errors are 0.48 and 0.40 points; a 0.98-point gap is about one and a half standard errors, thin support for a stable quality cost. Associativity guarantees compositional closure, not expressivity or sample efficiency. Equal storage still names different function classes. The paper states the results do not show an advantage at matched quality or matched training time.