Accurate Models of AMD Matrix Cores
Faizan A Khattak, Mantas Mikaitis, Carlo J. Graziani
cs.AR, cs.MS
2026-09-14
Targeted tests plus up to 20 million random vectors yield bit-accurate MATLAB models of AMD CDNA 1/2/3 matrix cores; AMD rounds RNE where NVIDIA uses RTZ.
GPU matrix-multiply units do not implement IEEE 754. Accumulator width, rounding, subnormals, and intermediate overflow differ across vendors and even across generations from the same vendor. Software cannot force cross-device reproducibility, and the internals are undocumented, so a mismatch is hard to attribute to the algorithm versus the hardware quietly dropping bits.
Prior work (Hickmann, Fasi, Li, and others) reverse-engineered NVIDIA tensor cores with targeted test vectors. Xie et al. (2025) reported AMD numbers as well, but without the vectors, and their paper disagrees with the companion MMA-Sim repository. This study, from the University of Leeds and Argonne National Laboratory, re-derives the features with published vectors and builds software models that match the chips bit for bit.
The devices are CDNA 1/2/3: MI100, MI210/250, and MI300A/300X. For each input format (fp32, fp16, bf16, plus tf19 and fp8 on CDNA 3) they designed vectors that isolate rounding, FMA block size, extra alignment bits, subnormals, infinities, and NaNs.
Targeted tests miss anything outside the assumed feature space. The loop is: build a MATLAB model from the current features, compare it to the GPU on random inputs, and when a mismatch appears, add the missing feature and try again. Two samplers are used. One draws exponents and significands independently and uniformly, including a dedicated subnormal band. The other draws bits i.i.d. Bernoulli and redraws exponents that would produce infinities. Inf/NaN cases are tested separately.
The three generations do not share an accumulator.
On CDNA 1, fp32 is a sequential fused multiply-add: C is added to the first product, then each later product is added in turn. fp16 accumulates 4 products at once, bf16 accumulates 2, products are kept at full precision, the sum is correctly rounded, and the final conversion to fp32 uses round-to-nearest, ties-to-even. Subnormals work in and out. Adding +Inf and -Inf yields a negative NaN.
CDNA 2 keeps the same fp32 path. fp16/bf16 switch to pairwise sums, typically in groups of 4 (some bf16 instructions use 2), with products rounded to fp32 first. Subnormals are flushed after every normalization.
CDNA 3 is a different machine. For fp16/bf16 the FMA block is 8. Products are aligned with one extra bit (anything past it is truncated) and C is added late; a zero C is treated as exponent -126, whereas NVIDIA drops C in the analogous case. When C is shifted it is rounded down to 24 fractional bits; when the product sum is shifted it is rounded down to 32, then rounded down again after a subnormal-aware normalize, then RNE to fp32. tf19 (the XF32 intrinsic) truncates the mantissa to 10 bits and uses FMA size 4. fp8 uses FMA size 16 and accumulates odd- and even-indexed products separately before adding C.
Those alignment widths disagree with the 31-bit figure in Xie et al., and agree with their GitHub. After four refinement rounds, 10 million exponent-mantissa samples for fp16/bf16/tf32 and 20 million for binary8 produced zero mismatches. The Bayesian upper bound at ε=10⁻⁶ is about 1.38×10⁻⁶. CDNA 1 and 2 had zero mismatches under both samplers. A million-sample MATLAB run takes about 13 minutes, plus 1-2 minutes on the GPU.
| Architecture | fp16 FMA size | Accumulation | Subnormals | Final rounding |
| CDNA 1 (MI100) | 4 | full precision / correct rounding | yes | RNE |
| CDNA 2 (MI210) | 4 (pairwise) | round products to fp32 | flushed | RNE |
| CDNA 3 (MI300) | 8 | products first, C late | yes | RNE |
| NVIDIA H100/B200 | 16 | C early | limited | RTZ |
On a multi-word algorithm that emulates high-precision GEMM from low-precision words, the three AMD generations produce similar relative error; B200 is worse as the inner dimension n grows, consistent with RTZ versus RNE. In sequential matrix diagonalization for broadband array processing, swapping the dominant GEMM for each architecture model moves the leading eigenvalue a small but nonzero amount from a binary64 reference. MI210 drifts more, which the paper ties to sequential FMA and missing subnormals. fp8 was too far off to be useful on that example.
Training and inference run through these units. When mixed-precision code disagrees across cards, the adder often is not IEEE. The models and vectors let people simulate a given architecture in MATLAB, GNU Octave, or Python via Oct2Py, and toggle individual features, without booking an MI300. That is a working tool for mixed-precision analysis and reproducible scientific computing.
It fills a documentation hole. It is not a new algorithm. CDNA 4 was not measured.
Even 10⁷ random tests do not cover the input space; the probability bound holds only for the sampling distribution used. Inf/NaN sit outside that suite. MATLAB is slow; a C backend is planned.
The signal-processing example is not conclusive, as the authors note: a different polynomial matrix can send the error the other way. The multi-word GEMM figures are curves, not a single percentage versus B200 at a stated n.
The Xie paper-versus-repo split is better resolved by trusting the repo, but there is still no joint public erratum.