DFM Mimir v1: An Open HRM Delivering Frontier Performance at 1B Parameters Using Only Permissible Post-Training Data
Peter Schneider-Kamp, Jacob Nielsen, Gianluca Barmina, Kenneth Enevoldsen, Lukas Galke Poech
cs.CL, cs.AI
2026-08-14
Denmark's DFM trains Mimir, a 1B model on the non-Transformer HRM architecture, using 161 permissible datasets (70.5B tokens); it lands 0.3 points behind Qwen 3.5 4B on English and sets a new Danish state of the art at 56.8.
Open model development runs on a monolithic recipe: trillions of web-scraped tokens for pre-training, then staged post-training on top. Two things make this route unusable for some teams. The compute bill alone runs to millions of dollars. And the data rights are murky: projects committed to permissibly sourced material simply cannot assemble a pre-training corpus. The Danish Foundation Models project hit exactly this wall: there is not enough properly licensed Danish text to pre-train from scratch.
The HRM-Text framework (Wang et al. 2026) offers a detour: skip large-scale pre-training and train directly on post-training-style instruction data. Mimir v1 is a full instantiation of that idea at 1B parameters, bilingual Danish-English, and the first published third-party build of the HRM architecture.
The architecture first. HRM (Hierarchical Reasoning Model) is not a Transformer decoder. It couples recurrent networks at two timescales: a high-level module (H-cycles) does slow abstract planning while a low-level module (L-cycles) does fast concrete computation over each input. The design mimics cortical hierarchy and buys inference efficiency, doing multi-step computation with fewer parameters and fewer generated tokens, at the cost of training instability, managed by truncating backpropagation to 5 steps. Mimir's configuration: hidden size 1,536, 32 layers with half activated, 2 H-cycles plus 3 L-cycles, RoPE, and the Gemma-4 tokenizer.
The data work is the substance of the report. 161 datasets, 70.5B tokens per epoch, all permissible: openly licensed, supplied by agreement, or covered by the EU text-and-data-mining research exception. Nothing with personal information or copyright problems. English takes 68.6% of tokens, Danish 24.7%. By category, Danish instruction and knowledge leads at 22.1%, English instruction takes 19.3%, the Sapient mixture 17%, math and reasoning 14.8%.
Two choices stand out. The original HRM-Text training data was dominated by multiple-choice tasks; Mimir shifts the balance toward free-form generation, with 56.1% of tokens requiring the model to produce an answer scored by exact match. And where license problems blocked a dataset, the team generated synthetic replacements with Gemma4 31B and audited them: 70 Sapient-synth transplant datasets totaling 75M tokens, reported as performing on par with or better than the originals.
Training cost: 8 NVIDIA B200 GPUs, 1.65M steps, just under 3 weeks, about 1.1 seconds per step. A national project can afford this.
Twenty benchmarks in three suites:
| Model | English avg | Math & Code avg | Danish avg |
| Mimir 1B | 69.0 | 64.1 | 56.8 |
| HRM-Text 1B | 66.1 | 46.9 | 21.7 |
| Qwen 3.5 4B | 69.3 | 65.0 | 49.2 |
| Gemma 4 E2B (5B, effective 2.3B) | 56.6 | 75.4 | 44.6 |
| SmolLM3 3B | 63.1 | 67.9 | 31.7 |
Specifics: the English average of 69.0 sits 0.3 behind Qwen 3.5 4B, a 4x larger model, with Mimir first on Winogrande (73.5) and DROP (83.1). On math and code, GSM8K at 89.9 is second overall and HumanEval at 56.7 leads its weight class; the suite average improves 36.7% over the original HRM-Text 1B. Danish is a blowout: 56.8 average across 11 benchmarks, 7.6 points clear of the runner-up Qwen 3.5 4B, first on grammatical error correction (85.6) and WikiQA (66.8).
The weak spots are visible too: MATH at 45.8 trails SmolLM3 3B's 62.2, and the whole math-and-code suite sits eleven points behind Gemma 4 E2B — a model with 2.3 effective parameters and a different pre-training budget.
For low-resource language communities this is a copyable recipe: roughly 70B tokens and one 8-GPU node for three weeks produced an open model that sets a national-language state of the art while staying within 0.3 points of a 4B model on English, with all 161 data sources listed. The transplant-dataset trick, synthesizing and auditing a licensed replacement for data you cannot clear, is useful to any team blocked on rights, and the paper claims no performance loss.
For architecture research, this is the HRM route's first independent, published replication, and it demonstrates code and symbolic reasoning on a non-Transformer backbone; the original HRM-Text scored 0.0 on HumanEval. The caveat is scope: one data point at 1B, and the whole approach bets that instruction data alone can carry capability. This result supports the bet here; the authors themselves say scaling behavior is unverified.
The authors list: math and code lag Gemma 4; assistant capability remains limited; reinforcement learning is unexplored for this architecture; the dataset has not reached fully open licensing; scaling behavior is future work.
A few more concerns from reading closely. Several of the 11 Danish benchmarks are close in style or origin to DFM's own training data (DaLA, GEC), so part of the Danish lead may be data affinity rather than general Danish competence; lærebogen alone, repeated 4x, is 11.8% of the corpus. All evaluation used greedy decoding on full datasets with no variance reported. The claim that transplant datasets match or beat the originals comes as one sentence, without a dedicated ablation table. And the inference-cost advantage, HRM's headline selling point, gets architectural argument but no measured comparison against same-size Transformers.