K-EXAONE 2.0 Technical Report
Eunbi Choi, Kibong Choi, Sehyun Chun, Seokhee Hong, Junwon Hwang, Hyojin Jeon, Ahra Jo, Hyunjik Jo, Yeonsik Jo, Minhyeok Jung, Doyoung Kim, Heegyu Kim, Joonkee Kim, Seonghwan Kim, Soyeon Kim, Sunkyoung Kim, Yireun Kim, Yongil Kim, Byungoh Ko, Changhun Lee, Dohaeng Lee, Haeju Lee, Jinsik Lee, Kyungmin Lee, Minwoo Lee, Wonkee Lee, Sangha Park, Sungjune Park, Kwangrok Ryoo, Kijung Seo, Minju Seo, Yongwoo Song, Sejong Yang, Heuiyeen Yeen, Stanley Jungkyu Choi, Yemuk Choi, Yongchan Chun, Jiwon Ham, Dasol Hong, Sujeong Im, Kijeong Jeon, Gerrard Jeongwon Jo, Hyeongjun Jo, Yujin Jo, Jiyeon Jung, Naeun Kang, Daeseong Kim, Euisoon Kim, Hayeon Kim, Hyosang Kim, Myoungshin Kim, Unsol Kim, Youchul Kim, Chaeeun Lee, ChaeYoon Lee, Edward Hwayoung Lee, Honglak Lee, Hwansoo Lee, Minkyung Lee, Sangeun Lee, Solji Lim, Woohyung Lim, Chanwoo Moon, Jueun Mun, Jimin Park, Seojeong Park, Yongmin Park, Hyerin Seo, Donghyeon Shin, Donghyun Son, Eunyong Son, Kaehyun Um, Sihoon Yang, Chang En Yea, Sihyuk Yi, Kyungjae Yoo, Chansik Yoon
cs.CL
2026-08-05
LG's open-weight 750B/37B MoE (upcycled from K-EXAONE, 256K context): about 10% over its predecessor across 24 benchmarks and about 30% on three coding ones; leads open peers on long-context retrieval and safety, trails on math and world knowledge.
This is a technical report recording how LG AI Research scaled its K-EXAONE from a 236B/23B MoE to 750B/37B, framed as "a step toward global frontier-scale foundation models". K-EXAONE is a "sovereign" foundation model built under a Korean compute program, positioned to keep capabilities such as training data and safety standards under domestic control.
The report's spine is not "we are first". The authors say flatly in the conclusion that "K-EXAONE 2.0 does not lead on every benchmark". The question it answers is where, after a more than threefold capacity increase, the capability lands and where it still falls short.
The model was not trained from scratch but upcycled from K-EXAONE: depth went from 48 to 78 layers (2 dense plus 76 MoE), experts per layer from 128 to 256, with top-8 routed plus one shared expert active per token. Attention retains a hybrid scheme (sliding window of 128 plus periodic global layers) to handle long context. A notable stability detail: the last 16 layers use "Clamped SwiGLU" to clamp activations at a threshold of 7.0; without it the last-layer activation spikes to 6862 and breaks FP8 training and serving.
Context was extended to 256K via two mid-training stages (400B tokens each, first to 64K then to 256K). On top of that, 8T tokens of continual pre-training added two synthetic data methods: Active Reading (for parametric knowledge) and thinking-augmented pre-training with latent intermediate reasoning.
Post-training has several stages: 350B tokens of SFT (with the router deliberately frozen to preserve expert specialization), then two-stage preference learning (first multi-task preference with an in-house group-wise method called GROU PER, then safety-aware preference handling refusals, jailbreak robustness, and abstention on unanswerable queries), and reasoning chains are carried across turns in agentic settings. For inference, two speculative-decoding draft heads are provided; the DSpark drafter (semi-autoregressive block diffusion) gives a 1.81 to 2.57x end-to-end speedup, beating the jointly trained MTP head (1.27 to 1.77x).
Twenty-four benchmarks across nine categories, all reported in reasoning mode. Versus the predecessor K-EXAONE, the average gain is just over 10%, with the standout items in coding and long context.
| Benchmark | K-EXAONE | K-EXAONE 2.0 | Gain |
| SWE-BENCH VERIFIED (coding) | 49.4 | 68.2 | +18.8 |
| TERMINAL-BENCH 2.1 (agentic) | 30.3 | 43.8 | +13.5 |
| OPENAI-MRCR (long-context retrieval) | 52.3 | 94.4 | +42.1 |
| POLYMATH (multilingual reasoning) | 57.4 | 71.3 | +13.9 |
| GPQA-DIAMOND (knowledge) | 79.1 | 82.2 | +3.1 |
Three coding benchmarks improve by roughly 30%. Long-context retrieval is the largest single jump (OPENAI-MRCR 52.3 to 94.4); the authors report perfect needle-in-a-haystack retrieval across all positions up to 256K.
Against comparable open-weight MoEs (Qwen3.5 397B/17B, GLM-5.1 754B/40B, DeepSeek V4 Pro 1.6T/49B), K-EXAONE 2.0 leads on two fronts: long-context retrieval (OPENAI-MRCR 94.4, above Qwen3.5's 93.0 and GLM-5.1's 71.5) and safety. Safety is the differentiation the report returns to: KGC-SAFETY 99.8 and ROK-FORTRESS 89.5 both top every listed competitor, and the in-house K-AUT-V2 safety taxonomy was expanded from 226 to 296 risk categories.
It also trails clearly. On world knowledge and math, GLM-5.1 and DeepSeek V4 Pro are generally higher: GPQA-DIAMOND 82.2 versus GLM's 90.1, AIME 2026 is 92.3 versus 95.3, and HLE (Humanity's Last Exam) sits at only 18.3, well behind DeepSeek V4 Pro's 28.7 and GLM-5.1's 37.7. Throughout, the report never directly benchmarks the closed frontier (GPT, Claude, Gemini); "global frontier" is an aspiration, not a measured result.
For anyone who wants a controllable, locally deployable, very large open model, K-EXAONE 2.0 is one of the few 750B-class Apache-2.0 options with 256K context, and it is stronger than same-class open peers on long-document retrieval and safety alignment. Its value is not the top score but that it walks a "sovereign" path to completion: scale, long context, and safety, all self-reported and all open-weight.
The engineering details are useful for people training large models: the Clamped SwiGLU fix for deep-layer activation blow-ups, upcycling rather than training from scratch, and SFT with a frozen router are the kinds of choices written only after real pain.
The foremost caveat: every K-EXAONE 2.0 score is self-reported by LG, with no third-party evaluation. KGC-SAFETY and KO-LONGBENCH are in-house benchmarks, so the model is graded on its authors' own exams and the 99.8 safety score should be discounted accordingly; ROK-FORTRESS is externally published but is a Korean-context safety benchmark where a Korean-trained model has a natural edge. Competitor scores marked with an asterisk are taken straight from each vendor's own report or leaderboard, with inconsistent methodology, so they are not strictly comparable.
The authors' own listed limitations are the standard large-model ones: possibly inappropriate or biased outputs, dependence on training-data statistics, and no guarantee of current information. The conclusion's "does not lead on every benchmark" is honest: world knowledge and math still trail the largest open-weight peers, and the closed frontier was never directly compared. "Global frontier-scale" is, for now, closer to a destination than an arrival.