Aleph Alpha open-sources Kolibri-1: 78B MoE with 1M context, llama.cpp support lands
jacek2023 · reddit · 2026-10-17
llama.cpp merged support (PR #30673) for Kolibri-1, Aleph Alpha's Apache 2.0 German-English MoE reasoning model: 78B total / 3.46B active parameters per token, 384 experts per layer (1 shared, 6 routed), 50 layers with 4:1 SWA:GQA attention trained via Muon. It supports explicit reasoning mode and tool calling, with 1,048,576-token context (≤262k recommended for serving). Pre-training ran 768 B200s for 21 days (392k GPU-hours, 6.4e23 FLOPS) on 20T tokens (62.5% EN, 23.9% DE, 13.6% code). FP8 weights need 78GB memory; minimum 2×A100 80GB or 1×H200/B200. Positioned as a sovereign open-weight model under the EU GPAI Code of Practice.
More from Models
- Hands-on: DeepSeek V4.1 Flash handles game VFX prompts but falls short of Opus — op7418 · 2026-10-17
- Microsoft completes pre-training of MAI-Base-2: 2.8T-param MoE, 4-12x more FLOPs-efficient than DeepSeek-V4-Pro-Base — NandoDF · 2026-10-17
- Screenshot of AI self-reflection goes viral: refusing to let judgment be 'for sale' — UltraRareAF · 2026-10-17
- Anthropic says Claude-designed quantum circuits cut cost of key operation ~17x — Dr_Singularity · 2026-10-17
- Training-free Confident Decoding skips final layers, cuts alignment tax, boosts reasoning — aiamblichus · 2026-10-17
- Gemini Pro subscriber details crash bugs and forced model downgrades in app — Kindly_Rough_4300 · 2026-10-17