Four LLMs Play Doom: Jev Averages 5.63 Kills, 4.5x a Finetuned Qwen3.5-4B
shniydder · reddit · 2026-09-20
The author wired Jev, Laya, a finetuned ModernCE-base-nli, and a LoRA-tuned Qwen3.5-4B to play the same-seeded ViZDoom scenarios, comparing decision quality and latency.
- Input: a deterministic Python adapter converts ViZDoom's object labels, bounding boxes, and HUD values into short text; output is one button action at 5 decisions/second, with the game clock not pausing.
- Local models ran on a single DGX Spark (GB10, 128GB unified memory); Jev used TypeSafe's hosted API. No model had Doom-specific training.
- Averages over 8 seeds: Jev scored 5.63 mean kills in Defend the Center, well ahead of Qwen3.5-4B (3.63), while Laya and ModernCE each managed 1.25. Survival times in Health Gathering were close (13.0s vs 11.3s).
- Latency: Laya p50 16ms and ModernCE 8ms locally, versus 117ms for Jev (API round trip) and 147ms for Qwen.
Takeaway: Jev clearly makes better decisions but pays a latency cost; tiny local models are fast yet mediocre at gameplay. Full methodology and a longer write-up are linked in the post.
Related event: Four LLMs play Doom: Jev leads with 5.63 kills, 15x latency gap(2 posts)→
More from Models
- Blogger vouches for rumored Gemini 4 Carbon: good taste, cost-effective — intellectronica · 2026-11-12
- The Zvi opens reaction thread on Gemini 4 Carbon: is it worth extended coverage? — TheZvi · 2026-11-12
- Walden Robotics' Sage model quadruples robot uptime on Toyota factory lines — adnothing · 2026-11-12
- Gemini won't let you keep chat history while opting out of AI training — Google bundles both into one setting — arshdeep · 2026-11-12
- Gemini 4 Carbon Flex tier now costs the same per token as GPT 6.1 Terra — mariofilhoml · 2026-11-12
- Real-world test: Kimi K3.1 Pro costs 5.4x less than Opus 5.5 per task despite 2x list-price gap — Arindam_1729 · 2026-11-12