Four LLMs play Doom: Jev leads with 5.63 mean kills but 15x higher latency
shniydder · reddit · 2026-09-20
A developer let Jev, Laya, a finetuned ModernCE-base-nli, and a finetuned Qwen3.5-4B (LoRA) play ViZDoom from identical seeds, inspired by TypeSafe's Jev Doom demo.
Setup: A deterministic Python adapter converts ViZDoom's object labels, bounding boxes, and HUD values into short text descriptions; each model outputs one action (turn/fire or move toward medkits). Game runs at 35 Hz targeting 5 decisions/sec, never pausing for model replies. Local models ran on a DGX Spark (GB10, 128GB unified memory); Jev used TypeSafe's hosted API. 8 seeds per controller, 30s episode cap, no Doom-specific training.
Results (means):
- Jev 1.13: 5.63 mean kills (Defend the Center), 13.03s survival (Health Gathering), but p50 latency 117.3ms including API round trip.
- Qwen3.5-4B LoRA: 3.63 kills, 11.31s survival, 146.8ms p50.
- Laya: 1.25 kills, 11.89s survival, 16.2ms p50.
- Finetuned ModernCE: 1.25 kills, 11.66s survival, best latency at 7.6ms p50.
Takeaway: Jev makes clearly better decisions but is an order of magnitude slower; local small models are near-instant yet kill 4x less. Longer write-up at morethanamachine.com.
Related event: Four LLMs play Doom: Jev leads with 5.63 kills, 15x latency gap(2 posts)→
More from Models
- Gemini 4 Carbon opens to all at $2/$10, price doubling to $4/$20 on Dec 1 — socialwithaayan · 2026-11-12
- Ten wild Gemini 4 Carbon demos in 24 hours: CUDA meadows, V8 engines, mech stomps — socialwithaayan · 2026-11-12
- How to cut Fable 5.2 token costs by 90%: Arc routing plus a lean CLAUDE.md — EXM7777 · 2026-11-12
- Grok Bot 1.0 launches with support for joining Meet, Zoom and Teams calls — mark_k · 2026-11-12
- Partial RuneBench results: Inkling 2 Small scores 5.314, far above original — mariofilhoml · 2026-11-12
- Grok Bot 1.0 Announced: Shareable Bots With Upgraded Collaboration — SpaceXSI · 2026-11-12