Reddit test finds Nemotron 3.5 Super VL best at obscure knowledge, beating GPT-OSS and Qwen
Kahvana · reddit · 2026-10-19
A Reddit user benchmarks models on obscure knowledge — Anno 1602 (1998) game edition rules with web search disabled. Nemotron 3.5 Super VL came closest, with only minor errors, while GPT-OSS 120B and Qwen3.5-122B hallucinated game editions, DeepSeek v4 Flash got upgrade requirements wrong, and Gemma 4 31B QAT omitted too many details. Running unsloth's NVFP4 quant on dual RTX 5060 Ti 16GB with layer split and Vulkan, it hits 8-10 t/s generation and 150-170 t/s processing.
More from Models
- Together AI teases mysterious 'Space Bunny' launch — togethercompute · 2026-10-19
- Pangram AI detector flags The Atlantic's human-written cover story as 100% AI — nitarshan · 2026-10-19
- Can a world model run in real time on just 12GB of VRAM? — yehiaserag · 2026-10-19
- Memory-native MoE MemT 2.0 (293B) keeps memory inside the forward pass, no retraining needed — MaterialButterfly135 · 2026-10-19
- MIST Framework Separates Execution from Memory to Diagnose Video World Model Failures — hkuhk · 2026-10-19
- Thought experiment: if billion-dollar companies stopped open-weight releases for a year — ForsookComparison · 2026-10-19