Open leaderboard for non-neural AI launches with NNAI-Bench v0.1, 1,800 synthetic questions
Silver_Employ2617 · reddit · 2026-10-19
A developer launched an open Hugging Face leaderboard for AI systems that use no neural networks at inference — symbolic/rule systems, cellular automata, trees, kernel methods, count models, search, and classical retrieval — alongside CAEL, their non-neural assistant.
NNAI-Bench v0.1:
- 1,800 original synthetic questions: 600 train / 300 validation / 900 public test.
- Six equal-weight task families: arithmetic, deductive logic, ordering, state tracking, string operations, table reasoning.
- The Python evaluator needs no ML libraries; code and data are MIT licensed.
Submission integrity: entries require predictions, a pinned source revision, reproduction steps, resource usage, test-set exposure, and disclosure of neural-derived training data; offline retrieval gets its own resource track; submissions are reviewed, and self-reported vs independently reproduced results are labeled separately.
The author openly notes the limitation — templates and labels are public, so a hand-written specialized solver could score highly — and asks for feedback on harder task families, leakage checks, and fair resource comparisons.
More from Research
- Agent reproduces black hole images from EHT raw data in 2 hours for ~$7 — fofrAI · 2026-10-19
- Researchers crack Google's 64-bit docid: it's a Jenkins hash fingerprint of the URL — gaganghotra_ · 2026-10-19
- The mysterious 'Chinese spirit neuron' may trace back to a Dream of the Red Chamber translation — Sauers_ · 2026-10-19
- Why native multimodal models won't win it all: they can't compress visual context — gabriberton · 2026-10-19
- 4-year bitter-lesson evolution of MLLM vision: from LLaVA crops to native-resolution images — gabriberton · 2026-10-19
- Data World paper: RL on synthetic data environments adds 20.9 points on DABstep-Hard — omarsar0 · 2026-10-19