Non-neural symbolic assistant CAEL scores 777/900, beating several small LLMs
Silver_Employ2617 · reddit · 2026-11-07
CAEL, a Python assistant built on symbolic rules, counts, and automata, released v0.3.9.1 with a maintainer-run 900-question fresh-transfer study.
Results:
- CAEL answered 777/900 correctly, far above its own 0.3.9 baseline (414) and neural peers like Falcon3-Mamba-7B (353), xLSTM-7B (273), and RWKV-7 (247) — though the neural baselines used fixed A/B/C/D next-token scoring, not their best chat performance.
- The more meaningful transfer result: on freshly worded questions CAEL scored 327/450 vs the baseline's 92/450, with all 123 abstentions counted wrong.
- The author discloses limitations: task families were known, development was adaptive, and isolation from peer models' pretraining data can't be certified.
- The project is source-available (not OSI open source); all questions, raw outputs, and verification scripts are public.
More from Research
- Near-identical neural net weights still diverge on ~3% of inputs after training — bravo_abad · 2026-11-08
- Removing negative-advantage samples drops robot zip-tie success from 86% to 60% — micoolcho · 2026-11-08
- DECIDE benchmark tests voice-agent decisions on 4,867 questions from 1,081 real calls, scoring 20 models — Aiden_Tech_Ai · 2026-11-08
- KV Cache Quantization Tested Across 35 Quants: 8-bit Is Essentially Free for Qwen3.8 27B — Hefty_Wolverine_553 · 2026-11-08
- YC amplifies demo of models handling semi-serious X-band phased-array design — ycombinator · 2026-11-08
- Analysis of 250k ICLR reviews finds 65% AI-written, scores collapsing to middle — TuhinChakr · 2026-11-08