Local agent testing: 8B–32B models invert math and multi-agent setups collapse into sycophancy

OpenComplaint524 · reddit · 2026-10-24

A developer benchmarking Qwen 2.5 (7B/14B/32B) and Llama 3.1 (8B/70B quantized) in local agent loops on a single GPU (Ollama/vLLM) reports two consistent failure modes:

1. The "plausible math" illusion

Even with strict JSON schemas (Outlines/Pydantic), 8B/14B models output mathematically inverted but plausible numbers — e.g. a 1:3.0 reward-to-risk as 1:0.3, or a below-NAV stock with a hallucinated PBR of 4.2
Forcing CoT helps 32B slightly but triples latency, and they still drop negative signs or botch compound percentages
Conclusion: treat LLMs strictly as semantic processors and NL routers; do all arithmetic in native Python injected as read-only context

2. Multi-agent groupthink

In bull/bear/risk-manager debates, a flawed but confident premise from Agent 1 almost always triggers sycophancy from Agents 2 and 3
A "skeptical auditor" persona rarely works on 8B/14B — they concede within 1–2 turns
The only reliable fix: a deterministic non-LLM Python gate that kills the workflow when numeric thresholds are violated

The author asks the community about stripping arithmetic from prompts, preventing sycophancy without hard-coded rules, and at what parameter size reliable structured tool-calling emerges under latency constraints.

Original post →

Local agent testing: 8B–32B models invert math and multi-agent setups collapse into sycophancy

More from coding & agent