Testing Qwen Small Models for Web Auditing
Annual_Manner_5901 · reddit · 2026-07-14
The author tested four models on a real webpage for SEO auditing using a custom CPU-first inference server (REAME), focusing on hallucinations, issue detection accuracy, and local CPU performance.
Key findings:
- Qwen2.5-1.5B hallucinated issues, falsely claiming all images lacked alt text and even repeating existing ones.
- OLMoE 7B-A1B was inconsistent, contradicting itself within the same response.
- Qwen3-30B-A3B found real issues but estimated a 159-character meta description as 210 characters, showing it approximates rather than counts.
- Qwen3.5-9B was the most reliable: accurate, no hallucinations, and correct length counts.
The author also highlighted an engineering pitfall: Qwen3.5's mix of attention and recurrent state structures (like Gated DeltaNet) causes crashes in llama.cpp during speculative decoding or KV truncation. Maintainers of local inference stacks must check compatibility for such hybrid models when sequence state rollbacks are needed. The repo is open-source with reproducible benchmarks.
More from Infra
- NVIDIA brings its Cosmos 3 Edge world model to Jetson for on-device robot control — liu_mingyu · 2026-07-21
- A silicon photonic reservoir chip compensates fiber distortion in real time at 28 Gbps — bravo_abad · 2026-07-21
- Chamath says open-sourcing Grok would push AI margins from models to infra and apps — Dan_Jeffries1 · 2026-07-21
- EU AI competitiveness is under pressure as firms double down on chips, ethics, and talent — nordicinst · 2026-07-21
- AI bottlenecks are shifting to memory, optics, yield control and power — thedealdirector · 2026-07-21
- llama.garden is using torrents and web seeds to decentralize LLM distribution — de4dee · 2026-07-21