Quantized Ammu 1.0 guardrail: 17M-param model beats GPT-6-luna off-topic F1 by 10 points
kalyan_kpl · x · 2026-10-29
Ammu 1.0 quantized off-topic guardrail models (Base 150M, Tiny 32M, Nano 17M, ONNX INT8) classify whether a user query matches the system prompt before it hits the main LLM. INT8 keeps up to 99.6% of FP32 performance, shrinks models up to 55.2%, and speeds CPU P50 latency up to 1.67x. Macro-F1: 89.64%/87.03%/85.26% — claimed +7.69 to +10.52 points over DeepSeek V4 Flash 0731 and GPT-6-luna. A cheap pre-filter for production LLM pipelines.
More from Models
- GPT-6.1 computer use costs $0.16 per human-minute task, an effective $9.60/hr — bytebot · 2026-10-29
- Researcher: one steering vector can make an RLHF model shout slurs, no prompt can match it — voooooogel · 2026-10-29
- OpenAI's math argument still unverified: no expert has publicly assessed it — maier_ak · 2026-10-29
- OpenAI model claims 59-page Lean proof of local smoothing and Kakeya in every dimension — maier_ak · 2026-10-29
- Developer hails Google's rumored 'Carbon' model as first reliable vibe-coding model from Google — cgarciae88 · 2026-10-29
- Sardine's real-world fraud test: Jev hits 93% recall vs 62% for LLM classification — jamdac · 2026-10-29