LLM redaction test at temp 0: some models leak hospital name every run, 24 runs cost $0.21
MaziyarPanahi · x · 2026-10-30
The author ran the same prompt at temperature 0, hosted, across multiple LLMs to black out the hospital name in medical text — 24 runs for $0.21 total. Kimi K3, Qwen3.8-27B and GLM-5.3-Flash succeeded every run; Mistral Large 4, MiMo-V2.6-Pro and DeepSeek-V4-Flash-Vision only on some runs; MiniMax-M3.1-Flash and Muse Spark never did. A neat hands-on look at real-world instruction-following variance in privacy redaction.
More from Models
- Liquid AI doubles usage limits for all Pro tiers as CEO taunts rival's rate-limit reset — janusch_patas · 2026-10-30
- Distillation debate: is Chinese AI pricing 'involution' or rational catch-up? — teortaxesTex · 2026-10-30
- JevBench v1.8 crowns mammoth-1 atop open-weights board, now prices models by GPU time — airesearch12 · 2026-10-30
- Mistral Large 4: 1T-parameter 'Le Chonk' now trainable in TRL with open SFT example — QGallouedec · 2026-10-30
- X account caught using outdated model to smear DeepSeek, blocks critics with receipts — teortaxesTex · 2026-10-30
- Opus 5.5 Explaining Interview Algorithms Rakes Up 50K Likes in Two Days — huangyun_122 · 2026-10-30