Promptfoo: Run LLM Prompt Tests Like Unit Tests in YAML with CI/CD
KhuyenTran16 · x · 2026-10-22
Open-source tool promptfoo (26.3k stars) turns evaluation of prompts, agents, and RAG into unit tests committed to your repo: each case declares input, provider, and expected behavior in YAML, so prompt changes can be rerun consistently across old cases, new cases, and different model providers. It supports comparing GPT, Claude, Gemini, DeepSeek and more, adds red-teaming/pentesting and vulnerability scanning, integrates with CLI and CI/CD, and is used by OpenAI and Anthropic.
Related event: Promptfoo Turns LLM Prompt Testing into YAML Unit Tests(2 posts)→
More from coding & agent
- Drop a PowerShell Script in a Folder, Get a Live MCP Tool — dfinke · 2026-10-23
- Auth'd MCP servers can deliver user-personalized skills, an overlooked design win — dsp_ · 2026-10-23
- Universe badge pairs RP2350 with an AGENTS.md so Copilot can code and deploy to it — mariorod1 · 2026-10-23
- Claude Code banked reset vanishes hours early, user demands Anthropic restore it — burny_tech · 2026-10-23
- Sudo L7 benchmark uses 76-word natural prompts vs 291-427 words for rivals — echen · 2026-10-23
- Claude recolors an AI-generated SVG into 4 sneaker colorways without touching stitching or shading — socialwithaayan · 2026-10-23