Cisco ships Agentic FAITH: one workflow for cybersecurity agent evals
aminkarbasi · x · 2026-10-24
Cisco Foundation AI announced Agentic FAITH, an extension of its FAITH evaluation framework that brings multi-step, tool-using agentic tasks into the same workflow teams already use for QA-style cybersecurity benchmarks.
- Unified workflow: run benchmarks, grade outcomes, and analyze results across both traditional and agentic tasks (query → eval → analysis).
- Coverage: includes tasks like exploring a repository with terminal tools; existing FAITH benchmarks span threat intelligence, vulnerability classification, and security knowledge.
- Demo: the blog walks through VulnLoc as an example of evaluating both an agent's process and final finding.
A ready-to-use evaluation option for researchers and engineers building security-focused agents.
More from Safety
- 'Black October': 48 hours of record cyberattacks hit PHP, GitLab, LiteLLM and California schools — AryHHAry · 2026-10-24
- Depthfirst and NVIDIA showcase post-trained Nemotron 3.5 Lightning for defensive cybersecurity at GTC — andreamichi · 2026-10-24
- Kimi K3.1 Pro tops Artificial Analysis Cyber Index among open models at 64%, near GPT-6 Sol — ArtificialAnlys · 2026-10-24
- DISARM paper: disclosure cuts manipulative LLM's belief shift from 15.3 to 3.3 points — MaartenSap · 2026-10-24
- Cisco's Antares: a security analyst agent that never leaves your laptop — aminkarbasi · 2026-10-24
- Users urge turning off Tailscale SSH immediately as security chatter spreads on X — sloppenheimer · 2026-10-24