PACT benchmark launches for enterprise assistant compliance; open-weight models top the board
baseten · x · 2026-09-01
Baseten is sponsoring PACT (Pressure-Applied Compliance Testing), a new benchmark for enterprise assistant rule-following in regulated domains such as HIPAA, hiring law, and GDPR, in partnership with Decagon.\n\nThe two highest-scoring models are open-weight. GLM-5.3 and GLM-5.3-Flash are the best-performing GLM models, ranking #5 and #8 overall.
Related event: PACT Benchmark Tests Enterprise AI Assistants Under Pressure(2 posts)→
More from Research
- Training Trick: Fine-Tune Only the Last Layers to Avoid the Moving-Target Problem — Sauers_ · 2026-10-18
- Models Can Be Trained to Sense Activation Probes and Generalize Across GPUs Zero-Shot — Sauers_ · 2026-10-18
- Models Can Learn to Detect Training vs. Deployment and Generalize to New Runtimes — Sauers_ · 2026-10-18
- How One Attention Head Acts as a Switch to Amplify Tiny Probe Differences — Sauers_ · 2026-10-18
- Model trained to express different preferences only when its activations are probed — Sauers_ · 2026-10-18
- On Qwen3-8B, even a kernel-split without a probe is detectable to a trained model — Sauers_ · 2026-10-18