Airbnb Cut Policy Classification Costs 99% With Gemma 4 E2B at 35ms Latency
petrusenko_max · x · 2026-11-20
Google's Gemma account highlights Airbnb Engineering's deployment: Gemma 4 E2B for policy classification, served on a single NVIDIA RTX Pro 6000 GPU. Max F1 jumped from 0.563 to 0.902, matching top frontier APIs, with 35ms median latency (85ms p99) vs 900ms for frontier models, at roughly 1% of frontier API spend at high volume.
The poster adds a key engineering caveat: 35ms median latency won't help if a policy change makes the classifier wrong while the old eval set still passes. Success came from thoughtful evals, domain data, and knowing exactly when a specialized small model beats a massive prompt.
More from coding & agent
- Joseph Suarez open-sources his 'skill issues.md' — called a $100M file — yacineMTB · 2026-11-20
- Google ships Genie 4 on Gemini API and AI Studio with live WASD steering — _philschmid · 2026-11-20
- Cognee builds company ontologies for agents with open-weight models and Postgres, no graph DB — philipvollet · 2026-11-20
- Developer uses Antigravity agent swarm to auto-build two full pinball games — AI_Andrew · 2026-11-20
- Ridges launches Ridgeline: label a GitHub issue, an agent fleet ships the PR — markjeffrey · 2026-11-20
- Claude drew an entire flyable game world with pure code — no images, no 3D models — prasenx · 2026-11-20