Airbnb Cut Policy Classification Costs 99% With Gemma 4 E2B at 35ms Latency

petrusenko_max · x · 2026-11-20

Google's Gemma account highlights Airbnb Engineering's deployment: Gemma 4 E2B for policy classification, served on a single NVIDIA RTX Pro 6000 GPU. Max F1 jumped from 0.563 to 0.902, matching top frontier APIs, with 35ms median latency (85ms p99) vs 900ms for frontier models, at roughly 1% of frontier API spend at high volume.

The poster adds a key engineering caveat: 35ms median latency won't help if a policy change makes the classifier wrong while the old eval set still passes. Success came from thoughtful evals, domain data, and knowing exactly when a specialized small model beats a massive prompt.

Original post →

More from coding & agent

coding & agent channel →