5 'OpenAI-Compatible' API Lies We Hit Building an AI Gateway

respan-81 · reddit · 2026-10-24

A team building an AI gateway (unified chat/images/voice/video) shares five pitfalls found while testing 'OpenAI-compatible' endpoints:

OpenAI isn't compatible with itself: from gpt-5.4, Chat Completions rejects reasoning and function tools in one request; from 5.6 models reason by default, so sending tools errors out
Free money: OpenRouter uses price "-1" for unknown, which parsed as a number made the 'route to cheapest model' option always pick those models
"stop" means "I called a tool": with forced toolchoice, OpenAI (and Gemini) return finishreason "stop" instead of "toolcalls", so agent loops never run the tool
Streams that just leave: some servers never send [DONE]; Gemini sometimes stops mid-answer with no finishReason, and half-answers were treated as complete
The signature only the first call gets: Gemini 3's thought signatures are only attached to the first of parallel calls; dropping them on history rebuild causes a 400 (ADK #3705)

Gateway partially open-sourced: respanai/respan-gateway (single Rust binary, 174 stars).

More from coding & agent