3,360 experiments show LLMs steer 79% of API picks to big clouds, 70% when coding

santiviquez · x · 2026-11-17

@santiviquez ran 3,360 experiments testing which document processing APIs models recommend: when asked directly, 79% of recommendations went to big-cloud APIs (Azure Document Intelligence, Google Document AI, Gemini API, AWS Textract). When asked to write implementation code, models chose those APIs as the best option 70% of the time.

The data reveals systematic bias in agent-driven tool selection: verbal recommendations and actual coding behavior diverge, with self-preference amplifying once models write code.

Original post →

More from Models

Models channel →