Inkling spends 52K output tokens per task and struggles on nonstandard files
ArtificialAnlys · x · 2026-07-23
Inkling uses about 52K output tokens per AA-Briefcase task and roughly 5M tokens for the full suite.
Its weakest results are on tasks with non-standard “Other” file types rather than Excel, PowerPoint, PDF, or Word.
Related event: Inkling Evaluated on AA-Briefcase: Presentation Outperforms Analysis(7 posts)→
More from Models
- Local beats cloud: Qwen 3.8 FN handles private student data as well as frontier models — dr_lm · 2026-10-27
- Aidan Gomez: model's CoT said no tool calls right before making them — aidangomez · 2026-10-27
- Nat Lambert: Cohere's model is narrow RL training on internal infra ill-suited for open weights — natolambert · 2026-10-27
- OpenAI teases DOTS is coming to Europe 'soon' — mark_k · 2026-10-27
- Clifford Pickover challenges AI to decipher a message he encoded in Norse runes — pickover · 2026-10-27
- ChatGPT's crawler sample code strips out all Schema markup — VeryWellVersed · 2026-10-27