Synap claims #1 on LongMemEval with 95.6% answer accuracy, optimizing a different metric
arpit_bhayani · x · 2026-10-29
Synap announces a #1 score of 95.6% on LongMemEval — optimizing final answer accuracy rather than Recall@20/200, with a Flash-class model (Gemini 3.8 Flash) generating answers and gpt-4o judging with the benchmark's own prompts. Arpit Bhayani highlights the interesting choice of focusing on accuracy and how much context must be retrieved to reach it.
More from coding & agent
- Running a $25k/month creator brand-deal agency solo with Grok Bot agents — VibeMarketer_ · 2026-10-29
- Dev says agents wrote 90% of his 5,000 commits this year — McDonaghMatthew · 2026-10-29
- 5,151 Commits in a Year, 90% Written by AI Agents on His Behalf — McDonaghMatthew · 2026-10-29
- autonomous_labs launches autonomous operator for Shopify and eBay stores — dee_hw · 2026-10-29
- Lyft Co-founder's Agent Toolbox: last30days at 66k Stars, New Tincan Lets Agents Help Each Other — EXM7777 · 2026-10-29
- Weaviate Podcast: MIT PhD's "Machine Studying" lets always-on agents prep without knowing the task — CShorten30 · 2026-10-29