LlamaIndex's Jerry Liu and Snorkel AI on why evals and RL environments remain unsolved
ajratner · x · 2026-09-30
Jerry Liu hosted a dinner with Snorkel AI's Vincent Chen on evals and RL environments: data/RL-env companies are booming, benchmarks get shredded each release, fairness attribution (input vs harness vs reward) is hard, and raw enterprise data must be "developed" into usable evals and training environments via new engineering paradigms involving domain experts.
More from coding & agent
- Open Dots: a 4.3k-star open-source, self-hosted alternative to OpenAI's Dots — matchaman11 · 2026-09-30
- ChatGPT subscription now works in 16+ partner products including Devin and Notion — charliermarsh · 2026-09-30
- Armin Ronacher: Pi's default theme now follows your terminal's color scheme — mitsuhiko · 2026-09-30
- Indian team builds an agent that turns Instagram Reel job posts into structured listings, with memory via Hindsight — Such-Pear-588 · 2026-09-30
- Creator Strategy Agent: problem identification and the idea behind it — pranay_naragoni · 2026-09-30
- Giving a content agent memory with Hindsight: a hands-on write-up — TripSuccessful6524 · 2026-09-30