Rachel Thomas highlights research on LLM instruction-following failures
fast.ai co-founder Rachel Thomas posted a thread reviewing papers and benchmarks on LLMs' unreliable instruction following, spotlighting Inverse IFEval, which tests models on out-of-distribution instructions such as flawed questions, comment-free code, mid-prompt changes, and banned bullet points.
2026-10-19 ~ 2026-10-19 · 3 related posts
- Rachel Thomas threads papers on LLM failure to follow instructions, starting with Inverse IFEval — math_rachel · 2026-10-19
- Inverse IFEval: A New Benchmark Testing Whether LLMs Can Follow Out-of-Distribution Instructions — math_rachel · 2026-10-19
- Inverse IFEval tests flawed questions, code without comments, mid-chat rule changes — math_rachel · 2026-10-19