Simple prompts with few examples match complex ICL methods on NLI, small-LLM study finds
rohanpaul_ai · x · 2026-11-11
A paper from Hellenic Open University et al. challenges the assumption that NLI needs complex prompting and sampling tricks.
- Tested Llama, Mistral, Gemma and Phi 4 small models on MNLI, QNLI and SciTail with one short, direct prompt and 0–15 examples
- Mistral 7B beat the 63.4% MNLI accuracy a published in-context sampling method got with the same model
- Gemma 2, Llama 3.1 and Mistral scored about the same with 1 example vs 15 on MNLI and SciTail
- Demonstration label choice matters in one-shot prompting, with entailment-primed examples performing best
Takeaway: minimal prompting consistently matches or outperforms ICS and MCQ-based methods, though efficacy is model- and dataset-dependent.
Related event: Simple Few-Shot Prompts Beat Complex Techniques for Small Models on NLI(2 posts)→
More from Research
- Cognitive scientists find AI generates ideas faster, but human teams stay more creative — CurieuxExplorer · 2026-11-11
- AI agents ran a biology lab end-to-end: AutoDNA completes DNA read-write cycle in 200 hours — CurieuxExplorer · 2026-11-11
- PufferLib: open-source RL library trains 60M samples/sec on one GPU — jsuarez · 2026-11-11
- Artificial Analysis Intelligence Index weights shift sharply across versions, chart reveals — HamelHusain · 2026-11-11
- Benchling's BenchBench-Refusals: GPT-6.1 Sol tops bio-safety refusals at 88.4%, unverified Opus 5.5 just 40.4% — nlarusstone · 2026-11-11
- Anthropic's top-cited AI scientists rack up 2x OpenAI's citations — NegotiationOk7535 · 2026-11-11