LongCoT-based RLHF: Llama-3.1-8B with 14K examples beats GPT-4o on chat benchmarks
xiye_nlp · x · 2026-10-01
AdithyaNLP will present a poster at COLM 2026 on a simple recipe: use long chain-of-thought (with a reward model) for RLHF instead of math/code RL, so models "think before they chat".
Key results:
- Built on Llama-3.1-8B-Instruct with only 14K training examples
- Beats GPT-4o on chat and creative writing
- Even outperforms Claude-3.7-Sonnet (thinking) on AlpacaEval2 and WildBench
The author also teases upcoming work on making LLMs learn from non-language data.
More from Models
- AI's clever quantization move is just sensitivity analysis, an old pruning trick — code_star · 2026-11-16
- AI progress doubled since GTA VI's first trailer; GPT-4 Turbo now ranks 379th — ArtificialAnlys · 2026-11-16
- Tencent's Hy4 Flash builds a 3D game with shield mechanics in 63 minutes, beating Hy4's 90 — ShunyuYao12 · 2026-11-16
- Hy4 Flash 3D voxel demo outpaces DeepSeek-V4.1 in same-prompt comparison — ShunyuYao12 · 2026-11-16
- Yacine MTB claims training LLMs is 'easy', says he's shocked by how simple it is — yacineMTB · 2026-11-16
- MiniMax M3.1 Flash lands on OpenRouter, with Inferact as first provider — Xianbao_QIAN · 2026-11-16