Preparing post-training data for coding LLM fine-tunes is 'black art', says practitioner
ayushthakur0 · x · 2026-10-13
Ayush Thakur argues preparing post-training data for fine-tuning LLMs on coding is like black art: you must consider the eval harness itself (especially when hard to configure), router layers like litellm injecting their own transformations so what users see differs from internal processing, and sampling choices — how much to sample per turn, where to set boundaries, how many tokens to pack per row. He asks whether any extensive study exists on this.
More from coding & agent
- PageIndex SDK v0.4 claims tree indexing now cheaper than vector indexing by ~15% — solyarisoftware · 2026-10-13
- Economist: agentic coding lets you write your own Fortran optimizers, not just ask for Stata packages — Apoorva__Lal · 2026-10-13
- Tencent open-sources Agent Memory: local long-term memory, 61% fewer tokens, 48%→76% accuracy — solyarisoftware · 2026-10-13
- Testing AI agents with simulations: replay real traces, 50 curated sims beat 1000 random ones — HamelHusain · 2026-10-13
- How humans and Copilot agents split the work to build a GitHub Copilot beginner course — DanWahlin · 2026-10-13
- Engineers distill their AGI debate into an essay on self-verifying, continuously improving agents — dotey · 2026-10-13