Artifact Distillation: training-free transfer lets Claude Haiku 4.5 beat Sonnet 4.5 at 60% cost
kalyan_kpl · x · 2026-10-31
A new paper introduces Artifact Distillation, sidestepping the two main drawbacks of knowledge distillation: it requires fine-tuning, and transferred knowledge stays opaque.
How it works:
- The teacher model (e.g. Claude Opus 4.5) solves training instances and externalizes its problem-solving into a queryable artifact bank
- Student models query the bank at inference time — no gradient updates, fully training-free
Results:
- On four LiveBench tasks, Claude Haiku 4.5 with an Opus 4.5-written bank scores 0.707 on spatial reasoning, beating unaided Sonnet 4.5 (0.647) at 60% of its cost; it matches Sonnet 4.5 on table joining at 89% of the cost
- The same raw bank lifts Qwen3.5-27B's pooled score from 0.730 to 0.800, showing transfer isn't limited to same-family models
More from Research
- Steering vectors explain out-of-context reasoning in fine-tuned LLMs — even backdoors — sebkrier · 2026-10-31
- Claude explains why LED streetlights make China's night lights understate GDP growth — teortaxesTex · 2026-10-31
- Tsinghua Open-Sources Falcon: Visual Target Speaker Extraction With 11x Lower Latency — aigclink · 2026-10-31
- Neuron steering experiment: models confabulate reasons for implanted word choices — Sauers_ · 2026-10-31
- GOFish: fisheye-trained 3D Gaussian Splatting surpasses 180° FoV and extracts collision-ready meshes — janusch_patas · 2026-10-31
- AI researcher argues 90% of research is too vague to be wrong as automated review costs drop to zero — RexDouglass · 2026-10-31