layerpool open-sources layer-split fine-tuning across machines with unchanged HF Trainer
Historical_Syrup_461 · reddit · 2026-11-23
- layerpool splits a transformers model by layers across multiple devices for fine-tuning: the main machine runs the training loop and keeps embeddings and the output head, while other devices hold the intermediate layers, each downloading only its own.
- Code changes are minimal — swap one import (import layerpool.hf as lp) — and Trainer, TRL, LoRA and QLoRA work unchanged. Supports NVIDIA/AMD/Apple Silicon/CPU/Colab/Kaggle. The author notes network speed is the bottleneck, so it targets models that don't fit on a single GPU otherwise.
- Repo: github.com/abdallah-swe/layerpool
More from Infra
- KV cache offload to RAM runs Qwen 3.8 with 8x200k streams on 4x3090 — Mr_Moonsilver · 2026-11-23
- OpenAI chart reveals unreleased GPT-6 Astra models; Domingos questions billions for <0.3 bits — pmddomingos · 2026-11-23
- wllama author: RPC-linked wasm instances show little performance loss, credits GGML RPC backend — ngxson · 2026-11-23
- wllama v4.0.0 ships: in-browser llama.cpp up to 2.8x faster with WebGPU hybrid — ngxson · 2026-11-23
- French dev laments missed chance to build green data center ecosystem on low-carbon grid — Dimillian · 2026-11-23
- Fchollet: deployment friction may be the best thing for AI infra, stretching the buildout timeline — npinto · 2026-11-23