llama.cpp PR adds tensor parallelism across multiple machines via -sm tensor
jacek2023 · reddit · 2026-10-06
A new llama.cpp RPC pull request (#26610 by am17an) adds a -sm tensor split mode enabling tensor parallelism across multiple computers.
This lets a single model be split across several machines for inference, removing single-node VRAM limits — a significant step for running large models locally.
More from Infra
- Polymarket puts 25% odds on data centers launching to space by end of 2026 — Polymarket · 2026-10-07
- Airbnb Engineering: replay real DB traffic instead of trusting synthetic benchmarks — rseroter · 2026-10-07
- Inference startup engy grows revenue 100% MoM, processing 4-5B Kimi K3 tokens daily — markjeffrey · 2026-10-07
- GPT-6 'leaks' hint at looped nested-model architecture, per Reddit speed-math analysis — QuackerEnte · 2026-10-07
- Reza Zadeh claims faster matrix multiplication algorithm, suspects labs near exponent 2 — Reza_Zadeh · 2026-10-07
- Ministack: open-source local AWS emulator runs 60+ services on one port, free LocalStack alternative — tom_doerr · 2026-10-07