High schooler clusters two Android phones over USB to run a 27B LLM locally at 30°C
Grouchy-Zebra9780 · reddit · 2026-10-13
A high school student wrote a custom C++ engine (a fork of llama.rn) that clusters two Android phones over a single USB cable to run a 27B LLM locally without a GPU.
Key details
- Model: Ternary-Bonsai-2-27B compressed to 2.06 bpw / 6.52 GiB, perplexity 14.26 vs 14.05 reference
- Cluster mode (Xiaomi + ROG 7): layers split 4.36 GiB / 2.15 GiB via custom RPC over USB at 2.8 tok/s; solo on a Xiaomi 15 hits 5.6 tok/s
- Why it doesn't overheat: weights are ternary (-1/0/1) in a Hadamard-rotated basis, kept folded in the GGUF; a runtime patch applies the Hadamard transform on the ARM CPU — additions/subtractions only, zero MAC operations, max temperature 30°C
The author published a demo video, engine logs, and a methodology writeup, and is seeking a lab placement or grant; next step is custom Vulkan compute shaders to boost throughput.
More from Infra
- Starlink Mini saves five at sea after sinking boat's radio fails — elonmusk · 2026-10-13
- NVIDIA maps France's AI ecosystem: 50-satellite constellation, Bull to build Vera Rubin servers — nvidia · 2026-10-13
- NVIDIA maps France's agentic computing push: satellites, sovereign models and quantum links — NVIDIA Blog · 2026-10-13
- LangChain built SmithDB, a new database just for agent traces in LangSmith — hwchase17 · 2026-10-13
- IMF Forecast: AI Adds No Measurable Productivity Yet, Bust Would Cost Twice the Boom's Gain — rohanpaul_ai · 2026-10-13
- NVIDIA ships a NURBS B-rep kernel on USD, but devs slam its nondeterministic geometry — rms80 · 2026-10-13