TensorFold ships macOS app for local LLM inference, reports 874 tok/s on RTX PRO 6000

HankYeomans · x · 2026-10-15

• Open-source inference engine TensorFold launched a Beta macOS app so users can launch the engine without the CLI; the app lives under /app in the repo and accepts PRs.
• It runs open models on your own Apple Silicon or NVIDIA GPUs via Metal/CUDA, exposes OpenAI- and Anthropic-compatible APIs, is Apache-2.0 licensed, and is part of NVIDIA's Inception startup program.
• Published benchmarks with named hardware: 874 tok/s greedy decode with drafting for Nemotron 3.5 Lightning (MLX affine 4-bit) on an RTX PRO 6000 Blackwell, 90–110ms prompt reuse, and 199 tok/s aggregate across 8 concurrent streams. Features include draft-propose/verify-commit parallel decoding, prompt reuse, and running one model across connected Macs. A satellite deployment is also in preparation.

Original post →

More from Infra

• CoreWeave's 2032 bond yield touches 15.1%, closes at record 14.5% as AI buildout leans on private credit — asusarla · 2026-10-15
• Three questions to ask your AI supplier: capacity ownership, financing, change notice — YvesMulkers · 2026-10-15
• BlackRock CEO Larry Fink warns halting data center construction could plunge the U.S. into recession — Polymarket · 2026-10-15
• Mine Bittensor Subnet 92 by pointing Claude Code at one link — airesearch12 · 2026-10-15
• OpenAI infra lead leaves to start up after locking in double-digit GWs of compute — DesireeCachette · 2026-10-15
• Who Sells Data directory maps 159 AI training data vendors, compares Scale vs Mercor — cneuralnetwork · 2026-10-15

Infra channel →