Open-source decision model Laya ported to Core ML: 99.5% ops on ANE, 3.7ms per decision
alexcovo_eth · x · 2026-09-22
A developer ported the open-source Jev-style decision model Laya to Core ML, getting 99.5% of ops on Apple's Neural Engine — 3.7ms per decision on an M5 Pro. The original Laya answers in 27ms, roughly 200x faster than a hosted LLM, and was demoed playing Tetris autonomously.
FluidUse 0.2.0 ships a Swift port producing choice/score/noul answers with calibrated probabilities in a single encoder pass, no generated tokens; fixed-length Core ML buckets (128/256/512/1024 tokens x 32 options), fp16 or e8 int8 embedding (30% smaller, same accuracy); accuracy matches PyTorch on all 10 published suites (3,899 examples), and the Tetris demo hits 15,800 decisions/min.
More from Infra
- SemiAnalysis tears down A20 on TSMC N2, DRAM revealed in iPhone 18 Pro Max package — dylan522p · 2026-09-22
- Bernstein's Intel road notes: servers sold out through 2027, real test comes in 2028 — BenBajarin · 2026-09-22
- Rollout scheduling: the underappreciated infra trick boosting inference efficiency — stochasticchasm · 2026-09-22
- Grok reportedly hit by a datacenter incident, details still unclear — Daniel_Farinax · 2026-09-22
- Decoupled agent runtime, mini harnesses: notes on a frontier lab's infra stack — stochasticchasm · 2026-09-22
- More inference than training machines means writing will soon beat reading — GregoryDiamos · 2026-09-22