LuxoBench: A New AI Benchmark Tasks Models With Building Real Electromechanical Devices
vincent_koc · x · 2026-09-12
Kyle (@pronouncedkyle) is building LuxoBench, a hardware engineering benchmark for AI models, named after the Pixar lamp.
- The first approach—asking models to design an "interactive desktop lamp"—produced only parts lists and overly simple build instructions, a poor test of real engineering
- The revised method starts from a render of a more complex device and asks each model to make it real and manufacturable, covering electronics, software, and mechanisms
- Judging criteria include cost, lead time, ease of build, and whether the result actually works
- The author is inviting collaborators and will share parts lists and instructions so others can try building too
More from coding & agent
- Cursor Projects hands-on: like Grok Bot for devs, with its own VM, files and memory — pswider · 2026-09-12
- Code Review Is Now an AI-vs-AI Loop Where Humans Just Copy-Paste — dotey · 2026-09-12
- Software Engineers Aren't Obsolete: Engineering Remains the Meta-Skill in the AI Era — dotey · 2026-09-12
- Notion MCP opened 50 auth tabs a day and crashed a user's Mac — menhguin · 2026-09-12
- Matt Pocock's skills repo now has more stars than React — mattpocockuk · 2026-09-12
- Codex beats average programmers: G-buffer shrunk from 24 to 16 bytes/pixel — ducha_aiki · 2026-09-12