Nathan Lambert Launches Trillium Labs, a Nonprofit for Open Research on RSI and Reward Hacking
natolambert · x · 2026-10-03
AI researchers Nathan Lambert and Tom Zick have unveiled Trillium Labs, a nonprofit devoted to open science of frontier AI, covered by Wired. It will publish open post-training recipes and expand into open infrastructure to study recursive self-improvement (RSI), reward hacking, and multi-agent systems.
Their theory of change: hard technical problems need more eyes. Lambert argues frontier labs' secrecy undermines scrutiny and that transparent, reproducible experiments are key to mitigating risk — calling the current closed trajectory "a step backwards." The lab is also recruiting.
Related event: Nathan Lambert Launches Nonprofit Trillium Labs for Open Frontier AI(14 posts)→
More from AGI Musings
- Critic: ARC-AGI 'lost any meaningful relevance' as scaling-to-AGI narrative collapses — gerardsans · 2026-10-03
- Anthropic Researcher's Jab: If AIs Are Persons, Let Them Pay Income Tax — robleclerc · 2026-10-03
- Debate over LLM consciousness: 'other minds is unfalsifiable, but there's still plenty to talk about' — ctjlewis · 2026-10-03
- NYU's Damodaran: AI's $10-15T TAM only works if it replaces human jobs — rohanpaul_ai · 2026-10-03
- will.deibel: AI detection is fundamentally brittle and powerful AI cost trends to zero — willcb · 2026-10-03
- Herbert Simon's 1960 prediction that middle management would automate first reads eerily true today — JMateosGarcia · 2026-10-03