NNsight 0.8 ships: vLLM becomes first-class runtime with rewritten execution engine
gsarti_ · x · 2026-10-19
NDIF team released NNsight 0.8, the interpretability library's biggest release yet, with a from-scratch rewritten execution engine and unchanged API (still model.trace(...)).
- vLLM is now a first-class runtime: optimized tracing scales well when extracting/intervening on activations across many layers and token positions
- Unified TransformersModel: backed by transformers.pipeline, so Whisper, LLaVA, Qwen3-VL, BERT encoder tasks, and PEFT adapters all trace with the same code
- Speed: greenlets replace OS threads; per-read overhead is 3x lower and stays flat as you touch more of the model
- Also includes .source for reaching inside forward passes and multi-GPU model support
More from Research
- Chris Potts clarifies: better models are easier to interpret, and agentic tools speed up interp research — ChrisGPotts · 2026-10-19
- Chris Potts critiques auto-interp evaluation: precision, recall, and the case for ex ante testing — ChrisGPotts · 2026-10-19
- Audio analysis veteran backs claim that conscious continuity is an illusion, flipping every 60ms — eigenhector · 2026-10-19
- Ropedia W0: One World Model Trained on 10,000 Hours of Egocentric Human Video for Any Robot — liuziwei7 · 2026-10-19
- Baker lab designs de novo cysteine proteases that cleave peptide bonds, published in Nature — DaveJuergens · 2026-10-19
- DKGI: agents treat knowledge graphs as files and co-evolve their own tools, +6.7 Hit@1 — yuz9yuz · 2026-10-19