inside-llm: open-source tool visualizes LLM internals and token-by-token quantization diffs
AlternativeEnough352 · reddit · 2026-11-16
A developer released inside-llm, an open-source MIT-licensed tool built on llama.cpp that shows what a GGUF model is doing as it generates text — and compares two quantizations of the same model token by token.
Features
- Stream: side-by-side generation with brightness as token probability and magenta marks where models A and B diverge; lockstep and free modes.
- Candidates: full distributions with entropy, effective option count, Jensen-Shannon divergence, and bidirectional KL.
- Gates: stage-by-stage visualization of the sampler chain (penalties, top-k, top-p, min-p, temperature, GBNF).
- Timeline & branching: entropy/divergence curves; click any candidate to continue from there, with exact KV-state snapshots for branch jumps.
- Logit lens: per-layer predictions as a 2D map and 3D relief.
- One-click Hugging Face download of quant pairs with tokenizer matching and VRAM fit estimation.
Stack: C++17 process linking llama.cpp, compact binary frames over WebSocket to a Svelte 5 + Canvas frontend; sessions can be recorded and replayed statically. Packages for Linux/Windows with CUDA or Vulkan; fully local, no telemetry.
Limits: both models must share a tokenizer and fit in VRAM; text only.
More from Models
- Anthropic tightens supported regions policy, closing loopholes for restricted countries' entities — J_MaestreVidal · 2026-11-16
- Wispr's self-trained Canto 1 speech model hits 3.4% WER on real-world dictation — rohanpaul_ai · 2026-11-16
- In real-world testing, Gemini 4 ranks a distant third for coding behind Claude Fable 5.2 and GPT-6 Astra — HarrySurden · 2026-11-16
- Why no GPT-6.2 yet? OpenAI may be prioritizing scarce compute for research over shipping intermediate checkpoints — kimmonismus · 2026-11-16
- AI influencer teases: November to be a good month for open models — mervenoyann · 2026-11-16
- repligate: Fable 5 models get cut off just for thinking about classifiers — repligate · 2026-11-16