Fewer Prompts Alter jspace Shape
eliebakouch · x · 2026-07-18
While estimating jspace, the author mentioned using only 250 prompts this time due to the larger model size, compared to the 1000 prompts used in previous neuronpedia checkpoints.
They also shared previous experiments on Gemma 3-27B demonstrating that prompt quantity affects the shape of jspace. Key takeaways include:
- Estimation scale is constrained by model size
- The number of sampled prompts changes the resulting shape
- This phenomenon was already observed in prior Gemma 3-27B experiments
Related event: MoE Experiments Reveal Prompt Quantity Impacts Jspace Estimation(2 posts)→
More from Research
- Datology webinar: Curation Studio matched top data baselines with under 20% compute — josh_wills · 2026-10-27
- KANLab: an interactive browser playground for Kolmogorov-Arnold Networks — Lukas Niehaus · 2026-10-27
- CMU's LLM2Cert synthesizes stability certificates verified via SMT and Lean — CarnegieMellonU · 2026-10-27
- NVIDIA open-sources FLORA, a robotics data engine ~125x cheaper than GPT-6 Astra — nvidia · 2026-10-27
- RSNA 2026 Deep Learning Labs return: eight hands-on AI classes over four days in Chicago — PRouzrokh · 2026-10-27
- Steering two orthogonal concept neurons makes a model compose "gnome" — Sauers_ · 2026-10-27