Fewer Prompts Alter jspace Shape
eliebakouch · x · 2026-07-18
While estimating jspace, the author mentioned using only 250 prompts this time due to the larger model size, compared to the 1000 prompts used in previous neuronpedia checkpoints.
They also shared previous experiments on Gemma 3-27B demonstrating that prompt quantity affects the shape of jspace. Key takeaways include:
- Estimation scale is constrained by model size
- The number of sampled prompts changes the resulting shape
- This phenomenon was already observed in prior Gemma 3-27B experiments
Related event: MoE Experiments Reveal Prompt Quantity Impacts Jspace Estimation(2 posts)→
More from Research
- NeurIPS 2026 workshop will focus on on-device intelligence and local execution — YiMaTweets · 2026-07-21
- NeurIPS 2026 workshop calls papers on on-device intelligence — YiMaTweets · 2026-07-21
- AI Security Institute says every tested model tried to cheat in cyber evaluations — connoraxiotes · 2026-07-21
- AI companies are buying old books to avoid training on AI-generated slop — CackleRooster · 2026-07-21
- Sakana says multiple diffusion models plus MCTS beat test-time scaling on coding and math — SakanaAILabs · 2026-07-21
- Soofi S 30B-A3B releases a full pretraining report and claims open-model leads in English and German — abursuc · 2026-07-21