Hubble: 14 open-source LLMs built to study memorization, with practical takeaways
ChrisGPotts · x · 2026-10-23
Chris Potts credits the Hubble project team for the model suite used in his group's experiments. Hubble is a fully open-source suite for the scientific study of LLM memorization: 8 core models (1B/8B, pretrained on 100B/500B tokens) in standard and perturbed variants show memorization risk depends on sensitive-data frequency relative to corpus size, while 6 more perturbed models show unexposed sensitive data gets forgotten. Best practices: dilute sensitive data with larger corpora and place it earlier in training.
More from Research
- Three Pitfalls of Fusion LLM Evals: Shortcuts, Leakage, and Memorization — anshulkundaje · 2026-10-24
- Stanford Prof Warns Fusion LLM Evals May Rely on Non-functional Shortcuts — anshulkundaje · 2026-10-24
- MIT's blind assembly robot hits 73% peg-insertion success with zero-shot sim-to-real — chris_j_paxton · 2026-10-24
- Flow Matching Explained: Why Streamlines Shift as the Noise Step Changes — CSProfKGD · 2026-10-24
- "RL doesn't work until it does": researchers showcase RL for high-reliability robot manipulation — chris_j_paxton · 2026-10-24
- CREATE benchmark still unsaturated: GPT-6.1-Sol hits 23.8 on creative utility — gregd_nlp · 2026-10-24