Emergence of Weekday Representations in Pre-training
Sauers_ · x · 2026-07-11
Goodfire's Silico demonstrated findings on how the representation of "days of the week" emerges during the pre-training process.
While the post lacks methodological details, the core message is clear: a model's internal representations aren't explicitly present from the start but gradually form during pre-training. The author considers the results "very cool" and worth checking out.
More from Research
- OpenAI Foundation funds UNC tumor antigen project, hiring senior experimental scientist — iskander · 2026-10-27
- Steering attention heads toward honesty: up moral patienthood, down psychopathy — edelwax · 2026-10-27
- Are transcoders overkill? New paper argues plain neurons suffice for circuit analysis — burny_tech · 2026-10-27
- EMNLP paper GENIE: a fine-grained metric locating where LLM stories are novel — gregd_nlp · 2026-10-27
- Hashed n-gram value embeddings push GPT to 3.2115 loss, beating 30M track record — RobertTLange · 2026-10-27
- MissionReport benchmark: frontier models score just ~60% on government status reporting — Exp_Mark · 2026-10-27