New Paper: Injecting Synthetic Moral Reflections During Pretraining Shifts Value Priorities in 3B Models
sethlazar · x · 2026-08-14
The Synthetic Persona Pretraining paper is out, proposing to inject synthetic morally-laden reflections into 10% of pretraining documents, training models up to 3B parameters. Early intervention significantly shifts the model's value priorities.
More from Research
- AutoDesign: Meta-optimizing agent harnesses beats Claude Design on poster generation — dair_ai · 2026-08-15
- Qwen3.8-27B identical architecture to 3.6, gains purely from training — Course_Latter · 2026-08-15
- Notion's real-world benchmark: open models hit 94-98% quality, cost as low as $0.02 per task — ivanhzhao · 2026-08-15
- MLA+MTP May Be a Bad Combo: An Arithmetic Intensity Perspective — tokenbender · 2026-08-14
- Notion's real-world benchmark: open models hit 94-98% quality, cost as low as $0.02 per task — ivanhzhao · 2026-08-14
- OpenRouter Launches Web Search Benchmarks for Models and Configurations — AravSrinivas · 2026-08-14