300M-Parameter Model Learns Six Languages In-Context from Synthetic Data Alone

A new paper introduces PFLM, a 300M-parameter byte-level transformer trained purely on synthetic data with no real language, which extends prior-fitted networks to natural language and can learn six languages in-context.

2026-10-06 ~ 2026-10-06 · 2 related posts

• PFLM: a 300M model pretrained on zero real languages learns languages in context — Lennart Carstens-Behrens · 2026-10-06
• 300M-Parameter Transformer Trained Only on Synthetic Data Learns 6 Languages In-Context — cbl007 · 2026-10-06