LFM3-1.7B impresses in medical long-context work: 20k tok/s prefill on a 3090Ti

AdRepulsive7837 · reddit · 2026-11-20

A developer working with medical records shares early hands-on results with LFM3-1.7B: feeding patient-sized chunks (128k tokens) of medical notes and medication lists for summarization and QA tasks, such as pinpointing when a drug was added or what treatment a doctor chose.

Qwen 27B handled the task but needed 20 minutes just to prefill on their machine. LFM3-1.7B turned out surprisingly fast and faithful — 20,000 tok/s prefill and 350 tok/s decode on a 3090Ti — with few critical hallucinations. Its main failure mode is confusing generic vs. brand drug names, which even offline GPT and Claude models stumble on. No quantitative evals yet; the author predicts 2027 will be the year major players push sub-10B models.

Original post →

More from Infra

Infra channel →