LFM3-1.7B impresses in medical long-context work: 20k tok/s prefill on a 3090Ti
AdRepulsive7837 · reddit · 2026-11-20
A developer working with medical records shares early hands-on results with LFM3-1.7B: feeding patient-sized chunks (128k tokens) of medical notes and medication lists for summarization and QA tasks, such as pinpointing when a drug was added or what treatment a doctor chose.
Qwen 27B handled the task but needed 20 minutes just to prefill on their machine. LFM3-1.7B turned out surprisingly fast and faithful — 20,000 tok/s prefill and 350 tok/s decode on a 3090Ti — with few critical hallucinations. Its main failure mode is confusing generic vs. brand drug names, which even offline GPT and Claude models stumble on. No quantitative evals yet; the author predicts 2027 will be the year major players push sub-10B models.
More from Infra
- Matt Slotnick questions Hunterbrook's AMD story: 'seems really flimsy' — matt_slotnick · 2026-11-20
- Prime Intellect breached: attacker got admin access to model-serving machines — himanshustwts · 2026-11-20
- The 'Costco for Compute' launches: 30+ GPU types from $0.20/h to undercut big cloud's 5x markup — ycombinator · 2026-11-20
- Anthropic needs ~$602B annual revenue to earn 20% on its ~$1T compute commitments — basedjensen · 2026-11-20
- AMD crashes 13% in 30 minutes after short seller alleges illegal China chip sales — econoar · 2026-11-19
- Anthropic CEO Dario Amodei: hard to predict our compute needs — 'it's a lot' — BenBajarin · 2026-11-19