Flama, an 8-year Python ML serving framework, publishes 83-page arXiv paper on its architecture
p3rdy · reddit · 2026-10-13
The author walks through Flama, a Python framework started in 2018 that treats ML models and LLMs as first-class citizens, now documented in an 83-page arXiv paper (2608.18733).
- A model is a file: every model ships as a single .flm containing weights, inference artifacts (tokenizer, config) and training metrics. Metadata precedes weights so flama model inspect reads it without loading; CI can gate promotion on metrics, and the same LLM file runs on vLLM or MLX.
- One model, four protocols: a single deployment serves OpenAI-, Anthropic- and Ollama-compatible routes plus a streaming chat UI.
- Three decoupled layers (dialect, canonical transport, backend) ensure tool calls and token usage work across protocols.
- The framework also offers dependency injection, Pydantic/Marshmallow schemas with OpenAPI docs, SQLAlchemy CRUD generation, DDD building blocks, JWT auth, MCP server support, upgrade codemods, and a Rust-accelerated core.
More from coding & agent
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Building an autonomous SEO agent in Hermes with three skills: programmatic pages, Keywords Everywhere, DataForSEO — EXM7777 · 2026-10-23
- Building a sports betting AI agent with onchain execution and human approval by default: architecture and lessons — Dry-Comfortable-2678 · 2026-10-23
- Capy 0.5.0 lets Claude Pro/Max subscribers reuse their plan quota, adds OIDC cloud access — garrytan · 2026-10-23
- You.com CEO Richard Socher: legacy code is human slop, let AI clean it up — RichardSocher · 2026-10-23
- Let agents design a single-line glance component so you can track them from the sidebar — blixt · 2026-10-23