What actually happens in the 3 seconds before your chatbot replies: tokens to RAG
Ok_Translator_227 · reddit · 2026-11-02
A Reddit deep-dive explains the full pipeline behind a real chatbot: tokens and context windows (with per-token cost), the illusion of memory (entire history is resent each turn), chunking and embeddings in a vector DB, RAG retrieval of the 3-5 nearest chunks, system prompts, function calling where code actually executes tools, and pre-launch evaluation plus safety guardrails. Bottom line: every step is a judgment call, and one bad call ruins the experience even with a great model.
More from coding & agent
- Open-source AnyShot: a Claude agent acts as cinematographer, cuts 6-hour renders to 3 minutes — Scobleizer · 2026-11-02
- With Chrome riddled with UAF bugs, could an agent swarm finish Servo? — georgejrjrjr · 2026-11-02
- Dialkit macOS: open-source live inspector for tuning iOS UI from your Mac — mikelikesdesign · 2026-11-02
- Developer ships v0.6.83 entirely via autonomous agents, zero human input — jasonkneen · 2026-11-02
- OpenCode MAU jumps 28% to 22M in October, biggest growth since March — ycombinator · 2026-11-02
- Open-source pixel-plays: browser games as isometric pixel dioramas, one JS file, zero dependencies — algo_diver · 2026-11-02