Is RAG Still the Default? Veteran Dev Says Long Context and Agents Are Taking Over
Warm-Reaction-456 · reddit · 2026-08-04
A software developer with 8 years of experience argues that RAG (Retrieval-Augmented Generation) should no longer be the automatic default for enterprise AI implementations.
After shipping over a dozen RAG projects, the author found that most disappointing failures weren't retrieval problems at all. In one case, a client wanted 40,000 documents embedded, but analysis revealed over half of employee queries were structured data questions (like refund totals). Only about 50 documents were actually useful; blindly embedding the rest would have just delivered outdated policies faster. Furthermore, retrieval assumes someone actively maintains the data library, which most companies fail to do.
With expanding context windows, the architectural paradigm is shifting:
- Long Context: For stable document sets, simply caching them in the model's context is often more effective than chunking and embedding.
- Agentic Search: For other needs, agents can act like junior analysts—running keyword queries and querying live APIs directly for real-time answers, rather than relying on stale vector copies.
The author advises teams to manually answer 5 core business questions before approving the next RAG budget to understand the actual technical problem.
More from AGI Musings
- The 'Lee Sedol Effect': How AI Will Outpace Top Human Experts — DeryaTR_ · 2026-08-04
- Similarweb Report: AI Mainstream Grows as Older Age Groups Increase Chatbot Usage — gaganghotra_ · 2026-08-04
- Finding Purpose: What Will You Do If ASI Outsmarts Humanity 100x? — Excellent-Photo9786 · 2026-08-04
- Framing AI Disruption: Can a 15-Year-Old Prodigy Revolutionize Your Field? — iskander · 2026-08-04
- After the Proof: How AI Will Radically Change Science — vishalmisra · 2026-08-04
- AI's Hallucinations Are Forcing Scholars to Rigorously Check References — anderssandberg · 2026-08-04