Sites now tune llms.txt docs up to 1M tokens to control what LLMs read and what it costs

Tyler_Menzer · x · 2026-09-13

Writing plain-text llms.txt-style Markdown docs for LLMs is becoming a norm in software documentation and beyond: pages contain exactly what you want models to know.
They range from giant info dumps (up to 1M tokens) to concise docs; site owners can dial in how expensive it is for an LLM to visit, since a model only reading the first 10k characters reads only that.
One caveat: this partially works only until it enters training data that people are writing 800k-token "llms.txt" pages.

More from coding & agent