Google's Gary Illyes: keep important content in HTML as LLMs crawl the web
lilyraynyc · x · 2026-09-16
SEO practitioner Lily Ray relays Gary Illyes' remarks at a conference about Google's indexing history and lessons for the LLM era.
- Around 2007-2010, Google had a freshness problem: it wasn't surfacing the newest information.
- It shipped Caffeine (Percolator) to serve fresher results, then built a web rendering engine to process JavaScript and emulate what users see in their browsers.
- Despite that capability, Illyes argues important content should still live in plain HTML — especially now, since LLM-based systems crawling the web may not render JS the way a browser does.
More from coding & agent
- Microsoft's MemSteer turns unlabeled user histories into agent memory supervision — MicrosoftResearch · 2026-11-02
- She fired herself from her own Instagram: AI agent runs the account, views up 82.3% — heyshrutimishra · 2026-11-02
- Sourcegraph: AI writes code faster than teams can review it, driving more outages — AI Engineer · 2026-11-02
- Tsinghua & Microsoft's HarnessMM cuts agent token use by up to 87.93% while raising success rate — rohanpaul_ai · 2026-11-02
- GitHub launches Agent HQ: Anthropic, OpenAI, Google coding agents in Copilot — emmanuelvivier · 2026-11-02
- DeepSeek open-sources Super-Agent, a coding agent that reads team chats to minimize human input — emmanuelvivier · 2026-11-02