LLMs as probability-guided search in token space: data coverage gaps are the real weakness
AlexTensor · x · 2026-09-22
A discussion argues the conditional probability model P(X|previous tokens) serves as a guide for search through combinatorial token space, with tokens sampled via various strategies. The catch: P() is unreliable where training data coverage is poor. The quoted reply likens agents using tools to A graph traversal with a decision tree—swapping lowest cost for probabilities learned from watching humans, which arguably isn't real conceptual understanding.
More from Research
- Google Distinguished Scientist to Give UCSB Talk on Reinventing the Camera with AI — docmilanfar · 2026-09-22
- FLA's KDA safe_gate fast path leaks future gates into prefix outputs; fix coming — YouJiacheng · 2026-09-22
- Halvar Flake: CS Is Turning From an Exact Field Into a Probabilistic One — soumitrashukla9 · 2026-09-22
- Formal CRC guarantee for a semantic cache breaks when the cache's own writes shape future calibration data — Reasonable_Royal_621 · 2026-09-22
- Category-aware expert RL framework hits 59% on SWE-bench Multilingual — Logics-MLLM · 2026-09-22
- Stanford's three-tier controller gets GPT-6 Astra to 79.13% on RoboMME with just 3.63 calls per episode — DJiafei · 2026-09-22