Test Shows LLMs Return Probabilities, Not Certain Answers

A developer tested Jev with a simple letter-counting app, showing LLMs return probability distributions instead of definite answers, often getting even trivial questions wrong.

2026-09-20 ~ 2026-09-20 · 2 related posts

• Testing Jev on counting letters: it returns probabilities and still flubs obvious questions — eptwts · 2026-09-20
• Counting the letter 'a' with an LLM: probabilities, not answers, reveal where LLMs fall short — eptwts · 2026-09-20