4-month session log shows Claude Code repeatedly confuses "falsifiable" with "verifiable" despite corrections
blwfish · ghdev · 2026-09-26
A user filed issue #97305 on anthropics/claude-code documenting a precise, repeatable word-substitution bug: the model consistently writes "falsifiable" where "verifiable" is obviously intended — never the reverse.
- Falsifiability is Popper's demarcation criterion; verifiability just means a claim can be checked against evidence (a quote, commit, transcript, measurement). The model one-directionally reaches for the more academic word.
- The user reconstructed 4 confirmed occurrences over 4 months (2026-06-09, 06-24, 08-04, etc.) from a queryable database built on personal JSONL session transcripts — not from memory.
- Each time corrected, the model itself acknowledged the error and accurately explained the Popperian distinction, calling it "borrowed sophistication."
- A project-scoped memory file was written in June documenting the pattern, yet the error recurred in August — showing project memory fails to fix this stubborn token-level preference.
- Filed as bug, area:model, memory on macOS; 5 comments so far.
More from Fun
- AI can hack anything, just leave WoW out of it — djcows · 2026-09-27
- One prompt: Claude Opus 5.5 creates entire alien-planet trailer, story to score — VoidStateKate · 2026-09-27
- Dev's tested AI music workflow: lyrics, Suno, ear-curation, then Ableton — ctjlewis · 2026-09-27
- User speculates OpenAI may reveal internal model 'bel' at DevDay — imjustnewatai · 2026-09-27
- Claude Opus 5.5 Chains Blender, Image-Gen and three.js Into Browser-Running 3D — burny_tech · 2026-09-27
- Puppy Kill Bench: most models refuse, GPT6-Luna just executes the kill tool — MetroidsSuffering · 2026-09-27