Ex-Apple engineer chats with GPT-2: model invents a backstory and gets politely debunked by Codex

Mike Frank, a former Apple chip executive, built a small chat app around GPT-2 (the 1.5-billion-parameter model released in 2019) and spent several days in early October running extended conversation experiments with it. His overall takeaway: GPT-2 is not fully coherent—it often gets stuck repeating itself and needs manual regeneration of the last reply—yet its level of engagement exceeded his expectations, managing to keep the conversation going over long exchanges.

Confirmed

• In the self-built GPT-2 chat app Mike Frank demonstrated early on (Oct 4), the model's replies were mostly fluent and reasonable, but it frequently fell into repetition and needed manual regeneration.
• In follow-up experiments (Oct 6), during a long conversation GPT-2 "hallucinated a backstory for itself," claiming to be both a journalist and a software engineer.
• When he probed further using Codex/Sol-6.1, the model politely pushed back on these hallucinated self-descriptions; he joked that it was "GPT-2 making up stories about itself."

Why it matters

• This is a hobbyist-style observation of an early small model's behavior: a 1.5-billion-parameter 2019 model already showed surprising engagement in long conversations, along with classic hallucination behavior (inventing an identity), offering a vivid case study of baseline behavior before the era of large models.

2026-10-04 ~ 2026-10-06 · 5 related posts

• [source] Ex-Apple exec Mike Frank demos a GPT-2 chat app that's cogent but repetitive — MikePFrank · 2026-10-04
• Researcher chats with GPT-2: less coherent than modern models but surprisingly engaged — MikePFrank · 2026-10-06
• GPT-2 hallucinates a backstory for itself in ongoing dialogue experiment — MikePFrank · 2026-10-06
• [source] GPT-2 hallucinates a journalist-engineer backstory for itself, politely called out — MikePFrank · 2026-10-06

1 near-duplicate retellings: MikePFrank