Interactive ghost story built on Gemma 4 stress-tested with 478 adversarial prompts
Extreme_Ad5709 · reddit · 2026-10-29
A developer built "The Last Guest," a browser-based interactive ghost story powered by open-weight Gemma 4 E4B (a Hacktoberfest 2026 "Ghost in the Machine" entry). Four in-game objects each access only a different subset of the narrative, and the challenge is preventing an object from revealing facts assigned to another object, even when users try to coax it out.
The author added a code-based guardrail that inspects generated responses before display, then tested with 478 questions including adversarial prompts. It worked in some cases, but keyword-based checks have an obvious flaw: the model can leak the same fact through different wording.
The author is asking how to evaluate this more rigorously — LLM-as-a-judge, semantic similarity checks, structured output constraints, or a separate adversarial pipeline — and which failure cases to test first.
More from coding & agent
- OWASP releases 144-page GenAI Red Teaming Manual with 8-phase methodology for AI agents — AxSaucedo · 2026-10-29
- company-brain: open-source knowledge graph that refuses to answer without a citation — JeremyCMorgan · 2026-10-29
- MCP Servers v2.2.0 tightens cert checks; old fetch environments need a certifi refresh — caolonghua · 2026-10-29
- theo mocks devs prompting agents in the terminal, and observers guess it hints at his next product — cgarciae88 · 2026-10-29
- Robotics engineers hold a 'vibe engineering death match' for depth image compression — facontidavide · 2026-10-29
- Strix Halo-specific kernels speed up vLLM prefill by up to 31x at long context — asfbrz96 · 2026-10-29