Apple's on-device model scores 45/49 in local classification tests but is vulnerable to prompt injection

NellWatson tested Apple's on-device Foundation Model on a SCADA event routing task, scoring 45/49 on held-out samples. A one-line queue description trick further improved small-model accuracy and cut Jev 1.14's susceptibility to injected answers from 7/20 to 1/20.

2026-10-25 ~ 2026-10-25 · 2 related posts

• Apple's on-device Foundation Model tested: 45/49 on routing, but easily prompt-injected — NellWatson · 2026-10-25
• Follow-up: one-line queue descriptions cut Jev's planted-answer rate from 7/20 to 1/20 — NellWatson · 2026-10-25