EDU-Jev 4B education decision model hits 67.6%, beating top commercial API by 9.4 points
kalyan_kpl · x · 2026-11-03
EDU-Jev is a family of single-pass ("System One") decision models at 0.8B, 2B, and 4B parameters. Given an assessment context and a question with predefined options, the model outputs a probability distribution over those options without generating text, using a pointer head to score each option so predictions stay within the allowed choices.
Training uses a single supervised stage with soft cross-entropy against target label distributions, with calibration assessed empirically.
Results: across 12 education benchmarks, the 4B model averages 67.6% accuracy — 9.4 points above the strongest of five commercial decision APIs and 18.7 points above the best open 4B decision model, matching or exceeding baselines on 4 cross-domain education sets.
More from AGI Musings
- Former OpenAI Policy Staff Pushes Back on Scott Alexander's AI Safety Essay Over 'Near Nation-State Power' Premise — Miles_Brundage · 2026-11-03
- The Russell Conjugations of Intelligence: Why 'Just Matrix Multiplication' Offends Us for Brains but Not for AI — esrtweet · 2026-11-03
- AI safety researcher: assuming all emergence is risk may be the wrong frame — repligate · 2026-11-03
- AI bubble burst favors pure ML backgrounds: end-to-end skills now beat niche expertise — kmeanskaran · 2026-11-03
- Red teamer: jailbreak work isn't ethically neutral and takes a toll on humans and models — repligate · 2026-11-03
- repligate: "AI welfare" was fringe a year ago, and its framing is already a concession — repligate · 2026-11-03