AI exam-question pipeline: ~80% of MCQs pass all checks, science questions still weak
SnooOpinions9858 · reddit · 2026-10-12
A developer building an AI question-bank feature for a high-school platform shares a working pipeline: LLM splits past papers into sub-questions tagged to curriculum objectives, a small model writes new questions, another model blind-answers them, and MCQ keys are verified in code against mark schemes. About 80% of MCQs pass all checks; structured/science questions are weaker until a mid-size model is used. They seek advice on PDF extraction (docling/marker), pgvector retrieval, batch API costs at scale, and eval approaches beyond human review.
More from coding & agent
- Claude Code Pushes Urgent Security Update Across All Installations — umeshonai · 2026-10-23
- Claude cut a 2.6-hour video export to 20 minutes — but only after being told to rebuild the server — byhewar · 2026-10-23
- 3 flight-booking rules my AI agent checks that I'd never do by hand — Efistoffeles · 2026-10-23
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Building an autonomous SEO agent in Hermes with three skills: programmatic pages, Keywords Everywhere, DataForSEO — EXM7777 · 2026-10-23
- Building a sports betting AI agent with onchain execution and human approval by default: architecture and lessons — Dry-Comfortable-2678 · 2026-10-23