K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs
Hao Liang, Qihan Lin, Zhaoyang Han, Xiaochen Ma, Zhen Hao Wong, Meiyi Qiang, Linzhuang Sun, Wentao Zhang
cs.CL
2026-05-11
From People's Education Press textbooks the team built a curriculum knowledge graph (9 node types, 14 relation types) and a 23,640-question benchmark where the best model reaches only 57%, then narrows the gap with 7,335 training samples.
Existing education benchmarks (C-Eval, CMMLU, GaokaoBench, EduEval) mostly test whether a model can answer exam questions, not whether it understands how curriculum knowledge is organized. The authors call the latter curriculum cognition, covering prerequisite chains, concept taxonomies, experiment-to-concept links, pedagogical sequencing, and the ability to ground concepts in textbook figures. A model that scores well on exams may still lack the structural understanding needed for tutoring, and that is the gap this work targets.
K12-KGraph is extracted from People's Education Press textbooks across mathematics, physics, chemistry, and biology, spanning primary, middle, and high school. The graph has nine node types (book, chapter, section, concept, skill, experiment, exercise, figure, visual element) and fourteen relation types. The text side holds 6,579 concepts, 1,364 skills, 652 experiments, and 1,171 exercises; the multimodal side has 7,388 figures and 10,203 visual elements.
From this graph the authors derive three artifacts. K12-Bench is 23,640 multiple-select questions in five task families: Ground (knowledge grounding, concept and skill ownership), Prereq (prerequisite reasoning), Neighbor (neighbor recommendation, such as hypernym or related-to), Evidence (experiment evidence chains), and Locate (cross-chapter indexing). K12-Train is 7,335 training samples, of which 2,267 are text-only QA and 5,068 are multimodal visual QA, all synthesized along the graph's nodes and edges.
Even strong models struggle with curriculum cognition.
| Model | Overall EM | Overall F1 |
| Gemini-3-Flash | 57.1% | 73.0% |
| Gemma-4-31B-IT | 46.4% | 69.5% |
| GPT-5.2 | 42.8% | 68.0% |
| Random baseline | 6.7% | 36.4% |
Prereq and Neighbor are hardest; even Gemini-3-Flash stays below 35 percent EM. LLaMA-3-8B is essentially at random (7.2 percent).
Training experiments show domain supervision helps. On Qwen3-4B-Base under a matched budget of about 2,300 samples, K12-Train-Text scores 1009.96 on GaokaoBench, above DataFlow's 985.91 and WizardLM's 922.75. On the multimodal side, K12-Train-Full reaches 39.9 percent on Gaokao-MM, above the base model's 32.4 percent and full DataFlow's 33.3 percent, and clearly beats the text-only and multimodal-only variants, which shows text and visual supervision are complementary.
For education-model builders, this is a diagnostic on a new axis: not just whether a model solves problems, but whether it grasps the structure of the knowledge. K12-Train beats much larger general-instruction subsets on several Gaokao benchmarks with very few samples (7,335), which shows curriculum-structured supervision is dense and worth using in vertical education models.
The authors concede several. The graph covers only four STEM subjects from one publisher (People's Education Press), with no humanities and no other editions, so representativeness is limited. The multimodal part is mostly figures and diagrams, with little video or interactive content. Validation relies on human annotators, though inter-annotator agreement is good (Fleiss' kappa 0.84). Training gains may also reflect sample quality and domain specificity, not solely the effect of curriculum grounding. And a benchmark of synthesized QA inevitably carries template traces, so models may learn some template regularities rather than genuine structural understanding.