K12-KGraph, a curriculum-aligned knowledge graph from Chinese K-12 textbooks, contains 10,685 nodes and 23,278 edges. Even Gemini-3-Flash scores only 57% on its 23,640-question benchmark.
Key facts
- 10,685 nodes in K12-KGraph.
- 23,278 edges connecting curriculum concepts.
- 23,640 questions in the benchmark.
- Gemini-3-Flash scores 57% accuracy.
- 2,267 training pairs for alignment.
K12-KGraph, released by a team of researchers, is a curriculum-aligned knowledge graph built from official Chinese K-12 textbooks According to @HuggingPapers. The graph comprises 10,685 nodes (concepts) and 23,278 edges (relationships), covering subjects such as mathematics, physics, and chemistry across all grade levels. The accompanying benchmark includes 23,640 questions, designed to test model understanding of curriculum structure and concept dependencies.
The key finding: Gemini-3-Flash, Google's latest frontier model, achieves only 57% accuracy on the benchmark. This suggests that even state-of-the-art LLMs struggle with structured, curriculum-specific reasoning—a task that requires not just factual recall but also knowledge of pedagogical ordering (e.g., that multiplication precedes division in the Chinese curriculum). The dataset also provides 2,267 training pairs for alignment tasks, though the team has not yet released baseline results from fine-tuned models.
What makes K12-KGraph different
Unlike generic knowledge graphs (e.g., Wikidata or ConceptNet), K12-KGraph is explicitly curriculum-aligned. Edges encode prerequisite relationships (e.g., "Pythagorean theorem" requires "right triangle") and grade-level constraints. This structure allows researchers to evaluate whether models understand not just what concepts exist, but when and in what order they should be taught. The 57% score from Gemini-3-Flash indicates a significant gap between general knowledge and curriculum-specific reasoning.
The 2,267 training pairs are sparse relative to the graph's size. For comparison, typical knowledge graph completion datasets like WN18RR have 40,000+ training triples. This data scarcity likely limits the effectiveness of supervised fine-tuning, making K12-KGraph a harder test for generalization.
What to watch
Watch for baseline results from fine-tuned models on the 23,640-question benchmark. If open-source models like Qwen2-72B or DeepSeek-R1 surpass 57% with the 2,267 training pairs, it would signal that curriculum alignment is solvable with limited data. Also monitor whether the team releases a larger training set.
