Can MLLMs Decode the Creative Leap? Introducing C4 for Cross-Concept Understanding
Ming Wang, Yuqing Zhang, Tingna Xie, Xiangju Li, Xiaocui Yang, Daling Wang, Shi Feng, Yifei Zhang
cs.AI, cs.CL, cs.MM
2026-08-07
A 221-item benchmark built from chengyu cross-concept substitution finds the strongest MLLM (GPT-5.5) recovers only 50.7%; candidate sets lift accuracy sharply, so the bottleneck is answer-space search, not perception.
Evaluating creativity in multimodal LLMs is hard because open-ended tasks have no fixed target to score against. This paper picks one creative subskill that can be scored exactly: cross-concept understanding. Its prototype is the familiar Chinese pastime of guessing a chengyu (four-character idiom) from a picture. The setter takes a character in the idiom and replaces it along an associative path (homophony, shape, meaning, or cultural allusion) with another drawable concept (leaf becomes coconut), then draws the result. The solver must reverse those displaced visual clues back to the original idiom. Answers are unique and the associative paths are human-annotated, so for once creativity is batchable, scorable, and graded. The authors formalize this as encoding (the substitution) and decoding (the recovery) and build a benchmark, C4-Eval.
The substrate is a manually annotated, third-party-reviewed chengyu cross-concept network: 47 idioms, 168 replaceable character slots, 758 deduplicated bridges. To build an item, fix an idiom, pick non-overlapping slots, follow bridge paths to imageable substitutes, and generate a picture (the target idiom never appears in it). Difficulty is set by bridge count and depth: L1 one slot one step, L2 two slots one step each, L3 one path deepened, L4 both deepened.
C4-Eval has 184 synthetic items (L1 to L4) plus 37 human-made figures from the web, covering 84 idioms and 884 primary scoring cases. Each item appears in five settings: T1 image-only free answer, T2 plus a cross-concept hint, T3 four-way candidates, T4 free answer with a structured explanation, T5 answer given, explanation only (unscored).
Ten multimodal models, five API and five local. The strongest closed model, GPT-5.5, reaches 50.7% primary accuracy; Kimi-K2.6 reaches 48.0%. The best open model, GLM-4.1V-9B, manages only 18.1%, far behind closed.
| Model | T1 image-only | T3 candidates | Primary |
| GPT-5.5 | 35.7 | 87.3 | 50.7 |
| Kimi-K2.6 | 32.6 | 85.5 | 48.0 |
| GLM-4.1V-9B (top open) | 5.0 | 53.4 | 18.1 |
The most informative finding is the gap between tasks. With four-way candidates (T3), accuracy jumps 17.3 to 56.0 points above the mean of the three open settings (T1, T2, T4). The visual clues are sufficient; models can recognize the answer inside a small candidate set. The failure is in open retrieval. The bridge hint (T2), which only lists possible relation types without naming the relevant clue, barely helps. On difficulty, slot count matters far more than bridge depth: the L1-to-L2 drop (adding a slot) is about three times the L2-to-L3 drop (deepening a path), and further depth adds almost nothing.
Of 221 items, 83 are solved by no model in any open setting; even with candidates, 6 defeat every model.
This is a rare benchmark where creativity becomes an exactly scorable task. The takeaway is useful for anyone evaluating or training MLLMs: current models read the clues in the picture fine, they just cannot push those clues back out into the open idiom space. That gap between seeing it and naming it is something you can measure and target separately. The chengyu is only the carrier; the encoding/decoding frame generalizes to any creative expression with a unique answer.
The carrier is tightly bound to Chinese idioms; cross-lingual and cross-cultural generalization is untested. Synthetic images come from a single generator (GPT Image 2), so visual style is narrow, and the complexity of real human-made figures is represented by only 37 items. The open-retrieval bottleneck conflates knowledge-retrieval failure with associative-reasoning failure, because T3 collapses both, and the paper does not separate them. At 221 items and 84 idioms the benchmark is small, and no confidence intervals are given for fine gaps between top models.