CaRGo-T: Causal Reasoning Graph-of-Thought improves Multimodal Humor Comprehension
Abhilash Nandy, Rahul Seetharaman, Aman Bansal, Rounak Saha, Manav Nitin Kapadnis, Millon Madhur Das, Pawan Goyal, Niloy Ganguly
cs.CL, cs.CV
2026-08-24
CaRGo-T has a VLM emit a code-form causal graph before answering. Understanding rises about 1–20% over reasoning baselines; detection gains are 1–3%.
Satire images, memes, and multimodal sarcasm live in mismatches among entities, events, and implied relations. A linear chain of thought often flattens that into a literal caption. Prior evals show strong VLMs still misread affect, miss object-action conflicts, and collapse figurative scenes to one reading. CoT in subjective, emotional items falls back on priors; self-reflection writes rationales that do not line up with the pixels. The missing piece is an intermediate form that states who caused what, then answers from that structure.
CaRGo-T (Causal Reasoning Graph-of-Thought) asks the model to emit a code-like causal reasoning graph, then the final answer. The graph is a thin subclass of a causal graph: cause-effect edges plus light attributes for objects, concepts, events, and participants, with no probabilities. Example: wearing heels to look fashionable causes foot pain. The prompt splits the job into Code, then Final Answer.
Zero-shot asks for the graph directly. For ICL, GPT-4o drafts a graph conditioned on the gold answer, and humans rewrite it into a fixed shape: entities and properties, then cause-effect pairs. At test time, K rewritten examples go into the prompt. No training, inference only. Baselines are Vanilla, CoT, CoD (Chain-of-Draft), and CCoT (scene graph in the prompt). Understanding uses BLEU, ROUGE-L, BERTScore, and their mean; detection uses accuracy and macro-F1. Backbones are GPT-4o, GPT-4o-mini, and MiniCPM-V-2.6. Data: 1,079 YesBut satire-understanding images, 559 MemeCap items, 2,541 YesBut detection images, 2,409 MMSD 2.0 sarcasm items.
CaRGo-T is often best within a backbone on understanding. Absolute scores stay low, and the headline lift depends on the comparator.
| Setting | Vanilla Avg | CoT Avg | CaRGo-T Avg |
| MiniCPM 0-shot satire | 0.3455 | 0.3457 | 0.3504 |
| GPT-4o-mini 0-shot satire | 0.3372 | 0.3043 | 0.3632 |
| GPT-4o 0-shot satire | 0.3571 | 0.3337 | 0.3726 |
| GPT-4o 2-shot satire | 0.3602 | 0.3551 | 0.3911 |
| GPT-4o 5-shot MemeCap | 0.3266 | 0.3328 | 0.3461 |
Versus CoT, GPT-4o-mini’s zero-shot satire mean is close to the paper’s 20% upper bound; versus Vanilla the gap is a few points. Extra shots diminish: GPT-4o’s lift over CoT shrinks from 11.66% at 0-shot to 5.86% at 5-shot. Detection is thinner. GPT-4o on MMSD 2.0 reaches 49.48% accuracy (CoT 48.07%); YesBut satire detection is 43.18% (CoT 42.7%), with F1 up by fractions of a point. Binary accuracy stuck near 50% means this task is still hard for current VLMs.
An information study on MiniCPM zero-shot satire shows slightly higher or equal KL from CaRGo-T token distributions versus CoT/CoD, and higher sentence-coverage LSF. GPT-4 as judge can infer the gold answer from the rationale on 45.11% of CaRGo-T traces, versus 40.78% for CoT and 37.64% for CCoT.
This is a prompting-format increment, not humor comprehension solved. Serializing a causal graph as code forces named entities and explicit links, which helps GPT-4o-mini more than GPT-4o: the smaller model writes code well and holds less parametric knowledge. It is a no-train skeleton worth trying for meme or satire understanding. It is not a detection accuracy fix.
There is no Limitations section. Understanding scores rest on BLEU/ROUGE/BERTScore; BERTScore sits at 0.85–0.88 for any fluent English, so most of the signal is tiny lexical overlap. Detection accuracy of 43%–50% makes 1%–3% gains fragile. ICL graphs need human rewriting, which does not scale. Zero-shot mostly uses closed models because open VLMs are weaker at code. There is no human preference study of whether the explanations actually land the joke. The graph is produced by the same VLM, not identified in the causal-discovery sense; “causal” is mostly the shape of the diagram.