SPK: Eliciting Structured Prior Knowledge for Interpretable Out-of-Distribution Detection in Real-Time Object Detection
Changshun Wu, Weicheng He, Xiaowei Huang, Saddek Bensalem
cs.CV, cs.LG
2026-08-20
SPK elicits a 5-D prior from a frozen detector; iForest drops YOLO+BDD Far FPR95 from 65.23 to 0.70 and beats fine-tuning Proximal-OoD on most cells.
Object detectors train on a closed category set, then in the open world they draw high-confidence boxes on things they never saw. Those errors are OoD hallucinations: nearby, lookalikes of known classes; farther out, background texture that lights up objectness. Current fixes run two ways. Score functions on the detector's high-dimensional features after the fact, or fine-tune the detector itself to suppress the boxes. Both still spin on an opaque representation. The priors the detector already learned stay unread.
SPK asks a different question: what priors inside the detector already say whether this prediction is trustworthy? Elicit them, pack them into a short interpretable vector, then reject.
Detector weights stay frozen. Hallucination samples are diagnostic supervision, not outlier-exposure reject labels, and they are not used to adapt the detector.
Semantic prior: a light four-layer residual conv head per ID class decodes that box's RoI feature into part-level concept activations. The concept vocabulary has three sources, ID parts, proximal-OoD parts, and background concepts. Proximal classes are mined from Objects365 with GPT-5 prompts over VOC/BDD labels; background comes from DTD after filtering. Part masks are auto-annotated, then lightly checked by hand. Training uses three losses: Dice on concept masks, suppression of channels that should stay dark, and group cross-entropy over {id, prox, bg}. Each group keeps its max channel, giving a three-way semantic response.
The geometric prior is the box area over the image area. The contextual prior treats the whole image as an RoI, builds an image embedding from channel-wise mean and std of the detector neck's multi-scale maps, then takes KNN distance to a class-conditioned bank of ID training images.
The five-D vector is \([s^{id}, s^{prox}, s^{bg}, r^{geo}, d^{ctx}]\). Downstream heads can be MDS, BAM, KNN, or Isolation Forest; the paper leads with iForest.
The calibrated protocol covers YOLO, Faster R-CNN, and RT-DETR, ID sets PASCAL-VOC and BDD-100K, Near-OoD and Far-OoD. Metrics are FPR95 (false positive rate at 95% ID recall, lower is better) and raw hallucination counts.
Swap the same iForest onto the SPK vector and FPR95 drops hard. YOLO on VOC Near/Far: 70.27 / 67.82 down to 14.25 / 11.84. YOLO on BDD Far: 65.23 down to 0.70. Faster R-CNN on BDD Near/Far: 63.25 / 62.78 down to 2.31 / 1.52.
Hallucination counts against Proximal-OoD, which does fine-tune the detector:
| Detector / data | Original | Proximal-OoD | SPK |
| YOLO VOC Near/Far | 946 / 440 | 134 / 60 | 135 / 52 |
| YOLO BDD Near/Far | 701 / 666 | 80 / 47 | 69 / 5 |
| Faster R-CNN VOC | 2150 / 1335 | 710 / 253 | 299 / 140 |
| Faster R-CNN BDD | 2576 / 1634 | 207 / 167 | 60 / 25 |
| RT-DETR VOC | 2311 / 1589 | 386 / 470 | 358 / 275 |
| RT-DETR BDD | 3145 / 1220 | 525 / 240 | 359 / 113 |
On YOLO VOC Near, SPK and Proximal-OoD are a tie (135 vs 134). The gap opens on Faster R-CNN and on BDD Far. Ablation: all three semantic losses, YOLO four-cell mean FPR95 is 9.16; drop Dice and it rises to 20.80. Semantics alone 21.17, plus geometry 12.36, plus context 9.16. All semantic heads for 10 BDD classes take about an hour on one A100 40GB.
The detector does not need a fine-tune, and rejection still beats Proximal-OoD's "fine-tune plus KNN" recipe. For a frozen YOLO or DETR already in production that is usable: add a five-D monitor, leave original weights alone. The vector is also readable. A weak id group and a strong prox or bg group is closer to incident analysis than a 256-D feature. The paper's own claim is blunt: get the representation right before inventing another scoring function.
There is no standalone limitations section. The authors note they cover only two hallucination sources, proximal objects and background, and that stopping generation is future work. Proximal mining is tied to GPT-5 plus Objects365; a new domain means a new mine. Part annotation still needs a human pass, and concept-vocab quality is not scored on its own. Geometry is only relative area, coarse. YOLO VOC Near count does not beat Proximal-OoD, so SOTA is not every cell. SPK is post-hoc rejection; the detector still draws the box first. Several older methods are compared only on the uncalibrated protocol.