TAPe+ML: A Compact Structured Representation for Multi-Task Computer Vision
Sergey Kurinov, Alexey Upatov
Comexp Research Lab, TAPe + ML Project, Nizhniy Novgorod, Russia
cs.CV, eess.IV
2026-09-15
TAPe+ML replaces pixels with proprietary T-bit relation graphs and a <100k-param core, reporting 88.1% ImageNet Top-1 and 65.3 COCO mAP50-95. The T-bit codec is not disclosed.
Production vision stacks still keep separate pipelines for classification, detection, and segmentation. YOLO and RF-DETR can do detection and segmentation in one network, but they still ingest pixel tensors, sit on large backbones, and train the two heads in parallel so neither task feeds the other. RF-DETR's backbone is even pre-trained on ImageNet just to make detection run.
A pixel pipeline is doing two jobs at once: recovering image structure from raw numbers, then solving the actual task. DINOv2 trained on 142 million images partly because pixels are unstable. A 30×50 RGB patch already has 256^4500 raw configurations under 8-bit encoding. TAPe+ML's bet is to write the image as a relational structure first, then let a tiny recognition core do the rest.
TAPe (Theory of Active Perception) maps an image to a set of T-bits plus edges among them. A T-bit is not a pixel. It is a subset of the most informative, interconnected visual elements, and there are far fewer of them than pixels. The same object under lighting changes, mild deformation, or noise is supposed to land on a similar T-bit set. The paper states input/output properties only. The construction algorithm is proprietary.
v3 is no longer a monolithic detector. A coordinator calls several small modules:
Trainable parameters stay under 100k. Classification switches from a learning-rate-sensitive gradient head to coordinate descent plus k-means, which the paper reports as about +3 points. New tasks can freeze the backbone and train a head, adapt the backbone, or train from scratch. After 10 to 15 labeled images the system can auto-label the rest. Segmentation is a three-stage contour pipeline: recover contextually meaningful edges, close them into polygons, then merge parts such as eyes or buttons into the parent object. Positive masks come from LVIS; negatives come from panoptic COCO. Backbone pre-training used about 5,000 images.
Numbers below are as reported. The paper notes that published detector baselines may differ in training and inference protocol.
| Task | TAPe+ML v3 | Reference |
| ImageNet-1k Top-1 | 88.1%, <100k params, trained from scratch | DINOv2 ViT-g/14 86.92% (1.1B); DINOv3 ViT-7B/16 88.4% (7B) |
| ImageNet-ReaL | 89.9% | ResNet-50 91.1%; DINOv3 90.4% |
| ImageNet-R / Sketch | 90.3% / 75.8% | ResNet-50 81.3% / 30.9%; DINOv3 91.1% / 71.3% |
| COCO box mAP50 / 50-95 | 84.7 / 65.3 | RF-DETR-2XL 78.5 / 60.1 (126.9M) |
| COCO Mask mAP50 / 50-95 | 80.7 / 58.4 | RF-DETR-Seg-2XL 73.1 / 49.9 |
| Imagenette, same 516k CNN | TAPe input 92% | raw pixels 47% |
For scene cuts on an Interstellar clip with the same HDBSCAN settings, TAPe indexed in 10–11 seconds, stored under 1 MB, and clustered in about 1 second. DINOv2 ViT-B/14 took about 5,220 seconds to index on 16 CPU cores, with a 9.7 MB index. In an industrial ore-blockage pilot with 30 images, YOLO26s reached 28%, TAPe head-only 45%, and backbone adaptation 65%; 85 images reached 85%; 500 images plus NMS reached 85.7–96%. Detection latency on a GTX 1070 Ti is 10.8 ms/frame; the full pipeline including masks is about 12 ms. On LVIS coverage after extra training, about 98.2% of objects received a meaningful mask, versus 44% for YOLO without LVIS training. An iBot-style SSL run stayed at loss 2.46 even at 120k pixel images, and reached 0.406 at 9k images once the input was TAPe.
If the numbers hold, the bottleneck in vision moves from scaling the network to writing a relational code before recognition. Edge devices and low-data industrial lines would feel that immediately: one core for classification, detection, and segmentation, with new classes starting from dozens of images as claimed.
They are not a reproducible baseline yet. The paper does not disclose how T-bits are computed from pixels. What is usable today is the problem framing: push structure out of network weights and into the input, then compose tiny specialized modules. Independent reruns under standard eval scripts are required before anyone should treat this as a drop-in replacement for YOLO or DINO.
The authors list tight boxes on small objects (the strict IoU band inside mAP50-95), COCO bias that forces backbone adaptation on industrial data, an undisclosed TAPe algorithm, and no 3D or tracking video experiments.
The scale of the claim is the larger issue. Under 100k parameters, trained from scratch, at 88.1% ImageNet Top-1 sits several orders of magnitude away from public tiny models. Some RF-DETR and YOLO latency numbers come from official T4 TensorRT tables, others from a 1070 Ti rerun. The 100k budget counts the coordinator and submodels, not the TAPe encoder's own compute graph. Ablations are incomplete because later modules inherit crops from earlier ones. Until the codec is public and the scores are reproduced with standard scripts, read every headline number as author-reported, not as confirmed SOTA.