Omni-IO Skills: Harnessing Your Agent Omni-Native
Yanlin Li, Mingyang Hao, Shengqiong Wu, Hao Fei, Mong-Li Lee, Wynne Hsu
cs.CL
2026-09-26
NUS and Oxford's 27-Skill harness lifts GPT-5.6 Sol and Claude Sonnet 5 input coverage from 40% and 38.89% to 100% on UniM-90, with relative SQCS at 74.94 and 77.78.
General-purpose agents already plan, write code, and iterate. Their production surface still splits across text, images, audio, video, documents, 3D assets, and code. A course-production job makes the gap concrete: lecture video and reference docs go in; slides, figures, narration, and an explainer come out, all sharing facts, style, and timing.
Growing native modalities inside one foundation model ties every new codec and alignment stage to a model update. Parking specialist engines next to the host leaves the operational mess untouched: which capability to pick, how to declare dependencies, where intermediates live, how to contain a local failure, how to restyle last turn's landing page without redoing product analysis. Coding agents cover software work. Coordinated media packages still need an extra layer.
That layer is an Agent Harness, a runtime shell between reasoning and heterogeneous backends. The question is whether a plug-in shell can make an existing agent omni-native without touching its planning core.
Omni-IO Skills, from the National University of Singapore and the University of Oxford, leaves the host unchanged. The current build has 19 Atomic, 2 Expert, and 6 Scenario Skills (27 total) covering 38 representative tasks across understanding, generation, reasoning, and retrieval, plus seven artifact types.
Jobs compile to a Declare Execution Graph (DEG). Nodes carry type, a self-contained prompt, params, and dependson. The graph must be acyclic and every reference must resolve, or planning restarts. Independent ready nodes share a Wave (a concurrent batch of unblocked tasks) and run together: stills and sound effects in parallel, image-to-video waiting on the still. A failed node cancels descendants; other branches continue; finished outputs stay. On a later turn, a stored asset becomes an already-done assetsource, and only the edited step reruns.
A promotion request expands Event Material into poster and video Experts, then image understanding, image gen, video, SFX, and code. Four Waves: analyze product shots; generate visuals and audio in parallel; assemble poster and promo; build the landing page. Restyle the page later, and the registered product analysis is reused.
UniM-90 is a fixed 90-item slice of UniM across the seven modalities and interleaved mixes, sampled without looking at either host's native coverage. Hosts are GPT-5.6 Sol and Claude Sonnet 5. Each runs as the stock Base Agent (built-in Skills and tools, no extra third-party add-ons) versus the same setup with Omni-IO Skills and its MCP services loaded. Prompts and budgets stay matched. Gaps between the two hosts are not treated as a model ranking.
τ is the share of items whose full input set can be received and processed. Absolute scores use the supported subset; relative scores multiply by τ so unsupported items zero out on the full set. SQCS jointly grades semantics and generation quality, ICS grades interleaved coherence, StS / LeS grade strict structure and modality coverage. Ingest-but-cannot-emit failures remain in the metric.
| Setup | τ | Rel. SQCS | Rel. ICS | Abs. StS |
| GPT-5.6 Sol | 40.00% | 26.99 | 34.61 | 47.84 |
| Omni-IO | 100% | 74.94 | 93.98 | 100.00 |
| Claude Sonnet 5 | 38.89% | 27.82 | 32.00 | 52.21 |
| Omni-IO | 100% | 77.78 | 83.28 | 99.78 |
Relative SQCS rises 47.95 and 49.96 points. LeS hits 100.00 on both hosts. Absolute SQCS still moves 67.49 to 74.94 and 71.53 to 77.78 on the supported subset, so coverage is not the whole story. Claude's absolute ICS only goes 82.29 to 83.28. Near-perfect structure scores read as the shell enforcing tool calls and layout.
S4 Education Sharing plus A2/A3 understanding and A6 image generation turns a drawing video and its narration into an 11-image Korean night-market tutorial. From Owala FreeSip photos, A1 extracts attributes, E1 makes a 4:5 poster, E2 an exactly 10-second 9:16 MP4, A16 a self-contained landing page. Both hosts deliver all three with bottle identity held constant.
Teams already on a coding agent get a path that does not wait for the next unified Omni backbone. Procedures, providers, and assets can grow separately. The fit is production work with several dependent deliverables and later revisions: course kits, campaign packs, game-asset bundles.
This is harness-level composition, not a new perceiver or generator. Quality still sits on the specialist backends. The code is public. DEG plus a persistent registry is closer to a runnable multimodal production loop than another ad-hoc tool wrapper. The step is incremental.
The paper has no limitations section; the conclusion stays uniformly positive. Relative metrics are structurally inflated. Taking τ from about 40% to 100% explains most of the 48-point relative SQCS/ICS jumps: items the Base Agent could not ingest now run. Absolute SQCS gains are about 6 to 7 points. Claude's absolute ICS is essentially flat.
The only baseline is the same host without this harness. There is no head-to-head with MM-ReAct, stock Agent Skills, or other Omni coordinators. No ablation of DEG scheduling, the registry, or Scenario versus Atomic-only. No latency, dollar cost, or per-node failure rates. Structure scores at 100 are consistent with format control, not with better pixels or copy.
UniM-90 has 90 items, and the main text does not report the slice's task mix. The qualitative cases are generation-heavy. A frozen reasoning core cannot fix a bad plan. Failed nodes do not roll back finished siblings, so a half-built kit can remain. Omni-native here is the execution layer, not a shared model representation.