SemaPLC: A Project-Grounded, Verification-Gated Agent Harness for PLC Code Generation
Yanlun Tu, Huacan Wang, Ziyue Zhou, Jie Zhou, Ningyan Zhu, Ge Chen, Wangyi Chen, Tengfei Zhou, Yifan Zhou, Dasheng Yang, Xiaofeng Mou, Hui Zhang, Yi Xu
cs.SE
2026-08-19
SemaPLC stops only when spec, compile, and runtime checks are logged. Function-track mean pass is 72.6%; project dynamic score is 52.2 versus at most 31.4.
PLCs run plants. They are mostly programmed in IEC 61131-3 Structured Text. Large language models can already emit standalone program organization units (POUs), with compiler-in-the-loop repair, model checking, and multi-agent iteration. Production logic is rarely an isolated POU. New code has to sit inside an existing project, reuse function blocks and variables, and then behave correctly at runtime. A program can compile and pass static checks and still misconfigure a timer, take a wrong transition, or drop an interlock.
Prior systems execute generated code to show that it can run, not to measure how reliably it runs at benchmark scale. SemaPLC splits project grounding from runtime behavior, and it binds the agent to a hard completion rule: the model may not stop because it likes its own output. Logged external checks have to pass.
The harness sits on a generic tool-use core. PLC details go through one MCP tool layer: syntax check, compile, deploy, force variables, sample traces. Three design rules operate together.
Project grounding retrieves structure from the existing project, reuses interfaces and function blocks, and edits a bounded scope instead of rewriting the plant. Multi-source verification covers a clause-by-clause spec audit, compiler diagnostics, and live runtime scenarios compared against traces. The verification gate allows at most two repairs per check; any edit voids every prior verdict; a claimed pass that is missing from the tool log is downgraded to unchecked.
Two tracks. The function track has 117 independent POUs from Agents4PLC; engineers repaired 43 tasks with defective oracles. A held-out formal judge requires at least 80% of properties to be shown satisfied; inconclusive and timeouts count as failure. The project-context track has 65 tasks over ten Spec2Control plants; generated logic must compile and run inside the full ST project. The dynamic score deploys the candidate and a hidden reference to the same runtime, with up to six scenarios, and compares traces on core output ports.
All methods use the same seven backbones, from MiniMax, Qwen, DeepSeek, and GLM through GPT-5.5. Baselines are LLM4PLC, AutoPLC, Agents4PLC, plus a bare ablation that strips skills and tools.
On the function track SemaPLC has the highest strict verified pass rate on every model, 72.6% mean, 8.8 points above Agents4PLC at 63.9%. On GPT-5.5 the scores are 82.1% versus 79.5%. The weakest SemaPLC model still scores 67.5%, above every baseline mean. Against bare, every model gains 8.5 to 33.3 points, with the weakest gaining most. Cross-model spread shrinks from 37.6 points to 14.6, and mean compile rate rises from 85.5% to 99.2%.
| Setting | Function verified pass | Project compile | Static | Dynamic |
| Strongest baseline | 63.9 (Agents4PLC) | 58.7–81.5 | 71.7–75.7 | ≤31.4 |
| SemaPLC | 72.6 | 89.4 | 81.6 | 52.2 |
Static scores sit within about 10 points of each other; dynamic scores reorder the methods. On GPT-5.5 the dynamic lead shrinks to 1.8 points (65.4 vs 63.6), and SemaPLC trails on static. Layer ablation on DeepSeek-V4-Flash moves dynamic from 23.1 (generate only) to 30.3, 43.7, then 54.1 as spec, compile, and runtime are added. Cost climbs from 34k tokens / 8.9 requests per task to 129k / 47.8. The formal pipeline returns a conclusive verdict on 0 of 174 properties across 32 timer-bearing programs, which is the hole runtime validation is meant to fill.
Project-track requests average 34.1 versus 6.9 for Agents4PLC. Function-track request counts are similar (6.5 vs 6.3) and wall-clock is shorter (71s vs 454s), because the baseline runs model checking on every iteration.
For industrial codegen, stopping at compile and static assertions cannot tell methods apart. Execution does. The tools are conventional; the completion discipline is the actual contribution. Gains are larger on weaker models, so the harness behaves like a model-agnostic reliability layer. Code is open. Copilots that look strong on static scores will be overrated in integration settings.
Dynamic scoring only covers a bounded scenario set from the hidden reference; unseen conditions are unmeasured. The lead shrinks on the strongest model, and SemaPLC loses on static with GPT-5.5, so the gate supplies reliability the model lacks rather than a fixed margin. The function-track oracle repaired 43 of 117 tasks, so numbers are not directly comparable to the original Agents4PLC paper. The agent derives its own injection scenarios from the spec while scoring scenarios come from the hidden reference; they share a requirement, which may still leak patterns. Project-track wall-clock is not cheap. Reliability is bought with extra model calls.