Writing lint rules for agent code: the hard part was refusing to duplicate ruff and oxlint
Goldziher · reddit · 2026-08-31
The author maintains a linter that compiles ruff, oxc, biome, and mago as libraries and adds its own ast-grep rule pack targeting agent output failure modes (stubs that compile, tests that assert nothing, silently discarded errors, suppression comments without reason).
Key constraint: a rule that duplicates a tier-1 backend is refused. A per-language deferral table avoids re-reporting ruff's C901 or PLR0913, oxlint's max-depth, etc. This deletes most rules you'd want to write first, as existing tools already cover them.
Surviving rules (JS/TS): test body with no assertion, function whose whole body is throw new Error("Not implemented"), empty .catch(() => {}); Python: test function with no assert and no pytest.raises, and raise NotImplementedError as whole body outside abstract methods.
Defaults: 13 of 26 rules ship disabled, each with corpus measurements. For example, a rule catching Rust's todo!() reported 324 findings, 320 in generated FFI glue; of 4 in hand-written code, 3 were real stubs and 1 was a mock — a 25% false positive rate on a sample of 4 is not evidence for enabling. Another rule was correct on every finding but 94.4% in vendored FFI bindings and only 1% in first-party code; correct and actionable are different questions.
Pattern: most noise is path-shaped rather than pattern-shaped; AST patterns cannot express "not in generated output".
Conclusion: the rule is the cheap part; deciding whether to ship it requires measuring against a real corpus and being willing to conclude no.
Project: github.com/Goldziher/poly (Rust, MIT)
More from coding & agent
- Warning: AI coding agents install unauthorized code in corporate networks — GaryMarcus · 2026-09-01
- Intent open-sourced: Large-scale agent coordination keeping agents on track — LukeW · 2026-09-01
- Agents Develop 'Classical Chinese' to Compress Long-Chain Task Context — wzenus · 2026-09-01
- Debugging multi-step LLM apps quickly turns into detective work — Sensitive-Parsnip-12 · 2026-08-31
- Tested 3 Claude Code plugins to reduce costs: Here is what worked — Marmelab · 2026-08-31
- Lanyon AI generates 32k lines of formally verified MHD solver code — jfischoff · 2026-08-31