Config-as-code AI factories: one factory learned another's benchmark scoring setup

vikvang1 · x · 2026-11-18

The author keeps his AI software factories fully configured as code, which lets one factory trivially adopt another's setup. While building benchmarks to test verification techniques on his mobile-factory, he didn't know how to score runs — so he simply tagged both factories in one Slack thread and had mobile-factory inspect and implement eng-factory-slim's existing scorer and LLM-as-a-judge configs as a starting point. A screenshot shows the two agents collaborating via PRs ($160.52 vs $0.58 runs), and one run even caught a harness bug where the iOS simulator crashed. Config-as-code turns factories into reusable knowledge for each other.

Original post →

More from coding & agent

coding & agent channel →