Gemma 4 dev-agent comp locks everyone to one 31B model — is the code graph the intended edge?
politefella0 · reddit · 2026-09-26
The Gemma 4 dev-agent competition allows only gemma-4-31b-it-qat-w4a16-ct for every agent and subagent, with optional per-agent PEFT LoRA adapters, prompts, sub-agents, and sandboxed skills. Scoring is SWE-Bench-style PASS/FAIL within a 12-hour total budget including sandbox setup. Over 4,000 entrants and 600+ submissions are in; it closes Dec 2.
The author's key observation: the harness provides getcodeneighbors, searchsimilarcode (cosine over pre-computed embeddings), and getcodesubgraph, while the paper track explicitly calls out Graph Reasoning and Code Comprehension with a released graph+embedding dataset — strong hints that graph-based repo comprehension is the intended winning path.
Three concrete debates follow:
- With identical tools for everyone, does the edge come from the model using them well (→ post-training/LoRA) or agent orchestration (→ harness)?
- Is SWE-Bench PASS/FAIL a fair proxy for a reliable dev agent, or does it reward test-gaming?
- What's the realistic VRAM/latency budget for repo-sized context under a single base model and 12h cap?
More from coding & agent
- Dev: AI makes MREs trivial, so he opens PRs with repros instead of issues — remilouf · 2026-09-26
- Team withdraws ICLR paper after Claude Code's token-budget fix skewed their evals — najoungkim · 2026-09-26
- 60% of DeepSeek Harness users run at least one third-party plugin, team says — vista8 · 2026-09-26
- AI agent rejects a shortcut that would technically satisfy the spec but betray user intent — gregmushen · 2026-09-26
- Pebrel: an AI-native, GPU-accelerated open-source terminal for Windows hits 2.3k stars on GitHub — lxfater · 2026-09-26
- Dev builds MCP bridge plugin to let AI agents drive WordPress sites, seeks testers — BillGet · 2026-09-26