Dev experiments with autoencoders over functions; low-dim search may be key to local FunSearch
cephaloform · x · 2026-10-24
Developer @cephaloform shared lessons from experiments with @VictorTaelin: training autoencoders over "functions" is painful because building the dataset without an agent in the loop is hard, and results were poor — he plans to rerun with a 3.8-27B model rewriting things.
He also notes the required batch size is the killer, and suspects local/consumer-GPU boosted FunSearch needs an audaciously low-dimensional search space — e.g. soft prompt tuning or a single LoRA on a single layer. The linked HF dataset crumb/functions-sample contains 10K–100K structured function text samples.
More from coding & agent
- Mubit trains proprietary models from your agent traces so you stop being an API tenant — SucceededMind · 2026-10-24
- Microsoft Open-Sources Frontier Tour Resource Hub with Code, Demos, and Labs — WirelessLife · 2026-10-24
- Rintawa: a solo dev's open-source Rust engine for persistent, simulated AI worlds — march-taylor · 2026-10-24
- Microsoft & HF turn database checks into RL rewards, lifting Qwen3.8-27B agents from 51.7% to 60.78% — SergioPaniego · 2026-10-24
- Stop asking agents for text PR summaries: use diagrams, screen recordings, and specs — _AustinCalvert_ · 2026-10-24
- Agent with SSH access successfully upgrades an ancient PHP 5.6 server end-to-end — DanielLockyer · 2026-10-24