Microsoft's TeleTune turns offline telemetry into agent skills, cutting tokens up to 75x
EliasEskin · x · 2026-10-14
Microsoft and UNC Chapel Hill introduce TeleTune: Evolving Agent Skills From Offline Telemetry, showing agents can learn useful skills from frozen logs people and agents already leave behind — no live environment needed during training.
Problem: user telemetry logs capture real software usage but (1) lack explicit task goals, (2) can't be replayed to test each skill update, and (3) interleave multiple tasks noisily.
Method: skill-guided progress — an offline metric using action-prediction errors on fixed logs; the system infers a goal per trajectory, proposes skill edits from prediction errors, and keeps only edits that improve the metric.
Results:
- +6.7% on WorkArena, +7.7% on Mind2Web (77.1% average success on WorkArena)
- 5–75x fewer optimization tokens vs. validating skill updates in live environments
- Skills transfer to unseen online tasks, beating baselines like AWWM (80.6% on Mind2Web)
More from coding & agent
- Intent ships PR review with inline diffs, Linear integration, board views and more — LukeW · 2026-10-14
- Intent ships big update: Home hub, inline-diff PR reviews, one-click Linear workspaces — LukeW · 2026-10-14
- Gemini 4 Argon ships in GitHub Copilot, beating prior Gemini models on every coding benchmark — rseroter · 2026-10-14
- Microsoft Copilot's real shift: from asking AI to delegating work to it — WirelessLife · 2026-10-14
- "Software is dead" debunked: vibe-coding on the fly costs tokens, compute and 15-minute waits — bendee983 · 2026-10-14
- gittifacts: an agent-native git platform built in a week on Cloudflare's challenge — threepointone · 2026-10-14