Self-Derived Activation-Steering Geometry Crystallized into Qwen2.5-7B Weights in 93 Seconds, +15.8%
Nearby_Indication474 · reddit · 2026-09-23
A Reddit user's AkbasCore 3.0 / SEASC experiment extracts a layerwise steering compass from Qwen2.5-7B-Instruct's own representations (no external teacher or dataset), drives activations along a controlled trajectory, then uses LoRA gradient updates to crystallize the geometry into the weights. After fully removing the steering motor and runtime adapter, a locked final test (excluded from training and checkpoint selection) still showed +15.79% on a single A100 in 93 seconds and 2.63 Wh. The system also auto-detected overshoot at update 70 and rolled back to the best state at update 40. The author stresses this is not 'beating LoRA' — LoRA is the pen, AkbasCore the motor, the model its own compass.
More from Research
- UCL study finds LLMs share a Fisher-Rao information geometry that enables low-distortion model interventions — UniversityCollegeLondon · 2026-09-23
- WFM trains on linked markdown wikis for agent memory, with a 10.5x faster GPU-to-GPU protocol — omarsar0 · 2026-09-23
- Oxford AI professor proposes desk-rejecting papers flagged by LLM watermark detection — TuhinChakr · 2026-09-23
- REAL: an embodied agent that explores real rooms and asks clarifying questions, accepted at ECCV 2026 — jiqizhixin · 2026-09-23
- OpenAI to publish 100+ solved open math problems; developer shares partial Erdős #993 result with Codex — ChrisGPT · 2026-09-23
- KVMem pages KV state to give agents million-token workspaces on a consumer GPU — rohanpaul_ai · 2026-09-23