Deep Dream and SAE Feature Maximization in Modern LLMs
torchcompiled · x · 2026-07-20
A reply to @matthen2's discussion on the Deep Dream effect in modern large models. The original post demonstrated optimizing an image to maximize the probability of a target caption. It found that Gemma 12B, which lacks a vision encoder, reads pixels just like processing token embeddings, printing recognizable objects on the canvas. Conversely, E4B, which features a vision encoder, tends to lean towards texture drift.
The responder noted that maximizing optimization using individual features identified by Sparse Autoencoders (SAE) might yield even more interesting results.
Related event: Reviving Deep Dream with modern LLMs(2 posts)→
More from Research
- 3D ResNet Paper Crosses 3,000 Citations Eight Years After CVPR 2018 — HirokatuKataoka · 2026-09-11
- Jeff Heaton's Intro to the Math of Neural Networks eBook Is Free to Download — blaizedsouza · 2026-09-11
- Mathematician Daniel Litt Launches Problem Repo to Track Human vs AI Progress: 15 Problems, 1 Solved — littmath · 2026-09-11
- Open ECDSA.fail challenge uses AI agents to shrink Shor's-algorithm quantum circuits for Bitcoin keys — StefanoGogioso · 2026-09-11
- Alex Townsend posts 200 open problems in numerical linear algebra for humans and AI agents — IgorCarron · 2026-09-11
- Navier-Stokes, Riemann, P vs NP: what this week's math buzzwords mean for you — koltregaskes · 2026-09-11