abs() as activation function: a one-hidden-layer MLP just trains fine
burny_tech · x · 2026-09-27
A developer shared a counterintuitive experiment: using abs() instead of ReLU as the activation in a vanilla MLP with just one hidden layer (10 neurons) trains fine, with a video of the run.
The author claims abs() is faster than ReLU and more expressive, joking it could train in real time on an ESP32 — a low-cost exploration of an unconventional activation function that drew surprise from others in the AI community.
More from Fun
- BrushArena streams its RL training runs live, letting anyone watch paintings evolve — xeophon · 2026-09-27
- AI bots on X are now so convincing that smart people interact with them unironically — iScienceLuvr · 2026-09-27
- One-shot Opus 5.5 video nails hand-drawn whiteboard explainer of Leidenfrost effect — Scobleizer · 2026-09-27
- FlovaAI video turns Voldemort's forbidden magic into a Hogwarts cafeteria lunch fix — HeyNayeem · 2026-09-27
- Open-source GTA4 loading-screen template powers 'Grand Theft Alignment IV' AI-industry meme — jwt0625 · 2026-09-27
- AI nostalgia: generative AI was supposed to be surreal, not slop menus at the bodega — andrey_kurenkov · 2026-09-27