Skip the weights: building toy word embeddings from hand-crafted feature axes
cephaloform · x · 2026-09-16
An explainer thread on building toy word embeddings without training: instead of generating random weights, hand-pick a single feature axis — like "how active" or "how positive" a word is, or multiply the two — score a set of words yourself as the reference set, then compare the polynomial-derived features against it to find the closest matches, constrained by a Markov-chain-decided part of speech. Essentially a minimal demonstration of embeddings and POS tagging showing features can be designed by hand rather than learned.
Related event: Markov Chains Plus One-Dimensional Projections Enable Hand-Computable AI(3 posts)→
More from Research
- Periodic Labs' Neon: trillion-param science model beats GPT-6 Astra with 1,300 H200s — LiamFedus · 2026-09-16
- ARC-AGI-4 revives Jeff Clune's call: preserve diverse stepping stones, not one fixed target — jeffclune · 2026-09-16
- Odyssey-3 world model spans humanoids to drones; driving model trained on just 20h data — CyberRobooo · 2026-09-16
- Odyssey-3: one foundation world model to drive robots, cars and drones — rohanpaul_ai · 2026-09-16
- 1,300 H200s + automated labs: Fedus's Neon model beats GPT-6 Astra on materials benchmark — giffmana · 2026-09-16
- Math's loudest AI skeptic Daniel Litt now expects to lose his 2030 bet — ziv_ravid · 2026-09-16