HISA author reproduces Periodic Weak Spots: block indexer avoids CSA's phase sensitivity
青稞AI · wechat · 2026-10-22
The HISA author breaks down how hierarchical coarse-to-fine candidate selection speeds up the sparse-attention indexer while keeping token-level attention quality, and reproduces the Periodic Weak Spots experiments comparing DSA, CSA and HISA.
Key findings: CSA shows severe phase sensitivity (accuracy swings from 1.95% to 75.78% across training phases, a 73.83-point gap), while HISA stays as stable as DSA with higher average accuracy (50.05% vs 45.41%), pinpointing block core-attention—not block indexing—as the culprit. The piece also surveys HISA variants combining layer compression (DeepSeek-V4.1-flash's reindex), head compression (MISA) and query-block compression (PIVOT).
More from Research
- RFD4-Proteina's 'underrated' docs detail partial diffusion for protein redesign — AllThingsApx · 2026-10-23
- MPI-Sec PhD Thesis on Vulnerability Detection Benchmarks Includes Pre-LLM Classic "Top Score on the Wrong Exam" — mboehme_ · 2026-10-23
- LiveNerf's 29-day data finds no nerfing of Claude Opus 5.5 — TheOnlyVibemaster · 2026-10-23
- New paper makes 9 falsifiable predictions toward a major AI-enabled terror attack by 2027-2028 — StephenLCasper · 2026-10-23
- HumanoidPose recovers humanoid robots' full-body joint states from RGB video alone — cholz · 2026-10-23
- Sophont's Holon-S1 predicts genes and biomarkers from routine pathology slides — iScienceLuvr · 2026-10-23