TTR-Compressor: test-time regression with closed-form solutions beats pooling for sequence compression
SphereLab · hf · 2026-10-23
Researchers from CUHK and Alibaba's Token Foundry propose Test-time Regression (TTR) as a new sequence compression paradigm beyond pooling and fixed projection: jointly fitting all tokens with closed-form solutions. In block-wise sparse attention, retrieval accuracy rises from 29.71 (pooling) to 50.92–51.44; in visual token merging, multimodal understanding scores consistently beat MLP-based merging. An open-source general-purpose compressor is available at spherelab.ai/ttr-compressor.
More from Infra
- Researcher: Putting a $200K GPU Server Purchase Order in a Paper Envelope — HildeKuehne · 2026-10-23
- Intel and AMD unite PyTorch CPU optimizations in oneDNN for a common x86 path — PyTorch · 2026-10-23
- Google and Meta open-source TorchTPU, a native PyTorch backend for TPUs — PyTorch · 2026-10-23
- Memory-bound at batch 1, compute-bound at batch N: how GPUs actually run LLMs — techNmak · 2026-10-23
- Intel demos AI Playground Home Agent: control local PyTorch compute from your phone via Slack — PyTorch · 2026-10-23
- TorchTPU and TorchNeuron Backends Land as PyTorch Becomes AI's Hardware Battleground — PyTorch · 2026-10-23