Pangram-4 Tech Report: Training a SOTA AI-Text Detector with Repeat2 Trick
RexDouglass · x · 2026-07-31
The Pangram-4 tech report details how to train a State-of-the-Art (SOTA) AI-text classifier.
- Backbone: Built on an undisclosed "popular open-weight MoE model", speculated to be at the scale of Mistral Small or Qwen3.5-122B-A10B.
- Fine-tuning Strategy: Uses LoRA to fine-tune select attention and dense layers, while fully training the classification heads. Interestingly, fine-tuning the routed experts tanked performance.
- Repeat2 Trick: Because causally masked models can only attend to leftward tokens, early predictions lack context. To solve this, the authors feed the input text twice (i.e., [text][text]) and score the predictions of the second copy.
More from Research
- GitHub Hit 'Data Structures in Practice': A Hardware-Aware Guide for System Engineers — tom_doerr · 2026-07-31
- Paper Reveals Deep Research Vulnerability: Misleading Info Triggers False Conclusions — Pengyu Zhu · 2026-07-31
- Study: Simple Image Transformations Easily Bypass Commercial AI Content Moderation — chaumian · 2026-07-31
- New Visual Pretraining Paradigm Beats Text with 1/4 Tokens — 量子位 · 2026-07-31
- Kimi K3 Architecture Explained: Building a 2.8T Parameter Open Model via 'Active Forgetting' — AccBalanced · 2026-07-31
- Review Paper on Reasoning Shortcuts in Neuro-Symbolic AI Published in JAIR — tetraduzione · 2026-07-31