Paired t-intervals beat CLMMs for small-sample Likert data, says HCI researcher
IanArawjo · x · 2026-10-19
Ian Arawjo poses a concrete scenario: a within-subjects study with 12 participants, 3 conditions, and Likert-scale data — how should you report confidence intervals? His answer: skip 'proper' approaches like CLMMs, which print garbage at this sample size, and just use paired t-intervals.
A screenshot from his forthcoming textbook shows simulation evidence that parametric statistics are robust here: paired t-intervals are slightly conservative but hold up even on non-normal LLM evaluation outputs, especially when paired with visualization. Bayesian and Wilcoxon alternatives either add philosophical baggage or fail outright on small samples.
Related event: Paired t intervals beat CLMM for small-sample Likert data, researcher says(3 posts)→
More from Research
- 1-gram wireless mesoscope images cortex-wide activity in freely moving mice — KordingLab · 2026-10-23
- Reversible computing veteran poses open problem: universal BARCS primitives from unbiased Josephson junctions — MikePFrank · 2026-10-23
- Arc Institute uses genomic language model to map 376M prokaryotic noncoding sequences into 5.14M clusters — tedgreenwald · 2026-10-23
- The rise of AI-assisted peer review: reviewers are making it into the appendix — abursuc · 2026-10-23
- Anthropic uses Claude to formally verify security properties in 100+ open-source libraries at RISC-V assembly level — AnthropicAI · 2026-10-23
- Researchers test 9 strategies to pick LLM teams, beating random selection by up to 36.1% — shangbinfeng · 2026-10-23