NCME talk: generative psychometrics validates LLM-generated test items before human data
GolinoHudson · x · 2026-11-19
Hudson Golino (University of Virginia) will present on Nov 18 (4 pm ET) at NCME's SIGIMIE talk series a "generative psychometrics" framework that pairs LLM-based automatic item generation with computational structural-validity checks.
The workflow uses construct definitions to guide item generation, then feeds LLM embeddings into Exploratory Graph Analysis (EGA), unique variable analysis, and bootEGA to flag redundant and unstable items — providing an in-silico screening stage before collecting human response data. The talk also covers how model dependence, prompting technique, and construct definition affect results.
More from Research
- ELLIS Tübingen safety team releases horizon-labs-agent-traces dataset on Hugging Face — NielsRogge · 2026-11-19
- Noam Brown confirms the math model claim; speculation points to an astra math fine-tune — haider1 · 2026-11-19
- Researchers say peer review has hit a breaking point as AI conference scale strains ICLR — LucaAmb · 2026-11-19
- Shanghai AI Lab's GameRealm: 120M frames of gameplay data for visual foundation models — Shanghai-AI-Laboratory · 2026-11-19
- KradleBench launches as a dynamic multi-agent benchmark pitting LLMs, world models and humans against each other — lmoroney · 2026-11-19
- KradleBench Launches: Minecraft Legend SethBling Dethroned by Model Astra 6.2 — fchollet · 2026-11-19