PCA vs EGA on LLM embeddings: EGA recovers 6-dim structure 97.6–100%, PCA nearly 0%

GolinoHudson · x · 2026-10-06

A research group benchmarked PCA vs Exploratory Graph Analysis (EGA) for estimating dimensional structure from LLM item embeddings in generative psychometrics.
Setup: 3 LLMs (GPT-4o among them), 2 embedding models (OpenAI text-embedding-3-small, Jina v3), 6 known personality dimensions, hundreds of Monte Carlo replications plus an empirical replication with the Multidimensional Schizotypy Scale.
Results: EGA recovered the correct 6-dimensional structure in 97.6–100% of conditions; PCA + Kaiser and PCA + parallel analysis achieved 0% exact recovery, frequently estimating 8–24 dimensions instead.
The authors attribute PCA's failure to a long tail of 'nuisance' dimensions from small item pools that cause eigenvalue-based methods to massively overextract.
Their AI-GENIE package shows redundancy and structural instability are distinct failure modes: Unique Variable Analysis removes redundant items, bootEGA flags items with unstable placement, and different LLMs showed different profiles on these failures.
On the real instrument, EGA came substantially closer to the human-response benchmark under both embedding models. Preprint: 'Estimating Dimensional Structure in Generative Psychometrics.'

More from Research