Hyperspherical Semantic Trajectory Analysis: Mapping Technological Diffusion across Academic Preprints, Patent Signals, and Compute Scaling
Muhammad Sukri Bin Ramli
cs.CL, cs.LG
2026-09-25
Hyperspherical clustering of 30K arXiv and USPTO texts finds LLM topics drift fastest (0.332); paper-volume velocity fails to Granger-predict frontier compute surges.
The standard proxy for technical progress in macroeconomics, Total Factor Productivity, is assembled from backward-looking output statistics and capital surveys, so a major breakthrough takes three to ten years to register in official numbers. Anyone wanting earlier signals turns to patents and papers, but classic bibliometrics leans on citation counts and taxonomy codes: citations need years to accumulate, and patent applications sit 18 to 36 months before entering public databases.
This paper skips the accounting layer and measures technology diffusion straight from raw text. It builds an unsupervised pipeline that turns arXiv preprints and USPTO patent applications into comparable time series, then tests one popular assumption along the way: whether publication activity predicts compute investment.
HSTA is a pipeline of off-the-shelf parts:
In the UMAP projection the 30,000 points split into two unsupervised islands, arXiv on the left and USPTO on the right: the embeddings separate academic and legal registers without any fine-tuning.
| Cluster | Drift Δ | Granger p | Offset (quarters) |
| Large Language Models (C6) | 0.332 | 0.81 | −18 |
| AI Systems (C2) | 0.234 | 0.76 | −22 |
| Computer Vision (C5) | 0.234 | 0.54 | −19 |
| Foundational Model Design (C1) | 0.174 | 0.28 | −13 |
| Neural Network Layers (C4) | 0.138 | 0.45 | −32 |
| Data Engineering (C7) | 0.031 | 0.14 | −16 |
| Hardware Architecture (C0) | 0.015 | 0.68 | −31 |
| Statistical ML (C3) | 0.012 | 0.33 | −10 |
The top-drifting LLM cluster comes with a clean qualitative check: before 2021 the dominant n-grams were masked language modeling, BERT fine-tuning, and contextual embeddings; after 2021 they were in-context learning, instruction tuning, RLHF, and prompt engineering. The drift tracks a real paradigm switch. Re-running with K = 6, 10, or 12 keeps both the two-island separation and the drift ordering stable.
The Granger tests fail across the board: p-values run from 0.14 to 0.81, none under 0.05, even as frontier training compute expanded from 10^16 to over 10^27 FLOPs over the same window. Paper-volume growth carries no predictive increment for that explosion.
The offsets are the counterintuitive part: in every cluster, patent peaks lead paper peaks by 10 to 32 quarters, roughly 2.5 to 8 years.
For anyone building innovation indicators, this is a cheap pipeline: sentence embeddings, clustering, and standard econometrics, all open-source, turn "how fast is a field's vocabulary churning" into a number trackable quarterly. The drift metric ports directly to monitoring your own corpus; a sudden rise in Δ for an internal topic most likely means the paradigm is shifting.
The negative result carries information too. "More papers means more compute coming" does not hold at quarterly frequency, so any text-based leading indicator of compute investment has to be conditioned on capital and hardware constraints. That is cold water on a common narrative.
Honest framing: every component is a standard tool, and the contribution lies in the combination and the cross-dataset linkage. This is incremental work.
The paper concedes two points: patents take 18 to 36 months from filing to publication, so the filing dates used for offsets are not on the same footing as arXiv indexing windows; and the 2026 volume spike is an artifact of dataset snapshot index bounds.
Negative offsets, with patents leading science by up to eight years, more likely reflect data conventions than commerce actually leading research. The paper itself scopes the finding to this specific streaming sample.
On the Granger side, ten years of quarterly data is roughly 40 observations, and a VAR(2) test there has low power. Failure to reject the null does not confirm the absence of predictive power, yet the paper's wording is "confirm." The 30,000 records are also a small slice of what these arXiv categories actually published over the decade, and the sampling rule is not described; silhouette 0.342 signals weak cluster structure to begin with; and drift values depend on the embedding model, with no comparison against LDA-style topic models and no significance tests on the drift differences.