LitPivot: Developing Well-Situated Research Ideas Through Dynamic Contextualization and Critique within the Literature Landscape
Hita Kambhamettu, Bhavana Dalvi Mishra, Andrew Head, Jonathan Bragg, Aakanksha Naik, Joseph Chee Chang, Pao Siangliulue
cs.HC, cs.AI
2026-04-03
LitPivot clusters papers around one facet of a draft and suggests rewrites. Expert grounding rose from 2.59 to 5.18 versus chat-with-papers, in a 17-person study.
A research idea has to connect to prior work and still stand apart from it. Change the idea, and the relevant papers change too. Literature tools organize a fixed pile of papers. Ideation tools score an idea against papers the user gathered beforehand. Reading that forces a rewrite, and a rewrite that forces a new reading list, is a loop those tools leave alone.
Ai2 and the University of Pennsylvania call that loop a literature-initiated pivot. Meeting notes from four finished CS projects at the authors' institution, two in HCI and two in NLP, mentioned literature in 7.5 meetings per project on average (SD 2.65) and contained 21 pivots: 9 to create distance from nearby work, 12 to line up with an existing practice. In a formative study, ordinary search led 2 of 5 researchers to change their idea. A probe that gave literature-grounded feedback led all 5 to change it. Drafting and checking novelty compete for the same stretch of attention. When the literature already covers the idea, people want a concrete revision of the current topic, not a replacement topic from the system.
LitPivot puts an editor on the left and a paper panel on the right. The draft is split into problem, contribution, and evaluation. Click one facet and the papers regroup around that span. The closest clusters are starred. The researcher picks a subset and gets a comparison plus a rewrite.
Retrieval starts with Asta Paper Finder, Ai2's open scientific search. Full text comes from Semantic Scholar. The pipeline extracts limitations, future work, contributions, motivations, evaluations, and methods, then adds papers cited in those sections when they are not already in the set. Sonnet-4 clusters. GPT-4o segments the draft and extracts facets. o3 builds the graph, writes the assessment, and drafts rewrites. Temperature is 0.7 on all three. The slower reasoning model is kept for the comparison steps.
Novelty is a bipartite graph. One side is contributions from the selected papers. The other is limitations and future-work statements those papers wrote down. An edge means a contribution already addresses that limitation. A new contribution that links only to handled items is marked incremental, a replication, or an extension. A link to something nobody has taken is marked conceptually new. A problem statement has to match at least one citable claim in the selected papers. An evaluation is checked for alignment with the problem, for whether it can be carried out, and for whether the measure is sensitive enough. After one facet changes, mismatched facets are flagged and missing core facets are requested. Suggestions can be accepted or rejected. The two checks stay separate because the formative study saw both needs: aligning a problem or an evaluation with the literature, and pulling a contribution away from its neighbors.
The pipeline was compared with IdeaSynth. Two experts, five ideas, 15 facets, same corpus and model, blind scores from 1 to 7. Rewrite suggestions moved from 3.40 to 5.27 and from 3.73 to 5.73. Pooled across 30 paired judgments, the mean gain was 1.93 points (dz=1.04). Critique quality gained 1.37 points pooled (dz=0.58). One rater's move from 3.60 to 4.40 was not significant (p=.15). Inter-rater alpha ran from -0.33 to 0.11.
Seventeen researchers in computer and information science ran a counterbalanced within-subjects study. 82% were PhD students. Each condition lasted 30 minutes and used an assigned seed idea. The baseline was retrieval-augmented question answering over the same set of about 100 papers, with no facets and no automatic critique.
| Measure | LitPivot | Baseline |
| Distinct papers selected, median | 7 | 3 |
| Literature understanding, 1-7 | 5.71 | 3.88 |
| Change in perceived novelty | +2.29 | +0.47 |
| Change in perceived utility | +0.53 | -0.76 |
| Change in perceived feasibility | +0.88 | -0.35, not significant after correction |
| Expert grounding, mean (median) | 5.18 (5) | 2.59 (2) |
| Helpfulness of outputs | 6.24 | 4.12 |
| Trust in outputs | 6.00 | 3.88 |
Experts scored one item: whether the idea was well argued from relevant literature. The median moved from 2 to 5. The mean difference was 2.59 (95% CI [1.56, 3.62], dz=1.29). Feasibility was not significant after correction (p=.152). Of the suggestions, 50% were kept as written, 45.6% were edited and then kept, and 4.4% were dropped. Thirteen people moved between facets flagged as out of sync. Four finished a facet before opening the next. Eleven pruned papers out of starred clusters. Six selected more papers on later passes.
Five of those participants then spent 30 minutes on their own projects. Two were testing a new idea, three were positioning a more mature one, and four said they would use the system again.
The increment is in the interaction, not in a new model. Scideator retrieved about 10 papers per idea. Users said it called ideas novel too often and missed important related work. IdeaSynth left most curation to the user and found no significant gain in confidence about the literature. On one shared corpus, against ordinary chat over papers, literature understanding moved with dz=1.42 and expert grounding with dz=1.29.
A half-formed idea can become a revision pass on whether it has already been done. Almost half the suggestions were edited before adoption, so the choice stayed with the person. Problem, contribution, and evaluation are cut the way empirical HCI and NLP papers are cut. Another field would need a different cut.
The evaluation stays inside HCI and NLP, with one person and a time limit. The main study assigned seed ideas in order to control how mature the project was. Perceived feasibility did not separate the conditions. The paper guesses that the topics were not the participants' own. The system measures grounding and differentiation only. How important the problem is, whether the idea is interesting, and whether it can be executed are not measured, and no experiment is run. The graph defines novelty as covering a gap the literature has already written down. Novelty from asking a different question, and papers that never state limitations, sit outside that graph. Paradigm-level ideas often match no stated limitation, so this check has no novelty to assign them.
Alpha on the technical evaluation was negative. The two experts did not share a significant preference on critique quality. Grounding scores were given by the last two authors of the paper, blind to condition and still internal. The rubric took three rounds to move alpha from 0.389 to 0.804, and the remaining disagreements were resolved by hand. About 100 papers were fixed in advance, and participants could not add any during the study. The open session was five people. The mechanism also pulls ideas toward the center of the literature, which can crowd out a direction that is not ready to cite. Half the suggestions were edited or rejected.