Computer Science Achievement and Writing Skills Predict Vibe Coding Proficiency
Sverrir Thorgeirsson, Theo B. Weidmann, Zhendong Su
CHI 2026
cs.HC, cs.SE
2026-03-15
In a preregistered ETH lab study (N=100), CS achievement predicted vibe coding at r=.39 and writing at r=.29; CS added about twice writing's unique variance in a joint model.
Tools like Replit, Lovable, and Cursor have made a language-first loop ordinary: describe a program in prose, watch it run, then iterate. Karpathy named the no-code variant vibe coding: you never inspect source, you only write prompts and judge behavior. Platform marketing often goes further. Replit has claimed that 75% of its users never write a line of code.
If that workflow is real, which skills actually predict who gets a working app? Clear writing, CS knowledge, or just being generally sharp? Prior studies were either tiny (N=19) or used single-turn, stateless prompting that does not match commercial multi-turn tools. This CHI 2026 paper from ETH Zürich runs a preregistered lab study that puts writing skill, CS achievement, domain-general reasoning, and vibe-coding performance on the same 100 students.
Students came from the ETH / University of Zurich volunteer pool. Eligibility: an intro CS course, prior LLM-for-programming experience, and English at C1 or above. One 1h45 lab session, paid 55 CHF. 56 women, 44 men, mean age 25.0. 51 in engineering or technology, 20 in the natural sciences, 11 in social sciences and the humanities. A priori power said N=85 was enough to detect r=0.30 at 80% power.
Four scores, all scaled to [0, 1]:
The custom platform hid all code. Chat on the left, live preview on the right, rollback, and a blurred token stream so people could see the agent working without reading source. Backend model: Claude Sonnet 4. Hiding code was the point: the construct is pure vibe coding, not prompt-then-patch. A blinded author scored each core feature on a 4-point rubric written before data collection. Instrument order and task order were counterbalanced. Starting with the essay did not lift vibe scores (0.446 vs 0.448).
Means: CS 0.53, ICAR16 0.56, writing 0.72, vibe 0.45. The decontextualized task was hardest (0.34); feature addition was easiest (0.55). In 297 of 300 submissions, students had clicked every instrumented feature in the sample app.
| Relationship | r | p |
| CS vs vibe | 0.386 | <0.001 |
| Writing vs vibe | 0.290 | 0.003 |
| Cognition vs vibe | 0.352 | <0.001 |
| CS vs writing | 0.126 | 0.213 (n.s.) |
After controlling for ICAR16, CS vs vibe stayed at partial r=0.281 (p=0.005). Writing dropped to 0.186 (p=0.066) and lost significance.
In hierarchical OLS, adding writing after CS moved R² from 0.150 to 0.208 (ΔR²=0.059). Adding CS after writing moved R² from 0.083 to 0.208 (ΔR²=0.125). CS contributed about twice the unique variance. Final-model betas: CS 0.356, writing 0.244, both significant. That joint model did not include ICAR16.
The preregistered claim that writing would matter more on the decontextualized task failed. Writing vs decontextualized r=0.239, slightly below feature addition (0.245); replication was 0.154 and not significant.
Exploratory: human-rated prompt quality correlated r=0.479 with vibe scores and r=0.353 with essay grades. Mediation: writing to prompt quality a=0.35, prompt quality to vibe b=0.43, indirect effect 0.152 (bootstrap 95% CI [0.061, 0.279]), about 52% of the writing-vibe link. The direct path was no longer significant. Prompt lexical diversity (MTLD) also tracked vibe scores, r=0.343.
Self-reported LLM-use frequency correlated negatively with vibe (r=-0.258) and with writing (r=-0.282), and was essentially zero with CS.
"You don't need to code to vibe" is too strong for this setting: small GUI apps, prompts only, code hidden. CS scores still predicted outcomes when nobody could read the generated source, which points at decomposition and mental models of state, not at patching. Writing helped on its own, but at about half the unique variance of CS, and the writing link was fragile once general reasoning was controlled.
For tools, prompt coaches that treat "write a clearer spec" as the bottleneck get some support from the mediation. For curricula, teaching prompt templates without CS fundamentals would skip the larger unique contributor.
This is correlation. It is evidence against yanking CS basics out of the syllabus. It is not a prescription that a writing class will make someone good at vibe coding.
All tasks were GUI miniapps. The only algorithmic probe was the meal planner's constraint logic (CS r=0.320, writing r=0.202). The writing instrument is new, with no large-scale norms. The lab was timed; some students were close to a working solution when the clock ran out. The sample is university students who already had intro CS and C1 English, not professional developers or true non-programmers.
Essay ICC(2,2) landed at 0.731, below the preregistered 0.75 target; the authors accepted a looser 0.7 cutoff. ICAR16 Cronbach's alpha was 0.625, so it stayed a covariate. Vibe scoring was done by one author-team member with no reported inter-rater reliability. The prompt-quality mediation is exploratory. The negative LLM-use correlation is a self-report Likert item in an exploratory analysis. Do not turn it into "AI makes people worse."