Does compulsivity weaken planning? A replication with three extra tests
Gillan et al. (2016) found that people high in a compulsivity dimension plan less in a simple two-step decision task. I re-analysed their open data (1,961 people) with my own code and a plan written before any analysis, then tested it in a new sample, asked whether alcohol problems matter on their own, measured reliability, and added reaction times.
In the two-step task, a planner adjusts after rare outcomes; a habit-driven player just repeats what was rewarded. The compulsivity effect held under every check: separately written code, the paper's own mixed model (written from scratch, since Python has no lme4), a population-averaged model, and 192 new people tested in a lab.
| CIT effect on planning (log-odds per SD) | Estimate | Test |
|---|---|---|
| Published (Gillan 2016, Study 2) | −0.058 | z = −4.71 |
| My mixed model, same data | −0.0585 | z = −4.65 |
| Independent sample (Seow 2021, n = 192) | −0.079 | one-sided p = .018 |
| Pooled, both samples | −0.061 | z = −5.07 |
| Anxious-depression (control dimension) | +0.004 | p = .76 |
Alcohol problems (AUDIT) looked linked to less planning in one model (p = .036) but not in another (p = .098), and not at all in the new sample (p = .86). Adding compulsivity removed most of it, while compulsivity without the alcohol items still predicted planning in both samples. In the general population, it is compulsivity, not drinking as such, that tracks weaker planning.
Split-half reliability was 0.67–0.74 in both samples, so noise is not what hides the effect. The effect itself is small (r ≈ −0.1): about 1,160 people are needed to detect it, and task behaviour added only 0.004 to predicting a person's compulsivity score. It is a group-level mechanism, not an individual test.
People slow down after a surprising transition. More compulsive people slowed down less, and this replicated in both samples. The RT marker is only weakly related to the choice measure (r ≈ 0.25), so averaging the two gave the strongest link to compulsivity in both samples and roughly halves the sample needed.
I also built the principled version, a drift-diffusion model that explains each choice and its reaction time together, and pre-registered how to judge it. It was well identified but did worse than the simple average: it reproduced choices but only a third of the slowing (30 vs 99 ms), because a fixed extra delay is the wrong shape for that effect.
- A post-hoc finding (the strength of planning tracks compulsivity better than its share) did not replicate in the smaller sample, and Sharp et al. (2023) had already reported it.
- A transition-learning account could not be tested: its key parameter cannot be recovered from this task (r ≈ 0.3).
- The mixed model is my own implementation, not lme4; it reproduces the published numbers, but it is not the reference software.
- Studies 1 and 2 are re-analyses. The new sample is small (192), and its authors had already reported the direction, so the test was not blind.
- General-population samples: few people drink at clinical levels, so the alcohol result may not hold in treatment-seeking groups.