Did the product get worse, or did acquisition change?
New-user retention fell as acquisition expanded. I investigated who was arriving before attributing the decline to the product.
- My role
- End-to-end diagnosis
- Original tools
- SQL · Excel · Tableau
- Deliverable
- Diagnosis & recommendations
The problem
Next-day retention was declining. Was this a product problem, a change in acquisition quality, or a wider disruption?
What I did
- Separated platform mix from changes within Android and iOS.
- Compared channels, acquisition volume, release timing, and the stages of the decline.
Both the user mix and retention within platforms changed.
Overall next-day retention fell from 22.20% to 19.45%. Android’s share of new users rose from 70% to 77%, while retention also declined within both platforms.
Account for the full 2.75-point decline
View data & definitions
Next-day returners divided by all new users in the registration cohort, with complete follow-up. Symmetric decomposition: mix changes use mean platform retention rates; within-platform changes use mean platform shares. This is an accounting allocation, not causal attribution.
| Component | Change (percentage points) |
|---|---|
| Platform mix | -1.015 |
| Android retention | -1.470 |
| iOS retention | -0.265 |
What this changed: The aggregate decline could not be read as a single product effect. I needed to investigate the acquisition mix and what changed inside Android.
More registrations came with weaker next-day return rates.
The historical investigation found weaker retention across the main paid app-store channels, while the cloud-drive referral channel stayed comparatively stable. Expansion timing and channel-level patterns made acquisition quality a stronger lead.
Acquisition volume and retention in one Android channel
View data & definitions
61 daily cohorts, November 1–December 31, with complete next-day follow-up. Next-day returners / registrations. Generic channel aliases are used. No regression effect or statistical significance is inferred from the reconstructed points.
| Registration date | New users | Next-day returners | Next-day retention |
|---|---|---|---|
| 2022-11-01 | 37 | 6 | 16.22% |
| 2022-11-02 | 43 | 7 | 16.28% |
| 2022-11-03 | 38 | 6 | 15.79% |
| 2022-11-04 | 36 | 6 | 16.67% |
| 2022-11-05 | 45 | 7 | 15.56% |
| 2022-11-06 | 48 | 8 | 16.67% |
| 2022-11-07 | 41 | 7 | 17.07% |
| 2022-11-08 | 43 | 7 | 16.28% |
| 2022-11-09 | 53 | 9 | 16.98% |
| 2022-11-10 | 52 | 8 | 15.38% |
| 2022-11-11 | 44 | 7 | 15.91% |
| 2022-11-12 | 51 | 8 | 15.69% |
| 2022-11-13 | 60 | 10 | 16.67% |
| 2022-11-14 | 54 | 8 | 14.81% |
| 2022-11-15 | 49 | 8 | 16.33% |
| 2022-11-16 | 60 | 9 | 15.00% |
| 2022-11-17 | 66 | 10 | 15.15% |
| 2022-11-18 | 56 | 10 | 17.86% |
| 2022-11-19 | 56 | 9 | 16.07% |
| 2022-11-20 | 69 | 11 | 15.94% |
| 2022-11-21 | 69 | 11 | 15.94% |
| 2022-11-22 | 58 | 9 | 15.52% |
| 2022-11-23 | 65 | 10 | 15.38% |
| 2022-11-24 | 77 | 13 | 16.88% |
| 2022-11-25 | 70 | 11 | 15.71% |
| 2022-11-26 | 62 | 9 | 14.52% |
| 2022-11-27 | 75 | 12 | 16.00% |
| 2022-11-28 | 83 | 13 | 15.66% |
| 2022-11-29 | 71 | 11 | 15.49% |
| 2022-11-30 | 69 | 12 | 17.39% |
| 2022-12-01 | 155 | 20 | 12.90% |
| 2022-12-02 | 174 | 22 | 12.64% |
| 2022-12-03 | 148 | 21 | 14.19% |
| 2022-12-04 | 133 | 18 | 13.53% |
| 2022-12-05 | 160 | 22 | 13.75% |
| 2022-12-06 | 165 | 23 | 13.94% |
| 2022-12-07 | 135 | 18 | 13.33% |
| 2022-12-08 | 135 | 18 | 13.33% |
| 2022-12-09 | 163 | 21 | 12.88% |
| 2022-12-10 | 153 | 19 | 12.42% |
| 2022-12-11 | 126 | 18 | 14.29% |
| 2022-12-12 | 140 | 19 | 13.57% |
| 2022-12-13 | 160 | 20 | 12.50% |
| 2022-12-14 | 139 | 19 | 13.67% |
| 2022-12-15 | 122 | 16 | 13.11% |
| 2022-12-16 | 145 | 19 | 13.10% |
| 2022-12-17 | 153 | 19 | 12.42% |
| 2022-12-18 | 126 | 18 | 14.29% |
| 2022-12-19 | 122 | 16 | 13.11% |
| 2022-12-20 | 147 | 19 | 12.93% |
| 2022-12-21 | 142 | 18 | 12.68% |
| 2022-12-22 | 116 | 15 | 12.93% |
| 2022-12-23 | 126 | 15 | 11.90% |
| 2022-12-24 | 146 | 18 | 12.33% |
| 2022-12-25 | 129 | 17 | 13.18% |
| 2022-12-26 | 111 | 14 | 12.61% |
| 2022-12-27 | 130 | 16 | 12.31% |
| 2022-12-28 | 140 | 17 | 12.14% |
| 2022-12-29 | 117 | 15 | 12.82% |
| 2022-12-30 | 110 | 14 | 12.73% |
| 2022-12-31 | 132 | 15 | 11.36% |
Compare all Android acquisition channels
| Channel | November new users | December new users | November retention | December retention |
|---|---|---|---|---|
| Cloud-drive referral | 6,000 | 7,400 | 17.00% | 17.00% |
| App store 1 | 2,000 | 4,800 | 20.00% | 17.00% |
| App store 2 | 1,700 | 4,300 | 16.00% | 13.00% |
| App store 3 | 1,500 | 3,400 | 19.00% | 16.00% |
| App store 4 | 1,200 | 2,300 | 19.00% | 16.00% |
| Other channels | 1,600 | 900 | 19.69% | 16.78% |
Recommendation: Evaluate acquisition on retained users and downstream value, alongside registration volume.
One explanation did not fit every stage of the decline.
Android weakened while iOS was initially steadier; later, both platforms declined. Releases overlapped acquisition changes, and regional activity added evidence of a wider disruption. Those overlaps limited a clean product-versus-channel attribution.
Separate platform-specific changes from shared movement
View data & definitions
Registration-date cohorts with complete next-day follow-up. The first six points use shorter available history. Overall retention is calculated from all returners divided by all registrations, not an unweighted mean of platform rates. This timeline is illustrative; it is not a release experiment or a causal estimate of external events.
| Registration date | Android pooled retention | iOS pooled retention | Overall pooled retention |
|---|---|---|---|
| 2022-11-01 | 18.62% | 32.16% | 23.18% |
| 2022-11-02 | 18.10% | 31.96% | 22.67% |
| 2022-11-03 | 17.97% | 31.94% | 22.67% |
| 2022-11-04 | 18.26% | 31.90% | 22.89% |
| 2022-11-05 | 18.20% | 31.96% | 22.74% |
| 2022-11-06 | 18.19% | 31.94% | 22.64% |
| 2022-11-07 | 18.23% | 31.96% | 22.73% |
| 2022-11-08 | 18.18% | 31.92% | 22.72% |
| 2022-11-09 | 18.27% | 31.95% | 22.72% |
| 2022-11-10 | 18.26% | 31.98% | 22.59% |
| 2022-11-11 | 18.12% | 32.00% | 22.47% |
| 2022-11-12 | 18.15% | 31.98% | 22.54% |
| 2022-11-13 | 18.11% | 32.02% | 22.51% |
| 2022-11-14 | 18.07% | 31.99% | 22.39% |
| 2022-11-15 | 18.05% | 31.98% | 22.31% |
| 2022-11-16 | 17.97% | 32.00% | 22.22% |
| 2022-11-17 | 17.96% | 32.02% | 22.21% |
| 2022-11-18 | 18.06% | 32.04% | 22.28% |
| 2022-11-19 | 18.03% | 32.06% | 22.26% |
| 2022-11-20 | 17.92% | 32.03% | 22.09% |
| 2022-11-21 | 17.80% | 32.04% | 21.91% |
| 2022-11-22 | 17.88% | 32.06% | 21.97% |
| 2022-11-23 | 17.96% | 32.06% | 22.09% |
| 2022-11-24 | 17.92% | 32.04% | 22.03% |
| 2022-11-25 | 17.79% | 32.02% | 21.81% |
| 2022-11-26 | 17.77% | 32.00% | 21.73% |
| 2022-11-27 | 17.92% | 32.03% | 21.87% |
| 2022-11-28 | 17.90% | 32.03% | 21.85% |
| 2022-11-29 | 17.78% | 32.04% | 21.75% |
| 2022-11-30 | 17.83% | 32.00% | 21.73% |
| 2022-12-01 | 17.57% | 32.13% | 21.44% |
| 2022-12-02 | 17.25% | 32.31% | 21.08% |
| 2022-12-03 | 17.14% | 32.43% | 20.92% |
| 2022-12-04 | 17.08% | 32.48% | 20.83% |
| 2022-12-05 | 16.92% | 32.59% | 20.59% |
| 2022-12-06 | 16.65% | 32.61% | 20.20% |
| 2022-12-07 | 16.51% | 32.63% | 20.05% |
| 2022-12-08 | 16.54% | 32.49% | 20.12% |
| 2022-12-09 | 16.52% | 32.33% | 20.06% |
| 2022-12-10 | 16.38% | 32.16% | 19.84% |
| 2022-12-11 | 16.31% | 32.01% | 19.76% |
| 2022-12-12 | 16.37% | 31.82% | 19.86% |
| 2022-12-13 | 16.45% | 31.67% | 19.93% |
| 2022-12-14 | 16.35% | 31.55% | 19.79% |
| 2022-12-15 | 16.31% | 31.41% | 19.70% |
| 2022-12-16 | 16.34% | 31.29% | 19.72% |
| 2022-12-17 | 16.25% | 31.15% | 19.65% |
| 2022-12-18 | 16.16% | 31.06% | 19.59% |
| 2022-12-19 | 16.06% | 30.94% | 19.53% |
| 2022-12-20 | 15.91% | 30.83% | 19.37% |
| 2022-12-21 | 15.78% | 30.74% | 19.19% |
| 2022-12-22 | 15.81% | 30.66% | 19.23% |
| 2022-12-23 | 15.75% | 30.52% | 19.25% |
| 2022-12-24 | 15.73% | 30.42% | 19.24% |
| 2022-12-25 | 15.65% | 30.24% | 19.07% |
| 2022-12-26 | 15.60% | 30.13% | 18.99% |
| 2022-12-27 | 15.65% | 30.02% | 19.06% |
| 2022-12-28 | 15.60% | 29.87% | 19.03% |
| 2022-12-29 | 15.41% | 29.69% | 18.86% |
| 2022-12-30 | 15.41% | 29.58% | 18.83% |
| 2022-12-31 | 15.34% | 29.40% | 18.72% |
Recommendation: Continue release-specific checks while tracking channels and platforms separately; timing alone is not enough to assign blame.
Investigate acquisition quality before treating the whole decline as a product failure.
What I delivered: A staged diagnosis that narrowed the acquisition, platform, release, and external explanations. The work supports recommendations, without claiming a measured retention recovery.
Methods, definitions & source notes
Based on my historical new-user retention investigation. Original analysis: SQL, Excel, and Tableau. This English presentation, exact decomposition, and Python/vector charts are portfolio reconstructions.
The cohorts contain 20,000 November registrants and 30,000 December registrants. Android channel rows partition Android users; platform rows partition each month. Rates are computed from integer returner counts.
| Month | Platform | New users | Next-day returners | Retention |
|---|---|---|---|---|
| November | Android | 14,000 | 2,520 | 18.00% |
| November | iOS | 6,000 | 1,920 | 32.00% |
| December | Android | 23,100 | 3,696 | 16.00% |
| December | iOS | 6,900 | 2,139 | 31.00% |
Releases, channel expansion, and external disruption overlapped. The original report’s correlations do not establish a controlled causal effect. The decomposition is newly calculated from the replacement figures, not a historical company attribution estimate.
Original reports, internal channel IDs, SQL, and customer records remain private.
Download chart data