Cochrane Modern RE — Flip Atlas v0.1.0

Corpus: 6386 pairwise meta-analyses across 582 Cochrane systematic reviews (Pairwise70).

Methods compared: DL (baseline), REML_only, REML+HKSJ+PI, bayesmeta half-normal.

Headline significance-flip rate (DL → REML+HKSJ+PI): 8.2% (DL baseline vs REML+HKSJ+PI modern RE)

Tier 1 — Significance flips by stratum

CI excludes null under baseline but includes null under comparator (or vice versa).

reproducibility status outcome type k stratum n total n comparable n flips flip rate comparable flip rate total
non_reproducible GIV 10<=k<20 7 7 0 0.000 0.000
non_reproducible GIV 5<=k<10 7 7 1 0.143 0.143
non_reproducible GIV k<5 3 3 0 0.000 0.000
non_reproducible GIV k>=20 14 14 1 0.071 0.071
non_reproducible binary 10<=k<20 46 43 3 0.070 0.065
non_reproducible binary 5<=k<10 62 61 8 0.131 0.129
non_reproducible binary k<5 86 86 15 0.174 0.174
non_reproducible binary k>=20 58 57 1 0.018 0.017
non_reproducible continuous 10<=k<20 72 72 12 0.167 0.167
non_reproducible continuous 5<=k<10 110 110 24 0.218 0.218
non_reproducible continuous k<5 163 163 38 0.233 0.233
non_reproducible continuous k>=20 52 52 3 0.058 0.058
reproducible GIV 10<=k<20 16 16 4 0.250 0.250
reproducible GIV 5<=k<10 20 20 1 0.050 0.050
reproducible GIV k<5 27 27 7 0.259 0.259
reproducible GIV k>=20 7 7 2 0.286 0.286
reproducible binary 10<=k<20 1144 1111 50 0.045 0.044
reproducible binary 5<=k<10 1380 1365 87 0.064 0.063
reproducible binary k<5 1833 1833 192 0.105 0.105
reproducible binary k>=20 1060 1032 36 0.035 0.034
reproducible continuous 10<=k<20 38 38 4 0.105 0.105
reproducible continuous 5<=k<10 63 63 8 0.127 0.127
reproducible continuous k<5 103 103 15 0.146 0.146
reproducible continuous k>=20 15 15 2 0.133 0.133

Tier 2 — Direction flips by stratum

Sign of pooled point estimate changes (protective → harmful or vice versa).

reproducibility status outcome type k stratum n total n comparable n flips flip rate comparable flip rate total
non_reproducible GIV 10<=k<20 7 7 0 0.000 0.000
non_reproducible GIV 5<=k<10 7 7 0 0.000 0.000
non_reproducible GIV k<5 3 3 0 0.000 0.000
non_reproducible GIV k>=20 14 14 0 0.000 0.000
non_reproducible binary 10<=k<20 46 43 0 0.000 0.000
non_reproducible binary 5<=k<10 62 61 0 0.000 0.000
non_reproducible binary k<5 86 86 1 0.012 0.012
non_reproducible binary k>=20 58 57 0 0.000 0.000
non_reproducible continuous 10<=k<20 72 72 4 0.056 0.056
non_reproducible continuous 5<=k<10 110 110 1 0.009 0.009
non_reproducible continuous k<5 163 163 1 0.006 0.006
non_reproducible continuous k>=20 52 52 2 0.038 0.038
reproducible GIV 10<=k<20 16 16 0 0.000 0.000
reproducible GIV 5<=k<10 20 20 0 0.000 0.000
reproducible GIV k<5 27 27 0 0.000 0.000
reproducible GIV k>=20 7 7 0 0.000 0.000
reproducible binary 10<=k<20 1144 1111 7 0.006 0.006
reproducible binary 5<=k<10 1380 1365 9 0.007 0.007
reproducible binary k<5 1833 1833 5 0.003 0.003
reproducible binary k>=20 1060 1032 2 0.002 0.002
reproducible continuous 10<=k<20 38 38 1 0.026 0.026
reproducible continuous 5<=k<10 63 63 1 0.016 0.016
reproducible continuous k<5 103 103 2 0.019 0.019
reproducible continuous k>=20 15 15 0 0.000 0.000

Tier 3 — Clinically-important flips (MID-available subset)

|Δ pooled effect| exceeds the MID for that outcome. Reported only on the MID-available subset; NA for outcomes not in the MID lookup table.

reproducibility status outcome type k stratum n total n comparable n flips flip rate comparable flip rate total
non_reproducible GIV 10<=k<20 7 0 0 — 0.000
non_reproducible GIV 5<=k<10 7 1 0 0.000 0.000
non_reproducible GIV k<5 3 0 0 — 0.000
non_reproducible GIV k>=20 14 0 0 — 0.000
non_reproducible binary 10<=k<20 46 8 0 0.000 0.000
non_reproducible binary 5<=k<10 62 8 0 0.000 0.000
non_reproducible binary k<5 86 15 0 0.000 0.000
non_reproducible binary k>=20 58 4 0 0.000 0.000
non_reproducible continuous 10<=k<20 72 10 0 0.000 0.000
non_reproducible continuous 5<=k<10 110 6 1 0.167 0.009
non_reproducible continuous k<5 163 5 0 0.000 0.000
non_reproducible continuous k>=20 52 4 0 0.000 0.000
reproducible GIV 10<=k<20 16 0 0 — 0.000
reproducible GIV 5<=k<10 20 2 0 0.000 0.000
reproducible GIV k<5 27 4 0 0.000 0.000
reproducible GIV k>=20 7 0 0 — 0.000
reproducible binary 10<=k<20 1144 132 3 0.023 0.003
reproducible binary 5<=k<10 1380 153 13 0.085 0.009
reproducible binary k<5 1833 194 7 0.036 0.004
reproducible binary k>=20 1060 131 3 0.023 0.003
reproducible continuous 10<=k<20 38 1 0 0.000 0.000
reproducible continuous 5<=k<10 63 4 0 0.000 0.000
reproducible continuous k<5 103 5 0 0.000 0.000
reproducible continuous k>=20 15 1 0 0.000 0.000