Can a pipeline turn Cochrane's open data into research-ready meta-analytic datasets at scale?
An automated Python pipeline that harvests Cochrane systematic reviews into standardised, ready-to-pool effect-size tables — the open Pairwise70 corpus of 501 meta-analyses spanning 82,379 individual studies.
The extractor walks the Cochrane Library, downloads each review's published data package, parses the archived CSV tables into one standardised effect-size record per study, and appends study year, author, and DOI — with resumable checkpointing so an interrupted run continues without loss or duplication. The result is a single open corpus that the meta-research community can pool, stress-test, and benchmark against.
Harvesting converts Cochrane's open-data repository into a continuously updated resource for methods research.
Automated extraction · 100% pilot concordance vs manual
One pipeline, one standardised schema, and a corpus deep enough to benchmark meta-analysis methods on real Cochrane evidence.
Dataset files
595
.rda files in data/ (per-version variants)
Unique meta-analyses
501
from 521 reviews ingested
Individual studies
82,379
summed across all 501 datasets
Median studies / MA
71
largest single MA pools 3,665
Binary outcomes
496/501
491 odds ratios; 5 continuous / GIV
Median I²
21.4%
computed across 474 estimable MAs
Heterogeneity across the corpus
Most Cochrane meta-analyses cluster at low heterogeneity — 164 of 474 have I²=0 — but a long right tail of highly heterogeneous syntheses remains. Hover any point for exact counts.
Heterogeneity is summarised on the 474 of 501 datasets where τ² is estimable (k≥2 with non-degenerate cells). Median τ² (REML) = 0.069; median τ = 0.263 with IQR 0.588 and 90th percentile 0.858.
Depth of the corpus
The corpus is not a collection of toy examples: 196 meta-analyses pool more than 100 studies each, and the median synthesis has 71. Hover any bar for exact counts.
Validation & Robustness
Does the extraction hold up?
The following figures describe reported pilot extraction and pooling-engine comparisons. Their underlying execution records, source tables and validation outputs are not included in this repository, so these checks cannot be independently reproduced from its files.
11 pilot pairwise datasets1,295 rows hand-checked100% concordance (95% CI 97–100)100 analyses re-run vs metafor87 reviews covered
max |Δθ| = 6.2e-6, max |ΔSE| = 3.1e-5 over 100 analyses
HKSJ sensitivity
10.0%
85 of 850 analyses change significance under Hartung-Knapp
CIs that widen
55.1%
under HKSJ; median CI-width ratio 1.054 across 158 reviews
The Hartung-Knapp-Sidik-Jonkman check matters: re-running 850 analyses, 45 results that were "significant" under the standard random-effects model lose significance once the small-sample correction is applied (264 → 219). A corpus this size makes that fragility measurable rather than anecdotal.
Read Across The Series
Across The Series
Pairwise70 is the shared substrate for a family of Cochrane meta-research projects — each isolating a different question about how reliable the published evidence base really is.