IPD Course > The Vanishing
Have you not seen how a thousand patients
can be reduced to a single number,
and in that number, lives are erased?
can be reduced to a single number,
and in that number, lives are erased?
The Aggregate Illusion
EVERY META-ANALYSIS YOU'VE EVER READ
A meta-analysis reports: "Treatment reduces mortality by 15%."
But which patients benefited? The young or the old? Those with mild disease or severe? Men or women?
The aggregate cannot answer.
For within the average, some patients were saved—and some were harmed.
But which patients benefited? The young or the old? Those with mild disease or severe? Men or women?
The aggregate cannot answer.
For within the average, some patients were saved—and some were harmed.
What Aggregates Hide
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● Responders ● Non-responders
THE TRAGEDY
"Overall benefit: 30%"
But who are the 30%? Without individual data, we cannot identify the responders from the non-responders. We cannot practice precision medicine.
But who are the 30%? Without individual data, we cannot identify the responders from the non-responders. We cannot practice precision medicine.
Oxford, 1985
EARLY BREAST CANCER TRIALISTS' COLLABORATIVE GROUP
Richard Peto gathered something unprecedented: the raw data
on every woman in every tamoxifen trial.
Not summaries. Not averages. Each patient, each tumor, each outcome.
What he discovered changed breast cancer treatment forever.
Not summaries. Not averages. Each patient, each tumor, each outcome.
What he discovered changed breast cancer treatment forever.
Early Breast Cancer Trialists' Collaborative Group. Lancet 1988;2:61-72
The published trials said:
"Tamoxifen works."
But the individual data revealed:
"Tamoxifen works."
But the individual data revealed:
ER+ vs ER−
Tamoxifen only worked in estrogen receptor positive tumors
WHAT WAS HIDDEN
No single trial was large enough to detect this interaction.
Only by combining individual patient data from all trials
could the truth emerge:
Giving tamoxifen to ER− patients was useless.
Giving tamoxifen to ER− patients was useless.
"In the aggregate, the individual disappears.
In the individual, the truth appears.
This is why we seek the hidden patient."
In the individual, the truth appears.
This is why we seek the hidden patient."
This is Individual Participant Data Meta-Analysis.
What is lost when we summarize?
What is found when we look closer?
What is found when we look closer?
Two Ways to Synthesize
Aggregate Data (AD)
- Study-level summaries
- Effect sizes from publications
- Mean age, % male, etc.
- Quick and accessible
- Cannot see within-study variation
Individual Participant Data (IPD)
- Patient-level raw data
- Every participant's characteristics
- Actual ages, actual outcomes
- Time-intensive to obtain
- Can see who responds and who doesn't
The Ecological Fallacy
THE DANGER OF AGGREGATE THINKING
Imagine two trials of a drug for heart failure.
Trial A: Mean age 55 years, Effect size 0.70 (benefit)
Trial B: Mean age 75 years, Effect size 0.90 (less benefit)
Tempting conclusion: "The drug works better in younger patients."
But this could be completely wrong.
Trial A: Mean age 55 years, Effect size 0.70 (benefit)
Trial B: Mean age 75 years, Effect size 0.90 (less benefit)
Tempting conclusion: "The drug works better in younger patients."
But this could be completely wrong.
The Trap
ECOLOGICAL FALLACY
Within each trial, the relationship between age and treatment effect
might be completely different from the between-trial relationship.
Perhaps in Trial A, older patients within that trial responded better. Perhaps in Trial B, younger patients within that trial responded better.
You cannot know without individual data.
Perhaps in Trial A, older patients within that trial responded better. Perhaps in Trial B, younger patients within that trial responded better.
You cannot know without individual data.
THE RULE
Between-study relationships ≠ Within-study relationships
Aggregate data can only show between-study patterns
What IPD Enables
1
True Effect Modification
Test whether treatment effect varies by patient characteristics (age, biomarkers, disease severity)
2
Time-to-Event Analysis
Use actual survival curves, not just hazard ratios. Handle censoring properly.
3
Consistent Definitions
Standardize outcome definitions, exposure timing, covariate categories across studies
4
Subgroup Credibility
Test interactions within studies, avoiding the ecological fallacy
"Aggregate data shows the forest.
Individual data shows each tree.
If you need to know which trees are sick—
you must walk among them."
Individual data shows each tree.
If you need to know which trees are sick—
you must walk among them."
Have you not seen how one collaboration
gathered data on 170,000 patients
and answered questions no single trial could ask?
gathered data on 170,000 patients
and answered questions no single trial could ask?
Oxford, 1994-Present
CHOLESTEROL TREATMENT TRIALISTS' COLLABORATION
The CTT gathered individual data from every major statin trial.
27 trials. 174,149 patients. Every baseline characteristic. Every cardiovascular event. Every death.
The published trials asked: "Do statins work?"
The CTT asked: "For whom do statins work?"
27 trials. 174,149 patients. Every baseline characteristic. Every cardiovascular event. Every death.
The published trials asked: "Do statins work?"
The CTT asked: "For whom do statins work?"
CTT Collaboration. Lancet 2010;376:1670-81
The Scale of IPD
27
Trials Combined
174,149
Individual Patients
5
Years of Data Gathering
THE INVESTMENT
Gathering IPD takes years of negotiation, data transfer, cleaning, and harmonization.
But the questions you can answer are worth the investment.
But the questions you can answer are worth the investment.
What CTT Discovered
✓
Benefit Proportional to LDL Reduction
Every 1 mmol/L LDL reduction = 22% lower CV events. True across all subgroups.
✓
No Age Threshold
Benefit continues even in patients >75 years (contradicting earlier AD analyses)
✓
Primary Prevention Works
Patients without prior CVD benefit proportionally to their baseline risk
✗
No Cancer Signal
Concerns about statins causing cancer were definitively refuted with IPD follow-up
Why Aggregate Data Failed
THE LIMITATION
Published trials categorized patients differently.
Trial A: "High risk" = 10-year CVD risk >20%
Trial B: "High risk" = Prior MI
Trial C: "High risk" = Diabetes
You cannot compare or combine what is defined differently.
IPD allowed the CTT to redefine everyone consistently.
Trial A: "High risk" = 10-year CVD risk >20%
Trial B: "High risk" = Prior MI
Trial C: "High risk" = Diabetes
You cannot compare or combine what is defined differently.
IPD allowed the CTT to redefine everyone consistently.
"When trials speak different languages,
IPD provides the translation.
What seemed contradictory becomes clear:
the same truth, measured differently."
IPD provides the translation.
What seemed contradictory becomes clear:
the same truth, measured differently."
IPD takes years to gather.
Aggregate data takes weeks.
When is the investment worth it?
Aggregate data takes weeks.
When is the investment worth it?
The Decision Tree
Should You Pursue IPD?
Is your question about treatment effect modification?
↓
YES
"Who benefits most?"
NO
"Does it work overall?"
↓
IPD likely needed
AD may suffice
When IPD Is Essential
1
Time-to-Event Outcomes
When survival curves matter, not just final hazard ratios. When you need to handle censoring properly.
2
Continuous Effect Modifiers
Testing whether treatment effect varies by age, BMI, biomarker level (not just "high" vs "low")
3
Outcome Definition Problems
When trials define outcomes differently and you need to standardize
4
Longer Follow-Up Available
When trialists have unpublished follow-up data you want to include
🔀
The Research Strategy Decision
You're planning a meta-analysis of corticosteroids for COVID-19 pneumonia. You have 10 eligible RCTs with published data. A colleague asks: "Should we try to get IPD, or just do an aggregate analysis?"
Your key question is: "Does benefit vary by disease severity and timing of treatment?"
Your key question is: "Does benefit vary by disease severity and timing of treatment?"
What do you recommend?
A
Just do AD - we need results quickly and IPD takes too long
B
Pursue IPD - our question is about effect modification by patient characteristics
C
Do subgroup analysis by trial-level severity categories from publications
When Aggregate Data Suffices
✓
Overall Treatment Effect
When your only question is "Does it work?" not "For whom?"
✓
Homogeneous Population
When trials enrolled similar patients and effect modification is unlikely
✓
Binary Outcomes, Short Follow-up
When censoring isn't an issue and outcomes are simple yes/no
✓
IPD Unobtainable
When trialists won't share, data is lost, or resources are unavailable
"Not every question requires the individual.
But every question about which individuals
demands their presence in your data."
But every question about which individuals
demands their presence in your data."
You have gathered the individual data.
Now: do you analyze it as one combined dataset
or trial by trial, then combine?
Now: do you analyze it as one combined dataset
or trial by trial, then combine?
Two Analytical Approaches
Two-Stage Approach
- Stage 1: Analyze each trial separately
- Stage 2: Meta-analyze the results
- Preserves trial structure
- Familiar (like standard MA)
- Cannot handle sparse data well
One-Stage Approach
- Analyze all data simultaneously
- Mixed-effects regression model
- Random effects for clustering
- Better for sparse data
- More flexible modeling
Two-Stage: The Familiar Path
STAGE 1 (Within Each Trial)
Estimate treatment effect for trial k: θk
Using individual patient data from that trial
STAGE 2 (Across Trials)
Meta-analyze θ1, θ2, ... θK
Using standard random-effects methods
ADVANTAGE
Easy to understand. Easy to explain.
Each trial's estimate is transparent.
Familiar forest plots and I2 statistics.
Each trial's estimate is transparent.
Familiar forest plots and I2 statistics.
One-Stage: The Powerful Path
MIXED-EFFECTS MODEL
Yij = β0 + β1Treatmentij + uj + εij
i = patient, j = trial, uj = random trial effect
ADVANTAGES
1. Handles trials with zero events (no continuity corrections)
2. More powerful for detecting interactions
3. Can model complex covariate relationships
4. Exact likelihood (no normal approximations)
2. More powerful for detecting interactions
3. Can model complex covariate relationships
4. Exact likelihood (no normal approximations)
Choosing Your Approach
One-Stage or Two-Stage?
Do trials have sparse events (rare outcomes)?
↓
YES (sparse)
↓
One-stage preferred
Avoids zero-cell problems
NO (common events)
↓
Either works
Results should be similar
"The two-stage preserves each trial's voice.
The one-stage hears all voices at once.
When data is sparse and events are rare—
the one-stage catches what the two-stage misses."
The one-stage hears all voices at once.
When data is sparse and events are rare—
the one-stage catches what the two-stage misses."
Have you not heard the tale of the drug
that saved babies' lives—
but only if given at the right time?
that saved babies' lives—
but only if given at the right time?
The Preterm Paradox
NEONATAL INTENSIVE CARE UNITS WORLDWIDE
Antenatal corticosteroids reduce mortality in preterm infants.
This was established in the 1970s.
But a puzzle remained: When should they be given?
24 hours before birth? 48 hours? A week?
Published trials couldn't answer—they didn't report timing consistently.
But a puzzle remained: When should they be given?
24 hours before birth? 48 hours? A week?
Published trials couldn't answer—they didn't report timing consistently.
Roberts D, et al. Cochrane Database Syst Rev 2017
IPD Revealed the Window
24h-7d
Optimal Window
>7 days
Benefit Wanes
THE DISCOVERY
By examining each baby's exact time from steroid to delivery,
IPD meta-analysis showed:
Maximum benefit: 24 hours to 7 days before birth
Reduced benefit: >7 days (lung maturity effect fades)
No benefit: <24 hours (not enough time to work)
Maximum benefit: 24 hours to 7 days before birth
Reduced benefit: >7 days (lung maturity effect fades)
No benefit: <24 hours (not enough time to work)
Lives Saved, Practice Changed
THE CLINICAL IMPACT
Before IPD: Clinicians gave steroids and hoped for the best.
After IPD: Guidelines now recommend repeat dosing if delivery hasn't occurred within 7 days of the first course.
This precision—impossible without individual data— has saved thousands of premature babies.
After IPD: Guidelines now recommend repeat dosing if delivery hasn't occurred within 7 days of the first course.
This precision—impossible without individual data— has saved thousands of premature babies.
Timing is Treatment
The same drug, given at the wrong time, may as well be placebo
Why Aggregate Data Couldn't Show This
THE PROBLEM
Trial A reported: "Steroid given within 48 hours"
Trial B reported: "Steroid given antenatally"
Trial C reported: "Steroid-to-delivery interval: median 3 days"
Different categories. Different definitions. Incompatible summaries.
Only by examining each baby's actual steroid-to-delivery time could the optimal window be identified.
Trial B reported: "Steroid given antenatally"
Trial C reported: "Steroid-to-delivery interval: median 3 days"
Different categories. Different definitions. Incompatible summaries.
Only by examining each baby's actual steroid-to-delivery time could the optimal window be identified.
"The drug was known to work.
But when to give it was unknown.
IPD turned 'sometime' into 'the right time'—
and in that precision, children lived."
But when to give it was unknown.
IPD turned 'sometime' into 'the right time'—
and in that precision, children lived."
The data exists.
Somewhere, in files and databases,
each patient's story is recorded.
The question is: will they share it?
Somewhere, in files and databases,
each patient's story is recorded.
The question is: will they share it?
The Quest for Data
60-80%
Typical IPD Retrieval Rate
6-24
Months to Gather
REALITY CHECK
You will likely not get 100% of trials.
Some investigators will refuse. Some data is lost.
Some companies won't share.
This is expected. Plan for it.
This is expected. Plan for it.
Where IPD Lives
1
Trialist Collaboration
Direct contact with trial investigators. Build relationships. Offer co-authorship.
2
Data Sharing Platforms
YODA Project, ClinicalStudyDataRequest.com, Vivli, ICPSR
3
Regulatory Agencies
EMA Policy 0070, FDA (limited), Health Canada
4
Journal Requirements
Many journals now require data sharing; check supplementary materials
The Approach
HOW TO ASK
1. Lead with your question—explain why IPD is essential
2. Offer co-authorship—make sharing worthwhile
3. Describe data security—how you'll protect their patients
4. Provide data dictionaries—specify exactly what you need
5. Set clear timelines—respect their time
2. Offer co-authorship—make sharing worthwhile
3. Describe data security—how you'll protect their patients
4. Provide data dictionaries—specify exactly what you need
5. Set clear timelines—respect their time
The Availability Bias Trap
THE DANGER
What if trials that share data are systematically different
from trials that don't share?
Industry trials: less likely to share
Negative trials: less likely to share
Older trials: data may be lost
Your IPD sample may be biased.
Industry trials: less likely to share
Negative trials: less likely to share
Older trials: data may be lost
Your IPD sample may be biased.
THE RULE
Always compare: IPD trials vs. non-IPD trials
Do they differ in effect size, sample size, funding source?
"The data exists. The question is trust.
Will they share what they have guarded?
Build the bridge carefully—
for on that bridge, patients' futures cross."
Will they share what they have guarded?
Build the bridge carefully—
for on that bridge, patients' futures cross."
You have gathered the data
from twelve trials, five countries, three decades.
But Trial A calls it "cardiovascular death"
and Trial B calls it "cardiac mortality".
Are they the same?
from twelve trials, five countries, three decades.
But Trial A calls it "cardiovascular death"
and Trial B calls it "cardiac mortality".
Are they the same?
The Tower of Babel Problem
EVERY IPD META-ANALYSIS
Trial from Japan: Age in years since 1900 (e.g., "54" = 1954 birth)
Trial from USA: Age in decimal years (e.g., 65.7)
Trial from UK: Age bands ("65-74")
Diabetes: HbA1c ≥ 6.5% vs. fasting glucose ≥ 126 vs. "physician diagnosis"
Outcome: "Major adverse cardiac event" (one trial includes stroke, another doesn't)
Before analysis: harmonize everything.
Trial from USA: Age in decimal years (e.g., 65.7)
Trial from UK: Age bands ("65-74")
Diabetes: HbA1c ≥ 6.5% vs. fasting glucose ≥ 126 vs. "physician diagnosis"
Outcome: "Major adverse cardiac event" (one trial includes stroke, another doesn't)
Before analysis: harmonize everything.
The Harmonization Process
1
Create a Master Data Dictionary
Define every variable you need: name, type, permitted values, derivation rules
2
Map Each Trial's Variables
Document how each trial's coding maps to your standardized definitions
3
Check with Trialists
Verify your interpretations. They know their data better than you.
4
Validate Transformations
Reproduce published results from the IPD. If they don't match, investigate.
🔀
The Definition Dilemma
You're harmonizing IPD from 8 diabetes trials. Your primary outcome is "cardiovascular death."
Six trials used a standard definition (ICD codes). Two trials from the 1990s used "investigator-assessed cardiac death" with no standardized criteria. These two trials show larger treatment effects.
Six trials used a standard definition (ICD codes). Two trials from the 1990s used "investigator-assessed cardiac death" with no standardized criteria. These two trials show larger treatment effects.
How do you handle this?
A
Include all 8 trials - more data is better and definitions are "close enough"
B
Exclude the 2 non-standardized trials to maintain outcome consistency
C
Primary analysis with 6 consistent trials; sensitivity analysis adding the 2 others
The Crucial Validation
THE TEST
Before any new analysis:
Reproduce each trial's published results from the IPD.
If your analysis gives RR = 0.78 but the publication says RR = 0.85,
something is wrong.
Find the discrepancy. Fix it. Then proceed.
Reproduce each trial's published results from the IPD.
If your analysis gives RR = 0.78 but the publication says RR = 0.85,
something is wrong.
Find the discrepancy. Fix it. Then proceed.
"Different languages, different rulers,
different ways to name the same disease.
Before you can combine, you must translate.
Before you translate, you must understand."
different ways to name the same disease.
Before you can combine, you must translate.
Before you translate, you must understand."
The treatment works on average.
But does it work the same for the young and the old?
For the mild and the severe?
For the one with the biomarker and the one without?
But does it work the same for the young and the old?
For the mild and the severe?
For the one with the biomarker and the one without?
The Interaction That Changed Oncology
EBCTCG, 1985-1998
The Early Breast Cancer Trialists' Group didn't just ask "Does tamoxifen work?"
They asked: "Does the effect differ by estrogen receptor status?"
The interaction was massive:
ER-positive: 47% reduction in recurrence
ER-negative: No benefit at all
They asked: "Does the effect differ by estrogen receptor status?"
The interaction was massive:
ER-positive: 47% reduction in recurrence
ER-negative: No benefit at all
EBCTCG. Lancet 1998;351:1451-67
pinteraction < 0.00001
The most important interaction in cancer medicine
Testing Treatment-Covariate Interactions
THE MODEL
Outcome = β1Treatment + β2Covariate + β3(Treatment × Covariate)
β3 is the interaction term - does treatment effect vary by covariate?
INTERPRETATION
If β3 is statistically significant:
The treatment effect differs between subgroups defined by the covariate.
If not: the treatment effect is similar across subgroups (or you lack power to detect a difference).
The treatment effect differs between subgroups defined by the covariate.
If not: the treatment effect is similar across subgroups (or you lack power to detect a difference).
The Critical Distinction
Between-Study Interaction
- Compares trial-level averages
- Ecological fallacy risk
- Confounded by trial design
- Low statistical power
- Can do with aggregate data
Within-Study Interaction
- Compares patients within each trial
- No ecological fallacy
- Randomization preserved
- Much higher power
- Requires IPD
THE KEY INSIGHT
IPD allows you to test within-study interactions,
where randomization ensures the comparison is fair.
This is the gold standard for effect modification.
This is the gold standard for effect modification.
🔀
The Age Interaction Question
Your IPD meta-analysis of anticoagulation for atrial fibrillation includes 12,000 patients. You test for age interaction and find p=0.03. Patients ≥75 years show larger benefit (RR 0.55) than patients <75 years (RR 0.72).
You tested 8 potential effect modifiers in your analysis.
You tested 8 potential effect modifiers in your analysis.
How do you interpret this finding?
A
Report that anticoagulation works better in older patients - the interaction is significant
B
Apply multiple testing correction; with 8 tests, p=0.03 is likely a false positive
C
Report as hypothesis-generating, note the multiple comparisons, request external validation
"The average hides the interaction.
The interaction reveals who benefits.
Test it within studies, not between—
for that is where the truth is found."
The interaction reveals who benefits.
Test it within studies, not between—
for that is where the truth is found."
IPD is not only for treatments.
When you seek to predict who will die,
who will recover, who will relapse—
the individual is everything.
When you seek to predict who will die,
who will recover, who will relapse—
the individual is everything.
Predicting Recovery After Brain Injury
IMPACT CONSORTIUM, 2005-PRESENT
After traumatic brain injury, families ask the devastating question:
"Will they ever wake up?"
Individual trials were too small to develop accurate prediction models.
IMPACT gathered IPD from 11 studies, 9,205 patients, and built a model that predicts 6-month outcomes from initial clinical features.
"Will they ever wake up?"
Individual trials were too small to develop accurate prediction models.
IMPACT gathered IPD from 11 studies, 9,205 patients, and built a model that predicts 6-month outcomes from initial clinical features.
IMPACT Study Group. PLoS Med 2008;5:e165
Why Prognosis Demands IPD
1
External Validation Across Populations
Develop in some studies, validate in others. True test of generalizability.
2
Non-linear Relationships
Explore how predictors relate to outcome: linear? threshold? U-shaped?
3
Multiple Predictor Interactions
Age + GCS + pupil reactivity may interact in ways aggregate data cannot reveal
4
Proper Handling of Missing Predictors
Multiple imputation at the patient level, not at the study level
Reporting Standards
TRIPOD
- Prediction model reporting
- Development and validation
- Calibration and discrimination
- 22 item checklist
PRISMA-IPD
- IPD meta-analysis reporting
- Data acquisition details
- Harmonization process
- Integrity checking
FOR IPD PROGNOSTIC MODELS
You need both: TRIPOD for the model itself,
PRISMA-IPD for the data synthesis process.
When Prediction Saves Lives
THE CLINICAL IMPACT
The IMPACT model is now embedded in clinical decision support systems worldwide.
When a patient arrives with traumatic brain injury, the model provides a probability of survival and probability of favorable outcome.
This guides conversations with families. This informs treatment intensity. This helps allocate ICU resources.
Built from individual data. Serving individual patients.
When a patient arrives with traumatic brain injury, the model provides a probability of survival and probability of favorable outcome.
This guides conversations with families. This informs treatment intensity. This helps allocate ICU resources.
Built from individual data. Serving individual patients.
"To predict one patient's future,
you must learn from thousands of pasts.
IPD holds those stories—
each one a teacher, if you will listen."
you must learn from thousands of pasts.
IPD holds those stories—
each one a teacher, if you will listen."
Have you not considered the small white tablet
that has been given to millions for their hearts?
They were told: "Take this, and you shall be protected."
But was every heart equally in need of protection?
that has been given to millions for their hearts?
They were told: "Take this, and you shall be protected."
But was every heart equally in need of protection?
The Story of the Netflix Prize
A TRUE STORY
In 2006, Netflix released 100 million "anonymized" movie ratings for a $1 million prize.
Researchers quickly re-identified users by matching ratings with public IMDb reviews.
One lawsuit alleged a closeted lesbian was outed through her viewing patterns.
The lesson: removing names isn't anonymization.
IPD contains enough combinations of age, diagnosis, treatment response, and dates to uniquely identify individuals. Privacy requires more than deleting the name column—it requires understanding how data combinations become fingerprints.
Researchers quickly re-identified users by matching ratings with public IMDb reviews.
One lawsuit alleged a closeted lesbian was outed through her viewing patterns.
The lesson: removing names isn't anonymization.
IPD contains enough combinations of age, diagnosis, treatment response, and dates to uniquely identify individuals. Privacy requires more than deleting the name column—it requires understanding how data combinations become fingerprints.
Oxford, 2009
ANTITHROMBOTIC TRIALISTS' COLLABORATION
The ATT gathered individual data from 95,000 patients
across 6 primary prevention trials of aspirin.
The published trials said: "Aspirin prevents heart attacks."
But the ATT asked the forbidden question:
"At what cost? And for whom?"
The published trials said: "Aspirin prevents heart attacks."
But the ATT asked the forbidden question:
"At what cost? And for whom?"
ATT Collaboration. Lancet 2009;373:1849-60
The Numbers Revealed
-12%
Heart Attacks Prevented
+32%
Major Bleeds Caused
THE BALANCE
For every 1,000 low-risk patients taking aspirin for 5 years:
2-3 heart attacks prevented
2-3 major bleeds caused
The benefit and harm cancel out.
2-3 heart attacks prevented
2-3 major bleeds caused
The benefit and harm cancel out.
The Risk-Based Decision
Should This Patient Take Aspirin for Primary Prevention?
10-Year CVD Risk?
↓
>20%
High Risk
10-20%
Intermediate
<10%
Low Risk
↓
Consider aspirin
(benefit > harm)
(benefit > harm)
Shared decision
(benefit ≈ harm)
(benefit ≈ harm)
Avoid aspirin
(harm > benefit)
(harm > benefit)
Why did published meta-analyses miss this?
THE BLINDNESS OF AGGREGATES
Published trials reported average benefits.
They could not show that high-risk patients gained
while low-risk patients lost.
Only by examining each patient's baseline risk,
each patient's outcomes,
could the interaction be revealed.
They could not show that high-risk patients gained
while low-risk patients lost.
Only by examining each patient's baseline risk,
each patient's outcomes,
could the interaction be revealed.
🔀
The Primary Prevention Dilemma
A 52-year-old man asks about aspirin for heart attack prevention. He has no prior CVD, cholesterol is borderline, BP is controlled, non-smoker. His 10-year CVD risk is 8%.
He read online that "aspirin prevents heart attacks" and wants your advice.
He read online that "aspirin prevents heart attacks" and wants your advice.
What do you tell him?
A
"Yes, take aspirin daily - it's proven to prevent heart attacks"
B
"At your risk level, aspirin's bleeding risk roughly equals its heart benefit - I'd recommend against it"
C
"Take baby aspirin - the lower dose is safer"
"The medicine that saves the endangered
may harm the secure.
Know your patient's risk before you prescribe.
This is what the individual data taught us."
may harm the secure.
Know your patient's risk before you prescribe.
This is what the individual data taught us."
95,000 patients. One truth: risk determines benefit.
Have you not seen how they debated for decades:
"Should the old be treated the same as the young?"
Some said: "Lower is always better."
Others said: "The old are fragile. Be cautious."
Who was right?
"Should the old be treated the same as the young?"
Some said: "Lower is always better."
Others said: "The old are fragile. Be cautious."
Who was right?
The J-Curve Controversy
CARDIOLOGY CONFERENCES, 1980s-2000s
For years, doctors argued:
The Aggressive Camp: "Every mmHg of BP reduction saves lives.
Treat everyone to target 120/80."
The Conservative Camp: "In the elderly, low BP causes falls, strokes, death.
There's a J-curve—too low is dangerous."
Individual trials were too small to settle it. Until the BPLTTC gathered the individual data.
The Aggressive Camp: "Every mmHg of BP reduction saves lives.
Treat everyone to target 120/80."
The Conservative Camp: "In the elderly, low BP causes falls, strokes, death.
There's a J-curve—too low is dangerous."
Individual trials were too small to settle it. Until the BPLTTC gathered the individual data.
The Blood Pressure Lowering Treatment Trialists
48
Trials Combined
344,716
Individual Patients
15+
Years of Follow-up
THE QUESTION
Does the benefit of BP lowering vary by age?
Is there a J-curve at very low pressures?
Do the very old (>80 years) still benefit?
Is there a J-curve at very low pressures?
Do the very old (>80 years) still benefit?
BPLTTC. Lancet 2021;397:1625-36
The Answer, At Last
Benefit at Every Age
Proportional risk reduction was consistent from age 55 to 85+
WHAT IPD SHOWED
Every 5 mmHg BP reduction:
Age 55-64: 10% lower major CV events
Age 65-74: 10% lower major CV events
Age 75-84: 10% lower major CV events
Age ≥85: Still 10% lower
pinteraction = 0.85. No evidence of age modification.
Age 55-64: 10% lower major CV events
Age 65-74: 10% lower major CV events
Age 75-84: 10% lower major CV events
Age ≥85: Still 10% lower
pinteraction = 0.85. No evidence of age modification.
What About the J-Curve?
THE RESOLUTION
The BPLTTC tested for a J-curve by examining patients
who achieved very low blood pressures.
Result: No J-curve in randomized comparisons.
The apparent J-curve in observational data was reverse causation— sick patients have low BP because they're sick, not sick because their BP is low.
Only IPD from RCTs could untangle this.
who achieved very low blood pressures.
Result: No J-curve in randomized comparisons.
The apparent J-curve in observational data was reverse causation— sick patients have low BP because they're sick, not sick because their BP is low.
Only IPD from RCTs could untangle this.
Treating Hypertension by Age
Should This Elderly Patient Get BP Treatment?
Patient Age?
↓
55-74
75-84
85+
↓
Treat if BP elevated
Proportional benefit maintained across all ages
Proportional benefit maintained across all ages
THE LESSON
Age alone is not a reason to withhold treatment.
Consider frailty, life expectancy, patient preference—but not age.
Consider frailty, life expectancy, patient preference—but not age.
🔀
The 82-Year-Old's Blood Pressure
An 82-year-old woman has BP 158/88. She's independent, cognitively intact, and has no history of falls. A colleague says: "She's too old for aggressive BP treatment. The J-curve, you know."
You remember the BPLTTC IPD meta-analysis.
You remember the BPLTTC IPD meta-analysis.
What do you do?
A
Agree with your colleague - accept higher BP targets for the elderly
B
Cite the BPLTTC evidence that benefit continues in the very elderly and initiate treatment
C
Wait for a cardiovascular event before treating
"They said the old were too fragile for medicine.
But the individual data showed otherwise:
At every age, the benefit endures.
Do not let age alone deny protection."
But the individual data showed otherwise:
At every age, the benefit endures.
Do not let age alone deny protection."
Have you not witnessed the race against time
when a clot blocks the brain?
Every minute, two million neurons die.
But when does the window close?
When is it too late to intervene?
when a clot blocks the brain?
Every minute, two million neurons die.
But when does the window close?
When is it too late to intervene?
Time Is Brain
1.9M
Neurons Lost Per Minute
14B
Synapses Lost Per Minute
THE QUESTION
Thrombolysis can dissolve the clot and restore blood flow.
But given too late, it causes bleeding into dying brain tissue.
What is the time window? 3 hours? 4.5 hours? 6 hours?
Individual trials disagreed. Guidelines were uncertain.
But given too late, it causes bleeding into dying brain tissue.
What is the time window? 3 hours? 4.5 hours? 6 hours?
Individual trials disagreed. Guidelines were uncertain.
The Stroke Thrombolysis Trialists
STT COLLABORATIVE GROUP, 2014
The STT gathered individual data from 6,756 patients
across 9 major alteplase trials.
They knew exactly when each patient's stroke began. They knew exactly when thrombolysis was given. They knew exactly who lived, who died, who recovered.
They could map benefit against time, minute by minute.
They knew exactly when each patient's stroke began. They knew exactly when thrombolysis was given. They knew exactly who lived, who died, who recovered.
They could map benefit against time, minute by minute.
STT Collaborative Group. Lancet 2014;384:1929-35
The Fading Window
0-3h
OR 1.75 for good outcome
3-4.5h
OR 1.26 for good outcome
4.5-6h
OR 1.15 (not significant)
THE DECAY OF BENEFIT
Every 15 minutes of delay reduced the benefit.
Treated at 90 min: 1 in 4 achieve excellent outcome
Treated at 180 min: 1 in 7 achieve excellent outcome
Treated at 270 min: 1 in 14 achieve excellent outcome
Treated at 90 min: 1 in 4 achieve excellent outcome
Treated at 180 min: 1 in 7 achieve excellent outcome
Treated at 270 min: 1 in 14 achieve excellent outcome
The Thrombolysis Decision
Acute Ischemic Stroke: Give Thrombolysis?
Time Since Symptom Onset?
↓
0-3h
Strong benefit
3-4.5h
Moderate benefit
4.5-6h
Uncertain
>6h
Harm likely
↓
TREAT
URGENTLY
URGENTLY
TREAT
if eligible
if eligible
Consider
imaging
imaging
Generally
avoid
avoid
Why did we need individual data?
THE LIMITATION OF AVERAGES
Trial A: "Patients treated within 3 hours" (average: 2.1 hours)
Trial B: "Patients treated within 6 hours" (average: 4.2 hours)
These overlapping, inconsistent windows couldn't be compared.
Only by knowing each patient's exact time
could the continuous decay of benefit be mapped.
Trial B: "Patients treated within 6 hours" (average: 4.2 hours)
These overlapping, inconsistent windows couldn't be compared.
Only by knowing each patient's exact time
could the continuous decay of benefit be mapped.
🔀
The 4-Hour Stroke
A 68-year-old presents with acute stroke. Symptom onset was 4 hours ago. CT shows no hemorrhage, NIHSS score is 12 (moderate). The family asks: "Is it too late for the clot-busting drug?"
Door-to-needle time will add 30 minutes, making total time ~4.5 hours.
Door-to-needle time will add 30 minutes, making total time ~4.5 hours.
What do you do?
A
"It's too late - the 3-hour window has passed"
B
"We're still within the 4.5-hour window - proceed with thrombolysis urgently"
C
"Wait for MRI to see if there's salvageable tissue"
"Every minute the clot remains,
two million neurons perish.
The IPD showed us the fading window.
Act quickly, or the window closes forever."
two million neurons perish.
The IPD showed us the fading window.
Act quickly, or the window closes forever."
You have gathered the individual data.
You open the files with hope.
And then you see it: empty cells.
Age: 67. Sex: Male. Smoking status: missing.
Outcome at 1 year: missing.
What now?
You open the files with hope.
And then you see it: empty cells.
Age: 67. Sex: Male. Smoking status: missing.
Outcome at 1 year: missing.
What now?
The Three Kinds of Missingness
1
Missing Completely at Random (MCAR)
Lab machine broke randomly. No relation to patient characteristics. Safe to ignore (but wasteful).
2
Missing at Random (MAR)
Older patients more likely to miss follow-up. Missingness related to observed variables. Imputation can help.
3
Missing Not at Random (MNAR)
Patients with poor outcomes drop out. Missingness related to the missing value itself. Dangerous. Requires sensitivity analysis.
Handling Missing Data
What To Do With Missing Values?
How much is missing?
↓
<5%
Minor
5-20%
Moderate
>20%
Substantial
↓
Complete case
may suffice
may suffice
Multiple
imputation
imputation
Impute +
sensitivity
sensitivity
Multiple Imputation: The Standard
THE PROCESS
1. Impute → 2. Analyze → 3. Pool
Create M imputed datasets, analyze each, combine using Rubin's rules
WHY MULTIPLE?
Single imputation pretends you know the missing value.
Multiple imputation (M=20-50 datasets) reflects uncertainty about what the missing value might have been.
This preserves valid standard errors and p-values.
Multiple imputation (M=20-50 datasets) reflects uncertainty about what the missing value might have been.
This preserves valid standard errors and p-values.
IPD-Specific Missing Data Challenges
1
Systematically Missing Variables
Trial A measured biomarker X. Trial B didn't. Can't impute what was never collected.
2
Multilevel Structure
Patients nested within trials. Imputation model must account for clustering.
3
Different Follow-up Durations
Trial A followed for 2 years. Trial B for 5 years. Survival analysis needs care.
When a Variable Is Missing Entirely
Trial Didn't Collect Your Key Covariate
Is it an effect modifier or confounder?
↓
Effect Modifier
↓
Restrict interaction
analysis to trials
with the variable
analysis to trials
with the variable
Confounder
↓
Sensitivity analysis
varying assumptions
about unmeasured
varying assumptions
about unmeasured
🔀
The Missing Biomarker
Your IPD-MA tests whether a new cancer drug works better in biomarker-positive patients. You have IPD from 8 trials. But 3 trials (40% of patients) didn't measure the biomarker at all.
These 3 trials tend to be older and smaller.
These 3 trials tend to be older and smaller.
How do you analyze the interaction?
A
Impute the biomarker status based on other patient characteristics
B
Analyze interaction only in the 5 trials with biomarker data; sensitivity analysis including all trials for overall effect
C
Exclude the 3 trials entirely from the meta-analysis
"The empty cell is not nothing.
It is a question: Why is this unknown?
Answer that question before you fill the gap—
for the reason for absence shapes the solution."
It is a question: Why is this unknown?
Answer that question before you fill the gap—
for the reason for absence shapes the solution."
Have you not seen the researcher
who gathered data from willing trialists
and declared victory?
But the unwilling held secrets.
And those secrets changed everything.
who gathered data from willing trialists
and declared victory?
But the unwilling held secrets.
And those secrets changed everything.
The Story of Tamiflu
A TRUE STORY
For years, governments stockpiled Tamiflu for pandemic flu, spending billions.
Cochrane reviewers requested trial data to verify efficacy. Roche refused, citing confidentiality.
For five years, the BMJ campaigned for transparency. When full Clinical Study Reports were finally released in 2014, the picture changed: Tamiflu reduced symptom duration by less than a day and didn't prevent complications.
Billions spent on a drug whose full evidence was locked away.
The Tamiflu saga transformed expectations—today, clinical trial transparency is becoming the norm, not the exception.
Cochrane reviewers requested trial data to verify efficacy. Roche refused, citing confidentiality.
For five years, the BMJ campaigned for transparency. When full Clinical Study Reports were finally released in 2014, the picture changed: Tamiflu reduced symptom duration by less than a day and didn't prevent complications.
Billions spent on a drug whose full evidence was locked away.
The Tamiflu saga transformed expectations—today, clinical trial transparency is becoming the norm, not the exception.
When IPD Is Selectively Available
THE EMPIRICAL FINDING
Studies have compared trials that share IPD vs. those that don't:
Industry-sponsored trials: Less likely to share
Trials with negative results: Less likely to share
Older trials: Data often lost
If these trials systematically differ in effect size,
your IPD-MA is biased.
Industry-sponsored trials: Less likely to share
Trials with negative results: Less likely to share
Older trials: Data often lost
If these trials systematically differ in effect size,
your IPD-MA is biased.
Ahmed I, et al. BMJ 2012;344:d7762
Always Compare: IPD vs. Non-IPD Trials
Assessing Availability Bias
Compare AD effects: trials with IPD vs. without
↓
Similar Effects
↓
Low concern
for bias
for bias
Different Effects
↓
High concern
Sensitivity analysis needed
Sensitivity analysis needed
The Availability Bias Checklist
✓
Report IPD retrieval rate
"We obtained IPD from 12/15 trials (80%)"
✓
Compare IPD vs. non-IPD trial characteristics
Sample size, funding, publication date, effect size from aggregate data
✓
Sensitivity analysis including non-IPD trials
Two-stage analysis combining IPD + AD from non-sharing trials
✓
Discuss reasons for non-sharing
Data lost? Refused? Never requested? Each has different implications.
🔀
The Reluctant Trialists
Your IPD-MA of a surgical intervention obtained data from 8 of 12 eligible trials (67%). The 4 non-sharing trials are all industry-funded and show smaller benefits in their publications (pooled OR 0.95) compared to the 8 IPD trials (pooled OR 0.72).
Your IPD analysis shows OR 0.70.
Your IPD analysis shows OR 0.70.
How do you report this?
A
Report IPD result (OR 0.70) as the main finding - IPD is gold standard
B
Report IPD result with prominent warning about availability bias and sensitivity analysis including AD from non-sharing trials
C
Wait until you get 100% IPD before publishing
The Two-Stage Hybrid Solution
COMBINING IPD + AD
Stage 1: Analyze IPD trials → get θIPD
Stage 2: Meta-analyze θIPD + θAD from non-IPD trials
Uses all available evidence, even when IPD incomplete
THE PRINCIPLE
IPD is not all-or-nothing.
Use IPD where you have it (for interaction testing).
Supplement with AD for overall effect estimation.
Transparency about what came from where.
Use IPD where you have it (for interaction testing).
Supplement with AD for overall effect estimation.
Transparency about what came from where.
"The willing may not represent the whole.
The open doors may hide the truth behind locked ones.
Always ask: Who refused to share?
And what might they be hiding?"
The open doors may hide the truth behind locked ones.
Always ask: Who refused to share?
And what might they be hiding?"
What did the Early Breast Cancer Trialists' Group discover that individual trials could not?
Tamoxifen works better than placebo
Tamoxifen only works in estrogen receptor positive tumors
Tamoxifen has significant side effects
Tamoxifen works better in younger women
What is the "ecological fallacy" in meta-analysis?
Assuming environmental factors don't affect treatment
Assuming between-study relationships apply to within-study (individual) relationships
Pooling studies from different ecological regions
Using outdated trials in modern meta-analyses
When is a one-stage IPD analysis preferred over two-stage?
When trials have sparse events or rare outcomes
When you have many large trials
When you want to create forest plots
When outcomes are common and follow-up is short
What should you do before analyzing IPD from multiple trials?
Immediately pool all data and run the main analysis
Exclude trials with missing variables
Reproduce each trial's published results from the IPD to verify data integrity
Convert all continuous variables to categorical
References
Key Sources Cited in This Course
- Riley RD, et al. Individual Participant Data Meta-Analysis: A Handbook for Healthcare Research. Wiley, 2021.
- Stewart LA, et al. PRISMA-IPD: Preferred reporting items for systematic reviews and meta-analyses of individual participant data. JAMA 2015;313:1657-65.
- Early Breast Cancer Trialists' Collaborative Group. Tamoxifen for early breast cancer. Cochrane Database Syst Rev 2001.
- Cholesterol Treatment Trialists' Collaboration. Efficacy and safety of LDL-lowering therapy. Lancet 2010;376:1670-81.
- Roberts D, et al. Antenatal corticosteroids for accelerating fetal lung maturation. Cochrane Database Syst Rev 2017.
- IMPACT Study Group. Predicting outcome after traumatic brain injury. PLoS Med 2008;5:e165.
- Debray TPA, et al. Get real in individual participant data meta-analysis. Int J Epidemiol 2015;44:1287-97.
- Burke DL, et al. Meta-analysis using individual participant data. Stat Med 2017;36:320-38.
✔
Course Complete
"In the aggregate, the individual disappears.
But you have learned to find them.
You have learned to ask: Who benefits? Who is harmed?
Now go—and let no patient vanish in the average."
But you have learned to find them.
You have learned to ask: Who benefits? Who is harmed?
Now go—and let no patient vanish in the average."
The Hidden Patient — Now You See Them.