Clue 1: Orient yourself
Which records are found before the sieve?
Use the labels and the picture's left-to-right, near-to-far, or before-and-after order. Name only what you can point to.
The question
Which records are found before the sieve?
Why it matters: Clinical recommendations affect real children and families. Fair comparisons, bias control, ethical limits, and honest uncertainty keep a promising result from becoming a harmful claim. Today you practice the professional reasoning behind that work: Systematic reviews reduce cherry-picking, while meta-analysis is credible only when studies and pooling assumptions support it.
On your WebXam
Deciding when heterogeneity means studies should not be pooled
For life
One result can have many causes; the outcome alone never tells you which.
Principle: Many roads, one ending
Five principles we return to
Having it is not using it
Same instructions, different switches
Same look, different cause
Change one thing and watch
Boss or doorman?
Decides the result or only allows it
Held down, not gone
Remove the brake and it returns
Many roads, one ending
One result can begin many ways
Try the everyday version first
A recipe search needs a sieve before ingredients enter one bowl
A sieve removes items that do not meet the rule before the rest are combined. The rule must be set first so the final mixture can be traced and checked.
Do not jump to the biology yet. Treat the picture as a small system. Track its parts, follow one change at a time, and keep more than one explanation open until the picture supplies a way to separate them.
Clue 2: Trace one change
Who sets the inclusion holes and when?
Follow one object, stage, or path. Point to the first place where the situation changes instead of jumping to the ending.
Clue 3: Keep the cause open
Why might some ingredients remain outside the bowl?
List more than one explanation that still fits. Name the extra observation that would help you separate those possibilities.
Now inspect the illustration
Work from the visible evidence. A useful answer names the part of the picture that supports it and leaves unknown causes open.
- 1Which records are found before the sieve?
- 2Who sets the inclusion holes and when?
- 3Why might some ingredients remain outside the bowl?
Tier 1 check
Finish with the everyday model
Use the everyday picture to answer today's question in plain words: Which records are found before the sieve?
You can complete today's required check without opening the technical details below.
Ready for the real names? Optional tier 2
Technical rules and limits
Rule 1: Predefine eligibility, search, outcomes, and analysis.
Rule 2: Assess risk of bias before pooling.
Rule 3: Interpret heterogeneity and pool only when the combined estimate has clear meaning.
Where the analogy stops: Studies are not interchangeable ingredients, and statistical weights do not erase design flaws.
Carry the previous idea forward
Bias is directional design or measurement error; confounding mixes effects; each threat needs a matched safeguard.
Today's technical takeaway
Systematic reviews reduce cherry-picking, while meta-analysis is credible only when studies and pooling assumptions support it.
Now map the same rules onto biology
Move from a PRISMA flow diagram to a defensible forest plot
Search shelves
Comprehensive literature search
Eligibility sieve
Predefined inclusion and exclusion rules
Pooling bowl
Meta-analysis when studies are compatible
Educational illustration, not a clinical photograph or a patient-specific study plan. Use the supplied evidence cards and claim ceiling.
Mateo's case file: evidence supplied in this lesson
EXP17-E1
The protocol was registered before screening and lists databases, eligibility, outcomes, and analysis.
Why it matters: The process is harder to change after seeing results.
EXP17-E2
Two studies use blinded age-5 VPI ratings, while one uses an unvalidated parent question at age 2.
Why it matters: Outcome differences may limit a meaningful pooled estimate.
EXP17-E3
The forest plot shows effects in different directions and major clinical differences across centers.
Why it matters: Heterogeneity must be explained rather than hidden by one average.
Make the clinical decision
You are the systematic-review editor deciding whether to approve pooling.
The authors want one dramatic number even though one study measures a different outcome and timing.
APresent the review and pool only a justified compatible subset.
BCombine every number because meta-analysis is always strongest.
CDiscard studies that disagree with the preferred answer.
Choose the synthesis plan and cite protocol, outcome compatibility, and heterogeneity evidence.
Evidence required
EXP17-E1 + EXP17-E2 Claim ceiling
You may synthesize and pool compatible studies when justified. You may not claim a pooled estimate repairs weak or incompatible evidence.Go deeper Optional tier 3
Everything required for today is above. Open these only if you want the explainer, source trail, or download files.
The everyday model and Tier 1 check are the complete required path for this lesson.
Use these checks to keep your place. They are not turned in through the portal.
Check off as you finish
- Worked through the everyday picture and answered its three questions.
- Completed the Tier 1 check in plain words.
Turn in: Experimental Design lesson 17: How Do We Combine Many Studies Into One Answer?
Go to Schoology to turn this in.
Submit one PDF. Put your first and last name in the document header. Name the file: FirstName LastName - Assignment Title - YYYY-MM-DD.pdf.
Open Schoology PDF upload helpIf you cannot get in, see Mr. Mendoza. Do not skip the work.
Optional legacy technical materials Open only if you want the original notes, vocabulary, artifact, and CER work
Before this page, you should know
- Bias is a built-in error in how a study is run or measured that pushes results one direction; (a measurement bias) and survival bias (a selection bias) are two examples.
- Confounding happens when a hidden third variable is tangled with both the thing you changed and the thing you measured, so it can masquerade as the cause.
If not, review
Today's new idea is only
A uses PRISMA to find and judge all the studies on a question, and a meta-analysis pools only the studies that are similar enough; too much heterogeneity means do not combine.
Goal: Explain how a pools studies using PRISMA, read a PRISMA flow diagram, and state when studies are too different to combine (heterogeneity).
Know by the end
- A is a structured, repeatable way to find all the studies on one question, judge their quality, and summarize them, reported under the 27-item checklist plus a flow diagram.
- Gold-standard reviews use a pre-registered protocol, a multi-database search, dual independent screening, a formal risk-of-bias check, and an assessment of .
- A meta-analysis is the statistics step that pools comparable studies into one summary estimate with a 95% CI and measures heterogeneity (often as I-squared); too-heterogeneous studies should not be pooled.
- The TOPS team noted an earlier of -timing trials could not statistically combine its four trials because they were too different (different ages, techniques, outcome definitions).
1
Systematic review and PRISMA
Model start: A finds and appraises all the studies on one question in a repeatable way, so the answer does not depend on which papers an author happened to like.
- A is a structured, repeatable way to find ____ the studies on one question, judge their quality, and summarize them.
- The is , a 27-item checklist plus a flow ____.
- Gold-standard features include a pre-registered protocol, a multi-database search, dual independent screening, a formal ____-of-bias check, and an assessment of .
2
Meta-analysis and heterogeneity
- A meta-analysis pools the included studies into one summary estimate with a 95% ____ interval and measures heterogeneity, often reported as I-squared.
- If studies are too ____, you should not pool them, exactly the call the -timing reviewers made.
- The danger to remember: garbage in, garbage out. Pooling biased studies gives a precise-looking but ____ answer.
Everything you need for today is on this page. These links are optional.
What to read
Read the short plain-language explanation written for this lesson. Plain-language explainer for this lesson
Why this source matters
This explanation gives you the background for today's idea without making you decode a research paper: A uses PRISMA to find and judge all the studies on a question, and a meta-analysis pools only the studies that are similar enough; too much heterogeneity means do not combine.
Words to unlock first
systematic reviewmeta-analysisheterogeneityPRISMA 2020publication bias
Reading moves
- Skim the title and abstract first to get the gist.
- Circle the one sentence that states the main claim.
- Box the evidence the authors give for that claim.
- Mark one sentence that confuses you, and move on.
Stop point
Stop after the final 'Use it now' section. The research citations are available separately for advanced readers.
Your output
Write one claim-evidence sentence: state the main idea, then name the example or evidence that supports it.
Where this fits
Tested on (Ohio WebXam)
Genetics of Disease · 072130
PLTW lesson
MI · Experimental Design domain · Systematic review and meta-analysis; PRISMA, heterogeneity, and publication bias
WebXam domain
Molecular and Genetic Technology
Evidence to produce
Use the flow diagram. Of the 4 included studies, suppose 3 measured speech with the same validated test and 1 used a completely different home-made test. As Data Scientist: (1) which studies could you reasonably pool, and which one would you keep separate, and why? (2) fill the '?' in the diagram: how many studies go into the meta-analysis under your decision? (3) write one sentence telling the PI what the review can honestly conclude.
Lab / skill
Biomedical Innovations (BI) · AP Biology
Everything required for today's decision is already in the case file and plain-language explainer. The links below are original papers and database records for teachers and advanced readers, not assigned student reading.
- Claim: Combining many studies can give a stronger answer than any single study, but only when it is done carefully.
- Evidence: requires ____ (name one safeguard), and a real -timing review could not pool its four trials because they were too ____.
- Reasoning: Explain why a careless meta-analysis of biased or mismatched studies can be worse than honest single studies.
How this is graded (rubric)
For: Use the flow diagram. Of the 4 included studies, suppose 3 measured speech with the same validated test and 1 used a completely different home-made test. As Data Scientist: (1) which studies could you reasonably pool, and which one would you keep separate, and why? (2) fill the '?' in the diagram: how many studies go into the meta-analysis under your decision? (3) write one sentence telling the PI what the review can honestly conclude.
| Criterion | Proficient | Developing | Beginning |
|---|---|---|---|
| Complete | Every required part of the artifact is present and filled in. | Most parts are present, but one is missing or left blank. | Several parts are missing. |
| Accurate | The science and data are correct and match the evidence. | Mostly correct, with a small factual slip. | Key science or data is wrong. |
| Scientific reasoning (CER) | States a claim, backs it with specific evidence, and explains the reasoning. | Has a claim and evidence, but the reasoning is thin or missing. | Gives an answer with no evidence or reasoning. |
| Professional communication | Clear, organized, and labeled the way a clinician or scientist would write it. | Readable but disorganized or missing labels. | Hard to follow. |
| Submitted | Turned in through the route named under Submit here and confirmed. | Turned in, but in the wrong place or unconfirmed. | Not turned in. |
How the model answer scores against this rubric
- CompleteProficient: Nothing is left blank: the model fills every part of "Use the flow diagram. Of the 4 included studies, suppose 3 measured speech with the same validated test and 1 used a completely different home-made test. As Data Scientist: (1) which studies could you reasonably pool, and which one would you keep separate, and why? (2) fill the '?' in the diagram: how many studies go into the meta-analysis under your decision? (3) write one sentence telling the PI what the review can honestly conclude.".
- AccurateProficient: Every number and claim matches the case evidence.
- Scientific reasoning (CER)Proficient: It names a claim, cites the specific evidence, and explains the reasoning, not just the answer.
- Professional communicationProficient: It is organized and labeled like a real chart note.
- SubmittedProficient: It would be attached to your class form or handed in, and confirmed.
What's next: Combining studies only works if the studies were done on real children fairly. But every study we pool was run on real babies and families. What rules protect children in research in the first place, and who decides what is allowed? We chase that next time.