What Could Fool Us Into the Wrong Conclusion?
This reading contains every idea and every piece of evidence needed for today's decision. The research links at the end are optional.
Why this matters
Precision cannot rescue a study that measures or compares the wrong things in a consistent direction.
The question you are trying to answer
Which problem changes the measurement itself?
Begin with the idea you already earned
An operational definition makes speech scoring repeatable, while patient-reported outcomes capture effects clinician scores can miss.
Study the analogy before the biology
- Which problem changes the measurement itself?
- Which adds another cause linked to both sides?
- Would a larger sample fix either problem?
Turn the analogy into three rules
Limit: Real studies can have many interacting biases and unmeasured confounders, not one visible mirror or magnet.
Map those rules onto the biology
Bias is systematic error from selection, measurement, missing data, or study conduct. A larger sample can make a biased estimate more precise without making it correct.
A confounder is linked to both the factor being studied and the outcome, but is not simply a step in the causal pathway.
Randomization, blinding, retention, matching, measurement, and adjusted analysis address different threats. No safeguard is perfect.
Read Mateo's labeled case evidence
Earlier-repair participants are more likely to receive care at high-volume centers with strong speech services.
Center resources could confound an observational comparison.
Assessors know each child's group and expect earlier repair to work.
Expectation could create directional measurement bias.
Follow-up is 92 percent in one group and 71 percent in the other, with access-related reasons.
Differential attrition could change the comparison.
Make the concrete decision
You are the adversarial methods reviewer before data lock.
The team says a sample of 5,000 makes bias and confounding disappear.
- Match each supplied threat with a design, measurement, or analysis safeguard.
- Accept the claim because large samples remove systematic error.
- List every possible bias without changing the protocol.
Choose the red-team response and propose one matched safeguard per evidence card.
Claim ceiling: You may reduce named threats. You may not claim adjustment removes all unmeasured bias or confounding.
Write the 10-year takeaway
Bias is directional design or measurement error; confounding mixes effects; each threat needs a matched safeguard.
- Why can a large sample still be biased?
- How is a confounder different from measurement bias?
Glossary in plain English
A systematic error in how data is collected or interpreted that tilts results in one direction, making conclusions less accurate or unfair.
When a hidden third factor influences both the suspected cause and the outcome, making a link look real when it may be misleading.
Keeping participants, researchers, or both unaware of who got which treatment so expectations cannot bias the results.
A study error where people with a condition remember past exposures differently than people without it, distorting the results.
A study bias where differences in patient outcomes come from which surgeon operated rather than from the treatment being tested.
Research citation trail (advanced)
You do not need these papers or database records to finish the lesson. They document where the plain-language explainer's claims come from and are intended for teachers or advanced readers.
- STROBE. Reporting guidance for observational studies.
- Ozawa et al. 2020, Surgical technique and timing in UCLP (surgeon as driver, Orthod Craniofac Res)
- Sabbagh et al. 2023, COVID-19 risk factors and orofacial clefts, five-country case-control (BMC Oral Health)
- Grosen et al. 2011, Risk of oral clefts in twins (survival bias in registries, Epidemiology)