Rough draft.This research track is under review with Dr. Atit's lab. Content and sequence may still change.
The Baby Mateo Case
Experimental Design domainBiomedical Innovations (BI)Lesson 8 of 20Your seat: Biostatistician on the cleft research team
Why Gene Studies Use Such Tiny P-Values
Discovery question
Why do gene studies demand a far tinier p-value than 0.05?
💡 Testing many hypotheses at once breeds false positives, so a genome scan must correct its threshold down to p < 5 x 10^-8 and demand replication.
The plan
Prerequisite check
Before this page, you should know
- An association means two things tend to occur together; an (or ) measures effect size, where 1.0 means no difference.
- A p-value is the probability of a result at least this extreme if there were truly no effect; by convention p < 0.05 is called statistically significant.
If not, review
Today's new idea is only
Testing many hypotheses at once breeds false positives, so a genome scan must correct its threshold down to p < 5 x 10^-8 and demand replication.
Learn first
What you will learn
Goal: Students will explain why testing many hypotheses at once inflates false positives, and apply a Bonferroni-style correction to reach the threshold p < 5 x 10^-8.
Know by the end
- A problem appears the moment you test many hypotheses on the same data, because each test gets its own chance to throw a .
- At p < 0.05, running 1,000,000 tests with no real effect yields on average about 50,000 false alarms by chance.
- The divides the 0.05 bar by the number of tests; 0.05 / 1,000,000 equals 5 x 10^-8, the threshold a real European GWAS held its hits to.
- Replication in a second independent sample is the strongest check; the (FDR, Benjamini-Hochberg) is a less strict alternative used heavily in gene-expression work.
Learn first
Model: One coin, then many coins, and the real GWAS threshold
A p-value of 0.05 means: if there were truly no effect, you would still see a result this striking about 1 time in 20 by chance alone. Run 1 test at p < 0.05 and you face about a 5 percent chance of a false alarm, which is tolerable. Run 20 tests, all with no real effect, and on average 1 of them crosses p < 0.05 anyway. Run 1,000,000 tests (a GWAS-sized scan), all with no real effect, and on average about 50,000 cross p < 0.05 by chance. This is the multiple-testing logic the synthesis dossier names as the central enemy of genetic studies [DOI:10.1002/bdr2.2216].
A genome-wide association study of in Europeans scanned across the genome and reported its top signals only when they cleared the standard bar of p < 5 x 10^-8 (that is 0.00000005), not 0.05 [PMID:31817908, DOI:10.3390/genes10121023]. Hits that did not clear that bar were treated as not yet trustworthy, and the strongest believable signals were expected to repeat in a second independent group before the field accepted them.
Do the work
Explore (work the model before reading on)
- What does a p-value of 0.05 mean for a single test, in your own words?
- Write out 5 x 10^-8 as a regular decimal. How many zeros after the point before the 5?
- Going from 20 tests to 1,000,000 tests, what happened to the expected number of false alarms, and why does running more tests create more flukes?
- A reasonable fix divides your 0.05 bar by the number of tests so the whole experiment still has only a 5 percent chance of any false alarm. For 1,000,000 tests, what is 0.05 / 1,000,000, and how does it compare to 5 x 10^-8?
- Suppose a team reports a SNP near IRF6 at p = 0.001. By the single-test rule that is significant. By the genome-wide rule, is it, and what would you tell the team to do before believing it?
- In one sentence, what pattern did your team find about p-values when you run many tests at once?
The plan
Guided notes
1
The multiple-testing trap
Model start: means each of many tests gets its own chance to throw a , so flukes pile up across a genome scan.
- When you test one hypothesis, p < 0.05 keeps your false-alarm rate at about ____ percent (5 percent).
- But a problem appears the moment you test many hypotheses on the same data, so the flukes pile up; running 1,000,000 tests yields about 50,000 false alarms by chance.
2
The Bonferroni fix
- The divides your significance bar by the number of tests: new bar = 0.05 / (number of tests).
- For about a million common-variant tests, 0.05 / 1,000,000 lands near ____ x 10^-8 (5 x 10^-8); this is exactly where the threshold comes from, and the real European GWAS held its hits to it.
3
Two more tools
- The (FDR), controlled by Benjamini-Hochberg, is a less strict alternative that controls the expected fraction of reported hits that are false, used heavily in gene-expression work.
- The strongest check of all is replication: a true signal should reappear in a second independent group; a hit that clears the bar once but never replicates is treated with suspicion. (5 x 10^-8 is the standard but debated; some argue for an even tinier 5 x 10^-9.)
Explore
Reading the Research
What to read
Read the title and the abstract only, not the whole paper. Sun B, et al. 2023. Ezh2-dependent methylation in oral epithelia promotes secondary palatogenesis. Birth Defects Res. [PMID:37435868]
Why this source matters
This is the published evidence behind today's idea: Testing many hypotheses at once breeds false positives, so a genome scan must correct its threshold down to p < 5 x 10^-8 and demand replication.
Words to unlock first
multiple testingfalse positiveBonferroni correctiongenome-wide significancefalse discovery rate
Reading moves
- Skim the title and abstract first to get the gist.
- Circle the one sentence that states the main claim.
- Box the evidence the authors give for that claim.
- Mark one sentence that confuses you, and move on.
Stop point
You do not need the methods or statistics yet. If a sentence is about lab technique or math you have not learned, mark it and skip it.
Your output
Write one claim-evidence sentence: what this source claims, and the one piece of evidence that backs it up.
Where this fits
Tested on (Ohio WebXam)
Genetics of Disease · 072130
PLTW lesson
MI · Experimental Design domain · Statistics for research, multiple testing and correction
WebXam domain
Bio-Molecular Technology
Evidence to produce
Decide for four scanned SNPs whether each clears genome-wide significance (p < 5 x 10^-8) with one sentence per call (IRF6: p = 2 x 10^-9; chromosome 8: p = 3 x 10^-6; unknown gene: p = 0.0004; chromosome 16: p = 1 x 10^-8), then name the single study that would make a cleared hit more trustworthy.
Lab / skill
Biomedical Innovations (BI) · Medical Interventions (MI)
Explore
Vetted readings for this lesson
- Sun B, et al. 2023. Ezh2-dependent methylation in oral epithelia promotes secondary palatogenesis. Birth Defects Res. [PMID:37435868]
- van Rooij IALM, et al. 2019. Nonsyndromic cleft lip/palate: GWAS in Europeans identifies risk loci. Genes. [PMID:31817908]
- Park JW, et al. 2007. Association between IRF6 and nonsyndromic cleft lip/palate in four populations. Genet Med. [PMID:17438386]
The plan
Track your progress today
Check these off as you work through the lesson, then submit. This tells Mr. Mendoza how you're doing so he can help the class. It does not replace turning in your producible.
Use the code Mr. Mendoza gave you, not your name. Saved on this device.
Check off as you finish
- Read the Model and answered the Explore questions.
- Filled in the guided notes in my own words.
- Defined the new vocabulary with an example.
- Built the producible: Decide for four scanned SNPs whether each clears genome-wide significance (p < 5 x 10^-8) with one sentence per call (IRF6: p = 2 x 10^-9; chromosome 8: p = 3 x 10^-6; unknown gene: p = 0.0004; chromosome 16: p = 1 x 10^-8), then name the single study that would make a cleared hit more trustworthy.
- Wrote my Claim, Evidence, and Reasoning exit ticket.
Pick your period and code first.
Check yourself
Exit ticket (Claim, Evidence, Reasoning)
- Claim: A SNP reported at p = 0.001 in a genome-wide scan is not yet trustworthy.
- Evidence: A genome scan runs about ____ (a million) tests, and the accepted bar is p < ____ (5 x 10^-8).
- Reasoning: Because so many tests run at once, a p-value of 0.001 would be expected to occur by chance about ____ (1,000) times, so it does not clear the genome-wide bar.
How this is graded (rubric)
For: Decide for four scanned SNPs whether each clears genome-wide significance (p < 5 x 10^-8) with one sentence per call (IRF6: p = 2 x 10^-9; chromosome 8: p = 3 x 10^-6; unknown gene: p = 0.0004; chromosome 16: p = 1 x 10^-8), then name the single study that would make a cleared hit more trustworthy.
| Criterion | Proficient | Developing | Beginning |
|---|---|---|---|
| Complete | Every required part of the artifact is present and filled in. | Most parts are present, but one is missing or left blank. | Several parts are missing. |
| Accurate | The science and data are correct and match the evidence. | Mostly correct, with a small factual slip. | Key science or data is wrong. |
| Scientific reasoning (CER) | States a claim, backs it with specific evidence, and explains the reasoning. | Has a claim and evidence, but the reasoning is thin or missing. | Gives an answer with no evidence or reasoning. |
| Professional communication | Clear, organized, and labeled the way a clinician or scientist would write it. | Readable but disorganized or missing labels. | Hard to follow. |
| Submitted | Turned in the right way (Schoology for routine work) and confirmed. | Turned in, but in the wrong place or unconfirmed. | Not turned in. |
How the model answer scores against this rubric
- CompleteProficient: Nothing is left blank: the model fills every part of "Decide for four scanned SNPs whether each clears genome-wide significance (p < 5 x 10^-8) with one sentence per call (IRF6: p = 2 x 10^-9; chromosome 8: p = 3 x 10^-6; unknown gene: p = 0.0004; chromosome 16: p = 1 x 10^-8), then name the single study that would make a cleared hit more trustworthy.".
- AccurateProficient: Every number and claim matches the case evidence.
- Scientific reasoning (CER)Proficient: It names a claim, cites the specific evidence, and explains the reasoning, not just the answer.
- Professional communicationProficient: It is organized and labeled like a real chart note.
- SubmittedProficient: It would be turned in on Schoology and confirmed.
Explore
Where this leads: careers
Biostatistician Computational geneticist Data scientist
What's next: Gene studies use a tiny p-value because a million tests breed a million chances to be fooled, and replication seals the deal. But even a rock-solid statistical link never proves IRF6 does anything to . How would we leave the spreadsheet, go to the bench, and test what IRF6 actually does to a developing ?