Summary and Schedule
This is a new lesson built with The Carpentries Workbench.
| Setup Instructions | Download files required for the lesson | |
| Duration: 00h 00m | 1. What is multiple testing |
What is multiple testing? How are false-positives related to the significance level? |
| Duration: 00h 15m | 2. Types of errors and error rates |
What are false positives and false negatives and how do they manifest in
a confusion matrix? What are some of real examples where false positives and false negatives have different implications and consequences? What are type I error rates? :::::::::::::::::::::::::::::::: |
| Duration: 00h 25m | 3. Family-wise error rate |
What is the family-wise error rate (FWER), and why is it important in
multiple testing scenarios? How does the Bonferroni procedure adjust p-values to control the FWER, and what are its limitations? :::::::::::::::::::::::::::::::: |
| Duration: 00h 45m | 4. False discovery rate |
How does correcting for the family-wise error rate (FWER) affect the
number of significant hits in large-scale data, such as RNA-Seq analysis
of 20,000 human genes? What is the interpretation of a p-value histogram? How can the Benjamini-Hochberg method be applied to control the false discovery rate (FDR) in RNA-Seq data, and what are the benefits of using this method over FWER correction? |
| Duration: 01h 10m | 5. Pairwise comparisons |
What are pairwise comparisons, and how do they relate to the broader
concept of multiple testing in statistical analysis? How can we effectively conduct and interpret pairwise comparisons to make valid statistical inferences while controlling for the family-wise error rate? |
| Duration: 01h 32m | 6. Summary | |
| Duration: 01h 35m | 7. References and Further Reading | |
| Duration: 01h 35m | Finish |
The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.
Authors
Sarah Kaspar
Carolyn Mukiri Kambona
Summary
At the end of this course, you will be able to:
- comprehend the concept of multiple testing and its implications, including the increased likelihood of observing low p-values by chance.
- be able to differentiate between different types of errors, (false positives and false negatives) and understand their significance in various research contexts.
- understand the concept of family-wise error rate (FWER) multiple testing scenarios and grasp methods like the Bonferroni procedure for controlling it.
- gain proficiency in interpreting adjusted p-values resulting from FWER correction and applying them in statistical analyses like ANOVA and post-hoc tests.
- appreciate the challenges posed by high-throughput data analysis and understand the importance of controlling FDR using methods like the Benjamini-Hochberg procedure.
- apply the concepts learned to real-world scenarios, such as analyzing RNA-Seq data by selecting appropriate error rates and methods based on their research questions.
Tools Required
- Statistical software R (download)
- Basic knowledge of programming in R is beneficial for understanding the coding examples.
Data Sets
For all example data in this lesson, links for downloading them are provided during the episodes, which will allow you to follow along with coding.
Preparation
For the coding exercises, you’ll need internet connection for loading
libraries and data. Ensure that R is installed and functional on your
system. Familiarize yourself with basic statistical concepts such as
hypothesis testing, and p-values as these will provide a foundation for
understanding multiple testing procedures.
Recommended prior learning: