Road to Rigor
Unraveling the Mind's Tendency: How Confirmation Bias Shapes Perception and Decision-Making.
Instructors
A raven wearing green sunglasses.
Konrad Kording
Professor
Meeting
Class
Book
A 1-hour interactive presentation.
Start meeting
A 3-hour interactive presentation. ANTON EDTING
Start Class
A 3-hour interactive textbook lesson.
Start Book

Unit Overview

What you will do
CMS TEST
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Fusce dapibus, tellus ac cursus commodo, tortor mauris condimentum nibh, ut fermentum massa justo sit amet risus.
Course highlights

Unit Lessons

(lessons marked with an * are included in the "Meeting" version of the unit)
1. Our biased brains
The Extent of the Problem
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
2. "Favored" vs. alternative hypotheses
Motive & Method
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
3. Researcher degrees of freedom
Solbing Rigor Problems
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
4. Mitigating bias through masking
The Toolbox
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
5. How good is your mask?
Power in Numbers
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
Solbing Rigor Problems
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
6. Analytical practices to mitigate bias
Power in Numbers
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
Solbing Rigor Problems
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
7. Data masking in machine learning models
Power in Numbers
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
Solbing Rigor Problems
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
8. Bonus biases that disrupt research
Power in Numbers
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
Solbing Rigor Problems
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
Our biased brains

This lesson introduces the concept of confirmation bias, highlighting how our brains tend to favor information that reinforces existing beliefs. Learn why these biases exist from both psychological and neuroscientific perspectives, and how they can inadvertently affect decision-making in research.
Motive & Method
This lesson focuses on the pitfalls of designing experiments that only test a single, “favored” hypothesis. Learn about and practice developing multiple, mutually exclusive hypotheses to challenge assumptions and reveal hidden biases.
Researcher degrees of freedom
This lesson explains how subjective decisions, ranging from data cleaning to statistical analysis, can unintentionally favor an expected outcome. Learn how “researcher degrees of freedom” can lead to overconfident or skewed results.
Mitigating bias through masking
This lesson introduces the concept of masking (or blinding) in experimental design. Learn how masking can prevent both participants and researchers from inadvertently influencing results with examples from clinical trials.
How good is your mask?
In this lesson, the focus shifts to the evaluation of masking effectiveness. Learn about factors that may inadvertently reveal experimental conditions and how to assess whether masking has been compromised.
How good is your mask?
This lesson introduces differences between exploratory and confirmatory data analysis. Learn how blending these approaches without clear distinction can lead to confirmation bias and discuss strategies to preserve the integrity of research findings.
How good is your mask?
This lesson draws parallels between experimental masking and data leakage in machine learning. Learn about the concept of data leakage and its impact on model evaluation, emphasizing the importance of proper data partitioning.
How good is your mask?
This final lesson broadens the discussion to other biases that can influence research, including conformity bias and cognitive dissonance. Learn how to tie these concepts back to confirmation bias, discovering how multiple biases can compound and affect scientific outcomes.
Unit Overview
Unit Overview
This unit is for learners who are expected to have already taken 2 to 3 other units and have a broad sense of what is meant by “rigor”, and what rigor solutions might look like.
Activities
minutes
The Extent of the Problem
The Extent of the Problem
Nullam id dolor id nibh ultricies vehicula ut id elit. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi leo risus, porta ac consectetur ac, vestibulum at eros. Donec id elit non mi porta gravida at. Content Identifying and solving rigor problems Reading 10 min Values & Ideology Short Activity 4 min How Widespread are the Issues? Short Activity 2 min Let’s put a face to this Video 6 min A Note on Hope Video 4 min Start Module
2
Activities
20
minutes
Motive & Method
Motive & Method
Solving Individual Rigor Problems 26 min Nullam id dolor id nibh ultricies vehicula ut id elit. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi leo risus, porta ac consectetur ac, vestibulum at eros. Donec id elit non mi porta gravida at. Content Identifying and solving rigor problems Reading 10 min Values & Ideology Short Activity 4 min How Widespread are the Issues? Short Activity 2 min Let’s put a face to this Video 6 min A Note on Hope Video 4 min Start Module
1
Activities
10
minutes
Solving Individual Rigor Problems
Solving Individual Rigor Problems
Solving Individual Rigor Problems 26 min Nullam id dolor id nibh ultricies vehicula ut id elit. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi leo risus, porta ac consectetur ac, vestibulum at eros. Donec id elit non mi porta gravida at. Content Identifying and solving rigor problems Reading 10 min Values & Ideology Short Activity 4 min How Widespread are the Issues? Short Activity 2 min Let’s put a face to this Video 6 min A Note on Hope Video 4 min Start Module
0
Activities
12
minutes