Summary:

This unit on randomization explores the critical role that randomization plays in experimental design, examining how proper randomization techniques can reduce bias and improve study validity. By understanding how to implement randomization correctly at every stage of research, students will learn essential strategies to design more rigorous, transparent, and reproducible studies. This unit is ideal for early-career researchers and advanced students who wish to strengthen their methodological skills and safeguard against experimental bias.

Why use this unit:

• This unit on randomization equips students with a fundamental understanding of how randomization helps to contain bias and strengthen causal inference in research.
• Each lesson blends theoretical insights with practical activities, ensuring that learners not only recognize the importance of randomization but can also implement various randomization strategies in their work.
• Real-world examples and interactive activities encourage students to reflect on their own research practices and develop more robust experimental designs.

‍

Lesson summary:

This lesson introduces the fundamental concept of randomization through an example study of lithium treatment in an ALS mouse model. Students learn how non-random assignment led to confounding between disease stage and treatment groups, compromising the study's conclusions.

Key takeaway:

Students should come away understanding that without proper randomization, confounding variables (both known and unknown) can invalidate experimental conclusions, making it impossible to determine if observed effects are "real."

Activity overview: (~5 minutes)

Students analyze experimental data to identify potential relationships between treatment assignment, survival, and other variables in the ALS mouse model example. They'll discover how disease stage became a confounding variable when mice were selected based on ease of catching.

Link to activity:

https://smi-ran-why-ran-v4.vercel.app/

Step-by-Step activity instructions:

1. [Instructor] Direct students to open the activity link to analyze the experimental data from the ALS mouse study.
2. Students are assigned different variables to explore their relationships with treatment assignment and survival.
3. Encourage students to look for patterns that might explain the apparent treatment effect.
4. Guide students to discover that mice assigned to the lithium treatment group were predominantly in earlier disease stages (longer "time to fall" on rotarod).
5. Discuss how this confounding occurred because mice in earlier disease stages were harder to catch, so they were more likely to be assigned to treatment after easier-to-catch mice were assigned to control.
6. [Instructor] Lead a discussion about how proper randomization would have distributed mice of all disease stages evenly between treatment groups.

Activity takeaway:

Students should recognize how non-random assignment created systematic differences between treatment groups that confounded the results, making it impossible to know if the treatment effect was real.

‍

Lesson summary:

This lesson examines common misconceptions about randomization and reveals how methods that seem random may actually introduce systematic bias. Students learn to distinguish between true randomization and other allocation approaches like alternation or manual allocation.

Key takeaway:

Students should understand that truly random allocation sequences appear different from human-generated "random" sequences, and that proper randomization implementation is critical for eliminating bias.

Activity #1 overview: (~5 minutes)

Students will examine three different allocation sequences, all claimed to be "randomized" in published studies, and determine which one represents true randomization.

Link to activity:

https://smi-ran-all-seqa-v1.vercel.app/

Step-by-Step activity instructions:

1. [Instructor] Ask students to open the activity link and examine the three different allocation sequences.
2. Have students analyze each sequence to determine which one represents true randomization.
3. Discuss the characteristics of each sequence (pattern, balance, runs of the same value).
4. Reveal to students that Sequence 3 shows genuine randomization, despite having runs and imbalances that might seem "non-random" to human intuition.
5. Point out that Sequence 1 (perfect alternation) and Sequence 2 (human-generated) both contain patterns that make them predictable.
6. [Instructor] Discuss real-world consequences of non-random allocation shown in examples from the lesson.

Activity takeaway:

Students should recognize that their intuitions about what "looks random" may be incorrect, and understand the key features that distinguish true randomization from other allocation methods.

Activity #2 overview: (~3 minutes)

Students will examine three different allocation sequences, alternation, manual allocation, and simple randomization, and compare patterns in the effect size, p-values, and balance between treatment groups.

Link to activity:

https://smi-ran-all-seqb-v1.vercel.app/

Step-by-Step activity instructions:

7. [Instructor] Ask students to open the activity link and examine the three different allocation sequences.
8. Have students analyze each sequence to determine how the effect size, p-values, and group balance patterns differ.
9. Discuss the characteristics of each sequence - are there concerns about how they differ from the true effect?
10. What patterns for these sequences differ at scale - are long runs in the simple randomization sequence as prominent? 
11. Do the students believe that a larger N will reduce bias for the alternating and manual allocations?
12. [Instructor] Discuss real-world consequences of non-random allocation shown in examples from the lesson.

Activity takeaway:

Students should recognize that alternation and manual allocation can lead to bias - inflated effect sizes or false positive findings. They might also identify that the impact of the long runs they noted in activity 1 seem to be less impactful at a higher scale. 

‍

Lesson summary:

This lesson guides students through the decision-making process for selecting the most appropriate randomization method based on study characteristics. It introduces three main approaches: simple randomization, block randomization, and stratified randomization.

Key takeaway:

Students should learn to assess their study design, sample size, and potential sources of variation to select the optimal randomization approach for their specific research context.

Activity overview: (~5 minutes)

Students will work through a flowchart to determine which randomization method is most appropriate for their research or a case study.

Link to activity:

https://smi-ran-ran-flo-v1.vercel.app/

Step-by-Step activity instructions:

1. [Instructor] Introduce the study options: students either use a study of their own, a study they know well, or use the case study.
2. Have students consider the study characteristics and potential sources of bias for their chosen study.
3. Guide students through the flowchart, answering questions about sample size, important covariates, or sources of bias..
4. Based on their answers, students will arrive at a recommendation for which randomization approach is most appropriate.
5. [Instructor] Facilitate a discussion about why the recommended method is suitable for the specific study characteristics.

Activity takeaway:

Students should understand the factors that influence the choice of randomization method and be able to apply this decision framework to their own research designs.

‍

Lesson summary:

This lesson explores simple randomization, which is the most basic randomization technique and is analogous to flipping a coin for each subject. Students learn about the benefits and limitations of simple randomization, particularly in smaller studies.

Key takeaway:

Students should understand how simple randomization provides the highest level of unpredictability but may result in unbalanced group sizes, especially in smaller studies.

Activity #1 overview: (~3 minutes)

Students will simulate the outcomes of simple randomization using a roulette wheel analogy to see the probability of achieving balanced group sizes with 24 mice.

Link to activity:

https://smi-ran-rou-whe-v0.vercel.app/

Step-by-Step activity instructions:

1. [Instructor] Have students open the activity link to simulate spinning a roulette wheel 24 times (representing 24 mice in the study).
2. Guide students to observe how many "black" spins they get out of 24, representing one treatment group.
3. Repeat the simulation multiple times to show the variation in outcomes.
4. Discuss the probability of getting exactly 12 black spins (balanced groups) versus more unbalanced outcomes.
5. Extend to simulations with larger sample sizes (100+) to demonstrate the law of large numbers.
6. [Instructor] Lead discussion about what problems the students predict for studies with uneven group sizes.

Activity takeaway:

Students should understand that simple randomization cannot guarantee balanced group sizes, especially in smaller studies, which is an important consideration when choosing a randomization method.

Activity #2 overview: (~5 minutes)

Students will determine which research tasks should be handled by masked versus unmasked team members to maintain the integrity of randomization in the NMDA receptor antagonist study.

Link to activity:

https://smi-ran-simple-ran-v1.vercel.app/

Step-by-Step activity instructions:

1. [Instructor] Introduce the concept of masking (blinding) in research and explain the two teams:
   
   • Team A (Access): Has access to treatment allocation information
   • Team B (Masked): Remains masked to which mouse receives which treatment
2. Direct students to the activity where they'll assign various research tasks to the appropriate team.
3. Have students work individually or in small groups to consider each task and the potential biases that could arise if performed by the wrong team.
4. For each task, students should:
   
   • Consider the type of bias it might introduce (selection, performance, observer)
   • Decide which team should handle it to minimize bias
   • Provide reasoning for their choice
5. Key tasks students will assign include:
   
   • Generating the random treatment allocation sequence
   • Preparing coded syringes with drugs or saline
   • Assessing and recording maze performance of mice
6. [Instructor] After completion, facilitate a discussion about why certain tasks must be performed by specific teams:
   
   • Why must Team A prepare the syringes? (to maintain masking)
   • Why must Team B conduct behavioral assessments? (to prevent observer bias)
   • Why separation of duties is crucial even in small lab settings
7. Discuss practical implementation challenges researchers might face in maintaining proper masking procedures.

Activity takeaway:

Students should understand that proper masking requires thoughtful assignment of research tasks to different personnel. They should recognize that effective randomization benefits are preserved only when combined with appropriate masking procedures that prevent selection, performance, and observer biases from influencing the results.

‍

Lesson summary:

This lesson introduces block randomization as a method to ensure balance in group sizes throughout the study. Students learn how blocking by time, space, or procedural factors can mitigate various sources of environmental variation.

Key takeaway:

Students should understand how block randomization creates "mini-experiments" that maintain balance between treatment groups while controlling for temporal, spatial, or procedural sources of variation.

Activity overview: (~3 minutes)

Students will practice generating block randomization sequences and understand how different block sizes and sample sizes impact treatment allocation.

Link to activity:

https://smi-ran-blk-ran-v4.vercel.app/

Step-by-Step activity instructions:

1. [Instructor] Guide students to open the activity link.
2. Have students experiment with different parameters:
   
   • Number of treatment groups
   • Block size
   • Target sample size
3. Encourage students to observe how the generated sequences maintain balance within each block.
4. [Instructor] Lead a discussion about how small versus large block sizes or small versus large sample sizes might impact balance or predictability.

Activity takeaway:

Students should understand how to implement block randomization and recognize its benefits for controlling environmental variation while maintaining balanced group sizes.

‍

Lesson summary:

This lesson focuses on stratified randomization as a technique to balance important covariates across treatment groups. Students learn how stratification can minimize the influence of known prognostic factors on study outcomes.

Key takeaway:

Students should understand how stratified randomization ensures balance of important baseline characteristics across treatment groups, improving the precision of treatment effect estimates.

Activity overview: (~5 minutes)

Students will explore different stratification approaches for a zebrafish aggression study and see how various stratification choices impact the estimated treatment effect.

Link to activity:

https://smi-ran-str-ran-v1.vercel.app/

Step-by-Step activity instructions:

1. [Instructor] Have students open the activity link to explore stratification strategies for the zebrafish study.
2. Students randomize the zebrafish using a block randomization approach to create two equally sized treatment groups.
3. Guide students to consider which variables (sex, family ID, size, swimming speed) are most important to balance.
4. Students play with stratification choices and pick one they believe is the most appropriate approach.
5. Have students explore outputs showing how different stratification approaches affect the study results.
6. [Instructor] Lead a discussion about the trade-offs between not stratifying and overstratification. Guide them to select a stratification choice they think strikes a balance.

Activity takeaway:

Students should understand how to identify and prioritize stratification variables, recognize the dangers of overstratification, and appreciate how stratification affects treatment effect estimates.

‍

Lesson summary:

This lesson expands the concept of randomization beyond just treatment allocation to encompass all aspects of experimental design. Students learn how randomizing time-based, space-based, and personnel-based factors can reduce various sources of bias.

Key takeaway:

Students should understand that comprehensive randomization strategies should address temporal, spatial, and personnel sources of variation to strengthen study validity.

Activity overview: (~5 minutes)

Students will identify additional opportunities for randomization in a spatial memory study with mice, beyond treatment allocation.

Link to activity:

https://smi-ran-ran-lab-v0.vercel.app/

Step-by-Step activity instructions:

1. [Instructor] Have students read about the spatial memory in mice study and review the study flowchart.
2. Guide students to identify potential time effects, space effects, and personnel effects that could influence study outcomes.
3. Encourage students to brainstorm randomization strategies to address each source of variation.
4. Guide students to prioritize their randomization possibilities according to how impactful the randomization step would be in countering bias and how challenging the randomization step would be to implement.
5. Compare their answers to the rest of the class and discuss practical implementation challenges and how to prioritize randomization efforts.
6. [Instructor] Lead a discussion about balancing methodological rigor with practical constraints in laboratory settings.

Activity takeaway:

Students should recognize that randomization extends beyond treatment assignment and develop strategies to implement comprehensive randomization in their own research.

‍

Lesson summary:

This final lesson examines common pitfalls in how randomization methods are reported in published research. Students learn about reporting guidelines and best practices for transparent documentation of randomization procedures.

Key takeaway:

Students should understand the importance of detailed reporting of randomization techniques for transparency, reproducibility, and proper evaluation of research quality.

Activity overview: (~5 minutes)

Students will evaluate methods sections from published papers and identify missing or inadequate details about randomization procedures.

Link to activity:

https://smi-ran-ran-lit-v1-ten.vercel.app/

Step-by-Step activity instructions:

1. [Instructor] Have students open the activity link to review published methods sections.
2. Guide students to identify methodology questions they have for the authors about their randomization procedures.
3. Have students drag and drop their questions to the corresponding buckets that relate to the reporting items recommended by the ARRIVE Guidelines.
4. Review the improved methods sections that now include all items from ARRIVE. 
5. [Instructor] Have students discuss which missing items were most important to their interpretation of the study. Discuss how inadequate reporting affects the ability to evaluate study quality and reproducibility.
6. [Instructor] Introduce established reporting guidelines like ARRIVE and CONSORT that set standards for thorough documentation of randomization.

Activity takeaway:

Students should recognize common deficiencies in randomization reporting and learn to document their own randomization procedures thoroughly for transparency and reproducibility.

‍

Concepts likely to challenge students:

• Understanding that true randomization can produce sequences that don't "look random" to human intuition, including runs and imbalances.
• Distinguishing between different randomization methods and selecting the most appropriate one for a specific research context.
• Recognizing the need to account for blocking and stratification in statistical analysis.
• Implementing comprehensive randomization strategies that address all potential sources of bias beyond just treatment allocation.
• Balancing methodological rigor with practical constraints in laboratory settings.
• Think we got it wrong? We want to improve! Email us at c4r@seas.upenn.edu.

Key terms to emphasize:

• Confounding variables/confounders
• Selection bias
• Performance bias
• Observer bias
• Simple randomization
• Block randomization
• Stratified randomization
• Allocation concealment
• Overstratification
• Masking/blinding

Final reminder for instructors:

This guide is intended as a flexible framework for presenting the Randomization unit. Instructors are encouraged to adapt the content to best suit their teaching style and the needs of their students. Feel free to expand on any section, incorporate additional examples, or integrate further interactive elements. Remember, this is your presentation: use it as a starting point and customize it to best serve your teaching.

‍