You want your research design to control for alternative explanations as much as possible so that you can draw usable and valid conclusions about cause and effect. Randomization is a tool of effective experimental design that assigns subjects to specific treatment groups and balances a variety of factors that could interfere with your results, like confounding and disturbance variables.

‍

The activity in this lesson gives you an opportunity to tinker with a dataset. You’ll be asked about relationships you might see between variables and to verify if the treatment effects shown in the plot still render as true after controlling for different confounding variables. You’ll also be able to see how results differ in the absence of randomization controls. Once you complete it, you will have the opportunity to discuss with your lab about how to make certain your results are true in future experiments.

‍

Eating more ice cream cannot actually increase the number of shark attacks in July. When those numbers rise in tandem, the reality is that summer in the northern hemisphere means more people go to the beach. The beach is where ice cream and shark attacks are on offer. It’s obvious, intuitive even, that the confounding variables here include temperature, time off from work or school, and a search for a good time. 

‍

What’s less obvious is how to control for interference in situations where the variables are murkier and the stakes for accurate results are much higher. For instance, in studies on heart disease, we’ll need to account for age as well as a history of smoking to understand the full picture. 

So what are these other variables, what do they do exactly, and how do you recognize them? If an interfering factor is related to the independent variable (your treatment) and affects the dependent variable (a specific outcome), you’re looking at a confounding variable. If an interfering factor is unrelated to the independent variable (and hence not confounded with the treatment), but still affects the dependent variable (outcome), it represents a disturbance variable.

Both of these interfering factors pose severe risks for the validity of your study and whether you can trust your results by introducing bias and reducing precision. If you have more women than men in your sample, for instance, the confounding/disturbance variable can either obscure any relationship that does exist between the independent and dependent variables or suggest that there is a relationship that isn’t there. Thus, when these factors are in play, you cannot properly attribute cause and effect. Similarly, these factors can often trick you into over- or underestimating the effects shown in your results.

‍

Knowing that randomization is important to mitigating this interference, you might look at the study you’ve designed and be tempted to do one of three things. Hint, these three things will introduce bias into your results:

1. Using a dice roll, coin flip, or similar to “randomly” select your treatment group (this risks groupings with confounding variables that you’re not seeing).
2. Assigning treatment groups in a systematic order like alternating assignments daily (this way leads to chaos because if participants are assigned treatments based on the day they arrive, certain days might have different types of participants with various confounding variables).
3. Attempting to balance the trial by deciding, for example, that the first three groups you’ve sorted are the control and the fourth will receive the drug treatment (meh, this is kind of sloppy, doesn’t really counter any confounding or disrupting variables, and it also introduces potential order effects that could influence the experimental outcome, we fear).

‍

These methods all have the potential to introduce bias as you assign subjects to treatment groups. What you really need is a method of randomization that you select before starting your experiment, which removes human error from the process. No vibes, only methodical processes.

‍

In our first activity, explore the relationships between the variables in the study to see if your lab can discover whether or not the results are true. This exploration shows what happens when you don’t prepare ahead of time, but, more importantly, it asks you to decipher relationships between variables as shown in a plot. Doing so will help you practice scanning your own results even if you did your best to account for confounding variables.

Randomization is a seemingly simple concept: just assign people (or more generically, “units” [e.g., mice, rats, flies, classrooms, clinics, families]) randomly to one treatment or intervention versus another. The importance of randomization may have been first recognized at the end of the nineteenth century, and formalized in the 1920s. Yet, since its inception, there have been errors in the implementation or interpretation of randomized experiments. (Vorland et al. 2021)

‍

Examining the consequences of not controlling for potential confounders is a crucial element of experimental design. There are situations, however, where researchers believe they are employing randomization correctly, but don’t stick the landing.

‍

A meta-review of problems with the implementation, analysis, and reporting of randomization within obesity and nutrition research (Vorland et al. 2021) identified 10 errors in randomized experiments:

‍

• Errors in implementing group allocation:some text
  • Representing nonrandom allocation methods as random.
  • Failing to adequately conceal allocation from investigators.
  • Not accounting for changes in allocation ratios.
  • Replacements are not randomly selected.
• Errors in the analysis of randomized experiments:some text
  • Failing to account for non-independence.
  • Basing conclusions on within-group statistical tests instead of between-groups tests.
  • Improper pooling of data.
  • Failing to account for missing data.
• Errors in the reporting of randomization:some text
  • Failing to fully describe randomization.
  • Failing to properly communicate inferences from randomized studies.

‍

The primary counterbalance the authors propose to lapses like this is to prepare ahead of time and collaborate with experts like biostatisticians or other researchers with methodological expertise to ensure correct experimental design.

‍

When errors are discovered, authors and editors have a responsibility to correct the scientific record, and journals should have procedures in place to do so expeditiously. The severity of the error, ranging from invalidating the conclusions to simply requiring clarification, means that different considerations exist for each type of error. (Vorland et al. 2021) 

‍

We’d rather not find ourselves in a position where errors invalidate our results. We want to avoid the retraction of a publication, a costly redesign, or worse – not being corrected and building a false foundation for other scientists. Conscientious and thorough experimental design, supported by randomization and other measures to deter false results, can help us.The rest of this unit will explore solutions to all of these things–woo!

1. Randomization is powerful because it mitigates the effects of confounding variables, which distort your study results.

2. We can minimize the risk of confounders by assigning subjects to treatment groups before we conduct an experiment.

3. Failing to randomize properly can lead to systematic differences between groups that are unrelated to the treatment.