Why Design of Experiments (DOE) is important for biologists

If you pushed us for a quick answer explaining why Design of Experiments (DOE) is so important for the life sciences, it would be this: 

Design of Experiments (DOE) is a time- and resource-efficient way for researchers to gain better, more reproducible, and robust insights. Crucially, the statistical framework surpasses the limitations of OFAT (methods where you analyze one factor at a time). It does this by helping you explore more of your possible experimental design space, and see how your factors interact—which helps you avoid identifying the wrong system state as optimal. DOE’s applications extend throughout the life sciences and beyond.

If you’re after a more thorough answer, stay with us and we’ll delve deeper into the 6 reasons why DOE is so beneficial for those working in the life sciences.

Curious to learn more, but wanted to get clued up on DOE first? Learn about:

• What Design of Experiments (DOE) actually is
• The DOE process, and get a step-by-step guide to how it works

6 important reasons why Design of Experiments (DOE) is beneficial for the life sciences

1. You're much more likely to identify the "correct" result

Many life scientists still try to use a simple OFAT method to answer complex biological questions. But, by definition, OFAT limits the size of the possible experimental design space that researchers can explore. This is because OFAT doesn’t consider the interactions between the complex, emergent, and divergent factors that characterize biological networks.

Which means, as Figure 1 shows, that by using OFAT researchers risk identifying a sub-optimal state that is some way from the "correct" one.

Figure 1: With OFAT (left) effects are easy to distinguish but there is no information about how factors interact. Design of Experiments (right) uses statistical techniques to explore combinations of factor settings and uncover the optimal results that probably would have been missed using OFAT

Meanwhile, Design of Experiments helps researchers identify the optimal state by offering a suite of techniques to design and analyze experiments that unpick complex systems or processes. In essence, it's a statistical framework empowering researchers to investigate the impact of changing multiple factors simultaneously on an experimental process—and importantly, explore their interactions.  

For example, let's say we wanted to consider the effect of 2 factors: The amount of a particular fertilizer and the strain on wheat growth. The optimal amount of fertilizer required may be different for each strain. If we don't investigate different combinations systematically, we'll end up under- or over-estimating effects. We may, for example, accidentally use too much or too little fertilizer for one strain but an optimal value for the other.

So, we could conclude that one strain does not give good yields. The statistical analyses that form the basis of DOE can help researchers avoid similar mistakes by clearly distinguishing real effects and interactions from noise.

2. It's great for tackling complex systems without bias

The empirical nature of DOE is useful for biologists who face the challenge of working with enormously complex systems, often without the benefit of well-developed theoretical frameworks. 

Physics, on the other hand, has theoretical frameworks to guide experimentation. For example, quantum mechanics predicted the Higgs Boson decades before CERN found the particle.¹

Biologists do not have the benefit of theoretical frameworks as robust as that. So, life scientists are forced to make seemingly arbitrary decisions about their focus of study and influential factors.

This can result in unconscious cognitive bias: it’s all too easy to develop OFAT experiments that confirm hypotheses.²

DOE helps avoid unconscious cognitive bias and allows you to take a holistic approach when formulating and testing your hypotheses. So, you can be more confident in your data than with OFAT approaches.

3. It gives you more confidence in your data 

DOE is a holistic and statistical approach to studying biological systems that can account for every component and their interactions. This helps scientists gain better, more reproducible and robust insights than is possible with OFAT.

Why? Because DOE encourages a broader, more structured mindset. By running a DOE, you'll have systematically investigated lots of factors and more points in your space than you would with OFAT. Plus, you'll have used statistical techniques to justify your inferences not just once, but multiple times, as iterating requires you to have tested previous inferences before exploring further. It's also partly because all these things make your experimentation more efficient. In essence, you're getting more valuable data in fewer experiments.

4. You get better value out of your negative results

Scientists are supposed to accept their negative results with the same grace as their posive results. But it’s hard to shake the feeling that negative results were a waste of time.

Regardless of whether you get the result you expected, DOE means that you learn more about the space you are investigating than with OFAT. So, you can turn those negative results to your benefit as you design the next step in your study.

You can, for example, use DOE to run screening designs to get a sense of “the lay of the land”. Design of Experiments screening allows you to identify factors that may have a material impact on a response and discard those that have little to no impact.

Screening designs, such as fractional factorials, give you a lot of statistical bang for your experimental buck by carefully balancing the sets of conditions tested. So, with relatively few runs you can get much better estimates of which effects are real without interference from previously unknown interactions than with OFAT approaches. In contrast to OFAT’s more ad hoc approach, systematically exploring your system gives you the confidence to say that the factors that didn’t have an effect were not just artifacts. 

On top of that, with DOE it's easy to quickly identify whether nothing is going to work, and decide to move on. Because sometimes, there's nothing to optimize. Whereas with OFAT, you can waste a lot of time and money throwing things at the wall and hoping to find something that sticks.

5. It's time- and resource-efficient

Almost 100 years ago, Fisher noted that “large and complex experiments have a much higher efficiency than simple ones.”³ And DOE is the perfect poster child for his point: It's both time- and resource-efficient.

For instance, using initial screening experiments to identify the most important factors saves time and resources. You can also use DOE with laboratory automation to increase your throughput. Design of Experiments reduces the time, materials, and experiments needed to yield a given amount of information compared with OFAT. 

Though if you choose to go down the automation route, naturally questions of integrations between your DOE software and the software controlling your lab automation come into play. The shift toward fully automated liquid handling experimentation requires either an automation engineer's specialist scripting or coding knowledge. The good news is that cloud-based DOE software has become more user-friendly and accessible,  allowing you to perform sophisticated DOEs on liquid handling automation with zero coding knowledge.

6. You're more likely to achieve your scientific goals

DOE also achieves higher precision and reduced variability when estimating the effects of each factor and their interactions than using OFAT.

For example, let's say you're modeling response surfaces based on a sequence of designed experiments. DOE helps you optimize outcomes by exploring a multidimensional design space of factors. 

This allows you to identify the ideal convergence of factors that shows us where the peak is on a ‘topographical map’ (Figure 2). If the design space is a mountain, the optimum response is the peak. But it’s easy to miss the peak using OFAT, while using DOE allows you to enjoy the experimental view. Analyzing interactions allows you to identify when you’re on a high ‘plateau’ rather than a single peak. These plateaus are more robust to variation, but, because they are revealed only by considering interactions, remain hidden by analyzing OFAT. Visualizations such as ‘topographical maps’ also help communicate the results to colleagues.

In other words? DOE allows you to build on a secure statistical foundation that increases the likelihood that your experiments will produce positive results. As Fisher told the Indian Statistical Congress during the 1930s: “To call in the statistician after the experiment is done may be no more than asking him to perform a postmortem examination: he may be able to say what the experiment died of”.⁴

Figure 2: Response surface methodology, not unlike a topographical map of a landscape

The best way to understand DOE's benefits is to dive right in

We hope this goes some way to convince you of OFAT’s limitations—and how much more powerful and efficient experimentation in the life sciences could be with DOE. 

If DOE’s statistical nature is still putting you off, we promise it’s not as hard as you might think it is. 

The only way to truly appreciate the valuable insights that DOE can bring is to get stuck in. The proof is in the pudding, as they say… give it a go and you won’t look back.

Interested in learning more about DOE? Make sure to check out our other DOE blogs, download our DOE for biologists ebook, or watch our DOE Masterclass webinar series.

References

1. Roa, A The Higgs boson: What makes it special? https://home.cern/news/series/lhc-physics-ten/higgs-boson-what-makes-it-special Accessed October 2022
2. Lendrem DW, Lendrem BC, Woods D et al. Lost in space: design of experiments and scientific exploration in a Hogarth Universe. Drug Discovery Today 2015;20:1365-1371
3. Fisher, RA The arrangement of field experiments. Journal of the Ministry of Agriculture 1926;33:503-515
4. Geraci M and Pearce A Transparency in a world of complexity: Basic guidelines for improved statistical reporting. The Journal of Child Health Care 2016;20:3-4.