Why should I use Design of Experiments in the Life Sciences?

• Methods analyzing one factor at a time (OFAT) limit the size of the possible experimental design space and, therefore, often identify the wrong system state as optimal
• Design of Experiments (DOE) is a statistical framework that empowers researchers to investigate the impact of multiple factors on an experimental process and to simultaneously explore interactions
• Design of Experiments enables researchers to gain better, more reproducible, and more robust insights, without previous programming knowledge
• Design of Experiments is time and resource-efficient
• This blog introduces some reasons why life scientists should consider using DOE instead of OFAT
• If you’d like to keep learning about DOE after you're done with this article, make sure to check out our other DOE blogs, download our DOE for biologists ebook, or watch our DOE Masterclass webinar series.

In 1843, pioneering experiments began in Rothamsted a few miles north of London.¹ An agriculturist and chemist aimed to determine the effect of various factors—which artificial fertilizer to apply, when and the quantity, optimal crop rotation, the best harvesting schedule, and so on—on the growth of wheat, barley, potatoes, and other crops.^1,2 Researchers have followed some plots at Rothamsted for more than 170 years making this, Jonathan Losos notes, “the longest continuously running experiment in the history of science.”¹ 

Understanding and unraveling the complex interplay of mutually reinforcing factors that drive plant growth poses a problem. But it’s one we can now solve using a statistical framework known as Design of Experiments (DOE). And DOE has applications throughout the life sciences and beyond—as we’ll see in future blogs.

Better experimentation

In our last DOE Foundations blog, we met the eminent statistician Ronald Fisher at a tea party in the 1920s, when he devised a method to determine if a lady really could taste whether tea or milk was poured into the cup first. Fisher worked at Rothamsted and recognized that varying one factor at a time (OFAT) wasn’t the best way to identify the combination of factors that optimizes yield. 

“No aphorism is more frequently repeated in connection with field trials, than that we must ask Nature few questions, or ideally, one question at a time,” Fisher wrote in a paper published in 1926. He added that “this view is wholly mistaken.”² “Nature,” Fisher pointed out, “will best respond to a logical and carefully thought out questionnaire; indeed, if we ask her a single question, she will often refuse to answer until some other topic has been discussed.”² 

Missing the point

Yet, almost a century later, many life scientists still try to ask single nature questions by using OFAT. But, by definition, OFAT limits the size of the possible experimental design space that researchers can explore. Critically, OFAT doesn’t consider the interactions between the complex, emergent and divergent factors that characterize biological networks. As figure 1 shows, considering OFAT can identify an optimal state that is some way from the ‘correct’ value. 

Figure 1: With OFAT (left) effects are easy to distinguish but there is no information about how factors interact. Design of Experiments (right) uses statistical techniques to explore combinations of factor settings and uncover the optimal results that probably would have been missed using OFAT

An empowering framework

Design of Experiments is a statistical framework that empowers researchers to investigate the impact of changing multiple factors simultaneously on an experimental process and, importantly, explore the interactions. Design of Experiments offers a suite of techniques that allows researchers to design and analyze experiments that unpick complex systems or processes. 

For example, we may want to consider the effect of two factors, the amount of a particular fertilizer and the strain, on wheat growth. The optimal amount of fertilizer required may be different for each strain, leading to effects being under or over-estimated if the different combinations are not systematically investigated. We may, for example, accidentally use too much or too little fertilizer for one strain but an optimal value for the other. So, we could conclude that one strain does not give good yields. The statistical analyses that form the basis of DOE can help researchers avoid similar mistakes by clearly distinguishing real effects and interactions from noise.

The challenge of enormously complex systems

The empirical nature of DOE is useful for biologists who face the challenge of working with enormously complex systems, often without the benefit of well-developed theoretical frameworks. 

Physics, on the other hand, has theoretical frameworks to guide experimentation. For example, quantum mechanics predicted the Higgs Boson decades before CERN found the particle.³ Biologists do not have the benefit of theoretical frameworks as robust as that. So, life scientists are forced to make seemingly arbitrary decisions about their focus of study and influential factors. This can result in unconscious cognitive bias: it’s all too easy to develop OFAT experiments that confirm hypotheses.⁴ Design of Experiments helps avoid unconscious cognitive bias and allows you to take a holistic approach when formulating and testing your hypotheses. So, you can be more confident in your data than with OFAT approaches.

Confidence in your data

So, DOE is a holistic and statistical approach to studying biological systems that can account for every component and their interactions. For instance, DOE can optimize biological assays, enabling scientists to gain better, more reproducible and more robust insights than is possible with OFAT. Cloud-based, user-friendly software enables scientists to easily perform sophisticated DOE, even if they don’t have much programming knowledge.

Value your negative results

Scientists are supposed to accept their negative results with the same grace as their positive results. But it’s hard to shake the feeling that negative results were a waste of time. Regardless of whether you get the result you expected, DOE means that you learn more about the space you are investigating than with OFAT. So, you can turn those negative results to your benefit as you design the next step in your study.

You can, for example, use DOE to run screening designs to get a sense of “the lay of the land”. Design of Experiments screening allows you to identify factors that may have a material impact on a response and discard those that have little to no impact. Screening designs, such as fractional factorials, give you a lot of statistical bang for your experimental buck by carefully balancing the sets of conditions tested. So, with relatively few runs you can get much better estimates of which effects are real without interference from previously unknown interactions than with OFAT approaches. In contrast to OFAT’s more ad hoc approach, systematically exploring your system gives you the confidence to say that the factors that didn’t have an effect were not just artifacts. 

Resource and time efficiency

Almost 100 years ago, Fisher noted that “large and complex experiments have a much higher efficiency than simple ones.”² And so it proved: DOE is time and resource-efficient. Using initial screening experiments to identify the most important factors saves time and resources. You can also use DOE with laboratory automation to increase your throughput. Design of Experiments reduces the time, materials, and experiments needed to yield a given amount of information compared with OFAT. 

In addition to these savings, DOE achieves higher precision and reduced variability when estimating the effects of each factor and their interactions than using OFAT. For example, modeling response surfaces based on a sequence of designed experiments optimizes outcomes by exploring a multidimensional design space of factors. 

Design of Experiments allows you to identify the ideal convergence of factors that shows us where the peak is on a ‘topographical map’ (figure 2). If the design space is a mountain, the optimum response is the peak. But it’s easy to miss the peak using OFAT, while using DOE allows you to enjoy the experimental view. Analyzing interactions allows you to identify when you’re on a high ‘plateau’ rather than a single peak. These plateaus are more robust to variation, but, because they are revealed only by considering interactions, remain hidden by analyzing OFAT. Visualizations such as ‘topographical maps’ also help communicate the results to colleagues.

In other words, DOE allows you to build on a secure statistical foundation that increases the likelihood that your experiments will produce positive results. As Fisher told the Indian Statistical Congress during the 1930s: “To call in the statistician after the experiment is done may be no more than asking him to perform a postmortem examination: he may be able to say what the experiment died of”.⁵

Figure 2: Response surface methodology, not unlike a topographical map of a landscape

Interested in learning more about DOE? Make sure to check out our other DOE blogs, download our DOE for biologists ebook, or watch our DOE Masterclass webinar series.

References

1. Losos, J Improbable Destinies: How Predictable is Evolution? Penguin Books 2017
2. Fisher, RA The arrangement of field experiments. Journal of the Ministry of Agriculture 1926;33:503-515
3. Roa, A The Higgs boson: What makes it special? https://home.cern/news/series/lhc-physics-ten/higgs-boson-what-makes-it-special Accessed October 2022
4. Lendrem DW, Lendrem BC, Woods D et al. Lost in space: design of experiments and scientific exploration in a Hogarth Universe. Drug Discovery Today 2015;20:1365-1371
5. Geraci M and Pearce A Transparency in a world of complexity: Basic guidelines for improved statistical reporting. The Journal of Child Health Care 2016;20:3-4.