November 21, 2022
- A design of experiments (DOE) campaign needs strategic and tactical planning, but much less than the traditional methodology of changing one factor at a time.
- Broadly, a DOE campaign encompasses screening, iteration and refinement, optimization; and assessing robustness. But it’s not a rigid flow. You can drop steps, as well as move backwards and forwards.
- Design of Experiments is iterative and flexible. Repeated iterations can move you rapidly from your initial ‘thought experiment’ to optimized data.
- Your campaign’s shape—the number and type of stages—depends on your goal. But there aren’t any rules about what you must do. DOE adapts the experiment to your needs.
- If you’d like to keep learning about DOE after you're done with this article, make sure to check out our other DOE blogs, download our DOE for biologists ebook, or watch our DOE Masterclass webinar series.
Science advances slowly: it’s more of a drawn-out campaign than a single, targeted strike. So, you need to plan strategically and tactically.
Military operations, like science, work to attain well-defined objectives. So, first, define the goals of your design of experiments (DOE) campaign: specify clearly what you are trying to achieve.
Secondly, ascertain the lay of the land by using DOE to understand and characterize your experimental space. A fine-grained design that samples many conditions across the landscape maps a detailed experimental topography. Design of Experiments approaches attain this in fewer stages, sometimes even a single stage, than assessing one factor at a time.
Thirdly, although scientists tend to be perfectionists, at least at heart, we do need to be pragmatic. Are we trying to be "good enough"? In other words, are we trying to exceed a threshold? A previous blog in this series considered strawberries as an example. So, perhaps you want to exceed a certain crop weight.
In other cases, you may be trying to optimize the output or minimize experimental noise. So, to go back to our strawberries, you may be trying to get the most fruit. Or perhaps you need stability despite changing conditions. Are you willing to sacrifice some yield for a strawberry crop that remains relatively stable and predictable despite the weather’s vagaries?
Stages in a DOE campaign
Different DOE designs fulfil different purposes. We’ll consider how to choose the design that is best suited to your purpose in a forthcoming blog. In the meantime, let’s look at the different stages in a DOE campaign, which broadly encompasses: screening, iteration and refinement, optimization, and assessing robustness (Figure 1). But it’s not a rigid flow. You can drop steps as needed and move backwards and forwards. Figure 2 shows the steps within any single stage.
Figure 1: Using the stages of DOE to deliver multifactorial optimization
Figure 2: The steps within any single stage of a DOE campaign
Screening
Screening aims to differentiate factors that are important to your system from those that are less influential. Screening is ideal when you have many factors and relatively little prior knowledge. In general, screening uses the maximums and minimums of the ranges for each expected factor. Screening also explores combinations to identify the importance of single factors and interactions with other variables. Limited prior knowledge can make setting a factor’s range difficult. But a DOE campaign allows you to revise your choices at each step, which is a big advantage. Even an educated guess is usually a "good enough" starting point.
At this stage, you’re not looking to fit high-quality predictive models. Instead, you’re aiming to determine which of many possible factors and interactions are really important in controlling your system’s behaviour. So, if you only have a few well-characterized factors you may be able to bypass screening and go straight to optimization. However, as we explained in our earlier blog, this can introduce bias. It’s best, therefore, to screen initially if you can spare the time and resources
Iteration and refinement
Sometimes you get lucky, hitting your success criteria with the first screen. However, in most cases, you are still learning about your system: which factors are important and within which ranges. You need to explore your space a bit more to find somewhere close to optimal. Strawberries generally need slightly acidic soil for optimal growth. Perhaps the soil was initially too alkaline, so you can refine the pH. Once you have found an acceptable place, then you can reduce the number of factors and explore the ones that remain in much more depth.
Optimization
The next stage involves creating a high-quality predictive model to infer optimal conditions for the system. This requires a specific design type which investigates the factors in more detail, defined using information from the previous stages. Optimization produces the sets of conditions that give you the best outcomes.
Returning to our strawberries, the model could predict the impact of variations in soil moisture and pH, shade and the proportion of clay, sand and silt in the loam. Exploring a small range and several repeated runs allows you to hone in on the optimal conditions and investigate the noise intrinsic to the system.
This stage can generate new hypotheses: do optimal conditions differ depending on whether the strawberries are destined for jam or centre court at Wimbledon? Your optimization run may tell you. Otherwise, you can always go back to earlier stages to find out.
Assessing robustness
You usually want to determine the extent to which the system is sensitive to changes in levels of your factors. A robust system will have smaller variability to changes than a more sensitive system. So, different strains of strawberry may be more or less susceptible to changes in soil moisture, pH and composition.
Reaching the goal
As the Prussian commander Helmuth von Moltke noted in 1880 “No plan of operations reaches with any certainty beyond the first encounter with the enemy’s main force.” The same applies to science. How often have your carefully planned experiments gone awry because of unexpected results or interactions?
Using DOE you run your experiments, get some data and infer the next steps. You don’t need to plan in detail upfront. You can adapt the plan in response to the changes and challenges. Breaking up your experiments in this way is an important advantage of using DOE techniques. DOE steadily increases the return on your experimental investment as you learn more rather than trying to do everything all at once. Incremental advances are always a better bet in the face of significant uncertainty.
A scientific campaign needs to adapt as new data emerges and you gain insights into the experimental topography. Design of experiments is flexible, while repeated iterations can move you rapidly from ‘thought experiment’ to optimized high-quality data (Figure 1). The shape of your campaign – the number and type of stages - depends on your goal. But there aren’t any rules about what you must do. DOE adapts the experiment design to your needs, rather than being forced to rethink your goals because of the inherent limitations when changing one factor at a time.
Interested in learning more about DOE? Make sure to check out our other DOE blogs, download our DOE for biologists ebook, or watch our DOE Masterclass webinar series.
