Design of experiments (DOE) is a statistical method for changing several process factors at once, in a planned pattern, so you can find out which settings actually drive an output and how those factors interact, using far fewer runs than testing one factor at a time.
The idea that trips people up is deliberately changing more than one thing at once. Plant instinct says hold everything steady and move one knob, but that instinct is wrong, and it has been provably wrong since the 1920s. A designed experiment moves several knobs together in a structured grid, then uses the pattern to separate each factor's real effect from the others and, crucially, to catch interactions that one-factor-at-a-time testing cannot see.
What is design of experiments?
Design of experiments is a branch of applied statistics for planning, running, and analyzing controlled tests that evaluate the factors controlling the value of an output (ASQ, What Is Design of Experiments?). The vocabulary is small: a factor is an input you can set (temperature, pressure, feed rate), a level is a value you set it to (low and high, say), the response is the output you measure (yield, strength, defect rate), and an interaction is when the effect of one factor depends on the setting of another.
The method traces to Ronald A. Fisher, who developed factorial experiments at the Rothamsted agricultural station in England in the 1920s and 1930s and set out the foundations in his 1935 book The Design of Experiments. Fisher was working on crops, but the math is indifferent to the subject; the same designs run today on injection molding, coating lines, and heat treat. The modern reference on the engineering side is Douglas Montgomery's Design and Analysis of Experiments the standard university text. A century on, the core logic is unchanged: structure the test up front and the analysis nearly falls out of the data.
Why is changing one factor at a time a trap?
Changing one factor at a time (OFAT) feels rigorous and is quietly misleading. It fails for two reasons. First, it is inefficient: to study several factors you run a long series of separate tests, and each factor gets studied at only one setting of the others. Second, and worse, OFAT is blind to interactions. If high temperature only helps when pressure is also high, an OFAT test that holds pressure low while sweeping temperature will conclude temperature does nothing, and it will be wrong.
A factorial design tests factors in combination, so every run pulls double duty and the interactions show up in the pattern. That is why DOE can study more factors in fewer total runs than OFAT while learning things OFAT structurally cannot. It also feeds directly into the improvement methods you already use: the confirmed drivers from a DOE become the settings you lock into statistical process control and the interactions you find often explain a stubborn defect that root cause analysis narrowed but could not close.
How do you run a design of experiments?
A DOE is mostly won before the first run, in the planning. The sequence below keeps a floor-level experiment honest.
- Define the response and the goal. Pick the one output that matters and how you will measure it: maximize yield, minimize warp, hit a target dimension. If the measurement system is noisy, fix that first, or the experiment measures your gauge, not your process.
- Choose factors and levels. List the inputs you suspect matter and set a low and a high level for each, wide enough to move the response but still safe to run. A fishbone diagram is a good way to gather candidate factors from the people who run the process.
- Pick the design. Decide between a full factorial (every combination) and a fractional factorial (a chosen subset) based on how many factors you have and how many runs you can afford. This is the central trade-off, covered below.
- Randomize and, where you can, replicate. Run the trials in random order so unknown drifts (a warming shop, a new material lot) do not line up with a factor and fool you. Replicating runs gives you an estimate of natural noise to judge effects against.
- Run the experiment as planned. Hold everything not under study as steady as you can, and record the actual settings and conditions, not just the intended ones. Discipline here is what makes the analysis trustworthy.
- Analyze the effects. Calculate each factor's main effect and the interactions, and separate real signals from noise. The output is a ranked picture of what drives the response and which factors do not matter.
- Confirm, then lock it in. Run confirmation trials at the recommended settings to prove the predicted result is real, then fold the winning settings into standard work and monitor them so the gain holds.
Full factorial or fractional factorial?
The core design choice is how much of the combination space to actually run. A full factorial at two levels tests every combination, which is 2 raised to the number of factors: 3 factors is 8 runs, 4 factors is 16, but 7 factors is 128 runs, and 10 factors is over a thousand. That completeness buys you clean estimates of every main effect and every interaction, but the run count explodes.
A fractional factorial runs a carefully chosen fraction of those combinations, halving or quartering the run count, at the cost of "confounding": some effects get tangled together and cannot be told apart. Fractional designs, introduced by statistician David Finney in 1945, are the workhorse of plant screening because early on you usually just want to know which few of many factors matter, and you can afford to blur the higher-order interactions you do not expect to be real.
| Design | Runs (2 levels) | You get | Best for |
|---|---|---|---|
| Full factorial | 2 to the power of factors (e.g. 4 factors = 16) | Every main effect and interaction, cleanly | A few factors you already believe matter |
| Fractional factorial | A half, quarter, or smaller fraction | Main effects, some interactions confounded | Screening many factors down to the vital few |
A common plant pattern is to screen with a fractional design first to knock a long factor list down to the two or three that move the response, then run a small full factorial on just those to map them and their interactions in detail.
What does DOE actually buy on the floor?
Two payoffs. First, efficiency: because factorial designs test factors in combination, you learn more from fewer runs than one-factor-at-a-time testing, and every run contributes to every factor's estimate (ASQ, Design of Experiments). Second, insight OFAT cannot give: the interactions. Finding that two settings only work in combination is often the difference between a defect you keep fighting and one you finally close. That is why DOE sits at the heart of both improvement and design work: it is the tool that turns a suspected driver into a proven one inside a DMAIC project, and the tool Design for Six Sigma uses to make a new design robust to variation before launch. The settings a DOE confirms are also what you feed into a process capability study to prove the process can hold spec.
Where does DOE go wrong?
The mistakes are almost always in setup, not statistics.
- Bad measurement. If the gauge cannot reliably tell two parts apart, the experiment measures noise. Validate the measurement system before you design a single run.
- Levels set too narrow. If low and high are barely different, no factor will look significant, and you will wrongly conclude nothing matters. Set levels wide enough to move the response.
- No randomization. Run the trials in a convenient order and a slow drift, a warming room, a shift change, can masquerade as a factor effect.
- Skipping confirmation. A DOE predicts; it does not prove. Confirmation runs at the recommended settings are what turn a prediction into a result you can stand behind.
- Over-fractionating. Squeezing too many factors into too few runs confounds effects so badly that the analysis cannot tell a main effect from an interaction. Match the fraction to what you actually need to learn.
How does DOE connect to your plant data?
A designed experiment is only as good as the data it generates, and on most floors that data is written on a run sheet, keyed into a spreadsheet later, and disconnected from the machine settings and quality results it needs to be paired with. That manual handling is where transcription errors creep in and where the analysis stalls. Harmony helps by digitizing station-level capture so experimental runs, the actual settings used, and the measured responses land as structured, timestamped records the moment they happen, ready to analyze instead of re-key. When the DOE confirms a winning setting, the same system carries it into the control charts that keep it in place. See how digitizing the floor first plays out in the CLS case study. Plan the experiment carefully, capture the data cleanly, and let it flow straight into the control that holds the gain. No rip-and-replace; the design logic is still Fisher's.