What is hypothesis testing in manufacturing?

It is a statistical method for deciding whether a difference in plant data, such as a change in scrap rate, fill weight, or cycle time, is a real effect or just normal variation. You state a null hypothesis of no difference, set a significance level, run a test, and reject the null only when the p-value falls below that level.

What is the difference between the null and alternative hypotheses?

The null hypothesis (H0) says nothing changed, no difference, no effect. The alternative hypothesis (H1) says a difference exists. You assume the null until the data forces you to reject it. You never prove the alternative; you either reject the null or fail to reject it.

What does a p-value of 0.05 mean?

A p-value is the probability of seeing a result at least as extreme as yours if the null hypothesis were true. A value of 0.05 means data this extreme would occur about 1 time in 20 by chance alone. If your significance level (alpha) is 0.05, you reject the null when the p-value is below it.

Which hypothesis test should I use to compare two machines?

For average measurements like cycle time or weight, use a two-sample t-test. For pass/fail or defect-rate data, use a two-proportion z-test or a chi-square test. If you are comparing variation rather than averages, use an F-test or Levene's test. Check normality and equal-variance assumptions first.

What are Type I and Type II errors?

A Type I error is a false alarm: rejecting a true null hypothesis and acting when nothing actually changed. A Type II error is a miss: failing to detect a real difference, usually because the sample was too small. The significance level alpha is your Type I error rate; larger samples reduce Type II risk.

What is the difference between statistical and practical significance?

Statistical significance means the difference is unlikely to be chance. Practical significance means the difference is big enough to matter on the floor. With a large sample, a tiny, meaningless difference can be statistically significant, so always report the size of the effect and a confidence interval, not just the p-value.

How is hypothesis testing related to SPC control charts?

A control chart is essentially continuous, automatic hypothesis testing: each plotted point tests whether the process is still in control, and an out-of-limit point is a rejected null hypothesis. Formal hypothesis tests are what you run on purpose when you change something and want to confirm before-and-after whether it worked.

Hypothesis Testing for Plant Data

How to tell a real process shift from ordinary noise, and which test to reach for.

Hypothesis testing is a statistical method for deciding whether a difference in plant data is real or just noise. You state a null hypothesis of no difference, an alternative that a difference exists, pick a significance level, run a test, and reject the null only when the p-value falls below that level.

Every plant argues about numbers. Line 2 scrapped 3.1% this week and 2.4% last week: did the new supplier make it worse, or is that swing normal? The night shift's fill weights look tighter than day shift's: real difference, or luck of the draw? Hypothesis testing is the tool that settles those arguments with a rule instead of a hunch. It does not tell you the truth; it tells you whether your data is strong enough to act on. That is exactly the discipline that keeps statistical process control and root cause analysis honest, and it belongs in the toolkit of any plant serious about lean manufacturing.

What Is Hypothesis Testing, in Plant Terms?

Hypothesis testing is a formal way to ask, "Is what I am seeing bigger than the random variation I would expect anyway?" You assume nothing changed (the null hypothesis), then check how surprising your data would be if that assumption were true. If the data would be very surprising under "nothing changed," you conclude something did change. If it would not be surprising, you have no evidence to act, and you say so plainly rather than reading tea leaves.

The key idea is that measured processes wobble on their own. Fill weights, cycle times, and defect rates all vary from part to part and hour to hour even when nothing is wrong. Hypothesis testing puts a number on how much of the difference you see could be that ordinary wobble, so you do not chase ghosts or, worse, ignore a real shift because it looked small.

The null curve is the world where nothing changed. If your result lands far out in the tail (the shaded rejection region), the p-value is small and you reject the null.

What Are the Null and Alternative Hypotheses?

The null hypothesis (H0) is the skeptic's position: no difference, no effect, nothing changed. It is what you assume until the data forces you to give it up. The alternative hypothesis (H1) is what you suspect is true: the means differ, the defect rate went up, the variance shrank. You never "prove" the alternative. You either reject the null (evidence for the alternative) or fail to reject it (not enough evidence), the same way a court returns "guilty" or "not guilty," never "innocent."

Write them before you look at the data, in the units of the process. For the supplier question: H0 is "scrap rate is the same for both suppliers"; H1 is "scrap rate differs." Decide up front whether you care about a difference in either direction (two-sided) or only one (one-sided, for example "the new fixture reduced cycle time"). One-sided tests are more sensitive but only legitimate when you truly do not care about the other direction.

What Is a P-Value, and What Does the Significance Level Mean?

The p-value is the probability of seeing a result at least as extreme as yours if the null hypothesis were true. A p-value of 0.02 means: if nothing really changed, data this lopsided would show up about 2 times in 100 by chance alone. The significance level (alpha) is the threshold you set in advance for how much of that chance you will tolerate before you act. When the p-value is below alpha, you reject the null; when it is at or above alpha, you fail to reject.

The near-universal default is alpha = 0.05, meaning you accept a 1-in-20 risk of crying wolf. That number is a convention, not a law. When acting on a false alarm is expensive, scrapping a good lot, halting a line, requalifying a supplier, engineers tighten alpha to 0.01 or lower, which is common practice when the cost of a wrong call on machined parts is high. Pick alpha before the test, not after you see the p-value, or you are just moving the goalposts.

Which Test Should You Reach For?

The right test depends on what kind of number you are comparing and how many groups you have. Continuous measurements (weights, times, dimensions) use tests on means and variances; counts and pass/fail data use tests on proportions. Here is the everyday map.

Question on the floor	Data type	Common test
Did the average fill weight shift from target?	One continuous measure vs a target	One-sample t-test
Do two machines run different average cycle times?	Two continuous groups	Two-sample t-test
Did a change help the same parts, before vs after?	Paired measurements	Paired t-test
Do three or more lines differ in mean output?	3+ continuous groups	One-way ANOVA
Did the scrap (defective) rate change?	Proportions / counts	Two-proportion z-test or chi-square
Is one process more variable than another?	Spread / variation	F-test or Levene's test

A starting map, not a rulebook. Small samples, non-normal data, and unequal variances change the choice, which is why you check assumptions before trusting any p-value.

Start from the kind of number you have, then count your groups. Variation questions branch off on their own to F-type tests.

How Do You Run a Hypothesis Test on the Floor? A 6-Step Procedure

State H0 and H1 in process units, before collecting data. Write the exact claim: "new coolant changes mean tool life" (two-sided) or "new coolant increases mean tool life" (one-sided). Ambiguous hypotheses produce arguable conclusions.
Set alpha and the minimum difference worth detecting. Pick 0.05, or tighter if a false alarm is costly. Decide how big a real difference has to be to matter, because a difference can be statistically real yet too small to care about.
Size the sample. Too few samples miss real shifts; absurdly many flag trivial ones. Use a sample-size calculation (from alpha, the effect you want to catch, and process variation) so you collect enough, and no more.
Collect data cleanly and check assumptions. Sample randomly, not just the easy parts. Then verify the test's assumptions, usually rough normality and comparable variances, with a histogram or normality check. Fix the test choice if they fail.
Run the test and read the p-value against alpha. Compute the statistic and p-value in your SPC software or a spreadsheet. Below alpha, reject the null; at or above, fail to reject. Report the p-value, not just "significant."
Translate the result into a decision and a size. Pair the yes/no with the estimated difference and a confidence interval: "night shift runs 0.4 g tighter (95% CI 0.2 to 0.6 g)." That interval, not the p-value alone, tells operations whether the effect is worth a change.

What Mistakes Wreck Hypothesis Tests on Real Plant Data?

The two classic errors have names. A Type I error is a false alarm: rejecting a true null, acting when nothing changed. Alpha is exactly your Type I risk. A Type II error is a miss: failing to catch a real difference, usually because the sample was too small. The two trade off; shrinking one enlarges the other unless you add data. Choosing alpha is choosing which mistake you would rather make.

Correct calls sit on the diagonal. A false alarm (Type I) is set by your alpha; a miss (Type II) usually means the sample was too small.

Three floor-level traps do the most damage. First, confusing statistical significance with practical significance: with a huge sample, a 0.05 g fill difference nobody can feel comes back "significant." Always report the size of the effect. Second, p-hacking: slicing the data twenty ways until something crosses 0.05. Run the test you planned, once. Third, violated assumptions: skewed data or wildly unequal variances make a t-test's p-value unreliable, which is when nonparametric tests earn their keep. When a test does confirm a real shift, feed the finding straight into your corrective and preventive action process so the fix is documented and verified, and use the same evidence to bound the problem with an is / is-not analysis before chasing causes.

Where Does Hypothesis Testing Fit With SPC and Cpk?

Think of the control chart as continuous, automatic hypothesis testing: every plotted point silently tests "is this process still in control?" and an out-of-limit point is a rejected null. Formal hypothesis tests are what you run when you deliberately change something and want to know if it worked, before-versus-after a countermeasure, machine A versus machine B, old spec versus new. They also underwrite capability studies: a process capability (Cpk) comparison across two setups is a variance-and-mean question, which is a hypothesis test in disguise. Read alongside control charts hypothesis testing turns "I think it got better" into "it got better, by this much, and here is the risk I am wrong."

Hypothesis testing by the numbers

The mechanics are standardized, and the reference sources are public. The most cited engineering reference, the U.S. NIST/SEMATECH e-Handbook of Statistical Methods, lays out the same tests plants use, t-tests, ANOVA, chi-square, and F-tests, with worked examples (NIST/SEMATECH e-Handbook of Statistical Methods). The default significance level of 0.05, and the definition of the p-value as the probability of data at least as extreme under the null, are the conventions taught by the American Society for Quality (ASQ, Hypothesis Testing). Alpha is deliberately tightened to 0.01 or lower where a wrong call is expensive, the standard practice for high-cost machined parts. None of this requires a statistician on staff, only the discipline to state the hypothesis, set alpha, and check assumptions before reading the p-value.

The bottleneck is rarely the math; it is clean, timestamped data. A test comparing this week to last is only as good as the numbers behind it, and paper logs reconciled at end of shift are full of gaps and rounding. Plants that capture counts, weights, and stop reasons automatically at the station can run a before-and-after test the same day a change is made, with samples they trust, which is the practical payoff of live floor data over your existing systems no rip-and-replace. See how one plant turned floor data into faster decisions in our CLS case study.