01 · Ask whether effects depend on conditions
Ask whether effects depend on conditions
Two way ANOVA models a continuous outcome with two categorical factors and their interaction. A warehouse might compare old and new picking methods at low and high congestion. The practical question may be whether the new method works particularly well when aisles are busy, rather than whether it is better on average. Define the response, independent unit, levels, and operating population before collecting data.
Factors are crossed when every method is observed at every congestion level. They are nested when levels occur only inside another factor, such as workers unique to each site. A crossed model is not the correct default for nested observations. Identify which factors were randomized and which were observed. Randomizing method within congestion strata supports stronger evidence about method than about naturally occurring congestion.
02 · Model means and interactions
Model means and interactions
The classical model is yijk = μ + αi + βj + (αβ)ij + εijk. The indices identify level i of A, level j of B, and replicate k. μ is an overall baseline; α and β are main effects; αβ is interaction; and ε is unexplained error. Coding constraints or reference levels identify the parameters. Conventional exact F tests assume independent normal errors with common variance across cells.
Interaction means the effect of one factor changes with the other factor. In a coded two by two model ŷ = b0 + bAA + bBB + bABAB, factors take −1 or +1. Main effects are twice their coefficients. The difference of differences is four times bAB. State the coding explicitly so coefficient signs have a clear operational meaning.
03 · Worked replicated warehouse study
Worked replicated warehouse study
Fictional times have two independent runs per cell: old method at low congestion, 11 and 13; old at high, 9 and 11; new at low, 10 and 12; new at high, 6 and 8. Cell means are 12, 10, 11, and 7 minutes. Code old as A = −1, new as A = +1, low as B = −1, and high as B = +1.
The grand mean is 10. New has a marginal mean of 9 versus 11 for old, so its average effect is −2 minutes. High congestion has a mean of 8.5 versus 11.5 for low, giving −3 minutes. This intentionally unusual direction must prompt a mechanism check in a real warehouse. New minus old is −1 at low congestion and −3 at high. Their difference is −2; therefore bAB = −0.5.
Each cell contributes 2 to within cell squared deviations. SSE = 8 with four error degrees of freedom, giving MSE = 2. Balanced design sums of squares are SSA = 8, SSB = 18, and SSAB = 2. F statistics are 4, 9, and 1, each with one numerator and four denominator degrees of freedom. Their p values are approximately 0.116, 0.0399, and 0.374. The interaction estimate is visible but highly uncertain with only two replicates per cell.
04 · Interpret conditional effects first
Interpret conditional effects first
An interaction plot connects cell means for one factor separately at each level of the other. Parallel lines suggest little interaction; nonparallel lines suggest conditional effects. The graph displays estimates, so uncertainty remains essential. A nonsignificant interaction test does not establish identical method effects at both congestion levels. Small samples can miss operationally substantial interactions.
Where interaction is credible and important, report simple effects at meaningful conditions. Retain model hierarchy: include main effects corresponding to retained interaction terms even when their individual p values are large. A crossover interaction can produce almost no average main effect while the intervention helps one condition and harms another. The decision should describe the condition under which the method is useful.
05 · Recognize identifiability limits
Recognize identifiability limits
An unreplicated two by two design supplies four observations. An intercept, two main effects, and an interaction require four parameters, leaving zero residual degrees of freedom. The full model fits perfectly and cannot estimate independent experimental error. A conventional interaction F test cannot be obtained from those four points alone.
Removing interaction creates a residual degree of freedom by assuming interaction is absent; its leftover contrast becomes error under that assumption. This cannot simultaneously provide a clean interaction test. Independently replicate, obtain defensible external error information, or state that the result is descriptive. Repeated scanner readings within one run measure repeatability rather than independently assigned process variation. They cannot be substituted for true replication.
06 · Handle imbalance and operational dependence
Handle imbalance and operational dependence
Missing cells or unequal cell counts make main effect tests depend on the fitted model and requested contrasts. In observational data, averages can reflect different workload compositions. Report adjusted means at a stated distribution of conditions and identify comparisons that are estimable. Understand the actual hypothesis instead of relying on a software sum of squares label.
Plot residuals by cell, fitted value, and run order. Worker, day, and truck dependence can require blocking or mixed models. A hard to change factor creates restricted randomization and may require split plot error terms. Unequal spread may call for transformation or a variance model. Binary missed cutoff outcomes require a suitable generalized model, such as logistic regression with an interaction, rather than ordinary ANOVA on individual zeros and ones.
07 · Judge quality and connect to DMAIC
Judge quality and connect to DMAIC
A strong application deliberately crosses factors, documents assignment, obtains independent replication, and translates interaction into operating conditions. A weak application tests every term in an unreplicated design or hides a harmful condition behind a favorable average. Another weak application deletes interaction simply because its p value exceeds 0.05, without considering power or the mechanism.
Analyze distinguishes competing explanations and reveals where an improvement may work. Improve confirms promising combinations on fresh runs across relevant conditions. Control records the operating region and monitors its continued relevance. Preserve cell counts, raw observations, conditional estimates, intervals, diagnostic plots, and the process mechanism. If the experiment contradicts expectations, investigate assignment, congestion definitions, and order mix before treating its averages as a universal law.
08 · Quantify the interaction uncertainty
Quantify the interaction uncertainty
In the worked coded model, every coefficient standard error is 0.5. The interaction coefficient is −0.5, so its 95% interval uses t with four error degrees of freedom: −0.5 ± 2.7764 × 0.5, or −1.8882 to 0.8882. Multiply by four to express the difference of differences: approximately −7.553 to 3.553 minutes. The interval includes both materially stronger benefit at high congestion and a reversal of that tendency. The correct follow-up is additional independent replication where the interaction matters, not a declaration that effects are equal.
A cell mean standard error under this balanced model is √(MSE/2) = 1 minute. Cell estimates from two runs are therefore imprecise even though their displayed numbers are whole minutes. Report simple effect intervals and actual cell counts. Where congestion is observational, examine why high congestion appears faster in these teaching data; the result could reflect a different order mix rather than an influence of crowding.
Connections
Related methods and phase use
Placed under Analyze because of the outcome it supports. Use the method in other phases when its purpose and assumptions fit the work; the phase list is a guide to selection, not a compulsory sequence.
Sources
Professional foundations and further reading
The explanation and fictional teaching examples are written for LeanTrainingLab. Linked primary sources provide professional foundations; example values and local conventions are stated in the guide. The accepted toolset draws on Six Sigma+Lean Toolset, second edition (Lunau, editor; Springer, 2013). No book text or examples are reproduced here.