01 · Purpose
Why and when to use it
Use it when the baseline depends on human or automated classifications such as damage/no damage.
Who takes part?
The measurement owner works with people who understand the process and the recorded data. Statistical or measurement-system decisions should involve someone able to check the assumptions and study design.
02 · Application
How to apply it
Start with an agreed measurement purpose, explicit units and trustworthy observations. Identify missing cases and assumptions before calculating.
- Define pass/fail rules and establish reference classifications.
- Select representative cases including difficult boundary cases.
- Have assessors classify the cases in randomised, blinded repeat rounds.
- Calculate within-assessor, between-assessor and reference agreement separately.
- Examine false accepts and false rejects; clarify the method and repeat the study if needed.
03 · Example
A worked measurement example
Original fictional teaching example. Small datasets illustrate the method; they are not proposed as sufficient study sizes. Supplied parameters are assumptions, not estimates from a real process.
| Illustrative result | Calculation | Meaning |
|---|---|---|
| Reference agreement | 36 of 40 decisions = 90% | Four decisions disagree with the reference |
| False accepts | 3 of 10 reference-fail decisions = 30% | Three failures incorrectly accepted |
| False rejects | 1 of 30 reference-pass decisions = 3.33% | One acceptable case incorrectly rejected |
04 · Interpretation
What the result tells you
Overall agreement hides the much larger false-accept rate. Check class balance and consequences, not just one combined percentage. Chance-adjusted statistics such as kappa can complement agreement but are sensitive to prevalence.
Completion check: An attribute agreement assessment with error direction and study limitations.
05 · Judgement
Mistakes and limitations
Avoid easy-only cases, unblinded repeats and assuming the reference is infallible. These 40 decisions are arithmetic examples, not an adequate universal study size.
07 · Reading
Sources and further reading
The tool list follows the accepted LeanTrainingLab overview and its reference to Six Sigma+Lean Toolset, second edition (Lunau, editor; Springer, 2013). The book chapters were not available for direct review during this build. Explanations, examples and worksheets are original teaching material. Further reading supplies professional background; local conventions are stated explicitly.