Before you investigate a quality problem, you need to know whether your measurement system can even detect it. MSE answers that question and tells you exactly where measurement variation lives.
Technical Guide · 2025-04-18 · 8 min · OpEx Excellence Team
The Question Nobody Asks First
When an appliance engineer sees variation in test data, the instinct is to investigate the product or the process. But there is a question that should come first: how much of this variation is real, and how much is coming from the measurement system itself? Measurement System Evaluation (MSE) answers this by decomposing total observed variation into its sources. It separates the contribution of the measurement system (operators, fixtures, instruments, environment) from the actual part-to-part differences you are trying to study. Without this step, every subsequent analysis is built on uncertain ground.
How MSE Decomposes Variation
MSE uses a structured study design where multiple parts are measured by multiple operators with multiple repetitions. The resulting data is analyzed using Components of Variation methods to quantify how much variation comes from each source. For a toaster browning consistency test, MSE might reveal: 45% of measurement variation comes from fixture positioning, 30% from ambient light interference, 15% from operator differences, and 10% from sensor repeatability. With this breakdown, the improvement path is clear: fix the fixture first, then add a light shield, and the measurement system will be well within acceptable limits.
MSE decomposes variation into its individual sources, telling you not just whether your measurement system has a problem, but exactly where the problem lives and how large each source is.
Real Example: Microwave Power Output Testing
A microwave manufacturer ran an MSE on their power output test station. The study showed the measurement system contributed 24% of total variation, which was marginal. The MSE variance decomposition pinpointed the dominant source: 18 of those 24 percentage points came from the water load temperature at the start of each test. Operators were not waiting for the water to reach a standard starting temperature. A simple procedural change (digital thermometer check before each test) dropped the measurement system contribution to 8% without any equipment investment. Without the MSE decomposition, the team might have purchased new power meters or rebuilt the test fixture, spending thousands to address sources that contributed only 6% of the problem.
When to Run MSE
MSE should be the first step in any structured investigation. Run MSE before any COV study, because COV results are only meaningful if the measurement system contribution is small enough to see through. Run MSE before any DOE, because experimental effects smaller than the measurement system noise will be invisible. Run MSE whenever you commission a new test station or change your measurement procedure. And run MSE periodically to verify that your measurement system has not drifted over time.
- Before COV: ensures variance decomposition reflects real process variation, not measurement noise
- Before DOE: ensures experimental effects can be detected above the measurement floor
- New test station commissioning: establishes baseline measurement capability before production begins
- After measurement procedure changes: confirms the new procedure did not introduce additional variation
- Periodic verification: catches measurement drift before it corrupts quality decisions
The Cost of Skipping MSE
Appliance testing labs deal with complex measurements: temperatures, forces, flow rates, acoustic levels, power consumption. Each of these has multiple potential sources of measurement error. When teams skip MSE and jump straight to process investigation, they risk chasing phantom variation that exists only in the measurement system. A refrigerator manufacturer spent six months investigating compressor efficiency variation before discovering that 41% of what they were measuring was test station noise. The six months of engineering time spent investigating a measurement artifact cost far more than the two-day MSE study that would have revealed the problem immediately.
Frequently Asked Questions
What is an acceptable measurement system contribution?
Generally, the measurement system should contribute less than 10% of total variation for the system to be considered excellent, 10-30% is marginal (may be acceptable depending on application), and above 30% means the measurement system needs improvement before it can be used for quality decisions.
How many parts and operators do I need for an MSE study?
A typical MSE study uses 10 parts measured by 2-3 operators with 2-3 repetitions each. This provides enough data to decompose variation into its sources while keeping the study practical for a production environment.
Can MSE be used for destructive testing?
Yes, but the study design must be modified. Since each part can only be measured once, nested designs are used instead of crossed designs. The analysis can still decompose variation sources, though with slightly less precision.
Try MSE with your own data
Upload your measurement data or use the built-in appliance examples to run a complete MSE analysis with interactive charts and variance decomposition.
Get started →Tags: MSE, measurement system evaluation, appliance testing, quality lab, variance decomposition