What the percentage actually measures
Every one of them measures execution, not assertion. A test that calls a function and asserts nothing gives exactly the same coverage as one that checks every result. This is not a subtle limitation; it is the whole story.
it("works", () => {
quote({ lines: [{ qty: 2, price: 890 }] }); // every line executed
}); // …and nothing asserted
| Metric | Counts |
|---|---|
| Line / statement | lines executed at least once |
| Branch | each side of each if, ?: and && |
| Function | functions called at least once |
| Path | combinations of branches — nobody reports this, it explodes |
What it is good for
- Finding what is never run at all. A module at 0% is either dead code or a gap somebody should know about.
- A ratchet. "Do not let it fall below where it is" is a reasonable rule; "get it to 100" is not.
- Branch coverage on a specific function you are about to change — it tells you which case you have not thought about.
Mandate 90% and people write tests that execute code without asserting, test getters, and add / istanbul ignore / to whatever is awkward. The number goes up and the suite gets worse. Goodhart’s law, with a CI badge.
The measurement that does mean something
A mutant that makes the suite fail is killed — some test noticed. A mutant that survives is a change to your production code that nobody would have caught, which is a precise description of a missing test.
Mutation score is expensive to compute (the suite runs once per mutant) and far more honest than coverage. Run it on the parts that matter — the pricing, the permissions, the thing that moves money — not on the whole repository.
// the original
if (order.qty > 0) { … }
// mutants
if (order.qty >= 0) { … } // boundary
if (order.qty < 0) { … } // negated
if (true) { … } // removed
The equivalent mutant
Those two lines behave identically: when qty is zero, adding 0 × price changes nothing. No test can distinguish them, so the mutant survives for ever and the score never reaches 100%.
Equivalent mutants are the reason mutation score is read as a direction rather than a target — and the reason every tool lets you mark one as "known, and impossible". Recognising them is part of using the technique.
if (line.qty > 0) total += line.qty * line.price;
if (line.qty >= 0) total += line.qty * line.price; // a mutant nobody can kill
What no measurement will tell you
Whether the behaviour is the one the business asked for. Whether the error message is usable. Whether the thing is fast enough, or accessible, or secure. Coverage and mutation score both measure the relationship between your tests and your code — neither has any opinion about whether the code is right.