Measure adoption
Measure both coverage—whether required telemetry exists—and effectiveness—whether it helps teams detect, investigate, and improve the service.
Core adoption measures
| Measure | Definition |
|---|---|
| Mandatory implementation coverage | verified mandatory items / applicable mandatory items |
| Waiver health | valid unexpired waivers / waivable gaps; also track overdue waivers |
| Trace coverage | expected traceable events with trace context / expected traceable events |
| Material-journey coverage | verified material journeys / identified material journeys |
| Evidence retrieval time | time from case/support reference to complete authorised evidence set |
| Detection coverage | material failure scenarios with tested alert/review / identified material scenarios |
| Runbook exercise coverage | tested critical runbooks / required critical runbooks |
| Finding closure | verified closed findings / due findings, plus age and severity |
| Recurrence | incidents sharing known cause or failed corrective action / incidents |
| Change safety coverage | applicable changes with completed safety evaluation / applicable changes |
Do not optimise coverage alone
A team can reach 100% checklist coverage while emitting unusable, unprotected, or ignored data. Pair coverage with:
- incident or exercise evidence retrieval time;
- percentage of alerts that produce a defined action;
- unexplained or unknown external-action states;
- schema and pipeline delivery quality;
- use of telemetry findings in release or remediation decisions; and
- operator/reviewer confidence in reconstruction.
Maturity model
| Level | Description |
|---|---|
| 0 — Unknown | No system profile or requirement ownership; evidence assembled ad hoc. |
| 1 — Recorded | Core Tier 1 records exist but correlation, protection, or operational use is inconsistent. |
| 2 — Investigable | Selected Tier 1/2 evidence is linked, protected, searchable, and exercised. |
| 3 — Operated | Dashboards, alerts, runbooks, change controls, and reviews drive routine decisions. |
| 4 — Improved | Governed evaluation, trend analysis, recurrence learning, and measured control effectiveness guide continuous improvement. |
Assess maturity per service or capability, not only organisation-wide. A shared platform may be Level 3 while an individual product has not instrumented material tool actions.
Reporting format
Report trend, target, owner, interpretation, and action. Avoid ranking teams by raw requirement count because system risk and applicability differ. Use the data to identify shared platform gaps and high-impact remediation.
Improvement loop
- Review coverage and effectiveness.
- Select the highest residual-risk or investigation gap.
- Improve instrumentation, control, view, alert, or runbook.
- Re-run the acceptance scenario.
- Update evidence and baseline.
- Share reusable schema or platform capability with other teams.
Done when: adoption measures lead to funded, owned improvements and can show that incident detection, investigation, or prevention is getting better over time.