Skip to content

3. Select requirements

The ORF catalogue contains 61 telemetry items: 11 Tier 1, 31 Tier 2, 13 Tier 3, and 6 Tier 4. Convert the catalogue into an owned implementation backlog for the scoped system.

Selection workflow

1. Start with the matrix obligation

Apply the system risk tier to every ORF item. Do not select only the categories that already have an obvious engineering owner; ORF deliberately spans product, engineering, operations, security, and governance.

2. Confirm technical applicability

The ORF matrix defines the implementation action. Not applicable is an additional playbook record status, not a matrix action or waiver. Use it only when the subject of the requirement does not exist anywhere within the approved system scope. Record the system fact, reviewer, and reassessment trigger—not just “N/A”. Examples:

Requirement Valid applicability rationale
Retrieval metrics or access logs “Not applicable: the production path has no retrieval, memory, or knowledge source.”
Tool-call telemetry “Not applicable: the model cannot invoke tools or perform external actions.”
Distributed service traces “Not applicable: the in-scope path is a single process with no remote service call.”
Human handover metrics “Not applicable: no handover function exists; user receives a documented fallback route.”

An applicability decision is different from a waiver and must not be used to avoid an applicable Mandatory item. If the system condition changes, the item becomes applicable immediately rather than waiting for the next periodic review.

3. Define the implementation outcome

For each applicable item, specify:

  • source component and responsible owner;
  • event, metric, or trace to be implemented;
  • environment and journey coverage;
  • data handling and retention;
  • dashboard, alert, investigation, or assurance use;
  • acceptance test and evidence location; and
  • target phase and delivery date.

Use the requirements register.

4. Handle waivers

A waiver should include:

  1. the requirement and underlying risk;
  2. why implementation is not currently proportionate or feasible;
  3. residual impact and likelihood;
  4. existing compensating controls;
  5. time-bound remediation or review plan;
  6. accountable owner and approving authority; and
  7. expiry date and reassessment triggers.

Do not use lack of tooling, capacity, or ownership as the only risk rationale. Those may explain the gap, but they do not reduce its potential impact.

5. Add system-specific requirements

ORF is a baseline. Add telemetry needed for risks unique to the service, such as payment confirmation, case-state transition, medical escalation, source freshness, multilingual performance, or high-impact irreversible tool actions.

Prioritisation within a phase

When several items share the same tier, implement in this order:

  1. signals required to reconstruct irreversible or consequential actions;
  2. signals required to detect safety, security, privacy, and availability incidents;
  3. correlation and platform-health signals that make other evidence trustworthy;
  4. operating dashboards and alerts; and
  5. analytical enhancements.

Quality check

Before approval, confirm that:

  • all 61 ORF IDs were considered;
  • every mandatory item is planned or verified;
  • every waiver has an approver and expiry;
  • every Not applicable record states a verifiable system condition, reviewer, and reassessment trigger;
  • every item has one accountable owner; and
  • selected telemetry has a defined operational or assurance use.

Done when: the approved register can be used directly to create engineering tasks and assurance tests without interpreting vague requirement statements.

Next: Design the telemetry