Reference architecture

The reference architecture separates telemetry production, collection and protection, storage, and use. A product may implement these layers with one platform or several tools; the logical responsibilities remain.

Logical architecture

┌──────────────────── AI service and dependencies ────────────────────┐
│ Channel → App → Orchestrator → Model → Retrieval → Tools / Agents  │
│                ↘ Guardrails ↔ Human review ↗                       │
└─────────────────────────────┬───────────────────────────────────────┘
                              │ logs, spans, metrics, events
                              ▼
┌──────────────── collection and protection ─────────────────────────┐
│ SDK / middleware → collector / queue → schema validation           │
│                     → redaction / routing → loss monitoring         │
└───────────────┬───────────────────┬───────────────────┬─────────────┘
                ▼                   ▼                   ▼
        Operational store    Restricted content   Audit evidence
        metrics / traces     approved raw data    integrity-protected
                └───────────────────┬───────────────────┘
                                    ▼
┌──────────────────────────── uses ───────────────────────────────────┐
│ dashboards │ alerts │ incident search │ evaluation │ assurance     │
└────────────────────────────────────────────────────────────────────┘

Instrumentation points

Point	Capture
Service entry	session/query/request IDs, channel, actor reference, intent classification, timestamp
Orchestrator/agent	task and subtask structure, agent version, step count, decisions, outcome
Model call	model ID, operation, parameters, token usage, latency, approved content/reference
Retrieval/memory	query reference, index/version, document IDs, classifications, scores, writes
Guardrail/policy	policy/version, category, decision, confidence where meaningful, action taken
Tool/API	tool and operation, safe parameters, identity/scope, attempt, outcome, side-effect reference
Human review	request, reviewer role, decision, reason, timestamp, resulting action
Service exit	final outcome, response/action reference, duration, safe/unsafe failure classification

Correlation pattern

Create trace context at the earliest trusted entry point. Propagate it across HTTP/RPC headers, queues, jobs, callbacks, and agent delegation. For asynchronous operations, retain a link to the initiating span even when a direct parent-child relationship is not valid.

Do not expose internal trace or event IDs as sequential user-facing identifiers. If users need a support reference, generate a separate non-sensitive reference that authorised staff can resolve to the trace.

Separate content from operational metadata

A useful default is two linked stores:

1. Operational telemetry contains low-sensitivity metadata, durations, outcomes, versions, identifiers, and aggregates used by engineers and operators.
2. Restricted evidence contains approved raw or redacted content required for investigation or evaluation and is accessible only for defined purposes.

Link the stores with a random evidence reference. This reduces the number of people and systems that can access prompt, response, retrieval, and tool content.

Integrity pattern for incident evidence

For flagged safety or security events:

• copy the approved evidence set into an append-only or integrity-protected location;
• record the source event IDs, capture time, case ID, classification, and evidence hash;
• audit every access and export;
• preserve the original event order and time source; and
• apply incident retention and legal-hold rules where relevant.

Integrity protection does not justify collecting unnecessary content.

Failure behaviour

Decide how the service behaves if telemetry is unavailable:

Situation	Possible policy
Non-consequential low-risk request	continue with local buffering and raise platform alert
High-impact external action	fail closed or require human approval if mandatory audit evidence cannot be written
Collector backpressure	bounded queue, safe sampling policy, no unbounded application memory growth
Redaction failure	quarantine or drop sensitive payload; emit metadata-only failure event
Evidence store unavailable	prevent privileged action or retain encrypted local evidence for bounded retry

The product and risk owners should approve these choices; they are business-control decisions, not only platform settings.

Multi-tenant platforms

Include service_id, tenant_id where allowed, environment, owner, and data classification on every event. Enforce tenant separation at ingestion and query time. A shared collector must not become a path for one product team to access another product's restricted content.

Done when: the architecture shows every producer, trust boundary, redaction point, store, consumer, access path, loss signal, and deletion path.

Next: Integrate with your stack