Off-Topic guardrail

GovTech's Off-Topic guardrail detects user prompts that fall outside an AI system's intended purpose. It scores relevance against your system prompt, so it needs no topic taxonomy of your own.

Why we built it

Existing solutions either required training a use-case-specific guardrail or configuring it with examples of on- and off-topic prompts — challenging in the absence of real production data. Instead, we created a rich dataset of synthetic system-prompt / user-prompt pairs (on- and off-topic) and trained a lightweight classifier.

In v1, we trained a bi-encoder classifier on top of jina-embeddings-v2-small-en and a cross-encoder classifier on top of stsb-roberta-base.

Access

Available via the Sentinel API (govtech/off-topic). For guidance on when and how to apply it, see Robustness improvements.

PaperMethodology and resultsBlogWhy we open-sourced an off-topic prompt guardrail