Skip to main content

Welcome to the Evaluation Playbook!

This playbook was developed by GovTech Singapore's AI Practice to help you evaluate AI systems of any kind in a rigorous manner with practical guidance and tools.

About us​

This playbook is jointly written by the AI Innovation squad and Responsible AI team in GovTech Singapore's AI Practice. We seek to build deep AI capabilities to close the gap between the frontier and adoption of AI for GovTech and the Whole-of-Government (WOG). To that end, the AI Innovation squad turns frontier capability into public good by tackling high-impact, high-complexity use cases with agencies, while the Responsible AI team helps to improve how the Singapore government develops, evaluates, deploys and monitors AI systems in a safe, trustworthy and ethical manner.

Check out our blog and Hugging Face!

Who this is for​

The playbook is primarily designed for application developers, particularly those in the Singapore public sector, who are building and launching AI products and want to approach evaluation with greater rigour and confidence. The playbook is also useful for product managers, CIO teams and others involved in developing, assessing or governing AI systems who want to understand AI evaluation and apply it in practice.

Although written with the public sector in mind, most of the principles, explanations and recommendations are broadly applicable to organisations developing any kinds of AI systems.

Find yourself below to see how the playbook can help you.

Application developers

You build and launch AI products in the Singapore public sector, whether traditional or LLM-based, and need to show they work before they ship.

How the playbook helps you
  • An end-to-end roadmap for evaluating AI systems, rigorous enough to trust and practical enough to follow.
  • Tools and resources to support your evaluation efforts.
Product managers

You own an AI product and want to know whether your current evaluation gives you enough confidence to improve it and make decisions about it.

How the playbook helps you
  • A review checklist for the evaluation practices behind your product.
  • Tools and resources to support your evaluation efforts.
CIO and central technology teams

You want to review evaluation maturity across products, and compare your organisation's practices with approaches used across the industry.

How the playbook helps you
  • A map of the landscape of evaluation paradigms across the industry, and how they compare with your current practices.
  • Clearer visibility of evaluation efforts across GovTech and the Whole-of-Government today.
  • A reference point to align your practices with public sector recommendations.

Purpose of this playbook​

With advances in AI capabilities, particularly in coding, teams can now build AI systems faster than ever. However, our ability to build AI systems has raced ahead of our ability to tell whether they actually work as intended.

This playbook aims to bridge that gap by helping you:

  1. Understand key evaluation concepts. Learn the key principles behind a trustworthy, end-to-end AI evaluation process, from defining objectives to interpreting results and productionalising evaluation for continuous improvement.
  2. Put them into practice. Follow practical, actionable guidance to design, execute and improve evaluations for your AI systems, both before and after deployment.
  3. Explore available tools and resources. Discover evaluation tools, frameworks and resources available across the Singapore public sector and the wider ecosystem.

While this playbook prescribes one structured approach to evaluating AI systems, there is no single correct way to evaluate. However, most of the underlying principles and practices in this playbook can be transferred to other evaluation frameworks, so we encourage you to keep an open mind and adapt the guidance to your own context.

Citation​

To cite this work, please use the following BibTeX citation:

@article{evaluation_playbook_govtech_singapore,
    title   = {Evaluation Playbook},
    author  = {GovTech Singapore},
    year    = {2026},
    month   = {October},
    url     = {https://playbooks.aip.gov.sg/evaluation-playbook/}
}

Alternatively, you may use the APA-formatted citation below:

GovTech Singapore (2026) Evaluation Playbook. URL https://playbooks.aip.gov.sg/evaluation-playbook/

Before you start​

You will get the most from the playbook as long as you have a system that you care about, of which it can be in any part of the development lifecycle.

No prior knowledge required

You do not need a background in evaluation research or statistics. The playbook introduces each concept where you need it.

Head to Getting Started for an overview of our evaluation framework and the structure of the playbook.

Last updated on

Was this page helpful?