Concluding Remarks

You have now reached the end of the playbook, here are some remarks to help you reflect on what you have learned and how to move forward with this knowledge in hand.

---

What you have come away with

Each stage of the evaluation process left you with something concrete, and each one built on the stage before it.

Look back to see how far you have come to make sure you have everything you need.

What began as a vague "what is good, and is my system good enough?" is now a question you can answer with confidence.

Core evaluation loop · stages 1 to 4

1. 1DefineWhat “good” means

   A stakeholder-agreed definition of “good”

   • What 'good' means for the system is clear and agreed among everyone.
   • The specifications are signed off before any result arrives.
2. 2DesignJudgement design and evaluation set design

   A judgement design per specification, plus a representative, realistic evaluation set

   • 'Good' is measurable: each specification has a judge, a judgement, and a 'good enough' bar.
   • The evaluation set reflects real-world usage, with its inputs and expected outputs defined.
3. 3ExecuteRun the system, then the judge

   Results that hold still across repeats, with appropriate level of traces behind them

   • You understand how much traces are needed to support and enrich the evaluation.
   • You understand how to make sure the evaluation is as consistent as possible.
4. 4ImproveDiagnose results and decide what to change

   Failures traced to their cause, and a prioritised list of changes

   • You know how to trace failures back to their root cause and validate them.
   • You know how to design and test changes to the system, and how to prioritise them.

Outer monitoring loop · stage 5

1. 5MonitorEnsure continuous evaluation even after launch

   An evaluation plan that runs continuously and is adapted to the post-launch context

   • You understand how evaluation can continue after launch.
   • You understand what to monitor and when to conduct a full evaluation again when needed.

Missing something?

If you feel like you are missing something, or that you are not ready to move on, go back to the relevant stage and review the guidance and artefacts there.

---

Where to go from here

Most of this playbook has been conceptual by design: a shared vocabulary for what “good” means, and a disciplined loop for measuring it and improving against it. An important next step is to put this into practice. The Tools and Resources annex collects evaluation tools, frameworks and resources from across the Singapore public sector and the wider ecosystem. Treat it as an appendix rather than another chapter: nothing in it is required reading, and it will be there to reach for whenever a practical question comes up.

That is the playbook. You are equipped, so go put the theory into practice. Happy evaluating!