Skip to main content

Getting Started

How we define evaluation​

Evaluations are structured tests that assess system behaviour against criteria that matter to you and your stakeholders.

Why evaluate?​

5TESTING6MAINTENANCE1PLANNING2ANALYSIS3DESIGN4IMPLEMENTATIONTHESOFTWAREDEVELOPMENTCYCLE
The traditional software development cycle has not changed, but rather the speed at which it operates has increased dramatically.

Within this cycle, evaluation has always been embedded deeply, most clearly in Testing (phase 5) and Maintenance (phase 6).

In recent times, the advent of AI capabilities has changed how teams build systems. Much of the implementation is now handed to AI, a practice known as "vibe-coding" that has quickly been accepted by the wider industry.

Tweet by Andrej Karpathy on 2 February 2025 describing a new kind of coding he calls vibe coding, where you fully give in to the vibes and forget the code exists, accepting all AI changes without reading the diffs and pasting in error messages until bugs go away.
The tweet that coined the term, February 2025
BBC News headline reading 'Vibe coding' named word of the year by Collins Dictionary, dated 6 November 2025, by technology reporter Laura Cress.
Less than a year later, Collins named it word of the year (BBC News, November 2025)

This has distorted the timeframes for each phase under the classic software development lifecycle. No longer do teams take weeks to implement and build a system, but days or even hours.

weeks → hoursTraditionalPlanBuildTestMaintainAI-acceleratedPlanTestMaintaintime saved
How AI has reshaped the software development lifecycle. (Anthropic, August 2026)

Notice that the time saved comes almost entirely from the build phase. You might expect evaluation to compress in the same way, and parts of it genuinely can: AI can draft test cases, run them at scale and even help judge outputs.

Yet, the heart of evaluation is not running tests, it is defining what "good" looks like and designing proper ways to test if "good" is achieved. A model building your system makes countless silent judgement calls about behaviour, tone and edge cases along the way, and what it settles on is not necessarily what you or your stakeholders would have chosen. Defining the criteria that matter, and checking the system thoroughly against them, remains human work.

If anything, handing the building over to AI makes evaluation all the more important. When code was written by hand, a baseline of scrutiny came built in: someone had at least read what they were shipping. Now a system can reach production without anyone having formed a clear picture of how it behaves. Evaluation is how you build that picture deliberately, and how you gather the evidence that the system performs as intended and is safe, fair and reliable. Skipping it costs the most in high-stakes settings, such as those within the public sector, where the impact of a system's failure can be catastrophic.

Hence, learning to evaluate our products rigorously has been growing in importance more than ever, and this playbook aims to address that.

How we approach evaluation​

Here in GovTech AI Practice, we approach evaluation through a "spec-driven lens". We believe that a sound evaluation is centred around understanding the criteria that matters to us and our stakeholders.

In doing so, everything else, from the evaluation design to the execution, will be grounded on the agreed-upon criteria, allowing our evaluations to answer the questions that matter to us and our stakeholders directly.

We can hence express our approach to evaluation in this diagram:

CORE EVALUATION LOOP · REPEATS AS YOU BUILDOUTER LOOP · AFTER LAUNCHafter launchfeeds back5MonitorAdapting evaluation design to be continuousin production environments.1DefineDefine what “good” looks like: intended tasks,expected outcomes and application requirements.2DesignDesign how to measure “good” and buildrealistic, trustworthy evaluation sets.a.Judgement DesignJudgement methods,required evidence,acceptance criteria.b.Evaluation SetDesignBuild realistic andtrustworthy evaluationsets.3ExecuteRun the test suite against the evaluation designconsistently and capture execution traces.a.Run the systemExecute the system andcollect artefacts andrelevant traces.b.Run the judgeRunning the evaluationdesign to assess capturedbehaviour and outcomes.4ImproveDiagnose test results and extract insights to guideapplication improvements and future evaluations.
An overview of our evaluation approach.

We believe that evaluation is an iterative process: the results of an evaluation serve to inform an improvement or a decision, whether to the system or to the evaluation methodology itself, which then require another evaluation.

We have therefore designed it as two connected loops:

  • Core evaluation loop (stages 1 to 4). The one you run as you build and test your system, to ensure it meets the criteria you and your stakeholders care about.
  • Outer monitoring loop (stage 5, Monitor). The one that enforces continuous evaluation even after launch, to ensure system performance and evaluation remains relevant over time.
Curious about other evaluation paradigms? Ours is not the only approach. Expand this box for a quick tour of common alternatives.

The evaluation ecosystem has grown quickly, with a wide range of open-source libraries, commercial platforms and established methods for assessing AI systems. Although non-exhaustive, here are some common paradigms you will encounter. Expand each one to see how it works, with its strengths and limitations:

Error-driven evals

Post-hoc analysis

Tools Parlance Labs, Braintrust

Ingests production traces after the fact to analyse observed failures and build targeted regression metrics.

Strengths
  • Targeted: fixes real friction observed in live systems
  • High-impact: focuses effort on proven failure cases
Limitations
  • Reactive: misses critical risks before deployment
  • Manual overhead: needs continuous trace inspection and mature logs

Metric-driven evals

CI/CD automation

Tools DeepEval, LangSmith, Arize

Runs predefined assertions and off-the-shelf LLM-as-a-judge metrics directly against inputs and test suites.

Strengths
  • Zero-touch: fast runs with no human in the loop
  • CI/CD ready: slots into pipelines for rapid iteration
Limitations
  • Surface scope: scores outputs only, missing tool calls and state drift
  • Generic defaults: prebuilt metrics lack domain and regulatory nuance

While these paradigms have their own merits, we ultimately favour a spec-driven paradigm because it grounds the evaluation in the criteria you and your stakeholders have agreed on, so the results answer the questions that matter to you directly and give you a firm basis for improving your system.

How the playbook is structured​

Here is an overview of the chapters, one per stage of the loop. Each card shows what you bring in, the takeaways you leave with, and an "Is this you?" situation describing the problem that chapter solves. If a situation sounds like yours, that chapter was written for you: head there first, and use You bring to check you have its starting materials in hand.

Core evaluation loop · repeats as you build
  1. DefineWhat “good” meansYou bring

    A product or prototype to evaluate, and access to the people accountable for it

    You leave with
    • A stakeholder-agreed list of the specifications for your product
    • Sign-off from the people accountable, so everyone evaluates for the same thing
  2. DesignJudgement design and evaluation set designYou bring

    A stakeholder-agreed list of the specifications

    You leave with
    • A judgement design for each specification: the artefacts, the judge and the judgement it returns
    • A basket of metrics, with acceptance criteria that say when each is good enough
    • A sourced, curated evaluation set that is representative, realistic and effective
  3. ExecuteRun the system, then the judgeYou bring

    Your evaluation objective, evaluation design and evaluation set

    You leave with
    • Runs and judgements consistent enough to trust across repeats
    • Traces captured at the depth your judgement design needs
    • An evaluation run that finishes within your time and cost budget
  4. ImproveDiagnose results and decide what to changeYou bring

    Results from a run you can trust and trace

    You leave with
    • Failures traced to the stage of the system that caused them
    • A decision, for each failure, on whether to fix the system or the evaluation
    • A prioritised list of changes, each with evidence behind it

After Improve, the loop returns to Define with what you have learned.

Outer loop · continues after launch
  1. MonitorKeep evaluation running in productionYou bring

    A trustworthy evaluation you can run on demand

    You leave with
    • An evaluation that runs continuously against live traffic
    • Signals for when the live system drifts from what “good” requires
    • A path that feeds production findings back into the loop
Don't know where to go?

If you are new and starting from scratch, begin with 1. Define.

Last updated on

Was this page helpful?