# ARC Framework Risk Register (WOG Version) > This is a markdown version of the Risks sheet from the ARC Framework risk register, intended for use with AI assistants when drafting the ARC self-assessment report. For the authoritative source and filterable view, refer to `risk_register.xlsx` or the [Interactive Risk Register](https://govtech-responsibleai.github.io/agentic-risk-capability-framework/arc_framework/risk-register/). **How to use this file:** - Skim the Elements table below to identify which components, design aspects, and capabilities are present in your system. - Then jump to the corresponding section of risks to find those relevant to your use case. - For the controls mapped to each risk, refer to the companion file `controls.md`. --- ## Elements Reference Agentic systems are decomposed into three types of elements: **Components** (the building blocks), **Design** (how they are assembled), and **Capabilities** (what the system can do). | ID | Name | Category | Description | | --- | --- | --- | --- | | CMP-01 | LLM | Component - LLM | A large language model (LLM) is a neural network trained on massive text data to learn statistical patterns of language, enabling it to understand, generate, and reason over natural language inputs. It is the core reasoning engine that processes instructions, interprets user inputs, and generates contextually appropriate responses by leveraging its trained language understanding and generation capabilities. There are a large variety of LLMs to choose from, with different sizes, capabilities, and architectures, from large closed-source LLMs such as OpenAI's GPT-5.2, Anthropic's Claude 4 Opus, or Gemini 3 Pro, to smaller open-weights models like Llama 3.1 8B, Qwen3 8B, and Mistral Small 3.1. | | CMP-02 | Instructions | Component - Instructions | Instructions are structured inputs provided to an LLM that define the task, constraints, and context, guiding how the model interprets inputs and generates outputs. They guide the LLM's decision process by conditioning prompts and tool use with objectives, policies, and guardrails. Forms include system prompts, policies, schemas, and rubrics, varying by framework and enforcement strictness. | | CMP-03 | Tools | Component - Tools | Tools are external functions, APIs, or resources attached to an LLM that extend its capabilities beyond text generation, enabling it to retrieve information, execute actions, and affect external systems (e.g., search, code execution, database queries, or file manipulation), typically through structured interfaces such as the Model Context Protocol (MCP) that constrain invocation, inputs, permissions, and outputs for agentic control and safety. | | CMP-04 | Memory | Component - Memory | Memory is a persistent or semi-persistent data store attached to an LLM that retains information across interactions—such as conversation history, user preferences, task state, and retrieved knowledge—enabling continuity, long-horizon reasoning, and personalization in agentic workflows. Memory may be implemented through mechanisms such as context windows, vector databases, episodic logs, or external state stores, each offering different trade-offs in latency, fidelity, and persistence. | | DES-01 | Agentic Architecture | Design - Agentic Architecture | The agentic architecture defines how multiple agents are structured, interconnected, and coordinated to collectively perform tasks that exceed the capabilities of a single agent. This includes orchestration patterns such as hierarchical delegation, parallel agent execution, sequential handoffs between specialised agents, shared or centralized planning components, and the communication protocols that govern information exchange and coordination across the system. | | DES-02 | Roles and Access Controls | Design - Roles and Access Controls | Roles and access controls define and enforce differentiated roles, permissions, and scopes of authority across agents and system components, specifying what actions each agent is allowed to perform and which resources it may access. This includes assigning functional roles to agents, configuring tool- and data-level permissions, scoping credentials or identities, and enforcing access policies that govern interactions with files, systems, services, or other agents. | | DES-03 | Monitoring and Traceability | Design - Monitoring and Traceability | Monitoring and traceability provide systematic visibility into agent behaviour, interactions, and decision pathways by recording, observing, and correlating agent actions and system events over time. This includes logging agent inputs and outputs, tracking tool invocations and state changes, capturing decision traces or execution paths, and surfacing telemetry or audit records that support analysis and review of system behaviour. | | CAP-01 | Planning and Goal Management | Capability - Cognitive | The capability to autonomously develop multi-step workflows and action plans for completing government service delivery tasks, policy implementation activities, or operational processes based on high-level objectives or citizen requests. Includes task prioritisation aligned with government priorities, monitoring execution progress, and dynamic replanning when encountering regulatory constraints, approval requirements, or operational obstacles. | | CAP-02 | Agent Delegation | Capability - Cognitive | The capability to autonomously delegate subtasks to specialised agents or systems across government agencies and coordinate their activities to achieve whole-of-government service delivery objectives. Includes identifying which agency systems or specialised agents are best suited for specific tasks, issuing instructions through approved interfaces, managing cross-agency dependencies and handoffs, and monitoring execution progress or failures across the multi-agent workflow. | | CAP-03 | Tool Use | Capability - Cognitive | The capability to autonomously evaluate available government systems, APIs, and MCP servers, and select the most appropriate ones for specific subtasks based on their capabilities, limitations, and suitability to the task requirements. Includes selecting between search engines, computational tools, code execution environments, or domain-specific government APIs accessible through GovTech's central MCP server registry or agency-specific MCP servers, determining when tool invocation is necessary, and sequencing or combining multiple tools through approved interfaces to complete complex cross-system tasks effectively. | | CAP-04 | Multimodal Understanding and Generation | Capability - Interaction | The capability to communicate with citizens, public officers, and external stakeholders across multiple modalities in support of government service delivery, using English as the official working language. Includes natural language conversation for explaining policies, generating official documents, and conducting interactive discussions, as well as multimodal understanding and generation such as processing identity documents, medical images, infrastructure photos, or creating multimedia content for government communications. | | CAP-05 | Official Communication | Capability - Interaction | The capability to autonomously compose, finalise, and dispatch authoritative communications that formally and legally represent a government agency to external parties (citizens, businesses, other agencies, regulators, courts, or media) via approved channels and formats, without prior human review or approval, thereby creating potential legal, regulatory, or reputational obligations. This includes sending legally binding correspondence, publishing official statements or press releases through Gov.sg channels, and responding to external enquiries using the agency's official identity. | | CAP-06 | Business Transactions | Capability - Interaction | The capability to autonomously initiate, authorise, and execute binding business transactions with external parties—such as payments, purchases, reservations, grants, or service commitments—within predefined authorisation limits, resulting in real financial, contractual, or operational obligations for the government agency. This includes processing payments or refunds, placing orders or subscriptions, booking services or reservations, disbursing grants or subsidies, and accepting or triggering contractual commitments on behalf of the agency. | | CAP-07 | Internet and Search Access | Capability - Interaction | The capability to autonomously access, browse, search, and retrieve information from the Internet and approved external data sources to augment the LLM's static training knowledge with external and up-to-date sources in support of government task execution and response generation. This includes issuing search queries, following and parsing web pages, extracting relevant facts or documents from trusted sources, and aggregating information from multiple online sources while adhering to government security policies and acceptable use guidelines. | | CAP-08 | Computer Use | Capability - Interaction | The capability to directly operate a computer's graphical user interface on behalf of government officers, enabling the agent to navigate government applications and execute tasks through mouse, keyboard, and window-based interactions. This includes moving the cursor, clicking buttons, entering text into forms, using keyboard shortcuts, switching between windows and applications, and navigating files and menus within the government desktop environment or virtual desktop infrastructure (VDI). | | CAP-09 | Other Programmatic Interfaces | Capability - Interaction | The capability to interact with government systems and external platforms through non-graphical, programmatic interfaces such as APIs, SDKs, and backend services to exchange data or trigger actions as part of task execution. This includes calling REST or GraphQL APIs through government API gateways (such as APEX), invoking cloud services on Government Commercial Cloud (GCC), publishing or consuming messages from queues or event streams, and performing operations such as code deployments, data updates, or system integrations across government enterprise platforms. | | CAP-10 | Code Execution | Capability - Operational | The capability to write, execute, and debug code in various programming languages to automate government tasks or solve computational problems. This includes implementing algorithms for policy calculations, writing scripts to process government data, compiling and running code in secure environments, debugging errors in automation workflows, and integrating with government systems through APIs or backend services. | | CAP-11 | File and Data Management | Capability - Operational | The capability to manage the full lifecycle of files and data by creating, reading, modifying, organising, converting, querying, and updating information across both unstructured artefacts (documents, spreadsheets, PDFs, media files) and structured data stores (SQL/NoSQL databases, data warehouses, vector stores) in support of government operational tasks. This includes ingesting and transforming datasets, generating or updating government documents, maintaining directory or schema structures, executing database queries or updates on government systems, and storing or retrieving embeddings or derived data products. | | CAP-12 | System Management | Capability - Operational | The capability to directly manage and configure government technical systems and infrastructure by adjusting system settings, controlling computing resources, and administering operational environments across on-premise data centres or Government Commercial Cloud platforms. This includes monitoring system health and performance, managing authentication credentials and access controls in compliance with IM8, provisioning or scaling compute and storage resources, configuring operating system or runtime parameters, and applying system-level optimisations to support reliable government service delivery. | --- ## Risks There are **46 risks** in total, grouped below by element category. ### Component - LLM *3 risk(s) in this category.* #### RISK-001: Use of untrusted or compromised LLMs - **Element:** LLM (Component - LLM) - **Failure mode:** External Manipulation - **Type:** Safety, Security - **Number of mapped controls:** 5 - **Sources:** https://arxiv.org/abs/2408.02946v6 **Description** This risk arises when LLMs obtained from untrusted or insufficiently vetted sources have been intentionally poisoned or backdoored during training or distribution, causing them to behave maliciously or unpredictably under specific conditions. Such models may leak sensitive information, bypass safeguards, or execute hidden behaviours that undermine system integrity and trust. **WOG context** Agencies can access LLMs through various modalities, including GovTech's LaunchPad AI platform, GCC AI services (Azure OpenAI, AWS Bedrock), commercial vendors, and open-source repositories like Hugging Face. When models are deployed without proper vetting through GovTech's security review process, or when fine-tuned models from external contractors are not adequately tested, agencies are exposed to risks from backdoored or poisoned models. For government systems handling citizen data via MyInfo, policy information, or operational decisions affecting Singaporeans, compromised models could leak sensitive data, generate misleading policy advice, or undermine public trust in digital government services. #### RISK-002: Insufficient alignment of LLM behaviour - **Element:** LLM (Component - LLM) - **Failure mode:** Agent Failure - **Type:** Safety, Security - **Number of mapped controls:** 5 - **Sources:** https://arxiv.org/abs/2406.10162 **Description** This risk arises when an LLM's learned objectives and behaviors do not reliably align with intended user goals, system instructions, or organizational policies, leading to inappropriate, unsafe, or undesired outputs. Misalignment may surface as failure to follow constraints, inconsistent reasoning, or behavior that diverges from expected norms in edge cases or complex scenarios. **WOG context** LLMs deployed in government contexts must reliably follow public service standards, agency-specific policies, regulatory requirements. Misalignment risks are particularly acute when agents handle statutory functions, citizen-facing services, or cross-agency workflows. An LLM that fails to align with the public service values, IM8 security policies, PDPA-equivalent data handling standards for public agencies, or agency-specific missions may produce outputs that violate the public service spirit or government protocols. #### RISK-003: Insufficient LLM capability and reliability - **Element:** LLM (Component - LLM) - **Failure mode:** Agent Failure - **Type:** Safety, Security - **Number of mapped controls:** 4 - **Sources:** https://arxiv.org/abs/2505.00212 **Description** This risk arises when an LLM lacks sufficient capability, robustness, or reasoning performance to correctly interpret instructions, handle edge cases, or detect unsafe situations. As a result, the model may produce incorrect, misleading, or unsafe outputs that create downstream safety or security failures in systems that rely on its judgments. **WOG context** Government services require high reliability as errors directly impact citizens' lives. An LLM lacking sufficient capability may fail to correctly interpret complex eligibility criteria, mishandle edge cases in policy implementation, or fail to recognise when a citizen's situation requires human officer intervention. Given Singapore's high digital service adoption rate and citizen expectations for accurate government information, capability gaps can erode public trust and cause tangible harm. --- ### Component - Instructions *2 risk(s) in this category.* #### RISK-008: Vague or underspecified instructions - **Element:** Instructions (Component - Instructions) - **Failure mode:** Agent Failure - **Type:** Safety, Security - **Number of mapped controls:** 3 - **Sources:** https://arxiv.org/abs/2502.13295, https://arxiv.org/abs/2505.13360v1 **Description** This risk arises when instructions provided to an LLM are ambiguous, incomplete, or poorly scoped, leading the model to make unintended assumptions when interpreting tasks or constraints. As a result, the LLM may behave unpredictably, bypass safeguards, or take actions that introduce safety or security risks. **WOG context** Government agents require precise instructions to operate within statutory boundaries, policy guidelines, and service standards. Vague instructions in the public sector context can lead agents to make assumptions about eligibility criteria, approval thresholds, or escalation procedures that contradict official policy. This is particularly risky when agents handle discretionary decisions, means-testing calculations, or cross-agency referrals where the boundaries of authority must be clearly defined. #### RISK-009: Unsanitised inputs in system instructions - **Element:** Instructions (Component - Instructions) - **Failure mode:** External Manipulation - **Type:** Safety, Security - **Number of mapped controls:** 2 - **Sources:** https://arxiv.org/abs/2502.15851v1, https://aclanthology.org/2025.naacl-long.425.pdf **Description** This risk arises when untrusted or user-controlled inputs are incorporated into system instructions without proper sanitisation or validation. As a result, malicious or malformed content may manipulate the model's behaviour, override intended constraints, or trigger unintended actions. **WOG context** Government agents often personalise responses using citizen data retrieved from MyInfo, Singpass profiles, or agency databases. When this data is incorporated into system prompts without sanitisation, attackers could craft malicious profile data or input fields that inject instructions into the agent's system prompt. This could manipulate the agent's behaviour to bypass security controls, reveal confidential information, or take unauthorised actions on behalf of the government. --- ### Component - Tools *4 risk(s) in this category.* #### RISK-004: Weak tool authentication and authorisation controls - **Element:** Tools (Component - Tools) - **Failure mode:** Tool or Resource Malfunction - **Type:** Security - **Number of mapped controls:** 3 - **Sources:** https://arxiv.org/abs/2504.08623, https://arxiv.org/abs/2505.14590, https://www.redhat.com/en/blog/model-context-protocol-mcp-understanding-security-risks-and-controls **Description** This risk arises when tools connected to an agent lack robust authentication or fine-grained authorisation mechanisms, allowing unauthorised access or misuse of tool capabilities. As a result, attackers or misbehaving agents may compromise the system by invoking sensitive actions, escalating privileges, or manipulating external resources beyond intended boundaries. **WOG context** Government agentic applications often integrate with multiple upstream and downstream API services to deliver end-to-end citizen services. Weak authentication or authorisation controls at any integration point can create vulnerabilities that propagate across the service chain. The interconnected nature of whole-of-government architecture means a single weak tool authentication mechanism could provide attack vectors across multiple agency systems and data sources. #### RISK-005: Lack of proper role-based access control for tools - **Element:** Tools (Component - Tools) - **Failure mode:** Tool or Resource Malfunction - **Type:** Security - **Number of mapped controls:** 2 - **Sources:** https://embracethered.com/blog/posts/2023/chatgpt-plugin-vulns-chat-with-code/ **Description** This risk arises when tools exposed to an agent do not enforce clear, role-based access controls, allowing agents to access capabilities or resources beyond their intended responsibilities. As a result, agents may perform unauthorised actions, misuse sensitive tools, or exceed their permitted scope, increasing the likelihood of security and operational failures. **WOG context** Government agents operate within defined roles corresponding to specific job functions—citizen service officers, policy analysts, enforcement personnel—each with distinct tool access requirements. Without proper RBAC, an agent designed for citizen enquiries might access enforcement databases, or a policy research agent might invoke transaction processing tools. This is particularly critical in Singapore's whole-of-government context where agents may need to interact with tools across multiple agencies through central WOG services. #### RISK-006: Tool poisoning by malicious actors - **Element:** Tools (Component - Tools) - **Failure mode:** External Manipulation - **Type:** Safety, Security - **Number of mapped controls:** 2 - **Sources:** https://www.mbgsec.com/archive/2025-05-03-mcp-untrusted-servers-and-confused-clients-plus-a-sneaky-exploit-embrace-the-red/ **Description** This risk arises when tools or their interfaces are intentionally modified, compromised, or replaced by malicious actors to introduce harmful or deceptive behaviour when invoked by an agent. As a result, the agent may unknowingly execute malicious actions, leak sensitive information, or produce manipulated outputs that undermine system integrity and trust. **WOG context** Agencies may deploy MCP servers and custom tools from various sources including vendor-provided solutions and open-source repositories. A poisoned tool in the government ecosystem could compromise data integrity, manipulate citizen service outcomes, or exfiltrate sensitive government data. #### RISK-007: Lack of input sanitisation - **Element:** Tools (Component - Tools) - **Failure mode:** Tool or Resource Malfunction - **Type:** Safety, Security - **Number of mapped controls:** 1 - **Sources:** https://arxiv.org/abs/2503.12188v1, https://www.cve.org/CVERecord?id=CVE-2024-7042 **Description** This risk arises when inputs passed from the agent to tools are not properly validated or sanitised, allowing malformed or malicious data to be processed. As a result, tools may be exploited through injection attacks, unintended command execution, or data corruption. **WOG context** Government agents process inputs from citizens, external systems, and inter-agency communications. Without proper sanitisation, these inputs could exploit vulnerabilities in connected government systems. Given that government tools often have elevated privileges to access citizen data and could perform statutory functions, successful injection attacks could have severe consequences for data integrity and system security. --- ### Component - Memory *2 risk(s) in this category.* #### RISK-010: Poisoned memory - **Element:** Memory (Component - Memory) - **Failure mode:** External Manipulation - **Type:** Safety, Security - **Number of mapped controls:** 3 - **Sources:** https://arxiv.org/abs/2505.11548v2, https://arxiv.org/abs/2402.07867, https://openreview.net/pdf?id=6SIymOqJlc **Description** This risk arises when the memory component of an agentic system is intentionally or inadvertently populated with malicious, misleading, or corrupted information. As a result, the agent may rely on compromised memory to make decisions, propagate false information, or exhibit persistent unsafe behaviour across interactions. **WOG context** Government agents with persistent memory may store citizen interaction histories, learned preferences, and contextual information across sessions. If an attacker can poison this memory - either through crafted interactions or by compromising memory storage - the agent could persistently provide incorrect information, develop biased decision patterns, or carry out long-term manipulation campaigns affecting citizen services. #### RISK-011: Sensitive data leakage across memory contexts - **Element:** Memory (Component - Memory) - **Failure mode:** Agent Failure - **Type:** Safety, Security - **Number of mapped controls:** 2 - **Sources:** https://arxiv.org/abs/2506.12699v2 **Description** This risk arises when the memory component retains or exposes sensitive information across sessions, tasks, or users with different scopes or authorisations. As a result, data may be inappropriately accessed or reused in unrelated contexts, leading to privacy breaches, confidentiality violations, or unauthorised disclosure. **WOG context** Government agents handle sensitive citizen data including NRIC numbers, income information, medical records, and family circumstances. Memory components that fail to properly isolate this data across sessions or users could leak information between citizens, enable unauthorised officers to access restricted data, or expose sensitive information when agents are queried about their prior interactions. --- ### Design - Agentic Architecture *3 risk(s) in this category.* #### RISK-012: Cascading errors in multi-agent architectures - **Element:** Agentic Architecture (Design - Agentic Architecture) - **Failure mode:** Agent Failure - **Type:** Safety, Security - **Number of mapped controls:** 2 - **Sources:** https://arxiv.org/abs/2408.00989v3, https://arxiv.org/pdf/2502.19145 **Description** This risk arises when errors or misjudgements produced by one agent propagate through interconnected agents within a multi-agent system. As a result, small failures may compound across agent interactions, leading to amplified errors, degraded system performance, or unintended outcomes at the system level. **WOG context** Singapore's whole-of-government service delivery often requires multi-agent coordination across agencies - a citizen's grant application might involve agents from MSF, IRAS, HDB, and CPF working together. Errors in one agent's assessment can cascade through the workflow, compounding at each stage. For example, an incorrect income verification from IRAS could affect MSF assistance eligibility, HDB grant calculations, and CPF top-up recommendations simultaneously. #### RISK-013: Man-in-the-middle attacks between agents - **Element:** Agentic Architecture (Design - Agentic Architecture) - **Failure mode:** External Manipulation - **Type:** Security - **Number of mapped controls:** 2 - **Sources:** https://arxiv.org/pdf/2502.14847 **Description** This risk arises when communication channels between agents are insufficiently secured, allowing an attacker to intercept, modify, or replay messages exchanged within the agentic system. As a result, agents may act on tampered information, leading to incorrect coordination, unauthorised actions, or compromised system behaviour. **WOG context** Inter-agency agent communications in Singapore's whole-of-government architecture may traverse different network segments, from agency intranets to GCC infrastructure to external partner networks. Attackers intercepting these communications could modify citizen data in transit, alter approval decisions, or inject false instructions. The risk is heightened for agents exchanging sensitive information like NRIC-linked data, financial details, or approval tokens. #### RISK-014: Feedback loops and runaway agent behaviour - **Element:** Agentic Architecture (Design - Agentic Architecture) - **Failure mode:** Agent Failure - **Type:** Safety, Security - **Number of mapped controls:** 2 **Description** This risk arises when agents repeatedly reinforce each other's decisions, outputs, or errors within an agentic architecture. As a result, feedback loops may form that escalate actions, consume excessive resources, or cause the system to persist in harmful or unintended behaviour without effective human intervention. **WOG context** Multi-agent government systems processing high volumes of citizen transactions could develop feedback loops where agents repeatedly validate each other's outputs without independent verification. In automated workflows for grants, permits, or enforcement actions, such loops could result in mass incorrect approvals, denial of legitimate applications, or resource exhaustion on government infrastructure. --- ### Design - Roles and Access Controls *2 risk(s) in this category.* #### RISK-015: Overly permissive roles and permissions - **Element:** Roles and Access Controls (Design - Roles and Access Controls) - **Failure mode:** Agent Failure - **Type:** Security - **Number of mapped controls:** 2 - **Sources:** https://cyberweapons.medium.com/escaping-reality-privilege-escalation-in-gen-ai-admin-panel-aka-the-chaos-of-a-misconfigured-b6ad73bf1b65 **Description** This risk arises when agents are granted roles or permissions that exceed their intended responsibilities or operational needs. As a result, agents may access sensitive resources, invoke high-impact capabilities, or perform unauthorised actions that increase the likelihood of security, privacy, or operational failures. **WOG context** Government agents require carefully scoped permissions aligned with the principle of least privilege as mandated by IM8 security policies. Overly permissive roles could enable agents to access citizen data across agencies unnecessarily, invoke transaction tools beyond their operational scope, or modify system configurations that should require elevated approval. #### RISK-016: Unauthorised privilege escalation - **Element:** Roles and Access Controls (Design - Roles and Access Controls) - **Failure mode:** Agent Failure - **Type:** Security - **Number of mapped controls:** 2 - **Sources:** https://arxiv.org/abs/2505.19301 **Description** This risk arises when agents are able to gain elevated roles or permissions beyond those initially granted, whether through misconfiguration, exploitation, or unintended system behaviour. As a result, agents may bypass intended controls, access restricted resources, or execute actions that undermine system security and governance. **WOG context** Government agents operating within Singapore's hierarchical approval structures must not be able to escalate their own privileges. An agent that can elevate its permissions could bypass financial approval thresholds, access restricted national security data, or approve transactions that should require senior officer authorisation. This risk is particularly acute for agents with tool-use capabilities that might exploit misconfigured permission systems. --- ### Design - Monitoring and Traceability *2 risk(s) in this category.* #### RISK-017: Delayed failure detection due to limited monitoring - **Element:** Monitoring and Traceability (Design - Monitoring and Traceability) - **Failure mode:** Agent Failure - **Type:** Safety, Security - **Number of mapped controls:** 2 - **Sources:** https://arxiv.org/abs/2401.13138 **Description** This risk arises when monitoring systems provide insufficient visibility into agent behaviour, system events, or execution outcomes. As a result, failures, anomalies, or unintended actions may go undetected for extended periods, increasing the impact and difficulty of remediation. **WOG context** Government agents processing thousands of citizen transactions daily require comprehensive monitoring to detect failures promptly. Delayed detection of incorrect eligibility determinations, erroneous payments, or data breaches can affect large numbers of citizens before remediation begins. #### RISK-018: Inability to audit failures due to missing decision traces - **Element:** Monitoring and Traceability (Design - Monitoring and Traceability) - **Failure mode:** Agent Failure - **Type:** Security - **Number of mapped controls:** 2 **Description** This risk arises when monitoring systems do not capture sufficient reasoning steps, decision pathways, or execution context for agent actions. As a result, operators may be unable to reconstruct failures, understand why specific outcomes occurred, or conduct effective audits and post-incident reviews. **WOG context** Government accountability requires the ability to explain and justify decisions affecting citizens. Agents making eligibility determinations, enforcement decisions, or service outcomes must maintain complete audit trails that satisfy public sector governance requirements. Without decision traces, agencies cannot respond to parliamentary questions, citizen appeals, or audit queries about specific cases. --- ### Capability - Cognitive *5 risk(s) in this category.* #### RISK-019: Generating plans that fail to meet the user's requirements - **Element:** Planning and Goal Management (Capability - Cognitive) - **Failure mode:** Agent Failure - **Type:** Safety, Security - **Number of mapped controls:** 3 - **Sources:** https://arxiv.org/pdf/2402.01622v4, https://aclanthology.org/2025.naacl-long.93.pdf **Description** This risk arises when an agent generates plans or goals that do not accurately reflect the user's stated objectives, constraints, or preferences. As a result, the system may pursue incorrect or suboptimal actions, waste resources, or deliver outcomes that do not satisfy user expectations. **WOG context** Government agents must accurately understand citizen needs and translate them into appropriate service delivery plans. Misunderstanding a citizen's actual requirements leads to wasted effort and frustrated citizens, undermining public trust. #### RISK-020: Generating plans that overlook safety implications - **Element:** Planning and Goal Management (Capability - Cognitive) - **Failure mode:** Agent Failure - **Type:** Safety - **Number of mapped controls:** 3 - **Sources:** https://garymarcus.substack.com/p/ai-still-lacks-common-sense-70-years **Description** This risk arises when an agent generates plans or goals without adequately considering basic safety, security, or practical constraints that would be apparent to a human. As a result, the system may propose or pursue actions that are unsafe, insecure, or inappropriate despite being technically feasible. **WOG context** Government agents must consider safety implications that extend beyond immediate task completion, including citizen welfare, regulatory compliance, and public interest. An agent focused solely on efficiency might overlook mandatory cooling-off periods, required safety checks, or situations where immediate action could cause harm. These considerations are especially important for agents handling healthcare, enforcement, or vulnerable citizen services. #### RISK-021: Incorrect task delegation between agents - **Element:** Agent Delegation (Capability - Cognitive) - **Failure mode:** Agent Failure - **Type:** Safety, Security - **Number of mapped controls:** 1 - **Sources:** https://arxiv.org/abs/2503.13657 **Description** This risk arises when an agent assigns tasks to other agents that do not match their capabilities, roles, or access permissions. As a result, tasks may be executed incorrectly, fail to complete, or introduce security and operational issues due to inappropriate delegation. **WOG context** Whole-of-government service delivery requires accurate routing of tasks to appropriate agency agents with correct capabilities and authorisations. Incorrect delegation wastes resources, delays service delivery, and may result in inappropriate actions being taken by agents lacking the necessary context or authority. #### RISK-022: Malicious or manipulative use of delegated agents - **Element:** Agent Delegation (Capability - Cognitive) - **Failure mode:** Agent Failure - **Type:** Safety, Security - **Number of mapped controls:** 4 - **Sources:** https://arxiv.org/abs/2507.06850 **Description** This risk arises when an agent deliberately assigns tasks to other agents in ways intended to bypass controls, obscure responsibility, or achieve malicious objectives. As a result, delegated agents may be coerced into performing unauthorised actions, amplifying harmful behaviour or evading detection within the system. **WOG context** In multi-agent government architectures, a compromised or manipulated agent could abuse delegation capabilities to launder requests through trusted agents, bypass access controls, or distribute malicious operations across multiple agents to evade detection. The trust relationships between government agents make such manipulation particularly dangerous. #### RISK-023: Incorrect tool selection or misuse - **Element:** Tool Use (Capability - Cognitive) - **Failure mode:** Agent Failure - **Type:** Safety, Security - **Number of mapped controls:** 3 - **Sources:** https://arxiv.org/abs/2411.13547 **Description** This risk arises when an agent selects an inappropriate tool or applies a tool incorrectly for a given task or action. As a result, the agent may produce erroneous outcomes, fail to complete the task effectively, or trigger unintended side effects due to misuse of tool capabilities. **WOG context** Government agents access numerous tools through MCP servers, APEX and direct integrations. Selecting the wrong tool or misusing tool parameters could result in incorrect citizen data being retrieved, wrong calculations being applied, or unintended transactions being initiated. The consequences range from service delays to incorrect statutory decisions. --- ### Capability - Interaction *15 risk(s) in this category.* #### RISK-024: Generation of undesirable content - **Element:** Multimodal Understanding and Generation (Capability - Interaction) - **Failure mode:** Agent Failure - **Type:** Safety - **Number of mapped controls:** 1 - **Sources:** https://arxiv.org/abs/2402.04249v2 **Description** This risk arises when an agent generates text, images, audio, or other media that contain toxic, hateful, sexual, or otherwise inappropriate content. As a result, the system may cause harm to users, violate organisational standards or regulations, or undermine trust in the system's outputs. **WOG context** Government agents communicating with citizens must maintain standards appropriate for official communications. Generating content that is offensive, discriminatory, or inappropriate would damage public trust in digital government services and potentially violate Singapore's laws on harassment, discrimination, or sedition. This is especially critical for citizen-facing services on Gov.sg platforms. #### RISK-025: Generation of unqualified advice in specialised domains - **Element:** Multimodal Understanding and Generation (Capability - Interaction) - **Failure mode:** Agent Failure - **Type:** Safety - **Number of mapped controls:** 1 - **Sources:** https://www.edpb.europa.eu/system/files/2025-04/ai-privacy-risks-and-mitigations-in-llms.pdf **Description** This risk arises when an agent generates advice or guidance in specialised domains such as medical, financial, or legal contexts without appropriate expertise, validation, or safeguards. As a result, users may act on incorrect or inappropriate information, leading to potential harm or adverse outcomes. **WOG context** Government agents may be asked for advice touching on medical, legal, financial, or other specialised domains. Providing unqualified advice in these areas could cause citizens to make harmful decisions—medical self-treatment instead of seeking care, legal strategies that prejudice their cases, or financial decisions that result in losses. Agents must recognise domain boundaries and direct citizens to appropriate qualified professionals. #### RISK-026: Generation of controversial or sensitive content - **Element:** Multimodal Understanding and Generation (Capability - Interaction) - **Failure mode:** Agent Failure - **Type:** Safety - **Number of mapped controls:** 1 - **Sources:** https://news.stanford.edu/stories/2025/05/ai-models-llms-chatgpt-claude-gemini-partisan-bias-research-study **Description** This risk arises when an agent generates content related to sensitive or controversial topics, such as political commentary or denigrating comments about competitors. As a result, the system may create reputational, legal, or compliance issues, or be perceived as biased, inappropriate, or misrepresentative of organisational views. **WOG context** Government agents must maintain political neutrality and avoid generating content on sensitive topics including race, religion, politics, and matters affecting Singapore's social cohesion. Agents generating inappropriate commentary could damage the government's reputation for neutrality and fairness. #### RISK-027: Regurgitating personally identifiable information - **Element:** Multimodal Understanding and Generation (Capability - Interaction) - **Failure mode:** Agent Failure - **Type:** Safety, Security - **Number of mapped controls:** 1 - **Sources:** https://www.edpb.europa.eu/system/files/2025-04/ai-privacy-risks-and-mitigations-in-llms.pdf **Description** This risk arises when an agent reproduces personally identifiable information in its generated outputs, whether drawn from training data, memory, or prior interactions. As a result, the system may violate privacy obligations, expose individuals to harm, or breach data protection requirements. **WOG context** Government agents process vast amounts of citizen PII including NRIC numbers, addresses, income information, medical records, and family details. Agents that regurgitate PII from training data, memory, or prior sessions could expose citizen information inappropriately. While public agencies have exemptions from PDPA, they are still bound by Public Sector Governance Act requirements and the Government Instruction Manual on data protection. #### RISK-028: Generation of non-factual or hallucinated content - **Element:** Multimodal Understanding and Generation (Capability - Interaction) - **Failure mode:** Agent Failure - **Type:** Safety - **Number of mapped controls:** 3 - **Sources:** https://arxiv.org/abs/2309.01219 **Description** This risk arises when an agent generates information that is inaccurate, fabricated, or unsupported by evidence while presenting it as factual. As a result, users may be misled, make incorrect decisions, or lose trust in the system's outputs. **WOG context** Government agents must provide accurate information as citizens rely on official sources for critical decisions about housing, retirement, healthcare, and legal matters. Hallucinated policy details, incorrect eligibility criteria, or fabricated procedures could cause citizens to miss application deadlines, misunderstand their entitlements, or take actions based on false information. This undermines the reliability of digital government services. #### RISK-029: Generation of copyrighted content - **Element:** Multimodal Understanding and Generation (Capability - Interaction) - **Failure mode:** Agent Failure - **Type:** Safety - **Number of mapped controls:** 1 - **Sources:** https://arxiv.org/abs/2407.07087 **Description** This risk arises when an agent generates content that reproduces or closely resembles copyrighted material without appropriate rights or attribution. As a result, the system may infringe intellectual property laws, expose the organisation to legal liability, or violate licensing and usage terms. **WOG context** Government agents generating content for official communications, educational materials, or public engagement must avoid reproducing copyrighted material. Singapore's Copyright Act applies to government use, and agencies must respect intellectual property rights. Agents generating content that infringes copyright could expose agencies to legal liability and reputational damage. #### RISK-030: Misrepresentation of authorship - **Element:** Official Communication (Capability - Interaction) - **Failure mode:** Agent Failure - **Type:** Safety - **Number of mapped controls:** 1 - **Sources:** https://fortune.com/article/customer-support-ai-cursor-went-rogue/ **Description** This risk arises when recipients are misled about whether an official communication was authored by a human or generated by an agent on behalf of the organisation. As a result, stakeholders may form incorrect assumptions about accountability, intent, or authority, potentially leading to trust, legal, or reputational issues. **WOG context** Government communications carry legal and administrative weight. Citizens receiving official letters, notices, or responses expect these to reflect considered human judgement by accountable officers. Misrepresenting AI-generated content as human-authored undermines this trust relationship and may create issues when citizens seek to appeal decisions or hold officers accountable. #### RISK-031: Inaccurate promises or statements in official communications - **Element:** Official Communication (Capability - Interaction) - **Failure mode:** Agent Failure - **Type:** Safety - **Number of mapped controls:** 3 - **Sources:** https://the-decoder.com/people-buy-brand-new-chevrolets-for-1-from-a-chatgpt-chatbot/ **Description** This risk arises when an agent makes commitments, assurances, or public statements that are incorrect, unsupported, or exceed the organisation's actual intentions or capabilities. As a result, the organisation may face reputational damage, legal exposure, or loss of public trust due to unmet expectations or misinformation. **WOG context** Government agents making official communications must not make promises or commitments that exceed their authority or agency capabilities. Incorrect statements about processing timelines, guaranteed outcomes, or special arrangements could create legitimate expectations that agencies cannot fulfil, potentially exposing the government to legal challenges or damaging public trust in official communications. #### RISK-032: Unauthorised execution of business transactions - **Element:** Business Transactions (Capability - Interaction) - **Failure mode:** Agent Failure - **Type:** Security - **Number of mapped controls:** 2 - **Sources:** https://www.emergingtechbrew.com/stories/2025/05/29/ai-agents-vulnerable-financial-attacks **Description** This risk arises when an agent initiates, authorises, or executes business transactions outside predefined approval thresholds, roles, or authorisation limits. As a result, the organisation may be exposed to unintended financial losses, binding contractual obligations, or operational commitments that were not properly sanctioned. **WOG context** Government agents with transaction capabilities must operate within the approval limits and delegation authorities defined by the agency. Unauthorised transactions—whether grant disbursements, procurement commitments, or payment authorisations—could result in improper use of public funds and audit findings. #### RISK-033: Leakage of transaction credentials - **Element:** Business Transactions (Capability - Interaction) - **Failure mode:** Tool or Resource Malfunction - **Type:** Security - **Number of mapped controls:** 2 - **Sources:** https://arxiv.org/pdf/2506.01055 **Description** This risk arises when credentials, tokens, or sensitive authentication information used to execute business transactions are exposed, mishandled, or improperly stored by an agent or its supporting systems. As a result, malicious parties may gain the ability to initiate unauthorised transactions, manipulate financial operations, or compromise transactional systems. **WOG context** Government transaction credentials provide access to public funds and sensitive operations. Credential leakage could enable fraudulent transactions, unauthorised procurement, or manipulation of financial systems. IM8 security requirements mandate strict credential protection. #### RISK-034: Prompt injection via malicious websites - **Element:** Internet and Search Access (Capability - Interaction) - **Failure mode:** External Manipulation - **Type:** Safety, Security - **Number of mapped controls:** 3 - **Sources:** https://unit42.paloaltonetworks.com/agentic-ai-threats/ **Description** This risk arises when an agent retrieves or processes content from malicious or untrusted websites that are designed to inject instructions or manipulative prompts into the system. As a result, the agent may follow unintended commands, override intended constraints, or take actions that compromise system behaviour or integrity. **WOG context** Government agents with internet access capabilities may retrieve content from websites for research, verification, or information augmentation. Malicious websites could embed prompt injection attacks in their content to manipulate agent behaviour—potentially causing agents to leak sensitive data, bypass security controls, or take unauthorised actions. This risk extends to government agents accessing external databases, news sources, or reference materials. #### RISK-035: Unreliable information or websites - **Element:** Internet and Search Access (Capability - Interaction) - **Failure mode:** Tool or Resource Malfunction - **Type:** Safety - **Number of mapped controls:** 1 - **Sources:** https://www.techradar.com/computing/artificial-intelligence/googles-ai-overviews-are-often-so-confidently-wrong-that-ive-lost-all-trust-in-them **Description** This risk arises when an agent retrieves and presents information from websites that are inaccurate, outdated, biased, or otherwise unreliable. As a result, users may be misinformed or make incorrect decisions based on content that has not been adequately validated or corroborated. **WOG context** Government agents providing information to citizens or supporting policy research must ensure accuracy and reliability. Retrieving information from unreliable sources could lead to incorrect policy advice, misinformed citizen guidance, or flawed analysis. #### RISK-036: Prompt injection risks through computer use - **Element:** Computer Use (Capability - Interaction) - **Failure mode:** External Manipulation - **Type:** Safety, Security - **Number of mapped controls:** 2 - **Sources:** https://arxiv.org/html/2505.13076v1, https://hiddenlayer.com/innovation-hub/indirect-prompt-injection-of-claude-computer-use/ **Description** This risk arises when an agent interacts with graphical user interfaces that display untrusted or adversarial content - such as web pages, documents, pop-ups, or form fields - crafted to embed hidden instructions or manipulative cues. As a result, the agent may misinterpret on-screen text as authoritative guidance, follow injected instructions, or perform unintended actions while operating the interface. **WOG context** Government agents with computer use capabilities operating desktop interfaces may encounter adversarial content displayed on screen. This could include malicious documents, manipulated web pages, or deceptive UI elements designed to hijack agent actions. This could lead to unauthorised data entry, incorrect form submissions, or compromise of government systems accessed through the interface." #### RISK-037: Exposure of sensitive data - **Element:** Computer Use (Capability - Interaction) - **Failure mode:** Agent Failure - **Type:** Safety, Security - **Number of mapped controls:** 1 - **Sources:** https://arxiv.org/html/2506.00618v3 **Description** This risk arises when an agent operating a computer interface accesses websites or applications that contain personally identifiable or sensitive information, particularly when authenticated as a user or organisation. As a result, the agent may inadvertently view, process, or disclose confidential data beyond its intended scope or authorisation. **WOG context** Agents with computer use capabilities authenticated to government systems via officer credentials may access screens containing sensitive citizen data, policy deliberations, or security-classified information beyond what is needed for the current task. This creates risk of data exposure through agent memory, logging, or inadvertent disclosure in subsequent interactions. #### RISK-038: Incorrect use of unfamiliar programmatic interfaces - **Element:** Other Programmatic Interfaces (Capability - Interaction) - **Failure mode:** Agent Failure - **Type:** Safety, Security - **Number of mapped controls:** 1 **Description** This risk arises when an agent interacts with programmatic interfaces it has not been trained or configured to use correctly, particularly bespoke or non-standard interfaces outside established protocols such as MCP servers. As a result, the agent may misinterpret interface semantics, invoke operations incorrectly, or produce unintended effects due to improper integration or usage. **WOG context** Singapore government operates numerous bespoke systems with agency-specific APIs beyond standardised API gateways. Agents interacting with legacy systems, custom agency APIs, or non-standard integrations may misinterpret interface requirements, invoke operations with incorrect parameters, or fail to handle agency-specific error conditions. This could result in failed transactions, data corruption, or system errors. --- ### Capability - Operational *8 risk(s) in this category.* #### RISK-039: Production or execution of poor or ineffective code - **Element:** Code Execution (Capability - Operational) - **Failure mode:** Agent Failure - **Type:** Security - **Number of mapped controls:** 5 - **Sources:** https://www.usenix.org/system/files/conference/usenixsecurity25/sec25cycle1-prepub-742-spracklen.pdf, https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/, https://proceedings.neurips.cc/paper_files/paper/2024/hash/bfd082c452dffb450d5a5202b0419205-Abstract-Datasets_and_Benchmarks_Track.html **Description** This risk arises when an agent generates or executes code that is incorrect, inefficient, insecure, or unsuitable for the intended task. As a result, the code may fail to achieve desired outcomes, introduce bugs or vulnerabilities, or cause operational disruptions when deployed or run. **WOG context** Government agents generating code for applications, system automation, or data processing must produce correct, efficient, and maintainable code. Poor quality code could result in application bugs affecting citizen services, system instabilities, or failed integrations. Code should meet government development standards and pass security scanning before deployment. #### RISK-040: Production or execution of vulnerable or malicious code - **Element:** Code Execution (Capability - Operational) - **Failure mode:** Agent Failure - **Type:** Security - **Number of mapped controls:** 7 - **Sources:** https://arxiv.org/pdf/2501.08200, https://arxiv.org/html/2504.21205v1 **Description** This risk arises when an agent executes code that contains security vulnerabilities or intentionally malicious logic, whether generated by the model or sourced externally. As a result, the system may be compromised through exploitation, unauthorised access, data leakage, or other harmful effects. **WOG context** Code executed in government environments must meet IM8 security standards and pass SHIP-HATS security scans including SAST analysis. Vulnerable code could expose citizen data, compromise GCC infrastructure, or create attack vectors into interconnected government systems. Malicious code, whether intentionally generated or sourced from compromised repositories, could exfiltrate data or establish persistent backdoors. #### RISK-041: Unintended overwriting or deletion of files or data - **Element:** File and Data Management (Capability - Operational) - **Failure mode:** Agent Failure - **Type:** Security - **Number of mapped controls:** 3 - **Sources:** https://syssec.dpss.inesc-id.pt/papers/pedro_icse25.pdf **Description** This risk arises when an agent modifies, overwrites, or deletes files, database tables, or datasets without explicit user instruction or authorisation. As a result, critical information may be lost or corrupted, leading to data integrity issues, operational disruption, or the need for costly recovery efforts. **WOG context** Government agents handling citizen records, policy documents, and operational databases must not inadvertently modify or delete critical data. Loss of citizen application records, corruption of financial databases, or deletion of audit trails could have serious consequences including inability to process citizen requests, financial reconciliation failures, or compliance violations. #### RISK-042: Database overload due to inefficient data operations - **Element:** File and Data Management (Capability - Operational) - **Failure mode:** Agent Failure - **Type:** Security - **Number of mapped controls:** 3 - **Sources:** https://www.tinybird.co/blog-posts/which-llm-writes-the-best-sql **Description** This risk arises when an agent issues poorly optimised, excessively frequent, or redundant queries against databases or data stores. As a result, system performance may degrade, resources may be exhausted, or critical services may become unavailable due to unnecessary load. **WOG context** Government databases supporting citizen services must remain responsive according to agreed-upon SLAs. Inefficient queries from agents could degrade performance for all users, causing service timeouts, transaction failures, or extended processing delays that affect citizens' ability to complete time-sensitive government transactions. #### RISK-043: Exposure of sensitive data through file or database access - **Element:** File and Data Management (Capability - Operational) - **Failure mode:** Agent Failure - **Type:** Security - **Number of mapped controls:** 2 - **Sources:** https://www.infosecurity-magazine.com/news/microsoft-365-copilot-zeroclick-ai/ **Description** This risk arises when an agent accesses, processes, or outputs personally identifiable or sensitive information stored in files or databases without appropriate safeguards. As a result, confidential data may be disclosed to unauthorised parties, leading to privacy breaches, regulatory non-compliance, or loss of trust. **WOG context** Government agents with database access may retrieve citizen PII including NRIC numbers, income details, medical records, and family information. Without proper safeguards, this data could be exposed through agent outputs, logs, or subsequent interactions. #### RISK-044: Prompt injection via malicious files or data - **Element:** File and Data Management (Capability - Operational) - **Failure mode:** External Manipulation - **Type:** Safety, Security - **Number of mapped controls:** 2 - **Sources:** https://www.hackthebox.com/blog/cve-2025-32711-echoleak-copilot-vulnerability, https://www.wired.com/story/here-come-the-ai-worms/ **Description** This risk arises when an agent ingests or processes maliciously crafted files or data that embed hidden instructions or manipulative content. As a result, the agent may follow unintended prompts, alter its behaviour, or execute actions that compromise system safety or integrity. **WOG context** Government agents processing citizen-submitted documents, business applications, or external data feeds may encounter files crafted to inject malicious instructions. This could affect agents processing documents through LifeSG or business filings through GoBusiness. Successful injection could manipulate processing outcomes or exfiltrate sensitive information. #### RISK-045: Misconfiguration of system resources - **Element:** System Management (Capability - Operational) - **Failure mode:** Agent Failure - **Type:** Security - **Number of mapped controls:** 3 - **Sources:** https://neurips.cc/virtual/2024/poster/97835, https://arxiv.org/html/2507.10584v1 **Description** This risk arises when an agent incorrectly configures system settings, infrastructure resources, or operational parameters. As a result, system performance, reliability, or security may be degraded, leading to service disruptions or unintended operational behaviour. **WOG context** Agents managing GCC infrastructure, agency application settings, or system configurations could introduce misconfigurations that affect service availability, security posture, or compliance status. #### RISK-046: System overload due to inefficient or excessive operations - **Element:** System Management (Capability - Operational) - **Failure mode:** Agent Failure - **Type:** Security - **Number of mapped controls:** 2 - **Sources:** https://arxiv.org/html/2407.20859v1, https://genai.owasp.org/llmrisk/llm102025-unbounded-consumption/ **Description** This risk arises when an agent issues poorly optimised, excessively frequent, or redundant system-level operations or queries. As a result, computing resources may be exhausted, system performance may degrade, or services may become unavailable due to unnecessary load. **WOG context** Government infrastructure must support consistent service delivery to Singapore's citizens. Agents performing excessive API calls, redundant system queries, or resource-intensive operations could degrade application performance, exhaust compute budgets, or cause cascading failures across interconnected government services. ---