Implementation

Many tools, frameworks, protocols, and platforms are available to support the development and implementation of AI agents. We introduce some of them in this section.

Libraries and Frameworks

LangGraph

LangGraph is a framework to orchestrate agentic workflows. It allows users to define agentic workflows as a group of nodes in a graph structure; this is conceptually similar to airflow and other workflow orchestration tools. It is flexible enough to allow human-in-the-loop design and LLM decision-making, making enabling complex agentic systems.

The main components for LangGraph include:

Nodes: Each node represents a task, which can be one call or a series of calls to an LLM
Conditional Nodes: These nodes control the workflow direction, either through an if-else call or a decision by a LLM.
Tools: Tools can be attached to nodes. Tools are usually built by developers for specific use cases. Examples of tools include web search tools and RAG tools.

Additionally, LangGraph provides off-the-shelf support for useful utilities such as map-reduce patterns in agentic calls (e.g. making multiple tool calls in parallel based on a previous node’s results), asynchronous execution, error handling and tracing using LangSmith.

The following is an example of a workflow designed using Langgraph. It is a corrective RAG workflow, which includes a re-write query and web search node to supplement the main RAG workflow.

LangGraph Corrective RAG

Image source: https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_crag/

Smolagents

Smolagents is a barebones Python library focused on building AI agents. It emphasises simplicity, with its core logic fitting within approximately 1,000 lines of code. A standout feature of Smolagents is its first-class support for Code Agents, where the LLM writes actions as Python code, offering potential performance advantages over the more traditional method of using JSON to define tool calls. Smolagents is also model-agnostic and tool-agnostic, supporting various LLMs (local transformers, Hugging Face Hub models, OpenAI, Anthropic) and integrating with tools from LangChain, Anthropic's MCP, and even Hugging Face Spaces. Furthermore, it offers Hub integrations for sharing and accessing tools.

Several key components underpin the Smolagents framework:

ReACT framework: Smolagents utilises multiple LLM calls for planning, reasoning about tool usage, observing the outcomes, and refining the results. This iterative process allows for more complex problem-solving.
ToolCallingAgent: In multi-step agents, the LLM can generate actions in the form of calls to external tools. The conventional method for specifying these actions involves the LLM generating a JSON structure containing the tool name and the arguments to be used .
CodeAgents: Instead of defining tool calls in JSON, CodeAgents write their actions directly as Python code snippets. This approach has been shown by research to offer several advantages, including using 30% fewer steps and achieving higher performance on challenging benchmarks. The reasoning behind this is that code languages are specifically designed to express computer actions effectively. Writing actions as code allows for better composability, object management, and generality, while also leveraging the inherent strengths LLMs possess in coding. To address the security concerns associated with executing arbitrary code, Smolagents supports execution in sandboxed environments via E2B or Docker, and also offers a secure Python interpreter as an option.
MultiAgent: Smolagents supports the creation of multi-agent systems where a manager or supervisor agent orchestrates the workflow of different specialised agents, each potentially adopting a specific persona (e.g. drafter, reviewer).
Integration with Hugging Face: Smolagents seamlessly integrates with the Hugging Face ecosystem, allowing users to utilise and contribute to the public repository of models and agents available on the Hugging Face Hub. Tools created within Smolagents can be easily shared to the Hub.

OpenAI Agents SDK

OpenAI Agents SDK is OpenAI's open-source framework designed to build agentic AI apps. This SDK offers a lightweight, easy-to-use package with very few abstractions . It represents a production-ready upgrade of OpenAI's previous experimentation for agents, Swarm. The SDK is built around a very small set of primitives, which include Agents, Handoffs, and Guardrails. Agents are LLMs equipped with instructions and tools, Handoffs enable agents to delegate to other agents for specific tasks, and Guardrails allow for the validation of inputs to agents. These primitives are powerful enough to express complex relationships between tools and agents, facilitating the building of real-world applications without a steep learning curve. The SDK also includes built-in tracing that lets you visualise and debug your agentic flows. The SDK features a built-in agent loop that handles calling tools, sending results to the LLM, and looping until the LLM is done. The SDK allows you to turn any Python function into a tool, with automatic schema generation and Pydantic-powered validation. Some of the built-in tools include:

Web search: Searches the internet
File search: Upload your own files which will get embedded for Vector Search
Computer use: Captures mouse and keyboard actions generated by the model, making it possible for developers to automate computer use tasks by directly translating these actions into executable commands within their environments.

Other Agentic Frameworks

As there are numerous agentic frameworks in the market, we do not review all of them. Apart from the three popular options listed above, the following are also gaining popularity at the time of writing:

CrewAI: An open-source framework for building agentic systems, with a no-code enterprise option, which allows users to create and deploy AI agents without writing code.
MCP-agent: Leverages MCP (Model Context Protocol) for providing context to LLMs, allowing them to maintain a coherent understanding of long conversations and complex tasks.
Pydantic AI: Integrates Pydantic, a popular Python data validation library, with AI models, ensuring that the inputs and outputs of AI models are valid and consistent.

Protocols and Standards

Model Context Protocol (MCP)

The Model Context Protocol is an open-source standard designed to facilitate connections between AI agents and various data sources, such as databases, APIs, and local files. It consists of three key components: the host, the client, and the server. The host is the application that integrates the agent, such as a chat application, a code assistant in an IDE, or another platform. Within the host, the client serves as the interface linking it to the server. The server houses the tools and handles task execution by connecting to databases, APIs, or local files as needed by the agent.

MCP General Architecture

Image source: https://modelcontextprotocol.io/introduction

The MCP protocol acts as a transport layer, enabling communication between the MCP host/client and the MCP server(s). It defines the standard for exchanging requests and responses, ensuring seamless interaction between the agent and the data sources.

For example, in a chat application, you ask, “What’s the weather in London?” The MCP host (the chat app) checks its connected MCP server(s) for available weather-related tools. It then forwards your query, along with the list of available tools, to an LLM. The LLM identifies that a weather API is needed. The MCP host then requests the relevant MCP server to call the weather API, retrieves the weather data for London, and sends it back to the LLM, which formulates the final response and delivers it to you in the chat.

Having a standard protocol like MCP to define the interactions between the model and tools greatly facilitates reusability. Tools created by one developer in an MCP server can be easily imported (for local runtime) or accessed (on remote servers) by clients (LLMs) created by other developers.

Responses API

The Responses API is OpenAI’s API primitive for leveraging OpenAI's built-in tools to build agents. It combines the simplicity of Chat Completions with the tool-use capabilities of the Assistants API. As model capabilities continue to evolve, the Responses API will provide a more flexible foundation for developers building agentic applications. With a single Responses API call, developers will be able to solve increasingly complex tasks using multiple tools and model turns.

To start, the Responses API will support new built-in tools like web search, file search, and computer use. These tools are designed to work together to connect models to the real world, making them more useful in completing tasks. It also brings with it several usability improvements including a unified item-based design, simpler polymorphism, intuitive streaming events, and SDK helpers like response.output_text to easily access the model’s text output.

The Chat Completions API is an industry standard for building AI applications, and OpenAI intends to continue supporting this API indefinitely. The Responses API was introduced to simplify workflows involving tool use, code execution, and state management and is a superset of the Chat Completions API, and it is likely to also be widely adopted for standardised communication between agents and tools.

Cloud-hosted Platforms for Building AI Agents

Amazon Bedrock Agents (AWS)

Amazon Bedrock Agents offers a service to abstract agentic development. Bedrock Agents employ the ReACT framework, which models agentic workflows as an interactive process of planning, acting and reasoning. These agents can be integrated with various AWS resources, orchestrated by Bedrock models, to perform tasks and execute code.

The following are some key features offered by Bedrock Agents:

Action Groups: LLMs can plan and execute lambda functions which perform specific actions according to user definitions, or execute search and retrieval using Bedrock Knowledge Bases.
Multi-agent routing: Supervisor agents can route tasks to various agents.
Tracing: AWS bedrock enables easy access to agent memory and logs for evaluation.
Ecosystem Integrations: Bedrock Agents are integrated with Bedrock models, knowledge bases, guardrails and AWS identity and governance frameworks.

Amazon Bedrock Agents

Image source: https://aws.amazon.com/solutions/guidance/automating-tasks-using-agents-for-amazon-bedrock/

Amazon Bedrock AI agents process user queries by converting them into embeddings, retrieving relevant information from knowledge bases, and executing actions through predefined API-driven workflows. Using ReAct prompting, they iteratively refine responses, validate inputs, and enhance decision-making with contextual understanding from conversation history and foundation models. This enables dynamic automation of complex tasks, such as claims processing, by leveraging retrieval-augmented generation (RAG) and intelligent orchestration.

Vertex AI Agent Builder (Google Cloud Platform)

Vertex AI Agent Builder allows users to utilize LLMs to orchestrate code pipelines. The pipeline can be both deterministic and generative in nature. Within the builder, we can create several agents, each which can be grounded by specific knowledge bases via search and retrieval. Vertex AI is integrated with Vertex AI model garden for selection of various models, including Gemini.

The application can also be integrated easily with various chat platforms, such as Telegram and Dialogflow fx.

Google Agent Space (Google Cloud Platform)

Google AgentSpace is an AI platform that integrates generative AI, enterprise search, and data connectivity, to enhance workflows and decision-making. It connects structured and unstructured data sources, providing users with AI-driven agents for automation and insights.

NotebookLLM is the main user interface for Google Agent Space. It is an AI-powered chatbot which allows users to upload files for analysis, generate summaries, and extract insights. NotebookLLM is also enhanced with agentic AI features: Users can build agents with specific roles to connect with documents, data, and knowledge bases from various platforms, using no-code or low-code tools.

Key Features

Multimodal AI
Powered by Gemini, Imagen, and Veo models, Agentspace processes text, images, and videos for comprehensive understanding.

Enterprise Search & Data Access
Google-quality search retrieves relevant information across hosted and third-party data sources, including documents, emails, and databases.

Pre-Built Connectors
Agentspace integrates with enterprise applications, such as Google Drive, Confluence, Jira, Microsoft SharePoint, Microsoft Outlook, Salesforce, and Slack.

AI Agents for Business Automation
Organizations can deploy AI agents across various business functions. In marketing, these agents assist with content generation, campaign reporting, and competitor analysis. For sales, they enhance lead qualification, provide customer insights, and automate follow-ups. In software and IT, they streamline code documentation, system monitoring, and troubleshooting. HR teams use AI for employee onboarding, policy documentation, and candidate evaluation. In research and development, AI agents support trend analysis, patent research, and industry benchmarking.

Security & Compliance
Agentspace is built on Google Cloud’s security framework, ensuring:

Data privacy for prompts, outputs, and training
Compliance with security frameworks (DRZ, AxT, CMEK, VPC-SC)
Role-based access controls and encryption
Generative AI indemnification and content moderation tools

Azure AI Agent Service: Building and Scaling AI Agents (Microsoft Azure)

The Azure AI Agent Service is a platform that simplifies developing, deploying, and scaling AI agents securely. It supports both single and multi-agent systems, offering enterprise-grade readiness for production environments.

Key features include:

Flexible Model Support: Developers can select models from Azure AI, including Azure OpenAI Service (GPT-4o, GPT-4o mini), Model Catalog entries like Llama 3 and Mistral Large, and Cohere-Command-R-Plus, ensuring adaptability for various agentic tasks.
Diverse Data Integration: The platform connects to multiple data sources for grounding agents, including:
Azure AI Search for document retrieval.
Microsoft Fabric and SharePoint (upcoming).
Bing Search for real-time web data.
Licensed data and local files for proprietary insights.
Robust Security: Features like BYO file storage, search index, thread storage, virtual networks, OBO authorization, and enhanced observability ensure compliance and security.
Accelerated Development: Integration with the Azure AI Foundry SDK aids in efficient agent creation.
Tool Integration: Supports tools like:
OpenAPI 3.0 Specified Tools and Azure Functions for custom actions.
Code Interpreter for on-the-fly code execution.
Orchestration Support: The platform integrates with frameworks like AutoGen and Semantic Kernel to manage complex multi-agent workflows.

Semantic Kernel: AI Orchestration Middleware

Semantic Kernel is an open-source middleware designed for adding AI capabilities to enterprise applications. It integrates LLMs and other AI tools through:

Plugins: These connect agents to APIs, databases, and business logic.
Planners: Leveraging LLM reasoning, planners coordinate task sequences for achieving goals.
Personas: Combining plugins and planners, personas enable contextual and personalised agent interactions.

Semantic Kernel supports .NET, Java, and Python, ensuring accessibility for diverse development teams. Its open-source nature and enterprise focus align with production-grade AI requirements.

Supporting Azure AI Services

Complementing these tools are key Azure AI services:

Azure OpenAI Service: Offers advanced LLMs like GPT-4o with multimodal capabilities for robust agent reasoning.
Azure AI Search: Facilitates Retrieval Augmented Generation (RAG) for grounding agent responses in relevant data.
Azure AI Content Understanding: Processes multimodal content (text, images, audio, etc.) with pre-built templates for streamlined workflows like document analysis and call center insights.

Together, Azure AI Agent Service, Semantic Kernel, and supporting Azure solutions provide developers with powerful tools to build intelligent, scalable agentic AI systems. One such example is a pipeline which can be built together with various Azure services, such as the Azure AI services, LLMs and AI Search.

TL;DR

Even though Agentic AI is a relatively new field, there is no lack of libraries and frameworks for developing agentic solutions. Cloud service providers are also providing platforms for building agents which can integrate seamlessly with other services in their ecosystems.