Agentic AI System Development: The BroadDigix Guide to Building AI That Operates, Not Just Responds
Whilst much of the conversation around building Agentic AI System Development are focused on the benefits they provide businesses, it is often overlooked how the technology chosen affects the overall performance of the system. Understanding the technology considerations behind building an agentic system will help any business leader looking to invest in an agentic AI solution understand if the proposed solution is built on the right foundations.
At BroadDigix we build our Agentic AI System Development using technology that can stand up to the demands of production.
Reasoning Layer
The choice of foundation model for an agents reasoning layer is an important consideration. Different models will perform better for certain tasks and fail differently.
When choosing a foundation model for a particular agentic implementation, BroadDigix considers:
Reasoning performance for the type of task. Output structure and quality of output. Reliability of following instructions. Ability to use tools. Size of context window. Latency for expected transaction volume. Cost per transaction.
There will never be a single model that will be the best option for all aspects listed above. For example an agent used for customer service (high volume and simple tasks) will have very different requirements to an agent used for internal research (complex and multi source). We always select the best model for the specific use case.
We test the models using realistic examples of the inputs the agent will receive in production. This means we ensure the model is working as expected for the real-world inputs not just the tests provided by the model provider.
Tool Architecture
Having an agent with good reasoning skills is only helpful if it has the tools to do something with it. The tools used by the agent will dictate what it can do.
At BroadDigix we design our agents with reliability in mind. We always design the agent to have fewer tools that can be relied on rather than having more tools that may not be reliable. Every tool the agent uses could be a point of failure so we will always design the agent to have the minimal amount of tools needed for the use case. We design each tool to be reliable in production.
All of our tools follow standard patterns seen in enterprise integration. We manage the authentication tokens for the APIs to ensure the agent doesn’t need restarting if the token expires. We manage rate limiting and retries on the APIs so that the agent isn’t failing due to issues with the API. We also have error handling to ensure the agent knows what to do for different errors.
Our Microsoft Dynamics integration is a key capability that we have developed from extensive work building agentic systems in enterprise. Our knowledge of how the Dynamics APIs work means we can design the agents to integrate with Dynamics in the best way possible.
Memory Architecture
The memory architecture of an Agentic AI System Development will determine how well it can perform complex, multi-step tasks. If the agent does not have the right memory architecture then it will lose context between steps and will not be able to perform the complex reasoning required in real business situations.
We design our memory architecture to fit the use case. At BroadDigix we use a tiered approach.
Working memory – This is the memory used to store the context of the current task. We need to ensure there is enough working memory for the most complex task the agent will carry out.
Episodic memory – This stores information from previous interactions that are relevant to the current interaction. Having an agent that can remember relevant information from previous interactions with the customer will result in a better customer experience. We will need to ensure the architecture works well to retrieve the correct information whilst ensuring we are not breaching privacy or increasing storage costs.
Semantic memory – This stores general knowledge about the domain the agent is working in (product information, policies, process guides etc). Ensuring we can retrieve this information correctly means the agents can utilise domain knowledge accurately.
One of the more technical aspects of any agentic AI system is the memory architecture between the different layers. We have a lot of experience in designing this meaning our agents will perform significantly better than less experienced teams.
Security Architecture
When building an agentic AI system for enterprise there are some additional security considerations. As the agents will be interacting with sensitive business data and performing actions that have impact, the security architecture of the agent must be designed with the same rigour as any other system that will be using sensitive data and performing impactful actions.
At BroadDigix we always consider the following 4 factors.
Authentication & Authorisation – All systems the agent interacts with will authenticate the agent as a specific service with defined permissions. The agent will only have access to the data and actions it has been authorised for. We follow least privilege.
Data – Any sensitive data the agent will interact with will be handled according to the data classification rules. Personal customer data, financial data and health information will be handled with the appropriate controls. Data will not be stored beyond the current task unless there is an authorised reason to do so.
Audit Logging – We ensure all actions carried out by the agent are logged. This helps with investigations, compliance and performance analysis.
Prompt Injection & Adversarial Inputs – When using external inputs with the agent there is a risk of adversarial inputs being used to try to influence the behaviour of the agent. We ensure all customer facing and external agents have protections in place against common prompt injection patterns.
Scalability Architecture
The scalability architecture of the agentic AI system will determine how well it will perform at higher volume. If the agent performs well at a lower volume but poorly at higher volumes then there will be a performance cap.
We design our systems to work within the expected volume range. We consider how well the infrastructure, integration and model invocations will perform at the volume we expect.
As part of our testing methodology we carry out load testing to ensure our agents perform well at high volumes. This will show any issues with performance at high volumes. This could include increased latency, hitting rate limits on integrations and memory usage. These will be resolved before going live.
Contact us at BroadDigix to speak to our engineering team about the technical architecture of your agentic AI system development requirements.
