Our first strategic response was PoliBot, a RAG-based AI assistant that streamlined the analysis phase and cut development cycles by over 60%. We created a helper file from the underlying source code and trained the LLM with real examples stored in Vector DB, which formed the foundation for the knowledge base/feature creation engine for the engineers and policy designers alike.

MVP outcomes: Early signs of success

Navigating these challenges required a significant effort, but the payoff from our initial MVP release has been immediate and clear. With our new agentic PoliBot handling the end-to-end process, we are already seeing policy rule tickets resolved ~40% faster than before. This powerful early metric has validated the entire journey, proving the immense operational value of evolving from a simple assistant to a truly autonomous teammate.

Lessons from our agentic AI journey

Our journey transforming PoliBot taught us invaluable lessons. For any leader looking to harness the power of agentic AI, here is the playbook we wished we had from day one.

1. Evolve a proven winner 

Our biggest advantage was starting with PoliBot, a tool the business already valued and trusted. Instead of building a revolutionary agent from scratch, we upgraded a proven success. This lowered the risk, leveraged existing buy-in, and allowed us to focus on the complex task of automation rather than selling a new concept, and was crucial for us to move from a standalone agent to a proper agentic AI solution.

2. Build the foundation before the agent 

We learned that an agent's reasoning is useless without the right infrastructure to act on it. Assessing the foundational "plumbing" — like APIs and communication protocols such as MCP — is a non-negotiable prerequisite before committing to an agentic solution.

3. Architect for trust from day one 

Trust cannot be an afterthought; it must be the foundation. Making every action transparent and auditable was the only way to earn the deep-seated buy-in from our risk, compliance and business partners who would ultimately rely on the agent.

 4. Test the thinking, not just the code 

We quickly discovered that testing an agent means testing its judgment, not just its output. Our quality assurance had to evolve to design tests that challenge the agent’s reasoning with ambiguous requests and unexpected tool errors, ensuring it can handle real-world uncertainty, not just ideal scenarios.

5. Make It a cross-functional mission 

An agentic AI project is a business transformation project, not just a siloed IT project. Our success was only possible through a deep partnership where the business defined the value, security built the guardrails, and technology delivered the "how."