Enhancing Support with AI Chatbot Assistant

The customer

The customer is a British-based security software and hardware company that develops products and services to protect users, networks and endpoints from cyberattacks.

They were seeking a partner to develop an accurate chatbot solution to showcase the capabilities of GenAI. The customer aimed to create a chatbot solution that could efficiently retrieve documents and provide technical support to staff members, thereby demonstrating the potential of GenAI.

Enhancing Support with AI Chatbot Assistant

The objective

The primary objective was to ensure the system’s accuracy, with all other performance metrics deemed secondary. However, this approach was subsequently reassessed when GPT3.5 was found to significantly outperform the original solution in terms of speed for the customers’ specific application use cases.​

This engagement saw the first internal usage of AI and associated testing, including demonstrating to customers’ AI Board.​

Critical success factors:

Accuracy

The critical success factor for this project was to ensure and measure the accuracy of the response given to users.

Production Capable

Design the Proof of Concept with a path to potentially productionise the solution if the success factors could be met.

Documentation

Provide the customer with comprehensive documentation to support the infrastructure and application, taking account of their limited knowledge of Azure and AI.

The results

This project demonstrated the successful integration of AI to enhance support services, with a focus on accuracy and potential for future production deployment.

Highlights:

Accuracy

GPT3.5 and GPT4 were tested back-to-back for accuracy and performance with the results provided to the customer. RAG testing tooling provided by Microsoft were usedto measure useful of the answers provided.

Infrastructure

Azure infrastructure was deployed with a well-architecture mindset allowing for a quick deployment to production.

Guardrails

Used Microsoft content filtering tooling to stop employees from "jailbreaking" the AI into giving inaccurate responses.

The Expert Thinking team were excellent from the start of our engagement, their communication was spot on. Jake was very knowledgeable and helpful. The quality of the product delivered was very high and his documentation was particularly excellent.

Enterprise Workflow and Automation Manager

Discover our AI Assessments.

We provide a series of packaged services designed to help you get the most out of AI, each taking a different aspect of AI, sifting out all of the hype to focus on business benefit, technology and implementation.