Now it’s time to choose your shortlist of AI support agent providers to start testing with.
I’d always recommend testing with a handful of vendors as they will all have a slightly different setup in terms of how they answer questions, some will work to your benefit, and others will work against your setup.
Just because one provider is optimal for one company, does not mean it is optimal for yours (ever hear the phrase “jack of all trades, master of none”?
This is most definitely the case in the world of AI agents, you are often much better suited looking for a specialist AI support agent.
If you are already using a customer support platform like Zendesk or Intercom, I would start by checking their respective marketplaces searching for something like: “AI support” or “AI chatbot”.
Googling phrases like “AI support agent”, “AI customer support”, “AI customer service” should bring a good list to begin with.
Most products will also create “Alternative to” pages which show comparisons of who they think their most similar competitors are.
While these lists are likely biased they can serve as a good research tool to begin your search.
Your goal here is to find say 5-15 vendors that you can then bring down to a shortlist of 2-5.
Now, the next question is, how do you reduce this list of 5-15 vendors, down to your shortlist?
Here are a few questions to ask yourself (and your vendors) that will help:
Consider:
This is an easy one - does it connect with the support tool you want to use going forward?
i.e. Intercom, Zendesk, HubSpot, Freshchat, HelpScout etc.
But also consider whether you might also want to provide AI agent responses in other tools:
There are a few different AI agent patterns that have emerged over the last few years, so make sure you understand which one(s) your AI agent can do for you (and don’t forget that you may only want a ‘co-pilot’ today, but this may change in the next few years…)
This is one of the biggest differences between different AI agent providers so make sure you find a provider whose pattern mimics the way you and your team work.
This should be table stakes for any AI agent.
No AI agent on the market today is in a place to 100% autonomously provide customer support with no input from a person.
This means that any AI agent tool you look at must provide a seamless way for tickets or conversations the AI can’t answer to be ‘handed over’ to a person to ensure that your customers will always get an answer.
It is also vital that the user experience is as easy as possible for a customer to speak to a human agent when they need to.
It is therefore worth asking your vendors:
When the AI is unable to assist, what are the ways and how does the customer speak to a person?
The day you first try out your AI agent will be the worst-performing it should ever be.
There are 2 major components to AI support agent performance:
To improve #2 you will need an AI agent tool which can ideally do some or all of the following:
If you don’t have at least some of these options available to you, it will make it very difficult for you to improve that all-important AI deflection or resolution rate over time, however good your AI agent provider does with #1.
The only way to accurately test this is to test.
I’ll go through this in more detail in Step 4 but in short, you will need a list of real questions, tickets or conversations that have been recently asked to your support.
I’d recommend somewhere between 50 and 250 to test with, depending on the variety of questions and complexity of your documentation (the more and the more complex, the closer to 250 you’ll want).
Any good AI agent provider should allow you to test this set of questions free of charge to see what responses you could expect to get from your new AI agent.
See Step 4 for more on testing answer quality.
AI models change all the time, you should however check that which AI agent provider you speak to is at least using the highest quality model from their respective model provider.
For instance, for OpenAI, the best-performing model right now is their GPT-4o model, don’t settle for anything less!
They may also tell you it is using a ‘fine-tuned’ model, which can vary considerably in interpretation.
It should also be noted that the best models can give the worst answers when set up incorrectly and the worst models can give market-leading answers and responses if they are set up by a team who knows what they are doing.
In short, I would recommend you largely ignore such statements and judge your AI agent solely on its answer quality.
This is where things can get complicated…
Each AI agent provider will likely offer very different, unique feature sets, which can make it quite hard to compare.
However almost all will have pricing that scales according to some kind of usage metric, here are the common ones:
The other ‘usage-based’ elements of pricing that are relatively commonplace are:
Aside from the usage-based comparison, you will have to look for features that sound useful to you as part of your comparison.
Common AI agent features you might see included in pricing lists include:
But just ensure you are always checking back to your goals and objectives here and aren’t blinded by shiny object syndrome!
Depending on your plan and the size of the company you are working with the level of support you get can vary massively.
Check the channels they use for support: email, live chat, their own AI agent, calls, video calls, and customer success teams.
And their availability: 24/7, 24/5, 9-5/7, 9-5/6, 9-5/5 (also check time zones).
You want to find the AI agent provider that best fits your needs.
Also don’t underestimate the value in having the provider help you get set up in the first place and check with them to ensure this is part of the service they provide to ensure you aren’t wasting time optimizing your set-up.
Security should always be a consideration.
While it is highly unlikely that any AI agent provider will use your conversations and tickets for training a model of theirs (or anyone else’s) it is always worth confirming this.
Depending on the size of your business and the customers you work with you may also need to check compliance with regulatory or security standards such as ISO 27001, GDPR or SOC II.
A lot of the time this should be a pretty simple and quick question for them to answer.