How do you trust an AI assistant before it goes live?

Kai is an AI assistant platform for businesses. Companies use it to set up a customer-facing chatbot that answers support questions, qualifies sales leads, and routes customers to the right team, without needing engineers to build it.

Think of it like hiring a customer service agent who never sleeps, handles thousands of conversations at once, and can be trained by the business team themselves, not the tech team.

Old-style chatbots followed rigid scripts and broke the moment a customer asked something unexpected. Kai understands natural language, learns from the company's own content, and knows when to hand off to a human.

Companies were interested. But something kept stopping them from committing.

Whenever they made a change to the assistant, whether updating its instructions, adding new content, or adjusting how it handled certain questions, they had no way to see what effect that change would have before it went live.

The only option was to deploy it and see what happened. For someone responsible for customer satisfaction, that wasn't good enough.

We ran 8 interviews with customer support managers and operations leads at enterprise companies that were actively evaluating Kai. These were people responsible for customer satisfaction scores, not engineering or product. They were the ones who would own the outcome if the AI got it wrong.

I expected the sessions to be about performance: accuracy rates, speed, resolution numbers. Instead, every single meeting turned to the same question.

"What happens when Kai gets it wrong, and who is accountable?"

They could see the AI worked in the demo. The question was whether they could be sure before putting it in front of their own customers. 3 out of 4 companies in advanced talks had paused at exactly this point.

Five different types of people needed to use the same testing space: a trainer uploading new content, an ops manager reviewing responses, an IT admin controlling who can make changes, a viewer monitoring results, and a super admin overseeing everything.

Each of them had a different idea of what 'safe to test' and 'ready to go live' meant. The design had to work for all of them.

People talked about what they needed the assistant to do, not how it worked underneath. They didn't care about the technology. They cared about the outcome.

They would happily spend more time on setup if they felt confident in the result. This wasn't about speed. It was about certainty.

Every time they updated something, they had no way to check what changed without going live. That single gap was the reason deals were stalling.

Every other AI platform treated testing as a separate activity. Something you did in a different environment before a demo, not something built into daily use.

Nobody had made testing a natural part of the setup experience itself. That gap was the opportunity.

Before any high-fidelity work, I mapped the five roles on a whiteboard against what each person needed to do, what they needed to see, and what they should never be able to touch. That exercise exposed the permission model problem early, before it could become a late-stage design fix.

Early concepts were rough: one put the test window in a drawer that slid out from the right, which kept it accessible but made it feel like an afterthought. Another treated it as a full-page mode you had to navigate to, which added the exact friction we were trying to remove. The version that worked kept the test window anchored beside the config screen at all times, visible without being forced on you.

I tested three structural approaches with two ops leads before committing to the final layout. The insight that shifted everything: people didn't want to open a test window. They wanted testing to already be there when they looked up.

When someone updates the assistant's instructions, changing how it introduces itself or how it handles a refund request, the test window shows the effect immediately. No saving. No refreshing. Just change and see.

When someone uploads a new document to train the assistant on, they can ask it questions about that document right away. A bad upload shows up in the test, not when a customer asks a question the assistant can't answer.

When someone adjusts the assistant's tone or the situations where it hands off to a human, the test window is right there to try it. Every screen, every change, same place to check.

The question 'what happens when it gets it wrong?' didn't go away. But the answer changed. Instead of 'we'll find out,' it became 'let me show you.'

Companies that had paused the deal could now see for themselves that changes were safe before going live. That shifted the conversation from risk to readiness.

Automatic checks after every update: right now, testing is something a person has to remember to do. The next step is running a standard set of test scenarios automatically every time something changes, so problems are flagged before anyone has to go looking for them.

A way to test together: currently one person tests and reports back. But the person who knows the edge cases and the person who signs off on go-live are often different people. A shared test session would let both of them look at the same thing at the same time.

Shared examples across companies: teams build their own test scenarios from scratch. But the same types of problems come up again and again across different businesses. A library of common test cases, built from anonymised examples, would give new teams a head start.

I went into this project thinking the problem was about making the AI smarter. Research showed the problem was simpler: people had no way to check whether it was ready.

The solution wasn't a new AI feature. It was giving people a safe place to find out for themselves, before it mattered.

One thing I'd do differently: I added user access levels late in the process, once I realised different people needed different permissions. That should have been part of the design from day one, not something bolted on once the gap became obvious.