Support chatbot that answers instantly and hands billing to a person

Business context

A scheduling software company of fifteen staff sells appointment booking to clinics and salons. Two people handle every support message, and most of them repeat questions already answered in the help center. The company asked us to build an AI customer support chatbot for the repeat questions, while keeping anything sensitive with a person. This is the support chatbot work we do for small teams.

Customers schedule their own time zones, so messages arrive overnight and on weekends. The queue was never empty, even when nothing was broken.

The problem

Most messages asked about something already written in the help center. Staff answered them by hand, one at a time. During working hours the first reply took about 40 minutes. A message that arrived after hours waited until the next morning, which is a long time when a customer is stuck in the middle of booking.

The help center was also out of date in places, but nobody could see which articles caused repeat questions because the questions sat in an inbox instead of a report.

What we built

The assistant runs inside Intercom. It answers from the help center only, and every answer cites the article it came from, so the customer can read the source in full.

We gave it three jobs and no others: explain how a feature works, point to the right help article, and ask one clarifying question when the request is unclear. It cannot discuss billing, change an account, cancel a plan, or promise a fix. Any of those topics triggers an immediate handover to the Intercom inbox with the whole conversation attached, so the customer never repeats themselves.

We wrote the refusal rules as plain sentences and tested them against two years of past support threads. A question that did not clearly match an article was treated as unknown and passed to a person.

The monthly report lists every question the assistant could not answer. That list feeds the help center, not the engineering backlog.

We set one more rule that mattered: the assistant may only use text that appears in an approved article. If an article covers half the question, the assistant answers that half and hands the rest to a person. That restriction is the main reason it stayed accurate.

Rollout

• Week 1: Read 18 months of support threads and grouped them by the question behind the words.
• Week 2 to 3: Built the help center index and the answer flow, then ran it in a sandbox where staff reviewed every draft reply.
• Week 4: Answered real conversations with a person checking each one before it sent.
• Week 5: Turned on automatic replies for questions that matched an article, with handover for everything else.

Before and after

Step	Before	After
First reply	Next morning for overnight messages	Seconds, at any hour
Billing questions	Answered by hand	Handed to a person with the chat attached
Help center	Updated when someone noticed	Fed by a monthly list of unanswered questions
Repeat questions	Staff time	Answered from the cited article

Results

By the end of the pilot, 68% of conversations were resolved without a handover. Median first response dropped from about nine hours to 24 seconds. The two support staff moved onto onboarding and account work that had been pushed aside for months.

The assistant found 31 gaps in the help center in the first quarter. Most were features that had changed and were never rewritten.

The remaining questions still ended with a person, which is where they belong. When someone asks to change a plan or disputes a charge, they should be talking to a colleague, not a bot.

Each month we reopen every wrong or incomplete answer with the transcript in front of us, then rewrite the article behind it. The accuracy review is short, and it is the reason the number kept improving after launch instead of drifting down.

Lessons learned

• The most useful output was the list of questions the assistant could not answer. We expected the chatbot itself to be the value, and the help center fixes did more for customer satisfaction. The WhatsApp order-status bot we built for an online store landed in the same place by answering fewer topics, better.
• We first let the assistant paraphrase articles in its own words. It sounded confident and was occasionally wrong. Quoting the passage directly fixed that.
• We assumed customers would dislike talking to a bot. Handover mattered more than the bot: as long as the escape hatch was obvious, people used it without complaint.