Teams building their first AI feature usually worry about capability. In our experience capability is rarely what sinks the project. Trust is.
One wrong answer costs more than ten missing ones
When we built a WhatsApp support assistant, the risk everyone raised first was that the model would not be clever enough to handle real questions. That is not what kills these projects.
What kills them is a single confident, invented answer given to a real customer who then acts on it. One of those undoes months of goodwill, and it undoes it publicly. A bot that cleanly resolves 60% of queries is worth far more than one that attempts them all and gets 15% wrong.
Once you accept that, the architecture more or less writes itself.
Answer only from your documents
Product docs, FAQs and resolved tickets get embedded and indexed. Retrieval happens before generation. If the answer is not in the source material, there is nothing to answer with.
That constraint is the feature. It also produces something we did not anticipate: every answer can be traced back to the exact documents it cited. That turned out to be the thing the support team used most — not to check the bot, but to find out which documentation was doing work and which was never retrieved at all.
Let it say it does not know
Below a confidence threshold, the assistant stops and hands the conversation to a human, with the transcript and the detected intent attached so the agent is not starting cold.
Handing off feels like a downgrade when you are demoing the thing. It is not. It is the mechanism that makes the other 60% believable, because a system that never escalates is a system that is guessing somewhere.
The metric worth watching is not resolution rate. It is escalation rate, broken down by topic. It tells you what the assistant does not know, which tells you what documentation to write next. It is the closest thing to a free product roadmap we have found.
Treat the model as one dependency among many
A model provider is a third-party API that sits outside your infrastructure boundary. It deserves the same rigour as any other external dependency: its own timeout, its own fallback path, monitoring on latency and spend, and an answer to the question of what the product does when it is unavailable.
The rest of the system needs the same care it would need without any AI in it. Real people fire off three messages in a row, and messaging platforms retry their webhooks, so we put a FIFO queue in front of the orchestration with per-conversation ordering. Without it you get replies out of sequence, or sent twice. That has nothing to do with the model and everything to do with whether the feature works.
Learn the platform’s rules before you design around them
The WhatsApp Business Platform has a 24-hour customer service window. Inside it you can reply freely; outside it you need pre-approved message templates.
That is not a technical footnote, it shapes the product. If somebody messages at 11pm and the assistant escalates, a human has until 11pm the next day to reply in a normal conversation. After that the interaction changes shape entirely. Anyone who has shipped on WhatsApp has hit this. Anyone who has not will find out the hard way.
Start with the problem, not the model
The best AI features are usually invisible: a smarter search, a better default, a task quietly automated. Nobody describes them as AI features, which is a good sign.
Keep humans in the loop wherever a mistake is expensive, and design the experience so people can correct the system easily. Then measure what it gets wrong as carefully as what it gets right.