The demo bot always gets the refund question right. It was built on the twenty questions the vendor expected. On day one a customer types "cancel my order i got charged twice??" about a plan you retired in March, and the bot cheerfully quotes a 30-day refund window you never offered. Now your support lead can honour a policy nobody wrote or argue with a customer who has a screenshot.
So judge an AI chatbot development company by its failures, not its demo. This checklist is for teams hiring a custom chatbot for support, sales or internal use. It covers the four places bots break under real users: questions they can't answer, stale sources, the handoff to a human, and who owns the system once the vendor leaves.
Bring your own questions
Before any sales call, pull thirty questions from your real inbox, tickets and call notes. Keep them ugly: typos, two questions in one message, an all-caps complaint, a question about a discontinued product, and at least five the bot must refuse, such as legal advice, a discount nobody approved or another customer's order. Send the same set to every company on your shortlist and ask them to run it against a prototype built on a sample of your content, not their showcase bot. A competent team will scope this as a one- or two-week paid discovery, because it's the test they'd run on their own work anyway.
Score the failures, not the hits. On an unanswerable question you want "I don't have that, here's who does", with no invented policy. When a reply is wrong and the company says "we'll tune the prompt", ask how they'll know the fix didn't break five other answers. Prompt edits regress unrelated answers constantly: a line added to stop refund promises can make the bot refuse to explain shipping. The right answer is a saved test set rerun on every change, with pass counts compared before anything ships. That is the discipline behind testing LLM code properly.
Ask where every answer comes from
Nearly every useful business chatbot is a retrieval system with a chat window on top. The model is the easy part, since any serious company calls the same frontier APIs you could. The engineering is in how your documents are chunked, how the right chunk is found and whether the bot cites it. Split a pricing table by fixed character count and the plan name lands in one chunk and its price in the next, so the bot quotes the Pro price for Starter. Ask to see the retrieved passages beside each answer in the prototype. If they can't show you, they can't debug it either.
Then ask about freshness. Your pricing page changes, the help centre gets rewritten, and a policy PDF is replaced with v2 while v1 stays in the index. Ask what triggers a re-index (a nightly job, a webhook from your CMS, or someone remembering to click a button) and what removes the old version. Stale answers are the most common quiet failure in production chatbots, and a demo never shows them because its content was loaded yesterday. Keeping RAG answers grounded covers the mechanics.
The handoff is the product
For a support bot, the moment the conversation passes to a human matters more than most answers before it. Ask what triggers escalation: low retrieval scores, a customer repeating the question, words like "chargeback" or "lawyer". Then ask what the agent receives. A good handoff opens a ticket in the helpdesk you already run, with the full transcript, the customer's identity if known and the sources the bot tried, so your agent doesn't open with "how can I help?" to someone who has explained the problem twice. A bad handoff is an email address printed in the chat.
If the bot takes actions, such as looking up an order, booking a call or issuing a refund, the questions get sharper. Which actions run without a human confirming? Does the bot hold an admin key when it only needs read access to orders? When the order API times out halfway through a refund, does the retry refund twice? A team that has shipped this answers with scoped keys, idempotency keys on every write, confirmation steps and audit logs, without looking anything up.
What you own after launch
Launch is where a chatbot's cost starts. Ask who owns the system prompt, the retrieval configuration, the evaluation set and the conversation logs, and get "you" written into the contract. Ask whether you can switch model providers without a rebuild. Prices and quality move every few months, and a bot hard-wired to one vendor's SDK is a bill you can't negotiate. If the company is reselling a hosted platform with your logo on it, that can be fine, but you should know before you sign, and an honest proposal includes weighing custom against SaaS.
Ask for the running cost per conversation at your expected volume, with model tokens, the vector database and hosting on separate lines. A flat retainer hides whether you're paying for engineering or a markup on API calls. Then ask what month one of monitoring looks like. The answer you want: an engineer reads every flagged conversation weekly and turns each bad one into a test case, so your thirty questions are sixty by the end of the month.
Location matters less than overlap
Searches for chatbot companies in India, Dubai, the USA or Bangalore are really searches for a price band or a time zone. Both are real concerns, and neither tells you whether the bot holds up. A team in Lahore or Chennai that runs your thirty questions, shows its sources and builds a clean handoff will beat a local firm that sends a slide deck. What matters is three or four hours of daily overlap in the first weeks, when wrong answers are found and fixed fastest, and knowing the name of the engineer writing the retrieval code.
So build the shortlist on evidence. Send the same question set to three companies, pay for a small prototype if you need to, and pick the one whose failures you understand best. If you want a team that works this way, our AI chatbot development work starts with exactly that test.
Questions, answered.
How do I choose an AI chatbot development company?
Give each company the same thirty real questions from your inbox, including at least five the bot must refuse, and have them run the set on a prototype built from your content. Pick the team whose bot admits what it doesn't know, cites its sources and hands off to a human with the full transcript.
How much does it cost to hire an AI chatbot development company?
Cost scales with content volume, number of integrations and whether the bot takes actions like refunds or bookings. Ask for the build cost and the running cost per conversation separately, with model tokens, vector database and hosting on their own lines, so a retainer can't hide an API markup.
Should I hire an AI chatbot development company in India or the USA?
Location changes price and time zone, not quality. What predicts success is whether the team passes your question test, shows the retrieved source behind every answer and gives you three or four hours of daily overlap in the first weeks after launch.
What should a custom AI chatbot proposal include?
How your content is chunked, retrieved and re-indexed when it changes; what triggers escalation and what the human agent receives; which actions the bot can take and with which scoped permissions. It should state that you own the prompts, evaluation set and conversation logs, and that the model provider can be swapped without a rebuild.
Why does a chatbot work in the demo but fail with real customers?
Demos run on the questions the vendor expected and content loaded the day before. Real users send typos, double questions and out-of-scope requests, and your source documents change, so a bot without a rerun test set and a re-indexing trigger drifts into wrong answers within weeks.