Every RAG development company has a demo that works. It's usually a chatbot answering questions about a clean product manual or a handful of tidy Markdown files. The demo proves the vendor can call an embedding API and a language model in the right order. It tells you almost nothing about what will happen when they meet your scanned contracts, your 40-tab spreadsheets and the policy PDF that exists in three versions.

Retrieval-augmented generation fails in boring places. It fails in parsing, in chunking, in permissions and in stale indexes, long before the model gets a chance to sound wrong. So the useful way to evaluate a vendor is to stop watching the answers and start asking about everything upstream of them. A team that has shipped this before will have specific opinions there. A team that hasn't will steer you back to the demo.

Ask what happens before the model

Hand the vendor a sample of your ugliest material and ask how they would ingest it. A good answer names the problems before you do: tables that flatten into nonsense when a PDF is converted to text, headers and footers repeated on every page and polluting every chunk, scanned pages that need OCR and come back with broken line order. They should talk about chunking by the document's real structure, such as sections, clauses or rows, rather than by a fixed character count that slices a sentence in half.

If the answer is "we'll use a standard loader and tune it later," that's the tell. Most of the quality in a production RAG system is decided at ingestion, and "later" usually means after you've paid for the build. Ask them to show you parsed output from your own files, not their architecture diagram. Ten minutes looking at what the chunks actually contain will teach you more than an hour of slides.

Make them show you an eval set

The single strongest signal is whether the company measures retrieval separately from generation. If the right passage never reaches the model, no prompt will rescue the answer. A serious team will want to build a set of real questions from your users, each paired with the document and section that should answer it, and then track how often that section shows up in the top results. When retrieval changes, they rerun the set and can tell you whether things got better or worse instead of guessing.

Ask what the system does when the answer isn't in your documents. The correct behaviour is to say so and point at what it did find, not to produce a fluent paragraph from the model's general knowledge. We've written about keeping RAG answers grounded in your sources, and the same principle applies to vendor selection: a company that can't describe how it tests for refusals hasn't tested for them. The broader habit of testing code that calls an LLM is worth asking about too, because a RAG pipeline is still software that needs regression checks.

Permissions and freshness are the real scope

Two requirements quietly double the size of most RAG projects, and weak proposals leave both out. The first is access control. If an intern and the CFO ask the same question, they may be entitled to different documents, which means permissions have to be enforced at retrieval time rather than hoped for in the prompt. Ask exactly where that filter runs and what happens when someone's access is revoked.

The second is freshness. Documents get edited, superseded and deleted, and the index has to follow. Ask how a changed file gets re-embedded, how a deleted one disappears from search, and how long both take. A vendor who quotes a fixed price without asking about either is quoting for a prototype, and a realistic view of what a RAG system costs depends heavily on these two answers.

India, the USA, or anywhere else

People search for a RAG development company in India or in the USA as if geography settles the question. It changes the rate card and the overlap in working hours, and both matter. It doesn't change whether the team knows how to parse a table or measure recall. Strong and weak RAG teams exist in Bangalore, Austin and Lahore alike, and the evaluation above works the same way wherever the invoice comes from.

What location should change is how you structure communication. If the overlap is two hours a day, insist on written weekly eval results rather than status calls, so progress is a number you can read rather than a mood you have to interpret. A distributed team that reports this way is often easier to manage than a local one that doesn't.

Buy a pilot, not a proposal

The cleanest way to choose is a short paid pilot on a slice of your real corpus, with the eval set as the deliverable that matters. Two to four weeks is usually enough to see whether retrieval holds up on your documents and whether the team communicates problems early or hides them. At the end you own the questions, the parsed data and a baseline score, which stays useful even if you pick someone else.

That's how we prefer to run RAG development at Duskel: messy documents first, measurement before polish, and a pilot that earns the next phase rather than assuming it. Whoever you hire, make them prove retrieval on your own material before you pay for the interface.

Questions, answered.

How do I choose a RAG development company?

Give shortlisted vendors a sample of your messiest real documents and ask how they would parse and chunk them. Then ask how they measure retrieval quality separately from answer quality. Teams that have shipped RAG before answer both specifically.

What should a RAG development company deliver?

Beyond the chat interface, you should get an ingestion pipeline that handles your formats, an evaluation set of real questions with expected sources, and a process for re-indexing changed or deleted documents. Access control at retrieval time should be included if different users see different content.

Is it cheaper to hire a RAG development company in India?

Hourly rates are usually lower, but the skills that decide RAG quality, like parsing, chunking and evaluation, aren't tied to a country. Judge vendors on how they handle your documents and how they report results, then weigh rate and time-zone overlap.

Why does a RAG demo work but fail on my documents?

Demos usually run on clean text, while real corpora have tables, scans, repeated headers and outdated versions. Most failures happen at ingestion and retrieval, so the model never sees the right passage. Better prompts don't fix a retrieval problem.

Should I start a RAG project with a pilot?

Yes, a short paid pilot on a slice of your real corpus is the most reliable way to judge a vendor. Make the evaluation set and a baseline retrieval score the main deliverables. You keep both even if you change teams.