The demo answers three cherry-picked questions and everyone in the room nods. Question 40 is where it dies: the retriever pulls a paragraph from the deprecated 2023 handbook, the model stitches it into a refund policy that was never real, and now your support lead is on the phone explaining to a customer why the bot promised them money. Nobody films that part. But that's the job — and it's where most of the cost and nearly all of the skill actually lives.
So before you post a req or sign a contract, be clear about what you're buying. An AI developer worth hiring is part backend engineer, part data plumber, and part experimentalist who treats the model as a flaky third-party dependency that returns a different answer to the same prompt at temperature 0.7 — not a magic box. Below is what the role really involves, what it costs across the three tiers you'll meet, and a one-hour test that separates people who ship from people who wrap.
What the job actually is
A competent web developer stands up a chatbot in an afternoon. What they usually can't do is make it correct on your data past a dozen rows. That gap is the whole discipline. It's chunking a 40-page contract so a retriever finds the indemnity clause instead of the header of one section glued to the footer of another. It's picking an embedding model, building a vector index that still returns in under 100ms at 400k rows instead of 12, and adding a reranker the moment cosine similarity starts handing back plausible-but-wrong neighbors. It's writing prompts as contracts with an explicit "if the documents disagree, refuse" path, then wrapping every tool call in real input validation because the model will pass `{"user_id": "the customer's account"}` as a literal string the first day you stop watching.
The tell of a real AI developer is that they raise failure modes before you do. They'll tell you retrieval recall moves the needle more than swapping GPT for Claude on most RAG systems, that a hallucination is usually a retrieval miss wearing a generation costume, and that the 200ms of latency you're fretting over is noise next to the rerank pass you'll be forced to add anyway. They think in evals, not vibes. If a candidate's plan for measuring quality is "we'll try it and see," you're paying senior rates for a prototype.
The three tiers you'll actually meet
At the bottom, roughly $25–50/hour, is the API-wrapper tier: freelancers who integrate a model and ship a clean chat UI. Fine for an internal tool over a small, tidy dataset. They stall the moment retrieval quality, token-cost control, or evals enter the picture, because none of that was in the tutorial they learned from. This budget pays off only if your problem is genuinely small and you can eat the rework.
The middle — about $60–120/hour, or a full-time salary in the $130k–180k US range — is where you find engineers who've shipped a RAG or agent system to real users and carry the scars. They can stand up an eval harness, instrument spend down to per-request token counts, and debug why an agent loops eleven times on a tool that keeps returning an empty list. Most B2B products should hire here. At the top — $150/hour and up, or specialist studios — you're paying for people who've run these systems under load: retrieval that stays fast across millions of vectors, multi-step agents that don't compound a 5% per-step error into garbage by step six, and the judgment to say "don't use an agent here, use a state machine." The gap between tiers isn't hours. It's how much of the failure surface they've already hit face-first.
How to test one in an hour
Skip the LeetCode. Hand the candidate a 15-document corpus with two files that deliberately contradict — an old leave policy and a superseding one — and ask for a question-answering flow that returns the current answer and refuses when the sources conflict. Watch what they reach for. Do they chunk with structure, or paste whole files into the context and hope? Do they add a reranker when naive similarity keeps surfacing the stale doc? When it hallucinates, do they blame the model or open the retrieval step? The strong ones narrate their uncertainty and instrument as they go instead of eyeballing outputs.
Then ask the cheap, revealing question: how would you know this still works next week, on questions you haven't seen? A weak answer is "I'd re-test it by hand." A strong answer names a golden eval set, scores retrieval and generation as separate numbers, and describes a regression check that fires when someone swaps the model or edits a prompt. That one answer predicts more than any line on a resume — it's the line between someone who builds demos and someone who operates a system.
The cost nobody quotes you
The build is the cheap part. The expensive, invisible line item is the loop after launch: evals, monitoring, and iteration as your data drifts and users ask things nobody scripted. An AI feature isn't a project you finish; it's a dependency you maintain against a provider who deprecates a model version on 90 days' notice and silently retunes the one you're on. If a quote or a candidate never mentions ongoing evaluation and observability, they're either underpricing the work or they've never watched a model quietly lose 8% accuracy overnight and had to prove it with numbers to a skeptical VP. That's why the salary and the studio rate look steep next to a plain web build — you're paying for someone who plans for month two, not just week one.
Dedicated hire or a studio
If AI is core to your product and you'll ship features for years, hire full-time at the middle-to-top tier — the compounding context of someone who has memorized the shape of your data outweighs any hourly saving. If you need one well-built system plus an internal team ready to inherit it, a studio that does this repeatedly gets you there faster and hands over the eval harness as a deliverable, not an afterthought. The wrong move is the cheapest one: a wrapper who ships a gorgeous demo, invoices, and vanishes — leaving you to discover on question 40 that nobody built the part that actually mattered.