The pilot cleared 500 invoices in a week. By month three the clerks were keying them by hand again — and nothing had crashed. What actually keeps an automated process alive past month twelve: idempotency keys on every side effect, durable execution state, and an exception queue with someone's name on it.
Four thousand supplier invoices a month, sixty vendors, one shared mailbox. In week one of the pilot we parsed 500 PDFs, matched line items to purchase orders, and posted the clean ones into the ERP with eleven human touches total. By month three, throughput was down to roughly 40% and two AP clerks were keying invoices by hand again. Nothing had crashed — no alert, no failed run. One vendor had started splitting a single delivery across two POs; another began emailing credit notes under the same subject line as its invoices. The matcher had no opinion about either, so it parked them in a 'needs review' status that appeared on no dashboard and belonged to no one. That queue held 1,900 documents by the time we looked.
That is how automated processes die, and the tooling is almost never the cause. The happy path is maybe 15% of the engineering and 100% of the demo. Month twelve depends on four behaviors nobody demos: what happens when step three commits and step four times out, when a vendor changes a template without telling anyone, when the same message arrives twice, and when a document genuinely needs a person. Build those four first and the happy path is whatever's left.
Every process has a distribution, and the labor lives in the tail. On that pipeline, 3,400 documents a month matched a single PO cleanly and posted untouched. The other 600 sorted into nine recognizable shapes: multi-PO deliveries, partial shipments, credit notes, foreign currency against a stale FX rate, duplicate submissions from the vendor's own automation, scanned faxes at 60% OCR confidence, and a one-off tail. Those 600 burned more clerk-hours than the 3,400 ever had, because the automation had thrown away the context needed to resolve them and the clerk rebuilt it from the original email thread.
So the first two days of an engagement go to sampling: pull 250 real historical items, bucket them by hand, count. It produces the only honest ROI number you'll get. The math is never 'we automate 4,000 invoices.' It's 'we automate 3,400 end to end, we cut handling time on 450 by attaching the evidence to the decision, and 150 stay manual because they're negotiations, not data entry.' A percentage quoted before the sampling is a guess, and the guess runs optimistic every time.
Every queue, webhook, and workflow engine worth deploying delivers at least once, so your steps will run twice. A 30-second timeout on the ERP post does not tell you the invoice failed; it tells you the response didn't arrive. Retry blindly and you've paid a supplier twice, and you'll hear about it from their remittance team. It's the most common bug we inherit from other people's automations, and it hides for months because it only fires when the downstream system is slow — which is exactly when nobody is reading logs.
The fix is boring and non-negotiable. Every side-effecting step gets an idempotency key derived from the business identity of the work, never from the run — for invoices, a hash of vendor ID, invoice number, and gross total. The step checks a dedupe table on that hash before posting, then writes the remote record ID back against the same key in the same transaction. A double delivery becomes a lookup that answers 'already posted, ERP document 88214.' Where the downstream system offers real idempotency keys, use theirs. Where it doesn't — most ERPs — read-before-write against the natural key covers everything but a true concurrent double-fire, and a unique index on the hash covers that.
The homegrown version keeps state in a `status` column and a script that wakes every five minutes hunting for rows to advance. It holds until the process has to wait — for an approver, a supplier reply, the nightly bank file — and then you're writing your own timeouts, your own backoff, and your own answer to what happens when the box reboots mid-step. We've watched approvals sit for eleven days. No in-process script survives that, and no cron loop stays readable past six states and four escape hatches; the last one we inherited had a boolean named `is_retry_2`.
Persist execution state as a first-class object: Temporal when the process is long-lived and money moves, n8n in queue mode against Postgres for mid-weight work, a plain durable state machine when the graph has fewer than ten nodes. The property matters, not the logo — resumable from the last completed step, and every step individually replayable without re-running the ones before it. Then a failed ERP call is one step you retry at 2pm after the vendor's maintenance window, not a workflow you rerun from the email parser.
Human review is a designed step, not where work goes to die. The version that stuck put every low-confidence item on one screen: parsed fields beside the source PDF page, the two candidate POs the matcher couldn't choose between with the line-item diff highlighted, and one-click accept-or-correct. Median exception handling went from about six minutes to under 90 seconds, because the clerk was making a decision instead of running an investigation. Every correction wrote back as a labeled example against the source document, which is how extraction accuracy actually improves — not by swapping models each quarter.
Set confidence thresholds as explicit numbers and revisit them monthly. Track exception rate per bucket, not in aggregate; aggregate hides the one vendor whose template changed behind 3,400 clean invoices. A bucket rate climbing two weeks running is the early warning. On that account, overall exception rate fell from 15% to roughly 4% over five months, and both spikes in between traced to real external changes — a vendor's new PDF generator that moved the invoice number into the footer, and an ERP release that started returning currency codes lowercase. We caught each within a day instead of at quarter close.
Ask what happens when step three commits and step four times out, and listen for whether 'idempotent' shows up before you say it. Ask where execution state lives and whether one failed item can be replayed alone, without rerunning its 3,999 siblings. Ask what the exception queue looks like on day one and whose name is on it in week six — an automation with no owner degrades quietly, and quiet reads like success. Then ask for the exception rate on a process they've operated for six months or more, broken out by bucket. Anyone who has run one has the number and will complain about it; anyone who has only shipped pilots will start talking about the tooling.
A software studio that ships and maintains its own products — KeepChats, Gwora and MoveProof — and builds the same way for clients. Founded and led by codewithumar.
Talk to the studio →Send the problem. You get one fixed number and a plan back within a business day.
We build software worth keeping — for clients, and for ourselves.
Founded & led by codewithumar