Shipping AI
Human-in-the-loop design: deciding when to ask a person
Route too much to review and you have rebuilt the manual process. Route too little and people stop trusting the output. Where the line goes.
5 min readByteWeave Studio
Every automation project with real consequences ends up with a review step. The interesting design question is not whether to have one, it is what reaches it, and most teams decide that by accident — a threshold picked in week two and never revisited.
It deserves more attention than that, because the threshold is where the accuracy of the model turns into the economics of the system.
The threshold is a business decision
Two numbers move against each other: the share of items handled automatically, and the error rate among them. Where you sit on that curve depends entirely on what a mistake costs. A wrong line item on an internal expense report is an annoyance. A wrong figure in a filed return is a different category of problem, and the same system should be tuned differently for each.
Frame it that way with the client rather than presenting a single accuracy number. "At this setting, 78% clears automatically and roughly one in three hundred of those is wrong" is a decision someone can actually make. An accuracy percentage on its own is not.
Ask for the smallest possible judgement
The most common design failure is escalating a whole item when only one part of it is uncertain. Showing a reviewer an entire document and asking them to check it re-creates the manual process the project was meant to remove, and reviewers behave accordingly: they skim, and skimming defeats the point.
Escalate the field, not the page. Show the value, show the region of the source it came from, and let the person confirm or correct one thing. On our extraction work, highlighting the source region roughly halved the time per correction, and it was a change to the interface rather than the model.
Design for the reviewer’s attention, not their diligence
A queue of items that are almost always correct trains people to approve without looking. This is not carelessness, it is a rational response to a low base rate, and no amount of instruction fixes it.
Two things help. Order the queue by uncertainty so the genuinely doubtful cases are seen first, while attention is fresh. And keep the queue short enough to finish, because a queue that can never be cleared stops functioning as a task and becomes background noise.
Corrections are the most valuable output
Every override is a labelled example produced by an expert, in context, for free. Systems that discard them are throwing away the best training data they will ever have, and this is a surprisingly common omission — the review interface writes the corrected value to the database and forgets what it replaced.
Store the original prediction, the correction, the confidence and the rubric or rule involved. Corrections cluster, and the clusters point at specific weaknesses far more reliably than aggregate accuracy. That feedback loop is usually worth more than the next model upgrade.
- Human-in-the-loop
- Product design
- Automation
- UX
More reading
Shipping AI
How to price an AI project when you can’t promise accuracy
Fixed-price assumes a known scope. Machine learning does not give you one up front. A structure that protects both sides without pretending.
5 min read
Shipping AI
What actually breaks in a production ML system
It is almost never the model weights. Four failure modes we keep meeting, and the monitoring that catches each one.
5 min read
Document AI
Why your invoice OCR works in testing and fails in production
The test folder is clean exports. The real intake is phone photographs. What changes between the two, and how to find out before a client does.
5 min read
Have a problem
worth solving?
Tell us what you're building. We'll help you figure out what's possible — and say so if we're not the right people for it.