Fintech
Invoice & statement extraction
Reads invoices, bank statements and utility bills at 97% field-level accuracy, and knows which fields it is unsure about.
- Field-level accuracy on standard documents
- 97%Field-level accuracy on standard documents
- Less manual processing time
- 85%Less manual processing time
- Fields cleared with no human review
- 78%Fields cleared with no human review
- Distinct layouts handled without templates
- 30+Distinct layouts handled without templates
The problem
Thousands of invoices were being keyed in by hand. Reconciliation waited on the typing, and reporting waited on reconciliation. The team had tried an off-the-shelf extraction API and abandoned it: it worked on the clean exports and failed on everything else, with no signal to say which was which.
The approach
A computer vision pipeline for invoices, bank statements and utility bills. It pulls each field, checks it against validation rules, and scores how sure it is, so only genuine edge cases reach a person.
Why templates were never going to work
No two documents arrive the same way. Some are scanned on an office multifunction device, some photographed on a phone in poor light, some exported clean from another system as a born-digital PDF. In the first month we counted more than thirty distinct layouts across four document families, and roughly a third of the intake had no consistent field positions at all.
Template matching handles the first few layouts and then charges you a new template for every vendor that joins. So the pipeline reads layout first and fields second: it works out where things sit on the page before it works out what they say. A new vendor format becomes a variation the model interprets rather than a configuration file somebody has to write.
How the pipeline is put together
A document lands in object storage and runs through four stages. Preprocessing deskews the page and normalises contrast, which matters more than it sounds, because most of the early accuracy gains came from fixing the input rather than the model. OCR produces text with coordinates. A layout pass groups that text into regions: header, party blocks, the line-item table, totals. Field extraction then interprets each region and emits typed values.
Validation runs last and is the least glamorous part of the system. Line items have to sum to the subtotal, tax has to reconcile against the rate, GSTIN and IFSC values are checked against their published formats, and dates are range-checked against the statement period. A field that fails validation is downgraded no matter how confident the model was, because a confident wrong answer costs more than an uncertain one.
Choosing the confidence threshold
Every field carries a confidence score, and the threshold deciding what goes straight through is the most consequential number in the system. We settled at 0.80 after running the pipeline over about nine thousand documents from the client’s own backlog. At that setting roughly 78% of fields clear without anyone looking at them.
Moving it in either direction fails in a specific way. Lower it and errors reach the ledger, people find them, and within a fortnight nobody trusts the tool. Raise it and the queue fills with fields the model had right, and you have rebuilt manual entry. We tuned against the real backlog rather than a clean benchmark set, because a benchmark set does not contain the photograph taken at an angle in a badly lit warehouse.
What we would do differently
We built the review interface last and should have built it first. For the first few weeks reviewers saw a field name and a value with no visual context, and their throughput was poor for a reason that had nothing to do with the model: they could not see where on the page the number had come from. Highlighting the source region roughly halved the time spent per correction.
We would also capture reviewer corrections as training data from day one. We added it later and it works, but the first two months of corrections, the most informative ones, from back when the model was at its worst, were never kept.
Questions we get asked
- Why not use an off-the-shelf invoice extraction API?
- The client had already tried one. It performed well on born-digital PDFs and poorly on scans and phone photographs, which were most of the intake, and it returned no per-field confidence, so there was no way to tell a reliable value from an unreliable one without checking every page by hand. Per-field confidence is the feature that makes straight-through processing safe, and it is what we built the pipeline around.
- What happens when a vendor sends a layout the system has never seen?
- Nothing breaks and no configuration is needed. Because layout understanding is separate from field extraction, an unfamiliar format is handled as a new arrangement of familiar regions. Confidence on the first few documents of a new format is typically lower, so more fields route to review; it recovers as those corrections feed back in.
- How is field-level accuracy actually measured?
- Against a manually keyed ground-truth set drawn from the client’s own documents, not a public benchmark. Accuracy is counted per field rather than per document, because a page with nineteen correct fields and one wrong one is not a failed page. It is one field for a person to confirm.
- Does anything reach the ledger without a human seeing it?
- Yes, by design. That is the whole point of the confidence threshold. Fields scoring above 0.80 and passing validation post straight through. Everything else queues for review with its source region highlighted. The threshold is configurable, and a client wanting full review can set it to 1.0 and still get the ordering and highlighting benefits.
Have a problem
worth solving?
Tell us what you're building. We'll help you work out what's actually possible, and say so if we're not the right people for it.