How AI Reads a Receipt in Three Seconds, and Where It Still Gets It Wrong

Photographing a receipt and getting back a finished expense record feels like a small magic trick. It is not one. It is four separate problems solved in sequence, and knowing what they are makes it obvious why some receipts scan perfectly and others do not.

Step one: finding the receipt

Before anything can be read, the paper has to be found in the photo. Your kitchen table, your hand and the pattern on the tablecloth are all in the frame. The software locates the four corners of the receipt, corrects the perspective so a photo taken at an angle becomes a flat rectangle, and increases the contrast between faded ink and greying thermal paper.

This step alone explains most scanning failures. A receipt lying on a busy surface, or curled so the corners lift, gives the next stage a distorted image to work with.

Step two: turning pixels into characters

Text recognition converts the image into characters. Modern recognition is very good at clean printed text and noticeably worse at the things receipts are made of: narrow thermal print, faded ink, dot-matrix output from an old till, and the ambiguity between a zero and an O or a 1 and an I in a cramped font.

At the end of this step the software has a wall of text with no meaning attached. It knows the characters. It does not yet know which of the eleven numbers on the page is the amount you paid.

Step three: understanding the layout

This is the interesting part, and where a language model earns its place. A receipt has a structure that every human reads without thinking: the shop name is at the top, the line items are in the middle, tax and total are at the bottom, and the card details come last.

The model works out which number is the total rather than a subtotal, a tax line, an item price, a discount, or the change you were given. It reads the date in whatever order that country writes dates. It identifies the merchant from a name that may be an abbreviation or a legal entity nobody would recognise.

• The total, distinguished from subtotal, tax and cash tendered
• The date, in local format, defaulting sensibly when the year is missing
• The merchant, mapped to a recognisable name
• The currency, which is often implied rather than printed

Step four: choosing a category

Finally the record needs a category, which is a judgement rather than a fact. A supermarket is usually groceries — unless the receipt is mostly fuel. A pharmacy is health, unless you bought shampoo. The model uses the merchant, the line items and your own past corrections to decide.

This is why correcting a category matters more than it looks. A correction is a signal for that merchant, and the same shop is filed correctly next time.

Where it still gets it wrong

Four cases account for nearly all remaining errors.

1. Faded thermal receipts. If you struggle to read the total, so will the model. Photograph these soon after you get them.
2. Long receipts photographed at an angle. The bottom of the page is where the total lives, and it is the part perspective distortion damages most.
3. Split payments. Half card, half cash, or a gift voucher applied at the end, produces several plausible totals.
4. Restaurant bills with a tip added by hand. The printed total and the amount you actually paid are different numbers, and only one of them is on the paper.

How to get a clean scan every time

• Flatten the receipt. A curl at the bottom costs you the total.
• Shoot from directly above, not at an angle across the table.
• Plain surface, even light, no shadow from your own hand.
• Include the whole receipt, edge to edge, with a little margin.
• Scan within a day or two for thermal paper, which fades quickly in a wallet.

Why this matters more than accuracy

People assume the value of receipt scanning is precision. It is not; a careful human typist is more accurate. The value is that it takes three seconds instead of forty, and three seconds is short enough that you actually do it at the till instead of promising yourself you will do it later.

Every expense tracking method fails at the same place: the gap between spending and recording. Closing that gap, even imperfectly, beats a perfect system you abandon in a fortnight.