WhatsApp Receipt OCR — How AI Reads a Receipt and Where It Gets It Wrong

The largest share of an exchange's daily workload is typing receipts by hand. The customer sends a photo on WhatsApp, the operator looks at it, and enters the amount, date, tracking number, and bank into the document by hand. A hundred receipts a day means a hundred repetitions of the same task — and each one a chance for a typo.

AI can eliminate this typing: read the receipt photo and fill the fields itself. But "reading receipts with AI" is as dangerous as it is useful, if you trust it blindly. This article is about how exactly this reading works, where it gets things wrong, and why its output must always stay verifiable.

This article complements Verifying Payment Receipts. That article addresses "did the money actually arrive"; this one addresses "how the receipt is read." Extraction and verification are two separate jobs, and neither replaces the other.

Two stages: cheap classification first, then full extraction

The common mistake is to think AI reads every incoming photo whole and complete in one go. In practice, a correct pipeline has two stages, and their order matters.

Stage one — cheap classification: before any extraction, a light, cheap check decides whether this photo is "a financial document at all." In an exchange's WhatsApp group, everything gets exchanged: selfies, stickers, a colleague's photo, an unrelated screenshot. If you run full extraction on every photo, you have wasted the AI cost on irrelevant things.

Stage two — full extraction: only the photos that stage one identified as a "financial document" go to the full model to have the fields extracted: amount, date, tracking number, source and destination bank.

Why does this split matter? Because of cost. A full extraction with a GPT model costs roughly 1.5 to 3 cents. If 300 photos come into the groups a day and only 100 of them are real receipts, without the cheap classification stage you are also paying the cost of 200 irrelevant photos.

In Nexto, this pre-classification stage does exactly this, and non-financial photos are skipped locally and for free (with a local tool like tesseract), rather than being sent to a paid model. The result is that you only pay for what is actually a receipt.

Where AI gets it wrong

This is the most important part of the work. AI is strong at reading text, but not flawless — and its errors fall precisely on the most sensitive parts: the digits.

• Look-alike letters and digits: the letter l and the digit 1, or the letter O and the digit 0. In a long tracking number, a single swap is enough to break automatic matching.
• Ambiguous digits: 1 and 7, 0 and 5, 3 and 8 are easily read swapped on a low-quality receipt or a blurry photo.
• The thousands separator: sometimes the amount's dot or comma is dropped or added, and 1,200,000 becomes 120,000 — one zero less, a tenfold error.
• Date: different formats (Jalali/Gregorian, handwritten) are a frequent source of date error.

A simple numeric example shows why this matters. Suppose the real receipt is 48,500,000 rials and AI reads the fifth digit as 8 instead of 5: 48,800,000. The difference is 300,000 rials. If this number enters the document without direct verification, the treasury discrepancy is found later — when it is harder to find.

The practical result of these facts is one principle: AI output is a suggestion, not a truth.

Why the output must be verifiable and correctable

If AI sometimes makes mistakes, the right design is not to make it more accurate so it never errs — that is impossible. The right design is to present the output in a way that lets the operator see and correct it at a glance.

That means the document form must show the suggested fields filled and ready, next to the receipt photo itself, so that the operator can compare the amount on screen with the amount in the photo. Confirmation should be one glance and one click; correcting a wrong number should be one glance and a short bit of typing.

This is completely different from "AI records it itself and moves on." The difference is in responsibility:

• In fully automatic mode, an AI error enters the ledger directly and no one notices until a discrepancy.
• In suggest-and-confirm mode, AI takes on the tedious work (typing) and the human keeps the important work (judgment and control).

The operator is no longer the typist of a hundred receipts; they are the confirmer of a hundred suggestions. This is both faster and safer.

In Nexto, incoming receipts from WhatsApp are read with AI and the suggested document fields — amount, date, tracking number, bank — are filled, but the final recording is by operator confirmation. That is why a currency exchange accounting software that has AI must take the ability to confirm and correct as seriously as it takes the extraction.

Extraction is not verification

One final point that is often confused: the fact that AI "read the receipt amount correctly" does not mean "the money arrived." Extraction means the number on the photo was seen correctly; verification means that deposit actually landed in your bank statement and is not reversible.

A forged receipt made with a photo editor may be perfectly clean and legible — and AI may extract it flawlessly. A clean extraction says nothing about whether the money is real. That is why, after extraction, matching against the actual account statement is necessary — the same separation of extraction from verification noted at the start of this article.

Checklist: what a correct OCR pipeline has

1. A cheap classification stage before full extraction, so cost is not wasted on irrelevant photos.
2. A local, free skip for non-financial photos.
3. Structured extraction into specific fields (amount, date, tracking, bank), not a raw text.
4. Showing the suggestion next to the photo for quick visual comparison.
5. Single-field correction before recording — the output is a suggestion, not a truth.
6. Matching against the account statement after extraction — because extraction is not verification.

Summary

AI can eliminate the manual typing of receipts, and this is a real saving. But doing it right means: cheap classification first so cost is not wasted, then full extraction; accepting that it errs on the sensitive digits; and designing the output so it is always verifiable and correctable. AI takes on the tedious work, but responsibility for the final number stays with the human.

Want to see how automatic reading of a WhatsApp receipt and its confirmation work in practice? Build a dedicated demo of the Nexto currency exchange accounting software and follow one receipt from the WhatsApp message to the confirmed document.