Would it be worth running this on 30 of your own purchase orders, redacted?
By whenNo deadline. A one-line reply to jon@vocabotics.com is plenty.
1What this shows
It reads a purchase order as it arrives, finds each line and checks it against that customer's contract price. It then drafts the acknowledgement, with any queries listed, for a person to check and send.
27/34
order lines right on 9 orders it had not seen
9/9
customer and PO number right
8/10
expected issues flagged, no false alarms
0/7
lines right on ward requisition forms
The orders came in four shapes: an NHS trust PO, a care group's own form, a plain email and a ward requisition form. For each one it finds the customer, PO number and delivery address. It pulls out every line, from the item code (allowing slips like "CB1001") or from the description. It checks each line against the contract price, and the order against its own rules, such as an NHS order with no PO number. It then drafts the acknowledgement for a person to approve.
Nine of the orders were written separately, from the price list and the check rules only. I scored them once, with the extractor frozen. Seven were fully right.
2Where it goes wrong
Both ward requisition forms failed completely. They set out their items as a plain-text table with the quantity at the end of the line, a layout my five build orders did not include. All 7 lines were missed, and so was one bed ordered without a mattress.
Every PO table and email came out right, including an item that is not on the price list. The failure was one layout nobody had thought of, which is how extractors usually break. I fixed it afterwards and the re-run read all 34 lines. Those orders were no longer unseen by then, so the figure to quote is 27 of 34.
3What made-up data cannot show
- How your real documents look. Scans, photographed forms and the habits of each ward are not in a set I wrote myself.
- Your real contract terms. Exceptions, rebates and unwritten rules are not in an invented price list.
- How a person uses the flags. A check only helps if the team trusts it enough to act on it.
What a real version would take
Thirty to 50 of your recent orders, redacted, become the test set, and I score the reader on them before anyone relies on it. Your price list and terms are the source of truth. Clean orders go through as drafts and anything flagged waits for a colleague.
AEvidence
Source: results/accuracy_holdout.md and results/accuracy_dev.md, scored once, extractor frozen
| Measure | 5 orders it was built on | 9 held-out orders |
|---|---|---|
| Customer account identified | 5/5 | 9/9 |
| PO number correct | 5/5 | 9/9 |
| Delivery address found, or rightly missing | 5/5 | 9/9 |
| Order lines right (item, quantity, price) | 21/21 | 27/34 |
| Expected issues flagged | 8/8 | 8/10 |
| Flags raised that were right | 8/8 | 8/8 |
| Orders fully right | 5/5 | 7/9 |
Source: results/accuracy_holdout.md
| Order | Shape | Lines right | Issue it should flag | Fully right |
|---|---|---|---|---|
| H01 | NHS PO table | 4/4 | none | |
| H02 | NHS PO table | 4/4 | price mismatch, line 3 | |
| H03 | NHS PO table | 4/4 | line total, line 3; missing PO | |
| H04 | Care PO table | 4/4 | discontinued, line 1 | |
| H05 | Care PO table | 4/4 | price mismatch, line 3; quantity check, line 4 | |
| H06 | 3/3 | none | ||
| H07 | 4/4 | missing delivery; unknown code, line 3 | ||
| H08 | Requisition form | 0/3 | bed without mattress | |
| H09 | Requisition form | 0/4 | line total, line 4 |
Source: results/extraction_holdout.json, results/order_lines_holdout.csv
Hello, we have a new resident arriving at the end of the month and need to get set up. Please can you send: 1 x bariatric low bed CB-1002 at £3,380; 1 x MT-2O03 alternating air mattress and pump at £1,180; 2 x pressure sensor floor mats; 1 x delivery and installation at £85. If the sensor mats are something you don't do, let me know and I'll source them elsewhere.
What it read. Four lines. CB-1002 at £3,380.00 and MT-2003 at £1,180.00 (read from the slip "MT-2O03") both match the contract price. Line 3, the floor mats, has no code and no match, so it is flagged as an unknown item. Line 4 is matched from its description at £85.00. At order level it flags a missing delivery address.
Source: results/accuracy_holdout_after_fix_not_heldout.md. Run after the failures were seen, so not a fair test
Source: this note
Source: not tested. Every document here was generated as a clean PDF
Next
- Reply with one line if a test on your own orders would be useful. No form, no commitment.
- Send 30 to 50 orders, redacted as you see fit. I score the reader on them and return a sheet in the same shape as above.
- Or start with a review. The Opportunity Review is
£1,950fixed over one week, remotely: a 60-minute call, a short questionnaire and up to three sample documents. You receive a written review of about eight pages. It is paid at booking and credited in full against a Discovery Sprint booked within 30 days.
Jon Ross
VOCABOTICS LTD