hsa-app/OCR/README.md

I want to create an OCR feature (ollama, whatever) that's 

1) local (assume I have a GPU with 24GB VRAM)
2) works on receipts (scanned or digital)
3) given a receipt, return an ordered list of rows of text in that receipt
4) example /home/jm/programming/HSAmanager/OCR/example/data/aldi.jpg

---

## Getting started (working prototype)

Local, GPU-backed, no cloud / no API key. Engine: **Qwen2.5-VL 7B** served by
**ollama**. The model transcribes the receipt top-to-bottom and returns one
string per physical row, columns joined left-to-right.

### One-time setup (already done on this machine)

- ollama binary installed (no root) at `~/.local/bin/ollama`.
- Model pulled: `ollama pull qwen2.5vl:7b` (~6 GB).

### Run

```bash
# 1. start the server (leave running; ~6 GB VRAM when the model loads)
~/.local/bin/ollama serve &

# 2. transcribe a receipt -> ordered, numbered rows
python3 OCR/receipt_ocr.py OCR/example/data/aldi.jpg

# raw JSON {"rows": [...]} instead of numbered text
python3 OCR/receipt_ocr.py OCR/example/data/aldi.jpg --json
```

No third-party Python deps — `receipt_ocr.py` uses the stdlib and the ollama
HTTP API. Override with `OLLAMA_MODEL` / `OLLAMA_HOST` env vars.

### Notes / next steps

- Works on the ALDI example with near-perfect line ordering and column joining.
- Caveat: the model sometimes *expands* thermal-print abbreviations
  (`Fire Roasted Tom` → `Fire Roasted Tomatoes`). Helpful for readability but
  not byte-for-byte faithful; tighten the prompt if you need verbatim text.
- Accuracy upgrade: `qwen2.5vl:32b` (q4 ≈ 20 GB, fits the 3090 but tighter/slower).
- This is local-only and complementary to `../receiptscan/`, which uses the
  Claude cloud API for line-item categorization + annotated rendering.

---

## Operating ollama (runbook)

ollama is the local model server both scripts talk to over HTTP
(`http://127.0.0.1:11434`). It must be running before you call either script.
Installed **without root** at `~/.local/bin/ollama` (the binary was extracted
from the `ollama-linux-amd64.tar.zst` GitHub release; `~/.local/bin` is on PATH,
so plain `ollama` works too).

### Start / check / stop

```bash
# Start the server (does NOT survive a reboot or terminal close — restart it):
ollama serve &                     # or: nohup ollama serve > /tmp/ollama.log 2>&1 &

# Is it up? (prints a JSON version string if so)
curl -fsS http://127.0.0.1:11434/api/version

# Is the process alive?
pgrep -af "ollama serve"

# Stop it:
pkill -f "ollama serve"
```

After a reboot the server is **not** running — just `ollama serve &` again. The
pulled models persist on disk (under `~/.ollama`), so you don't re-download them.

### Models

```bash
ollama list            # models on disk
ollama ps              # models currently loaded in VRAM (and how long until idle-unload)
ollama pull <model>    # download a model, e.g. ollama pull qwen2.5:14b
ollama rm <model>      # delete a model to reclaim disk
```

Currently pulled: `qwen2.5vl:7b` (OCR), `mistral-small3.2:24b` (grouping, default),
plus `qwen2.5:7b` / `qwen2.5:14b` (earlier grouping experiments — `ollama rm`
them if you want the disk back).

### VRAM / GPU notes

- A model loads into VRAM on first request and **auto-unloads after ~5 min idle**
  (so `ollama ps` is often empty — that's normal; the next call reloads it in a
  few seconds).
- Only run one large model at a time on the 24 GB 3090. The OCR model (~6 GB) and
  the grouping model (~15 GB) can coexist, but bumping grouping to
  `qwen2.5:32b`/`qwen2.5vl:32b` (~20 GB) leaves little room — expect a reload
  when switching between OCR and grouping.
- Force-unload now (free VRAM without stopping the server):
  `ollama stop <model>`.
- Keep a model resident longer / shorter via the request `keep_alive` field, or
  globally with `OLLAMA_KEEP_ALIVE` (e.g. `OLLAMA_KEEP_ALIVE=30m ollama serve`).

### Pointing the scripts elsewhere

Both scripts honor `OLLAMA_HOST` and `OLLAMA_MODEL`:

```bash
OLLAMA_MODEL=qwen2.5:14b python3 OCR/receipt_group.py        # try another model
OLLAMA_HOST=http://other-box:11434 python3 OCR/receipt_ocr.py img.jpg  # remote server
```

### Logs / troubleshooting

- If a script hangs or errors connecting, the server is probably down — check
  with the `curl` above and restart.
- Startup logs (GPU discovery, VRAM, errors) go wherever you redirected `serve`
  (e.g. `/tmp/ollama.log`); tail that if a model fails to load.
- "model not found" → you haven't `ollama pull`ed it (or typo'd the tag); see
  `ollama list`.


# grouping the OCR results
next task is ... given an OCR output, like:
```text
 1  ALDI
 2  Store #145
 3  1501 Rockville Pike
 4  Rockville, MD
 5  https://help.aldi.us
 6  Your cashier today was Wendy
 7  382175 Unsalted Peanuts 2.29 FA
 8  382175 Unsalted Peanuts 2.29 FA
 9  384773 4 lb. Sugar 2.79 FA
10  384773 4 lb. Sugar 2.79 FA
11  382437 Fire Roasted Tomatoes 2.30 FA
12  2 @ 1.15
13  371607 Canned Cat Food 2.28 NB
14  4 @ 0.57
15  371607 Canned Cat Food 0.57 NB
16  416787 1% Milk, Gallon 3.38 FA
17  343557 Protein Powder 18.49 FA
18  416645 Chocolate Milk 1.91 FA
19  382260 Plain NF Greek Yogurt 2.79 FA
20  382260 Plain NF Greek Yogurt 2.79 FA
21  297956 PureandSimple Bars 3.99 FA
22  356508 Broccoli Crowns 2.21 FA
23  1.17 lb x 1.89/lb
24  469529 Corn Tortillas 1.95 FA
25  609748 Assorted Cashews 6.49 FA
26  634568 Chicken Skewers 7.49 FA
27  382473 Shredded Mozzarella 3.29 FA
28  341878 Yellow Onions 1.85 FA
29  535290 ABF B/S Thighs 6.61 FA
30  272135 Salmon Portions 9.35 FA
31  272135 Salmon Portions 8.54 FA
32  382310 Feta Crumbles 1.29 FA
33  356508 Broccoli Crowns 2.53 FA
34  1.34 lb x 1.89/lb
35  262137 Stuffed Olives 2.89 FA
36  356490 Bagged Avocados 2.99 FA
37  356607 Red Delic Apples 2.75 FA
38  356527 Celery 1.89 FA
39  382653 Indian Sauces 3.69 FA
40  356628 Seedless Cucumber 0.89 FA
41  356684 WildTwist Apf LRW 5.44 FA
42  (G) 3.10 lb - (T) 0.06 lb
43  (N) 3.04 lb x 1.79/lb
44  282119 Pineapples 1.89 FA
45  356419 Mangoes 3.80 FA
46  4 @ 0.95
47  356691 Zucchini 2.01 FA
48  1.69 lb x 1.19/lb
49  356522 Cantaloupe 1.89 FA
50  356504 Blueberries 9.95 FA
51  5 @ 1.99
52  388137 Large Eggs 2.92 FA
53  2 @ 1.46
54  262747 Bananas LRW 1.45 FA
55  (G) 2.97 lb - (T) 0.01 lb
56  (N) 2.96 lb x 0.49/lb
57  262747 Bananas LRW 1.13 FA
58  (G) 2.32 lb - (T) 0.01 lb
59  (N) 2.31 lb x 0.49/lb
60  356427 Multi-Peppers 3pk. 2.69 FA
61  341876 Red Grapes LRW 5.48 FA
62  (G) 3.96 lb - (T) 0.02 lb
63  (N) 3.94 lb x 1.39/lb
64  479744 Campari Tomatoes 2.99 FA
65  479744 Campari Tomatoes 2.99 FA
66  569269 Protein Bread 3.99 FA
67  343989 Family AsstCookie 1.99 FA
68  356646 Strawberries 2.09 FA
69  356646 Strawberries 2.09 FA
70  356646 Strawberries 2.09 FA
71  VISA 172.42
72  ************0294 ONLINE
73  06/21/26 12:43 Ref/Seq # 0
74  Auth# 00401D
75  AID A000000031010
76  TVR 0000000000
77  IAD 06021203A00000
78  TSI 0000 ARC 00 EntryMode 07
79  ++APPROVED++
80  SUBTOTAL 172.24
81  B-Taxable @6.00% 0.18
82  A-Taxable @0.00% 0.00
83  AMOUNT DUE 172.42
84  TOTAL $172.42
85  60 ITEMS
86  Credit Card $ 172.42
87  *7367 L411/005/064 06/21/26 12:43PM
88  ************
89  Sign up for ALDI emails
90  for a sneak peek on the weekly ad!
91  www.aldi.us/signup
```

write a LLM call, still using my local ollama (you can switch the model). I want the output to be an object roughly like that:

store name:
store branch:
store address:
list of rows that belong together as one item purchased
total in dollars.
(anything else is relevant here)?

you can hardcode the lines for this test and let's call the LLM to see what it does

the output needs to be a new script (receipt_group.py) that (for now) read this examle as is from an example file that you will create, and with the correct prompt organizes the information correctly. 

## Grouping — implemented

`receipt_group.py` reads OCR rows (default
`example/data/aldi_ocr.txt`) and asks a local ollama text model to reconstruct
the receipt: store name/branch/address, line items (merging multi-row items
such as weighed produce and `qty @ price` lines), total, and metadata
(cashier, datetime, payment, subtotal, tax, item count).

```bash
python3 OCR/receipt_group.py                       # groups only: each item + its rows
python3 OCR/receipt_group.py OCR/example/data/aldi_ocr.txt --json   # full object + reconciliation
```

- Default model: `mistral-small3.2:24b` (~15 GB). Override with `OLLAMA_MODEL`.
  Tried qwen2.5:7b (many merge errors) and qwen2.5:14b (~90%); Mistral Small got
  the grouping right. Swapping models is just the env var — no code change.
- **Why an LLM groups, not a regex**: detecting item vs detail rows is trivial,
  but deciding *which* item a detail row attaches to is not — wrapped
  descriptions, discount lines, and qty/weight lines can sit above or below the
  item, so a positional rule breaks. The prompt tells the model to attach each
  detail row to the item its **math** reconciles with (count × unit, weight ×
  rate = the item's charged price), never by a fixed above/below position.
- **Completeness clause** in the prompt: every input row must land in exactly one
  item's `rows`, copied verbatim, and the model must walk the input top-to-bottom
  to confirm nothing was dropped. This fixed the model silently swallowing Large
  Eggs' `2 @ 1.46` line. Prompting lowers the odds of a drop but cannot guarantee
  it — hence the backstop below.
- **Deterministic reconciliation pass** (the part you *can* trust), shown in
  `--json` under `validation`: Python recomputes each item from its detail math,
  sums items vs the printed subtotal, checks total = subtotal + tax, and sums
  unit quantities vs the printed `N ITEMS`. When the model slips (e.g. dropping a
  `qty @` line), the unit-count check flags it (59 vs 60) even though dollars
  still balance.