# receiptscan

Sends a receipt photo to Claude, classifies the receipt into a budget category,
and (for groceries) locates + categorizes every line item, then renders an
annotated PNG: each line item is lightly shaded by food category with a category
tag in the margin, plus a per-category subtotal sidebar.

## Run

```bash
/home/jm/.local/go/bin/go -C /home/jm/programming/HSAmanager/receiptscan run .
```

- Image path is hard-coded to `/home/jm/aldi.jpg` (or pass one as an argument).
- Reads `CLAUDE_API_KEY` from `../.env` (or `ANTHROPIC_API_KEY` in the env).
- Output: `<image>_annotated.png` next to the input.
- Go is installed locally at `/home/jm/.local/go` (no root). `GOTOOLCHAIN=local`
  avoids auto-downloading a toolchain.

### Useful env flags
- `RECEIPT_CACHE=read` — re-render from the saved `<image>.cache.json` without
  calling the API (fast iteration on rendering).
- `RECEIPT_DEBUG=1` — print the raw API response, the row-fit, and per-item row
  assignments to stderr.
- `RECEIPT_ANALYZE=1` — dump detected text rows vs. model boxes and exit.

## Model query

- Model: `claude-opus-4-8` (returns true 1:1 image-pixel coordinates; Haiku does
  not localize accurately and produced vertically-stretched boxes).
- Adaptive thinking + `output_config.effort: high`. Without thinking the model
  often returned a placeholder/empty list.
- Structured output via `output_config.format` (`json_schema`, see schema in
  `main.go` → `analyze`): `receipt_type`, `currency`, `total_amount`, and
  `line_items[]` of `{name, price, category, box{x,y,width,height}}`.
- The model intermittently returns an empty `line_items` for a grocery receipt,
  so the call is retried up to 4× until items come back.

### Prompt

```
You are reading a photographed grocery store receipt. The image is exactly
<W> pixels wide and <H> pixels tall. All coordinates you return must be in
pixels in that space, with (0,0) at the top-left.

1. Set receipt_type to "groceries" and read the grand total (total_amount)
   with its currency.

2. This receipt has many purchased line items (dozens). Return EVERY one of
   them - do not stop early and do not return an empty list. For each line item:
   - name and price exactly as printed,
   - a tight pixel rectangle (box) enclosing that whole printed line, from the
     item name on the left through its price on the right,
   - a food category, exactly one of: dairy, meat, bakery_bread, berries,
     non_berry_fruits, vegetables, desserts, processed_food, pet_stuff,
     other_groceries.
   Use other_groceries only when nothing more specific fits. Do NOT include
   subtotal, tax, payment, or total rows as line items.

Return only the structured object.
```

Budget categories for `receipt_type`: house_maintenance, activities, restaurant,
groceries, hobbies, alcohol, medical, education, other.

## How box alignment works

The model's returned y-coordinates are vertically stretched (~1.35× the true
line pitch) and only roughly placed, so boxes are **not** drawn at the model
coordinates. Instead the image itself is used to place them:

1. Detect horizontal text-row bands (dark-pixel projection in the item column).
2. Keep only **item rows**: bands with ink in the price column (right) AND the
   item-code column (far left), at or below the model's first item. This
   excludes centered headers, weight/quantity sub-lines, paper-edge shadows, and
   the payment/total block.
3. The first N price-bearing rows (N = item count from the model) are the N
   items, in order — map item *i* → row *i*.

This keeps alignment correct regardless of the model's coordinate drift.