diff --git a/spec.md b/DESIGN.md similarity index 97% rename from spec.md rename to DESIGN.md index 570f1bf..4319f89 100644 --- a/spec.md +++ b/DESIGN.md @@ -1,3 +1,11 @@ +================================================================================ +DESIGN JOURNAL — historical, chronological record of decisions and rationale. +For the current expected behavior of the app, see SPEC.md (the source of truth). +This file explains WHY things were built the way they were; SPEC.md says WHAT the +app does today. When behavior changes, update SPEC.md; append context here only if +the reasoning is worth preserving. +================================================================================ + HSA Receipt Tracker — Requirements (v1) Purpose Capture and archive HSA-eligible receipts for future reimbursement and tax substantiation. No parsing, no OCR, no reporting. diff --git a/SPEC.md b/SPEC.md new file mode 100644 index 0000000..471340e --- /dev/null +++ b/SPEC.md @@ -0,0 +1,309 @@ +# HSA Receipt Tracker — Specification + +This is the source of truth for the app's **current** expected behavior, organized +by domain. It describes what the app does today, not how it got here — for the +history and rationale of each decision see [DESIGN.md](DESIGN.md); for the +version-by-version log see [CHANGELOG.md](CHANGELOG.md). + +Keywords **MUST**, **SHOULD**, and **MUST NOT** mark hard requirements vs. +recommendations. When behavior changes, amend the relevant section here in the same +commit. + +--- + +## 1. Purpose & scope + +Capture and archive HSA-eligible receipts for two household users, for future +reimbursement and tax substantiation. The app is mobile-first (phone camera +capture matters). It deliberately does **not** do reimbursement tracking, in-place +editing, or general reporting beyond the Tally view. + +--- + +## 2. Users, authentication & authorization + +- Two users, both with full shared access — there is no per-user data isolation; + everything is visible to every authorized user. +- Authentication is **OIDC with PKCE** against an Authelia issuer (`ISSUER_URL`), + using the registered client (`OIDC_CLIENT_ID` / `OIDC_CLIENT_SECRET`). +- The login flow: + - `GET /login` generates a PKCE verifier, state, and nonce, stores them in a + short-lived (10 min) encrypted `hsa_login` cookie, and redirects to Authelia. + - `GET /callback` validates state (CSRF), exchanges the code with the PKCE + verifier, verifies the ID token and nonce, then reads claims. Groups and email + come from the ID token, falling back to the UserInfo endpoint (Authelia serves + these from UserInfo by default). + - The user identity (**subject**) is the email, or the preferred username if no + email is present. +- **Authorization gate:** the user MUST be a member of `REQUIRED_GROUP` (exact, + case-sensitive match). Otherwise the callback returns **403**. +- On success a `hsa_session` cookie is set for **12 hours**. Cookies are + `HttpOnly`, `SameSite=Lax`, and `Secure` whenever `REDIRECT_URL` is `https://`. + The session is an AES-encrypted token; the key is `SHA-256(SESSION_SECRET)`. +- `GET /logout` clears the session and redirects to `/login`. +- **Public routes** (no session required): `/healthz`, `/static/*`, `/login`, + `/callback`, `/logout`. **Every other route requires a valid session**; missing + or invalid sessions redirect to `/login`. + +--- + +## 3. Configuration + +All runtime config is read from environment variables (a local `.env` is loaded +best-effort on startup; see [.env.example](.env.example)). Required vars (startup +fails if any is missing): `ISSUER_URL`, `OIDC_CLIENT_ID`, `OIDC_CLIENT_SECRET`, +`REDIRECT_URL`, `REQUIRED_GROUP`, `SESSION_SECRET`. + +| Var | Default | Meaning | +|---|---|---| +| `LISTEN_ADDR` | `:8080` | HTTP listen address | +| `DB_PATH` | `./data/hsa.db` | SQLite database file | +| `STORAGE_DIR` | `./data` | storage **root** (receipts/attachments live under it) | +| `MAX_UPLOAD_MB` | `32` | per-file upload cap (reject larger) | +| `CONFIG_PATH` | `./config.json` | people + categories catalog | +| `CLAUDE_API_KEY` | _(empty)_ | Anthropic key; empty **disables** AI classification | +| `CLASSIFY_MODEL` | `claude-haiku-4-5-20251001` | classification model id | +| `BACKUP_DIR` | `./data/dbbackup` | scheduled metadata-only backups | +| `BACKUP_INTERVAL` | `168h` | backup frequency (Go duration); `0` disables | +| `BACKUP_KEEP` | `8` | most-recent backups to retain | + +**Catalog (`config.json`):** defines `persons` (last/first/middle) and +`categories` (name + authored `examples`). On startup the app seeds the canonical +person labels (`First Last`) and category names into the DB (insert-if-absent), so +labels renamed via Manage survive restarts. The per-category `examples` are used +only to build the classifier prompt and are intentionally **not** stored in the DB. +A missing/invalid catalog is a fatal startup error. + +--- + +## 4. Data model (SQLite) + +- **categories** — `id`, `label` (unique). Seeded; renamable via Manage. +- **people** — `id`, `label` (unique). Seeded from catalog; renamable via Manage. +- **receipts** — `id` (UUID), `uploaded_by`, `uploaded_at`, `receipt_date`, + `amount_cents` (integer — money is never a float), `category_id` (FK, required), + `person_id` (FK, nullable), `file_path`, `image_data` (BLOB), `file_size_bytes`, + `original_filename`, `mime_type`, `deleted_at` (nullable, soft delete). +- **attachments** — same shape as receipts minus the receipt-specific fields, plus + `receipt_id` (FK). Supplementary files for a receipt; no amount/date/category of + their own. +- **tags** — `id`, `label` (unique, **case-insensitive** via `COLLATE NOCASE`). +- **receipt_tags** — (`receipt_id`, `tag_id`) many-to-many, composite primary key. + +`PRAGMA foreign_keys=ON`, `journal_mode=WAL`, `busy_timeout=5000` are set on open. + +--- + +## 5. Upload & receipt creation + +The upload form (`GET /{$}`) and its handler (`POST /upload`) are the core flow. + +### 5.1 Fields & validation +A submission MUST include: an **amount**, a **date** (`YYYY-MM-DD`), a **category**, +and a **receipt file**. The **who** (person) and **tags** are optional, as are +additional attachment files. + +- **Amount** is parsed to integer cents; accepts an optional `$`, spaces, and + thousands separators; rejects empty, non-numeric, more than two decimals, zero, + and negatives. +- **Category** MUST reference an existing category; **who**, if provided, MUST + reference an existing person. +- **File** MUST be detected as `image/*` or `application/pdf` and be within + `MAX_UPLOAD_MB`. Same allowlist and cap apply to each attachment. +- On any validation error the form re-renders with messages and **nothing is + written**; the user's entered values, including the tag selection, are preserved. + +### 5.2 Image orientation normalization +On upload, a JPEG carrying an EXIF Orientation tag of 2..8 is rotated/flipped +upright, re-encoded (JPEG q≈90), and the tag dropped, so the stored image is +upright everywhere. Images with no tag, an upright tag (1), non-JPEG images, PDFs, +or anything that fails to decode pass through **byte-for-byte**. This applies to +the receipt, every attachment, and the image sent to the classifier. (No +content-based guessing; new uploads only.) + +### 5.3 AI auto-fill (classification) +When `CLAUDE_API_KEY` is set, attaching a file triggers `POST /classify`, which +sends the (orientation-normalized) image to the configured model and returns +suggested amount, date, category, and who to **pre-fill** the form. It mutates no +state — the user reviews and submits normally. + +- Classification is **forced tool-use**, so the result is always structured. The + **person** is constrained to the canonical labels (kept only on exact match); + the **category** is constrained to the configured set, falling back to the last + (most general) category if the model returns something off-list. Date/amount are + dropped if null-ish. +- The user can tick **Skip AI** to suppress the call and fill fields by hand. The + footnote always shows which model runs (or that auto-fill is off). +- After a successful classification the per-call cost is shown in cents (¢), + computed locally from the response's token usage and a hardcoded per-model rate + table. An unknown model id shows token usage but omits the ¢ figure (never a + guess). + +### 5.4 Duplicate warning +Whenever date and amount are both known, the form calls `GET /duplicates` and warns +if a non-deleted receipt already has the **same `receipt_date` and `amount_cents`**. +The warning lists each match (amount, date, category, who, uploader, upload time, +filename) and **disables submit** until the user ticks "Add it anyway". No match → +submit proceeds normally. + +### 5.5 Attachments +The form accepts optional extra files in the same submission (a second page, an +EOB, an itemized list). They inherit the parent receipt's identity, carry no +metadata of their own, and are saved after the receipt row exists. AI runs only on +the primary receipt, never on attachments. + +### 5.6 Tags +The form has an **"Add tags"** control after **Who** that opens an in-page modal +with an alphabetical chip mosaic. The user taps chips to select/deselect and can +create a new tag inline (auto-selected for this receipt). Selected tags submit with +the form. + +- The tag catalog is **shared/global** and **free-form**. +- A new tag is written to the catalog **only when the receipt is actually saved** + (create-if-missing by label), so abandoned uploads add nothing. Labels are + trimmed and matched/deduped **case-insensitively** (first-seen casing kept). +- A receipt's tags are shown on the confirmation page and in the recent lists. +- The classifier does **not** suggest tags — tagging is a manual act. + +### 5.7 Storage (dual-write & file layout) +Each receipt and attachment is written in **both** places: +1. **Filesystem**, under the storage root with a browsable, dated, amount-tagged + name: + - receipts: `/receipts//_
_.` + - attachments: `/attachments//_
_._att` + - Year/month/day and amount come from the **receipt** date/amount (attachments + use their parent's), so an expense's files sort together. Path components + derive only from date + amount (no user filename → no path-traversal surface). + - Name collisions get a numeric suffix on the stem (`_1`, `_2`, …) via exclusive + create (`O_CREATE|O_EXCL`), so concurrent uploads never overwrite. +2. **DB blob** (`image_data`), so the single `.db` file is a complete, portable + dataset (metadata + all images). + +`original_filename` and `mime_type` are kept as metadata. The file is written +before the DB row, so a crash in between can leave a harmless orphan file (never a +row missing its data); orphans are not auto-cleaned. The DB blob is the source of +truth for serving. + +After a successful save, a confirmation page shows category, who, amount, date, +filename, attachment count, and tags, with links to add another / Manage / Export. + +--- + +## 6. Viewing & serving files + +- `GET /receipt/{id}/file` and `GET /attachment/{id}/file` serve the stored bytes + **from the DB blob** (so serving works even if the on-disk copy is gone), with + the original MIME type and `Content-Disposition: inline`. + +--- + +## 7. Recent lists + +Two read-only listings of non-deleted receipts, 10 per page with a "Load next 10" +control (offset paging): + +- `GET /recent` — ordered by **upload date** (`uploaded_at` desc). +- `GET /recent/receipts` — ordered by **receipt date** (`receipt_date` desc). + +Each row shows: amount (linking to the file), receipt date, category, the **who** +abbreviated to initials + last name (e.g. "Jean-Michel Tremblay" → "JM. Tremblay"; +each hyphenated first-name part is initialed), the upload date, paperclip links for +any attachments, and the receipt's tag chips. + +--- + +## 8. Tally + +`GET /tally` — a read-only person × year matrix of summed amounts (dollars): + +- One row per person, one column per year that has data; receipts with no who are + grouped under **"Unassigned"** (sorted last). +- Right-margin column: total per person across years. Bottom-margin row: grand + total per year. Bottom-right: overall grand total. +- Excludes soft-deleted rows. Years are bucketed by the **receipt** date. + +--- + +## 9. Manage lists + +`GET /manage` lets authorized users curate the lookup lists: + +- **Categories** and **People**: rename existing entries (`POST + /manage/categories/rename`, `/manage/people/rename`) and add new ones (`POST + /manage/categories`, `/manage/people`). Renames propagate everywhere because + receipts reference these by id. +- On startup, stray partial-name people (e.g. a leftover "Jude") are merged into the + unambiguous canonical person ("Jude Tremblay"), reassigning their receipts first, + so no receipt loses its who. Seeding is idempotent. +- Tags are **not** managed here (see §13 gaps). + +--- + +## 10. Export + +- `GET /export/db` — downloads a consistent point-in-time copy of the SQLite + database produced via `VACUUM INTO` (the live file is never locked/served + directly). Because images are stored as blobs, this single file is the **complete + dataset**. `Content-Type: application/octet-stream`, filename + `hsa-export-YYYY-MM-DD.db`. +- `GET /export/db?blobs=false` — a **metadata-only** copy with image blobs stripped + (≈99% smaller); filename `hsa-export-YYYY-MM-DD-metadata.db`. +- Read-only; mutates no app state. Amounts stay integer cents in the export. + +--- + +## 11. Scheduled backups + +A background goroutine writes **metadata-only** snapshots (blobs stripped from both +receipts and attachments) into `BACKUP_DIR`: + +- Frequency `BACKUP_INTERVAL` (default weekly; `0` disables), retaining the most + recent `BACKUP_KEEP` (default 8). +- Files are named `hsa_sqlite_backup_.db` — **one per day**; a same-day + re-run is a no-op, so frequent restarts never spam the directory. +- Runs once at startup (if the latest backup is due), then on the interval. Failures + are logged, never fatal. Images are recoverable from the on-disk files and the + full `/export/db`; the daily backup recovers metadata. + +--- + +## 12. Durability & crash recovery + +- WAL mode makes the DB self-healing: an interrupted write is rolled forward if + committed, discarded otherwise, bringing the DB up at its last committed state. +- On open the app runs `PRAGMA quick_check`; a crash-interrupted write passes, but + genuine corruption fails fast and the app **refuses to start**, pointing the + operator at `BACKUP_DIR` / a `.db` export to restore. + +--- + +## 13. Deployment & operations + +- The app builds as a fully static binary (`CGO_ENABLED=0`; the SQLite driver is + pure Go), runnable on any `linux/`. +- It runs as a **user systemd service** behind a reverse proxy. Tagged releases are + built and deployed by Forgejo Actions: a release tag `X.Y.Z` is built, tested, + staged into `~/hsa-app/releases/hsa-app-V/`, then **activated** (symlink swap + + service restart); pre-release tags (e.g. `0.0.0a2`) are built and staged only. + See [deploy/INSTALL.md](deploy/INSTALL.md). + +--- + +## 14. Known gaps / not yet implemented + +These are intentionally absent today (the schema or storage layer may support some; +the UI/endpoint does not): + +- **Deleting receipts:** `deleted_at` and a `SoftDelete` storage method exist and + all listings/tally already exclude deleted rows, but **no route or UI triggers a + delete** — there is currently no way to delete a receipt from the app. +- **Editing** existing receipts (fix by deleting and re-adding — once delete exists). +- **Adding attachments or tags to an already-saved receipt** (no edit/detail page). +- **Duplicate same-person highlighting:** the warning matches on date + amount only; + it does not yet weight or highlight a same-person match differently. +- **Tag features:** no tag-based filtering/search, no tally-by-tag, and no + editing/merging/deleting tags in Manage (free-form tags will accumulate; a cleanup + surface is future work). +- **Archive/zip export** (`/export/archive`): not implemented; only `/export/db`. +- **HEIC** images are not decodable in pure Go; browser camera uploads arrive as + JPEG in practice.