- Open() now runs PRAGMA quick_check after WAL recovery. WAL mode already makes opening self-healing (committed writes rolled forward, an interrupted write discarded), so a crash mid-write recovers automatically; quick_check fails fast only on genuine corruption, pointing the operator at BACKUP_DIR. - spec item 12 documents the durability/recovery model and the orphan-file caveat. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
356 lines
No EOL
18 KiB
Markdown
356 lines
No EOL
18 KiB
Markdown
HSA Receipt Tracker — Requirements (v1)
|
||
Purpose
|
||
Capture and archive HSA-eligible receipts for future reimbursement and tax substantiation. No parsing, no OCR, no reporting.
|
||
Users
|
||
|
||
Two users, both with full access (shared visibility).
|
||
Authentication via Authelia OIDC.
|
||
Authorization via membership in an Authelia group (e.g. hsa-users).
|
||
Not in the group → 403. No other roles or gradations.
|
||
|
||
Core flow
|
||
|
||
User opens app on phone (mobile-first UI; camera access matters).
|
||
Takes a photo of a receipt, or selects an existing image / PDF from device.
|
||
Form prompts for: amount, date, category.
|
||
Submit → image stored to disk, metadata row inserted into DB.
|
||
Confirmation page, with options to view list or add another.
|
||
|
||
Data model
|
||
receipts table:
|
||
|
||
id (UUID)
|
||
uploaded_by (Authelia username or email)
|
||
uploaded_at (server timestamp)
|
||
receipt_date (user-supplied date on the receipt)
|
||
amount_cents (integer — never store money as float)
|
||
category (enum)
|
||
file_path (relative path on disk — the filesystem copy)
|
||
image_data (BLOB — the receipt file bytes, also stored in the DB itself)
|
||
file_size_bytes (integer — convenience for listings/exports)
|
||
original_filename (preserved for reference)
|
||
mime_type
|
||
deleted_at (nullable — soft delete)
|
||
|
||
Categories (fixed list, hardcoded for v1):
|
||
|
||
Medical
|
||
Dental
|
||
Vision
|
||
Pharmacy
|
||
Other
|
||
|
||
Storage
|
||
|
||
Receipt images/PDFs are stored in BOTH places on upload:
|
||
1. Filesystem at a configurable path, filename randomized (UUID) on save
|
||
(file_path) — used as the primary path for serving.
|
||
2. As a BLOB inside the SQLite database (image_data column) — so the single
|
||
.db file is a complete, self-contained dataset (metadata + files).
|
||
|
||
Rationale: the filesystem copy keeps serving simple/efficient; the DB blob makes
|
||
backup and export trivial ("hand over one file" gets everything, even without the
|
||
files directory). Written once at upload; no edit in v1, so the two copies never
|
||
diverge. Acceptable cost because scale is tiny (two users, small files).
|
||
original_filename and mime_type kept as metadata for download/serving.
|
||
Backups remain JM's responsibility outside the app.
|
||
|
||
Auth integration
|
||
|
||
OIDC with PKCE against https://auth.jmopines.com.
|
||
Session cookie after successful callback.
|
||
/login, /callback, /logout, /healthz are public; everything else requires a valid session.
|
||
Group claim (hsa-users) gates access; otherwise 403.
|
||
|
||
Deployment
|
||
|
||
Runs as a systemd service in an LXC.
|
||
Caddy reverse proxy at https://hsa.jmopines.com (TBD: maisym.com vs jmopines.com).
|
||
SQLite DB + filesystem storage. No external dependencies (no Redis, no Postgres, no S3).
|
||
|
||
Operations
|
||
|
||
Soft delete supported (set deleted_at, hide from default list views).
|
||
No edit functionality in v1 — fix mistakes by deleting and re-adding.
|
||
|
||
Database export
|
||
|
||
Authenticated users (hsa-users group) can download the data for offline use.
|
||
|
||
Two endpoints:
|
||
|
||
GET /export/db — downloads a consistent copy of the SQLite database file. Because
|
||
images are stored as BLOBs in the DB, this single file IS the complete dataset
|
||
(metadata + all receipt images). This is the primary export.
|
||
- Must NOT serve the live DB file directly (avoids locking/corruption against the
|
||
running app). Use SQLite's online backup API (or VACUUM INTO a temp file) to
|
||
produce a point-in-time snapshot, then stream that.
|
||
- Content-Type: application/octet-stream; filename like hsa-export-YYYY-MM-DD.db.
|
||
|
||
GET /export/archive (optional convenience) — downloads a zip with the image files
|
||
extracted to normal files (named by original_filename) plus a CSV/JSON of the
|
||
metadata, for someone who wants the pictures as browseable files rather than
|
||
inside a DB.
|
||
- Streamed zip to avoid buffering large archives in memory.
|
||
- filename like hsa-export-YYYY-MM-DD.zip.
|
||
|
||
Notes:
|
||
- Amounts remain integer cents in the export; consumers divide by 100 for dollars.
|
||
- Read-only operation; no app state is mutated.
|
||
|
||
Out of scope for v1
|
||
|
||
OCR / image content parsing
|
||
Reports, totals, dashboards
|
||
CSV / tax-software export
|
||
Reimbursement tracking (paid vs pending status)
|
||
In-place editing of existing receipts
|
||
Multi-tenancy or per-user data isolation
|
||
Notification / reminders
|
||
|
||
================================================================================
|
||
v2 — additions
|
||
================================================================================
|
||
These supersede the v1 "out of scope" entries for OCR-assisted entry (now present
|
||
via the classifier) and "Reports, totals, dashboards" (see Tally below). Same two
|
||
users, same auth model, same storage. Mobile-first still applies.
|
||
|
||
1. Skip-AI toggle on upload
|
||
|
||
A control on the upload form lets the user opt out of AI parsing for the current
|
||
receipt — for receipts they know are too hard to read, or to avoid spending an API
|
||
call on a bad result.
|
||
- Default: AI parsing ON (auto-fill runs when a file is attached).
|
||
- When "skip AI" is selected, NO /classify call is made; the user fills amount,
|
||
date, category, and who by hand.
|
||
- The model footnote stays visible but reads as disabled when skip is selected,
|
||
so the user always knows whether a call will happen and which model it uses.
|
||
- Skipping is per-upload, not a saved preference.
|
||
|
||
2. Duplicate-transaction warning before insert
|
||
|
||
Once a receipt has both a date and a dollar amount (whether typed or AI-filled),
|
||
check the DB for an already-posted receipt that looks like the same transaction,
|
||
and make the user confirm before inserting a possible duplicate.
|
||
- Match: same receipt_date AND same amount_cents, among non-soft-deleted rows.
|
||
When a "who" is set on both, prefer/highlight a same-person match; a match with
|
||
a different or absent person is still shown as a weaker warning.
|
||
- On a match, show the existing receipt's details: date, amount, category, who,
|
||
uploaded_by, uploaded_at, original_filename (and a link to view it).
|
||
- User chooses: Cancel (abort — nothing inserted) or Approve (insert anyway, as a
|
||
deliberate duplicate). No new schema; this is a pre-insert read + confirm step.
|
||
- No match → insert proceeds as today with no extra prompt.
|
||
|
||
3. Tally tab
|
||
|
||
A totals view (read-only). Bucketed by the YEAR of receipt_date.
|
||
- Matrix: one row per person, one column per year that has data; each cell is the
|
||
summed amount for that person in that year.
|
||
- Include an "Unassigned" row for receipts with no "who".
|
||
- Right margin column: grand total per year (all persons).
|
||
- Bottom margin row: grand total per person across all years.
|
||
- Bottom-right cell: overall grand total tracked.
|
||
- Excludes soft-deleted rows. Amounts shown in dollars (cents / 100).
|
||
|
||
4. Recent uploads tab (by upload date)
|
||
|
||
A list of the most recently ADDED receipts, ordered by uploaded_at descending.
|
||
- Show the 10 most recent, with a "Load next 10" control that pages further back
|
||
(offset or cursor based).
|
||
- Each row: receipt_date, amount, category, who, original_filename, and a link to
|
||
view/download. Excludes soft-deleted rows.
|
||
|
||
5. Recent receipts tab (by receipt date)
|
||
|
||
Identical to #4 but ordered by receipt_date descending instead of uploaded_at —
|
||
"newest receipts" rather than "newest uploads". Same 10 + "Load next 10" paging.
|
||
|
||
6. People catalog integrity (bug)
|
||
|
||
The Manage page currently shows partial-name duplicates (e.g. both "Jude" and
|
||
"Jude Tremblay"). Only the canonical full names seeded from config.json
|
||
("First Last", per Person.Label) should exist as people.
|
||
- Remove stray partial entries; keep only the config-seeded canonical labels.
|
||
- Before deleting a partial entry, reassign any receipts that point at it to the
|
||
matching canonical person so no receipt loses its "who".
|
||
- Seeding must be idempotent: re-seeding from config.json must not create a second
|
||
row for a person who already exists under the canonical label.
|
||
|
||
7. Per-parse cost shown in cents (¢)
|
||
|
||
After a receipt is classified, show the cost of that single AI call, in cents,
|
||
using the cent sign (e.g. "0.3¢"). The cost is computed locally from the API
|
||
response — no extra API call needed.
|
||
- The Messages API response includes a usage object (input_tokens,
|
||
output_tokens, and cache_creation/cache_read token counts). The classifier
|
||
should capture these and return them alongside the suggestion.
|
||
- Cost = input_tokens × input_price + output_tokens × output_price, using the
|
||
active model's per-token rates. For the default model (Haiku 4.5): $1 per 1M
|
||
input tokens and $5 per 1M output tokens — i.e. $0.000001/input-token and
|
||
$0.000005/output-token. Cache-read tokens bill at ~0.1× input; treat them at
|
||
the input rate unless we add exact cache pricing later.
|
||
- Display in cents with the ¢ sign next to where the model footnote shows the
|
||
model name, so the user sees both which model ran and what the scan cost.
|
||
|
||
Where the rates come from (this is the only maintenance cost of the feature):
|
||
- Token counts are exact and free — they come straight from the response's
|
||
usage object, no estimation.
|
||
- Per-token PRICES are not available from any API (the Models API exposes
|
||
capabilities, not dollars), so they live as hardcoded constants in a small
|
||
rate table keyed by exact model id:
|
||
haiku-4-5 → $1/1M in, $5/1M out
|
||
opus-4-8 → $5/1M in, $25/1M out
|
||
sonnet-4-6 → $3/1M in, $15/1M out
|
||
- This table is low-maintenance: Anthropic prices a specific model id once and
|
||
ships price changes as NEW model ids, so an existing id's rate does not move.
|
||
A new row is only needed when we adopt a new model — i.e. exactly when we'd be
|
||
changing CLASSIFY_MODEL anyway.
|
||
- Unknown model id (not in the table) → show the token counts but omit the ¢
|
||
figure (or "cost: n/a"), never a guessed number. A stale table degrades
|
||
gracefully instead of lying.
|
||
|
||
(Balance/spend indicator: dropped. The Anthropic API has no remaining-balance
|
||
endpoint, and a cumulative-spend readout was not wanted. Per-query cost above is
|
||
the only cost surface.)
|
||
|
||
9. Human-readable on-disk file layout
|
||
|
||
Replace the flat UUID filenames with a dated, amount-tagged layout under the
|
||
storage root, so the files directory is browsable on its own:
|
||
|
||
<STORAGE_DIR>/<YYYY>/<MM>_<DD>_<dollars>.<cents>.<ext>
|
||
|
||
- Year folder and MM/DD come from the RECEIPT date (not upload date); dollars
|
||
and cents come from amount_cents (cents zero-padded to two digits, dollars not
|
||
padded). Example: a $42.50 JPEG dated 2026-06-08 → 2026/06_08_42.50.jpeg.
|
||
- The amount's decimal dot and the extension dot coexist fine — the extension is
|
||
just the final dot-segment ("jpeg"); the stem is "06_08_42.50". (If that ever
|
||
feels ambiguous, the accepted alternatives are an underscore "06_08_42_50" or
|
||
bare cents "06_08_4250" — pick one and keep it consistent.)
|
||
- All path components derive only from the date and amount (digits, underscores,
|
||
one dot), never from the user-supplied original filename, so there is no path-
|
||
traversal surface. original_filename stays as metadata in the DB.
|
||
- Collisions (same date + amount + ext — legitimately possible since duplicates
|
||
can be approved) get a numeric suffix on the STEM, starting at _1: the second
|
||
file becomes "06_08_42.50_1.jpeg", the third "_2", etc. Use exclusive create
|
||
(O_CREATE|O_EXCL) and increment the suffix on "already exists" so two concurrent
|
||
uploads can't race onto the same name.
|
||
- The chosen relative path is stored in receipts.file_path as today; the dual
|
||
write still also stores the bytes as the DB blob, and serving continues to work
|
||
from the blob regardless of the on-disk name.
|
||
- Applies to NEW uploads only — existing UUID-named files keep their file_path;
|
||
no backfill/rename of historical files in scope.
|
||
- saveFile must create the year subdirectory (MkdirAll) before writing.
|
||
|
||
10. Additional attachments on a receipt
|
||
|
||
When uploading a receipt, the user can also attach one or more EXTRA files in the
|
||
SAME submission (a second page, an itemized list, an EOB, a photo from another
|
||
angle). Attachments are supplementary files for the same HSA expense — they are
|
||
not standalone receipts and carry no amount/date/category/who of their own; they
|
||
inherit the parent receipt's identity (including its date, which drives their
|
||
on-disk name). Adding attachments to an ALREADY-SAVED receipt is not supported yet
|
||
(no edit/detail page) — attachments are captured only at receipt-creation time.
|
||
|
||
Data model — new `attachments` table:
|
||
- id (UUID)
|
||
- receipt_id (FK → receipts.id; the parent expense)
|
||
- uploaded_by (Authelia username/email)
|
||
- uploaded_at (server timestamp)
|
||
- file_path (relative path on disk — see layout below)
|
||
- image_data (BLOB — the bytes, stored in the DB too, same as receipts)
|
||
- file_size_bytes
|
||
- original_filename
|
||
- mime_type
|
||
- deleted_at (nullable — soft delete)
|
||
Index on receipt_id (and on deleted_at) for listing a receipt's live attachments.
|
||
|
||
Storage layout change — split receipts and attachments under the root:
|
||
- Receipts move from <STORAGE_DIR>/<YYYY>/… to <STORAGE_DIR>/receipts/<YYYY>/…
|
||
(the item-9 dated name is unchanged; only the "receipts/" prefix is added).
|
||
- Attachments go to <STORAGE_DIR>/attachments/<YYYY>/…, named from the PARENT
|
||
receipt's stem plus an attachment marker, e.g. a JPEG attached to the
|
||
$42.50 receipt dated 2026-06-08 →
|
||
attachments/2026/06_08_42.50_att.jpeg
|
||
with the same exclusive-create disambiguation as receipts: the second
|
||
attachment becomes _att_1, the third _att_2, etc. (Year/MM/DD/amount come from
|
||
the parent receipt, so an expense's receipt and its attachments sort together.)
|
||
- Same dual-write as receipts: bytes on disk AND as a DB blob, so the single .db
|
||
export stays a complete dataset (receipts + attachments + images).
|
||
- Existing receipts keep their stored file_path verbatim — file_path is the
|
||
source of truth for serving location, so old (ROOT/<YYYY>/…) and new
|
||
(ROOT/receipts/<YYYY>/…) paths coexist with no backfill. (In practice the DB
|
||
was reset, so there are no legacy files to move.)
|
||
|
||
UI and endpoints:
|
||
- The upload form gets an additional, OPTIONAL multi-file input ("Additional
|
||
files", accept images + PDF, multiple). These ride along with the normal
|
||
receipt submission to POST /upload — there is no separate attach endpoint.
|
||
- Submission order: the receipt row is inserted first (so its id exists), then
|
||
each attached file is saved and linked to it. Same MIME allowlist (images +
|
||
PDF) and per-file size cap (MAX_UPLOAD_MB) as the receipt. No AI runs on
|
||
attachments. AI auto-fill still reads only the primary receipt image.
|
||
- The confirm page lists the saved receipt plus the count/filenames of any
|
||
attachments.
|
||
- GET /attachment/{id}/file serves an attachment's bytes from the DB blob (mirror
|
||
of GET /receipt/{id}/file), so the recent list can link them.
|
||
|
||
Lifecycle / interactions:
|
||
- Soft-deleting a receipt also hides its attachments (filter attachments by the
|
||
parent's deleted_at, or soft-delete the children alongside the parent).
|
||
- Attachments never affect Tally, duplicate detection, or the receipt counts —
|
||
those operate on receipts only.
|
||
- Export: attachments ride along automatically in the .db blob export; an archive
|
||
/zip export (if/when added) lists them under their receipt.
|
||
|
||
Out of scope (for now):
|
||
- Adding attachments to an already-saved receipt (would need an edit/detail page).
|
||
- No AI parsing of attachments; no per-attachment metadata beyond the file.
|
||
- No reordering UI beyond upload order.
|
||
|
||
11. Scheduled metadata-only DB backups
|
||
|
||
The app writes a periodic, metadata-only snapshot of the database to a local
|
||
directory so the receipt metadata always has a recent recoverable copy, without an
|
||
external cron job.
|
||
- Built on the existing snapshot primitive (VACUUM INTO, then blob-strip): the
|
||
backup excludes image blobs from BOTH receipts and attachments, so it is tiny.
|
||
The on-disk files and the full-blob export (/export/db) remain the source for
|
||
the images themselves.
|
||
- On by default, weekly. Config: BACKUP_DIR (default ./data/dbbackup),
|
||
BACKUP_INTERVAL (Go duration sets the frequency, default 168h; set 0 to
|
||
disable), BACKUP_KEEP (most-recent copies to retain, default 8).
|
||
- Files are named hsa_sqlite_backup_<YYYY_MM_DD>.db — one per day, sorting
|
||
chronologically; older copies beyond BACKUP_KEEP are pruned. A same-day re-run
|
||
is a no-op (the dated file already exists).
|
||
- Restart-safe: on startup it backs up immediately only if the newest existing
|
||
backup is older than the interval, so frequent restarts don't spam the dir and
|
||
a long gap is covered right away. Runs in a background goroutine for the life
|
||
of the process; failures are logged, never fatal.
|
||
|
||
Storage-root note: STORAGE_DIR is the storage ROOT (not the receipts subdir).
|
||
Receipts live under <STORAGE_DIR>/receipts/<year>/ and attachments under
|
||
<STORAGE_DIR>/attachments/<year>/ (default STORAGE_DIR=./data).
|
||
|
||
12. Crash recovery and durability at boot
|
||
|
||
The database must survive an app crash or kill mid-write without corruption or a
|
||
half-written row, and recover automatically on the next boot — no manual repair.
|
||
- SQLite runs in WAL mode (PRAGMA journal_mode=WAL, set on open). Every write is
|
||
an atomic transaction, so an interrupted write is either fully applied or not at
|
||
all. Opening the file is self-healing: SQLite rolls forward committed
|
||
transactions from the WAL and discards any incomplete tail, bringing the DB up
|
||
at its last committed state. There is no custom "revert" logic — SQLite's own
|
||
recovery is the mechanism, and reimplementing it would be less safe.
|
||
- On open the app runs PRAGMA quick_check to confirm the recovered file is sound.
|
||
A crash-interrupted write passes (WAL recovery already healed it). A failure
|
||
means genuine corruption (e.g. disk failure); the app refuses to start with a
|
||
message pointing at BACKUP_DIR, so the operator restores rather than running on
|
||
a corrupt DB.
|
||
- Dual-write caveat: a receipt's (or attachment's) file is written to disk BEFORE
|
||
its DB row is inserted, so a crash in between can leave an orphan file with no
|
||
row. This is harmless — extra bytes on disk, never a row missing its data — and
|
||
is not auto-cleaned. The DB/blob is the source of truth for serving.
|
||
- Recovery from true corruption (beyond crash-consistency) is restore-from-backup:
|
||
the metadata-only daily backup (item 11) recovers the records; the full .db
|
||
export (/export/db, blobs included) recovers records + images. |