Normalize receipt image orientation from EXIF; add changelog (0.0.1)
All checks were successful
Build and Test / build-and-test (push) Successful in 37s

Bake the EXIF Orientation rotation into uploaded JPEG pixels (and strip
the tag) so receipts are upright in every consumer, not just EXIF-aware
viewers. Acts only when orientation is known (tag 2..8); images with no
tag, tag 1, non-JPEG, or PDFs pass through byte-for-byte. Wired into the
receipt, attachments, and AI-classify paths. Documented as spec item 13.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Jean-Michel Tremblay 2026-06-19 20:26:03 -04:00
parent ef2b0665b8
commit 7ba0d5abe7
8 changed files with 320 additions and 2 deletions

20
CHANGELOG.md Normal file
View file

@ -0,0 +1,20 @@
# Changelog
All notable changes to this project are documented here. Versions are git tags;
release tags `X.Y.Z` are built and deployed automatically (pre-release tags such
as `0.0.0a1` are built and staged only).
## [0.0.1] - 2026-06-19
### Added
- Upright receipt images (EXIF orientation normalization). Phone cameras often
store a photo sideways with an EXIF "Orientation" tag that only some viewers
honor, so receipts could appear rotated. On upload the indicated rotation is now
baked into the pixels (and the tag dropped) so the stored image is upright in
every consumer — browser, download, the AI classifier, and any future export.
- Applies only when orientation is actually known (a JPEG with an EXIF tag of
2..8). Images with no tag, an already-upright tag, non-JPEG images, and PDFs
are left byte-for-byte untouched — no guessing from image content.
- Runs on the primary receipt, every additional attachment, and the image sent
to the AI classifier. New uploads only; existing stored receipts are unchanged.
- See spec.md item 13.

4
go.mod
View file

@ -4,7 +4,9 @@ go 1.26.4
require (
github.com/coreos/go-oidc/v3 v3.18.0
github.com/disintegration/imaging v1.6.2
golang.org/x/oauth2 v0.36.0
modernc.org/sqlite v1.52.0
)
require (
@ -14,9 +16,9 @@ require (
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8 // indirect
golang.org/x/sys v0.42.0 // indirect
modernc.org/libc v1.72.3 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
modernc.org/sqlite v1.52.0 // indirect
)

35
go.sum
View file

@ -1,27 +1,62 @@
github.com/coreos/go-oidc/v3 v3.18.0 h1:V9orjXynvu5wiC9SemFTWnG4F45v403aIcjWo0d41+A=
github.com/coreos/go-oidc/v3 v3.18.0/go.mod h1:DYCf24+ncYi+XkIH97GY1+dqoRlbaSI26KVTCI9SrY4=
github.com/disintegration/imaging v1.6.2 h1:w1LecBlG2Lnp8B3jk5zSuNqd7b4DXhcjwek1ei82L+c=
github.com/disintegration/imaging v1.6.2/go.mod h1:44/5580QXChDfwIclfc/PCwrr44amcmDAg8hxG0Ewe4=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/go-jose/go-jose/v4 v4.1.4 h1:moDMcTHmvE6Groj34emNPLs/qtYXRVcd6S7NHbHz3kA=
github.com/go-jose/go-jose/v4 v4.1.4/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8 h1:hVwzHzIUGRjiF7EcUjqNxk3NCfkPxbDKRdnNE1Rpg0U=
golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/mod v0.33.0 h1:tHFzIWbBifEmbwtGz65eaWyGiGZatSrT9prnU8DbVL8=
golang.org/x/mod v0.33.0/go.mod h1:swjeQEj+6r7fODbD2cqrnje9PnziFuw4bmLbBZFrQ5w=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/tools v0.42.0 h1:uNgphsn75Tdz5Ji2q36v/nsFSfR/9BRFvqhGBaJGd5k=
golang.org/x/tools v0.42.0/go.mod h1:Ma6lCIwGZvHK6XtgbswSoWroEkhugApmsXyrUmBhfr0=
modernc.org/cc/v4 v4.28.2 h1:3tQ0lf2ADtoby2EtSP+J7IE2SHwEJdP8ioR59wx7XpY=
modernc.org/cc/v4 v4.28.2/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.34.0 h1:yRLPFZieg532OT4rp4JFNIVcquwalMX26G95WQDqwCQ=
modernc.org/ccgo/v4 v4.34.0/go.mod h1:AS5WYMyBakQ+fhsHhtP8mWB82KTGPkNNJDGfGQCe0/A=
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/gc/v3 v3.1.2 h1:ZtDCnhonXSZexk/AYsegNRV1lJGgaNZJuKjJSWKyEqo=
modernc.org/gc/v3 v3.1.2/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.72.3 h1:ZnDF4tXn4NBXFutMMQC4vtbTFSXhhKzR73fv0beZEAU=
modernc.org/libc v1.72.3/go.mod h1:dn0dZNnnn1clLyvRxLxYExxiKRZIRENOfqQ8XEeg4Qs=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.52.0 h1:p4dhYh2tXZCiyaqHwRVJDjIGKWyXayiQpThxgDzJaxo=
modernc.org/sqlite v1.52.0/go.mod h1:tcNzv5p84E0skkmJn038y+hWJbLQXQqEnQfeh5r2JLM=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=

113
internal/web/orientation.go Normal file
View file

@ -0,0 +1,113 @@
package web
import (
"bytes"
"encoding/binary"
"github.com/disintegration/imaging"
)
// normalizeOrientation returns image bytes that are physically upright.
//
// Phone cameras store a photo in the sensor's native orientation plus an EXIF
// "Orientation" tag telling viewers to rotate it. We bake that rotation into the
// pixels (and drop the tag) so the stored receipt is upright in every consumer —
// the browser, downloads, the AI classifier, a future PDF export — not just ones
// that happen to honor EXIF.
//
// It only acts when the orientation is actually KNOWN: a JPEG carrying an EXIF
// Orientation tag of 2..8. With no tag (the common case — there's no way to know
// which way is up) or an already-upright tag of 1, the original bytes are returned
// verbatim, so we neither guess nor needlessly recompress. PDFs and non-JPEG
// images pass through untouched, as does anything that fails to decode.
func normalizeOrientation(data []byte, mime string) []byte {
if mime != "image/jpeg" {
return data
}
switch exifOrientation(data) {
case 0, 1:
return data // no metadata to act on, or already upright
}
// imaging.AutoOrientation reads the EXIF tag and rotates/flips to upright.
img, err := imaging.Decode(bytes.NewReader(data), imaging.AutoOrientation(true))
if err != nil {
return data // undecodable — keep the original rather than lose it
}
var buf bytes.Buffer
if err := imaging.Encode(&buf, img, imaging.JPEG, imaging.JPEGQuality(90)); err != nil {
return data
}
return buf.Bytes()
}
// exifOrientation returns the EXIF Orientation tag (1..8) of a JPEG, or 0 when the
// image has no such tag. It is a minimal, allocation-free scan: it walks the JPEG
// segment markers to the APP1 (Exif) block and reads tag 0x0112 from IFD0.
func exifOrientation(data []byte) int {
if len(data) < 4 || data[0] != 0xFF || data[1] != 0xD8 { // SOI
return 0
}
i := 2
for i+4 <= len(data) {
if data[i] != 0xFF {
return 0 // not at a marker boundary — give up
}
marker := data[i+1]
if marker == 0xDA || marker == 0xD9 { // SOS (image data begins) or EOI
return 0
}
segLen := int(data[i+2])<<8 | int(data[i+3])
if segLen < 2 {
return 0
}
segStart, segEnd := i+4, i+2+segLen
if segEnd > len(data) {
return 0
}
if marker == 0xE1 { // APP1
if o, ok := orientationFromAPP1(data[segStart:segEnd]); ok {
return o
}
}
i = segEnd
}
return 0
}
// orientationFromAPP1 reads the Orientation tag out of an APP1 segment's payload
// (the "Exif\0\0" header followed by a TIFF block). Returns (0, false) for any
// malformed or non-Exif segment.
func orientationFromAPP1(seg []byte) (int, bool) {
const hdr = "Exif\x00\x00"
if len(seg) < len(hdr)+8 || string(seg[:len(hdr)]) != hdr {
return 0, false
}
tiff := seg[len(hdr):]
var bo binary.ByteOrder
switch {
case tiff[0] == 'I' && tiff[1] == 'I':
bo = binary.LittleEndian
case tiff[0] == 'M' && tiff[1] == 'M':
bo = binary.BigEndian
default:
return 0, false
}
ifdOff := int(bo.Uint32(tiff[4:8]))
if ifdOff < 8 || ifdOff+2 > len(tiff) {
return 0, false
}
count := int(bo.Uint16(tiff[ifdOff : ifdOff+2]))
for k, entry := 0, ifdOff+2; k < count; k, entry = k+1, entry+12 {
if entry+12 > len(tiff) {
return 0, false
}
if bo.Uint16(tiff[entry:entry+2]) == 0x0112 { // Orientation
val := int(bo.Uint16(tiff[entry+8 : entry+10]))
if val >= 1 && val <= 8 {
return val, true
}
return 0, false
}
}
return 0, false
}

View file

@ -0,0 +1,115 @@
package web
import (
"bytes"
"encoding/binary"
"image"
"image/color"
"image/jpeg"
"testing"
"github.com/disintegration/imaging"
)
// jpegBytes encodes a w×h test image as JPEG (no EXIF).
func jpegBytes(t *testing.T, w, h int) []byte {
t.Helper()
img := image.NewRGBA(image.Rect(0, 0, w, h))
for y := 0; y < h; y++ {
for x := 0; x < w; x++ {
img.Set(x, y, color.RGBA{uint8(x * 30), uint8(y * 30), 100, 255})
}
}
var buf bytes.Buffer
if err := jpeg.Encode(&buf, img, &jpeg.Options{Quality: 90}); err != nil {
t.Fatalf("encode jpeg: %v", err)
}
return buf.Bytes()
}
// withOrientation splices a minimal big-endian EXIF APP1 segment carrying the
// given Orientation value into a JPEG (right after the SOI marker).
func withOrientation(t *testing.T, jpg []byte, orientation uint16) []byte {
t.Helper()
tiff := new(bytes.Buffer)
tiff.WriteString("MM") // big-endian
binary.Write(tiff, binary.BigEndian, uint16(0x002A)) // magic
binary.Write(tiff, binary.BigEndian, uint32(8)) // IFD0 at offset 8
binary.Write(tiff, binary.BigEndian, uint16(1)) // one entry
binary.Write(tiff, binary.BigEndian, uint16(0x0112)) // tag: Orientation
binary.Write(tiff, binary.BigEndian, uint16(3)) // type: SHORT
binary.Write(tiff, binary.BigEndian, uint32(1)) // count
binary.Write(tiff, binary.BigEndian, orientation) // value (in high 2 bytes)
binary.Write(tiff, binary.BigEndian, uint16(0)) // value padding
binary.Write(tiff, binary.BigEndian, uint32(0)) // next IFD: none
payload := append([]byte("Exif\x00\x00"), tiff.Bytes()...)
app1 := new(bytes.Buffer)
app1.Write([]byte{0xFF, 0xE1})
binary.Write(app1, binary.BigEndian, uint16(len(payload)+2)) // segment length
app1.Write(payload)
out := make([]byte, 0, len(jpg)+app1.Len())
out = append(out, jpg[:2]...) // SOI
out = append(out, app1.Bytes()...)
out = append(out, jpg[2:]...)
return out
}
func TestExifOrientation(t *testing.T) {
plain := jpegBytes(t, 4, 2)
if got := exifOrientation(plain); got != 0 {
t.Errorf("plain JPEG: orientation = %d, want 0 (no tag)", got)
}
for _, o := range []uint16{1, 3, 6, 8} {
tagged := withOrientation(t, plain, o)
if got := exifOrientation(tagged); got != int(o) {
t.Errorf("tagged orientation %d: got %d", o, got)
}
}
if got := exifOrientation([]byte("not a jpeg")); got != 0 {
t.Errorf("non-JPEG: orientation = %d, want 0", got)
}
}
func TestNormalizeOrientationPassThrough(t *testing.T) {
plain := jpegBytes(t, 4, 2)
// Non-JPEG mime → untouched.
png := []byte("fake png bytes")
if got := normalizeOrientation(png, "image/png"); !bytes.Equal(got, png) {
t.Error("png should pass through unchanged")
}
// JPEG without EXIF → untouched (can't know, don't guess or recompress).
if got := normalizeOrientation(plain, "image/jpeg"); !bytes.Equal(got, plain) {
t.Error("EXIF-less JPEG should pass through byte-for-byte")
}
// JPEG already upright (orientation 1) → untouched.
up := withOrientation(t, plain, 1)
if got := normalizeOrientation(up, "image/jpeg"); !bytes.Equal(got, up) {
t.Error("orientation-1 JPEG should pass through byte-for-byte")
}
}
func TestNormalizeOrientationRotates(t *testing.T) {
// Stored 4×2 (wide) image tagged orientation 6 ("rotate 90° CW to view") must
// come out upright — i.e. tall (2×4) — with the EXIF tag stripped.
wide := jpegBytes(t, 4, 2)
tagged := withOrientation(t, wide, 6)
out := normalizeOrientation(tagged, "image/jpeg")
if bytes.Equal(out, tagged) {
t.Fatal("orientation-6 JPEG should have been re-encoded upright")
}
if got := exifOrientation(out); got != 0 {
t.Errorf("output still carries orientation tag %d; want stripped (0)", got)
}
img, err := imaging.Decode(bytes.NewReader(out))
if err != nil {
t.Fatalf("decode output: %v", err)
}
b := img.Bounds()
if b.Dx() != 2 || b.Dy() != 4 {
t.Errorf("output dims = %dx%d, want 2x4 (rotated upright)", b.Dx(), b.Dy())
}
}

View file

@ -62,6 +62,7 @@ func (s *Server) handleClassify(w http.ResponseWriter, r *http.Request) {
http.Error(w, "only images and PDFs are allowed", http.StatusUnsupportedMediaType)
return
}
data = normalizeOrientation(data, mimeType) // upright image reads better for the AI
sug, err := s.classifier.Classify(r.Context(), time.Now(), data, mimeType)
if err != nil {

View file

@ -139,6 +139,8 @@ func (s *Server) handleUpload(w http.ResponseWriter, r *http.Request) {
mimeType = detectMime(data)
if !allowedMime(mimeType) {
errs = append(errs, "Only images and PDFs are allowed.")
} else {
data = normalizeOrientation(data, mimeType) // bake in EXIF rotation
}
}
@ -264,6 +266,7 @@ func readAttachments(r *http.Request) (out []pendingAttachment, errs []string) {
errs = append(errs, "Attachments must be images or PDFs.")
continue
}
adata = normalizeOrientation(adata, amime) // bake in EXIF rotation
out = append(out, pendingAttachment{
data: adata,
mimeType: amime,

31
spec.md
View file

@ -353,4 +353,33 @@ half-written row, and recover automatically on the next boot — no manual repai
is not auto-cleaned. The DB/blob is the source of truth for serving.
- Recovery from true corruption (beyond crash-consistency) is restore-from-backup:
the metadata-only daily backup (item 11) recovers the records; the full .db
export (/export/db, blobs included) recovers records + images.
export (/export/db, blobs included) recovers records + images.
13. Upright receipt images (EXIF orientation normalization)
Phone cameras often save a photo in the sensor's native orientation plus an EXIF
"Orientation" tag that says "rotate me when displaying". Some viewers honor the
tag and some don't, so a receipt that looks fine in one place shows up sideways in
another (and the bytes are stored verbatim, so the problem follows the file). On
upload, bake the indicated rotation into the pixels so the stored image is upright
for EVERY consumer — browser, download, the AI classifier, a future PDF export —
not just EXIF-aware ones.
- Act only when orientation is actually KNOWN. A JPEG carrying an EXIF
Orientation tag of 2..8 is decoded, rotated/flipped upright, re-encoded
(JPEG, q≈90), and the tag dropped. A tag of 1 (already upright) or NO tag at
all — the common case, where there is genuinely no way to know which way is up
— leaves the original bytes byte-for-byte untouched. We never infer orientation
from image content or guess; an unrotatable image is left as-is, never wrong.
- Scope: JPEG only (where camera orientation tags live in practice). PDFs and
other image types pass through unchanged, as does any file that fails to decode
(kept rather than lost).
- Applies to the primary receipt, every additional attachment (item 10), and the
image sent to the AI classifier (an upright image reads more reliably). It runs
after the MIME allowlist check and before the dual-write, so file_size_bytes and
both stored copies (disk + DB blob) reflect the normalized bytes.
- Client side cannot help: there is no browser/camera API to disable capture
rotation, so this is necessarily a server-side fix.
Out of scope: content-based auto-rotation (detecting "up" without metadata),
rotating PDFs, and backfilling already-stored receipts.