Record pivot to readpst and real-mailbox validation findings in ADR-0008

This commit is contained in:
iamdoubz
2026-07-01 15:07:12 -05:00
parent 59a098f471
commit 1efd8bf7dd
+60 -3
View File
@@ -18,7 +18,7 @@ required.
to it.
- **Live calendar:** abstract behind a `calendar::CalendarSource` trait. Phase 6 ships the PST
source. Microsoft Graph (with explicit user consent) and local ICS files are future sources
behind the same trait; Graph is the *one* allowed remote call and only for calendar metadata,
behind the same trait; Graph is the _one_ allowed remote call and only for calendar metadata,
never meeting content, and only with consent.
- **Reminders:** action items become **local OS notifications** (Windows toast); optional
calendar entry creation is deferred and, where added, stays local/consented.
@@ -33,6 +33,63 @@ required.
a fallback and treat parse failures as non-fatal; Graph path (future) introduces OAuth + the
product's only remote dependency, so it must be strictly opt-in and clearly labeled.
## Update (2026-07-01) — `readpst` (libpst) instead of the `outlook-pst` crate
T6.1 implementation revealed `outlook-pst` (the Rust crate this ADR originally named) is a
byte-level, "clean room" implementation of the raw MS-PST binary format — B-tree pages,
allocation maps, block trailers — with **no MAPI object-model layer**: no `folder.name()` or
`message.subject()`, just numeric property IDs and hand-decoded `PropertyValue`s. Reaching
"appointment with attendees" would mean hand-implementing named-property resolution
(`PidLidAppointmentStartWhole` etc. aren't fixed low prop IDs) and per-message recipient-table
parsing against the MS-PST/MS-OXPROPS spec directly, with no test `.pst` fixture available to
verify correctness against.
Decision: shell out to **`readpst`** (the **libpst** project, not libpff — the two are separate,
similarly-purposed C libraries) instead. `readpst -S -e -t a -o <dir> <file.pst>` dumps every
appointment as its own `.ics` (iCalendar) file; WA parses those with a small hand-rolled RFC 5545
VEVENT reader (`calendar::parse_vevents` — SUMMARY/ORGANIZER/DTSTART/DTEND/ATTENDEE/UID only, no
timezone database, floating and `Z` times both treated as UTC). This trades a pure-Rust dependency
for an external-binary one (`readpst` must be on `PATH` — obtainable via MSYS2, Cygwin, or
prebuilt Windows binaries; WA surfaces a clear `CalError::ToolMissing` if it isn't found, no
different in kind from Ollama's guided-install treatment, T5.7) in exchange for a **far** smaller,
better-tested WA-side surface. Most PST "password protection" is a UI-level gate over a fixed,
keyless obfuscation that `readpst` already reverses transparently, not real encryption — the
`password` field is accepted for forward-compat but currently unused; genuinely IRM-protected
files surface as a non-fatal parse error rather than being supported.
`CalendarSource::attendees(event_id)` (a second, separate trait method in the original design) was
dropped: PST/ICS list attendees inline per-appointment, so there's no live "fetch attendees for
event X" round trip against the source to make — `import()` now returns `Vec<ImportedEvent>`
(event + its attendees together), and storage persists both.
## Update (2026-07-01) — validated against a real 7.2GB mailbox
Ran `readpst` (libpst v0.6.63, a ~2014 Windows build — an older release than current libpst
master) against a real, in-use corporate `.pst` and fed every resulting `.ics` file through the
real `parse_vevents`. Results: **1738/1738 files produced exactly one valid event each** —
subjects, `DTSTART`/`DTEND` (including the `VALUE=DATE-TIME` parameter form, correctly ignored by
the parser) all came through cleanly, zero parse failures.
One real, confirmed gap: **this libpst v0.6.63 build never emits `ORGANIZER` or `ATTENDEE` lines
at all** — 0/1738 events had either, including confirmed real meetings ("Denver", location
"Microsoft Teams Meeting", `STATUS:TENTATIVE`). `write_schedule_part_data()`'s attendee-writing
logic either doesn't exist in this old release or isn't reached; not something WA's parser can
compensate for — the data simply never reaches WA. Practical effect: **T6.1/T6.2 deliver real
event import (subject/organizer-as-None/start/end/location) today; T6.5's attendee-populated
naming dropdown has nothing to populate from PST until this is resolved.** Also found and fixed:
this readpst build doesn't support `-8` (force UTF-8 output) at all ("invalid option -- 8") —
removed from the invocation; a non-UTF-8-encoded item's `.ics` file is skipped rather than
misread (`collect_ics_files` already treats a non-UTF-8-readable file as skippable).
Also confirmed empirically: `-t a` does mean **appointments**, not attachments, despite this
build's own `-h` output literally printing `-t[eajc] ... a = attachment` — the printed usage
string is wrong/stale in this build; the real behavior (verified by watching it skip-count
non-appointment items per folder, then produce exactly the Calendar folder's real appointments)
matches current libpst master's `OTMODE_APPOINTMENT` mapping for `'a'`.
## Revisit if
`outlook-pst` proves insufficient in practice (promote libpff to primary), or users need live
Exchange more than PST (prioritize the Graph source).
A newer libpst build (or a different install path) turns out to emit `ORGANIZER`/`ATTENDEE`
correctly — re-test T6.5 once one is available. Otherwise: `readpst` proves insufficient in
practice in some other way, or users need live Exchange more than PST (prioritize the Graph
source).