Record pivot to readpst and real-mailbox validation findings in ADR-0008
This commit is contained in:
@@ -18,7 +18,7 @@ required.
|
||||
to it.
|
||||
- **Live calendar:** abstract behind a `calendar::CalendarSource` trait. Phase 6 ships the PST
|
||||
source. Microsoft Graph (with explicit user consent) and local ICS files are future sources
|
||||
behind the same trait; Graph is the *one* allowed remote call and only for calendar metadata,
|
||||
behind the same trait; Graph is the _one_ allowed remote call and only for calendar metadata,
|
||||
never meeting content, and only with consent.
|
||||
- **Reminders:** action items become **local OS notifications** (Windows toast); optional
|
||||
calendar entry creation is deferred and, where added, stays local/consented.
|
||||
@@ -33,6 +33,63 @@ required.
|
||||
a fallback and treat parse failures as non-fatal; Graph path (future) introduces OAuth + the
|
||||
product's only remote dependency, so it must be strictly opt-in and clearly labeled.
|
||||
|
||||
## Update (2026-07-01) — `readpst` (libpst) instead of the `outlook-pst` crate
|
||||
|
||||
T6.1 implementation revealed `outlook-pst` (the Rust crate this ADR originally named) is a
|
||||
byte-level, "clean room" implementation of the raw MS-PST binary format — B-tree pages,
|
||||
allocation maps, block trailers — with **no MAPI object-model layer**: no `folder.name()` or
|
||||
`message.subject()`, just numeric property IDs and hand-decoded `PropertyValue`s. Reaching
|
||||
"appointment with attendees" would mean hand-implementing named-property resolution
|
||||
(`PidLidAppointmentStartWhole` etc. aren't fixed low prop IDs) and per-message recipient-table
|
||||
parsing against the MS-PST/MS-OXPROPS spec directly, with no test `.pst` fixture available to
|
||||
verify correctness against.
|
||||
|
||||
Decision: shell out to **`readpst`** (the **libpst** project, not libpff — the two are separate,
|
||||
similarly-purposed C libraries) instead. `readpst -S -e -t a -o <dir> <file.pst>` dumps every
|
||||
appointment as its own `.ics` (iCalendar) file; WA parses those with a small hand-rolled RFC 5545
|
||||
VEVENT reader (`calendar::parse_vevents` — SUMMARY/ORGANIZER/DTSTART/DTEND/ATTENDEE/UID only, no
|
||||
timezone database, floating and `Z` times both treated as UTC). This trades a pure-Rust dependency
|
||||
for an external-binary one (`readpst` must be on `PATH` — obtainable via MSYS2, Cygwin, or
|
||||
prebuilt Windows binaries; WA surfaces a clear `CalError::ToolMissing` if it isn't found, no
|
||||
different in kind from Ollama's guided-install treatment, T5.7) in exchange for a **far** smaller,
|
||||
better-tested WA-side surface. Most PST "password protection" is a UI-level gate over a fixed,
|
||||
keyless obfuscation that `readpst` already reverses transparently, not real encryption — the
|
||||
`password` field is accepted for forward-compat but currently unused; genuinely IRM-protected
|
||||
files surface as a non-fatal parse error rather than being supported.
|
||||
|
||||
`CalendarSource::attendees(event_id)` (a second, separate trait method in the original design) was
|
||||
dropped: PST/ICS list attendees inline per-appointment, so there's no live "fetch attendees for
|
||||
event X" round trip against the source to make — `import()` now returns `Vec<ImportedEvent>`
|
||||
(event + its attendees together), and storage persists both.
|
||||
|
||||
## Update (2026-07-01) — validated against a real 7.2GB mailbox
|
||||
|
||||
Ran `readpst` (libpst v0.6.63, a ~2014 Windows build — an older release than current libpst
|
||||
master) against a real, in-use corporate `.pst` and fed every resulting `.ics` file through the
|
||||
real `parse_vevents`. Results: **1738/1738 files produced exactly one valid event each** —
|
||||
subjects, `DTSTART`/`DTEND` (including the `VALUE=DATE-TIME` parameter form, correctly ignored by
|
||||
the parser) all came through cleanly, zero parse failures.
|
||||
|
||||
One real, confirmed gap: **this libpst v0.6.63 build never emits `ORGANIZER` or `ATTENDEE` lines
|
||||
at all** — 0/1738 events had either, including confirmed real meetings ("Denver", location
|
||||
"Microsoft Teams Meeting", `STATUS:TENTATIVE`). `write_schedule_part_data()`'s attendee-writing
|
||||
logic either doesn't exist in this old release or isn't reached; not something WA's parser can
|
||||
compensate for — the data simply never reaches WA. Practical effect: **T6.1/T6.2 deliver real
|
||||
event import (subject/organizer-as-None/start/end/location) today; T6.5's attendee-populated
|
||||
naming dropdown has nothing to populate from PST until this is resolved.** Also found and fixed:
|
||||
this readpst build doesn't support `-8` (force UTF-8 output) at all ("invalid option -- 8") —
|
||||
removed from the invocation; a non-UTF-8-encoded item's `.ics` file is skipped rather than
|
||||
misread (`collect_ics_files` already treats a non-UTF-8-readable file as skippable).
|
||||
|
||||
Also confirmed empirically: `-t a` does mean **appointments**, not attachments, despite this
|
||||
build's own `-h` output literally printing `-t[eajc] ... a = attachment` — the printed usage
|
||||
string is wrong/stale in this build; the real behavior (verified by watching it skip-count
|
||||
non-appointment items per folder, then produce exactly the Calendar folder's real appointments)
|
||||
matches current libpst master's `OTMODE_APPOINTMENT` mapping for `'a'`.
|
||||
|
||||
## Revisit if
|
||||
`outlook-pst` proves insufficient in practice (promote libpff to primary), or users need live
|
||||
Exchange more than PST (prioritize the Graph source).
|
||||
|
||||
A newer libpst build (or a different install path) turns out to emit `ORGANIZER`/`ATTENDEE`
|
||||
correctly — re-test T6.5 once one is available. Otherwise: `readpst` proves insufficient in
|
||||
practice in some other way, or users need live Exchange more than PST (prioritize the Graph
|
||||
source).
|
||||
|
||||
Reference in New Issue
Block a user