feat(raw): raw derive reads PDF and Outlook MSG; doctor shows each import filter
CI / verify (push) Successful in 6m41s
CI / pwsh (push) Successful in 2m24s
Release / release (push) Successful in 34s
Release / wiki-mirror (push) Skipped

PDF is read with pypdf (page markers, scan warning, encrypted files refused), Outlook .msg with olefile as a container like an .eml, and a .msg attached to a mail is quoted. Both packages are in tools/requirements.txt and imported only when such a file is derived, so a missing one stops that format alone. doctor reports one import-filter line per package-backed format.

Files changed:
- CHANGES.md
- VERSION
- docs/how-to/ingest-a-source.md
- docs/reference/commands/instance-health.md
- docs/reference/commands/raw-material-and-uploads.md
- tools/CONTRACT.md
- tools/README.md
- tools/chemenu/commands/doctor.py
- tools/chemenu/commands/raw_cmd.py
- tools/chemenu/filters/__init__.py
- tools/chemenu/filters/mail.py
- tools/chemenu/filters/msg.py
- tools/chemenu/filters/pdf.py
- tools/chemenu/tests/binary_formats.py
- tools/chemenu/tests/test_raw_derive_binary.py
- tools/requirements.txt

Co-Authored-By: Claude Code <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
torbenandClaude Code committed 2026-10-08 14:02:16 +02:00
1 parent 6659ca5409
commit 0e6a50aa98
16 files changed
+812 -17

No files matched your search

+4 -2
View File
@@ -66,7 +66,9 @@ excepted.
`jsonschema` and `PyYAML` are hard dependencies, not optional extras: schema
validation is the tool's whole safety net, so `cli.py` fails loudly with the
fix rather than degrading silently.
fix rather than degrading silently. `pypdf` and `olefile` are in `requirements.txt` too
but are imported only when a PDF or an Outlook `.msg` is derived, so a missing one stops
that format alone; `doctor` reports each.
## Layout
@@ -112,7 +114,7 @@ tools/
kb_state.py the KB version (.wikitool-kb.json) and the migration chain
corpus_diff.py invariant comparison of kb/ between two revisions
web_capture.py `raw fetch`'s core: fetch a page, decide its charset, derive Markdown-like text from the HTML - standard library only, deterministic
filters/ the import filters behind `raw derive`: a registry (`__init__.py`) and one module per format family (HTML, mail, Office, ZIP), each registering the extensions it reads - standard library, or a package imported lazily so a missing one fails only its own format
filters/ the import filters behind `raw derive`: a registry (`__init__.py`) and one module per format family (HTML, mail, Outlook MSG, Office, PDF, ZIP), each registering the extensions it reads - standard library, or a package imported lazily so a missing one fails only its own format
repo_capture.py `raw capture`/`raw status`'s core: resolve a ref rule, fetch it into a bare cache, select files by glob as blobs, the `_capture.json` manifest - git with the host's credentials, never a prompt; plus the one write back, a single-file commit pushed without force for `export guidelines`
guideline_export.py `export guidelines`' core: select pages by `search` predicates, render them into one deterministic GUIDELINES.md, and recognise an export again (`EXPORT_MARKER`) - with no CLI attached
search/ pluggable search backends, plus service.py - the search core