torbenandClaude Code 0e6a50aa98 feat(raw): raw derive reads PDF and Outlook MSG; doctor shows each import filter ...

PDF is read with pypdf (page markers, scan warning, encrypted files refused), Outlook .msg with olefile as a container like an .eml, and a .msg attached to a mail is quoted. Both packages are in tools/requirements.txt and imported only when such a file is derived, so a missing one stops that format alone. doctor reports one import-filter line per package-backed format.

Files changed:
- CHANGES.md
- VERSION
- docs/how-to/ingest-a-source.md
- docs/reference/commands/instance-health.md
- docs/reference/commands/raw-material-and-uploads.md
- tools/CONTRACT.md
- tools/README.md
- tools/chemenu/commands/doctor.py
- tools/chemenu/commands/raw_cmd.py
- tools/chemenu/filters/__init__.py
- tools/chemenu/filters/mail.py
- tools/chemenu/filters/msg.py
- tools/chemenu/filters/pdf.py
- tools/chemenu/tests/binary_formats.py
- tools/chemenu/tests/test_raw_derive_binary.py
- tools/requirements.txt

Co-Authored-By: Claude Code <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2

2026-10-08 14:02:16 +02:00

..

fixtures/sp

fix: demo corpus follows the decided project pages - three states, seed with their items, fixtures re-recorded (#156)

2026-09-30 14:39:32 +02:00

__init__.py

Chemenu 2.1.0 - deterministischer Wissenskompiler

2026-09-01 16:26:14 +02:00

binary_formats.py

feat(raw): raw derive reads PDF and Outlook MSG; doctor shows each import filter

2026-10-08 14:02:16 +02:00

conftest.py

fix(preflight): asset mode retries a move that fails, names the cause and rolls back (#197)

2026-10-07 12:23:13 +02:00

record_sp_fixtures.py

feat: live tracker suite - WIKITOOL_TASKS_CONFIG override, real-tracker tests for Super Productivity and CalDAV, nightly workflow and test image (#156)

2026-09-30 13:23:24 +02:00

sp_headless.py

feat: live tracker suite - WIKITOOL_TASKS_CONFIG override, real-tracker tests for Super Productivity and CalDAV, nightly workflow and test image (#156)

2026-09-30 13:23:24 +02:00

test_api.py

search: Pfad und Titel vollstaendig in der Trefferzeile, Trunkierung wird benannt (schliesst #100)

2026-09-15 21:26:39 +02:00

test_blocks.py

feat: lint reports sections that hold only template placeholders as Unfilled Template Sections (#94)

2026-10-04 17:11:37 +02:00

test_bugreport_launcher.py

fix: tools/bugreport launcher finds a Python 3.8+ itself and never starts a Store alias (#166)

2026-10-02 13:12:19 +02:00

test_bugreport.py

feat: PowerShell 7 preflight and launcher - preflight.ps1, wikitool.ps1, doctor policy and Mark of the Web checks, pwsh CI job (#151, B)

2026-10-01 09:52:05 +02:00

test_caldav.py

tasks: CalDAV provider (Nextcloud Tasks/iOS), review reports unknown values; bump stops pointing at release (#139)

2026-09-25 20:43:53 +02:00

test_cite_cmd.py

fix: comparison and source pages accept the sources: cite add writes; sources may cite sources (#173)

2026-10-04 21:35:24 +02:00

test_cli_contract.py

AGENTS.md § Gates is the one gate list; cli_contract.GATES and docs verify hold it

2026-10-07 10:10:48 +02:00

test_cli.py

feat: export guidelines - the guideline pages as a generated GUIDELINES.md, pushed into the captured repositories behind the Guideline Push Gate (#179)

2026-10-06 06:48:07 +02:00

test_config_shapes.py

feat: the shapes of the four hand-written .wikitool-*.json files live in one record each (config_shapes.py); loaders, doctor and docs/reference/configuration.md read them (#193)

2026-10-07 08:44:50 +02:00

test_conventions.py

stack: Typ project und Collection kb/gtd/ (#123)

2026-09-19 21:13:24 +02:00

test_corpus_cache.py

feat: MCP-Leseserver, Bibliotheksgrenze, Haertung des Lesepfads, Publish-Remote-Gate scharf (2.4.0)

2026-09-02 07:19:32 +02:00

test_corpus_diff.py

stack: Konfidenz-Mechanismus ersatzlos entfernt, Korpus migriert (schliesst #60, #86)

2026-09-10 19:51:48 +02:00

test_dist_cmd.py

feat: docs/ carries the operator's how-to guides, a configuration reference and the architecture and telemetry explanations carved out of README.md, INSTALL.md, INSTALL-MCP.md and EVALS.md; the last two retired (#190)

2026-10-07 05:13:56 +02:00

test_dist_upgrade.py

feat: the test suite no longer ships, and dist upgrade deletes what a release stops shipping (#113)

2026-10-03 20:12:32 +02:00

test_docs_verify.py

AGENTS.md § Gates is the one gate list; cli_contract.GATES and docs verify hold it

2026-10-07 10:10:48 +02:00

test_doctor.py

feat: the shapes of the four hand-written .wikitool-*.json files live in one record each (config_shapes.py); loaders, doctor and docs/reference/configuration.md read them (#193)

2026-10-07 08:44:50 +02:00

test_evals.py

feat: lint reports sections that hold only template placeholders as Unfilled Template Sections (#94)

2026-10-04 17:11:37 +02:00

test_export_guidelines.py

feat: export guidelines - the guideline pages as a generated GUIDELINES.md, pushed into the captured repositories behind the Guideline Push Gate (#179)

2026-10-06 06:48:07 +02:00

test_frontmatter_io.py

stack: Konfidenz-Mechanismus ersatzlos entfernt, Korpus migriert (schliesst #60, #86)

2026-09-10 19:51:48 +02:00

test_git_publish.py

AGENTS.md § Gates is the one gate list; cli_contract.GATES and docs verify hold it

2026-10-07 10:10:48 +02:00

test_hermetic_env.py

fix: Testisolation - kb_dir repointet config.ROOT, lint loest Kollektionen gegen den uebergebenen Baum auf (4.1.1, #44)

2026-09-03 06:33:38 +02:00

test_import_chronicle.py

Chemenu 2.1.0 - deterministischer Wissenskompiler

2026-09-01 16:26:14 +02:00

test_index_build.py

stack: Konfidenz-Mechanismus ersatzlos entfernt, Korpus migriert (schliesst #60, #86)

2026-09-10 19:51:48 +02:00

test_install_doc.py

feat: INSTALL.md is the English installation tutorial; new tutorial docs/tutorials/first-source-to-first-answer.md with a shipped sample source; README.md links it as its first-ingestion step (#191)

2026-10-07 06:38:40 +02:00

test_instructions_cmd.py

feat: stack development in three phases - stack-dev (design), stack-build, stack-close, handed over through tracker states (#168)

2026-10-02 20:33:07 +02:00

test_instructions_shell.py

feat: docs/ carries the operator's how-to guides, a configuration reference and the architecture and telemetry explanations carved out of README.md, INSTALL.md, INSTALL-MCP.md and EVALS.md; the last two retired (#190)

2026-10-07 05:13:56 +02:00

test_kb_collections.py

stack: Typ project und Collection kb/gtd/ (#123)

2026-09-19 21:13:24 +02:00

test_kb_scan.py

fix: a wikilink wrapped across a line break is its own lint finding; rename and rm see it (#115)

2026-10-03 11:19:10 +02:00

test_lint.py

feat: lint reports sections that hold only template placeholders as Unfilled Template Sections (#94)

2026-10-04 17:11:37 +02:00

test_log_append.py

fix: log append reports an unreadable --body-file as ERROR, not a traceback (#145)

2026-09-26 16:03:44 +02:00

test_markdown_code.py

Chemenu 2.1.0 - deterministischer Wissenskompiler

2026-09-01 16:26:14 +02:00

test_mcp_server.py

search: Pfad und Titel vollstaendig in der Trefferzeile, Trunkierung wird benannt (schliesst #100)

2026-09-15 21:26:39 +02:00

test_migrate_cmd.py

feat: wikitool move - eine kb-Seite folgt ihrem Subtype ins berechnete Verzeichnis (#56)

2026-09-04 22:50:22 +02:00

test_new_page.py

feat: people live on their organization's page until promoted; organization subtype, member-of, broken_anchors lint (#172)

2026-10-04 10:51:46 +02:00

test_page_ops.py

fix: comparison and source pages accept the sources: cite add writes; sources may cite sources (#173)

2026-10-04 21:35:24 +02:00

test_pipeline_l0.py

feat: docs/ carries the operator's how-to guides, a configuration reference and the architecture and telemetry explanations carved out of README.md, INSTALL.md, INSTALL-MCP.md and EVALS.md; the last two retired (#190)

2026-10-07 05:13:56 +02:00

test_portability.py

feat(raw): raw derive unpacks ZIP archives and mails with attachments into a folder

2026-10-08 13:04:01 +02:00

test_preflight_pwsh.py

fix(preflight): asset mode retries a move that fails, names the cause and rolls back (#197)

2026-10-07 12:23:13 +02:00

test_preflight.py

fix(preflight): asset mode retries a move that fails, names the cause and rolls back (#197)

2026-10-07 12:23:13 +02:00

test_provenance.py

stack: Konfidenz-Mechanismus ersatzlos entfernt, Korpus migriert (schliesst #60, #86)

2026-09-10 19:51:48 +02:00

test_raw_capture.py

feat: raw capture / raw status / --replaces-bundle - documentation from git repositories as a bundle, with drift reporting (#177)

2026-10-05 11:59:59 +02:00

test_raw_cmd.py

feat: incoming/ as a queue - raw pending picks the next entry, raw accept takes a whole folder, a file in a subdirectory of incoming/ is refused (#112)

2026-10-03 09:49:20 +02:00

test_raw_derive_binary.py

feat(raw): raw derive reads PDF and Outlook MSG; doctor shows each import filter

2026-10-08 14:02:16 +02:00

test_raw_derive.py

feat(raw): raw derive unpacks ZIP archives and mails with attachments into a folder

2026-10-08 13:04:01 +02:00

test_raw_fetch.py

feat: incoming/ as a queue - raw pending picks the next entry, raw accept takes a whole folder, a file in a subdirectory of incoming/ is refused (#112)

2026-10-03 09:49:20 +02:00

test_reference_doc.py

AGENTS.md § Gates is the one gate list; cli_contract.GATES and docs verify hold it

2026-10-07 10:10:48 +02:00

test_review.py

feat: live tracker suite - WIKITOOL_TASKS_CONFIG override, real-tracker tests for Super Productivity and CalDAV, nightly workflow and test image (#156)

2026-09-30 13:23:24 +02:00

test_run_budget.py

feat!: installation only from a release, into an empty folder; upstream merge/verify and private-instance.md removed, dist adopt, shell-neutral instructions (#153)

2026-10-01 22:12:09 +02:00

test_search.py

feat: Windows portability - path separators, LF line endings, UTF-8 decoding and output, msvcrt lock fallback (#152)

2026-10-01 21:15:03 +02:00

test_source_hygiene.py

docs: die Zahl der nachgezogenen Pfadliterale korrigiert (33, nicht 27)

2026-09-17 09:01:50 +02:00

test_sp_recorded.py

fix: demo corpus follows the decided project pages - three states, seed with their items, fixtures re-recorded (#156)

2026-09-30 14:39:32 +02:00

test_superproductivity.py

fix: Super Productivity API path unwraps the {ok, data} envelope, drops the inbox project, health waits for the renderer (#162)

2026-09-30 12:17:39 +02:00

test_task_cmd.py

feat: live tracker suite - WIKITOOL_TASKS_CONFIG override, real-tracker tests for Super Productivity and CalDAV, nightly workflow and test image (#156)

2026-09-30 13:23:24 +02:00

test_tasks_config.py

feat: live tracker suite - WIKITOOL_TASKS_CONFIG override, real-tracker tests for Super Productivity and CalDAV, nightly workflow and test image (#156)

2026-09-30 13:23:24 +02:00

test_tasks_protocol.py

stack: Provider-Schicht für Aufgaben-Tracker mit Super-Productivity-Adapter (#124)

2026-09-19 21:46:56 +02:00

test_telemetry_emit.py

session: Harness-Session-Variable schliesst die Luecke im Session-Id-Fallback (Telemetrie-Join, Iteration-Budget-Gate); SIGPIPE-Nebenbefund im Emitter behoben

2026-09-16 19:18:00 +02:00

test_telemetry_policy.py

stack: Telemetrie-Default nach Installationsform, Byte-Deckel und Session-Retention (schliesst #55)

2026-09-10 23:38:22 +02:00

test_telemetry_scrub.py

Chemenu 2.1.0 - deterministischer Wissenskompiler

2026-09-01 16:26:14 +02:00

test_titles.py

feat: path budget - a file's path stays at 160 characters or fewer; new, rename, move and raw accept refuse more, lint reports Long Paths (#163)

2026-09-30 23:08:36 +02:00

test_toc.py

feat!: docs/ sorted into tutorials, how-to, reference, explanation plus unshipped docs/dev/; explanation pages and developer guide moved, human docs English (#187)

2026-10-06 20:45:09 +02:00

test_touch.py

fix: stale wiki/ path literals nach kb/ nachgezogen, mit Test-Guard gegen die naechste Umbenennung

2026-09-17 08:59:25 +02:00

test_trace_ingest.py

feat: Windows portability - path separators, LF line endings, UTF-8 decoding and output, msvcrt lock fallback (#152)

2026-10-01 21:15:03 +02:00

test_tracker_live.py

feat: live tracker suite - WIKITOOL_TASKS_CONFIG override, real-tracker tests for Super Productivity and CalDAV, nightly workflow and test image (#156)

2026-09-30 13:23:24 +02:00

test_type_resolver.py

fix: comparison and source pages accept the sources: cite add writes; sources may cite sources (#173)

2026-10-04 21:35:24 +02:00

test_types_cmd.py

feat: people live on their organization's page until promoted; organization subtype, member-of, broken_anchors lint (#172)

2026-10-04 10:51:46 +02:00

test_upload_cmd.py

stack: MCP submit-Tool mit Upload Review Gate und Quarantäne-Schreibpfad (schliesst #32)

2026-09-11 09:51:37 +02:00

test_upload.py

stack: MCP submit-Tool mit Upload Review Gate und Quarantäne-Schreibpfad (schliesst #32)

2026-09-11 09:51:37 +02:00

test_util.py

tools: fail() prints ON FAILURE lines on stderr after ERROR (#143)

2026-09-26 14:53:51 +02:00

test_version_cmd.py

feat: changelog entries stay compact - one paragraph per topic, a size budget in docs verify and version release, version bump names required migration documents (#184)

2026-10-06 16:48:02 +02:00

test_wiki_mirror.py

feat(workflow): Gitea-Wiki als generierter Spiegel von docs/ (#192)

2026-10-07 23:10:19 +02:00

test_work_cmd.py

Chemenu 2.1.0 - deterministischer Wissenskompiler

2026-09-01 16:26:14 +02:00

test_xref.py

fix: comparison and source pages accept the sources: cite add writes; sources may cite sources (#173)

2026-10-04 21:35:24 +02:00

tracker_live.py

feat: live tracker suite - WIKITOOL_TASKS_CONFIG override, real-tracker tests for Super Productivity and CalDAV, nightly workflow and test image (#156)

2026-09-30 13:23:24 +02:00