tools: Export- und Eingangsordner im Backup-Ziel - Überschneidungsregel nur noch auf <ziel>/git und den eigenen Maschinenordner, Laufzeitprüfung auf Vorfahren in upload collect und export run (#216); 10.1.0-beta.16
CI / verify (push) Successful in 8m17s
CI / pwsh (push) Successful in 2m52s
Release / converters (push) Successful in 26s
Release / release (push) Successful in 37s
Release / wiki-mirror (push) Skipped

Files changed:
- CHANGES.md
- VERSION
- docs/how-to/back-up-to-a-synced-folder.md
- docs/how-to/collect-from-a-synced-folder.md
- docs/how-to/export-to-a-synced-folder.md
- docs/reference/commands/exports.md
- docs/reference/commands/raw-material-and-uploads.md
- docs/reference/configuration.md
- tools/CONTRACT.md
- tools/chemenu/backup.py
- tools/chemenu/collect.py
- tools/chemenu/commands/export_cmd.py
- tools/chemenu/commands/upload_cmd.py
- tools/chemenu/config_shapes.py
- tools/chemenu/page_export.py
- tools/chemenu/tests/test_collect.py
- tools/chemenu/tests/test_export_run.py
- tools/chemenu/upload.py

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
torbenandClaude Sonnet 5.5 committed 2026-10-16 11:19:12 +02:00
1 parent b48a35b224
commit 02bf3b0403
18 files changed
+312 -41

No files matched your search

+1 -1
View File
@@ -113,7 +113,7 @@ tools/wikitool export run work-notes
- Every Markdown file starts with a frontmatter block whose `wikitool_export` line says `kind=pages instance=<origin URL without userinfo, or local> commit=<newest commit touching this page>`, followed by `title`, `type`, `summary`, `modified` where the page has them (`frontmatter: summary`, the default; `minimal` keeps only the marker, `full` the whole frontmatter), then the page text. A file keeps its path below `kb/`. A wikilink to an exported page becomes a relative Markdown link (`links: relative`, the default) or plain text (`links: text`); a link to a page that is not exported is always plain text. Footnote markers stay; code is never rewritten. LF endings, no timestamp: the same pages give the same bytes, and a commit that touches one page rewrites that page's file alone - plus any file whose links change because a page entered or left the selection. The manifest's `commit` is the newest commit touching any selected page; all of them come from one `git log`, however many pages are selected.
- Handler `command`: the Markdown is rendered into a staging directory, the program of `export.commands` runs with no shell, stdin closed and a timeout of 300 s, and receives the Markdown directory and an empty output directory as its last two arguments. On exit 0 the regular files in the output directory (recursively) become the target's content; any other end leaves the target unchanged and prints the end of its stderr. The staging directory is always removed.
- The folder is only ever touched by what it holds in its manifest, `.wikitool-export.json`: the files an earlier run wrote, each with its sha256. A run removes only manifest files that are no longer exported (and folders it created, once empty), rewrites a manifest file that is missing, leaves every other file byte for byte as it was, and writes nothing - not even a modification time - for a file that did not change. A run that changes nothing writes nothing at all.
- Refuses a target, before writing anything into it: a non-empty folder with no manifest (it is somebody's, not an export), a manifest file that was changed in the folder since the last run (naming it), a foreign file where an exported one belongs, an empty selection (it would delete the earlier export), an unknown field in a predicate. A missing folder is created.
- Refuses a target, before writing anything into it: a non-empty folder with no manifest (it is somebody's, not an export), a folder that holds or lies in backup data (a `.chemenu-backup.json` - any machine's, also one this checkout does not know - or a bare git repository), a manifest file that was changed in the folder since the last run (naming it), a foreign file where an exported one belongs, an empty selection (it would delete the earlier export), an unknown field in a predicate. A missing folder is created.
- Refuses everything, before any target: uncommitted changes in `kb/` or `types/` (untracked files included), no git checkout, a page under `kb/` with unreadable frontmatter, an unknown target name, a broken `export` section.
- Output per target: `new` (with the page titles), `changed`, `removed`, the count `unchanged`, or `failed` with the reason. A page that newly enters a selection is reported as new, not gated. Exit 0 when every target ran; exit 1 when any failed - the others were still served.
- A target with `after_publish: true` is also refreshed by every `publish` that committed, `--no-push` included; a failure there is a warning, the commit stands, and this command is the repeat.
@@ -531,6 +531,7 @@ tools/wikitool upload collect onedrive
- Everything a channel delivers lands in `quarantine/<id>/` unchanged, with a manifest that names `channel`, `handler` and `origin`; the submitter is `unknown`. Nothing reaches `incoming/` or `raw/` - that is `upload accept`, behind its gate, and an agent may run `collect` because collecting gives no trust.
- `folder`: only the top level of `path`. A file is taken once its modification time and, where the platform has it, its creation time are older than `settle_seconds`, its extension and size fit, and its name passes the filename rules. It is removed from the folder only after its copy is fully written and the source still has the size and sha256 that were read; if it changed meanwhile, nothing is kept and it stays. A file whose sha256 already waits in the quarantine is reported as a duplicate and removed from the folder, because its content is kept. Subdirectories, hidden files (names starting with `.` or `~$`, or the hidden or system attribute on Windows), symbolic links, other extensions and oversized files are left untouched and reported.
- `command`: the program of `commands.<name>` runs without a shell, with stdin closed, a 300 s timeout and an empty staging directory under `quarantine/.staging/` as its last argument. On exit 0 each regular file at the top of that directory with an allowed extension and size becomes a submission; on anything else nothing is taken and the end of stderr is printed. The staging directory is always removed.
- `folder` never empties backup or export data: a `path` that holds a `.chemenu-backup.json` (a machine's folder of the backup, any machine's) or a `.wikitool-export.json` (an export), is a bare git repository, or lies in one of these, is that channel's error - nothing is read and nothing removed. A `path` above such folders is fine, since only its top level is read.
- Channels are independent: when one fails the others still run, and the exit status is 1 if any channel or file failed. With no pulled channel it prints `nothing to collect` and exits 0.
- Exits 1, before it does anything, while an `mcp-upload/` of an earlier version still holds submissions, and when the `intake` section is malformed.
- Not idempotent: a second run finds the folder emptied.