export: export run writes selected kb/ pages as a derived, read-only version into configured target folders (markdown and command handlers, manifest, after_publish), raw accept also refuses manifest-listed files, doctor checks the section (#214); 10.0.0-beta.13

Files changed: - CHANGES.md - VERSION - docs/how-to/README.md - docs/how-to/back-up-to-a-synced-folder.md - docs/how-to/collect-from-a-synced-folder.md - docs/how-to/export-guidelines.md - docs/how-to/export-to-a-synced-folder.md - docs/reference/commands/README.md - docs/reference/commands/exports.md - docs/reference/commands/git.md - docs/reference/commands/guideline-export.md - docs/reference/commands/instance-health.md - docs/reference/commands/raw-material-and-uploads.md - docs/reference/configuration.md - docs/reference/exit-codes.md - instructions/setup-instance.md - raw/CONTRACT.md - tools/CONTRACT.md - tools/README.md - tools/chemenu/backup.py - tools/chemenu/cli_contract.py - tools/chemenu/commands/doctor.py - tools/chemenu/commands/export_cmd.py - tools/chemenu/commands/git_publish.py - tools/chemenu/commands/raw/_common.py - tools/chemenu/commands/raw/accept.py - tools/chemenu/commands/raw/capture.py - tools/chemenu/config_shapes.py - tools/chemenu/guideline_export.py - tools/chemenu/page_export.py - tools/chemenu/reference_doc.py - tools/chemenu/repo_capture.py - tools/chemenu/tests/test_config_shapes.py - tools/chemenu/tests/test_export_run.py Co-Authored-By: Claude Code <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
2026-10-13 05:53:48 +02:00
1 parent ecefe0a033
commit 99945c4841
@@ -42,7 +42,7 @@ ON FAILURE and NEVER, which address an agent.
  - [Provenance](#provenance)
  - [Raw material and uploads](#raw-material-and-uploads)
  - [Git](#git)
-  - [Guideline export](#guideline-export)
+  - [Exports](#exports)
  - [Workshop runs and session budget](#workshop-runs-and-session-budget)
  - [Types, instructions and docs](#types-instructions-and-docs)
  - [Telemetry](#telemetry)
@@ -140,6 +140,7 @@ publish               write non-idempotent budget:counted              exit:0,1,
 backup init           write idempotent     budget:counted              exit:0,1    Create the backup target's bare repository and make it `origin` when there is none.
 backup run            write idempotent     budget:counted              exit:0,1    Mirror this machine's files that are not in git into `<target>/<machine>/`.
 export guidelines     write idempotent     budget:counted              exit:0,1,42 Render the guideline pages into one generated `GUIDELINES.md`, or push it into the captured repositories that opted in (**Guideline Push Gate**).
+export run            write idempotent     budget:counted              exit:0,1    Write the derived, read-only reading copy of `kb/` into the folders the `export` section configures - the selected pages as Markdown, or converted by a program of this checkout.
 work new              write non-idempotent budget:counted              exit:0,1    Scaffold `work/<runkey>/` for one workshop run.
 work close            write non-idempotent budget:counted              exit:0,1    Delete a finished workshop.
 budget status         read  idempotent     budget:exempt               exit:0      Show the current session's `wikitool` call count and recent command history.
@@ -1671,7 +1672,7 @@ Capture documentation from a git repository into `incoming/<bundle>/`, with a ma
 - `--ref` is a branch name (`main`) or a tag pattern (`v*`, any of `*?[`), which names the newest matching tag by version order. It resolves to one commit - for an annotated tag, the commit it points at.
 - The commit is fetched by ref name, shallowly, into a bare cache repository per URL under `tools/.wikitool_capture/` (gitignored). Files are read from it as blobs, never through a checkout, so they are byte-identical to the repository's - no line-ending conversion, no filter.
 - `--path` globs (repeatable) select files by their path in the repository, as git's `:(glob)` pathspec does: `*`, `?` and `[...]` do not cross `/`; `**` as a whole segment spans any number of directories; a glob without wildcards also takes everything below it as a directory. Files keep their repository paths inside the bundle.
- Never captured, each named in the output with its reason: a file whose first line starts with `<!-- wikitool:export`, a symlink, a submodule, a file over 25 MiB, a path with a segment starting with `.`, a file named `_capture.json`, and a Git LFS pointer.
+- Never captured, each named in the output with its reason: a file whose first line starts with `<!-- wikitool:export` or a leading frontmatter block with a `wikitool_export:` key, a symlink, a submodule, a file over 25 MiB, a path with a segment starting with `.`, a file named `_capture.json`, and a Git LFS pointer.
 - `incoming/<bundle>/_capture.json` records `schema`, `repo`, `ref`, `commit`, `paths`, `captured` (UTC), `fidelity`, `authority` and `files`. It is the only declaration that and how this instance follows the repository.
 - Only `ssh://`, `https://` and the scp form `user@host:path`; a URL with a password or token in it is refused, since the manifest is committed. Git runs with the host's own credentials and configuration, restricted to those protocols, and never prompts: a repository that asks for credentials is reported as unreachable. Each git call times out after 120 s.
 - `--update <raw-bundle>` takes URL, ref rule and globs from that bundle's manifest and writes the current state to `incoming/<same bundle name>/`; `--fidelity`/`--authority` default to the manifest's values, and passing one is the way to correct it.
@@ -1806,7 +1807,7 @@ Promote one or more files, or one folder, from `incoming/` into `raw/`.
 - 0    success
 - 1    raw accept: A file does not exist or is not directly in `incoming/`; two files in one call share a filename; a target path already exists; or a target path would be over the path budget (160 UTF-16 code units below the instance root)
 - 1    raw accept incoming/<folder>: The folder is not directly in `incoming/`, is combined with another argument, `--page` or `--replaces`, holds no file, or holds a hidden entry, a symlink or a special file; a target path is over the budget; or the folder name is already occupied under `raw/`
- 1    A file - given directly, inside a folder, or as `--replaces`/`--replaces-bundle` material - is a guideline export: its first line, after an optional BOM, starts with `<!-- wikitool:export`
+- 1    A file - given directly, inside a folder, or as `--replaces`/`--replaces-bundle` material - is an export of this wiki: it carries the export marker (a first line starting with `<!-- wikitool:export`, or a `wikitool_export:` key in a leading frontmatter block), or it is byte for byte a file in the manifest of a configured `export` target
 - 1    raw accept: `--fidelity`/`--authority` is missing, or names `unknown` or a value outside the schema's enum
 - 1    raw accept: The target name is already occupied anywhere under `raw/` by something the call does not own
 - 1    raw accept: `--page` names an unknown page or one with no `raw_files:` yet, an existing `raw_files:` entry is missing on disk, a file to be moved has more than one owning page, or `--page` would overwrite an already-set `fidelity`/`authority` with a different value
@@ -1821,7 +1822,7 @@ Promote one or more files, or one folder, from `incoming/` into `raw/`.

 - raw accept: A file does not exist or is not directly in `incoming/`; two files in one call share a filename; a target path already exists; or a target path would be over the path budget (160 UTF-16 code units below the instance root) -> Fix the named argument and retry once. A file inside a subdirectory of `incoming/` is accepted with its whole folder (`raw accept incoming/<folder>`) or moved up into `incoming/` first. For a path over the budget, rename the file in `incoming/` to something shorter - the refusal comes before anything moves, so `incoming/` and `raw/` are unchanged
 - raw accept incoming/<folder>: The folder is not directly in `incoming/`, is combined with another argument, `--page` or `--replaces`, holds no file, or holds a hidden entry, a symlink or a special file; a target path is over the budget; or the folder name is already occupied under `raw/` -> Nothing moved. Fix what the message names and retry once. For an occupied name held by a captured bundle, the folder is its new edition: `raw accept incoming/<bundle> --replaces-bundle <raw-bundle>`. Any other occupied name: rename the folder in `incoming/` - such a folder has no replacement form
- A file - given directly, inside a folder, or as `--replaces`/`--replaces-bundle` material - is a guideline export: its first line, after an optional BOM, starts with `<!-- wikitool:export` -> Not fixed by retrying: it is generated from a wiki's `kb/` and never goes back into `raw/`. Remove it from `incoming/`; nothing was moved
+- A file - given directly, inside a folder, or as `--replaces`/`--replaces-bundle` material - is an export of this wiki: it carries the export marker (a first line starting with `<!-- wikitool:export`, or a `wikitool_export:` key in a leading frontmatter block), or it is byte for byte a file in the manifest of a configured `export` target -> Not fixed by retrying: it is generated from a wiki's `kb/` and never goes back into `raw/`. Remove it from `incoming/`; nothing was moved
 - raw accept: `--fidelity`/`--authority` is missing, or names `unknown` or a value outside the schema's enum -> Pass both with a valid value, then retry once
 - raw accept: The target name is already occupied anywhere under `raw/` by something the call does not own -> Not fixed by retrying: the refusal names `--replaces` (same source, new edition) and renaming in `incoming/` (a separate source) as the two routes, and neither is the tool's to pick. Show the message to the user and wait
 - raw accept: `--page` names an unknown page or one with no `raw_files:` yet, an existing `raw_files:` entry is missing on disk, a file to be moved has more than one owning page, or `--page` would overwrite an already-set `fidelity`/`authority` with a different value -> Fix the named argument and retry once; a different capture value on an existing page is a new edition - `--replaces`
@@ -2178,11 +2179,12 @@ Reconcile with `<remote>/<branch>`, then stage all changes, commit, and push.

 - 0    success
 - 0    The machine-data backup after the push failed: a `WARN` line names the reason, and the commit and the push stand
+- 0    The export after the commit failed: a `WARN` line names the target and the reason, and the commit stands
 - 1    git failed - `git add`, `git commit`, `git push`, or the reconcile's automatic rebase (a conflict outside the generated files, an edited `kb/log.md`, or an uncommitted change on a path the incoming commits change)
 - 1    The push target (`--branch`) is not the checked-out branch, or HEAD is detached; the unborn branch of a fresh `git init` is not this case
 - 1    No `--no-push`, and the remote is not configured or cannot be reached; nothing was committed
 - 1    `--yes`/`-y` was passed - the flag does not exist and fails with an explicit error
- 1    The settings files do not pass their check, `remotes.allowed_push_urls` is not a list of strings, or the `backup` section is malformed; nothing was committed
+- 1    The settings files do not pass their check, `remotes.allowed_push_urls` is not a list of strings, the `backup` section is malformed, or the `export` section is malformed while a target sets `after_publish`; nothing was committed
 - 42   The remotes settings file of an earlier version is still there
 - 42   Mass-Update Gate: `--threshold` (default 10) or more counted files would be committed, or the `--confirm` token does not match this changeset
 - 42   Rebase-Review Gate: `<remote>/<branch>` moved and both sides changed the same file; the output lists the upstream commits, the overlapping files and their diff
@@ -2191,11 +2193,12 @@ Reconcile with `<remote>/<branch>`, then stage all changes, commit, and push.
 **ON FAILURE**

 - The machine-data backup after the push failed: a `WARN` line names the reason, and the commit and the push stand -> Show the `WARN` line to the user. Do not publish again - once the cause is fixed, `tools/wikitool backup run` repeats the backup
+- The export after the commit failed: a `WARN` line names the target and the reason, and the commit stands -> Show the `WARN` line to the user. Do not publish again - once the cause is fixed, `tools/wikitool export run` repeats the export
 - git failed - `git add`, `git commit`, `git push`, or the reconcile's automatic rebase (a conflict outside the generated files, an edited `kb/log.md`, or an uncommitted change on a path the incoming commits change) -> Do not retry and do not force - report and ask the user. `publish` has already made its one retry of a rejected push itself, where a reconcile resolved the rejection
 - The push target (`--branch`) is not the checked-out branch, or HEAD is detached; the unborn branch of a fresh `git init` is not this case -> Check out the branch you mean to publish, or pass `--branch <checked-out branch>`, then retry once
 - No `--no-push`, and the remote is not configured or cannot be reached; nothing was committed -> Show the message to the user and ask whether to commit locally with `--no-push`. Never push by hand - the next `publish` that reaches the remote sends that commit
 - `--yes`/`-y` was passed - the flag does not exist and fails with an explicit error -> Drop it. The Mass-Update Gate is cleared only with `--confirm <token>` from the gate's own refusal output
- The settings files do not pass their check, `remotes.allowed_push_urls` is not a list of strings, or the `backup` section is malformed; nothing was committed -> Show the error to the user and stop - a malformed section is not permission, and fixing it is theirs to do
+- The settings files do not pass their check, `remotes.allowed_push_urls` is not a list of strings, the `backup` section is malformed, or the `export` section is malformed while a target sets `after_publish`; nothing was committed -> Show the error to the user and stop - a malformed section is not permission, and fixing it is theirs to do
 - The remotes settings file of an earlier version is still there -> Show the output to the user verbatim and stop; `tools/wikitool config migrate` moves it, and the user decides when
 - Mass-Update Gate: `--threshold` (default 10) or more counted files would be committed, or the `--confirm` token does not match this changeset -> Show the user the command's full output verbatim and stop. Once they have approved it, run the re-run line the output prints, which carries `--confirm <token>`. Without that token, or with a wrong, invented or superseded one, it exits 42 again with the current state
 - Rebase-Review Gate: `<remote>/<branch>` moved and both sides changed the same file; the output lists the upstream commits, the overlapping files and their diff -> Show the user the command's full output verbatim and stop. Once they have approved it, run the re-run line the output prints, which carries `--confirm-rebase <token>`. Without that token, or with a wrong, invented or superseded one, it exits 42 again with the current state
@@ -2225,6 +2228,7 @@ Reconcile with `<remote>/<branch>`, then stage all changes, commit, and push.
 - `--path` (repeatable) scopes the whole operation - gate count, staging and commit - to that subtree, with one exception: uncommitted generated files outside `--path` (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md` under `kb/`) are staged and committed too, and a line names them - whether a reconcile or the change itself wrote them. They stay out of the gate count, and the `--confirm` token and the re-run line cover the `--path` given. Every other file outside `--path` is left as it is.
 - Commit message: `--message`, then a `Files changed:` paragraph listing every committed path. When `--message` ends in a paragraph git reads as a trailer block (`Co-Authored-By:` and the like), that block stays the last paragraph and the list goes in front of it, since git reads trailers from the last paragraph only. `git interpret-trailers` decides whether there is such a block, and the list moves only when git reads the same trailers from the result; otherwise it is appended at the end. `--message` is not part of any gate token.
 - Machine-data backup: with a `backup` section in `.wikitool-local.json` whose `machine_data` is true, a call that pushed then mirrors this machine's files that are not in git into `<target>/<machine>/` - the same step as `backup run`. A failure there prints a `WARN` line with the reason and still exits 0, since the commit and the push stand. No backup with `--no-push`, after a failed push, or when there was nothing to push. A malformed `backup` section stops the call with exit 1 before anything is committed, `--no-push` included.
+- Export after publish: every target of the `export` section with `after_publish: true` is refreshed once a commit was made - the same step as `export run`, `--no-push` included, since the export reads the committed `kb/`. A failure there prints a `WARN` line with the reason and still exits 0, since the commit stands; `export run` repeats it. A malformed `export` section stops the call with exit 1 before anything is committed, but only when some target sets `after_publish`.
 - After a successful commit or push whose changed files include `tools/`, `types/`, `instructions/`, `AGENTS.md` or a path ending in `CONTRACT.md`, prints one reminder line: the phase past this point (an issue-body rewrite, `docs/` staleness, a changelog entry's accuracy) is not covered by `docs verify`, `instructions verify` or `pytest`. It is not a gate: no exit code change, nothing to clear, and silent for an ordinary content publish.

 **SEE ALSO**
@@ -2233,6 +2237,7 @@ Reconcile with `<remote>/<branch>`, then stage all changes, commit, and push.
 - `instructions/gates.md` - the gate procedure
 - `instructions/setup-instance.md` - the first publish of a new instance
 - `wikitool backup run` - the machine-data backup on its own
+- `wikitool export run` - the page export on its own

 #### `backup init`

@@ -2337,7 +2342,7 @@ Mirror this machine's files that are not in git into `<target>/<machine>/`.
 - `wikitool doctor` - the `backup` line reports the age of the last run
 - `docs/how-to/back-up-to-a-synced-folder.md` - the whole procedure, restoring included

-### Guideline export
+### Exports

 #### `export guidelines`

@@ -2405,6 +2410,63 @@ Render the guideline pages into one generated `GUIDELINES.md`, or push it into t
 - `wikitool raw capture` - how a repository becomes a target
 - `instructions/gates.md` - the gate procedure

+#### `export run`
+
+Write the derived, read-only reading copy of `kb/` into the folders the `export` section configures - the selected pages as Markdown, or converted by a program of this checkout.
+
+**SYNOPSIS**
+
+- `wikitool export run [<target> ...]` - Every configured target, or the named ones. Needs an `export` section in `.wikitool-local.json`; the user writes it by hand.
+
+**PROPERTIES**
+
+- effect: write
+- idempotent: yes
+- atomic: Per file within a target (a temporary file in the target, then a move over its place), the manifest last; targets are independent, so a failed target does not undo another
+- budget: counted
+- network: no
+
+**EXAMPLES**
+
+- `tools/wikitool export run`
+- `tools/wikitool export run work-notes`
+
+**EXIT STATUS**
+
+- 0    success
+- 1    No `export` section, an unknown target name, or a broken section (a predicate that does not parse, a path inside the instance or in the way of another target, an `intake` folder or the backup target)
+- 1    `kb/` or `types/` has uncommitted changes, the instance is not a git checkout, or a page under `kb/` has unreadable frontmatter
+- 1    A target failed: a foreign folder, a file changed in the target, an empty selection, an unknown field, a converter that did not exit 0
+
+**ON FAILURE**
+
+- No `export` section, an unknown target name, or a broken section (a predicate that does not parse, a path inside the instance or in the way of another target, an `intake` folder or the backup target) -> Show the message to the user; the section is theirs to write by hand, never yours
+- `kb/` or `types/` has uncommitted changes, the instance is not a git checkout, or a page under `kb/` has unreadable frontmatter -> Fix the named pages, or publish the changes first, then retry once
+- A target failed: a foreign folder, a file changed in the target, an empty selection, an unknown field, a converter that did not exit 0 -> Show the target's reason to the user; the other targets were still served. Never delete or edit files in a target folder to make a run pass
+
+**NEVER**
+
+- Never edit a file in a target folder to change an export - the next run refuses or overwrites it; change the page, or the target's `include`, instead.
+- Never put an export folder into `incoming/` or `raw/`.
+
+**NOTES**
+
+- An export is derived: regenerated from `kb/` on every run, never edited, never read back into the wiki. Each target names its pages with an allowlist of `search --field` predicates (`include`: a list of selections, AND within one, OR across them), a handler (`markdown`, or `command` for the operator's converter) and the folder (`path`, outside the instance). Nothing is exported that no selection names.
+- Every Markdown file starts with a frontmatter block whose `wikitool_export` line says `kind=pages instance=<origin URL without userinfo, or local> commit=<newest commit touching a selected page>`, followed by `title`, `type`, `summary`, `modified` where the page has them (`frontmatter: summary`, the default; `minimal` keeps only the marker, `full` the whole frontmatter), then the page text. A file keeps its path below `kb/`. A wikilink to an exported page becomes a relative Markdown link (`links: relative`, the default) or plain text (`links: text`); a link to a page that is not exported is always plain text. Footnote markers stay; code is never rewritten. LF endings, no timestamp: the same pages give the same bytes.
+- Handler `command`: the Markdown is rendered into a staging directory, the program of `export.commands` runs with no shell, stdin closed and a timeout of 300 s, and receives the Markdown directory and an empty output directory as its last two arguments. On exit 0 the regular files in the output directory (recursively) become the target's content; any other end leaves the target unchanged and prints the end of its stderr. The staging directory is always removed.
+- The folder is only ever touched by what it holds in its manifest, `.wikitool-export.json`: the files an earlier run wrote, each with its sha256. A run removes only manifest files that are no longer exported (and folders it created, once empty), rewrites a manifest file that is missing, leaves every other file byte for byte as it was, and writes nothing - not even a modification time - for a file that did not change. A run that changes nothing writes nothing at all.
+- Refuses a target, before writing anything into it: a non-empty folder with no manifest (it is somebody's, not an export), a manifest file that was changed in the folder since the last run (naming it), a foreign file where an exported one belongs, an empty selection (it would delete the earlier export), an unknown field in a predicate. A missing folder is created.
+- Refuses everything, before any target: uncommitted changes in `kb/` or `types/` (untracked files included), no git checkout, a page under `kb/` with unreadable frontmatter, an unknown target name, a broken `export` section.
+- Output per target: `new` (with the page titles), `changed`, `removed`, the count `unchanged`, or `failed` with the reason. A page that newly enters a selection is reported as new, not gated. Exit 0 when every target ran; exit 1 when any failed - the others were still served.
+- A target with `after_publish: true` is also refreshed by every `publish` that committed, `--no-push` included; a failure there is a warning, the commit stands, and this command is the repeat.
+- `raw accept` refuses a file whose sha256 a target's manifest lists, and one that carries the marker: an export never goes back into `raw/`.
+
+**SEE ALSO**
+
+- `wikitool search` - the same predicates, to preview a selection
+- `wikitool backup run` - the same kind of folder, for this machine's own files
+- `docs/how-to/export-to-a-synced-folder.md`
+
 ### Workshop runs and session budget

 #### `work new`
@@ -4073,6 +4135,7 @@ Check that this instance is correctly configured.
 - Legacy settings (`legacy-settings`): one `FAIL` for each per-feature settings file of an earlier version (telemetry, remotes, upload, task tracker) that is still there. Nothing reads them any more, so the settings they hold are not in effect; the fix is `tools/wikitool config migrate`.
 - Synced folder (`sync-folder`): a `WARN` when the git directory, links and junctions resolved, lies under a folder named by `OneDrive`, `OneDriveCommercial` or `OneDriveConsumer` (compared without regard to case on Windows) or under `~/Library/CloudStorage/` (macOS); `OK` otherwise, never a `FAIL`.
 - Backup (`backup`): the `backup` section absent is `OK`, malformed is a `FAIL`. Configured, a `WARN` when `<target>/git` is not a bare repository, `origin` does not push there, or - with `machine_data` true - no machine-data backup exists yet; otherwise `OK` with the age of the last machine-data backup.
+- Page export (`export`): the `export` section absent is `OK`, malformed is a `FAIL`; otherwise `OK` listing each target with its handler, folder and whether it runs after `publish`.
 - Intake channels (`upload-intake`): the channels of the `intake` section with their handler, and how many submissions wait in `quarantine/`. Absent is `OK` and means no channel - no `submit` tool, nothing to collect; malformed is a `FAIL`, and so is an `mcp-upload/` of an earlier version that still holds submissions (the message names the command that moves it). A `folder` channel whose `path` does not exist is a `WARN`.
 - Task tracker: the `tasks` section present, absent or malformed - absent is `OK` (no tracker configured), malformed is a `FAIL`. When `WIKITOOL_TASKS_CONFIG` is set, the profile it names is read instead of the `tasks`, `superproductivity` and `caldav` sections and the finding names it; a set variable that names no file is a `FAIL`, never `OK`.
 - For a configured `superproductivity` provider, the configured `access` path's own state: `access: "api"` reports whether its local REST API answers `GET /health` with a ready renderer right now, `access: "snapshot"` whether a backup file is ready. The other access path is never attempted, and neither state is ever a `FAIL`.