diff --git a/CHANGES.md b/CHANGES.md index 14ae7e7..242208e 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -14,7 +14,7 @@ place only: `instructions/dev/version-parts.md` § The candidate model. --- -## 10.1.0-beta.20 - 2026-10-16 - publish-and-ci: Wartezeit zählt ab freiem Runner, nicht ab dem Push - offene Läufe eines früheren Pushs laufen zuerst, und der Runner hält die Push-Reihenfolge nicht ein +## 10.1.0-beta.21 - 2026-10-16 - Export: include "all" wählt die ganze KB, index: true schreibt index.md und INDEX.md im Ziel **Author:** Torben Nehmer @@ -30,6 +30,7 @@ place only: `instructions/dev/version-parts.md` § The candidate model. - wiki-ingest: ein Lauf endet nach einer Quelle mit Stopp und /clear-Empfehlung - keine Schleife über mehrere Einträge im selben Kontext; capture-session delegiert wiki-ingest nicht mehr an Subagents (#219) - tools/bugreport: eine nach dem Sammeln korrigierte Chronologie wird mit --bundle/--chronology in das vorhandene Bündel getauscht (gleiche Verarbeitung, kein zweiter Lauf, kein zweites Bündel); bug-report.md prüft die Chronologie vor dem Sammeln (#220) - Export- und Eingangsordner dürfen im Backup-Ziel neben git/ und den Maschinenordnern liegen: die Überschneidungsregel gilt nur noch für /git und den eigenen Maschinenordner, upload collect und export run weisen Ordner in oder unter Backup-/Export-Daten fremder Maschinen zur Laufzeit ab +- Export: include "all" wählt die ganze KB, index: true schreibt index.md und INDEX.md im Ziel **Low impact** - instructions/dev: Konverter-Vergleich – eigene Ableitung gegen Referenzwerkzeuge am privaten Korpus, mit Grundlinie diff --git a/VERSION b/VERSION index e5055b1..06227d5 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -10.1.0-beta.20 +10.1.0-beta.21 diff --git a/docs/how-to/export-to-a-synced-folder.md b/docs/how-to/export-to-a-synced-folder.md index 7080a0c..9265e82 100644 --- a/docs/how-to/export-to-a-synced-folder.md +++ b/docs/how-to/export-to-a-synced-folder.md @@ -20,10 +20,17 @@ look like: [configuration reference](../reference/configuration.md#export-to-a-f ## Pick the pages and the format -- **Which pages.** Only what a target's `include` selects is written; there is no "everything". A - selection is a list of `search` predicates that must all hold - `tags=shared`, `kind=system`, - `!draft` - and a page that fits any selection of the target is exported. Try a selection with - `tools/wikitool search --field tags=shared` first. +- **Which pages.** Only what a target's `include` selects is written. `"include": "all"` selects + every page, also one that lies directly under `kb/`, and no generated file. Otherwise a selection + is a list of `search` predicates that must all hold - `tags=shared`, `kind=system`, `!draft` - and + a page that fits any selection of the target is exported: `"include": [["tags=shared"]]`. Try a + selection with `tools/wikitool search --field tags=shared` first. A missing or empty `include` is + an error, not "all". With `"all"`, every page you add later goes into the export too - on a target + with `after_publish` at the next `publish`. +- **An entry point.** The folder holds the pages only. `"index": true` adds `index.md` in the root + of the folder and an `INDEX.md` per collection, so someone who opens the folder finds the + collections, their areas and the pages with their summaries by following links. The files list what + the target exports and nothing else, and the link text is the page's title. - **Which format.** Two ways, equally valid: - `markdown` writes the pages as `.md` files. It needs nothing but the folder. - `command` hands those files to a converter of your own and exports what it produces - Word @@ -69,6 +76,10 @@ look like: [configuration reference](../reference/configuration.md#export-to-a-f (the default) the title, type, summary and modified date, `full` the page's whole frontmatter. `links: relative` (the default) turns a link to an exported page into a relative Markdown link; `links: text` leaves the page's name. A link to a page that is not exported is always plain text. + The index files link relatively whatever `links` says. + + To put the whole of `kb/` into the folder with an entry point, write `"include": "all"` and + `"index": true` instead of the `include` selection above. 3. **Tell the agent:** > Export the pages. @@ -84,8 +95,8 @@ every file names the commit its page comes from. Publish first. ## What a run does to the folder -The folder holds the pages and one file of the stack's own, `.wikitool-export.json`, which lists -what the last run wrote. +The folder holds the pages, the index files if the target asks for them, and one file of the stack's +own, `.wikitool-export.json`, which lists what the last run wrote. - **Only its own files.** A run removes a file only when it is listed and no longer exported, and a folder only when the run created it. Anything else in the folder stays as it is. @@ -102,6 +113,11 @@ what the last run wrote. A file of a page the commit did not touch changes only when its text does: a link to a page that has just entered the selection becomes a relative link, one to a page that left it plain text. The first run after updating to a version that brought this rewrites every file once. +- **An index follows the pages.** A collection's `INDEX.md` changes when one of its pages does, and + `index.md`, which lists them all, with every commit to an exported page. Setting `index` back to + `false` removes the index files at the next run and leaves every other file as it is. A page + `index.md` in a collection, next to the generated `INDEX.md`, stops the target: a synced folder on + Windows would make them one file. - **A growing selection needs no confirmation.** New pages are reported as new; read the list before the sync client uploads them. @@ -150,7 +166,8 @@ program's stderr. The staging directory is always removed. ## What does not come back in An export is never a source. Every Markdown file starts with a frontmatter block whose first key is -`wikitool_export`, and `tools/wikitool raw accept` and `raw capture` refuse a file that carries it. +`wikitool_export` - the index files too - and `tools/wikitool raw accept` and `raw capture` refuse a +file that carries it. A converter may lose that key, so `raw accept` also refuses any file in `incoming/` that is byte for byte one of the files in the manifest of a configured target - and warns, without refusing, when a target has no manifest yet or the `export` section is broken. diff --git a/docs/reference/commands/exports.md b/docs/reference/commands/exports.md index 93537b9..a5404d9 100644 --- a/docs/reference/commands/exports.md +++ b/docs/reference/commands/exports.md @@ -109,7 +109,8 @@ tools/wikitool export run work-notes **Behaviour** -- An export is derived: regenerated from `kb/` on every run, never edited, never read back into the wiki. Each target names its pages with an allowlist of `search --field` predicates (`include`: a list of selections, AND within one, OR across them), a handler (`markdown`, or `command` for the operator's converter) and the folder (`path`, outside the instance). Nothing is exported that no selection names. +- An export is derived: regenerated from `kb/` on every run, never edited, never read back into the wiki. Each target names its pages with `include`: either the string `"all"` (every page `search` knows, also one directly under `kb/`; never a generated or meta file) or an allowlist of `search --field` predicates (a list of selections, AND within one, OR across them), a handler (`markdown`, or `command` for the operator's converter) and the folder (`path`, outside the instance). A missing or empty `include` is an error, never "all": nothing is exported that the target does not name. +- With `index: true` a target also gets `index.md` in its root and an `INDEX.md` per collection (and per area the catalog gives a file of its own), built from the exported pages alone: collections and areas with their counts, then tables of page, type, summary and last modified, with relative Markdown links whatever `links` says. No date of the day and no hint for an agent; the marker is `kind=index` with the newest commit of the pages the file lists, so a commit to one collection rewrites that collection's `INDEX.md` and `index.md` only. The files are ordinary files of the run (manifest, atomic write, removed once no longer generated); the `command` handler receives them with the pages. A page `/index.md` beside the generated `INDEX.md` stops the target. - Every Markdown file starts with a frontmatter block whose `wikitool_export` line says `kind=pages instance= commit=`, followed by `title`, `type`, `summary`, `modified` where the page has them (`frontmatter: summary`, the default; `minimal` keeps only the marker, `full` the whole frontmatter), then the page text. A file keeps its path below `kb/`. A wikilink to an exported page becomes a relative Markdown link (`links: relative`, the default) or plain text (`links: text`); a link to a page that is not exported is always plain text. Footnote markers stay; code is never rewritten. LF endings, no timestamp: the same pages give the same bytes, and a commit that touches one page rewrites that page's file alone - plus any file whose links change because a page entered or left the selection. The manifest's `commit` is the newest commit touching any selected page; all of them come from one `git log`, however many pages are selected. - Handler `command`: the Markdown is rendered into a staging directory, the program of `export.commands` runs with no shell, stdin closed and a timeout of 300 s, and receives the Markdown directory and an empty output directory as its last two arguments. On exit 0 the regular files in the output directory (recursively) become the target's content; any other end leaves the target unchanged and prints the end of its stderr. The staging directory is always removed. - The folder is only ever touched by what it holds in its manifest, `.wikitool-export.json`: the files an earlier run wrote, each with its sha256. A run removes only manifest files that are no longer exported (and folders it created, once empty), rewrites a manifest file that is missing, leaves every other file byte for byte as it was, and writes nothing - not even a modification time - for a file that did not change. A run that changes nothing writes nothing at all. @@ -132,7 +133,7 @@ tools/wikitool export run work-notes - `0` - success - `1` - No `export` section, an unknown target name, or a broken section (a predicate that does not parse, a path inside the instance or in the way of another target, an `intake` folder or a folder the backup writes into: `/git` or this machine's folder) - `1` - `kb/` or `types/` has uncommitted changes, the instance is not a git checkout, or a page under `kb/` has unreadable frontmatter -- `1` - A target failed: a foreign folder, a folder in backup data, a file changed in the target, an empty selection, an unknown field, a converter that did not exit 0 +- `1` - A target failed: a foreign folder, a folder in backup data, a file changed in the target, an empty selection, an unknown field, two files that differ only in case (also a page and a generated index file), a converter that did not exit 0 **See also** diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 6886bfb..46254cd 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -644,7 +644,7 @@ them into a staging directory and hands that to a converter of this checkout. | Field | Tier | Type | Required | Default | Description | |-------|------|------|----------|---------|-------------| -| `targets` | checkout, gate | object | yes | - | Target name (lower-case letters, digits and hyphens) to target; the name is the argument of `export run`. Every target has `handler` (`markdown`: the pages as Markdown files; `command`: the same Markdown run through a converter of this checkout), `path` (an absolute path outside the instance; it must not lie in or contain another target or an `intake` folder, and not be, lie in or contain `/git` or `/`, nor lie in any other machine's folder or a bare repository) and `include` (an array of selections, each an array of `search --field` predicates: all predicates of a selection must hold, and a page that fits any selection is exported; there is no "everything"). Optional: `frontmatter` (`minimal`: only the export marker; `summary`, the default: also `title`, `type`, `summary` and `modified` where the page has them; `full`: the page's whole frontmatter), `links` (`relative`, the default: a wikilink to an exported page becomes a relative Markdown link; `text`: plain text; a link to a page that is not exported is always plain text) and `after_publish` (default false: refresh this target after every `publish` that committed). The handler `command` takes `command` (required: the name of an entry of `commands`). A field the handler does not take is an error. | +| `targets` | checkout, gate | object | yes | - | Target name (lower-case letters, digits and hyphens) to target; the name is the argument of `export run`. Every target has `handler` (`markdown`: the pages as Markdown files; `command`: the same Markdown run through a converter of this checkout), `path` (an absolute path outside the instance; it must not lie in or contain another target or an `intake` folder, and not be, lie in or contain `/git` or `/`, nor lie in any other machine's folder or a bare repository) and `include` (the string `"all"`: every page `search` knows, a page directly under `kb/` included and no generated or meta file; or an array of selections, each an array of `search --field` predicates: all predicates of a selection must hold, and a page that fits any selection is exported. A missing or empty `include` is an error, never "all"). Optional: `frontmatter` (`minimal`: only the export marker; `summary`, the default: also `title`, `type`, `summary` and `modified` where the page has them; `full`: the page's whole frontmatter), `links` (`relative`, the default: a wikilink to an exported page becomes a relative Markdown link; `text`: plain text; a link to a page that is not exported is always plain text), `index` (default false; true: also write `index.md` in the target's root and an `INDEX.md` per collection, and per area the catalog gives a file of its own, listing the exported pages only, with relative Markdown links whatever `links` says) and `after_publish` (default false: refresh this target after every `publish` that committed). The handler `command` takes `command` (required: the name of an entry of `commands`). A field the handler does not take is an error. | | `commands` | checkout, names a command | object | no | `{}` | Command name to command: an array of the program and its arguments, run by the `command` handler without a shell, with stdin closed and a timeout of 300 s. The last two arguments appended are the directory holding the Markdown and an empty output directory. Exit 0 makes the regular files in the output directory the target's content; anything else leaves the target unchanged and prints the end of the program's stderr. | - **Absent:** No export: `export run` stops with exit 1 naming the section, `publish` exports nothing and `doctor` reports `OK`. @@ -672,6 +672,12 @@ Example: "links": "relative", "after_publish": true }, + "whole-kb": { + "handler": "markdown", + "path": "C:\\Users\\\\OneDrive - \\Chemenu-KB", + "include": "all", + "index": true + }, "as-word": { "handler": "command", "command": "pandoc-docx", @@ -696,12 +702,24 @@ Example: ``` -Within one target, `include` is a list of selections, each a list of `search` predicates (`tags=docs`, -`entity_type=system`, `!source_url`): a page is exported when it satisfies every predicate of at least -one selection. Only what `include` selects is written - there is no way to exclude by default. The -syntax is checked when the section is read, the field names against the corpus when `export run` -runs. `path` must not lie in or contain the instance, another target, an `intake` folder or -`backup.target`. +Within one target, `include` is either the string `"all"` or a list of selections, each a list of +`search` predicates (`tags=docs`, `entity_type=system`, `!source_url`): a page is exported when it +satisfies every predicate of at least one selection. `"all"` selects every page `search` knows without +a predicate - one directly under `kb/` too, never a generated or meta file (`kb/index.md`, `kb/log.md`, +`kb/provenance.md`, `kb/CONTRACT.md`, `kb/CONVENTIONS.md`, an `INDEX.md`, a `COLLECTION.md`). Only what +`include` selects is written - there is no way to exclude by default, and a missing or empty `include` +is an error, never "all". The syntax is checked when the section is read, the field names against the +corpus when `export run` runs. `path` must not lie in or contain the instance, another target, an +`intake` folder or `backup.target`. + +With `"index": true` a run also writes `index.md` in the root of the target and an `INDEX.md` per +collection - and per area the catalog under `kb/` would give a file of its own. They list the exported +pages only (Page, Type, Summary, Last Modified), link with relative Markdown links whatever `links` +says, and carry no date of the day: Last Modified is the page's own `modified`, `date` or `created`, +empty where it has none. The column and heading labels are English. Each starts with a frontmatter +block whose only key is `wikitool_export` (`kind=index`), so `raw accept` refuses it like a page. A +page `/index.md` beside the generated `/INDEX.md` stops the target, since a +synced folder on Windows would make them one file. Every Markdown file starts with a frontmatter block whose first key is `wikitool_export`, carrying the kind, the instance and the newest commit that touched that page. `frontmatter` decides what follows: diff --git a/raw/CONTRACT.md b/raw/CONTRACT.md index 96a0ab4..fa13962 100644 --- a/raw/CONTRACT.md +++ b/raw/CONTRACT.md @@ -99,9 +99,9 @@ both ways out - the whole folder, or the file moved up into `incoming/`. own output and `raw accept` refuses it, given on its own, inside a folder, or as `--replaces`/`--replaces-bundle` material, naming each one. The marker is either a first line (after an optional BOM) that starts with `