export: export run writes selected kb/ pages as a derived, read-only version into configured target folders (markdown and command handlers, manifest, after_publish), raw accept also refuses manifest-listed files, doctor checks the section (#214); 10.0.0-beta.13
Files changed: - CHANGES.md - VERSION - docs/how-to/README.md - docs/how-to/back-up-to-a-synced-folder.md - docs/how-to/collect-from-a-synced-folder.md - docs/how-to/export-guidelines.md - docs/how-to/export-to-a-synced-folder.md - docs/reference/commands/README.md - docs/reference/commands/exports.md - docs/reference/commands/git.md - docs/reference/commands/guideline-export.md - docs/reference/commands/instance-health.md - docs/reference/commands/raw-material-and-uploads.md - docs/reference/configuration.md - docs/reference/exit-codes.md - instructions/setup-instance.md - raw/CONTRACT.md - tools/CONTRACT.md - tools/README.md - tools/chemenu/backup.py - tools/chemenu/cli_contract.py - tools/chemenu/commands/doctor.py - tools/chemenu/commands/export_cmd.py - tools/chemenu/commands/git_publish.py - tools/chemenu/commands/raw/_common.py - tools/chemenu/commands/raw/accept.py - tools/chemenu/commands/raw/capture.py - tools/chemenu/config_shapes.py - tools/chemenu/guideline_export.py - tools/chemenu/page_export.py - tools/chemenu/reference_doc.py - tools/chemenu/repo_capture.py - tools/chemenu/tests/test_config_shapes.py - tools/chemenu/tests/test_export_run.py Co-Authored-By: Claude Code <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
torbenandClaude Code committed
1 parent
ecefe0a033
commit
99945c4841
33 files changed
+2316
-96
No files matched your search
@@ -15,6 +15,8 @@ Working with the wiki:
|
||||
browser, or through a fetch program of your own
|
||||
- [Capture documentation from a git repository](capture-repository-docs.md) - and keep it current
|
||||
- [Export guidelines to captured repositories](export-guidelines.md)
|
||||
- [Export to a synced folder](export-to-a-synced-folder.md) - a read-only version of selected pages
|
||||
for a reader outside the repository, OneDrive and Microsoft 365 included
|
||||
- [Query the wiki](query-the-wiki.md) - asking, and searching yourself
|
||||
- [Maintain the wiki](maintain-the-wiki.md) - lint, provenance, and the gates on the way to a publish
|
||||
- [Run the weekly review](run-the-weekly-review.md) - commitments in a task tracker against the
|
||||
|
||||
@@ -156,3 +156,7 @@ Only one device publishes to `<target>/git`. Two devices pushing into the same s
|
||||
writers again, only one step removed - and the sync client resolves that with conflict copies, not
|
||||
with a merge. An instance on a second device gets a target of its own. The sync client is a backup
|
||||
here, not a way to work on one instance from two devices.
|
||||
|
||||
A folder the sync client uploads can also hold a read-only version of selected pages, for readers
|
||||
outside the repository: [Export pages to a synced folder](export-to-a-synced-folder.md). It is a
|
||||
separate folder with its own settings, and never the backup target.
|
||||
@@ -131,3 +131,7 @@ exits 0 with "nothing to collect".
|
||||
|
||||
The agent does the same at the start of an ingest when you name no file: `upload collect`, then
|
||||
`upload list`, then a report of what waits. It never runs `upload accept` for you.
|
||||
|
||||
The other direction - a read-only version of selected pages written out to a synced folder - is
|
||||
[Export pages to a synced folder](export-to-a-synced-folder.md). An export is never a source: the
|
||||
export marker keeps it out of `raw/`.
|
||||
@@ -50,5 +50,7 @@ did not see.
|
||||
|
||||
## Related
|
||||
|
||||
- [Guideline export commands](../reference/commands/guideline-export.md)
|
||||
- [Export commands](../reference/commands/exports.md)
|
||||
- [Export pages to a synced folder](export-to-a-synced-folder.md) - the other export, a read-only
|
||||
version of `kb/` pages
|
||||
- [Why the gates are code](../explanation/why-gates-are-code.md)
|
||||
@@ -0,0 +1,155 @@
|
||||
# Export to a synced folder
|
||||
|
||||
Give a reader outside the repository a read-only version of part of `kb/`: the selected pages,
|
||||
written as files into a folder - typically one a sync client such as OneDrive uploads, so that a
|
||||
Microsoft 365 tenant can index them. The export is derived. It is regenerated from the committed
|
||||
pages on every run, nobody edits it, and nothing in it flows back into the wiki. What the settings
|
||||
look like: [configuration reference](../reference/configuration.md#export-to-a-folder).
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Pick the pages and the format](#pick-the-pages-and-the-format)
|
||||
- [Set it up](#set-it-up)
|
||||
- [What a run does to the folder](#what-a-run-does-to-the-folder)
|
||||
- [Export after every publish](#export-after-every-publish)
|
||||
- [Convert the pages with a program of your own](#convert-the-pages-with-a-program-of-your-own)
|
||||
- [What does not come back in](#what-does-not-come-back-in)
|
||||
- [Related](#related)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## Pick the pages and the format
|
||||
|
||||
- **Which pages.** Only what a target's `include` selects is written; there is no "everything". A
|
||||
selection is a list of `search` predicates that must all hold - `tags=shared`, `kind=system`,
|
||||
`!draft` - and a page that fits any selection of the target is exported. Try a selection with
|
||||
`tools/wikitool search --field tags=shared` first.
|
||||
- **Which format.** Two ways, equally valid:
|
||||
- `markdown` writes the pages as `.md` files. It needs nothing but the folder.
|
||||
- `command` hands those files to a converter of your own and exports what it produces - Word
|
||||
documents with `pandoc`, say.
|
||||
|
||||
Whether a Microsoft 365 tenant indexes `.md` files for Copilot is not settled for every tenant.
|
||||
Try a small `markdown` target first; if the tenant does not pick the files up, the same pages go
|
||||
through a `command` target as a format it does.
|
||||
|
||||
## Set it up
|
||||
|
||||
1. **Pick the folder**: a folder inside the synced folder, for example
|
||||
`C:\Users\<you>\OneDrive - <company>\Chemenu-Lesefassung`. It must be empty or not exist yet,
|
||||
and it must lie outside the instance, outside every other export target, outside every `intake`
|
||||
folder and outside `backup.target`.
|
||||
2. **Write the `export` section** into `.wikitool-local.json` at the root of the instance - by hand,
|
||||
since its fields decide what leaves this machine and which program runs, and `config set`
|
||||
refuses them:
|
||||
|
||||
```json
|
||||
{
|
||||
"export": {
|
||||
"targets": {
|
||||
"shared": {
|
||||
"handler": "markdown",
|
||||
"path": "C:\\Users\\<you>\\OneDrive - <company>\\Chemenu-Lesefassung",
|
||||
"include": [["tags=shared"]],
|
||||
"frontmatter": "summary",
|
||||
"links": "relative",
|
||||
"after_publish": false
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`frontmatter` decides what follows the export marker in each file: `minimal` nothing, `summary`
|
||||
(the default) the title, type, summary and modified date, `full` the page's whole frontmatter.
|
||||
`links: relative` (the default) turns a link to an exported page into a relative Markdown link;
|
||||
`links: text` leaves the page's name. A link to a page that is not exported is always plain text.
|
||||
3. **Tell the agent:**
|
||||
|
||||
> Export the pages.
|
||||
|
||||
The agent runs `tools/wikitool export run`. It reports per target which pages are new, which
|
||||
changed and which were removed, and how many stayed as they were. Name a target to run only that
|
||||
one: `tools/wikitool export run shared`.
|
||||
4. **Check it:** `tools/wikitool doctor`. The `export` line lists the targets; a `FAIL` there names
|
||||
what is wrong with the section.
|
||||
|
||||
The export reads the committed pages. A `kb/` or `types/` with uncommitted changes is refused, so
|
||||
every file names the commit its pages come from. Publish first.
|
||||
|
||||
## What a run does to the folder
|
||||
|
||||
The folder holds the pages and one file of the stack's own, `.wikitool-export.json`, which lists
|
||||
what the last run wrote.
|
||||
|
||||
- **Only its own files.** A run removes a file only when it is listed and no longer exported, and a
|
||||
folder only when the run created it. Anything else in the folder stays as it is.
|
||||
- **Nobody edits an export.** A listed file that someone changed stops the target before anything is
|
||||
written, and the message names the file. Move or delete it and run again. A listed file that is
|
||||
missing is written again.
|
||||
- **A folder that is not an export is refused.** A folder that holds files and no manifest is
|
||||
somebody's folder; the run writes nothing into it.
|
||||
- **An empty selection is an error.** A run that finds no page does not delete the earlier export.
|
||||
- **An unchanged export stays untouched.** A run that has nothing to change writes nothing, not even
|
||||
a modification time - the sync client has nothing to upload.
|
||||
- **A growing selection needs no confirmation.** New pages are reported as new; read the list
|
||||
before the sync client uploads them.
|
||||
|
||||
Targets are independent: one that fails is reported with its reason and the others still run; the
|
||||
command then exits 1.
|
||||
|
||||
## Export after every publish
|
||||
|
||||
With `"after_publish": true` on a target, every `tools/wikitool publish` that committed refreshes
|
||||
it afterwards - with `--no-push` too, since the export reads the commit, not the remote. If that
|
||||
export fails, `publish` prints a `WARN` with the reason and still exits 0: the commit and the push
|
||||
stand. Fix what the line names and run `tools/wikitool export run`; another `publish` does not
|
||||
repeat it. A malformed `export` section stops `publish` before it commits, but only when some target
|
||||
sets `after_publish`.
|
||||
|
||||
## Convert the pages with a program of your own
|
||||
|
||||
A target with `"handler": "command"` names an entry of `commands`:
|
||||
|
||||
```json
|
||||
{
|
||||
"export": {
|
||||
"targets": {
|
||||
"as-word": {
|
||||
"handler": "command",
|
||||
"command": "pandoc-docx",
|
||||
"path": "C:\\Users\\<you>\\OneDrive - <company>\\Chemenu-Word",
|
||||
"include": [["tags=shared"]]
|
||||
}
|
||||
},
|
||||
"commands": {
|
||||
"pandoc-docx": ["pwsh", "-NoProfile", "-File", "C:\\tools\\pandoc-docx.ps1"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The pages are rendered into a staging directory first. The program runs without a shell, with stdin
|
||||
closed and a limit of 300 seconds, and gets two more arguments: the directory with the Markdown
|
||||
files and an empty directory for its output. On exit 0 the regular files it left there (below any
|
||||
subdirectory, symlinks left out and reported) become the content of the target; the manifest and
|
||||
all the rules above apply to them as to Markdown. Any other end - a non-zero exit, a timeout, a
|
||||
program that does not start - leaves the target exactly as it was and prints the end of the
|
||||
program's stderr. The staging directory is always removed.
|
||||
|
||||
## What does not come back in
|
||||
|
||||
An export is never a source. Every Markdown file starts with a frontmatter block whose first key is
|
||||
`wikitool_export`, and `tools/wikitool raw accept` and `raw capture` refuse a file that carries it.
|
||||
A converter may lose that key, so `raw accept` also refuses any file in `incoming/` that is byte
|
||||
for byte one of the files in the manifest of a configured target - and warns, without refusing,
|
||||
when a target has no manifest yet or the `export` section is broken.
|
||||
|
||||
## Related
|
||||
|
||||
- [Configuration reference](../reference/configuration.md#export-to-a-folder)
|
||||
- [Export commands](../reference/commands/exports.md)
|
||||
- [Export guidelines to captured repositories](export-guidelines.md) - the other export, one file
|
||||
pushed into a repository
|
||||
- [Back up to a synced folder](back-up-to-a-synced-folder.md) - the same kind of folder, for the
|
||||
backup instead
|
||||
@@ -15,7 +15,7 @@ collected in [exit codes and gates](../exit-codes.md).
|
||||
- [Provenance](#provenance)
|
||||
- [Raw material and uploads](#raw-material-and-uploads)
|
||||
- [Git](#git)
|
||||
- [Guideline export](#guideline-export)
|
||||
- [Exports](#exports)
|
||||
- [Workshop runs and session budget](#workshop-runs-and-session-budget)
|
||||
- [Types, instructions and docs](#types-instructions-and-docs)
|
||||
- [Telemetry](#telemetry)
|
||||
@@ -101,11 +101,12 @@ Full records: [git.md](git.md)
|
||||
- [`backup init`](git.md#backup-init) - Create the backup target's bare repository and make it `origin` when there is none.
|
||||
- [`backup run`](git.md#backup-run) - Mirror this machine's files that are not in git into `<target>/<machine>/`.
|
||||
|
||||
## Guideline export
|
||||
## Exports
|
||||
|
||||
Full records: [guideline-export.md](guideline-export.md)
|
||||
Full records: [exports.md](exports.md)
|
||||
|
||||
- [`export guidelines`](guideline-export.md#export-guidelines) - Render the guideline pages into one generated `GUIDELINES.md`, or push it into the captured repositories that opted in (**Guideline Push Gate**).
|
||||
- [`export guidelines`](exports.md#export-guidelines) - Render the guideline pages into one generated `GUIDELINES.md`, or push it into the captured repositories that opted in (**Guideline Push Gate**).
|
||||
- [`export run`](exports.md#export-run) - Write the derived, read-only reading copy of `kb/` into the folders the `export` section configures - the selected pages as Markdown, or converted by a program of this checkout.
|
||||
|
||||
## Workshop runs and session budget
|
||||
|
||||
|
||||
@@ -1,6 +1,13 @@
|
||||
# Guideline export
|
||||
# Exports
|
||||
|
||||
Writing the generated `GUIDELINES.md` into the repositories the wiki has captured.
|
||||
Derived, read-only versions of `kb/`: the generated `GUIDELINES.md` for the repositories the wiki has captured, and the pages exported into configured folders.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [`export guidelines`](#export-guidelines)
|
||||
- [`export run`](#export-run)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
<!-- wikitool:reference-commands -->
|
||||
## `export guidelines`
|
||||
@@ -74,4 +81,62 @@ tools/wikitool export guidelines --confirm <token> --push --field tags=guideline
|
||||
- `wikitool search` - the same predicates, to preview the selection
|
||||
- `wikitool raw capture` - how a repository becomes a target
|
||||
- `instructions/gates.md` - the gate procedure
|
||||
|
||||
## `export run`
|
||||
|
||||
Write the derived, read-only reading copy of `kb/` into the folders the `export` section configures - the selected pages as Markdown, or converted by a program of this checkout.
|
||||
|
||||
**Synopsis**
|
||||
|
||||
- `wikitool export run [<target> ...]` - Every configured target, or the named ones. Needs an `export` section in `.wikitool-local.json`; the user writes it by hand.
|
||||
|
||||
**Options**
|
||||
|
||||
```text
|
||||
Arguments:
|
||||
[TARGET]... Targets to run; default: every configured target.
|
||||
|
||||
Options:
|
||||
-h, --help Show this message and exit.
|
||||
```
|
||||
|
||||
**Examples**
|
||||
|
||||
```bash
|
||||
tools/wikitool export run
|
||||
tools/wikitool export run work-notes
|
||||
```
|
||||
|
||||
**Behaviour**
|
||||
|
||||
- An export is derived: regenerated from `kb/` on every run, never edited, never read back into the wiki. Each target names its pages with an allowlist of `search --field` predicates (`include`: a list of selections, AND within one, OR across them), a handler (`markdown`, or `command` for the operator's converter) and the folder (`path`, outside the instance). Nothing is exported that no selection names.
|
||||
- Every Markdown file starts with a frontmatter block whose `wikitool_export` line says `kind=pages instance=<origin URL without userinfo, or local> commit=<newest commit touching a selected page>`, followed by `title`, `type`, `summary`, `modified` where the page has them (`frontmatter: summary`, the default; `minimal` keeps only the marker, `full` the whole frontmatter), then the page text. A file keeps its path below `kb/`. A wikilink to an exported page becomes a relative Markdown link (`links: relative`, the default) or plain text (`links: text`); a link to a page that is not exported is always plain text. Footnote markers stay; code is never rewritten. LF endings, no timestamp: the same pages give the same bytes.
|
||||
- Handler `command`: the Markdown is rendered into a staging directory, the program of `export.commands` runs with no shell, stdin closed and a timeout of 300 s, and receives the Markdown directory and an empty output directory as its last two arguments. On exit 0 the regular files in the output directory (recursively) become the target's content; any other end leaves the target unchanged and prints the end of its stderr. The staging directory is always removed.
|
||||
- The folder is only ever touched by what it holds in its manifest, `.wikitool-export.json`: the files an earlier run wrote, each with its sha256. A run removes only manifest files that are no longer exported (and folders it created, once empty), rewrites a manifest file that is missing, leaves every other file byte for byte as it was, and writes nothing - not even a modification time - for a file that did not change. A run that changes nothing writes nothing at all.
|
||||
- Refuses a target, before writing anything into it: a non-empty folder with no manifest (it is somebody's, not an export), a manifest file that was changed in the folder since the last run (naming it), a foreign file where an exported one belongs, an empty selection (it would delete the earlier export), an unknown field in a predicate. A missing folder is created.
|
||||
- Refuses everything, before any target: uncommitted changes in `kb/` or `types/` (untracked files included), no git checkout, a page under `kb/` with unreadable frontmatter, an unknown target name, a broken `export` section.
|
||||
- Output per target: `new` (with the page titles), `changed`, `removed`, the count `unchanged`, or `failed` with the reason. A page that newly enters a selection is reported as new, not gated. Exit 0 when every target ran; exit 1 when any failed - the others were still served.
|
||||
- A target with `after_publish: true` is also refreshed by every `publish` that committed, `--no-push` included; a failure there is a warning, the commit stands, and this command is the repeat.
|
||||
- `raw accept` refuses a file whose sha256 a target's manifest lists, and one that carries the marker: an export never goes back into `raw/`.
|
||||
|
||||
**Properties**
|
||||
|
||||
- Changes files: yes
|
||||
- Idempotent: yes
|
||||
- Atomic: Per file within a target (a temporary file in the target, then a move over its place), the manifest last; targets are independent, so a failed target does not undo another
|
||||
- Counts toward the session call budget: yes
|
||||
- Can reach the network: no
|
||||
|
||||
**Exit codes**
|
||||
|
||||
- `0` - success
|
||||
- `1` - No `export` section, an unknown target name, or a broken section (a predicate that does not parse, a path inside the instance or in the way of another target, an `intake` folder or the backup target)
|
||||
- `1` - `kb/` or `types/` has uncommitted changes, the instance is not a git checkout, or a page under `kb/` has unreadable frontmatter
|
||||
- `1` - A target failed: a foreign folder, a file changed in the target, an empty selection, an unknown field, a converter that did not exit 0
|
||||
|
||||
**See also**
|
||||
|
||||
- `wikitool search` - the same predicates, to preview a selection
|
||||
- `wikitool backup run` - the same kind of folder, for this machine's own files
|
||||
- `docs/how-to/export-to-a-synced-folder.md`
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -134,6 +134,7 @@ tools/wikitool publish --confirm-rebase <token> --message "ingest: docker-cheats
|
||||
- `--path` (repeatable) scopes the whole operation - gate count, staging and commit - to that subtree, with one exception: uncommitted generated files outside `--path` (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md` under `kb/`) are staged and committed too, and a line names them - whether a reconcile or the change itself wrote them. They stay out of the gate count, and the `--confirm` token and the re-run line cover the `--path` given. Every other file outside `--path` is left as it is.
|
||||
- Commit message: `--message`, then a `Files changed:` paragraph listing every committed path. When `--message` ends in a paragraph git reads as a trailer block (`Co-Authored-By:` and the like), that block stays the last paragraph and the list goes in front of it, since git reads trailers from the last paragraph only. `git interpret-trailers` decides whether there is such a block, and the list moves only when git reads the same trailers from the result; otherwise it is appended at the end. `--message` is not part of any gate token.
|
||||
- Machine-data backup: with a `backup` section in `.wikitool-local.json` whose `machine_data` is true, a call that pushed then mirrors this machine's files that are not in git into `<target>/<machine>/` - the same step as `backup run`. A failure there prints a `WARN` line with the reason and still exits 0, since the commit and the push stand. No backup with `--no-push`, after a failed push, or when there was nothing to push. A malformed `backup` section stops the call with exit 1 before anything is committed, `--no-push` included.
|
||||
- Export after publish: every target of the `export` section with `after_publish: true` is refreshed once a commit was made - the same step as `export run`, `--no-push` included, since the export reads the committed `kb/`. A failure there prints a `WARN` line with the reason and still exits 0, since the commit stands; `export run` repeats it. A malformed `export` section stops the call with exit 1 before anything is committed, but only when some target sets `after_publish`.
|
||||
- After a successful commit or push whose changed files include `tools/`, `types/`, `instructions/`, `AGENTS.md` or a path ending in `CONTRACT.md`, prints one reminder line: the phase past this point (an issue-body rewrite, `docs/` staleness, a changelog entry's accuracy) is not covered by `docs verify`, `instructions verify` or `pytest`. It is not a gate: no exit code change, nothing to clear, and silent for an ordinary content publish.
|
||||
|
||||
**Properties**
|
||||
@@ -149,11 +150,12 @@ tools/wikitool publish --confirm-rebase <token> --message "ingest: docker-cheats
|
||||
|
||||
- `0` - success
|
||||
- `0` - The machine-data backup after the push failed: a `WARN` line names the reason, and the commit and the push stand
|
||||
- `0` - The export after the commit failed: a `WARN` line names the target and the reason, and the commit stands
|
||||
- `1` - git failed - `git add`, `git commit`, `git push`, or the reconcile's automatic rebase (a conflict outside the generated files, an edited `kb/log.md`, or an uncommitted change on a path the incoming commits change)
|
||||
- `1` - The push target (`--branch`) is not the checked-out branch, or HEAD is detached; the unborn branch of a fresh `git init` is not this case
|
||||
- `1` - No `--no-push`, and the remote is not configured or cannot be reached; nothing was committed
|
||||
- `1` - `--yes`/`-y` was passed - the flag does not exist and fails with an explicit error
|
||||
- `1` - The settings files do not pass their check, `remotes.allowed_push_urls` is not a list of strings, or the `backup` section is malformed; nothing was committed
|
||||
- `1` - The settings files do not pass their check, `remotes.allowed_push_urls` is not a list of strings, the `backup` section is malformed, or the `export` section is malformed while a target sets `after_publish`; nothing was committed
|
||||
- `42` - The remotes settings file of an earlier version is still there
|
||||
- `42` - Mass-Update Gate: `--threshold` (default 10) or more counted files would be committed, or the `--confirm` token does not match this changeset
|
||||
- `42` - Rebase-Review Gate: `<remote>/<branch>` moved and both sides changed the same file; the output lists the upstream commits, the overlapping files and their diff
|
||||
@@ -165,6 +167,7 @@ tools/wikitool publish --confirm-rebase <token> --message "ingest: docker-cheats
|
||||
- `instructions/gates.md` - the gate procedure
|
||||
- `instructions/setup-instance.md` - the first publish of a new instance
|
||||
- `wikitool backup run` - the machine-data backup on its own
|
||||
- `wikitool export run` - the page export on its own
|
||||
|
||||
## `backup init`
|
||||
|
||||
|
||||
@@ -38,6 +38,7 @@ tools/wikitool doctor --json
|
||||
- Legacy settings (`legacy-settings`): one `FAIL` for each per-feature settings file of an earlier version (telemetry, remotes, upload, task tracker) that is still there. Nothing reads them any more, so the settings they hold are not in effect; the fix is `tools/wikitool config migrate`.
|
||||
- Synced folder (`sync-folder`): a `WARN` when the git directory, links and junctions resolved, lies under a folder named by `OneDrive`, `OneDriveCommercial` or `OneDriveConsumer` (compared without regard to case on Windows) or under `~/Library/CloudStorage/` (macOS); `OK` otherwise, never a `FAIL`.
|
||||
- Backup (`backup`): the `backup` section absent is `OK`, malformed is a `FAIL`. Configured, a `WARN` when `<target>/git` is not a bare repository, `origin` does not push there, or - with `machine_data` true - no machine-data backup exists yet; otherwise `OK` with the age of the last machine-data backup.
|
||||
- Page export (`export`): the `export` section absent is `OK`, malformed is a `FAIL`; otherwise `OK` listing each target with its handler, folder and whether it runs after `publish`.
|
||||
- Intake channels (`upload-intake`): the channels of the `intake` section with their handler, and how many submissions wait in `quarantine/`. Absent is `OK` and means no channel - no `submit` tool, nothing to collect; malformed is a `FAIL`, and so is an `mcp-upload/` of an earlier version that still holds submissions (the message names the command that moves it). A `folder` channel whose `path` does not exist is a `WARN`.
|
||||
- Task tracker: the `tasks` section present, absent or malformed - absent is `OK` (no tracker configured), malformed is a `FAIL`. When `WIKITOOL_TASKS_CONFIG` is set, the profile it names is read instead of the `tasks`, `superproductivity` and `caldav` sections and the finding names it; a set variable that names no file is a `FAIL`, never `OK`.
|
||||
- For a configured `superproductivity` provider, the configured `access` path's own state: `access: "api"` reports whether its local REST API answers `GET /health` with a ready renderer right now, `access: "snapshot"` whether a backup file is ready. The other access path is never attempted, and neither state is ever a `FAIL`.
|
||||
|
||||
@@ -262,7 +262,7 @@ tools/wikitool raw capture --update raw/2026/10/service-docs
|
||||
- `--ref` is a branch name (`main`) or a tag pattern (`v*`, any of `*?[`), which names the newest matching tag by version order. It resolves to one commit - for an annotated tag, the commit it points at.
|
||||
- The commit is fetched by ref name, shallowly, into a bare cache repository per URL under `tools/.wikitool_capture/` (gitignored). Files are read from it as blobs, never through a checkout, so they are byte-identical to the repository's - no line-ending conversion, no filter.
|
||||
- `--path` globs (repeatable) select files by their path in the repository, as git's `:(glob)` pathspec does: `*`, `?` and `[...]` do not cross `/`; `**` as a whole segment spans any number of directories; a glob without wildcards also takes everything below it as a directory. Files keep their repository paths inside the bundle.
|
||||
- Never captured, each named in the output with its reason: a file whose first line starts with `<!-- wikitool:export`, a symlink, a submodule, a file over 25 MiB, a path with a segment starting with `.`, a file named `_capture.json`, and a Git LFS pointer.
|
||||
- Never captured, each named in the output with its reason: a file whose first line starts with `<!-- wikitool:export` or a leading frontmatter block with a `wikitool_export:` key, a symlink, a submodule, a file over 25 MiB, a path with a segment starting with `.`, a file named `_capture.json`, and a Git LFS pointer.
|
||||
- `incoming/<bundle>/_capture.json` records `schema`, `repo`, `ref`, `commit`, `paths`, `captured` (UTC), `fidelity`, `authority` and `files`. It is the only declaration that and how this instance follows the repository.
|
||||
- Only `ssh://`, `https://` and the scp form `user@host:path`; a URL with a password or token in it is refused, since the manifest is committed. Git runs with the host's own credentials and configuration, restricted to those protocols, and never prompts: a repository that asks for credentials is reported as unreachable. Each git call times out after 120 s.
|
||||
- `--update <raw-bundle>` takes URL, ref rule and globs from that bundle's manifest and writes the current state to `incoming/<same bundle name>/`; `--fidelity`/`--authority` default to the manifest's values, and passing one is the way to correct it.
|
||||
@@ -477,7 +477,7 @@ tools/wikitool raw accept incoming/chemenu --replaces-bundle raw/2026/10/chemenu
|
||||
- `0` - success
|
||||
- `1` - raw accept: A file does not exist or is not directly in `incoming/`; two files in one call share a filename; a target path already exists; or a target path would be over the path budget (160 UTF-16 code units below the instance root)
|
||||
- `1` - raw accept incoming/<folder>: The folder is not directly in `incoming/`, is combined with another argument, `--page` or `--replaces`, holds no file, or holds a hidden entry, a symlink or a special file; a target path is over the budget; or the folder name is already occupied under `raw/`
|
||||
- `1` - A file - given directly, inside a folder, or as `--replaces`/`--replaces-bundle` material - is a guideline export: its first line, after an optional BOM, starts with `<!-- wikitool:export`
|
||||
- `1` - A file - given directly, inside a folder, or as `--replaces`/`--replaces-bundle` material - is an export of this wiki: it carries the export marker (a first line starting with `<!-- wikitool:export`, or a `wikitool_export:` key in a leading frontmatter block), or it is byte for byte a file in the manifest of a configured `export` target
|
||||
- `1` - raw accept: `--fidelity`/`--authority` is missing, or names `unknown` or a value outside the schema's enum
|
||||
- `1` - raw accept: The target name is already occupied anywhere under `raw/` by something the call does not own
|
||||
- `1` - raw accept: `--page` names an unknown page or one with no `raw_files:` yet, an existing `raw_files:` entry is missing on disk, a file to be moved has more than one owning page, or `--page` would overwrite an already-set `fidelity`/`authority` with a different value
|
||||
|
||||
@@ -31,6 +31,7 @@ something you run.
|
||||
- [Site handlers](#site-handlers)
|
||||
- [Fetch routes](#fetch-routes)
|
||||
- [Backup to a synced folder](#backup-to-a-synced-folder)
|
||||
- [Export to a folder](#export-to-a-folder)
|
||||
- [Files of earlier versions](#files-of-earlier-versions)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
@@ -93,7 +94,7 @@ Each file is a JSON object of sections, one per feature, beside two keys of its
|
||||
the id; the id names the directory of the secrets file below. The sections are [telemetry](#telemetry),
|
||||
[remotes](#allowed-push-urls), [tasks, superproductivity and caldav](#task-tracker) and
|
||||
[intake](#intake-channels), [sites](#site-handlers), [fetch](#fetch-routes) and
|
||||
[backup](#backup-to-a-synced-folder). `config set` names the sections it knows, and `doctor` reports a
|
||||
[backup](#backup-to-a-synced-folder) and [export](#export-to-a-folder). `config set` names the sections it knows, and `doctor` reports a
|
||||
section it does not read as a `WARN`.
|
||||
|
||||
**One place per value.** A value in the file its tier does not name is an error naming the file,
|
||||
@@ -631,6 +632,96 @@ Example:
|
||||
`config set` exits 42 for them, and only you write them. The procedure, restoring included:
|
||||
[back-up-to-a-synced-folder.md](../how-to/back-up-to-a-synced-folder.md).
|
||||
|
||||
## Export to a folder
|
||||
|
||||
The `export` section of `.wikitool-local.json` declares folders that hold a derived, read-only
|
||||
version of `kb/` - for a reader outside the repository, such as a Microsoft 365 tenant that indexes
|
||||
a OneDrive folder. An export is regenerated from the committed pages on every run and is never
|
||||
edited; what someone changes in the folder is refused or overwritten, never merged back. Each
|
||||
target has a handler: `markdown` writes the selected pages as Markdown files, `command` renders
|
||||
them into a staging directory and hands that to a converter of this checkout.
|
||||
|
||||
<!-- wikitool:reference-config-export -->
|
||||
| Field | Tier | Type | Required | Default | Description |
|
||||
|-------|------|------|----------|---------|-------------|
|
||||
| `targets` | checkout, gate | object | yes | - | Target name (lower-case letters, digits and hyphens) to target; the name is the argument of `export run`. Every target has `handler` (`markdown`: the pages as Markdown files; `command`: the same Markdown run through a converter of this checkout), `path` (an absolute path outside the instance; it must not lie in or contain another target, an `intake` folder or `backup.target`) and `include` (an array of selections, each an array of `search --field` predicates: all predicates of a selection must hold, and a page that fits any selection is exported; there is no "everything"). Optional: `frontmatter` (`minimal`: only the export marker; `summary`, the default: also `title`, `type`, `summary` and `modified` where the page has them; `full`: the page's whole frontmatter), `links` (`relative`, the default: a wikilink to an exported page becomes a relative Markdown link; `text`: plain text; a link to a page that is not exported is always plain text) and `after_publish` (default false: refresh this target after every `publish` that committed). The handler `command` takes `command` (required: the name of an entry of `commands`). A field the handler does not take is an error. |
|
||||
| `commands` | checkout, names a command | object | no | `{}` | Command name to command: an array of the program and its arguments, run by the `command` handler without a shell, with stdin closed and a timeout of 300 s. The last two arguments appended are the directory holding the Markdown and an empty output directory. Exit 0 makes the regular files in the output directory the target's content; anything else leaves the target unchanged and prints the end of the program's stderr. |
|
||||
|
||||
- **Absent:** No export: `export run` stops with exit 1 naming the section, `publish` exports nothing and `doctor` reports `OK`.
|
||||
- **Malformed:** An error. `export run` stops with exit 1 and `doctor` reports `FAIL`; `publish` stops with exit 1 before it commits anything, but only when some target sets `after_publish`. A broken section is never read as "no export".
|
||||
|
||||
Example:
|
||||
|
||||
```json
|
||||
{
|
||||
"export": {
|
||||
"targets": {
|
||||
"work-notes": {
|
||||
"handler": "markdown",
|
||||
"path": "C:\\Users\\<user>\\OneDrive - <company>\\Chemenu-Lesefassung",
|
||||
"include": [
|
||||
[
|
||||
"kind=system"
|
||||
],
|
||||
[
|
||||
"tags=shared",
|
||||
"!draft"
|
||||
]
|
||||
],
|
||||
"frontmatter": "summary",
|
||||
"links": "relative",
|
||||
"after_publish": true
|
||||
},
|
||||
"as-word": {
|
||||
"handler": "command",
|
||||
"command": "pandoc-docx",
|
||||
"path": "C:\\Users\\<user>\\OneDrive - <company>\\Chemenu-Word",
|
||||
"include": [
|
||||
[
|
||||
"kind=system"
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"commands": {
|
||||
"pandoc-docx": [
|
||||
"pwsh",
|
||||
"-NoProfile",
|
||||
"-File",
|
||||
"C:\\tools\\pandoc-docx.ps1"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
<!-- /wikitool:reference-config-export -->
|
||||
|
||||
Within one target, `include` is a list of selections, each a list of `search` predicates (`tags=docs`,
|
||||
`entity_type=system`, `!source_url`): a page is exported when it satisfies every predicate of at least
|
||||
one selection. Only what `include` selects is written - there is no way to exclude by default. The
|
||||
syntax is checked when the section is read, the field names against the corpus when `export run`
|
||||
runs. `path` must not lie in or contain the instance, another target, an `intake` folder or
|
||||
`backup.target`.
|
||||
|
||||
Every Markdown file starts with a frontmatter block whose first key is `wikitool_export`, carrying
|
||||
the kind, the instance and the commit of the newest selected page. `frontmatter` decides what follows:
|
||||
`minimal` nothing, `summary` the title, type, summary and modified date, `full` the page's whole
|
||||
frontmatter. Wikilinks between exported pages become relative Markdown links (`links: relative`) or
|
||||
plain text (`links: text`); a link to a page that is not exported is always plain text. Footnote
|
||||
markers and code stay as they are.
|
||||
|
||||
A target folder holds `.wikitool-export.json`, which lists the files a run wrote with their digests.
|
||||
A run removes only listed files that are no longer exported, creates only folders it needs and
|
||||
removes only those it created, and leaves every other file untouched. It stops before writing for a
|
||||
listed file someone changed, for a folder that is not empty and has no manifest, and for a selection
|
||||
that comes up empty. A run that finds nothing to change writes nothing, not even a modification time.
|
||||
|
||||
`targets` and `commands` decide what leaves this checkout and which program runs, so they are marked
|
||||
`gate` and `names a command`: `config set` exits 42 for them, and only you write them. `raw accept`
|
||||
also refuses a file whose sha256 is listed in the manifest of a configured target, for an export
|
||||
that went through a converter and lost its marker. The procedure:
|
||||
[export-to-a-synced-folder.md](../how-to/export-to-a-synced-folder.md).
|
||||
|
||||
## Files of earlier versions
|
||||
|
||||
Until version 10 every feature above had a file of its own at the root: `.wikitool-telemetry.json`,
|
||||
|
||||
@@ -10,7 +10,7 @@ What each exit code means, and how a caller reacts to it, is [AGENTS.md § Tool
|
||||
|
||||
### guideline-push
|
||||
|
||||
- [`export guidelines`](commands/guideline-export.md#export-guidelines)
|
||||
- [`export guidelines`](commands/exports.md#export-guidelines)
|
||||
|
||||
### mass-update
|
||||
|
||||
@@ -45,6 +45,6 @@ What each exit code means, and how a caller reacts to it, is [AGENTS.md § Tool
|
||||
- [`publish`](commands/git.md#publish): Mass-Update Gate: `--threshold` (default 10) or more counted files would be committed, or the `--confirm` token does not match this changeset
|
||||
- [`publish`](commands/git.md#publish): Rebase-Review Gate: `<remote>/<branch>` moved and both sides changed the same file; the output lists the upstream commits, the overlapping files and their diff
|
||||
- [`publish`](commands/git.md#publish): Publish-Remote Gate: `remotes.allowed_push_urls` is set and the push URL of `--remote` is not listed in it, or `--remote` resolves to no push URL
|
||||
- [`export guidelines`](commands/guideline-export.md#export-guidelines): Guideline Push Gate: `--push` without `--confirm`, or with a token that does not match the current set of targets
|
||||
- [`export guidelines`](commands/exports.md#export-guidelines): Guideline Push Gate: `--push` without `--confirm`, or with a token that does not match the current set of targets
|
||||
- [`config set`](commands/configuration.md#config-set): The field names a program to run or affects a gate
|
||||
<!-- /wikitool:reference-exit-codes -->
|
||||
Reference in new issue
Block a user