Files
torbenandClaude Sonnet 5.5 db09e89223
CI / verify (push) Successful in 8m17s
CI / pwsh (push) Successful in 2m54s
Release / converters (push) Successful in 28s
Release / release (push) Successful in 36s
Release / wiki-mirror (push) Skipped
tools: Export - include "all" wählt die ganze KB, index: true schreibt index.md und INDEX.md in den Zielordner (#217); 10.1.0-beta.21
Files changed:
- CHANGES.md
- VERSION
- docs/how-to/export-to-a-synced-folder.md
- docs/reference/commands/exports.md
- docs/reference/configuration.md
- raw/CONTRACT.md
- tools/CONTRACT.md
- tools/README.md
- tools/chemenu/catalog.py
- tools/chemenu/commands/export_cmd.py
- tools/chemenu/commands/index_build.py
- tools/chemenu/config_shapes.py
- tools/chemenu/export_index.py
- tools/chemenu/guideline_export.py
- tools/chemenu/page_export.py
- tools/chemenu/tests/test_config_shapes.py
- tools/chemenu/tests/test_export_run.py
- tools/chemenu/tests/test_index_build.py

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
2026-10-16 15:44:21 +02:00

44 KiB

Configuration

The environment variables wikitool reads and the two tiered settings files an operator may create at the root of an instance. None of them is needed for an instance to work: an instance installed from a release runs against the origin's public release feed with no configuration at all.

Every setting of a feature - telemetry, the publish allowlist, the task tracker, the intake channels - is a section of those two files, and the shape of each section - its fields, their tier, what an absent or a malformed section means, an example - is generated from the record wikitool itself reads it with, so it cannot describe a field the tool does not take. What a setting does around it is written by hand.

Three more files at the root are written by wikitool and never by hand: .wikitool-release.json, .wikitool-kb.json and .wikitool-tools.json (AGENTS.md invariant 1). The last one has its own section below, because the preflight that writes it is something you run.

Contents

Environment variables

Variable Purpose Fallback
WIKI_AUTHOR Overrides the author name of new source pages git config user.name. Without either, new stops with an ERROR
WIKITOOL_SESSION_ID Scopes the Iteration Budget Gate to a task rather than a terminal, and names the directory a trace lands in A session variable the harness sets itself, where one is known (for example CLAUDE_CODE_SESSION_ID), else the parent process id (instructions/session-setup.md)
WIKITOOL_UPDATE_URL The release feed version check, version notes and dist upgrade --latest ask The value in .wikitool-release.json, else the origin's feed
WIKITOOL_UPDATE_TOKEN A Gitea token for the release feed, sent only to downloads on the feed's own host None. Not needed against the origin, only against a feed in a private repository
WIKITOOL_TASKS_CONFIG Path to a tracker profile that task, review and doctor read instead of the tasks, superproductivity and caldav sections of the settings, for running one checkout against several trackers in turn (below) The sections in wikitool.json and .wikitool-local.json. A path to a file that does not exist is an error, never "no tracker configured"
WIKITOOL_SECRET_<NAME> The value behind a settings reference {"ref": "<name>"}, for CI and containers - <NAME> is the name upper-cased The secrets file in the user's configuration directory (below)
CHEMENU_ROOT Which corpus the package works on, for a caller outside the checkout itself The checkout the package sits in. tools/wikitool behaves the same without it
WIKI_TRACE 0/1 turns telemetry off or on, beating both the default and the telemetry section The installation-form default (below)
WIKI_TRACE_DIR Writes traces somewhere other than reports/telemetry/ reports/telemetry/
WIKI_TRACE_CONTENT 0 keeps only lengths and digests of prompts and replies instead of their text Text is kept
WIKI_TRACE_MAX_CONTENT Per-attribute content cap in characters 60 KiB
WIKI_TRACE_MAX_SESSION_BYTES Byte cap per session trace; beats the section's max_session_bytes 5 MiB
WIKI_TRACE_KEEP_SESSIONS How many session directories retention keeps; beats the section's keep_sessions 250

Tool paths

The preflight writes this file, not you. .wikitool-tools.json at the root holds the absolute paths of Python, git and ripgrep, as the preflight found them on this machine - and of the 1Password CLI op when it is installed. op is optional: the preflight records it when it finds it, reports ABSENT when it does not, and never asks for it. Only an op:// reference in the tiered settings needs it. wikitool starts git and rg from there instead of from PATH. It is per checkout and gitignored, because a path on one machine says nothing about the next.

If the file is missing or incomplete, tools/wikitool does not start (exit 42) and names the preflight. If a tool sits somewhere the preflight does not look, name it: tools/preflight.sh --set rg=<path>, or in PowerShell pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1 --set rg=<path>. Never edit the file by hand. doctor reports under tool-paths whether every path still works.

Tiered settings

Two files at the root hold the settings of the features that read them this way, and every field declares the tier it belongs to:

Tier File Holds
instance wikitool.json, committed What belongs to the corpus and is the same in every clone
checkout .wikitool-local.json, gitignored What is true only for this clone on this machine
secret neither Tokens, passwords and website sessions - both files hold only a reference, and a session from raw login none (below)

Each file is a JSON object of sections, one per feature, beside two keys of its own:

{
  "schema": 1,
  "instance_id": "6f1c0e52-3d1b-4b8e-9a51-2f0e7d1c9b44",
  "<section>": {"<field>": "<value>"}
}

instance_id sits in wikitool.json only. The first wikitool config set creates the file and the id; the id names the directory of the secrets file below. The sections are telemetry, remotes, tasks, superproductivity and caldav and intake, sites, fetch and backup and export. config set names the sections it knows, and doctor reports a section it does not read as a WARN.

One place per value. A value in the file its tier does not name is an error naming the file, the key and the right tier - never an override. The same holds for a key set in both files.

References. A field of tier secret takes only a reference; a field marked private - a host name, an internal URL - takes a plain value in .wikitool-local.json or a reference in either file. No other field takes one.

Reference Resolves through
{"ref": "<name>"} - lower-case letters, digits and single _ WIKITOOL_SECRET_<NAME>, else the key <name> in the secrets file
{"ref": "op://<vault>/<item>/<field>"} op read, the 1Password CLI the preflight recorded

The secrets file is ~/.config/chemenu/<instance_id>/secrets.json ($XDG_CONFIG_HOME instead of ~/.config when it is set; %APPDATA%\chemenu\<instance_id>\secrets.json on Windows), a JSON object of names to strings. Outside Windows it must be readable by its owner only (chmod 600); a file others can read is refused.

A reference is resolved when a command asks for that value, never when the files are read. A name nobody provides, an op that is not installed, or an item 1Password does not have ends the command with exit 1; a 1Password that is locked, signed out or waiting for an approval ends it with exit 42, for a human to unlock. Setting 1Password up: use-1password-for-settings.md.

Never in the committed file: a field that names a program to run or affects a gate - not even as a reference. config set does not write such a field either (exit 42); the user edits .wikitool-local.json by hand, and doctor checks it afterwards. The fields marked gate or names a command in the tables below are these: the publish allowlist, the intake channels and the external fetchers.

wikitool doctor reports one line per tier - settings-instance, settings-checkout, settings-secrets - and for the references only whether they resolve. docs verify holds a committed wikitool.json to the placement rules above, a plain secret or private value above all. Neither file ships with dist export. Why the tiers exist, and what keeping secrets out of the tree does not protect against: configuration-tiers.md.

Telemetry

Telemetry is off in an instance. Every instance carries the .wikitool-release.json of its release, and that is how chemenu.telemetry.policy recognises that nobody asked for telemetry. The same holds for every further clone of the instance, because the file is committed with the first publish. Only a clone of the origin repository, used to develop the stack, has no stamp. There the traces are the instrument the stack measures itself with, and they are on.

To turn the default around, add a telemetry section to .wikitool-local.json, by hand or with wikitool config set telemetry.enabled true. The consent to write prompts to this disk belongs to the checkout, so the section has no instance-tier field and no gate.

Field Tier Type Required Default Description
enabled checkout boolean no the installation-form default Telemetry on or off. WIKI_TRACE beats it in both directions. Any other value is read by its truthiness, so "no" means on.
max_session_bytes checkout positive integer no 5242880 Byte cap per session trace. WIKI_TRACE_MAX_SESSION_BYTES beats it. A number written as a string is accepted.
keep_sessions checkout positive integer no 250 How many session directories retention keeps. WIKI_TRACE_KEEP_SESSIONS beats it. A number written as a string is accepted.
  • Absent: The installation-form default applies: on in a clone of the origin repository, off in an instance.
  • Malformed: The hook that reads it never fails on it: a value it cannot use counts as absent, and an unreadable .wikitool-local.json as an absent section. doctor reports each such deviation as a WARN under telemetry, naming the field.

Example:

{
  "telemetry": {
    "enabled": true,
    "max_session_bytes": 5242880,
    "keep_sessions": 250
  }
}

wikitool doctor reports the current state: on or off, why, and how much has been recorded against both caps. What the caps do and what never reaches a trace: telemetry-and-evaluation.md.

The telemetry hook runs on every tool call and stays silent, so it reads only this section, leniently, and never exits 42: a .wikitool-telemetry.json left over from an earlier version is ignored there (below).

Allowed push URLs

The remotes section of .wikitool-local.json restricts where publish may push.

Field Tier Type Required Default Description
allowed_push_urls checkout, gate array of strings yes - The push URLs publish may write to, compared with the URL the remote resolves to for a push - not with its name. An empty list allows no push at all.
  • Absent: Any remote is allowed; the Publish-Remote Gate does not apply.
  • Malformed: publish stops with an error before committing anything. A broken section is never read as "no restriction".

Example:

{
  "remotes": {
    "allowed_push_urls": [
      "<url>"
    ]
  }
}

With the section, publish refuses with exit 42 (the Publish-Remote Gate) before pushing to a URL not on the list. Only you edit it: the field is marked gate, so config set exits 42 for it. The gate has no token and no flag, and an agent that edits the list to get past a refusal is opening a gate on its own initiative.

Task tracker

Optional. The weekly review (tools/wikitool review, skill gtd-weekly-review) checks the project pages under kb/gtd/ against a task tracker. Which tracker that is, and its connection details, are three sections: tasks (which provider, the review's thresholds), and the section of the provider chosen, superproductivity or caldav. What is the same in every clone - the provider, the thresholds, the list names - sits in wikitool.json; the host, the user name and the folder this machine uses in .wikitool-local.json; a token or password is a reference to a secret, never a value.

Field Tier Type Required Default Description
provider instance string yes - Which adapter the review goes through.
stalled_waiting_days instance positive integer yes - A waiting-for item older than this many days counts as overdue. A number written as a string is accepted.
unpaged_project_weeks instance positive integer yes - A tracker project with no kb/ page is reported once it is older than this many weeks. A number written as a string is accepted.
someday_stale_months instance positive integer yes - A someday/maybe item untouched for this many months counts as stale. A number written as a string is accepted.
  • Absent: No tracker is configured - a valid state. The weekly review has nothing to check against.
  • Malformed: An error, and doctor reports FAIL. A broken section is never read as "no tracker configured".

Example:

{
  "tasks": {
    "provider": "superproductivity",
    "stalled_waiting_days": 14,
    "unpaged_project_weeks": 3,
    "someday_stale_months": 5
  }
}

The thresholds are configuration, not schema. A value the provider cannot supply at all - a waiting item with no follow-up date, a tracker project with no creation date - is not skipped silently but reported as its own finding (waiting_no_follow_up/project_age_unknown).

Super Productivity

Field Tier Type Required Default Description
access instance string yes - Which way in: api, the local REST API (reads the current state and carries the write path), or snapshot, the newest backup snapshot (read-only).
api_base_url checkout, private string no "http://127.0.0.1:3876" With access: "api": where the app's REST API listens. An empty value means the default.
api_token secret string no - With access: "api", required: the token from the app's settings. Only ever a reference.
backups_dir checkout string no - With access: "snapshot", required: the directory the app writes its backup snapshots to; ~ is expanded.
  • Absent: Read only when tasks.provider is superproductivity.
  • Malformed: An error when it is read: a field of the other access path beside access, or a missing api_token or backups_dir, stops the command with exit 1.

Examples:

{
  "superproductivity": {
    "access": "api",
    "api_base_url": "http://127.0.0.1:3876",
    "api_token": {
      "ref": "sp_token"
    }
  }
}
{
  "superproductivity": {
    "access": "snapshot",
    "backups_dir": "~/.config/superProductivity/backups"
  }
}

access is mandatory and has no fallback. The fields of the other access path are an error, not ignored: a backups_dir beside access: "api" stops the command with exit 1, as does an api_token missing from it. It chooses between two mutually exclusive ways in:

  • access: "snapshot" for an instance run headless: it reads only the newest backup snapshot under backups_dir. That works without the app running, but only reads. The tracker cannot be written from there.

  • access: "api" for a desktop instance: it talks only to the local REST API. That answers only while the app runs, but it gives the current state and carries the write path. api_token is required (as a reference), because every endpoint except GET /health demands Authorization: Bearer <token>.

What can write:

  • tools/wikitool new project creates a tracker project of the same name only with access: "api", and even there not automatically. The command stops (exit 42) and asks a human to create it, then continues with --resume.
  • tools/wikitool task new creates a single item without a kb/ page.
  • tools/wikitool task close --id marks an item done. It never deletes one.

With access: "snapshot", all three refuse completely with exit 1, because the tracker is read-only from there. tools/wikitool task list --project only reads, so it answers with either access.

CalDAV

Field Tier Type Required Default Description
url checkout, private string yes, not empty - The account's CalDAV calendar home set. A trailing / is added when missing.
username checkout, private string yes, not empty - The login name, which can differ from the user id in the URL.
app_password secret string yes, not empty - An app password is recommended over the account password. Only ever a reference.
inbox_list instance string yes, not empty - The name of the one inbox list.
someday_list instance string yes, not empty - The name of the one someday/maybe list.
exclude_lists instance array of strings no [] Existing plain task lists that are not projects.
  • Absent: Read only when tasks.provider is caldav.

Example:

{
  "caldav": {
    "url": {
      "ref": "op://Chemenu/caldav/url"
    },
    "username": "<login>",
    "app_password": {
      "ref": "caldav_app_password"
    },
    "inbox_list": "Inbox",
    "someday_list": "Someday",
    "exclude_lists": [
      "..."
    ]
  }
}

caldav is the standards-based second adapter (RFC 4791/5545). It is verified against Nextcloud Tasks, with iOS Reminders as the mobile client, and built for that split: tick items off on the phone, maintain them at the desk. Unlike Super Productivity, there is only one way in. CalDAV is always a network access, so no access field is needed.

  • url may be an alias - Nextcloud accepts a short name there - because every later address comes only from the hrefs the server returns, never from this URL joined with a name.
  • inbox_list and someday_list name the two fixed lists, exactly one of each per instance.
  • A list with a VEVENT component never counts as a project, so exclude_lists is needed only for plain task lists.

A project is a list whose only supported component is VTODO. tools/wikitool new project creates it automatically with MKCALENDAR - without asking, unlike Super Productivity, because CalDAV has a real create command for lists. The uniqueness check runs against every list in the account, including excluded lists, the inbox, someday and mixed calendars. If a new name collides with any of them, nothing is created and the collision is named. Renaming the existing list in Nextcloud stays a manual step.

task new and task close work without restriction on caldav, which has no read-only mode like access: "snapshot". When ticking an item off, task close changes only STATUS, COMPLETED, PERCENT-COMPLETE, LAST-MODIFIED and DTSTAMP on the existing .ics resource. Every other property, including an unknown X- property or an alarm, is kept unchanged. A resource that changed since it was read (an ETag conflict) is not written, and the command stops with exit 1. Nothing is ever deleted. The lists of finished projects stay after archiving, because the adapter never deletes a list. Remove them by hand in Nextcloud when you want to.

Profiles

WIKITOOL_TASKS_CONFIG points at a profile: a file in the format of .wikitool-local.json that holds only the sections tasks, superproductivity and caldav (and schema, and live_test_version for the live suite). While it is set, task, review and doctor read those three sections from the profile instead of from wikitool.json and .wikitool-local.json, so one checkout can be run against several trackers in turn. A profile is checked like the local file, except that a plain instance-tier value is allowed in it; a secret is a reference there too, never a value. The live suite takes its throwaway tokens from WIKITOOL_SECRET_<NAME>.

Intake channels

The intake section of .wikitool-local.json declares the ways a document gets into the quarantine quarantine/ from somewhere other than this terminal. Each channel has a name and a handler:

Handler Source How it arrives
mcp The MCP read server's optional submit tool Pushed by a remote caller. At most one channel has this handler.
folder A directory outside the instance that a sync client fills (OneDrive and the like) Pulled by wikitool upload collect; what is taken is removed from the folder.
command A program of this checkout, named in commands Pulled by wikitool upload collect; the program leaves its files in an empty staging directory.

All of them end in the same place: a submission nobody has looked at, which wikitool upload accept promotes into incoming/ after a human has cleared the Upload Review Gate. What the authentication in front of the server has to guarantee for the mcp handler is in Set up the MCP read server; how to collect from a synced folder or a program is in Collect documents from a synced folder. Every field is marked gate or names a command: the channels are the user's to set by hand, and config set exits 42 for them.

Field Tier Type Required Default Description
channels checkout, gate object yes - Channel name (lower-case letters, digits and hyphens) to channel. The name goes into the manifest of what the channel delivers and is the argument of upload collect. Every channel has handler (mcp, folder or command), max_bytes (the largest single file) and allowed_extensions (each starting with ., compared case-insensitively). Then per handler: mcp takes identity_header (the request header the authenticating middleware sets to the submitter's identity, default X-Forwarded-User) and, required, quota_submissions_per_day and quota_bytes_per_day (per submitter, over any rolling 24 hours); folder takes path (required: an absolute path outside the instance, read without descending into subdirectories; it must not be or lie in <backup.target>/git or <backup.target>/<machine>, and upload collect refuses a folder that holds or lies in any machine folder, bare repository or export) and settle_seconds (default 60: a file whose modification or creation time is younger is left alone); command takes command (required: the name of an entry of commands). At most one channel has the handler mcp. A field the handler does not take is an error.
commands checkout, names a command object no {} Command name to command: an array of the program and its arguments, run by the command handler without a shell, with stdin closed and a timeout of 300 s, and a staging directory appended as the last argument. Exit 0 turns each regular file at the top level of that directory into a submission; anything else takes nothing over and prints the end of the program's stderr.
  • Absent: No channel: the submit tool is not registered and upload collect has nothing to collect. A valid state; doctor reports OK.
  • Malformed: An error. The MCP server does not start, upload collect stops with exit 1 and doctor reports FAIL. A broken section is never read as "no channel".

Example:

{
  "intake": {
    "channels": {
      "mcp": {
        "handler": "mcp",
        "identity_header": "X-Forwarded-User",
        "max_bytes": 10485760,
        "allowed_extensions": [
          ".md",
          ".pdf"
        ],
        "quota_submissions_per_day": 20,
        "quota_bytes_per_day": 52428800
      },
      "onedrive": {
        "handler": "folder",
        "path": "C:\\Users\\<user>\\OneDrive - <company>\\Chemenu-Eingang",
        "max_bytes": 52428800,
        "allowed_extensions": [
          ".pdf",
          ".docx",
          ".msg",
          ".eml",
          ".md"
        ],
        "settle_seconds": 60
      },
      "scanner": {
        "handler": "command",
        "command": "scanner-pull",
        "max_bytes": 52428800,
        "allowed_extensions": [
          ".pdf"
        ]
      }
    },
    "commands": {
      "scanner-pull": [
        "pwsh",
        "-NoProfile",
        "-File",
        "C:\\tools\\scanner-pull.ps1"
      ]
    }
  }
}

Size and quota are your deployment's decision, not defaults worth copying blindly. A field that does not belong to the channel's handler is an error, not ignored: identity_header and the quota_* fields belong to mcp, path and settle_seconds to folder, command to command. A folder path must be absolute and lie outside the instance, and upload collect reads only its top level.

Site handlers

tools/wikitool raw fetch knows how some websites serve their pages. For a URL of such a website a site handler fetches the page in the website's standard form, leaves the page furniture out of the derived text and prints hints about the page as CHECK lines; today there are two, both for learn.microsoft.com: learn-microsoft for every page of the host and learn-microsoft-powershell-module for a PowerShell module's overview page, which offers the cmdlet pages it lists as subpages for raw fetch --scope. Handlers are JSON files: the stack's, and the instance's own under sites/ (see How to write a site handler). The sites section is what the instance decides about the handlers, and it is optional: without it every handler applies, in the locale en-us.

  • disabled switches handlers off by name. A page of that website is then handled as any other: fetched under the URL as typed, with its page furniture in the derived text. A name no handler has is an error, so a typo cannot leave a handler on. Switching off a special handler leaves the general one of the same website in force.
  • locale is the language segment a handler for a multi-language website puts into the URL it fetches. The header of the derived text keeps the URL as typed under requested_url.
  • notes adds hint lines of the instance's own, printed after a fetch or an --html derivation of a page of that host, whether or not a handler exists for it. An entry names a host (exact, lower case), optionally a path (a regular expression matched against the start of the URL path, after the standard form was applied) and one to three single-line lines. A host and path pattern appears once; an entry whose path is left out applies to the whole host. A line is plain text and is output as it is, so it is for a reminder to the reader of the capture, never for an instruction the tool should carry out.
  • max_pages and pause_seconds bound raw fetch --scope: the most subpages one page may offer before the fetch is refused, and the pause before each subpage request, which keeps the load on the website low.
Field Tier Type Required Default Description
disabled instance array of strings no [] The names of the site handlers that do not apply. raw fetch then treats a page of that website as it does any other. A name no handler has is an error when raw fetch reads the section, so a typo cannot leave a handler on.
locale instance string no "en-us" The locale a handler for a multi-language website fetches, in the website's own spelling and lower case. An empty value means the default.
notes instance array of objects no [] Hint lines of the instance's own, printed as CHECK lines after a fetch or an --html derivation of a page of that host. Each entry has host (exact, lower case), an optional path (a regular expression matched against the start of the URL path) and lines (one to 3 single-line strings of at most 300 characters). A host and path pattern appears once.
max_pages instance positive integer no 200 The most subpages raw fetch --scope fetches for one page. A page that offers more is refused before the first subpage is requested. A number written as a string is accepted.
pause_seconds instance number no 1 The pause, in seconds, before each subpage request of raw fetch --scope. 0 means none. A number written as a string is accepted.
  • Absent: Every handler applies, with the locale en-us and no further hint lines.
  • Malformed: An error when raw fetch reads it: it stops with exit 1 before fetching anything. A broken section is never read as "no handlers".

Example:

{
  "sites": {
    "disabled": [
      "learn-microsoft-powershell-module"
    ],
    "locale": "en-us",
    "max_pages": 200,
    "pause_seconds": 1,
    "notes": [
      {
        "host": "learn.microsoft.com",
        "path": "^/[a-z-]+/azure/",
        "lines": [
          "Check the Azure retirement notice at the top of the page."
        ]
      }
    ]
  }
}

All three fields are instance-tier: what a website needs cleaned is the same in every clone, and none of them names a program or opens a gate. A section that does not read correctly stops raw fetch with exit 1 before it fetches anything.

Fetch routes

tools/wikitool raw fetch fetches every page the same way unless the fetch section says otherwise for its host: anonymously, with no cookie and no browser. routes maps a host to another way, and is instance-tier, so every clone fetches the host the same way:

  • "session" fetches the host's pages with the session the user stored with tools/wikitool raw login <host>, through the same built-in fetch. The host needs a site handler with a session block - its login page and how a page shows it arrived signed in (sites/CONTRACT.md). No stored session, or a page without that sign, stops the fetch with exit 42 for the user to sign in again. The session is a secret: it is stored in the secrets file above under session_<host> (every run of other characters as one _, so www.example.org is session_www_example_org), or comes from WIKITOOL_SECRET_SESSION_<HOST>, and no settings field refers to it. It is never written to 1Password.
  • Any other value names a fetcher of fetchers: a program of this checkout, run without a shell with the URL appended. fetchers names a program to run, so it lives only in .wikitool-local.json, written by hand - config set refuses it with exit 42, and a fetcher in wikitool.json is an error. A route to a fetcher this checkout does not define stops a fetch of that host with exit 1, so a clone without the program fails loudly instead of fetching anonymously.

The whole procedure, with the browser component it needs for signing in: fetch-pages-behind-a-login.md.

Field Tier Type Required Default Description
routes instance object no {} Host (exact, lower case) to route: "session" fetches with the user's session - the host needs a site handler that names its login page - and any other value names a fetcher of fetchers. A host not listed is fetched as before.
fetchers checkout, names a command object no {} Fetcher name to command: an array of the program and its arguments, run without a shell with the URL appended. Exit 0 and the page's HTML on stdout is a fetch; anything else stops raw fetch with exit 1 and the program's stderr.
  • Absent: Every page is fetched by the built-in anonymous fetch, as before.
  • Malformed: An error when raw fetch or raw login reads it: the command stops with exit 1 before fetching anything. A route to a fetcher this checkout does not define stops a fetch of that host the same way.

Example:

{
  "fetch": {
    "routes": {
      "www.heise.de": "session",
      "intranet.example.org": "homelab"
    },
    "fetchers": {
      "homelab": [
        "/usr/local/bin/fetch-page",
        "--html"
      ]
    }
  }
}

Backup to a synced folder

The backup section of .wikitool-local.json backs this checkout up into a folder a sync client uploads - OneDrive and the like - while the working tree itself stays outside it. The target gets two kinds of content:

<target>/git/          bare repository, the checkout's origin; publish pushes here
<target>/<machine>/    mirror of this machine's files that are not in git, plus .chemenu-backup.json

tools/wikitool backup init creates <target>/git and makes it origin when the checkout has none. With machine_data on, every publish that pushed mirrors .wikitool-local.json, ENVIRONMENT.md, what waits in incoming/ and the extra paths into <target>/<machine>/; tools/wikitool backup run does the same on its own. The secrets file is never among them - it lies outside the instance, and a backup reads nothing outside it. Plain private values in .wikitool-local.json, an internal host name for one, do reach the synced folder.

Field Tier Type Required Default Description
target checkout, gate string yes - The backup folder: an absolute path outside the instance. backup init creates <target>/git in it, backup run writes only to <target>/<machine>/. Everything else in the folder is the sync folder's: an export target or an intake folder may sit beside git and the machine folders (<target>/Export, <target>/Inbox), but none may be, or lie in, one of them.
machine_data checkout, gate boolean no false Mirror this machine's files after every publish that pushed, and with backup run: .wikitool-local.json, ENVIRONMENT.md, what waits in incoming/, and extra.
machine_name checkout string no the host name The name of this machine's folder under the target. Every character outside letters, digits, - and _ becomes -, and leading and trailing - are dropped; a name that is empty afterwards, or git in any case, is an error.
extra checkout, gate array of strings no [] Further files or directories to mirror, relative to the instance. One that resolves outside the instance, or into .git/, is an error.
  • Absent: No backup: publish behaves as without it, and backup init and backup run stop with exit 1 naming the section.
  • Malformed: An error. publish stops with exit 1 before it commits anything, backup init and backup run stop with exit 1, and doctor reports FAIL. A broken section is never read as "no backup".

Example:

{
  "backup": {
    "target": "C:\\Users\\<user>\\OneDrive - <company>\\Chemenu-Backup",
    "machine_data": true,
    "machine_name": "WORK-LAPTOP",
    "extra": [
      ".claude/settings.local.json"
    ]
  }
}

target, machine_data and extra decide what leaves this machine, so they are marked gate: config set exits 42 for them, and only you write them. The procedure, restoring included: back-up-to-a-synced-folder.md.

Export to a folder

The export section of .wikitool-local.json declares folders that hold a derived, read-only version of kb/ - for a reader outside the repository, such as a Microsoft 365 tenant that indexes a OneDrive folder. An export is regenerated from the committed pages on every run and is never edited; what someone changes in the folder is refused or overwritten, never merged back. Each target has a handler: markdown writes the selected pages as Markdown files, command renders them into a staging directory and hands that to a converter of this checkout.

Field Tier Type Required Default Description
targets checkout, gate object yes - Target name (lower-case letters, digits and hyphens) to target; the name is the argument of export run. Every target has handler (markdown: the pages as Markdown files; command: the same Markdown run through a converter of this checkout), path (an absolute path outside the instance; it must not lie in or contain another target or an intake folder, and not be, lie in or contain <backup.target>/git or <backup.target>/<machine>, nor lie in any other machine's folder or a bare repository) and include (the string "all": every page search knows, a page directly under kb/ included and no generated or meta file; or an array of selections, each an array of search --field predicates: all predicates of a selection must hold, and a page that fits any selection is exported. A missing or empty include is an error, never "all"). Optional: frontmatter (minimal: only the export marker; summary, the default: also title, type, summary and modified where the page has them; full: the page's whole frontmatter), links (relative, the default: a wikilink to an exported page becomes a relative Markdown link; text: plain text; a link to a page that is not exported is always plain text), index (default false; true: also write index.md in the target's root and an INDEX.md per collection, and per area the catalog gives a file of its own, listing the exported pages only, with relative Markdown links whatever links says) and after_publish (default false: refresh this target after every publish that committed). The handler command takes command (required: the name of an entry of commands). A field the handler does not take is an error.
commands checkout, names a command object no {} Command name to command: an array of the program and its arguments, run by the command handler without a shell, with stdin closed and a timeout of 300 s. The last two arguments appended are the directory holding the Markdown and an empty output directory. Exit 0 makes the regular files in the output directory the target's content; anything else leaves the target unchanged and prints the end of the program's stderr.
  • Absent: No export: export run stops with exit 1 naming the section, publish exports nothing and doctor reports OK.
  • Malformed: An error. export run stops with exit 1 and doctor reports FAIL; publish stops with exit 1 before it commits anything, but only when some target sets after_publish. A broken section is never read as "no export".

Example:

{
  "export": {
    "targets": {
      "work-notes": {
        "handler": "markdown",
        "path": "C:\\Users\\<user>\\OneDrive - <company>\\Chemenu-Lesefassung",
        "include": [
          [
            "kind=system"
          ],
          [
            "tags=shared",
            "!draft"
          ]
        ],
        "frontmatter": "summary",
        "links": "relative",
        "after_publish": true
      },
      "whole-kb": {
        "handler": "markdown",
        "path": "C:\\Users\\<user>\\OneDrive - <company>\\Chemenu-KB",
        "include": "all",
        "index": true
      },
      "as-word": {
        "handler": "command",
        "command": "pandoc-docx",
        "path": "C:\\Users\\<user>\\OneDrive - <company>\\Chemenu-Word",
        "include": [
          [
            "kind=system"
          ]
        ]
      }
    },
    "commands": {
      "pandoc-docx": [
        "pwsh",
        "-NoProfile",
        "-File",
        "C:\\tools\\pandoc-docx.ps1"
      ]
    }
  }
}

Within one target, include is either the string "all" or a list of selections, each a list of search predicates (tags=docs, entity_type=system, !source_url): a page is exported when it satisfies every predicate of at least one selection. "all" selects every page search knows without a predicate - one directly under kb/ too, never a generated or meta file (kb/index.md, kb/log.md, kb/provenance.md, kb/CONTRACT.md, kb/CONVENTIONS.md, an INDEX.md, a COLLECTION.md). Only what include selects is written - there is no way to exclude by default, and a missing or empty include is an error, never "all". The syntax is checked when the section is read, the field names against the corpus when export run runs. path must not lie in or contain the instance, another target, an intake folder or backup.target.

With "index": true a run also writes index.md in the root of the target and an INDEX.md per collection - and per area the catalog under kb/ would give a file of its own. They list the exported pages only (Page, Type, Summary, Last Modified), link with relative Markdown links whatever links says, and carry no date of the day: Last Modified is the page's own modified, date or created, empty where it has none. The column and heading labels are English. Each starts with a frontmatter block whose only key is wikitool_export (kind=index), so raw accept refuses it like a page. A page <collection>/index.md beside the generated <collection>/INDEX.md stops the target, since a synced folder on Windows would make them one file.

Every Markdown file starts with a frontmatter block whose first key is wikitool_export, carrying the kind, the instance and the newest commit that touched that page. frontmatter decides what follows: minimal nothing, summary the title, type, summary and modified date, full the page's whole frontmatter. Wikilinks between exported pages become relative Markdown links (links: relative) or plain text (links: text); a link to a page that is not exported is always plain text. Footnote markers and code stay as they are.

A target folder holds .wikitool-export.json, which lists the files a run wrote with their digests. A run removes only listed files that are no longer exported, creates only folders it needs and removes only those it created, and leaves every other file untouched. It stops before writing for a listed file someone changed, for a folder that is not empty and has no manifest, and for a selection that comes up empty. A run that finds nothing to change writes nothing, not even a modification time.

targets and commands decide what leaves this checkout and which program runs, so they are marked gate and names a command: config set exits 42 for them, and only you write them. raw accept also refuses a file whose sha256 is listed in the manifest of a configured target, for an export that went through a converter and lost its marker. The procedure: export-to-a-synced-folder.md.

Files of earlier versions

Until version 10 every feature above had a file of its own at the root: .wikitool-telemetry.json, .wikitool-remotes.json, .wikitool-upload.json and .wikitool-tasks.json. Nothing reads them any more. A command that needs the setting such a file held stops with exit 42 and names tools/wikitool config migrate, doctor reports each one as a FAIL, and only the telemetry hook ignores it. config migrate moves the content into the sections above (the upload file becomes the channel mcp of intake), each value into the file its tier names and each secret into the secrets file, and then removes the old files; it is run once per checkout. .wikitool-tools.json is not among them and stays. The migration step for an instance is 10.0.0-settings-tiers.md.