chemenu/docs/reference/configuration.md

# Configuration

The environment variables `wikitool` reads and the two tiered settings files an operator may create at
the root of an instance. None of them is needed for an instance to work: an instance installed from a
release runs against the origin's public release feed with no configuration at all.

Every setting of a feature - telemetry, the publish allowlist, the task tracker, the intake
channels - is a section of those two files, and the shape of each section - its fields, their tier, what
an absent or a malformed section means, an example - is generated from the record `wikitool` itself
reads it with, so it cannot describe a field the tool does not take. What a setting does around it is
written by hand.

Three more files at the root are written by `wikitool` and never by hand: `.wikitool-release.json`,
`.wikitool-kb.json` and `.wikitool-tools.json` ([AGENTS.md invariant 1](../../AGENTS.md#invariants)).
The last one has its own section [below](#tool-paths), because the preflight that writes it is
something you run.

<!-- wikitool:toc -->
## Contents

- [Environment variables](#environment-variables)
- [Tool paths](#tool-paths)
- [Tiered settings](#tiered-settings)
- [Telemetry](#telemetry)
- [Allowed push URLs](#allowed-push-urls)
- [Task tracker](#task-tracker)
  - [Super Productivity](#super-productivity)
  - [CalDAV](#caldav)
  - [Profiles](#profiles)
- [Intake channels](#intake-channels)
- [Site handlers](#site-handlers)
- [Fetch routes](#fetch-routes)
- [Backup to a synced folder](#backup-to-a-synced-folder)
- [Export to a folder](#export-to-a-folder)
- [Files of earlier versions](#files-of-earlier-versions)
<!-- /wikitool:toc -->

## Environment variables

| Variable | Purpose | Fallback |
|----------|---------|----------|
| `WIKI_AUTHOR` | Overrides the author name of new source pages | `git config user.name`. Without either, `new` stops with an `ERROR` |
| `WIKITOOL_SESSION_ID` | Scopes the Iteration Budget Gate to a task rather than a terminal, and names the directory a trace lands in | A session variable the harness sets itself, where one is known (for example `CLAUDE_CODE_SESSION_ID`), else the parent process id ([instructions/session-setup.md](../../instructions/session-setup.md)) |
| `WIKITOOL_UPDATE_URL` | The release feed `version check`, `version notes` and `dist upgrade --latest` ask | The value in `.wikitool-release.json`, else the origin's feed |
| `WIKITOOL_UPDATE_TOKEN` | A Gitea token for the release feed, sent only to downloads on the feed's own host | None. Not needed against the origin, only against a feed in a private repository |
| `WIKITOOL_TASKS_CONFIG` | Path to a tracker profile that `task`, `review` and `doctor` read instead of the `tasks`, `superproductivity` and `caldav` sections of the settings, for running one checkout against several trackers in turn ([below](#profiles)) | The sections in `wikitool.json` and `.wikitool-local.json`. A path to a file that does not exist is an error, never "no tracker configured" |
| `WIKITOOL_SECRET_<NAME>` | The value behind a settings reference `{"ref": "<name>"}`, for CI and containers - `<NAME>` is the name upper-cased | The secrets file in the user's configuration directory ([below](#tiered-settings)) |
| `CHEMENU_ROOT` | Which corpus the package works on, for a caller outside the checkout itself | The checkout the package sits in. `tools/wikitool` behaves the same without it |
| `WIKI_TRACE` | `0`/`1` turns telemetry off or on, beating both the default and the `telemetry` section | The installation-form default ([below](#telemetry)) |
| `WIKI_TRACE_DIR` | Writes traces somewhere other than `reports/telemetry/` | `reports/telemetry/` |
| `WIKI_TRACE_CONTENT` | `0` keeps only lengths and digests of prompts and replies instead of their text | Text is kept |
| `WIKI_TRACE_MAX_CONTENT` | Per-attribute content cap in characters | 60 KiB |
| `WIKI_TRACE_MAX_SESSION_BYTES` | Byte cap per session trace; beats the section's `max_session_bytes` | 5 MiB |
| `WIKI_TRACE_KEEP_SESSIONS` | How many session directories retention keeps; beats the section's `keep_sessions` | 250 |

## Tool paths

**The preflight writes this file, not you.** `.wikitool-tools.json` at the root holds the absolute
paths of Python, git and ripgrep, as the preflight found them on this machine - and of the
1Password CLI `op` when it is installed. `op` is optional: the preflight records it when it finds
it, reports `ABSENT` when it does not, and never asks for it. Only an `op://` reference in the
[tiered settings](#tiered-settings) needs it. `wikitool` starts git
and rg from there instead of from `PATH`. It is per checkout and gitignored, because a path on one
machine says nothing about the next.

If the file is missing or incomplete, `tools/wikitool` does not start (exit 42) and names the
preflight. If a tool sits somewhere the preflight does not look, name it:
`tools/preflight.sh --set rg=<path>`, or in PowerShell
`pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1 --set rg=<path>`. Never edit the
file by hand. `doctor` reports under `tool-paths` whether every path still works.

## Tiered settings

Two files at the root hold the settings of the features that read them this way, and every field
declares the tier it belongs to:

| Tier | File | Holds |
|------|------|-------|
| `instance` | `wikitool.json`, committed | What belongs to the corpus and is the same in every clone |
| `checkout` | `.wikitool-local.json`, gitignored | What is true only for this clone on this machine |
| `secret` | neither | Tokens, passwords and website sessions - both files hold only a reference, and a session from `raw login` none ([below](#fetch-routes)) |

Each file is a JSON object of sections, one per feature, beside two keys of its own:

```json
{
  "schema": 1,
  "instance_id": "6f1c0e52-3d1b-4b8e-9a51-2f0e7d1c9b44",
  "<section>": {"<field>": "<value>"}
}
```

`instance_id` sits in `wikitool.json` only. The first `wikitool config set` creates the file and
the id; the id names the directory of the secrets file below. The sections are [telemetry](#telemetry),
[remotes](#allowed-push-urls), [tasks, superproductivity and caldav](#task-tracker) and
[intake](#intake-channels), [sites](#site-handlers), [fetch](#fetch-routes) and
[backup](#backup-to-a-synced-folder) and [export](#export-to-a-folder). `config set` names the sections it knows, and `doctor` reports a
section it does not read as a `WARN`.

**One place per value.** A value in the file its tier does not name is an error naming the file,
the key and the right tier - never an override. The same holds for a key set in both files.

**References.** A field of tier `secret` takes only a reference; a field marked private - a host
name, an internal URL - takes a plain value in `.wikitool-local.json` or a reference in either
file. No other field takes one.

| Reference | Resolves through |
|-----------|------------------|
| `{"ref": "<name>"}` - lower-case letters, digits and single `_` | `WIKITOOL_SECRET_<NAME>`, else the key `<name>` in the secrets file |
| `{"ref": "op://<vault>/<item>/<field>"}` | `op read`, the 1Password CLI the preflight recorded |

The secrets file is `~/.config/chemenu/<instance_id>/secrets.json` (`$XDG_CONFIG_HOME` instead of
`~/.config` when it is set; `%APPDATA%\chemenu\<instance_id>\secrets.json` on Windows), a JSON object of
names to strings. Outside Windows it must be readable by its owner only (`chmod 600`); a file
others can read is refused.

A reference is resolved when a command asks for that value, never when the files are read. A name
nobody provides, an `op` that is not installed, or an item 1Password does not have ends the command
with exit 1; a 1Password that is locked, signed out or waiting for an approval ends it with exit 42,
for a human to unlock. Setting 1Password up: [use-1password-for-settings.md](../how-to/use-1password-for-settings.md).

**Never in the committed file:** a field that names a program to run or affects a gate - not even
as a reference. `config set` does not write such a field either (exit 42); the user edits
`.wikitool-local.json` by hand, and `doctor` checks it afterwards. The fields marked `gate` or
`names a command` in the tables below are these: the publish allowlist, the intake channels and the
external fetchers.

`wikitool doctor` reports one line per tier - `settings-instance`, `settings-checkout`,
`settings-secrets` - and for the references only whether they resolve. `docs verify` holds a
committed `wikitool.json` to the placement rules above, a plain secret or private value above
all. Neither file ships with `dist export`. Why the tiers exist, and what keeping secrets out of
the tree does not protect against: [configuration-tiers.md](../explanation/configuration-tiers.md).

## Telemetry

**Telemetry is off in an instance.** Every instance carries the `.wikitool-release.json` of its
release, and that is how `chemenu.telemetry.policy` recognises that nobody asked for telemetry. The
same holds for every further clone of the instance, because the file is committed with the first
publish. Only a clone of the origin repository, used to develop the stack, has no stamp. There the
traces are the instrument the stack measures itself with, and they are **on**.

To turn the default around, add a `telemetry` section to `.wikitool-local.json`, by hand or with
`wikitool config set telemetry.enabled true`. The consent to write prompts to this disk belongs to
the checkout, so the section has no instance-tier field and no gate.

<!-- wikitool:reference-config-telemetry -->
| Field | Tier | Type | Required | Default | Description |
|-------|------|------|----------|---------|-------------|
| `enabled` | checkout | boolean | no | the installation-form default | Telemetry on or off. `WIKI_TRACE` beats it in both directions. Any other value is read by its truthiness, so `"no"` means on. |
| `max_session_bytes` | checkout | positive integer | no | `5242880` | Byte cap per session trace. `WIKI_TRACE_MAX_SESSION_BYTES` beats it. A number written as a string is accepted. |
| `keep_sessions` | checkout | positive integer | no | `250` | How many session directories retention keeps. `WIKI_TRACE_KEEP_SESSIONS` beats it. A number written as a string is accepted. |

- **Absent:** The installation-form default applies: on in a clone of the origin repository, off in an instance.
- **Malformed:** The hook that reads it never fails on it: a value it cannot use counts as absent, and an unreadable `.wikitool-local.json` as an absent section. `doctor` reports each such deviation as a `WARN` under `telemetry`, naming the field.

Example:

```json
{
  "telemetry": {
    "enabled": true,
    "max_session_bytes": 5242880,
    "keep_sessions": 250
  }
}
```
<!-- /wikitool:reference-config-telemetry -->

`wikitool doctor` reports the current state: on or off, why, and how much has been recorded against
both caps. What the caps do and what never reaches a trace:
[telemetry-and-evaluation.md](../explanation/telemetry-and-evaluation.md).

The telemetry hook runs on every tool call and stays silent, so it reads only this section, leniently,
and never exits 42: a `.wikitool-telemetry.json` left over from an earlier version is ignored there
([below](#files-of-earlier-versions)).

## Allowed push URLs

The `remotes` section of `.wikitool-local.json` restricts where `publish` may push.

<!-- wikitool:reference-config-remotes -->
| Field | Tier | Type | Required | Default | Description |
|-------|------|------|----------|---------|-------------|
| `allowed_push_urls` | checkout, gate | array of strings | yes | - | The push URLs `publish` may write to, compared with the URL the remote resolves to for a push - not with its name. An empty list allows no push at all. |

- **Absent:** Any remote is allowed; the Publish-Remote Gate does not apply.
- **Malformed:** `publish` stops with an error before committing anything. A broken section is never read as "no restriction".

Example:

```json
{
  "remotes": {
    "allowed_push_urls": [
      "<url>"
    ]
  }
}
```
<!-- /wikitool:reference-config-remotes -->

With the section, `publish` refuses with exit 42 (the Publish-Remote Gate) before pushing to a URL not
on the list. Only you edit it: the field is marked `gate`, so `config set` exits 42 for it. The gate
has no token and no flag, and an agent that edits the list to get past a refusal is opening a gate on
its own initiative.

## Task tracker

**Optional.** The weekly review (`tools/wikitool review`, skill `gtd-weekly-review`) checks the
project pages under `kb/gtd/` against a task tracker. Which tracker that is, and its connection
details, are three sections: `tasks` (which provider, the review's thresholds), and the section of the
provider chosen, `superproductivity` or `caldav`. What is the same in every clone - the provider, the
thresholds, the list names - sits in `wikitool.json`; the host, the user name and the folder this
machine uses in `.wikitool-local.json`; a token or password is a reference to a secret, never a value.

<!-- wikitool:reference-config-tasks -->
| Field | Tier | Type | Required | Default | Description |
|-------|------|------|----------|---------|-------------|
| `provider` | instance | string | yes | - | Which adapter the review goes through. |
| `stalled_waiting_days` | instance | positive integer | yes | - | A waiting-for item older than this many days counts as overdue. A number written as a string is accepted. |
| `unpaged_project_weeks` | instance | positive integer | yes | - | A tracker project with no `kb/` page is reported once it is older than this many weeks. A number written as a string is accepted. |
| `someday_stale_months` | instance | positive integer | yes | - | A someday/maybe item untouched for this many months counts as stale. A number written as a string is accepted. |

- **Absent:** No tracker is configured - a valid state. The weekly review has nothing to check against.
- **Malformed:** An error, and `doctor` reports `FAIL`. A broken section is never read as "no tracker configured".

Example:

```json
{
  "tasks": {
    "provider": "superproductivity",
    "stalled_waiting_days": 14,
    "unpaged_project_weeks": 3,
    "someday_stale_months": 5
  }
}
```
<!-- /wikitool:reference-config-tasks -->

The thresholds are configuration, not schema. A value the provider cannot supply at all - a
waiting item with no follow-up date, a tracker project with no creation date - is not skipped silently
but reported as its own finding (`waiting_no_follow_up`/`project_age_unknown`).

### Super Productivity

<!-- wikitool:reference-config-superproductivity -->
| Field | Tier | Type | Required | Default | Description |
|-------|------|------|----------|---------|-------------|
| `access` | instance | string | yes | - | Which way in: `api`, the local REST API (reads the current state and carries the write path), or `snapshot`, the newest backup snapshot (read-only). |
| `api_base_url` | checkout, private | string | no | `"http://127.0.0.1:3876"` | With `access: "api"`: where the app's REST API listens. An empty value means the default. |
| `api_token` | secret | string | no | - | With `access: "api"`, required: the token from the app's settings. Only ever a reference. |
| `backups_dir` | checkout | string | no | - | With `access: "snapshot"`, required: the directory the app writes its backup snapshots to; `~` is expanded. |

- **Absent:** Read only when `tasks.provider` is `superproductivity`.
- **Malformed:** An error when it is read: a field of the other access path beside `access`, or a missing `api_token` or `backups_dir`, stops the command with exit 1.

Examples:

```json
{
  "superproductivity": {
    "access": "api",
    "api_base_url": "http://127.0.0.1:3876",
    "api_token": {
      "ref": "sp_token"
    }
  }
}
```

```json
{
  "superproductivity": {
    "access": "snapshot",
    "backups_dir": "~/.config/superProductivity/backups"
  }
}
```
<!-- /wikitool:reference-config-superproductivity -->

`access` is **mandatory and has no fallback**. The fields of the other access path are an error, not ignored: a `backups_dir` beside `access: "api"` stops the command with exit 1, as does an `api_token` missing from it. It chooses between two mutually exclusive ways in:

- **`access: "snapshot"`** for an instance run headless: it reads only the newest backup snapshot
  under `backups_dir`. That works without the app running, but only reads. The tracker cannot be
  written from there.

- **`access: "api"`** for a desktop instance: it talks only to the local REST API. That answers only
  while the app runs, but it gives the current state and carries the write path. `api_token` is
  required (as a reference), because every endpoint except `GET /health` demands `Authorization: Bearer <token>`.

What can write:

- `tools/wikitool new project` creates a tracker project of the same name only with `access: "api"`,
  and even there not automatically. The command stops (exit 42) and asks a human to create it, then
  continues with `--resume`.
- `tools/wikitool task new` creates a single item without a `kb/` page.
- `tools/wikitool task close --id` marks an item done. It never deletes one.

With `access: "snapshot"`, all three refuse completely with exit 1, because the tracker is read-only
from there. `tools/wikitool task list --project` only reads, so it answers with either `access`.

### CalDAV

<!-- wikitool:reference-config-caldav -->
| Field | Tier | Type | Required | Default | Description |
|-------|------|------|----------|---------|-------------|
| `url` | checkout, private | string | yes, not empty | - | The account's CalDAV calendar home set. A trailing `/` is added when missing. |
| `username` | checkout, private | string | yes, not empty | - | The login name, which can differ from the user id in the URL. |
| `app_password` | secret | string | yes, not empty | - | An app password is recommended over the account password. Only ever a reference. |
| `inbox_list` | instance | string | yes, not empty | - | The name of the one inbox list. |
| `someday_list` | instance | string | yes, not empty | - | The name of the one someday/maybe list. |
| `exclude_lists` | instance | array of strings | no | `[]` | Existing plain task lists that are not projects. |

- **Absent:** Read only when `tasks.provider` is `caldav`.

Example:

```json
{
  "caldav": {
    "url": {
      "ref": "op://Chemenu/caldav/url"
    },
    "username": "<login>",
    "app_password": {
      "ref": "caldav_app_password"
    },
    "inbox_list": "Inbox",
    "someday_list": "Someday",
    "exclude_lists": [
      "..."
    ]
  }
}
```
<!-- /wikitool:reference-config-caldav -->

`caldav` is the standards-based second adapter (RFC 4791/5545). It is verified against Nextcloud
Tasks, with iOS *Reminders* as the mobile client, and built for that split: tick items off on the
phone, maintain them at the desk. Unlike Super Productivity, there is only one way in. CalDAV is
always a network access, so no `access` field is needed.

- `url` may be an alias - Nextcloud accepts a short name there - because every later address comes
  only from the `href`s the server returns, never from this URL joined with a name.
- `inbox_list` and `someday_list` name the two fixed lists, exactly one of each per instance.
- A list with a `VEVENT` component never counts as a project, so `exclude_lists` is needed only for
  plain task lists.

A project is a list whose only supported component is `VTODO`. `tools/wikitool new project` creates
it automatically with `MKCALENDAR` - without asking, unlike Super Productivity, because CalDAV has a
real create command for lists. The uniqueness check runs against **every** list in the account,
including excluded lists, the inbox, someday and mixed calendars. If a new name collides with any of
them, nothing is created and the collision is named. Renaming the existing list in Nextcloud stays a
manual step.

`task new` and `task close` work without restriction on `caldav`, which has no read-only mode like
`access: "snapshot"`. When ticking an item off, `task close` changes only `STATUS`, `COMPLETED`,
`PERCENT-COMPLETE`, `LAST-MODIFIED` and `DTSTAMP` on the existing `.ics` resource. Every other
property, including an unknown `X-` property or an alarm, is kept unchanged. A resource that changed
since it was read (an ETag conflict) is not written, and the command stops with exit 1. Nothing is
ever deleted. The lists of finished projects stay after archiving, because the adapter never deletes
a list. Remove them by hand in Nextcloud when you want to.

### Profiles

`WIKITOOL_TASKS_CONFIG` points at a profile: a file in the format of `.wikitool-local.json` that holds
only the sections `tasks`, `superproductivity` and `caldav` (and `schema`, and `live_test_version`
for the live suite). While it is set, `task`, `review` and `doctor` read those three sections from the
profile instead of from `wikitool.json` and `.wikitool-local.json`, so one checkout can be run against
several trackers in turn. A profile is checked like the local file, except that a plain instance-tier
value is allowed in it; a secret is a reference there too, never a value. The live suite takes its
throwaway tokens from `WIKITOOL_SECRET_<NAME>`.

## Intake channels

The `intake` section of `.wikitool-local.json` declares the ways a document gets into the
quarantine `quarantine/` from somewhere other than this terminal. Each channel has a name and a
handler:

| Handler | Source | How it arrives |
|---|---|---|
| `mcp` | The MCP read server's optional `submit` tool | Pushed by a remote caller. At most one channel has this handler. |
| `folder` | A directory outside the instance that a sync client fills (OneDrive and the like) | Pulled by `wikitool upload collect`; what is taken is removed from the folder. |
| `command` | A program of this checkout, named in `commands` | Pulled by `wikitool upload collect`; the program leaves its files in an empty staging directory. |

All of them end in the same place: a submission nobody has looked at, which `wikitool upload accept`
promotes into `incoming/` after a human has cleared the Upload Review Gate. What the authentication
in front of the server has to guarantee for the `mcp` handler is in
[Set up the MCP read server](../how-to/set-up-the-mcp-server.md#optional-the-submit-path); how to
collect from a synced folder or a program is in
[Collect documents from a synced folder](../how-to/collect-from-a-synced-folder.md). Every field is
marked `gate` or `names a command`: the channels are the user's to set by hand, and `config set`
exits 42 for them.

<!-- wikitool:reference-config-intake -->
| Field | Tier | Type | Required | Default | Description |
|-------|------|------|----------|---------|-------------|
| `channels` | checkout, gate | object | yes | - | Channel name (lower-case letters, digits and hyphens) to channel. The name goes into the manifest of what the channel delivers and is the argument of `upload collect`. Every channel has `handler` (`mcp`, `folder` or `command`), `max_bytes` (the largest single file) and `allowed_extensions` (each starting with `.`, compared case-insensitively). Then per handler: `mcp` takes `identity_header` (the request header the authenticating middleware sets to the submitter's identity, default `X-Forwarded-User`) and, required, `quota_submissions_per_day` and `quota_bytes_per_day` (per submitter, over any rolling 24 hours); `folder` takes `path` (required: an absolute path outside the instance, read without descending into subdirectories; it must not be or lie in `<backup.target>/git` or `<backup.target>/<machine>`, and `upload collect` refuses a folder that holds or lies in any machine folder, bare repository or export) and `settle_seconds` (default 60: a file whose modification or creation time is younger is left alone); `command` takes `command` (required: the name of an entry of `commands`). At most one channel has the handler `mcp`. A field the handler does not take is an error. |
| `commands` | checkout, names a command | object | no | `{}` | Command name to command: an array of the program and its arguments, run by the `command` handler without a shell, with stdin closed and a timeout of 300 s, and a staging directory appended as the last argument. Exit 0 turns each regular file at the top level of that directory into a submission; anything else takes nothing over and prints the end of the program's stderr. |

- **Absent:** No channel: the `submit` tool is not registered and `upload collect` has nothing to collect. A valid state; `doctor` reports `OK`.
- **Malformed:** An error. The MCP server does not start, `upload collect` stops with exit 1 and `doctor` reports `FAIL`. A broken section is never read as "no channel".

Example:

```json
{
  "intake": {
    "channels": {
      "mcp": {
        "handler": "mcp",
        "identity_header": "X-Forwarded-User",
        "max_bytes": 10485760,
        "allowed_extensions": [
          ".md",
          ".pdf"
        ],
        "quota_submissions_per_day": 20,
        "quota_bytes_per_day": 52428800
      },
      "onedrive": {
        "handler": "folder",
        "path": "C:\\Users\\<user>\\OneDrive - <company>\\Chemenu-Eingang",
        "max_bytes": 52428800,
        "allowed_extensions": [
          ".pdf",
          ".docx",
          ".msg",
          ".eml",
          ".md"
        ],
        "settle_seconds": 60
      },
      "scanner": {
        "handler": "command",
        "command": "scanner-pull",
        "max_bytes": 52428800,
        "allowed_extensions": [
          ".pdf"
        ]
      }
    },
    "commands": {
      "scanner-pull": [
        "pwsh",
        "-NoProfile",
        "-File",
        "C:\\tools\\scanner-pull.ps1"
      ]
    }
  }
}
```
<!-- /wikitool:reference-config-intake -->

Size and quota are your deployment's decision, not defaults worth copying blindly. A field that does
not belong to the channel's handler is an error, not ignored: `identity_header` and the `quota_*`
fields belong to `mcp`, `path` and `settle_seconds` to `folder`, `command` to `command`. A `folder`
`path` must be absolute and lie outside the instance, and `upload collect` reads only its top level.

## Site handlers

`tools/wikitool raw fetch` knows how some websites serve their pages. For a URL of such a website a
*site handler* fetches the page in the website's standard form, leaves the page furniture out of the
derived text and prints hints about the page as `CHECK` lines; today there are two, both for
`learn.microsoft.com`: `learn-microsoft` for every page of the host and
`learn-microsoft-powershell-module` for a PowerShell module's overview page, which offers the cmdlet
pages it lists as subpages for `raw fetch --scope`. Handlers are JSON files: the stack's, and the instance's own under `sites/`
(see [How to write a site handler](../how-to/write-a-site-handler.md)).
The `sites` section is what the instance decides about the handlers, and it is optional: without it every
handler applies, in the locale `en-us`.

- `disabled` switches handlers off by name. A page of that website is then handled as any other:
  fetched under the URL as typed, with its page furniture in the derived text. A name no handler has
  is an error, so a typo cannot leave a handler on. Switching off a special handler leaves the
  general one of the same website in force.
- `locale` is the language segment a handler for a multi-language website puts into the URL it
  fetches. The header of the derived text keeps the URL as typed under `requested_url`.
- `notes` adds hint lines of the instance's own, printed after a fetch or an `--html` derivation of
  a page of that host, whether or not a handler exists for it. An entry names a `host` (exact, lower
  case), optionally a `path` (a regular expression matched against the start of the URL path, after
  the standard form was applied) and one to three single-line `lines`. A host and path pattern appears
  once; an entry whose `path` is left out applies to the whole host. A line is plain text and is
  output as it is, so it is for a reminder to the reader of the capture, never for an instruction
  the tool should carry out.
- `max_pages` and `pause_seconds` bound `raw fetch --scope`: the most subpages one page may offer
  before the fetch is refused, and the pause before each subpage request, which keeps the load on
  the website low.

<!-- wikitool:reference-config-sites -->
| Field | Tier | Type | Required | Default | Description |
|-------|------|------|----------|---------|-------------|
| `disabled` | instance | array of strings | no | `[]` | The names of the site handlers that do not apply. `raw fetch` then treats a page of that website as it does any other. A name no handler has is an error when `raw fetch` reads the section, so a typo cannot leave a handler on. |
| `locale` | instance | string | no | `"en-us"` | The locale a handler for a multi-language website fetches, in the website's own spelling and lower case. An empty value means the default. |
| `notes` | instance | array of objects | no | `[]` | Hint lines of the instance's own, printed as `CHECK` lines after a fetch or an `--html` derivation of a page of that host. Each entry has `host` (exact, lower case), an optional `path` (a regular expression matched against the start of the URL path) and `lines` (one to 3 single-line strings of at most 300 characters). A host and path pattern appears once. |
| `max_pages` | instance | positive integer | no | `200` | The most subpages `raw fetch --scope` fetches for one page. A page that offers more is refused before the first subpage is requested. A number written as a string is accepted. |
| `pause_seconds` | instance | number | no | `1` | The pause, in seconds, before each subpage request of `raw fetch --scope`. `0` means none. A number written as a string is accepted. |

- **Absent:** Every handler applies, with the locale `en-us` and no further hint lines.
- **Malformed:** An error when `raw fetch` reads it: it stops with exit 1 before fetching anything. A broken section is never read as "no handlers".

Example:

```json
{
  "sites": {
    "disabled": [
      "learn-microsoft-powershell-module"
    ],
    "locale": "en-us",
    "max_pages": 200,
    "pause_seconds": 1,
    "notes": [
      {
        "host": "learn.microsoft.com",
        "path": "^/[a-z-]+/azure/",
        "lines": [
          "Check the Azure retirement notice at the top of the page."
        ]
      }
    ]
  }
}
```
<!-- /wikitool:reference-config-sites -->

All three fields are instance-tier: what a website needs cleaned is the same in every clone, and none
of them names a program or opens a gate. A section that does not read correctly stops `raw fetch`
with exit 1 before it fetches anything.

## Fetch routes

`tools/wikitool raw fetch` fetches every page the same way unless the `fetch` section says
otherwise for its host: anonymously, with no cookie and no browser. `routes` maps a host to another
way, and is instance-tier, so every clone fetches the host the same way:

- `"session"` fetches the host's pages with the session the user stored with
  `tools/wikitool raw login <host>`, through the same built-in fetch. The host needs a site handler
  with a `session` block - its login page and how a page shows it arrived signed in
  ([sites/CONTRACT.md](../../sites/CONTRACT.md)). No stored session, or a page without that sign,
  stops the fetch with exit 42 for the user to sign in again. The session is a secret: it is
  stored in the secrets file [above](#tiered-settings) under `session_<host>` (every run of other
  characters as one `_`, so `www.example.org` is `session_www_example_org`), or comes from
  `WIKITOOL_SECRET_SESSION_<HOST>`, and no settings field refers to it. It is never written to
  1Password.
- Any other value names a fetcher of `fetchers`: a program of this checkout, run without a shell
  with the URL appended. `fetchers` names a program to run, so it lives only in
  `.wikitool-local.json`, written by hand - `config set` refuses it with exit 42, and a fetcher in
  `wikitool.json` is an error. A route to a fetcher this checkout does not define stops a fetch of
  that host with exit 1, so a clone without the program fails loudly instead of fetching
  anonymously.

The whole procedure, with the browser component it needs for signing in:
[fetch-pages-behind-a-login.md](../how-to/fetch-pages-behind-a-login.md).

<!-- wikitool:reference-config-fetch -->
| Field | Tier | Type | Required | Default | Description |
|-------|------|------|----------|---------|-------------|
| `routes` | instance | object | no | `{}` | Host (exact, lower case) to route: `"session"` fetches with the user's session - the host needs a site handler that names its login page - and any other value names a fetcher of `fetchers`. A host not listed is fetched as before. |
| `fetchers` | checkout, names a command | object | no | `{}` | Fetcher name to command: an array of the program and its arguments, run without a shell with the URL appended. Exit 0 and the page's HTML on stdout is a fetch; anything else stops `raw fetch` with exit 1 and the program's stderr. |

- **Absent:** Every page is fetched by the built-in anonymous fetch, as before.
- **Malformed:** An error when `raw fetch` or `raw login` reads it: the command stops with exit 1 before fetching anything. A route to a fetcher this checkout does not define stops a fetch of that host the same way.

Example:

```json
{
  "fetch": {
    "routes": {
      "www.heise.de": "session",
      "intranet.example.org": "homelab"
    },
    "fetchers": {
      "homelab": [
        "/usr/local/bin/fetch-page",
        "--html"
      ]
    }
  }
}
```
<!-- /wikitool:reference-config-fetch -->

## Backup to a synced folder

The `backup` section of `.wikitool-local.json` backs this checkout up into a folder a sync client
uploads - OneDrive and the like - while the working tree itself stays outside it. The target gets
two kinds of content:

```
<target>/git/          bare repository, the checkout's origin; publish pushes here
<target>/<machine>/    mirror of this machine's files that are not in git, plus .chemenu-backup.json
```

`tools/wikitool backup init` creates `<target>/git` and makes it `origin` when the checkout has none.
With `machine_data` on, every `publish` that pushed mirrors `.wikitool-local.json`, `ENVIRONMENT.md`,
what waits in `incoming/` and the `extra` paths into `<target>/<machine>/`; `tools/wikitool backup
run` does the same on its own. The secrets file is never among them - it lies outside the instance,
and a backup reads nothing outside it. Plain private values in `.wikitool-local.json`, an internal
host name for one, do reach the synced folder.

<!-- wikitool:reference-config-backup -->
| Field | Tier | Type | Required | Default | Description |
|-------|------|------|----------|---------|-------------|
| `target` | checkout, gate | string | yes | - | The backup folder: an absolute path outside the instance. `backup init` creates `<target>/git` in it, `backup run` writes only to `<target>/<machine>/`. Everything else in the folder is the sync folder's: an `export` target or an `intake` folder may sit beside `git` and the machine folders (`<target>/Export`, `<target>/Inbox`), but none may be, or lie in, one of them. |
| `machine_data` | checkout, gate | boolean | no | `false` | Mirror this machine's files after every `publish` that pushed, and with `backup run`: `.wikitool-local.json`, `ENVIRONMENT.md`, what waits in `incoming/`, and `extra`. |
| `machine_name` | checkout | string | no | the host name | The name of this machine's folder under the target. Every character outside letters, digits, `-` and `_` becomes `-`, and leading and trailing `-` are dropped; a name that is empty afterwards, or `git` in any case, is an error. |
| `extra` | checkout, gate | array of strings | no | `[]` | Further files or directories to mirror, relative to the instance. One that resolves outside the instance, or into `.git/`, is an error. |

- **Absent:** No backup: `publish` behaves as without it, and `backup init` and `backup run` stop with exit 1 naming the section.
- **Malformed:** An error. `publish` stops with exit 1 before it commits anything, `backup init` and `backup run` stop with exit 1, and `doctor` reports `FAIL`. A broken section is never read as "no backup".

Example:

```json
{
  "backup": {
    "target": "C:\\Users\\<user>\\OneDrive - <company>\\Chemenu-Backup",
    "machine_data": true,
    "machine_name": "WORK-LAPTOP",
    "extra": [
      ".claude/settings.local.json"
    ]
  }
}
```
<!-- /wikitool:reference-config-backup -->

`target`, `machine_data` and `extra` decide what leaves this machine, so they are marked `gate`:
`config set` exits 42 for them, and only you write them. The procedure, restoring included:
[back-up-to-a-synced-folder.md](../how-to/back-up-to-a-synced-folder.md).

## Export to a folder

The `export` section of `.wikitool-local.json` declares folders that hold a derived, read-only
version of `kb/` - for a reader outside the repository, such as a Microsoft 365 tenant that indexes
a OneDrive folder. An export is regenerated from the committed pages on every run and is never
edited; what someone changes in the folder is refused or overwritten, never merged back. Each
target has a handler: `markdown` writes the selected pages as Markdown files, `command` renders
them into a staging directory and hands that to a converter of this checkout.

<!-- wikitool:reference-config-export -->
| Field | Tier | Type | Required | Default | Description |
|-------|------|------|----------|---------|-------------|
| `targets` | checkout, gate | object | yes | - | Target name (lower-case letters, digits and hyphens) to target; the name is the argument of `export run`. Every target has `handler` (`markdown`: the pages as Markdown files; `command`: the same Markdown run through a converter of this checkout), `path` (an absolute path outside the instance; it must not lie in or contain another target or an `intake` folder, and not be, lie in or contain `<backup.target>/git` or `<backup.target>/<machine>`, nor lie in any other machine's folder or a bare repository) and `include` (the string `"all"`: every page `search` knows, a page directly under `kb/` included and no generated or meta file; or an array of selections, each an array of `search --field` predicates: all predicates of a selection must hold, and a page that fits any selection is exported. A missing or empty `include` is an error, never "all"). Optional: `frontmatter` (`minimal`: only the export marker; `summary`, the default: also `title`, `type`, `summary` and `modified` where the page has them; `full`: the page's whole frontmatter), `links` (`relative`, the default: a wikilink to an exported page becomes a relative Markdown link; `text`: plain text; a link to a page that is not exported is always plain text), `index` (default false; true: also write `index.md` in the target's root and an `INDEX.md` per collection, and per area the catalog gives a file of its own, listing the exported pages only, with relative Markdown links whatever `links` says) and `after_publish` (default false: refresh this target after every `publish` that committed). The handler `command` takes `command` (required: the name of an entry of `commands`). A field the handler does not take is an error. |
| `commands` | checkout, names a command | object | no | `{}` | Command name to command: an array of the program and its arguments, run by the `command` handler without a shell, with stdin closed and a timeout of 300 s. The last two arguments appended are the directory holding the Markdown and an empty output directory. Exit 0 makes the regular files in the output directory the target's content; anything else leaves the target unchanged and prints the end of the program's stderr. |

- **Absent:** No export: `export run` stops with exit 1 naming the section, `publish` exports nothing and `doctor` reports `OK`.
- **Malformed:** An error. `export run` stops with exit 1 and `doctor` reports `FAIL`; `publish` stops with exit 1 before it commits anything, but only when some target sets `after_publish`. A broken section is never read as "no export".

Example:

```json
{
  "export": {
    "targets": {
      "work-notes": {
        "handler": "markdown",
        "path": "C:\\Users\\<user>\\OneDrive - <company>\\Chemenu-Lesefassung",
        "include": [
          [
            "kind=system"
          ],
          [
            "tags=shared",
            "!draft"
          ]
        ],
        "frontmatter": "summary",
        "links": "relative",
        "after_publish": true
      },
      "whole-kb": {
        "handler": "markdown",
        "path": "C:\\Users\\<user>\\OneDrive - <company>\\Chemenu-KB",
        "include": "all",
        "index": true
      },
      "as-word": {
        "handler": "command",
        "command": "pandoc-docx",
        "path": "C:\\Users\\<user>\\OneDrive - <company>\\Chemenu-Word",
        "include": [
          [
            "kind=system"
          ]
        ]
      }
    },
    "commands": {
      "pandoc-docx": [
        "pwsh",
        "-NoProfile",
        "-File",
        "C:\\tools\\pandoc-docx.ps1"
      ]
    }
  }
}
```
<!-- /wikitool:reference-config-export -->

Within one target, `include` is either the string `"all"` or a list of selections, each a list of
`search` predicates (`tags=docs`, `entity_type=system`, `!source_url`): a page is exported when it
satisfies every predicate of at least one selection. `"all"` selects every page `search` knows without
a predicate - one directly under `kb/` too, never a generated or meta file (`kb/index.md`, `kb/log.md`,
`kb/provenance.md`, `kb/CONTRACT.md`, `kb/CONVENTIONS.md`, an `INDEX.md`, a `COLLECTION.md`). Only what
`include` selects is written - there is no way to exclude by default, and a missing or empty `include`
is an error, never "all". The syntax is checked when the section is read, the field names against the
corpus when `export run` runs. `path` must not lie in or contain the instance, another target, an
`intake` folder or `backup.target`.

With `"index": true` a run also writes `index.md` in the root of the target and an `INDEX.md` per
collection - and per area the catalog under `kb/` would give a file of its own. They list the exported
pages only (Page, Type, Summary, Last Modified), link with relative Markdown links whatever `links`
says, and carry no date of the day: Last Modified is the page's own `modified`, `date` or `created`,
empty where it has none. The column and heading labels are English. Each starts with a frontmatter
block whose only key is `wikitool_export` (`kind=index`), so `raw accept` refuses it like a page. A
page `<collection>/index.md` beside the generated `<collection>/INDEX.md` stops the target, since a
synced folder on Windows would make them one file.

Every Markdown file starts with a frontmatter block whose first key is `wikitool_export`, carrying
the kind, the instance and the newest commit that touched that page. `frontmatter` decides what follows:
`minimal` nothing, `summary` the title, type, summary and modified date, `full` the page's whole
frontmatter. Wikilinks between exported pages become relative Markdown links (`links: relative`) or
plain text (`links: text`); a link to a page that is not exported is always plain text. Footnote
markers and code stay as they are.

A target folder holds `.wikitool-export.json`, which lists the files a run wrote with their digests.
A run removes only listed files that are no longer exported, creates only folders it needs and
removes only those it created, and leaves every other file untouched. It stops before writing for a
listed file someone changed, for a folder that is not empty and has no manifest, and for a selection
that comes up empty. A run that finds nothing to change writes nothing, not even a modification time.

`targets` and `commands` decide what leaves this checkout and which program runs, so they are marked
`gate` and `names a command`: `config set` exits 42 for them, and only you write them. `raw accept`
also refuses a file whose sha256 is listed in the manifest of a configured target, for an export
that went through a converter and lost its marker. The procedure:
[export-to-a-synced-folder.md](../how-to/export-to-a-synced-folder.md).

## Files of earlier versions

Until version 10 every feature above had a file of its own at the root: `.wikitool-telemetry.json`,
`.wikitool-remotes.json`, `.wikitool-upload.json` and `.wikitool-tasks.json`. Nothing reads them any
more. A command that needs the setting such a file held stops with exit 42 and names
`tools/wikitool config migrate`, `doctor` reports each one as a `FAIL`, and only the telemetry hook
ignores it. `config migrate` moves the content into the sections above (the upload file becomes the channel `mcp` of `intake`), each value into the file its
tier names and each secret into the secrets file, and then removes the old files; it is run once per
checkout. `.wikitool-tools.json` is not among them and stays. The migration step for an instance is
[10.0.0-settings-tiers.md](../../instructions/migrations/10.0.0-settings-tiers.md).