# Configuration The environment variables `wikitool` reads and the two tiered settings files an operator may create at the root of an instance. None of them is needed for an instance to work: an instance installed from a release runs against the origin's public release feed with no configuration at all. Every setting of a feature - telemetry, the publish allowlist, the task tracker, the MCP `submit` limits - is a section of those two files, and the shape of each section - its fields, their tier, what an absent or a malformed section means, an example - is generated from the record `wikitool` itself reads it with, so it cannot describe a field the tool does not take. What a setting does around it is written by hand. Three more files at the root are written by `wikitool` and never by hand: `.wikitool-release.json`, `.wikitool-kb.json` and `.wikitool-tools.json` ([AGENTS.md invariant 1](../../AGENTS.md#invariants)). The last one has its own section [below](#tool-paths), because the preflight that writes it is something you run. ## Contents - [Environment variables](#environment-variables) - [Tool paths](#tool-paths) - [Tiered settings](#tiered-settings) - [Telemetry](#telemetry) - [Allowed push URLs](#allowed-push-urls) - [Task tracker](#task-tracker) - [Super Productivity](#super-productivity) - [CalDAV](#caldav) - [Profiles](#profiles) - [Intake channels](#intake-channels) - [Site handlers](#site-handlers) - [Fetch routes](#fetch-routes) - [Backup to a synced folder](#backup-to-a-synced-folder) - [Files of earlier versions](#files-of-earlier-versions) ## Environment variables | Variable | Purpose | Fallback | |----------|---------|----------| | `WIKI_AUTHOR` | Overrides the author name of new source pages | `git config user.name`. Without either, `new` stops with an `ERROR` | | `WIKITOOL_SESSION_ID` | Scopes the Iteration Budget Gate to a task rather than a terminal, and names the directory a trace lands in | A session variable the harness sets itself, where one is known (for example `CLAUDE_CODE_SESSION_ID`), else the parent process id ([instructions/session-setup.md](../../instructions/session-setup.md)) | | `WIKITOOL_UPDATE_URL` | The release feed `version check`, `version notes` and `dist upgrade --latest` ask | The value in `.wikitool-release.json`, else the origin's feed | | `WIKITOOL_UPDATE_TOKEN` | A Gitea token for the release feed, sent only to downloads on the feed's own host | None. Not needed against the origin, only against a feed in a private repository | | `WIKITOOL_TASKS_CONFIG` | Path to a tracker profile that `task`, `review` and `doctor` read instead of the `tasks`, `superproductivity` and `caldav` sections of the settings, for running one checkout against several trackers in turn ([below](#profiles)) | The sections in `wikitool.json` and `.wikitool-local.json`. A path to a file that does not exist is an error, never "no tracker configured" | | `WIKITOOL_SECRET_` | The value behind a settings reference `{"ref": ""}`, for CI and containers - `` is the name upper-cased | The secrets file in the user's configuration directory ([below](#tiered-settings)) | | `CHEMENU_ROOT` | Which corpus the package works on, for a caller outside the checkout itself | The checkout the package sits in. `tools/wikitool` behaves the same without it | | `WIKI_TRACE` | `0`/`1` turns telemetry off or on, beating both the default and the `telemetry` section | The installation-form default ([below](#telemetry)) | | `WIKI_TRACE_DIR` | Writes traces somewhere other than `reports/telemetry/` | `reports/telemetry/` | | `WIKI_TRACE_CONTENT` | `0` keeps only lengths and digests of prompts and replies instead of their text | Text is kept | | `WIKI_TRACE_MAX_CONTENT` | Per-attribute content cap in characters | 60 KiB | | `WIKI_TRACE_MAX_SESSION_BYTES` | Byte cap per session trace; beats the section's `max_session_bytes` | 5 MiB | | `WIKI_TRACE_KEEP_SESSIONS` | How many session directories retention keeps; beats the section's `keep_sessions` | 250 | ## Tool paths **The preflight writes this file, not you.** `.wikitool-tools.json` at the root holds the absolute paths of Python, git and ripgrep, as the preflight found them on this machine - and of the 1Password CLI `op` when it is installed. `op` is optional: the preflight records it when it finds it, reports `ABSENT` when it does not, and never asks for it. Only an `op://` reference in the [tiered settings](#tiered-settings) needs it. `wikitool` starts git and rg from there instead of from `PATH`. It is per checkout and gitignored, because a path on one machine says nothing about the next. If the file is missing or incomplete, `tools/wikitool` does not start (exit 42) and names the preflight. If a tool sits somewhere the preflight does not look, name it: `tools/preflight.sh --set rg=`, or in PowerShell `pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1 --set rg=`. Never edit the file by hand. `doctor` reports under `tool-paths` whether every path still works. ## Tiered settings Two files at the root hold the settings of the features that read them this way, and every field declares the tier it belongs to: | Tier | File | Holds | |------|------|-------| | `instance` | `wikitool.json`, committed | What belongs to the corpus and is the same in every clone | | `checkout` | `.wikitool-local.json`, gitignored | What is true only for this clone on this machine | | `secret` | neither | Tokens, passwords and website sessions - both files hold only a reference, and a session from `raw login` none ([below](#fetch-routes)) | Each file is a JSON object of sections, one per feature, beside two keys of its own: ```json { "schema": 1, "instance_id": "6f1c0e52-3d1b-4b8e-9a51-2f0e7d1c9b44", "
": {"": ""} } ``` `instance_id` sits in `wikitool.json` only. The first `wikitool config set` creates the file and the id; the id names the directory of the secrets file below. The sections are [telemetry](#telemetry), [remotes](#allowed-push-urls), [tasks, superproductivity and caldav](#task-tracker) and [intake](#intake-channels), [sites](#site-handlers), [fetch](#fetch-routes) and [backup](#backup-to-a-synced-folder). `config set` names the sections it knows, and `doctor` reports a section it does not read as a `WARN`. **One place per value.** A value in the file its tier does not name is an error naming the file, the key and the right tier - never an override. The same holds for a key set in both files. **References.** A field of tier `secret` takes only a reference; a field marked private - a host name, an internal URL - takes a plain value in `.wikitool-local.json` or a reference in either file. No other field takes one. | Reference | Resolves through | |-----------|------------------| | `{"ref": ""}` - lower-case letters, digits and single `_` | `WIKITOOL_SECRET_`, else the key `` in the secrets file | | `{"ref": "op:////"}` | `op read`, the 1Password CLI the preflight recorded | The secrets file is `~/.config/chemenu//secrets.json` (`$XDG_CONFIG_HOME` instead of `~/.config` when it is set; `%APPDATA%\chemenu\\secrets.json` on Windows), a JSON object of names to strings. Outside Windows it must be readable by its owner only (`chmod 600`); a file others can read is refused. A reference is resolved when a command asks for that value, never when the files are read. A name nobody provides, an `op` that is not installed, or an item 1Password does not have ends the command with exit 1; a 1Password that is locked, signed out or waiting for an approval ends it with exit 42, for a human to unlock. Setting 1Password up: [use-1password-for-settings.md](../how-to/use-1password-for-settings.md). **Never in the committed file:** a field that names a program to run or affects a gate - not even as a reference. `config set` does not write such a field either (exit 42); the user edits `.wikitool-local.json` by hand, and `doctor` checks it afterwards. The fields marked `gate` or `names a command` in the tables below are these: the publish allowlist, the intake channels and the external fetchers. `wikitool doctor` reports one line per tier - `settings-instance`, `settings-checkout`, `settings-secrets` - and for the references only whether they resolve. `docs verify` holds a committed `wikitool.json` to the placement rules above, a plain secret or private value above all. Neither file ships with `dist export`. Why the tiers exist, and what keeping secrets out of the tree does not protect against: [configuration-tiers.md](../explanation/configuration-tiers.md). ## Telemetry **Telemetry is off in an instance.** Every instance carries the `.wikitool-release.json` of its release, and that is how `chemenu.telemetry.policy` recognises that nobody asked for telemetry. The same holds for every further clone of the instance, because the file is committed with the first publish. Only a clone of the origin repository, used to develop the stack, has no stamp. There the traces are the instrument the stack measures itself with, and they are **on**. To turn the default around, add a `telemetry` section to `.wikitool-local.json`, by hand or with `wikitool config set telemetry.enabled true`. The consent to write prompts to this disk belongs to the checkout, so the section has no instance-tier field and no gate. | Field | Tier | Type | Required | Default | Description | |-------|------|------|----------|---------|-------------| | `enabled` | checkout | boolean | no | the installation-form default | Telemetry on or off. `WIKI_TRACE` beats it in both directions. Any other value is read by its truthiness, so `"no"` means on. | | `max_session_bytes` | checkout | positive integer | no | `5242880` | Byte cap per session trace. `WIKI_TRACE_MAX_SESSION_BYTES` beats it. A number written as a string is accepted. | | `keep_sessions` | checkout | positive integer | no | `250` | How many session directories retention keeps. `WIKI_TRACE_KEEP_SESSIONS` beats it. A number written as a string is accepted. | - **Absent:** The installation-form default applies: on in a clone of the origin repository, off in an instance. - **Malformed:** The hook that reads it never fails on it: a value it cannot use counts as absent, and an unreadable `.wikitool-local.json` as an absent section. `doctor` reports each such deviation as a `WARN` under `telemetry`, naming the field. Example: ```json { "telemetry": { "enabled": true, "max_session_bytes": 5242880, "keep_sessions": 250 } } ``` `wikitool doctor` reports the current state: on or off, why, and how much has been recorded against both caps. What the caps do and what never reaches a trace: [telemetry-and-evaluation.md](../explanation/telemetry-and-evaluation.md). The telemetry hook runs on every tool call and stays silent, so it reads only this section, leniently, and never exits 42: a `.wikitool-telemetry.json` left over from an earlier version is ignored there ([below](#files-of-earlier-versions)). ## Allowed push URLs The `remotes` section of `.wikitool-local.json` restricts where `publish` may push. | Field | Tier | Type | Required | Default | Description | |-------|------|------|----------|---------|-------------| | `allowed_push_urls` | checkout, gate | array of strings | yes | - | The push URLs `publish` may write to, compared with the URL the remote resolves to for a push - not with its name. An empty list allows no push at all. | - **Absent:** Any remote is allowed; the Publish-Remote Gate does not apply. - **Malformed:** `publish` stops with an error before committing anything. A broken section is never read as "no restriction". Example: ```json { "remotes": { "allowed_push_urls": [ "" ] } } ``` With the section, `publish` refuses with exit 42 (the Publish-Remote Gate) before pushing to a URL not on the list. Only you edit it: the field is marked `gate`, so `config set` exits 42 for it. The gate has no token and no flag, and an agent that edits the list to get past a refusal is opening a gate on its own initiative. ## Task tracker **Optional.** The weekly review (`tools/wikitool review`, skill `gtd-weekly-review`) checks the project pages under `kb/gtd/` against a task tracker. Which tracker that is, and its connection details, are three sections: `tasks` (which provider, the review's thresholds), and the section of the provider chosen, `superproductivity` or `caldav`. What is the same in every clone - the provider, the thresholds, the list names - sits in `wikitool.json`; the host, the user name and the folder this machine uses in `.wikitool-local.json`; a token or password is a reference to a secret, never a value. | Field | Tier | Type | Required | Default | Description | |-------|------|------|----------|---------|-------------| | `provider` | instance | string | yes | - | Which adapter the review goes through. | | `stalled_waiting_days` | instance | positive integer | yes | - | A waiting-for item older than this many days counts as overdue. A number written as a string is accepted. | | `unpaged_project_weeks` | instance | positive integer | yes | - | A tracker project with no `kb/` page is reported once it is older than this many weeks. A number written as a string is accepted. | | `someday_stale_months` | instance | positive integer | yes | - | A someday/maybe item untouched for this many months counts as stale. A number written as a string is accepted. | - **Absent:** No tracker is configured - a valid state. The weekly review has nothing to check against. - **Malformed:** An error, and `doctor` reports `FAIL`. A broken section is never read as "no tracker configured". Example: ```json { "tasks": { "provider": "superproductivity", "stalled_waiting_days": 14, "unpaged_project_weeks": 3, "someday_stale_months": 5 } } ``` The thresholds are configuration, not schema. A value the provider cannot supply at all - a waiting item with no follow-up date, a tracker project with no creation date - is not skipped silently but reported as its own finding (`waiting_no_follow_up`/`project_age_unknown`). ### Super Productivity | Field | Tier | Type | Required | Default | Description | |-------|------|------|----------|---------|-------------| | `access` | instance | string | yes | - | Which way in: `api`, the local REST API (reads the current state and carries the write path), or `snapshot`, the newest backup snapshot (read-only). | | `api_base_url` | checkout, private | string | no | `"http://127.0.0.1:3876"` | With `access: "api"`: where the app's REST API listens. An empty value means the default. | | `api_token` | secret | string | no | - | With `access: "api"`, required: the token from the app's settings. Only ever a reference. | | `backups_dir` | checkout | string | no | - | With `access: "snapshot"`, required: the directory the app writes its backup snapshots to; `~` is expanded. | - **Absent:** Read only when `tasks.provider` is `superproductivity`. - **Malformed:** An error when it is read: a field of the other access path beside `access`, or a missing `api_token` or `backups_dir`, stops the command with exit 1. Examples: ```json { "superproductivity": { "access": "api", "api_base_url": "http://127.0.0.1:3876", "api_token": { "ref": "sp_token" } } } ``` ```json { "superproductivity": { "access": "snapshot", "backups_dir": "~/.config/superProductivity/backups" } } ``` `access` is **mandatory and has no fallback**. The fields of the other access path are an error, not ignored: a `backups_dir` beside `access: "api"` stops the command with exit 1, as does an `api_token` missing from it. It chooses between two mutually exclusive ways in: - **`access: "snapshot"`** for an instance run headless: it reads only the newest backup snapshot under `backups_dir`. That works without the app running, but only reads. The tracker cannot be written from there. - **`access: "api"`** for a desktop instance: it talks only to the local REST API. That answers only while the app runs, but it gives the current state and carries the write path. `api_token` is required (as a reference), because every endpoint except `GET /health` demands `Authorization: Bearer `. What can write: - `tools/wikitool new project` creates a tracker project of the same name only with `access: "api"`, and even there not automatically. The command stops (exit 42) and asks a human to create it, then continues with `--resume`. - `tools/wikitool task new` creates a single item without a `kb/` page. - `tools/wikitool task close --id` marks an item done. It never deletes one. With `access: "snapshot"`, all three refuse completely with exit 1, because the tracker is read-only from there. `tools/wikitool task list --project` only reads, so it answers with either `access`. ### CalDAV | Field | Tier | Type | Required | Default | Description | |-------|------|------|----------|---------|-------------| | `url` | checkout, private | string | yes, not empty | - | The account's CalDAV calendar home set. A trailing `/` is added when missing. | | `username` | checkout, private | string | yes, not empty | - | The login name, which can differ from the user id in the URL. | | `app_password` | secret | string | yes, not empty | - | An app password is recommended over the account password. Only ever a reference. | | `inbox_list` | instance | string | yes, not empty | - | The name of the one inbox list. | | `someday_list` | instance | string | yes, not empty | - | The name of the one someday/maybe list. | | `exclude_lists` | instance | array of strings | no | `[]` | Existing plain task lists that are not projects. | - **Absent:** Read only when `tasks.provider` is `caldav`. Example: ```json { "caldav": { "url": { "ref": "op://Chemenu/caldav/url" }, "username": "", "app_password": { "ref": "caldav_app_password" }, "inbox_list": "Inbox", "someday_list": "Someday", "exclude_lists": [ "..." ] } } ``` `caldav` is the standards-based second adapter (RFC 4791/5545). It is verified against Nextcloud Tasks, with iOS *Reminders* as the mobile client, and built for that split: tick items off on the phone, maintain them at the desk. Unlike Super Productivity, there is only one way in. CalDAV is always a network access, so no `access` field is needed. - `url` may be an alias - Nextcloud accepts a short name there - because every later address comes only from the `href`s the server returns, never from this URL joined with a name. - `inbox_list` and `someday_list` name the two fixed lists, exactly one of each per instance. - A list with a `VEVENT` component never counts as a project, so `exclude_lists` is needed only for plain task lists. A project is a list whose only supported component is `VTODO`. `tools/wikitool new project` creates it automatically with `MKCALENDAR` - without asking, unlike Super Productivity, because CalDAV has a real create command for lists. The uniqueness check runs against **every** list in the account, including excluded lists, the inbox, someday and mixed calendars. If a new name collides with any of them, nothing is created and the collision is named. Renaming the existing list in Nextcloud stays a manual step. `task new` and `task close` work without restriction on `caldav`, which has no read-only mode like `access: "snapshot"`. When ticking an item off, `task close` changes only `STATUS`, `COMPLETED`, `PERCENT-COMPLETE`, `LAST-MODIFIED` and `DTSTAMP` on the existing `.ics` resource. Every other property, including an unknown `X-` property or an alarm, is kept unchanged. A resource that changed since it was read (an ETag conflict) is not written, and the command stops with exit 1. Nothing is ever deleted. The lists of finished projects stay after archiving, because the adapter never deletes a list. Remove them by hand in Nextcloud when you want to. ### Profiles `WIKITOOL_TASKS_CONFIG` points at a profile: a file in the format of `.wikitool-local.json` that holds only the sections `tasks`, `superproductivity` and `caldav` (and `schema`, and `live_test_version` for the live suite). While it is set, `task`, `review` and `doctor` read those three sections from the profile instead of from `wikitool.json` and `.wikitool-local.json`, so one checkout can be run against several trackers in turn. A profile is checked like the local file, except that a plain instance-tier value is allowed in it; a secret is a reference there too, never a value. The live suite takes its throwaway tokens from `WIKITOOL_SECRET_`. ## Intake channels The `intake` section of `.wikitool-local.json` declares the ways a document gets into the quarantine `quarantine/` from somewhere other than this terminal. Each channel has a name and a handler: | Handler | Source | How it arrives | |---|---|---| | `mcp` | The MCP read server's optional `submit` tool | Pushed by a remote caller. At most one channel has this handler. | | `folder` | A directory outside the instance that a sync client fills (OneDrive and the like) | Pulled by `wikitool upload collect`; what is taken is removed from the folder. | | `command` | A program of this checkout, named in `commands` | Pulled by `wikitool upload collect`; the program leaves its files in an empty staging directory. | All of them end in the same place: a submission nobody has looked at, which `wikitool upload accept` promotes into `incoming/` after a human has cleared the Upload Review Gate. What the authentication in front of the server has to guarantee for the `mcp` handler is in [Set up the MCP read server](../how-to/set-up-the-mcp-server.md#optional-the-submit-path); how to collect from a synced folder or a program is in [Collect documents from a synced folder](../how-to/collect-from-a-synced-folder.md). Every field is marked `gate` or `names a command`: the channels are the user's to set by hand, and `config set` exits 42 for them. | Field | Tier | Type | Required | Default | Description | |-------|------|------|----------|---------|-------------| | `channels` | checkout, gate | object | yes | - | Channel name (lower-case letters, digits and hyphens) to channel. The name goes into the manifest of what the channel delivers and is the argument of `upload collect`. Every channel has `handler` (`mcp`, `folder` or `command`), `max_bytes` (the largest single file) and `allowed_extensions` (each starting with `.`, compared case-insensitively). Then per handler: `mcp` takes `identity_header` (the request header the authenticating middleware sets to the submitter's identity, default `X-Forwarded-User`) and, required, `quota_submissions_per_day` and `quota_bytes_per_day` (per submitter, over any rolling 24 hours); `folder` takes `path` (required: an absolute path outside the instance, read without descending into subdirectories) and `settle_seconds` (default 60: a file whose modification or creation time is younger is left alone); `command` takes `command` (required: the name of an entry of `commands`). At most one channel has the handler `mcp`. A field the handler does not take is an error. | | `commands` | checkout, names a command | object | no | `{}` | Command name to command: an array of the program and its arguments, run by the `command` handler without a shell, with stdin closed and a timeout of 300 s, and a staging directory appended as the last argument. Exit 0 turns each regular file at the top level of that directory into a submission; anything else takes nothing over and prints the end of the program's stderr. | - **Absent:** No channel: the `submit` tool is not registered and `upload collect` has nothing to collect. A valid state; `doctor` reports `OK`. - **Malformed:** An error. The MCP server does not start, `upload collect` stops with exit 1 and `doctor` reports `FAIL`. A broken section is never read as "no channel". Example: ```json { "intake": { "channels": { "mcp": { "handler": "mcp", "identity_header": "X-Forwarded-User", "max_bytes": 10485760, "allowed_extensions": [ ".md", ".pdf" ], "quota_submissions_per_day": 20, "quota_bytes_per_day": 52428800 }, "onedrive": { "handler": "folder", "path": "C:\\Users\\\\OneDrive - \\Chemenu-Eingang", "max_bytes": 52428800, "allowed_extensions": [ ".pdf", ".docx", ".msg", ".eml", ".md" ], "settle_seconds": 60 }, "scanner": { "handler": "command", "command": "scanner-pull", "max_bytes": 52428800, "allowed_extensions": [ ".pdf" ] } }, "commands": { "scanner-pull": [ "pwsh", "-NoProfile", "-File", "C:\\tools\\scanner-pull.ps1" ] } } } ``` Size and quota are your deployment's decision, not defaults worth copying blindly. A field that does not belong to the channel's handler is an error, not ignored: `identity_header` and the `quota_*` fields belong to `mcp`, `path` and `settle_seconds` to `folder`, `command` to `command`. A `folder` `path` must be absolute and lie outside the instance, and `upload collect` reads only its top level. ## Site handlers `tools/wikitool raw fetch` knows how some websites serve their pages. For a URL of such a website a *site handler* fetches the page in the website's standard form, leaves the page furniture out of the derived text and prints hints about the page as `CHECK` lines; today there are two, both for `learn.microsoft.com`: `learn-microsoft` for every page of the host and `learn-microsoft-powershell-module` for a PowerShell module's overview page, which offers the cmdlet pages it lists as subpages for `raw fetch --scope`. Handlers are JSON files: the stack's, and the instance's own under `sites/` (see [How to write a site handler](../how-to/write-a-site-handler.md)). The `sites` section is what the instance decides about the handlers, and it is optional: without it every handler applies, in the locale `en-us`. - `disabled` switches handlers off by name. A page of that website is then handled as any other: fetched under the URL as typed, with its page furniture in the derived text. A name no handler has is an error, so a typo cannot leave a handler on. Switching off a special handler leaves the general one of the same website in force. - `locale` is the language segment a handler for a multi-language website puts into the URL it fetches. The header of the derived text keeps the URL as typed under `requested_url`. - `notes` adds hint lines of the instance's own, printed after a fetch or an `--html` derivation of a page of that host, whether or not a handler exists for it. An entry names a `host` (exact, lower case), optionally a `path` (a regular expression matched against the start of the URL path, after the standard form was applied) and one to three single-line `lines`. A host and path pattern appears once; an entry whose `path` is left out applies to the whole host. A line is plain text and is output as it is, so it is for a reminder to the reader of the capture, never for an instruction the tool should carry out. - `max_pages` and `pause_seconds` bound `raw fetch --scope`: the most subpages one page may offer before the fetch is refused, and the pause before each subpage request, which keeps the load on the website low. | Field | Tier | Type | Required | Default | Description | |-------|------|------|----------|---------|-------------| | `disabled` | instance | array of strings | no | `[]` | The names of the site handlers that do not apply. `raw fetch` then treats a page of that website as it does any other. A name no handler has is an error when `raw fetch` reads the section, so a typo cannot leave a handler on. | | `locale` | instance | string | no | `"en-us"` | The locale a handler for a multi-language website fetches, in the website's own spelling and lower case. An empty value means the default. | | `notes` | instance | array of objects | no | `[]` | Hint lines of the instance's own, printed as `CHECK` lines after a fetch or an `--html` derivation of a page of that host. Each entry has `host` (exact, lower case), an optional `path` (a regular expression matched against the start of the URL path) and `lines` (one to 3 single-line strings of at most 300 characters). A host and path pattern appears once. | | `max_pages` | instance | positive integer | no | `200` | The most subpages `raw fetch --scope` fetches for one page. A page that offers more is refused before the first subpage is requested. A number written as a string is accepted. | | `pause_seconds` | instance | number | no | `1` | The pause, in seconds, before each subpage request of `raw fetch --scope`. `0` means none. A number written as a string is accepted. | - **Absent:** Every handler applies, with the locale `en-us` and no further hint lines. - **Malformed:** An error when `raw fetch` reads it: it stops with exit 1 before fetching anything. A broken section is never read as "no handlers". Example: ```json { "sites": { "disabled": [ "learn-microsoft-powershell-module" ], "locale": "en-us", "max_pages": 200, "pause_seconds": 1, "notes": [ { "host": "learn.microsoft.com", "path": "^/[a-z-]+/azure/", "lines": [ "Check the Azure retirement notice at the top of the page." ] } ] } } ``` All three fields are instance-tier: what a website needs cleaned is the same in every clone, and none of them names a program or opens a gate. A section that does not read correctly stops `raw fetch` with exit 1 before it fetches anything. ## Fetch routes `tools/wikitool raw fetch` fetches every page the same way unless the `fetch` section says otherwise for its host: anonymously, with no cookie and no browser. `routes` maps a host to another way, and is instance-tier, so every clone fetches the host the same way: - `"session"` fetches the host's pages with the session the user stored with `tools/wikitool raw login `, through the same built-in fetch. The host needs a site handler with a `session` block - its login page and how a page shows it arrived signed in ([sites/CONTRACT.md](../../sites/CONTRACT.md)). No stored session, or a page without that sign, stops the fetch with exit 42 for the user to sign in again. The session is a secret: it is stored in the secrets file [above](#tiered-settings) under `session_` (every run of other characters as one `_`, so `www.example.org` is `session_www_example_org`), or comes from `WIKITOOL_SECRET_SESSION_`, and no settings field refers to it. It is never written to 1Password. - Any other value names a fetcher of `fetchers`: a program of this checkout, run without a shell with the URL appended. `fetchers` names a program to run, so it lives only in `.wikitool-local.json`, written by hand - `config set` refuses it with exit 42, and a fetcher in `wikitool.json` is an error. A route to a fetcher this checkout does not define stops a fetch of that host with exit 1, so a clone without the program fails loudly instead of fetching anonymously. The whole procedure, with the browser component it needs for signing in: [fetch-pages-behind-a-login.md](../how-to/fetch-pages-behind-a-login.md). | Field | Tier | Type | Required | Default | Description | |-------|------|------|----------|---------|-------------| | `routes` | instance | object | no | `{}` | Host (exact, lower case) to route: `"session"` fetches with the user's session - the host needs a site handler that names its login page - and any other value names a fetcher of `fetchers`. A host not listed is fetched as before. | | `fetchers` | checkout, names a command | object | no | `{}` | Fetcher name to command: an array of the program and its arguments, run without a shell with the URL appended. Exit 0 and the page's HTML on stdout is a fetch; anything else stops `raw fetch` with exit 1 and the program's stderr. | - **Absent:** Every page is fetched by the built-in anonymous fetch, as before. - **Malformed:** An error when `raw fetch` or `raw login` reads it: the command stops with exit 1 before fetching anything. A route to a fetcher this checkout does not define stops a fetch of that host the same way. Example: ```json { "fetch": { "routes": { "www.heise.de": "session", "intranet.example.org": "homelab" }, "fetchers": { "homelab": [ "/usr/local/bin/fetch-page", "--html" ] } } } ``` ## Backup to a synced folder The `backup` section of `.wikitool-local.json` backs this checkout up into a folder a sync client uploads - OneDrive and the like - while the working tree itself stays outside it. The target gets two kinds of content: ``` /git/ bare repository, the checkout's origin; publish pushes here // mirror of this machine's files that are not in git, plus .chemenu-backup.json ``` `tools/wikitool backup init` creates `/git` and makes it `origin` when the checkout has none. With `machine_data` on, every `publish` that pushed mirrors `.wikitool-local.json`, `ENVIRONMENT.md`, what waits in `incoming/` and the `extra` paths into `//`; `tools/wikitool backup run` does the same on its own. The secrets file is never among them - it lies outside the instance, and a backup reads nothing outside it. Plain private values in `.wikitool-local.json`, an internal host name for one, do reach the synced folder. | Field | Tier | Type | Required | Default | Description | |-------|------|------|----------|---------|-------------| | `target` | checkout, gate | string | yes | - | The backup folder: an absolute path outside the instance. `backup init` creates `/git` in it, `backup run` writes only to `//`. | | `machine_data` | checkout, gate | boolean | no | `false` | Mirror this machine's files after every `publish` that pushed, and with `backup run`: `.wikitool-local.json`, `ENVIRONMENT.md`, what waits in `incoming/`, and `extra`. | | `machine_name` | checkout | string | no | the host name | The name of this machine's folder under the target. Every character outside letters, digits, `-` and `_` becomes `-`, and leading and trailing `-` are dropped; a name that is empty afterwards, or `git` in any case, is an error. | | `extra` | checkout, gate | array of strings | no | `[]` | Further files or directories to mirror, relative to the instance. One that resolves outside the instance, or into `.git/`, is an error. | - **Absent:** No backup: `publish` behaves as without it, and `backup init` and `backup run` stop with exit 1 naming the section. - **Malformed:** An error. `publish` stops with exit 1 before it commits anything, `backup init` and `backup run` stop with exit 1, and `doctor` reports `FAIL`. A broken section is never read as "no backup". Example: ```json { "backup": { "target": "C:\\Users\\\\OneDrive - \\Chemenu-Backup", "machine_data": true, "machine_name": "WORK-LAPTOP", "extra": [ ".claude/settings.local.json" ] } } ``` `target`, `machine_data` and `extra` decide what leaves this machine, so they are marked `gate`: `config set` exits 42 for them, and only you write them. The procedure, restoring included: [back-up-to-a-synced-folder.md](../how-to/back-up-to-a-synced-folder.md). ## Files of earlier versions Until version 10 every feature above had a file of its own at the root: `.wikitool-telemetry.json`, `.wikitool-remotes.json`, `.wikitool-upload.json` and `.wikitool-tasks.json`. Nothing reads them any more. A command that needs the setting such a file held stops with exit 42 and names `tools/wikitool config migrate`, `doctor` reports each one as a `FAIL`, and only the telemetry hook ignores it. `config migrate` moves the content into the sections above (the upload file becomes the channel `mcp` of `intake`), each value into the file its tier names and each secret into the secrets file, and then removes the old files; it is run once per checkout. `.wikitool-tools.json` is not among them and stays. The migration step for an instance is [10.0.0-settings-tiers.md](../../instructions/migrations/10.0.0-settings-tiers.md).