settings: three tiers - wikitool.json (instance), .wikitool-local.json (checkout), secrets by reference incl. 1Password CLI; new command config set (#207)

Every settings field declares its tier; a value in the wrong file is an error naming file, key and tier. A field naming a program or affecting a gate cannot be declared instance. Secrets are references only: {"ref": "<name>"} via WIKITOOL_SECRET_<NAME> or a user-only secrets file, {"ref": "op://..."} via op read (locked 1Password: exit 42). The preflight records op as an optional tool. doctor gains one line per tier, docs verify checks a committed wikitool.json for plaintext secrets, CI resolves a real op:// value through 1password/load-secrets-action. No feature reads its settings this way yet; without the two files every other command is unchanged. Files changed: - .gitea/workflows/ci.yml - .gitignore - AGENTS.md - CHANGES.md - INSTALL.md - VERSION - docs/explanation/README.md - docs/explanation/configuration-tiers.md - docs/how-to/README.md - docs/how-to/use-1password-for-settings.md - docs/reference/commands/README.md - docs/reference/commands/configuration.md - docs/reference/commands/instance-health.md - docs/reference/commands/types-instructions-and-docs.md - docs/reference/configuration.md - docs/reference/exit-codes.md - instructions/dev/doc-pull-through.md - instructions/dev/testing-conventions.md - instructions/preflight.md - tools/CONTRACT.md - tools/README.md - tools/chemenu/cli.py - tools/chemenu/cli_contract.py - tools/chemenu/commands/_util.py - tools/chemenu/commands/config_cmd.py - tools/chemenu/commands/docs_verify.py - tools/chemenu/commands/doctor.py - tools/chemenu/config.py - tools/chemenu/config_shapes.py - tools/chemenu/prerequisites.py - tools/chemenu/settings.py - tools/chemenu/tests/conftest.py - tools/chemenu/tests/test_dist_cmd.py - tools/chemenu/tests/test_preflight.py - tools/chemenu/tests/test_preflight_pwsh.py - tools/chemenu/tests/test_settings.py - tools/chemenu/toolpaths.py - tools/preflight.ps1 - tools/preflight.sh - tools/prerequisites.txt Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
2026-10-10 15:34:28 +02:00
1 parent 059a0a1840
commit f6aaadc413
@@ -21,6 +21,7 @@ collected in [exit codes and gates](../exit-codes.md).
 - [Telemetry](#telemetry)
 - [Distribution and versioning](#distribution-and-versioning)
 - [Content migrations](#content-migrations)
+- [Configuration](#configuration)
 - [Instance health](#instance-health)
 <!-- /wikitool:toc -->

@@ -157,6 +158,12 @@ Full records: [content-migrations.md](content-migrations.md)
 - [`migrate done`](content-migrations.md#migrate-done) - Record one migration as applied, advancing `kb_version` in `.wikitool-kb.json`.
 - [`migrate baseline`](content-migrations.md#migrate-baseline) - Declare `kb_version` once, for an instance predating `.wikitool-kb.json`.

+## Configuration
+
+Full records: [configuration.md](configuration.md)
+
+- [`config set`](configuration.md#config-set) - Set one value of the tiered settings, in the file its tier names.
+
 ## Instance health

 Full records: [instance-health.md](instance-health.md)
@@ -0,0 +1,67 @@
+# Configuration
+
+Writing the tiered settings in `wikitool.json` and `.wikitool-local.json`.
+
+<!-- wikitool:reference-commands -->
+## `config set`
+
+Set one value of the tiered settings, in the file its tier names.
+
+**Synopsis**
+
+- `wikitool config set <section>.<field> <value>`
+- `wikitool config set <section>.<field> --ref <name|op://vault/item/field>`
+
+**Options**
+
+```text
+Arguments:
+  key    <section>.<field>  [required]
+  value  The plain value; a string as given, anything else as JSON.
+
+Options:
+  --ref <str>  A reference instead of a value: <name> or
+               op://<vault>/<item>/<field>.
+  -h, --help   Show this message and exit.
+```
+
+**Examples**
+
+```bash
+tools/wikitool config set tracker.stale_days 14
+tools/wikitool config set tracker.token --ref tracker_token
+tools/wikitool config set tracker.url --ref op://Chemenu/tracker/url
+```
+
+**Behaviour**
+
+- Writes a plain value of a field of tier `instance` into `wikitool.json`, and one of tier `checkout` - a private field's included - into `.wikitool-local.json`.
+- With `--ref`, writes a reference instead of a value into `wikitool.json`: `<name>` resolves through `WIKITOOL_SECRET_<NAME>`, then the secrets file in the user's configuration directory; `op://<vault>/<item>/<field>` resolves through the 1Password CLI. Only a secret or private field takes one.
+- Removes the same key from the other file, so a value never has two places, and says so.
+- Creates `wikitool.json` with a fresh `instance_id` the first time it writes anything; the secrets file is found by that id.
+- Checks both files as they would stand after the write against every registered section, and writes nothing when that fails.
+- Prints the key and the file it wrote, never the value. Resolves no reference.
+
+**Properties**
+
+- Changes files: yes
+- Idempotent: yes
+- Atomic: Each file is replaced in one rename; both are checked before either is written
+- Counts toward the session call budget: yes
+- Can reach the network: no
+
+**Exit codes**
+
+- `0` - success
+- `1` - The key names no registered section or field, or is not `<section>.<field>`
+- `1` - A plain value given for a field of tier `secret`
+- `1` - `--ref` given for a field that takes a plain value, or a malformed reference
+- `1` - The value is not of the field's type, or the files would not pass their check afterwards
+- `42` - The field names a program to run or affects a gate
+
+**See also**
+
+- `wikitool doctor` - whether both files pass and every reference resolves
+- `docs/reference/configuration.md` - the files, the tiers and the references
+- `docs/how-to/use-1password-for-settings.md` - setting 1Password up as the source
+<!-- /wikitool:reference-commands -->
@@ -39,6 +39,7 @@ tools/wikitool doctor --json
 - Task tracker: `.wikitool-tasks.json` present, absent or malformed - absent is `OK` (no tracker configured), malformed is a `FAIL`. When `WIKITOOL_TASKS_CONFIG` is set, that file is read instead and the finding names it; a set variable that names no file is a `FAIL`, never `OK`.
 - For a configured `superproductivity` provider, the configured `access` path's own state: `access: "api"` reports whether its local REST API answers `GET /health` with a ready renderer right now, `access: "snapshot"` whether a backup file is ready. The other access path is never attempted, and neither state is ever a `FAIL`.
 - For a configured `caldav` provider, whether the server is reachable and Basic auth succeeds - never a `FAIL`; only a broken config block is.
+- Settings (`settings-instance`, `settings-checkout`, `settings-secrets`): one line per tier. `wikitool.json` and `.wikitool-local.json` each present or absent - absent is `OK` - and a `FAIL` naming file, key and the right tier when a value sits in the wrong file, a secret or a committed private field holds a plain value, or a value fails its field; a section this version does not read is a `WARN`. For the references: how many resolve, never a value - an unresolvable one is a `FAIL`, one 1Password will not hand out until a human unlocks it a `WARN`.
 - Session id source: `OK` for `WIKITOOL_SESSION_ID` or a registered harness variable, `WARN` only for the bare parent-pid fallback.
 - Telemetry: on or off and why - installation-form default, `.wikitool-telemetry.json`, or `WIKI_TRACE` - and the current session's count and byte total against both caps; never a `FAIL`. A `WARN` when `.wikitool-telemetry.json` is not read as written - unreadable JSON, not an object, `enabled` not a JSON boolean, a cap not a positive integer, or an unknown key - naming the field; telemetry itself drops or reinterprets such a value without a word, so this is where it shows.
 - Exits 1 only on a `FAIL`; a missing remote, session id or `VERSION` is a `WARN`, not a fault.
@@ -61,7 +62,7 @@ tools/wikitool doctor --json

 - `instructions/setup-instance.md` - the setup steps most findings point back to
 - `instructions/preflight.md` - what `tool-paths` and `install-dir` point back to
- `docs/reference/configuration.md` - the per-checkout configuration files
+- `docs/reference/configuration.md` - the per-checkout configuration files and the tiered settings
 - `docs/explanation/telemetry-and-evaluation.md` - telemetry state and caps
 - `instructions/session-setup.md` - setting `WIKITOOL_SESSION_ID`
 <!-- /wikitool:reference-commands -->
@@ -283,7 +283,8 @@ tools/wikitool docs verify
 - Collections: every directory under `kb/` has a `COLLECTION.md` and no directory outside it does; every collection declares `profile:` and a `required_by_stack:` that agrees with the stack's own list.
 - Types: every type the stack lists (currently `source` and `project`) has a type-spec of that name whose schema requires the field the stack list names (`raw_files:`/`state:`); every file under `types/` declaring `type: types/type-spec.md` validates against `types/type-spec.schema.yaml`; every subtype template `types/<type>.<value>.md` (any `<value>` but `guidance`) sits beside a type-spec declaring `subtype_field:`, names a value that field's schema enum allows, and carries no frontmatter; no pre-migration `type: entity` blocks are left in the contracts.
 - `kb/CONVENTIONS.md`, if it exists at all, names all three tool-owned section headings; every stage contract is present.
- The `.gitignore` canaries clear in both directions: nothing ignored under `raw/`/`kb/`, `incoming/` ignored, everything ignored under `reports/` and the published skill directories.
+- The `.gitignore` canaries clear in both directions: nothing ignored under `raw/`/`kb/`, `incoming/` ignored, everything ignored under `reports/` and the published skill directories, `.wikitool-local.json` ignored and `wikitool.json` tracked.
+- A `wikitool.json` at the root holds every value in the place its tier allows - no secret and no private field as plain text, no `checkout` field at all except as a reference - and a well-formed `schema` and `instance_id`. The local file is not read.
 - No `.md`/`.template` file `dist export` would ship cites an issue number. A `<!-- dist:strip-start/end -->` region is exempt: the check reads the export plan's text, from which it is already gone.
 - Every reference file `docs toc` covers carries the current table-of-contents region for its own headings - missing and stale are one check.
 - Every relative markdown link in one of those reference files resolves to an existing file. A target's `#anchor` suffix is stripped first, and code fences and inline code spans are masked before scanning, so link syntax shown as an example is not mistaken for a real reference.
@@ -312,6 +313,7 @@ tools/wikitool docs verify
 - `1` - The `<!-- wikitool:commands -->` region of `tools/CONTRACT.md` is stale
 - `1` - A type-spec's own frontmatter fails its schema
 - `1` - A subtype template `types/<type>.<value>.md` has no type-spec with a `subtype_field:` beside it, names a value outside that field's enum, or carries frontmatter
+- `1` - `wikitool.json` holds a secret or private value as plain text, or a value its tier puts elsewhere
 - `1` - A shipped `.md`/`.template` cites an issue number
 - `1` - A region of the human reference under `docs/reference/` is stale
 - `1` - A page of the human reference is missing or lacks its marker pair, or a page under `docs/reference/commands/` belongs to no group
@@ -1,7 +1,7 @@
 # Configuration

-The environment variables `wikitool` reads, and the per-checkout files an operator may create at the
-root of an instance. None of them is needed for an instance to work: an instance installed from a
+The environment variables `wikitool` reads, the per-checkout files an operator may create at the
+root of an instance, and the two tiered settings files. None of them is needed for an instance to work: an instance installed from a
 release runs against the origin's public release feed with no configuration at all.

 The shape of each file - its fields, what an absent or a malformed file means, an example - is
@@ -24,6 +24,7 @@ something you run.
  - [Super Productivity](#super-productivity)
  - [CalDAV](#caldav)
 - [MCP submissions](#mcp-submissions)
+- [Tiered settings](#tiered-settings)
 <!-- /wikitool:toc -->

 ## Environment variables
@@ -35,6 +36,7 @@ something you run.
 | `WIKITOOL_UPDATE_URL` | The release feed `version check`, `version notes` and `dist upgrade --latest` ask | The value in `.wikitool-release.json`, else the origin's feed |
 | `WIKITOOL_UPDATE_TOKEN` | A Gitea token for the release feed, sent only to downloads on the feed's own host | None. Not needed against the origin, only against a feed in a private repository |
 | `WIKITOOL_TASKS_CONFIG` | Path to a tracker configuration that `task`, `review` and `doctor` read instead of `.wikitool-tasks.json`, for running one checkout against several trackers in turn | `.wikitool-tasks.json` at the root. A path to a file that does not exist is an error, never "no tracker configured" |
+| `WIKITOOL_SECRET_<NAME>` | The value behind a settings reference `{"ref": "<name>"}`, for CI and containers - `<NAME>` is the name upper-cased | The secrets file in the user's configuration directory ([below](#tiered-settings)) |
 | `CHEMENU_ROOT` | Which corpus the package works on, for a caller outside the checkout itself | The checkout the package sits in. `tools/wikitool` behaves the same without it |
 | `WIKI_TRACE` | `0`/`1` turns telemetry off or on, beating both the default and `.wikitool-telemetry.json` | The installation-form default ([below](#telemetry)) |
 | `WIKI_TRACE_DIR` | Writes traces somewhere other than `reports/telemetry/` | `reports/telemetry/` |
@@ -84,7 +86,10 @@ both caps. What the caps do and what never reaches a trace:
 ## Tool paths

 **The preflight writes this file, not you.** `.wikitool-tools.json` at the root holds the absolute
-paths of Python, git and ripgrep, as the preflight found them on this machine. `wikitool` starts git
+paths of Python, git and ripgrep, as the preflight found them on this machine - and of the
+1Password CLI `op` when it is installed. `op` is optional: the preflight records it when it finds
+it, reports `ABSENT` when it does not, and never asks for it. Only an `op://` reference in the
+[tiered settings](#tiered-settings) needs it. `wikitool` starts git
 and rg from there instead of from `PATH`. It is per checkout and gitignored, because a path on one
 machine says nothing about the next.

@@ -330,3 +335,61 @@ Example:
 <!-- /wikitool:reference-config-upload -->

 Size and quota are your deployment's decision, not defaults worth copying blindly.
+
+## Tiered settings
+
+Two files at the root hold the settings of the features that read them this way, and every field
+declares the tier it belongs to:
+
+| Tier | File | Holds |
+|------|------|-------|
+| `instance` | `wikitool.json`, committed | What belongs to the corpus and is the same in every clone |
+| `checkout` | `.wikitool-local.json`, gitignored | What is true only for this clone on this machine |
+| `secret` | neither | Tokens and passwords - both files hold only a reference |
+
+Each file is a JSON object of sections, one per feature, beside two keys of its own:
+
+```json
+{
+  "schema": 1,
+  "instance_id": "6f1c0e52-3d1b-4b8e-9a51-2f0e7d1c9b44",
+  "<section>": {"<field>": "<value>"}
+}
+```
+
+`instance_id` sits in `wikitool.json` only. The first `wikitool config set` creates the file and
+the id; the id names the directory of the secrets file below. No feature reads its settings this
+way yet, so this version declares no section: `config set` names the sections it knows, and
+`doctor` reports a section it does not read as a `WARN`.
+
+**One place per value.** A value in the file its tier does not name is an error naming the file,
+the key and the right tier - never an override. The same holds for a key set in both files.
+
+**References.** A field of tier `secret` takes only a reference; a field marked private - a host
+name, an internal URL - takes a plain value in `.wikitool-local.json` or a reference in either
+file. No other field takes one.
+
+| Reference | Resolves through |
+|-----------|------------------|
+| `{"ref": "<name>"}` - lower-case letters, digits and single `_` | `WIKITOOL_SECRET_<NAME>`, else the key `<name>` in the secrets file |
+| `{"ref": "op://<vault>/<item>/<field>"}` | `op read`, the 1Password CLI the preflight recorded |
+
+The secrets file is `~/.config/chemenu/<instance_id>/secrets.json` (`$XDG_CONFIG_HOME` instead of
+`~/.config` when it is set; `%APPDATA%\chemenu\<instance_id>\secrets.json` on Windows), a JSON object of
+names to strings. Outside Windows it must be readable by its owner only (`chmod 600`); a file
+others can read is refused.
+
+A reference is resolved when a command asks for that value, never when the files are read. A name
+nobody provides, an `op` that is not installed, or an item 1Password does not have ends the command
+with exit 1; a 1Password that is locked, signed out or waiting for an approval ends it with exit 42,
+for a human to unlock. Setting 1Password up: [use-1password-for-settings.md](../how-to/use-1password-for-settings.md).
+
+**Never in the committed file:** a field that names a program to run or affects a gate - not even
+as a reference. `config set` does not write such a field either; the user edits
+`.wikitool-local.json` by hand, and `doctor` checks it afterwards.
+
+`wikitool doctor` reports one line per tier - `settings-instance`, `settings-checkout`,
+`settings-secrets` - and for the references only whether they resolve. `docs verify` holds a
+committed `wikitool.json` to the placement rules above, a plain secret or private value above
+all. Neither file ships with `dist export`. Why the tiers exist, and what keeping secrets out of
+the tree does not protect against: [configuration-tiers.md](../explanation/configuration-tiers.md).
@@ -38,4 +38,5 @@ What each exit code means, and how a caller reacts to it, is [AGENTS.md § Tool
 - [`publish`](commands/git.md#publish): Rebase-Review Gate: `<remote>/<branch>` moved and both sides changed the same file; the output lists the upstream commits, the overlapping files and their diff
 - [`publish`](commands/git.md#publish): Publish-Remote Gate: `.wikitool-remotes.json` exists and the push URL of `--remote` is not listed in it, or `--remote` resolves to no push URL
 - [`export guidelines`](commands/guideline-export.md#export-guidelines): Guideline Push Gate: `--push` without `--confirm`, or with a token that does not match the current set of targets
+- [`config set`](commands/configuration.md#config-set): The field names a program to run or affects a gate
 <!-- /wikitool:reference-exit-codes -->