feat: the shapes of the four hand-written .wikitool-*.json files live in one record each (config_shapes.py); loaders, doctor and docs/reference/configuration.md read them (#193)

The telemetry, remotes, upload and tasks loaders take field names, defaults, coercions and their Expected: text from the record and accept exactly what they did before. doctor WARNs on a telemetry file the policy silently reads differently. configuration.md carries one generated region per file, also checked in an instance. docs reference --apply now refreshes several regions in one page in sequence instead of keeping only the last.

Files changed:
- CHANGES.md
- VERSION
- docs/how-to/set-up-the-mcp-server.md
- docs/reference/README.md
- docs/reference/commands/instance-health.md
- docs/reference/commands/types-instructions-and-docs.md
- docs/reference/configuration.md
- instructions/dev/doc-pull-through.md
- instructions/setup-instance.md
- tools/CONTRACT.md
- tools/README.md
- tools/chemenu/commands/docs_verify.py
- tools/chemenu/commands/doctor.py
- tools/chemenu/commands/git_publish.py
- tools/chemenu/config.py
- tools/chemenu/config_shapes.py
- tools/chemenu/reference_doc.py
- tools/chemenu/tasks/caldav.py
- tools/chemenu/tasks/config.py
- tools/chemenu/tasks/superproductivity.py
- tools/chemenu/telemetry/policy.py
- tools/chemenu/tests/test_config_shapes.py
- tools/chemenu/tests/test_doctor.py
- tools/chemenu/tests/test_reference_doc.py
- tools/chemenu/upload.py

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2

tools/CONTRACT.md

@@ -2559,7 +2559,7 @@ Check the docs that mirror the code.
 - Every reference file `docs toc` covers carries the current table-of-contents region for its own headings - missing and stale are one check.
 - Every relative markdown link in one of those reference files resolves to an existing file. A target's `#anchor` suffix is stripped first, and code fences and inline code spans are masked before scanning, so link syntax shown as an example is not mistaken for a real reference.
 - Every file under `docs/` sits in one of its areas - `tutorials/`, `how-to/`, `reference/`, `explanation/` or `dev/` - never directly in `docs/` or in another directory.
-- If `docs/` exists: every generated region of the human reference under `docs/reference/` matches what `docs reference` would write, every `cli_contract.GROUPS` entry has its page under `docs/reference/commands/`, and no page there belongs to no group. The skill catalogue and the page types are checked only in the origin repo (no `.wikitool-release.json`): they describe the stack's shipped defaults, which an instance's own type-specs and instructions may differ from.
+- If `docs/` exists: every generated region of the human reference under `docs/reference/` matches what `docs reference` would write, every `cli_contract.GROUPS` entry has its page under `docs/reference/commands/`, and no page there belongs to no group. That includes the four configuration regions in `docs/reference/configuration.md`. The skill catalogue and the page types are checked only in the origin repo (no `.wikitool-release.json`): they describe the stack's shipped defaults, which an instance's own type-specs and instructions may differ from.
 - `INSTALL.md` carries one generated `<!-- wikitool:prerequisites -->` region per platform value of `tools/prerequisites.txt` (`prerequisites-<platform>` for a platform-specific one), each current; and the `<!-- setup-question: <key> -->` markers in `instructions/setup-instance.md` and `INSTALL.md` name the same set of keys, so a question the agent asks is never one the human guide leaves out, nor the reverse.
 - `VERSION` parses and the newest versioned `CHANGES.md` entry names it. An entry that crosses the compatibility boundary from the last release carries `**Breaking Change:**`, has a migration document or `**Migration:** none required`, and its `**Migration:**` line names every required migration document targeting it.
 - The newest versioned `CHANGES.md` entry is at most 32 000 bytes (UTF-8) as `version notes` prints it, and each of its `###` sections is exactly one paragraph of at most 1 000 bytes, heading line excluded. Older entries and a changelog with no versioned entry are not checked.
@@ -2744,6 +2744,7 @@ Regenerate the human reference under `docs/reference/`.
 
 - Rewrites the generated region of every page of the human command reference from the `cli_contract` records: one `docs/reference/commands/<group>.md` per `cli_contract.GROUPS` entry (`<!-- wikitool:reference-commands -->`), the overview in `docs/reference/commands/README.md` (`<!-- wikitool:reference-commands-index -->`), and the gates and exit-42 causes in `docs/reference/exit-codes.md` (`<!-- wikitool:reference-exit-codes -->`).
 - Also, in the origin repo only, the skill catalogue `docs/reference/skills.md` (`<!-- wikitool:reference-skills -->`: one row per shipped `instructions/<name>/SKILL.md`, its description split at the sentence starting "Use when") and the page types `docs/reference/page-types.md` (`<!-- wikitool:reference-page-types -->`: one section per `root: kb` type-spec, from what `types describe` reads). Both describe the stack's shipped defaults. In an instance - a tree with `.wikitool-release.json` - they are left alone and the output says so: an instance's adopted type-specs and own instructions may differ, and rewriting a shipped page there would block the next `dist upgrade`.
+- Also the shapes of the four hand-written configuration files in `docs/reference/configuration.md`, one region each (`<!-- wikitool:reference-config-telemetry -->`, `-remotes`, `-tasks`, `-upload`) inside the hand-written section on that file, rendered from the records in `chemenu.config_shapes` the loaders read the files with. Those records ship with `tools/`, so these regions are written and checked in an instance as well. Several regions in one page are refreshed one after the other into the same text.
 - Renders each command's summary, synopsis, options, examples, notes, properties in words, exit codes with their causes and see-also - but not the ON FAILURE reactions or NEVER, which address an agent; `tools/CONTRACT.md` keeps those.
 - Refreshes the table of contents of each page it writes in the same step.
 - Never creates a page or places a region: a page's title and one-sentence head are written by hand. A group's page is named after the group - lower case, every run of other characters one hyphen.
@@ -3636,7 +3637,7 @@ Check that this instance is correctly configured.
 - For a configured `superproductivity` provider, the configured `access` path's own state: `access: "api"` reports whether its local REST API answers `GET /health` with a ready renderer right now, `access: "snapshot"` whether a backup file is ready. The other access path is never attempted, and neither state is ever a `FAIL`.
 - For a configured `caldav` provider, whether the server is reachable and Basic auth succeeds - never a `FAIL`; only a broken config block is.
 - Session id source: `OK` for `WIKITOOL_SESSION_ID` or a registered harness variable, `WARN` only for the bare parent-pid fallback.
-- Telemetry: on or off and why - installation-form default, `.wikitool-telemetry.json`, or `WIKI_TRACE` - and the current session's count and byte total against both caps; never a `FAIL`.
+- Telemetry: on or off and why - installation-form default, `.wikitool-telemetry.json`, or `WIKI_TRACE` - and the current session's count and byte total against both caps; never a `FAIL`. A `WARN` when `.wikitool-telemetry.json` is not read as written - unreadable JSON, not an object, `enabled` not a JSON boolean, a cap not a positive integer, or an unknown key - naming the field; telemetry itself drops or reinterprets such a value without a word, so this is where it shows.
 - Exits 1 only on a `FAIL`; a missing remote, session id or `VERSION` is a `WARN`, not a fault.
 - Read-only and exempt from the Iteration Budget Gate.
 

tools/README.md

@@ -83,13 +83,14 @@ tools/
     cli.py                 Typer app: registers every command, runs the budget gate, renders `-h`/`--help` from cli_contract
     cli_contract.py         one data record per command (name, synopsis, properties, exit status) - the source `-h`, the index, CONTRACT.md's generated region and the human reference under docs/reference/ render from
     config.py              repo layout: root resolution and every path under it
+    config_shapes.py       one record per hand-written .wikitool-*.json (telemetry, remotes, upload, tasks): what its loader accepts, its defaults and `Expected:` text, and what docs/reference/configuration.md renders; stdlib-only, since the telemetry policy reads it
     api.py                 the in-process entry point - point Chemenu at a corpus and read it
     errors.py              ChemenuError / ValidationError / BackendError
     toolpaths.py           where git and rg are started from: .wikitool-tools.json, bare name only without the file
     filelock.py            an exclusive lock on an open file, flock on POSIX and msvcrt on Windows - the only module that imports either
     prerequisites.py       prerequisites.txt read from Python, plus the platform and long-path questions `doctor` asks
     install_doc.py         INSTALL.md held to the instructions: its prerequisites lists generated from prerequisites.txt, its setup questions matched to setup-instance.md's markers
-    reference_doc.py       the human reference under docs/reference/: one page per command group, an overview and the exit-42 causes, rendered from the records without what addresses an agent; and, in the origin repo only, the skill catalogue and the page types, rendered from the shipped SKILL.md files and type-specs
+    reference_doc.py       the human reference under docs/reference/: one page per command group, an overview and the exit-42 causes, rendered from the records without what addresses an agent; and, in the origin repo only, the skill catalogue and the page types, rendered from the shipped SKILL.md files and type-specs; and the configuration-file shapes, from config_shapes.py
     corpus_cache.py        one parsed corpus per commit, never cached while the tree is dirty
     kb_scan.py             page iteration/loading over kb/
     blocks.py              generated regions in a page body, found by marker rather than by heading

tools/chemenu/commands/docs_verify.py

@@ -1296,7 +1296,8 @@ def check_breaking_change_for_boundary() -> list[str]:
         "If `docs/` exists: every generated region of the human reference under "
         "`docs/reference/` matches what `docs reference` would write, every `cli_contract.GROUPS` "
         "entry has its page under `docs/reference/commands/`, and no page there belongs to no "
-        "group. The skill catalogue and the page types are checked only in the origin repo "
+        "group. That includes the four configuration regions in "
+        "`docs/reference/configuration.md`. The skill catalogue and the page types are checked only in the origin repo "
         "(no `.wikitool-release.json`): they describe the stack's shipped defaults, which an "
         "instance's own type-specs and instructions may differ from.",
         "`INSTALL.md` carries one generated `<!-- wikitool:prerequisites -->` region per "
@@ -1704,6 +1705,13 @@ def contract_command(
         "shipped defaults. In an instance - a tree with `.wikitool-release.json` - they are left "
         "alone and the output says so: an instance's adopted type-specs and own instructions may "
         "differ, and rewriting a shipped page there would block the next `dist upgrade`.",
+        "Also the shapes of the four hand-written configuration files in "
+        "`docs/reference/configuration.md`, one region each "
+        "(`<!-- wikitool:reference-config-telemetry -->`, `-remotes`, `-tasks`, `-upload`) inside "
+        "the hand-written section on that file, rendered from the records in "
+        "`chemenu.config_shapes` the loaders read the files with. Those records ship with "
+        "`tools/`, so these regions are written and checked in an instance as well. Several "
+        "regions in one page are refreshed one after the other into the same text.",
         "Renders each command's summary, synopsis, options, examples, notes, properties in words, "
         "exit codes with their causes and see-also - but not the ON FAILURE reactions or NEVER, "
         "which address an agent; `tools/CONTRACT.md` keeps those.",
@@ -1737,7 +1745,6 @@ def reference_command(
 ):
     """Regenerate the human reference under `docs/reference/`."""
     missing = []
-    changed = []
     generated, left_alone = reference_doc.applicable(reference_doc.regions())
     if left_alone:
         typer.echo(
@@ -1746,17 +1753,22 @@ def reference_command(
             + " - they describe the stack's shipped defaults; `wikitool types describe <type>` "
             "and `wikitool instructions list` show this instance's own."
         )
+    # A page may carry several regions (`configuration.md` has one per file),
+    # so each is refreshed into the text the previous one left, not into what
+    # is on disk - or only the last region of a page would survive the write.
+    texts: dict[Path, tuple[str, str]] = {}
     for region in generated:
         path = config.ROOT / region.path
         if not path.is_file():
             missing.append(f"{region.path} (create it with `{blocks.open_marker(region.name)}`)")
             continue
-        before = path.read_text(encoding="utf-8")
-        after = reference_doc.refresh(before, region)
+        before, current = texts.get(path) or (path.read_text(encoding="utf-8"),) * 2
+        after = reference_doc.refresh(current, region)
         if after is None:
             missing.append(f"{region.path} (add `{blocks.open_marker(region.name)}`)")
-        elif after != before:
-            changed.append((path, after))
+        else:
+            texts[path] = (before, after)
+    changed = [(path, after) for path, (before, after) in texts.items() if after != before]
 
     if missing:
         fail(

tools/chemenu/commands/doctor.py

@@ -21,7 +21,7 @@ from typing import Optional
 import typer
 from rich.console import Console
 
-from chemenu import cli_contract, config, conventions, kb_collections, prerequisites, toolpaths, version as version_mod
+from chemenu import cli_contract, config, config_shapes, conventions, kb_collections, prerequisites, toolpaths, version as version_mod
 from chemenu.commands import git_publish, instructions_cmd
 from chemenu.commands._util import rel_path
 from chemenu.session import ENV_VAR as SESSION_ENV_VAR
@@ -466,7 +466,8 @@ def check_telemetry() -> Check:
     Never `FAIL`s, the same line `check_publish_remotes` and
     `check_environment` draw: both an enabled and a disabled tree are
     legitimate states, and a `FAIL` would make one of them mandatory by the
-    back door. The tree walk below is fine here - `doctor` is not a hot path,
+    back door. It `WARN`s on a `.wikitool-telemetry.json` the policy does not
+    read as written, since the policy itself drops that in silence. The tree walk below is fine here - `doctor` is not a hot path,
     unlike the `stat` `emit()` does on every append.
     """
     from chemenu.telemetry import policy as telemetry_policy
@@ -482,12 +483,38 @@ def check_telemetry() -> Check:
         else 0
     )
     state = "on" if pol.enabled else "off"
-    return Check(
-        "telemetry", "OK",
+    detail = (
         f"{state} ({pol.reason}); {session_count}/{pol.keep_sessions} session(s), "
         f"{total_bytes:,} byte(s) under {rel_path(root)} "
-        f"(cap {pol.max_session_bytes:,} byte(s)/session)",
+        f"(cap {pol.max_session_bytes:,} byte(s)/session)"
     )
+    deviations = _telemetry_file_deviations()
+    if deviations:
+        return Check(
+            "telemetry", "WARN",
+            f"{detail}. {config.TELEMETRY_FILENAME} is not read as written: {'; '.join(deviations)}",
+            f"Correct {config.TELEMETRY_FILENAME} to the shape in "
+            "docs/reference/configuration.md#telemetry, or delete it to get the default",
+        )
+    return Check("telemetry", "OK", detail)
+
+
+def _telemetry_file_deviations() -> list[str]:
+    """What the telemetry policy silently drops or reinterprets in this
+    checkout's `.wikitool-telemetry.json` - nothing when the file is absent.
+
+    The policy has to stay silent (a hook handler reads it on every tool
+    call); this is where the silence becomes visible."""
+    path = config.ROOT / config.TELEMETRY_FILENAME
+    if not path.is_file():
+        return []
+    try:
+        data = _json.loads(path.read_text(encoding="utf-8"))
+    except (OSError, _json.JSONDecodeError) as exc:
+        return [f"unreadable ({exc}), so the whole file is ignored"]
+    if not isinstance(data, dict):
+        return ["not a JSON object, so the whole file is ignored"]
+    return config_shapes.deviations(config_shapes.TELEMETRY, data)
 
 
 def check_upload_intake() -> Check:
@@ -788,7 +815,10 @@ def run_doctor() -> list[Check]:
         "`WARN` only for the bare parent-pid fallback.",
         "Telemetry: on or off and why - installation-form default, `.wikitool-telemetry.json`, "
         "or `WIKI_TRACE` - and the current session's count and byte total against both caps; "
-        "never a `FAIL`.",
+        "never a `FAIL`. A `WARN` when `.wikitool-telemetry.json` is not read as written - "
+        "unreadable JSON, not an object, `enabled` not a JSON boolean, a cap not a positive "
+        "integer, or an unknown key - naming the field; telemetry itself drops or reinterprets "
+        "such a value without a word, so this is where it shows.",
         "Exits 1 only on a `FAIL`; a missing remote, session id or `VERSION` is a `WARN`, not a "
         "fault.",
         "Read-only and exempt from the Iteration Budget Gate.",

tools/chemenu/commands/git_publish.py

@@ -62,7 +62,7 @@ from typing import NamedTuple, Optional
 
 import typer
 
-from chemenu import cli_contract, config, toolpaths
+from chemenu import cli_contract, config, config_shapes, toolpaths
 from chemenu.commands._util import fail, needs_clearance, success
 from chemenu.telemetry import emit
 
@@ -123,14 +123,15 @@ def read_allowed_push_urls() -> Optional[list[str]]:
             "fix the file or delete it deliberately."
         )
         return None
-    urls = data.get("allowed_push_urls")
-    if not isinstance(urls, list) or not all(isinstance(u, str) for u in urls):
+    shape = config_shapes.REMOTES
+    checked = config_shapes.check(shape, data)
+    if checked.problems:
         fail(
-            f"{config.PUBLISH_REMOTES_FILENAME} has no usable `allowed_push_urls` list of "
-            "strings. Expected: {\"schema\": 1, \"allowed_push_urls\": [\"<url>\"]}"
+            f"{config.PUBLISH_REMOTES_FILENAME} has no usable allowed-URL list: {checked.message()}. "
+            f"Expected: {config_shapes.render_expected(shape)}"
         )
         return None
-    return urls
+    return checked.values["allowed_push_urls"]
 
 
 def push_url_for(remote: str) -> Optional[str]:

tools/chemenu/config.py

@@ -222,15 +222,17 @@ LICENSE_FILES = ("LICENSE", "LICENSE-CONTENT", "NOTICE")
 RELEASE_STAMP_FILENAME = ".wikitool-release.json"
 
 # Per-checkout telemetry opt-in/opt-out plus its two quantity caps (see
-# `telemetry/policy.py`). Same shape as `PUBLISH_REMOTES_FILENAME` below: it
+# `telemetry/policy.py`). Same posture as `PUBLISH_REMOTES_FILENAME` below: it
 # answers a question about *this* checkout, so it is per-checkout and
 # gitignored, ships no `.template`, and its absence is a legitimate state -
 # the installation-form default (keyed off `RELEASE_STAMP_FILENAME`) applies.
 # `instructions/setup-instance.md`'s Telemetry decision point writes it from
-# the operator's answer.
+# the operator's answer. Its fields are the record `config_shapes.TELEMETRY`,
+# as for the three files below - each has exactly one.
 TELEMETRY_FILENAME = ".wikitool-telemetry.json"
 
-# Which push targets `publish` may write to, for a checkout that says so. The
+# Which push targets `publish` may write to, for a checkout that says so
+# (fields: `config_shapes.REMOTES`). The
 # danger this addresses is one checkout's content reaching another checkout's
 # remote - a private instance pushing its own `kb/` to a public repository,
 # where it cannot be taken back.
@@ -246,8 +248,8 @@ TELEMETRY_FILENAME = ".wikitool-telemetry.json"
 # instructions/gates.md.
 PUBLISH_REMOTES_FILENAME = ".wikitool-remotes.json"
 
-# Opt-in for the MCP server's `submit` tool (Gitea #32): identity header name,
-# size deckel, extension allowlist, per-submitter quota. Same shape as the two
+# Opt-in for the MCP server's `submit` tool (Gitea #32); its fields are the
+# record `config_shapes.UPLOAD`. Same posture as the two
 # above - per-checkout, gitignored, no `.template` - but its absence means
 # something stronger than "unrestricted": **the write path does not exist at
 # all**, the tool is not registered. The safe direction, and a structural
@@ -256,7 +258,8 @@ UPLOAD_CONFIG_FILENAME = ".wikitool-upload.json"
 
 # The task-tracker provider opt-in (Gitea #124, D25/D30): which provider this
 # instance's GTD review reads/writes through, its connection details, and the
-# review's three staleness thresholds. Same shape as the three files above -
+# review's three staleness thresholds - fields in `config_shapes.TASKS`. Same
+# posture as the three files above -
 # per-checkout, gitignored once a credential lands in it, no `.template` - and
 # its absence is a legitimate state, the same posture `UPLOAD_CONFIG_FILENAME`
 # takes: an instance with no tracker configured runs `doctor` and everything

tools/chemenu/config_shapes.py

@@ -0,0 +1,596 @@
+"""The shapes of the per-checkout configuration files an operator writes by hand.
+
+Four files - `.wikitool-telemetry.json`, `.wikitool-remotes.json`,
+`.wikitool-upload.json` and `.wikitool-tasks.json` (names in `chemenu.config`)
+- each have exactly one record here, and that record is the only place their
+shape is written down. The loader that reads a file takes its field names,
+required fields, defaults, coercions, conditions and the `Expected:` text of its
+error messages from the record; `docs reference --apply` renders the same record
+into `docs/reference/configuration.md`. A second description of a shape, in a
+loader or in prose, is what used to drift.
+
+The records describe what the loaders **accept today**, leniency included:
+`int()` on a number given as a string, `bool()` on `enabled`, unknown top-level
+keys ignored. Tightening any of that would reject a file an instance already
+has, which is a different kind of change than writing the shape down.
+
+`check()` never raises. Each loader keeps its own posture towards a problem -
+`publish` stops through `fail()`, the upload and tracker loaders raise
+`ValidationError`, and telemetry drops what it cannot use in silence, because a
+hook handler reads it on every tool call. `deviations()` is the strict reading
+`doctor` uses to report what that silence would otherwise hide.
+
+The three machine-written files beside them - `.wikitool-release.json`,
+`.wikitool-kb.json`, `.wikitool-tools.json` - are deliberately not here: nobody
+writes them by hand, so they have no shape for an operator to look up.
+
+Stdlib only, and importing nothing from this package but `chemenu.config`:
+`chemenu.telemetry.policy` reads its record, and a hook handler imports that on
+every tool call without the venv.
+"""
+from __future__ import annotations
+
+import json
+from dataclasses import dataclass, field as dc_field
+from typing import Any, Callable, Optional
+
+from chemenu import config
+
+IGNORE = "ignore"
+REJECT = "reject"
+
+_ABSENT = object()
+
+
+# ---------------------------------------------------------------------------
+# The record types
+# ---------------------------------------------------------------------------
+
+
+@dataclass(frozen=True)
+class Field:
+    """One key of a configuration object.
+
+    `json_type` is the JSON type a conforming file uses (`boolean`, `integer`,
+    `string`, `array`, `object`, or None for any). `coerce` is what the loader
+    applies to the value it finds - it may accept more than `json_type`, which
+    is the leniency `deviations()` reports - and raises `TypeError`/`ValueError`
+    on a value it cannot use. `check` then judges the coerced value and returns
+    a problem, or None. A field with `fields` is a nested object."""
+
+    name: str
+    type: str
+    description: str
+    required: bool = False
+    default: Any = None
+    default_text: str = ""
+    json_type: Optional[str] = None
+    coerce: Optional[Callable[[Any], Any]] = None
+    check: Optional[Callable[[Any], Optional[str]]] = None
+    choices: tuple[str, ...] = ()
+    empty_is_missing: bool = False
+    required_why: str = ""
+    lenient: str = ""
+    fields: tuple["Field", ...] = ()
+
+
+@dataclass(frozen=True)
+class Shape:
+    """One configuration object: a whole file, or a provider's section of one.
+
+    `key` is the file name for a file and the section's key for a section.
+    `discriminator` names a field whose value picks one of `variants` - an
+    extra set of fields valid only alongside that value. `section_key` names a
+    field whose value picks one of `sections`, a nested object stored under the
+    chosen value's own key."""
+
+    key: str
+    title: str
+    purpose: str
+    fields: tuple[Field, ...]
+    examples: tuple[dict, ...]
+    absent: str = ""
+    malformed: str = ""
+    unknown_keys: str = IGNORE
+    unknown_text: str = ""
+    unknown_why: str = ""
+    discriminator: str = ""
+    variants: tuple[tuple[str, tuple[Field, ...]], ...] = ()
+    section_key: str = ""
+    sections: tuple["Shape", ...] = ()
+
+    def field(self, name: str) -> Field:
+        for candidate in self.fields + tuple(f for _, fs in self.variants for f in fs):
+            if candidate.name == name:
+                return candidate
+        raise KeyError(name)
+
+    def variant_fields(self, value: Any) -> Optional[tuple[Field, ...]]:
+        return dict(self.variants).get(value)
+
+    def choices(self, field: Field) -> tuple[str, ...]:
+        """The values `field` may take: its own list, or - for the
+        discriminator and the section key - the variants and sections that
+        exist, so a new one cannot be added without the choice following."""
+        if field.name == self.discriminator:
+            return tuple(value for value, _ in self.variants)
+        if field.name == self.section_key:
+            return tuple(s.key for s in self.sections)
+        return field.choices
+
+
+@dataclass(frozen=True)
+class Problem:
+    field: str
+    message: str
+
+
+@dataclass
+class Checked:
+    """What `check()` found: the coerced values (defaults filled in), the
+    fields the file actually set and the loader accepted, the problems a
+    loader that rejects would reject on, the fields accepted only by coercion,
+    and the keys no field names."""
+
+    values: dict = dc_field(default_factory=dict)
+    given: set = dc_field(default_factory=set)
+    problems: list = dc_field(default_factory=list)
+    coerced: list = dc_field(default_factory=list)
+    unknown: list = dc_field(default_factory=list)
+
+    def message(self) -> str:
+        return "; ".join(p.message for p in self.problems)
+
+
+# ---------------------------------------------------------------------------
+# Coercions and conditions
+# ---------------------------------------------------------------------------
+
+
+def _exact_str(value: Any) -> str:
+    if not isinstance(value, str):
+        raise TypeError("not a string")
+    return value
+
+
+def _str_list(value: Any) -> list[str]:
+    if not isinstance(value, list) or not all(isinstance(item, str) for item in value):
+        raise TypeError("not a list of strings")
+    return list(value)
+
+
+def _extensions(value: Any) -> tuple[str, ...]:
+    return tuple(sorted({str(ext).lower() for ext in value}))
+
+
+def _positive(value: int) -> Optional[str]:
+    return None if value > 0 else "must be positive"
+
+
+def _dotted_extensions(value: tuple[str, ...]) -> Optional[str]:
+    if not value:
+        return "must not be empty"
+    not_dotted = [ext for ext in value if not ext.startswith(".")]
+    return f"must each start with '.': {not_dotted}" if not_dotted else None
+
+
+def _positive_integer(name: str, description: str, **kwargs: Any) -> Field:
+    return Field(
+        name, "positive integer", description, json_type="integer", coerce=int,
+        check=_positive, lenient="A number written as a string is accepted.", **kwargs,
+    )
+
+
+_SCHEMA = Field(
+    "schema", "any", "Accepted and not read. Conventionally `1`; no version of this file is "
+    "told apart by it yet.",
+)
+
+
+# ---------------------------------------------------------------------------
+# The records
+# ---------------------------------------------------------------------------
+
+TELEMETRY = Shape(
+    key=config.TELEMETRY_FILENAME,
+    title="Telemetry",
+    purpose="Turns telemetry on or off for this checkout, against the installation-form default, "
+    "and sets its two quantity caps.",
+    fields=(
+        _SCHEMA,
+        Field(
+            "enabled", "boolean", "Telemetry on or off. `WIKI_TRACE` beats it in both directions.",
+            default_text="the installation-form default", json_type="boolean", coerce=bool,
+            lenient="Any other value is read by its truthiness, so `\"no\"` means on.",
+        ),
+        _positive_integer(
+            "max_session_bytes", "Byte cap per session trace. `WIKI_TRACE_MAX_SESSION_BYTES` beats it.",
+            default=5 * 1024 * 1024,
+        ),
+        _positive_integer(
+            "keep_sessions", "How many session directories retention keeps. "
+            "`WIKI_TRACE_KEEP_SESSIONS` beats it.", default=250,
+        ),
+    ),
+    examples=({"enabled": True, "max_session_bytes": 5242880, "keep_sessions": 250},),
+    absent="The installation-form default applies: on in a clone of the origin repository, off in "
+    "an instance.",
+    malformed="Never an error. Unreadable JSON or a non-object counts as an absent file, and a cap "
+    "that is not a positive integer as an absent cap. `doctor` reports each such deviation as a "
+    "`WARN` under `telemetry`, naming the field.",
+    unknown_text="Ignored; `doctor` reports them as a `WARN`.",
+)
+
+REMOTES = Shape(
+    key=config.PUBLISH_REMOTES_FILENAME,
+    title="Allowed push URLs",
+    purpose="Restricts the URLs `publish` may push to.",
+    fields=(
+        _SCHEMA,
+        Field(
+            "allowed_push_urls", "array of strings", "The push URLs `publish` may write to, "
+            "compared with the URL the remote resolves to for a push - not with its name. An "
+            "empty list allows no push at all.",
+            required=True, json_type="array", coerce=_str_list,
+        ),
+    ),
+    examples=({"schema": 1, "allowed_push_urls": ["<url>"]},),
+    absent="Any remote is allowed; the Publish-Remote Gate does not apply.",
+    malformed="`publish` stops with an error before committing anything. A broken file is never "
+    "read as \"no restriction\".",
+    unknown_text="Ignored.",
+)
+
+UPLOAD = Shape(
+    key=config.UPLOAD_CONFIG_FILENAME,
+    title="MCP submissions",
+    purpose="Turns on the MCP read server's `submit` tool and sets its limits.",
+    fields=(
+        _SCHEMA,
+        Field(
+            "identity_header", "string", "The request header the authenticating middleware sets to "
+            "the submitter's identity on every request.",
+            default="X-Forwarded-User", json_type="string", coerce=str, empty_is_missing=True,
+            lenient="An empty value means the default.",
+        ),
+        _positive_integer("max_bytes", "Largest single submission, in bytes.", required=True),
+        Field(
+            "allowed_extensions", "array of strings", "The file extensions `submit` accepts, each "
+            "starting with `.`. Compared case-insensitively.",
+            required=True, json_type="array", coerce=_extensions, check=_dotted_extensions,
+            lenient="Entries are read as strings and lower-cased; duplicates collapse.",
+        ),
+        Field(
+            "quota", "object", "Per-submitter limits over any rolling 24 hours.",
+            required=True, json_type="object",
+            fields=(
+                _positive_integer("submissions_per_day", "Submissions one submitter may make.",
+                                  required=True),
+                _positive_integer("bytes_per_day", "Bytes one submitter may submit.", required=True),
+            ),
+        ),
+    ),
+    examples=({
+        "schema": 1, "identity_header": "X-Forwarded-User", "max_bytes": 10485760,
+        "allowed_extensions": [".md", ".pdf"],
+        "quota": {"submissions_per_day": 20, "bytes_per_day": 52428800},
+    },),
+    absent="The `submit` tool is not registered: the write path does not exist.",
+    malformed="An error. The MCP server does not start and `doctor` reports `FAIL`. A broken file "
+    "is never read as \"no limits\".",
+    unknown_text="Ignored.",
+)
+
+SUPERPRODUCTIVITY = Shape(
+    key="superproductivity",
+    title="Super Productivity",
+    purpose="Connection settings for the Super Productivity adapter.",
+    fields=(
+        Field(
+            "access", "string", "Which way in: `api`, the local REST API (reads the current state "
+            "and carries the write path), or `snapshot`, the newest backup snapshot (read-only).",
+            required=True, json_type="string",
+            required_why="there is no default and no fallback between the two",
+        ),
+    ),
+    discriminator="access",
+    variants=(
+        ("api", (
+            Field(
+                "api_base_url", "string", "Where the app's REST API listens.",
+                default="http://127.0.0.1:3876", json_type="string", coerce=str,
+                empty_is_missing=True, lenient="An empty value means the default.",
+            ),
+            Field(
+                "api_token", "string", "The token from the app's settings.",
+                required=True, json_type="string", coerce=_exact_str, empty_is_missing=True,
+                required_why="every endpoint but GET /health requires Authorization: Bearer <token>",
+            ),
+        )),
+        ("snapshot", (
+            Field(
+                "backups_dir", "string", "The directory the app writes its backup snapshots to; "
+                "`~` is expanded.",
+                required=True, json_type="string", coerce=_exact_str, empty_is_missing=True,
+                required_why="the read path has nothing to read otherwise",
+            ),
+        )),
+    ),
+    examples=(
+        {"access": "api", "api_base_url": "http://127.0.0.1:3876", "api_token": "..."},
+        {"access": "snapshot", "backups_dir": "~/.config/superProductivity/backups"},
+    ),
+    unknown_keys=REJECT,
+    unknown_why="a section names only one access path's own fields",
+    unknown_text="Rejected: a section names only one access path's own fields, so a `backups_dir` "
+    "beside `access: \"api\"` is an error, not ignored.",
+)
+
+CALDAV = Shape(
+    key="caldav",
+    title="CalDAV",
+    purpose="Connection settings for the CalDAV adapter.",
+    fields=(
+        Field("url", "string", "The account's CalDAV calendar home set. A trailing `/` is added "
+              "when missing.", required=True, json_type="string", coerce=_exact_str,
+              empty_is_missing=True),
+        Field("username", "string", "The login name, which can differ from the user id in the URL.",
+              required=True, json_type="string", coerce=_exact_str, empty_is_missing=True),
+        Field("app_password", "string", "An app password is recommended over the account password.",
+              required=True, json_type="string", coerce=_exact_str, empty_is_missing=True),
+        Field("inbox_list", "string", "The name of the one inbox list.",
+              required=True, json_type="string", coerce=_exact_str, empty_is_missing=True),
+        Field("someday_list", "string", "The name of the one someday/maybe list.",
+              required=True, json_type="string", coerce=_exact_str, empty_is_missing=True),
+        Field("exclude_lists", "array of strings", "Existing plain task lists that are not "
+              "projects.", default=[], json_type="array", coerce=_str_list),
+    ),
+    examples=({
+        "url": "https://<host>/remote.php/dav/calendars/<user>/", "username": "<login>",
+        "app_password": "...", "inbox_list": "Inbox", "someday_list": "Someday",
+        "exclude_lists": ["..."],
+    },),
+    unknown_keys=REJECT,
+    unknown_text="Rejected.",
+)
+
+TASKS = Shape(
+    key=config.TASKS_CONFIG_FILENAME,
+    title="Task tracker",
+    purpose="Picks the task tracker the weekly review reads and writes through, its connection "
+    "settings and the review's thresholds.",
+    fields=(
+        _SCHEMA,
+        Field("provider", "string", "Which adapter the review goes through. The provider's own "
+              "settings sit in a section of the same name.",
+              required=True, json_type="string", coerce=str),
+        Field(
+            "thresholds", "object", "The weekly review's three thresholds.",
+            required=True, json_type="object",
+            fields=(
+                _positive_integer("stalled_waiting_days", "A waiting-for item older than this many "
+                                  "days counts as overdue.", required=True),
+                _positive_integer("unpaged_project_weeks", "A tracker project with no `kb/` page is "
+                                  "reported once it is older than this many weeks.", required=True),
+                _positive_integer("someday_stale_months", "A someday/maybe item untouched for this "
+                                  "many months counts as stale.", required=True),
+            ),
+        ),
+    ),
+    section_key="provider",
+    sections=(SUPERPRODUCTIVITY, CALDAV),
+    examples=({
+        "schema": 1, "provider": "superproductivity",
+        "thresholds": {"stalled_waiting_days": 14, "unpaged_project_weeks": 3, "someday_stale_months": 5},
+        "superproductivity": {"access": "api", "api_base_url": "http://127.0.0.1:3876", "api_token": "..."},
+    },),
+    absent="No tracker is configured - a valid state. The weekly review has nothing to check "
+    "against.",
+    malformed="An error, and `doctor` reports `FAIL`. A broken file is never read as \"no tracker "
+    "configured\".",
+    unknown_text="Ignored, so the section of a provider not chosen right now can stay in the file.",
+)
+
+SHAPES = (TELEMETRY, REMOTES, UPLOAD, TASKS)
+
+
+# ---------------------------------------------------------------------------
+# Checking a value against a record
+# ---------------------------------------------------------------------------
+
+_JSON_TYPES: dict[str, Callable[[Any], bool]] = {
+    "boolean": lambda v: isinstance(v, bool),
+    "integer": lambda v: isinstance(v, int) and not isinstance(v, bool),
+    "string": lambda v: isinstance(v, str),
+    "array": lambda v: isinstance(v, list),
+    "object": lambda v: isinstance(v, dict),
+}
+
+_TYPE_NAMES = {bool: "a boolean", int: "an integer", float: "a number", str: "a string",
+               list: "an array", dict: "an object", type(None): "null"}
+
+
+def _described(value: Any) -> str:
+    return _TYPE_NAMES.get(type(value), type(value).__name__)
+
+
+def _article(noun: str) -> str:
+    return f"{'an' if noun[:1] in 'aeiou' else 'a'} {noun}"
+
+
+def _check_fields(
+    shape: Shape, fields: tuple[Field, ...], data: dict, prefix: str, when: str, out: Checked,
+    values: dict,
+) -> None:
+    for spec in fields:
+        path = prefix + spec.name
+        raw = data.get(spec.name, _ABSENT)
+        if raw is _ABSENT or (spec.empty_is_missing and not raw):
+            if spec.required:
+                why = f" - {spec.required_why}" if spec.required_why else ""
+                empty = " or empty" if spec.empty_is_missing else ""
+                out.problems.append(Problem(path, f"{path} is missing{empty}, and required{when}{why}"))
+            else:
+                values[spec.name] = spec.default
+            continue
+        if spec.fields:
+            if not isinstance(raw, dict):
+                out.problems.append(Problem(path, f"{path} must be an object, got {_described(raw)}"))
+                continue
+            nested: dict = {}
+            before = len(out.problems)
+            _check_fields(shape, spec.fields, raw, f"{path}.", when, out, nested)
+            if len(out.problems) == before:
+                values[spec.name] = nested
+                out.given.add(path)
+            continue
+        try:
+            value = spec.coerce(raw) if spec.coerce is not None else raw
+        except (TypeError, ValueError):
+            out.problems.append(Problem(path, f"{path} must be {_article(spec.type)}, got {_described(raw)}"))
+            continue
+        choices = shape.choices(spec)
+        if choices and value not in choices:
+            why = f" - {spec.required_why}" if spec.required_why else ""
+            out.problems.append(Problem(
+                path, f"{path} must be one of {', '.join(repr(c) for c in choices)}{why}; got {raw!r}"
+            ))
+            continue
+        problem = spec.check(value) if spec.check is not None else None
+        if problem:
+            out.problems.append(Problem(path, f"{path} {problem}"))
+            continue
+        if spec.json_type is not None and not _JSON_TYPES[spec.json_type](raw):
+            out.coerced.append(Problem(
+                path, f"{path} is {_described(raw)}, not a JSON {spec.json_type}, and is read as "
+                f"{value!r}",
+            ))
+        values[spec.name] = value
+        out.given.add(path)
+
+
+def check(shape: Shape, data: Any) -> Checked:
+    """`data` read against `shape`: coerced values, problems, and what was
+    only accepted leniently. Never raises - each loader decides what a problem
+    means for it."""
+    out = Checked()
+    if not isinstance(data, dict):
+        out.problems.append(Problem("", f"must contain a JSON object, not {_described(data)}"))
+        return out
+    _check_fields(shape, shape.fields, data, "", "", out, out.values)
+    known = {f.name for f in shape.fields}
+
+    if shape.discriminator:
+        chosen = out.values.get(shape.discriminator)
+        variant = shape.variant_fields(chosen) if shape.discriminator in out.given else None
+        if variant is None:
+            # The discriminator is already a problem; an "unknown key" on top
+            # would only name fields of a variant nobody picked.
+            return out
+        _check_fields(shape, variant, data, "", f" when {shape.discriminator}: {chosen!r}", out,
+                      out.values)
+        known |= {f.name for f in variant}
+
+    if shape.section_key:
+        known |= {s.key for s in shape.sections}
+        chosen = out.values.get(shape.section_key)
+        if shape.section_key in out.given and not isinstance(data.get(chosen), dict):
+            out.problems.append(Problem(
+                chosen, f"missing or non-object {chosen!r} section holding that provider's own "
+                "connection settings",
+            ))
+
+    out.unknown = sorted(set(data) - known)
+    if out.unknown and shape.unknown_keys == REJECT:
+        why = f" - {shape.unknown_why}" if shape.unknown_why else ""
+        out.problems.append(Problem("", f"does not take {out.unknown}{why}"))
+    return out
+
+
+def deviations(shape: Shape, data: Any) -> list[str]:
+    """Everything in `data` that a strict reading of `shape` would not take as
+    written: problems, values accepted only by coercion, and unknown keys. For
+    a file whose loader is silent about all three - telemetry - this is what
+    `doctor` reports."""
+    checked = check(shape, data)
+    found = [p.message for p in checked.problems] + [p.message for p in checked.coerced]
+    if shape.unknown_keys == IGNORE:
+        found += [f"unknown key {key!r} is ignored" for key in checked.unknown]
+    return found
+
+
+# ---------------------------------------------------------------------------
+# Rendering
+# ---------------------------------------------------------------------------
+
+
+def render_expected(shape: Shape) -> str:
+    """The `Expected:` text of a loader's error message: the record's example,
+    or its examples joined by "or" where a discriminator offers several."""
+    return " or ".join(json.dumps(example) for example in shape.examples)
+
+
+def _cell(text: str) -> str:
+    return text.replace("|", "\\|").replace("\n", " ")
+
+
+def _default_cell(spec: Field) -> str:
+    if spec.required or spec.fields:
+        return "-"
+    if spec.default_text:
+        return spec.default_text
+    if spec.default is None:
+        return "-"
+    return f"`{json.dumps(spec.default)}`"
+
+
+def _rows(fields: tuple[Field, ...], prefix: str = "") -> list[str]:
+    rows = []
+    for spec in fields:
+        path = prefix + spec.name
+        required = "yes" if spec.required else "no"
+        if spec.required and spec.empty_is_missing:
+            required = "yes, not empty"
+        description = " ".join(part for part in (spec.description, spec.lenient) if part)
+        rows.append(
+            f"| `{path}` | {spec.type} | {required} | {_cell(_default_cell(spec))} | {_cell(description)} |"
+        )
+        rows += _rows(spec.fields, f"{path}.")
+    return rows
+
+
+def _table(fields: tuple[Field, ...]) -> list[str]:
+    return ["| Field | Type | Required | Default | Description |",
+            "|-------|------|----------|---------|-------------|", *_rows(fields)]
+
+
+def _example(example: dict) -> list[str]:
+    return ["```json", json.dumps(example, indent=2), "```"]
+
+
+def _render_section(shape: Shape) -> list[str]:
+    lines = [f"**The `{shape.key}` section.** {shape.purpose}", "", *_table(shape.fields), ""]
+    for value, fields in shape.variants:
+        lines += [f"With `{shape.discriminator}: \"{value}\"`:", "", *_table(fields), ""]
+    lines += [f"- **Unknown keys:** {shape.unknown_text}", ""]
+    for example in shape.examples:
+        lines += [*_example({shape.key: example}), ""]
+    return lines
+
+
+def render_markdown(shape: Shape) -> str:
+    """The reference text for one file: its fields, what an absent and a
+    malformed file mean, what happens to unknown keys, and an example."""
+    lines = [*_table(shape.fields), ""]
+    lines += [
+        f"- **Absent:** {shape.absent}",
+        f"- **Malformed:** {shape.malformed}",
+        f"- **Unknown keys:** {shape.unknown_text}",
+        "",
+    ]
+    for section in shape.sections:
+        lines += _render_section(section)
+    lines += ["A complete example:" if shape.sections else "Example:", ""]
+    for example in shape.examples:
+        lines += [*_example(example), ""]
+    return "\n".join(lines).rstrip("\n") + "\n"

tools/chemenu/reference_doc.py

@@ -27,6 +27,11 @@ pages describe the stack's shipped defaults and are `origin_only`: generated
 and checked in the origin repo, where `types/*.md` is exactly what ships as
 `.template`, and left alone in an instance - one carrying a release stamp,
 the same line the telemetry default draws.
+
+`docs/reference/configuration.md` carries four more regions, one per
+hand-written configuration file, rendered from the records in `config_shapes`.
+Those records are the tool's own, read by the loaders and shipped with
+`tools/`, so their regions are generated and checked in an instance too.
 """
 from __future__ import annotations
 
@@ -34,7 +39,7 @@ import re
 from pathlib import Path
 from typing import Callable, NamedTuple, Optional
 
-from chemenu import blocks, cli_contract, config, toc
+from chemenu import blocks, cli_contract, config, config_shapes, toc
 from chemenu import version as version_mod
 
 REFERENCE_DIR = "docs/reference"
@@ -43,6 +48,7 @@ COMMANDS_INDEX = f"{COMMANDS_DIR}/README.md"
 EXIT_CODES = f"{REFERENCE_DIR}/exit-codes.md"
 SKILLS = f"{REFERENCE_DIR}/skills.md"
 PAGE_TYPES = f"{REFERENCE_DIR}/page-types.md"
+CONFIGURATION = f"{REFERENCE_DIR}/configuration.md"
 
 COMMANDS_REGION = "reference-commands"
 INDEX_REGION = "reference-commands-index"
@@ -50,6 +56,18 @@ EXIT_CODES_REGION = "reference-exit-codes"
 SKILLS_REGION = "reference-skills"
 PAGE_TYPES_REGION = "reference-page-types"
 
+# One region per hand-written configuration file, each placed inside the
+# section of `configuration.md` that carries the file's behaviour in prose.
+# Rendered from the `config_shapes` record named beside it, which ships with
+# `tools/`, so unlike the two pages above they are generated and checked in an
+# instance as well.
+CONFIG_REGIONS = (
+    ("reference-config-telemetry", "TELEMETRY"),
+    ("reference-config-remotes", "REMOTES"),
+    ("reference-config-tasks", "TASKS"),
+    ("reference-config-upload", "UPLOAD"),
+)
+
 _SLUG_RE = re.compile(r"[^a-z0-9]+")
 
 
@@ -408,6 +426,12 @@ def regions(
     # its own skills and types for the real ones.
     found.append(Region(SKILLS, SKILLS_REGION, lambda: render_skills(shipped_skills()), origin_only=True))
     found.append(Region(PAGE_TYPES, PAGE_TYPES_REGION, lambda: render_page_types(page_types()), origin_only=True))
+    # The record is looked up at render time, so a test replacing one sees it.
+    found += [
+        Region(CONFIGURATION, name,
+               lambda attr=shape_name: config_shapes.render_markdown(getattr(config_shapes, attr)))
+        for name, shape_name in CONFIG_REGIONS
+    ]
     return found
 
 
@@ -422,7 +446,10 @@ def refresh(text: str, region: Region) -> Optional[str]:
     if blocks.find(text, region.name) is None:
         return None
     content = region.render().strip("\n")
-    wrapped = blocks.open_marker(region.name) + "\n" + content + "\n" + blocks.close_marker(region.name)
+    # The trailing newline keeps a blank line between the region and prose
+    # that follows it; `blocks.replace` was written for regions that end a
+    # page, and the configuration page is the first to carry them mid-page.
+    wrapped = blocks.open_marker(region.name) + "\n" + content + "\n" + blocks.close_marker(region.name) + "\n"
     return toc.upsert(blocks.replace(text, region.name, wrapped))
 
 

tools/chemenu/tasks/caldav.py

@@ -82,6 +82,7 @@ import icalendar
 import urllib.error
 import urllib.request
 
+from chemenu import config_shapes
 from chemenu.errors import ValidationError
 from chemenu.tasks.protocol import (
     OpenItem,
@@ -102,16 +103,6 @@ WAITING_CATEGORY = "waiting"
 
 SOURCE_KIND = "api"
 
-_EXPECTED_CONFIG = (
-    '{"url": "https://<host>/remote.php/dav/calendars/<user>/", "username": "<login>", '
-    '"app_password": "...", "inbox_list": "Inbox", "someday_list": "Someday", '
-    '"exclude_lists": ["..."]} - see INSTALL.md'
-)
-
-_REQUIRED_STRING_FIELDS = ("url", "username", "app_password", "inbox_list", "someday_list")
-_KNOWN_FIELDS = frozenset({*_REQUIRED_STRING_FIELDS, "exclude_lists"})
-
-
 @dataclass(frozen=True)
 class CalDAVConfig:
     """This provider's own section of `.wikitool-tasks.json`
@@ -129,40 +120,21 @@ class CalDAVConfig:
 
     @classmethod
     def from_dict(cls, data: dict[str, Any]) -> "CalDAVConfig":
-        if not isinstance(data, dict):
-            raise ValidationError(f"caldav config must be an object. Expected: {_EXPECTED_CONFIG}")
-
-        missing = [
-            key for key in _REQUIRED_STRING_FIELDS
-            if not isinstance(data.get(key), str) or not data[key]
-        ]
-        if missing:
+        shape = config_shapes.CALDAV
+        checked = config_shapes.check(shape, data)
+        if checked.problems:
             raise ValidationError(
-                f"caldav config is missing or has an empty {', '.join(missing)}. "
-                f"Expected: {_EXPECTED_CONFIG}"
+                f"caldav config: {checked.message()}. Expected: {config_shapes.render_expected(shape)}"
             )
-
-        extra = sorted(set(data) - _KNOWN_FIELDS)
-        if extra:
-            raise ValidationError(
-                f"caldav config does not take {extra} - unexpected field(s). Expected: {_EXPECTED_CONFIG}"
-            )
-
-        exclude_lists = data.get("exclude_lists", [])
-        if not isinstance(exclude_lists, list) or not all(isinstance(x, str) for x in exclude_lists):
-            raise ValidationError(
-                f"caldav config: 'exclude_lists' must be a list of strings if given. "
-                f"Expected: {_EXPECTED_CONFIG}"
-            )
-
-        url = data["url"] if data["url"].endswith("/") else data["url"] + "/"
+        values = checked.values
+        url = values["url"] if values["url"].endswith("/") else values["url"] + "/"
         return cls(
             url=url,
-            username=data["username"],
-            app_password=data["app_password"],
-            inbox_list=data["inbox_list"],
-            someday_list=data["someday_list"],
-            exclude_lists=tuple(exclude_lists),
+            username=values["username"],
+            app_password=values["app_password"],
+            inbox_list=values["inbox_list"],
+            someday_list=values["someday_list"],
+            exclude_lists=tuple(values["exclude_lists"]),
         )
 
 

tools/chemenu/tasks/config.py

@@ -12,14 +12,14 @@ from __future__ import annotations
 from dataclasses import dataclass
 from typing import Any
 
-from chemenu import config
+from chemenu import config, config_shapes
 from chemenu.errors import ValidationError
 
-# The providers this package ships an adapter for. Checked at read time so a
-# typo in `provider` fails here, at the one place that knows the full list,
-# rather than surfacing later as an unhelpful "unknown provider" from whatever
-# code tried to dispatch on it.
-KNOWN_PROVIDERS = ("superproductivity", "caldav")
+# The providers this package ships an adapter for: the sections of the file's
+# record (`config_shapes.TASKS`). Checked at read time so a typo in `provider`
+# fails here, rather than surfacing later as an unhelpful "unknown provider"
+# from whatever code tried to dispatch on it.
+KNOWN_PROVIDERS = tuple(section.key for section in config_shapes.TASKS.sections)
 
 
 @dataclass(frozen=True)
@@ -67,56 +67,16 @@ def read_config(root: "Any") -> "TasksConfig | None":
             "tracker the weekly review talks to, so a broken file is not treated as 'no "
             "tracker configured' - fix it or delete it deliberately."
         ) from exc
-    if not isinstance(data, dict):
-        raise ValidationError(f"{label} must contain a JSON object.")
-
-    expected = (
-        '{"schema": 1, "provider": "superproductivity", '
-        '"thresholds": {"stalled_waiting_days": 14, "unpaged_project_weeks": 3, '
-        '"someday_stale_months": 5}, "superproductivity": {"access": "api", '
-        '"api_base_url": "http://127.0.0.1:3876", "api_token": "..."} '
-        '(or {"access": "snapshot", "backups_dir": "..."} - see INSTALL.md)}'
-    )
-    try:
-        provider = str(data["provider"])
-        raw_thresholds = data["thresholds"]
-        stalled_waiting_days = int(raw_thresholds["stalled_waiting_days"])
-        unpaged_project_weeks = int(raw_thresholds["unpaged_project_weeks"])
-        someday_stale_months = int(raw_thresholds["someday_stale_months"])
-    except (KeyError, TypeError, ValueError) as exc:
+    shape = config_shapes.TASKS
+    checked = config_shapes.check(shape, data)
+    if checked.problems:
         raise ValidationError(
-            f"{label} is missing or misshapes a required field ({exc}). "
-            f"Expected: {expected}"
-        ) from exc
-
-    if provider not in KNOWN_PROVIDERS:
-        raise ValidationError(
-            f"{label}: unknown provider {provider!r}. "
-            f"Known: {', '.join(KNOWN_PROVIDERS)}"
+            f"{label}: {checked.message()}. Expected: {config_shapes.render_expected(shape)}"
         )
-    for field_name, value in (
-        ("stalled_waiting_days", stalled_waiting_days),
-        ("unpaged_project_weeks", unpaged_project_weeks),
-        ("someday_stale_months", someday_stale_months),
-    ):
-        if value <= 0:
-            raise ValidationError(
-                f"{label}: thresholds.{field_name} must be positive."
-            )
-
-    provider_config = data.get(provider)
-    if not isinstance(provider_config, dict):
-        raise ValidationError(
-            f"{label}: missing or non-object {provider!r} section "
-            f"holding that provider's own connection settings. Expected: {expected}"
-        )
-
+    provider = checked.values["provider"]
+    thresholds = checked.values["thresholds"]
     return TasksConfig(
         provider=provider,
-        thresholds=Thresholds(
-            stalled_waiting_days=stalled_waiting_days,
-            unpaged_project_weeks=unpaged_project_weeks,
-            someday_stale_months=someday_stale_months,
-        ),
-        provider_config=provider_config,
+        thresholds=Thresholds(**thresholds),
+        provider_config=data[provider],
     )

tools/chemenu/tasks/superproductivity.py

@@ -125,6 +125,7 @@ from datetime import date, datetime, timezone
 from pathlib import Path
 from typing import Any, Optional
 
+from chemenu import config_shapes
 from chemenu.errors import HumanInterventionRequired, ValidationError
 from chemenu.tasks.protocol import (
     OpenItem,
@@ -150,11 +151,13 @@ WAITING_TAG_TITLE = "waiting"
 # this module ever writes it.
 INBOX_PROJECT_ID = "INBOX_PROJECT"
 
-DEFAULT_API_BASE_URL = "http://127.0.0.1:3876"
+# This section's shape - its fields per `access`, and these values - is the
+# record `config_shapes.SUPERPRODUCTIVITY`.
+DEFAULT_API_BASE_URL = config_shapes.SUPERPRODUCTIVITY.field("api_base_url").default
 
 ACCESS_API = "api"
 ACCESS_SNAPSHOT = "snapshot"
-KNOWN_ACCESS = (ACCESS_API, ACCESS_SNAPSHOT)
+KNOWN_ACCESS = tuple(value for value, _ in config_shapes.SUPERPRODUCTIVITY.variants)
 
 # `electron/backup.ts` writes exactly this shape - a fixed-width timestamp,
 # no prefix. Restricting the glob to it (Gitea #133) is what keeps a manually
@@ -165,10 +168,6 @@ _SNAPSHOT_GLOB = (
     "[0-9][0-9][0-9][0-9][0-9][0-9].json"
 )
 
-_EXPECTED_API_CONFIG = '{"access": "api", "api_base_url": "http://127.0.0.1:3876", "api_token": "..."}'
-_EXPECTED_SNAPSHOT_CONFIG = '{"access": "snapshot", "backups_dir": "~/.config/superProductivity/backups"}'
-
-
 @dataclass(frozen=True)
 class SuperProductivityConfig:
     """This provider's own section of `.wikitool-tasks.json`
@@ -185,53 +184,22 @@ class SuperProductivityConfig:
 
     @classmethod
     def from_dict(cls, data: dict[str, Any]) -> "SuperProductivityConfig":
-        if not isinstance(data, dict):
+        shape = config_shapes.SUPERPRODUCTIVITY
+        checked = config_shapes.check(shape, data)
+        if checked.problems:
             raise ValidationError(
-                "superproductivity config must be an object. Expected one of: "
-                f"{_EXPECTED_API_CONFIG} or {_EXPECTED_SNAPSHOT_CONFIG}"
+                f"superproductivity config: {checked.message()}. "
+                f"Expected one of: {config_shapes.render_expected(shape)}"
             )
-        access = data.get("access")
-        if access not in KNOWN_ACCESS:
-            raise ValidationError(
-                "superproductivity config needs 'access', either 'api' or 'snapshot' - "
-                "required, no default and no fallback between them (Gitea #133). Got: "
-                f"{access!r}. Expected one of: {_EXPECTED_API_CONFIG} or {_EXPECTED_SNAPSHOT_CONFIG}"
-            )
-
-        if access == ACCESS_API:
-            extra = sorted(set(data) - {"access", "api_base_url", "api_token"})
-            if extra:
-                raise ValidationError(
-                    f"superproductivity config: access: 'api' does not take {extra} - a "
-                    "section names only one access path's own fields (Gitea #133). "
-                    f"Expected: {_EXPECTED_API_CONFIG}"
-                )
-            api_token = data.get("api_token")
-            if not isinstance(api_token, str) or not api_token:
-                raise ValidationError(
-                    "superproductivity config: api_token is required when access: 'api' - "
-                    "every endpoint but GET /health requires Authorization: Bearer <token>. "
-                    f"Expected: {_EXPECTED_API_CONFIG}"
-                )
-            api_base_url = str(data.get("api_base_url") or DEFAULT_API_BASE_URL)
-            return cls(access=access, backups_dir=None, api_base_url=api_base_url, api_token=api_token)
-
-        extra = sorted(set(data) - {"access", "backups_dir"})
-        if extra:
-            raise ValidationError(
-                f"superproductivity config: access: 'snapshot' does not take {extra} - a "
-                "section names only one access path's own fields (Gitea #133). "
-                f"Expected: {_EXPECTED_SNAPSHOT_CONFIG}"
-            )
-        backups_dir = data.get("backups_dir")
-        if not backups_dir:
-            raise ValidationError(
-                "superproductivity config: backups_dir is required when access: 'snapshot' - "
-                f"the read path has nothing to read otherwise. Expected: {_EXPECTED_SNAPSHOT_CONFIG}"
+        values = checked.values
+        if values["access"] == ACCESS_API:
+            return cls(
+                access=ACCESS_API, backups_dir=None,
+                api_base_url=values["api_base_url"], api_token=values["api_token"],
             )
         return cls(
-            access=access,
-            backups_dir=Path(backups_dir).expanduser(),
+            access=ACCESS_SNAPSHOT,
+            backups_dir=Path(values["backups_dir"]).expanduser(),
             api_base_url=DEFAULT_API_BASE_URL,
             api_token=None,
         )

tools/chemenu/telemetry/policy.py

@@ -33,10 +33,12 @@ import os
 from dataclasses import dataclass
 from pathlib import Path
 
-from chemenu import config
+from chemenu import config, config_shapes
 
-DEFAULT_MAX_SESSION_BYTES = 5 * 1024 * 1024
-DEFAULT_KEEP_SESSIONS = 250
+# The file's shape - and these two defaults with it - is the record in
+# `chemenu.config_shapes`, which is stdlib-only for exactly this import.
+DEFAULT_MAX_SESSION_BYTES = config_shapes.TELEMETRY.field("max_session_bytes").default
+DEFAULT_KEEP_SESSIONS = config_shapes.TELEMETRY.field("keep_sessions").default
 
 ENV_ENABLED = "WIKI_TRACE"
 ENV_MAX_SESSION_BYTES = "WIKI_TRACE_MAX_SESSION_BYTES"
@@ -76,6 +78,12 @@ def _positive_int(value: object) -> int | None:
     return parsed if parsed > 0 else None
 
 
+def _given(checked: "config_shapes.Checked", name: str) -> int | None:
+    """A cap the file sets and the record accepts, else None - so the
+    environment and the default decide, as for an absent file."""
+    return checked.values[name] if name in checked.given else None
+
+
 def _read_config(root: Path) -> dict:
     try:
         raw = (root / config.TELEMETRY_FILENAME).read_text(encoding="utf-8")
@@ -101,8 +109,10 @@ def _root_defaults(root: Path) -> _RootDefaults:
         else f"{config.RELEASE_STAMP_FILENAME} present - distributed instance, default off"
     )
 
-    data = _read_config(root)
-    file_enabled = bool(data["enabled"]) if "enabled" in data else None
+    # Silent on the hot path: a value the record does not accept is dropped,
+    # never an error. `doctor` reports it (`config_shapes.deviations`).
+    checked = config_shapes.check(config_shapes.TELEMETRY, _read_config(root))
+    file_enabled = checked.values.get("enabled")
     file_reason = (
         f"{config.TELEMETRY_FILENAME} sets enabled={file_enabled}"
         if file_enabled is not None
@@ -114,8 +124,8 @@ def _root_defaults(root: Path) -> _RootDefaults:
         form_reason=form_reason,
         file_enabled=file_enabled,
         file_reason=file_reason,
-        file_max_bytes=_positive_int(data.get("max_session_bytes")),
-        file_keep_sessions=_positive_int(data.get("keep_sessions")),
+        file_max_bytes=_given(checked, "max_session_bytes"),
+        file_keep_sessions=_given(checked, "keep_sessions"),
     )
     _cache[root] = result
     return result

tools/chemenu/tests/test_config_shapes.py

@@ -0,0 +1,608 @@
+"""`chemenu.config_shapes` - one record per hand-written configuration file,
+which the loaders read and `docs reference` renders.
+
+The drop-in half compares every loader against a copy of the loader as it stood
+before the records existed (the `_legacy_*` oracles below, kept verbatim in
+what they accept): any file it accepted must still be accepted with the same
+values, and any file it rejected must still be rejected. Only the wording of a
+rejection may differ.
+"""
+from __future__ import annotations
+
+import ast
+import dataclasses
+import json
+import re
+import sys
+from pathlib import Path
+
+import pytest
+import typer
+from typer.testing import CliRunner
+
+from chemenu import blocks, config, config_shapes, reference_doc
+from chemenu.cli import app
+from chemenu.commands import docs_verify, git_publish
+from chemenu.errors import ValidationError
+from chemenu.tasks import caldav, superproductivity
+from chemenu.tasks import config as tasks_config
+from chemenu.telemetry import policy
+
+runner = CliRunner()
+
+PACKAGE = Path(config_shapes.__file__).parent
+
+
+# --- the module itself -------------------------------------------------------
+
+
+def test_the_module_imports_only_the_stdlib_and_chemenu_config():
+    """A hook handler imports `chemenu.telemetry` on every tool call without
+    the venv, and the telemetry policy imports this module."""
+    tree = ast.parse(Path(config_shapes.__file__).read_text(encoding="utf-8"))
+    imported = set()
+    for node in ast.walk(tree):
+        if isinstance(node, ast.Import):
+            imported |= {alias.name for alias in node.names}
+        elif isinstance(node, ast.ImportFrom):
+            if node.module == "chemenu":
+                imported |= {f"chemenu.{alias.name}" for alias in node.names}
+            elif node.module != "__future__":
+                imported.add(node.module)
+    foreign = {
+        name for name in imported
+        if name != "chemenu.config" and name.split(".")[0] not in sys.stdlib_module_names
+    }
+    assert foreign == set()
+
+
+def test_the_telemetry_policy_reads_the_record():
+    assert policy.config_shapes is config_shapes
+    assert policy.DEFAULT_MAX_SESSION_BYTES == config_shapes.TELEMETRY.field("max_session_bytes").default
+    assert policy.DEFAULT_KEEP_SESSIONS == config_shapes.TELEMETRY.field("keep_sessions").default
+
+
+def test_one_record_per_file():
+    assert [shape.key for shape in config_shapes.SHAPES] == [
+        config.TELEMETRY_FILENAME, config.PUBLISH_REMOTES_FILENAME,
+        config.UPLOAD_CONFIG_FILENAME, config.TASKS_CONFIG_FILENAME,
+    ]
+
+
+LOADERS = ("telemetry/policy.py", "commands/git_publish.py", "upload.py", "tasks/config.py",
+           "tasks/superproductivity.py", "tasks/caldav.py")
+
+
+def test_no_hand_written_expected_text_is_left_in_a_loader():
+    pattern = re.compile(r"_EXPECTED_\w*\s*=|\bexpected\s*=\s*\(|Expected: \{\\?\"")
+    offenders = [
+        relative for relative in LOADERS
+        if pattern.search((PACKAGE / relative).read_text(encoding="utf-8"))
+    ]
+    assert offenders == []
+
+
+def test_the_known_names_follow_the_records():
+    assert tasks_config.KNOWN_PROVIDERS == ("superproductivity", "caldav")
+    assert superproductivity.KNOWN_ACCESS == ("api", "snapshot")
+
+
+# --- the legacy oracles: the loaders as they were before the records --------
+
+
+def _legacy_upload(data):
+    if not isinstance(data, dict):
+        raise ValidationError("not an object")
+    try:
+        identity_header = str(data.get("identity_header") or "X-Forwarded-User")
+        max_bytes = int(data["max_bytes"])
+        extensions = tuple(sorted({str(ext).lower() for ext in data["allowed_extensions"]}))
+        quota = data["quota"]
+        submissions_per_day = int(quota["submissions_per_day"])
+        bytes_per_day = int(quota["bytes_per_day"])
+    except (KeyError, TypeError, ValueError) as exc:
+        raise ValidationError("misshaped") from exc
+    if max_bytes <= 0 or submissions_per_day <= 0 or bytes_per_day <= 0:
+        raise ValidationError("not positive")
+    if not extensions or [ext for ext in extensions if not ext.startswith(".")]:
+        raise ValidationError("extensions")
+    return (identity_header, max_bytes, extensions, submissions_per_day, bytes_per_day)
+
+
+def _legacy_tasks(data):
+    if not isinstance(data, dict):
+        raise ValidationError("not an object")
+    try:
+        provider = str(data["provider"])
+        raw = data["thresholds"]
+        thresholds = (int(raw["stalled_waiting_days"]), int(raw["unpaged_project_weeks"]),
+                      int(raw["someday_stale_months"]))
+    except (KeyError, TypeError, ValueError) as exc:
+        raise ValidationError("misshaped") from exc
+    if provider not in ("superproductivity", "caldav"):
+        raise ValidationError("provider")
+    if any(value <= 0 for value in thresholds):
+        raise ValidationError("not positive")
+    if not isinstance(data.get(provider), dict):
+        raise ValidationError("section")
+    return (provider, thresholds, data.get(provider))
+
+
+def _legacy_sp(data):
+    if not isinstance(data, dict):
+        raise ValidationError("not an object")
+    access = data.get("access")
+    if access not in ("api", "snapshot"):
+        raise ValidationError("access")
+    if access == "api":
+        if set(data) - {"access", "api_base_url", "api_token"}:
+            raise ValidationError("extra")
+        token = data.get("api_token")
+        if not isinstance(token, str) or not token:
+            raise ValidationError("token")
+        return ("api", None, str(data.get("api_base_url") or "http://127.0.0.1:3876"), token)
+    if set(data) - {"access", "backups_dir"}:
+        raise ValidationError("extra")
+    if not data.get("backups_dir"):
+        raise ValidationError("backups_dir")
+    return ("snapshot", Path(data["backups_dir"]).expanduser(), "http://127.0.0.1:3876", None)
+
+
+_CALDAV_STRINGS = ("url", "username", "app_password", "inbox_list", "someday_list")
+
+
+def _legacy_caldav(data):
+    if not isinstance(data, dict):
+        raise ValidationError("not an object")
+    if [k for k in _CALDAV_STRINGS if not isinstance(data.get(k), str) or not data[k]]:
+        raise ValidationError("missing")
+    if set(data) - {*_CALDAV_STRINGS, "exclude_lists"}:
+        raise ValidationError("extra")
+    exclude = data.get("exclude_lists", [])
+    if not isinstance(exclude, list) or not all(isinstance(x, str) for x in exclude):
+        raise ValidationError("exclude_lists")
+    url = data["url"] if data["url"].endswith("/") else data["url"] + "/"
+    return (url, data["username"], data["app_password"], data["inbox_list"], data["someday_list"],
+            tuple(exclude))
+
+
+def _legacy_remotes(data):
+    urls = data.get("allowed_push_urls")
+    if not isinstance(urls, list) or not all(isinstance(u, str) for u in urls):
+        raise ValidationError("urls")
+    return urls
+
+
+def _legacy_telemetry(data):
+    def positive(value):
+        try:
+            parsed = int(value)
+        except (TypeError, ValueError):
+            return None
+        return parsed if parsed > 0 else None
+
+    data = data if isinstance(data, dict) else {}
+    return (
+        bool(data["enabled"]) if "enabled" in data else None,
+        positive(data.get("max_session_bytes")),
+        positive(data.get("keep_sessions")),
+    )
+
+
+# --- the loaders today, reduced to the same tuples --------------------------
+
+
+def _upload(data, tmp_path):
+    (tmp_path / config.UPLOAD_CONFIG_FILENAME).write_text(json.dumps(data), encoding="utf-8")
+    from chemenu import upload
+
+    cfg = upload.read_config(tmp_path)
+    return (cfg.identity_header, cfg.max_bytes, cfg.allowed_extensions, cfg.submissions_per_day,
+            cfg.bytes_per_day)
+
+
+def _tasks(data, tmp_path):
+    (tmp_path / config.TASKS_CONFIG_FILENAME).write_text(json.dumps(data), encoding="utf-8")
+    cfg = tasks_config.read_config(tmp_path)
+    t = cfg.thresholds
+    return (cfg.provider, (t.stalled_waiting_days, t.unpaged_project_weeks, t.someday_stale_months),
+            cfg.provider_config)
+
+
+def _sp(data, tmp_path):
+    cfg = superproductivity.SuperProductivityConfig.from_dict(data)
+    return (cfg.access, cfg.backups_dir, cfg.api_base_url, cfg.api_token)
+
+
+def _caldav(data, tmp_path):
+    cfg = caldav.CalDAVConfig.from_dict(data)
+    return (cfg.url, cfg.username, cfg.app_password, cfg.inbox_list, cfg.someday_list,
+            cfg.exclude_lists)
+
+
+def _remotes(data, tmp_path):
+    (tmp_path / config.PUBLISH_REMOTES_FILENAME).write_text(json.dumps(data), encoding="utf-8")
+    try:
+        return git_publish.read_allowed_push_urls()
+    except typer.Exit as exc:
+        raise ValidationError("fail()") from exc
+
+
+def _telemetry(data, tmp_path):
+    (tmp_path / config.TELEMETRY_FILENAME).write_text(json.dumps(data), encoding="utf-8")
+    policy.reset_cache()
+    defaults = policy._root_defaults(tmp_path.resolve())
+    policy.reset_cache()
+    return (defaults.file_enabled, defaults.file_max_bytes, defaults.file_keep_sessions)
+
+
+UPLOAD_OK = {
+    "schema": 1, "identity_header": "X-Forwarded-User", "max_bytes": 10485760,
+    "allowed_extensions": [".md", ".txt", ".pdf", ".html", ".csv", ".json", ".png", ".jpg"],
+    "quota": {"submissions_per_day": 20, "bytes_per_day": 52428800},
+}
+TASKS_OK = {
+    "schema": 1, "provider": "superproductivity",
+    "thresholds": {"stalled_waiting_days": 14, "unpaged_project_weeks": 3, "someday_stale_months": 5},
+    "superproductivity": {"access": "api", "api_base_url": "http://127.0.0.1:3876", "api_token": "t"},
+}
+CALDAV_OK = {
+    "url": "https://host/remote.php/dav/calendars/me", "username": "me", "app_password": "pw",
+    "inbox_list": "Inbox", "someday_list": "Someday", "exclude_lists": ["Shopping"],
+}
+
+
+def _with(base, **changes):
+    data = json.loads(json.dumps(base))
+    for key, value in changes.items():
+        target = data
+        *parents, leaf = key.split("__")
+        for parent in parents:
+            target = target[parent]
+        if value is _DROP:
+            del target[leaf]
+        else:
+            target[leaf] = value
+    return data
+
+
+_DROP = object()
+
+CASES = [
+    # upload: the documented file, then every tolerance and every rejection
+    ("upload", UPLOAD_OK),
+    ("upload", _with(UPLOAD_OK, schema=_DROP)),
+    ("upload", _with(UPLOAD_OK, identity_header=_DROP)),
+    ("upload", _with(UPLOAD_OK, identity_header="")),
+    ("upload", _with(UPLOAD_OK, max_bytes="10485760")),
+    ("upload", _with(UPLOAD_OK, quota__bytes_per_day="100")),
+    ("upload", _with(UPLOAD_OK, allowed_extensions=[".MD", ".md", ".Pdf"])),
+    ("upload", _with(UPLOAD_OK, surplus="ignored")),
+    ("upload", _with(UPLOAD_OK, quota__surplus=1)),
+    ("upload", _with(UPLOAD_OK, max_bytes=0)),
+    ("upload", _with(UPLOAD_OK, max_bytes="ten")),
+    ("upload", _with(UPLOAD_OK, max_bytes=_DROP)),
+    ("upload", _with(UPLOAD_OK, allowed_extensions=["md"])),
+    ("upload", _with(UPLOAD_OK, allowed_extensions=[])),
+    ("upload", _with(UPLOAD_OK, allowed_extensions=".md")),
+    ("upload", _with(UPLOAD_OK, quota=[1, 2])),
+    ("upload", _with(UPLOAD_OK, quota__submissions_per_day=-1)),
+    ("upload", [1, 2]),
+    # tasks
+    ("tasks", TASKS_OK),
+    ("tasks", _with(TASKS_OK, schema=_DROP)),
+    ("tasks", _with(TASKS_OK, thresholds__stalled_waiting_days="14")),
+    ("tasks", _with(TASKS_OK, caldav={"url": "unused"})),
+    ("tasks", _with(TASKS_OK, surplus=True)),
+    ("tasks", _with(TASKS_OK, thresholds__surplus=1)),
+    ("tasks", _with(TASKS_OK, provider="asana")),
+    ("tasks", _with(TASKS_OK, provider=_DROP)),
+    ("tasks", _with(TASKS_OK, thresholds__someday_stale_months=0)),
+    ("tasks", _with(TASKS_OK, thresholds__unpaged_project_weeks=_DROP)),
+    ("tasks", _with(TASKS_OK, thresholds="soon")),
+    ("tasks", _with(TASKS_OK, superproductivity=_DROP)),
+    ("tasks", _with(TASKS_OK, superproductivity="api")),
+    ("tasks", _with(TASKS_OK, provider="caldav")),
+    ("tasks", "not an object"),
+    # the Super Productivity section
+    ("sp", {"access": "api", "api_base_url": "http://127.0.0.1:3876", "api_token": "t"}),
+    ("sp", {"access": "api", "api_token": "t"}),
+    ("sp", {"access": "api", "api_token": "t", "api_base_url": ""}),
+    ("sp", {"access": "snapshot", "backups_dir": "~/.config/superProductivity/backups"}),
+    ("sp", {"access": "api"}),
+    ("sp", {"access": "api", "api_token": ""}),
+    ("sp", {"access": "api", "api_token": 7}),
+    ("sp", {"access": "api", "api_token": "t", "backups_dir": "/x"}),
+    ("sp", {"access": "snapshot", "backups_dir": "/x", "api_token": "t"}),
+    ("sp", {"access": "snapshot"}),
+    ("sp", {"access": "snapshot", "backups_dir": ""}),
+    ("sp", {"access": "auto", "backups_dir": "/x"}),
+    ("sp", {"backups_dir": "/x"}),
+    ("sp", ["api"]),
+    # the CalDAV section
+    ("caldav", CALDAV_OK),
+    ("caldav", _with(CALDAV_OK, exclude_lists=_DROP)),
+    ("caldav", _with(CALDAV_OK, url="https://host/dav/")),
+    ("caldav", _with(CALDAV_OK, url=_DROP)),
+    ("caldav", _with(CALDAV_OK, username="")),
+    ("caldav", _with(CALDAV_OK, app_password=5)),
+    ("caldav", _with(CALDAV_OK, exclude_lists="Shopping")),
+    ("caldav", _with(CALDAV_OK, exclude_lists=[1])),
+    ("caldav", _with(CALDAV_OK, backups_dir="/x")),
+    # remotes
+    ("remotes", {"schema": 1, "allowed_push_urls": ["git@example.org:me/my-wiki.git"]}),
+    ("remotes", {"allowed_push_urls": []}),
+    ("remotes", {"allowed_push_urls": ["a"], "surplus": 1}),
+    ("remotes", {"allowed_push_urls": "not-a-list"}),
+    ("remotes", {"allowed_push_urls": [1]}),
+    ("remotes", {}),
+    # telemetry: never rejected, only read differently
+    ("telemetry", {"enabled": True, "max_session_bytes": 5242880, "keep_sessions": 250}),
+    ("telemetry", {}),
+    ("telemetry", {"enabled": "no"}),
+    ("telemetry", {"enabled": 0}),
+    ("telemetry", {"enabled": None}),
+    ("telemetry", {"max_session_bytes": "100", "keep_sessions": 3.7}),
+    ("telemetry", {"max_session_bytes": 0, "keep_sessions": -2}),
+    ("telemetry", {"max_session_bytes": "lots", "keep_sessions": [1]}),
+    ("telemetry", {"surplus": 1, "schema": 1}),
+    ("telemetry", [True]),
+]
+
+ORACLES = {
+    "upload": (_legacy_upload, _upload),
+    "tasks": (_legacy_tasks, _tasks),
+    "sp": (_legacy_sp, _sp),
+    "caldav": (_legacy_caldav, _caldav),
+    "remotes": (_legacy_remotes, _remotes),
+    "telemetry": (_legacy_telemetry, _telemetry),
+}
+
+
+def _outcome(function, data, *args):
+    try:
+        return ("accepted", function(data, *args))
+    except ValidationError:
+        return ("rejected", None)
+
+
+@pytest.mark.parametrize("loader,data", CASES, ids=[f"{n}-{i}" for i, (n, _) in enumerate(CASES)])
+def test_every_loader_accepts_and_rejects_exactly_what_it_did_before(loader, data, tmp_path, monkeypatch):
+    legacy, current = ORACLES[loader]
+    monkeypatch.setattr(config, "ROOT", tmp_path)
+    assert _outcome(current, data, tmp_path) == _outcome(legacy, data)
+
+
+def test_the_cases_cover_both_outcomes_for_every_loader(tmp_path):
+    for loader, (legacy, _) in ORACLES.items():
+        if loader == "telemetry":
+            continue
+        outcomes = {_outcome(legacy, data)[0] for name, data in CASES if name == loader}
+        assert outcomes == {"accepted", "rejected"}, loader
+
+
+def test_a_rejection_keeps_its_cause_and_renders_the_expected_text(tmp_path):
+    (tmp_path / config.UPLOAD_CONFIG_FILENAME).write_text(
+        json.dumps(_with(UPLOAD_OK, max_bytes=0)), encoding="utf-8"
+    )
+    from chemenu import upload
+
+    with pytest.raises(ValidationError) as caught:
+        upload.read_config(tmp_path)
+    message = str(caught.value)
+    assert "max_bytes must be positive" in message
+    assert f"Expected: {config_shapes.render_expected(config_shapes.UPLOAD)}" in message
+
+
+# --- the loaders take their expectations from the record --------------------
+
+
+def _add_required(shape, name="added"):
+    return dataclasses.replace(
+        shape, fields=shape.fields + (config_shapes.Field(name, "string", "x", required=True),)
+    )
+
+
+def _not_required(shape, name, default):
+    fields = tuple(
+        dataclasses.replace(f, required=False, default=default) if f.name == name else f
+        for f in shape.fields
+    )
+    return dataclasses.replace(shape, fields=fields)
+
+
+def test_upload_follows_a_field_made_required_or_optional(tmp_path, monkeypatch):
+    from chemenu import upload
+
+    (tmp_path / config.UPLOAD_CONFIG_FILENAME).write_text(json.dumps(UPLOAD_OK), encoding="utf-8")
+    monkeypatch.setattr(config_shapes, "UPLOAD", _add_required(config_shapes.UPLOAD))
+    with pytest.raises(ValidationError, match="added is missing"):
+        upload.read_config(tmp_path)
+
+    monkeypatch.undo()
+    (tmp_path / config.UPLOAD_CONFIG_FILENAME).write_text(
+        json.dumps(_with(UPLOAD_OK, max_bytes=_DROP)), encoding="utf-8"
+    )
+    with pytest.raises(ValidationError):
+        upload.read_config(tmp_path)
+    monkeypatch.setattr(config_shapes, "UPLOAD", _not_required(config_shapes.UPLOAD, "max_bytes", 99))
+    assert upload.read_config(tmp_path).max_bytes == 99
+
+
+def test_tasks_follows_a_field_made_required(tmp_path, monkeypatch):
+    (tmp_path / config.TASKS_CONFIG_FILENAME).write_text(json.dumps(TASKS_OK), encoding="utf-8")
+    assert tasks_config.read_config(tmp_path) is not None
+    monkeypatch.setattr(config_shapes, "TASKS", _add_required(config_shapes.TASKS))
+    with pytest.raises(ValidationError, match="added is missing"):
+        tasks_config.read_config(tmp_path)
+
+
+def test_remotes_follows_a_field_made_required(tmp_path, monkeypatch):
+    monkeypatch.setattr(config, "ROOT", tmp_path)
+    (tmp_path / config.PUBLISH_REMOTES_FILENAME).write_text(
+        json.dumps({"allowed_push_urls": ["u"]}), encoding="utf-8"
+    )
+    assert git_publish.read_allowed_push_urls() == ["u"]
+    monkeypatch.setattr(config_shapes, "REMOTES", _add_required(config_shapes.REMOTES))
+    with pytest.raises(typer.Exit):
+        git_publish.read_allowed_push_urls()
+
+
+def test_caldav_follows_a_field_made_required(monkeypatch):
+    data = _with(CALDAV_OK, exclude_lists=_DROP)
+    assert caldav.CalDAVConfig.from_dict(data).exclude_lists == ()
+    fields = tuple(
+        dataclasses.replace(f, required=True) if f.name == "exclude_lists" else f
+        for f in config_shapes.CALDAV.fields
+    )
+    monkeypatch.setattr(config_shapes, "CALDAV", dataclasses.replace(config_shapes.CALDAV, fields=fields))
+    with pytest.raises(ValidationError, match="exclude_lists is missing"):
+        caldav.CalDAVConfig.from_dict(data)
+
+
+def test_superproductivity_follows_a_field_added_to_a_variant(monkeypatch):
+    data = {"access": "api", "api_token": "t", "timeout": "5"}
+    with pytest.raises(ValidationError, match="does not take"):
+        superproductivity.SuperProductivityConfig.from_dict(data)
+    shape = config_shapes.SUPERPRODUCTIVITY
+    api = dict(shape.variants)["api"] + (config_shapes.Field("timeout", "string", "x"),)
+    variants = (("api", api),) + tuple(v for v in shape.variants if v[0] != "api")
+    monkeypatch.setattr(config_shapes, "SUPERPRODUCTIVITY", dataclasses.replace(shape, variants=variants))
+    assert superproductivity.SuperProductivityConfig.from_dict(data).api_token == "t"
+
+
+def test_telemetry_follows_a_condition_in_the_record(tmp_path, monkeypatch):
+    (tmp_path / config.TELEMETRY_FILENAME).write_text(json.dumps({"keep_sessions": 500}), encoding="utf-8")
+    policy.reset_cache()
+    assert policy.resolve(tmp_path).keep_sessions == 500
+    shape = config_shapes.TELEMETRY
+    fields = tuple(
+        dataclasses.replace(f, check=lambda v: None if v < 100 else "too many") if f.name == "keep_sessions" else f
+        for f in shape.fields
+    )
+    monkeypatch.setattr(config_shapes, "TELEMETRY", dataclasses.replace(shape, fields=fields))
+    policy.reset_cache()
+    assert policy.resolve(tmp_path).keep_sessions == policy.DEFAULT_KEEP_SESSIONS
+    policy.reset_cache()
+
+
+# --- deviations: what doctor reports for telemetry -------------------------
+
+
+@pytest.mark.parametrize("data,field", [
+    ({"enabled": "yes"}, "enabled"),
+    ({"enabled": 1}, "enabled"),
+    ({"max_session_bytes": 0}, "max_session_bytes"),
+    ({"keep_sessions": "250"}, "keep_sessions"),
+    ({"keep_sessions": 2.5}, "keep_sessions"),
+    ({"enabled": True, "verbose": True}, "verbose"),
+])
+def test_a_telemetry_deviation_names_its_field(data, field):
+    found = config_shapes.deviations(config_shapes.TELEMETRY, data)
+    assert found and all(field in entry for entry in found)
+
+
+def test_a_conforming_telemetry_file_has_no_deviation():
+    for example in config_shapes.TELEMETRY.examples:
+        assert config_shapes.deviations(config_shapes.TELEMETRY, example) == []
+    assert config_shapes.deviations(config_shapes.TELEMETRY, {"schema": 1, "enabled": False}) == []
+
+
+def test_every_example_passes_its_own_record():
+    for shape in config_shapes.SHAPES:
+        for example in shape.examples:
+            assert config_shapes.check(shape, example).problems == [], shape.key
+    for section in config_shapes.TASKS.sections:
+        for example in section.examples:
+            assert config_shapes.check(section, example).problems == [], section.key
+
+
+# --- the generated regions --------------------------------------------------
+
+
+def _page() -> str:
+    return (config.ROOT / reference_doc.CONFIGURATION).read_text(encoding="utf-8")
+
+
+def _all_fields(shape, prefix=""):
+    for spec in shape.fields + tuple(f for _, fs in shape.variants for f in fs):
+        yield prefix + spec.name
+        yield from (prefix + spec.name + "." + sub.name for sub in spec.fields)
+
+
+def test_the_page_carries_one_region_per_file_and_every_field_in_it():
+    page = _page()
+    for name, attr in reference_doc.CONFIG_REGIONS:
+        shape = getattr(config_shapes, attr)
+        region = blocks.find(page, name)
+        assert region is not None, name
+        for path in _all_fields(shape):
+            assert f"`{path}`" in region, (name, path)
+        for section in shape.sections:
+            for path in _all_fields(section):
+                assert f"`{path}`" in region, (name, path)
+
+
+@pytest.mark.parametrize("relative", [reference_doc.CONFIGURATION, "docs/how-to/set-up-the-mcp-server.md"])
+def test_no_json_example_for_these_files_stands_outside_a_region(relative):
+    text = blocks.mask_regions((config.ROOT / relative).read_text(encoding="utf-8"))
+    keys = {f.name for shape in config_shapes.SHAPES for f in shape.fields if f.name != "schema"}
+    keys |= {f.name for s in config_shapes.TASKS.sections for f in s.fields}
+    keys |= {f.name for s in config_shapes.TASKS.sections for _, fs in s.variants for f in fs}
+    for block in re.findall(r"```json\n(.*?)```", text, re.S):
+        assert not any(f'"{key}"' in block for key in keys), block
+
+
+def _only_config_regions(monkeypatch):
+    real = reference_doc.regions
+    monkeypatch.setattr(
+        reference_doc, "regions",
+        lambda *a, **k: [r for r in real(*a, **k) if r.path == reference_doc.CONFIGURATION],
+    )
+
+
+def _hand_written(text: str) -> str:
+    for name, _ in reference_doc.CONFIG_REGIONS:
+        text = re.sub(
+            re.escape(blocks.open_marker(name)) + ".*?" + re.escape(blocks.close_marker(name)),
+            blocks.open_marker(name), text, flags=re.S,
+        )
+    return text
+
+
+def test_apply_fills_every_region_of_the_page_and_a_second_run_changes_nothing(tmp_path, monkeypatch):
+    """Four regions in one file: each is refreshed into the text the one
+    before it left, or only the last would survive."""
+    target = tmp_path / reference_doc.CONFIGURATION
+    target.parent.mkdir(parents=True)
+    real = _page()
+    emptied = real
+    for name, _ in reference_doc.CONFIG_REGIONS:
+        emptied = blocks.replace(emptied, name, blocks.open_marker(name) + "\n" + blocks.close_marker(name) + "\n")
+    target.write_text(emptied, encoding="utf-8")
+    monkeypatch.setattr(config, "ROOT", tmp_path)
+    _only_config_regions(monkeypatch)
+
+    result = runner.invoke(app, ["docs", "reference", "--apply"])
+    assert result.exit_code == 0, result.output
+    applied = target.read_text(encoding="utf-8")
+    assert applied == real
+    assert _hand_written(applied) == _hand_written(emptied)
+
+    result = runner.invoke(app, ["docs", "reference", "--apply"])
+    assert "already current" in result.output
+    assert target.read_text(encoding="utf-8") == applied
+
+
+def test_a_changed_record_without_apply_fails_docs_verify(monkeypatch):
+    assert not [i for i in docs_verify.check_reference_regions() if "configuration.md" in i]
+    monkeypatch.setattr(config_shapes, "REMOTES", _add_required(config_shapes.REMOTES))
+    issues = [i for i in docs_verify.check_reference_regions() if reference_doc.CONFIGURATION in i]
+    assert len(issues) == 1
+    assert "reference-config-remotes" in issues[0] and "docs reference --apply" in issues[0]
+
+
+def test_the_configuration_regions_are_checked_in_an_instance_too():
+    regions = [r for r in reference_doc.regions() if r.path == reference_doc.CONFIGURATION]
+    assert [r.name for r in regions] == [name for name, _ in reference_doc.CONFIG_REGIONS]
+    assert not any(r.origin_only for r in regions)

tools/chemenu/tests/test_doctor.py

@@ -521,6 +521,49 @@ def test_telemetry_never_fails_either_way(instance):
     assert _status(doctor.run_doctor(), "telemetry") == "OK"
 
 
+def _telemetry_with(text: "str | None"):
+    """`check_telemetry` against a `.wikitool-telemetry.json` holding `text`
+    (absent for None), plus what the policy itself makes of it."""
+    from chemenu.telemetry import policy
+
+    path = config.ROOT / config.TELEMETRY_FILENAME
+    if text is None:
+        path.unlink(missing_ok=True)
+    else:
+        path.write_text(text, encoding="utf-8")
+    policy.reset_cache()
+    try:
+        return doctor.check_telemetry(), policy.resolve(config.ROOT)
+    finally:
+        policy.reset_cache()
+
+
+@pytest.mark.parametrize("text", [None, '{"enabled": false}', '{"schema": 1, "enabled": true, '
+                                  '"max_session_bytes": 1000, "keep_sessions": 9}'])
+def test_telemetry_stays_ok_for_an_absent_or_conforming_file(instance, text):
+    check, _ = _telemetry_with(text)
+    assert check.status == "OK"
+    assert check.fix is None
+
+
+@pytest.mark.parametrize("text,named,policy_reads", [
+    ("{not json", "unreadable", (True, 5 * 1024 * 1024, 250)),
+    ("[true]", "not a JSON object", (True, 5 * 1024 * 1024, 250)),
+    ('{"enabled": "no"}', "enabled", (True, 5 * 1024 * 1024, 250)),
+    ('{"max_session_bytes": 0}', "max_session_bytes", (True, 5 * 1024 * 1024, 250)),
+    ('{"keep_sessions": "12"}', "keep_sessions", (True, 5 * 1024 * 1024, 12)),
+    ('{"enabled": false, "keep_session": 12}', "keep_session", (False, 5 * 1024 * 1024, 250)),
+])
+def test_telemetry_warns_on_what_the_policy_reads_differently(instance, text, named, policy_reads):
+    """The policy stays silent and reads the file exactly as before; `doctor`
+    is where the deviation becomes visible, by field."""
+    check, pol = _telemetry_with(text)
+    assert check.status == "WARN"
+    assert named in check.detail
+    assert config.TELEMETRY_FILENAME in check.fix
+    assert (pol.enabled, pol.max_session_bytes, pol.keep_sessions) == policy_reads
+
+
 def test_telemetry_reports_the_current_session_count_and_byte_total(instance, monkeypatch):
     from chemenu.telemetry.writer import TRACE_FILE, trace_root
 

tools/chemenu/tests/test_reference_doc.py

@@ -266,6 +266,7 @@ def _page_type(description: str = "A fixture type.") -> reference_doc.PageType:
 
 @pytest.fixture
 def tree(tmp_path, monkeypatch):
+    real_root = config.ROOT
     monkeypatch.setattr(config, "ROOT", tmp_path)
     records = {"frobnicate": _record("frobnicate"), "gate keep": _record("gate keep", ("keep",))}
     monkeypatch.setattr(cli_contract, "all_records", lambda: dict(records))
@@ -283,6 +284,13 @@ def tree(tmp_path, monkeypatch):
         path.write_text(
             f"{head}\n{blocks.open_marker(name)}\n{blocks.close_marker(name)}\n", encoding="utf-8"
         )
+    # The configuration regions render from the real records, not from the
+    # fixture's; the current page keeps them out of what these tests count.
+    # test_config_shapes.py covers that page.
+    configuration = tmp_path / reference_doc.CONFIGURATION
+    configuration.write_text(
+        (real_root / reference_doc.CONFIGURATION).read_text(encoding="utf-8"), encoding="utf-8"
+    )
     return tmp_path, records
 
 

tools/chemenu/upload.py

@@ -49,10 +49,10 @@ from dataclasses import dataclass
 from pathlib import Path
 from typing import Any, Optional
 
-from chemenu import config
+from chemenu import config, config_shapes
 from chemenu.errors import ValidationError
 
-DEFAULT_IDENTITY_HEADER = "X-Forwarded-User"
+DEFAULT_IDENTITY_HEADER = config_shapes.UPLOAD.field("identity_header").default
 
 # Submissions older than this stop counting toward a submitter's quota. A
 # rolling window rather than a calendar day - "resets at midnight" is a
@@ -99,46 +99,20 @@ def read_config(root: "Path | str") -> Optional[UploadConfig]:
             "upload tool is offered at all, so a broken file is not treated as 'no limits' - "
             "fix it or delete it deliberately."
         ) from exc
-    if not isinstance(data, dict):
-        raise ValidationError(f"{config.UPLOAD_CONFIG_FILENAME} must contain a JSON object.")
-
-    expected = (
-        '{"schema": 1, "identity_header": "X-Forwarded-User", "max_bytes": 10485760, '
-        '"allowed_extensions": [".md", ".pdf"], '
-        '"quota": {"submissions_per_day": 20, "bytes_per_day": 52428800}}'
-    )
-    try:
-        identity_header = str(data.get("identity_header") or DEFAULT_IDENTITY_HEADER)
-        max_bytes = int(data["max_bytes"])
-        raw_extensions = data["allowed_extensions"]
-        extensions = tuple(sorted({str(ext).lower() for ext in raw_extensions}))
-        quota = data["quota"]
-        submissions_per_day = int(quota["submissions_per_day"])
-        bytes_per_day = int(quota["bytes_per_day"])
-    except (KeyError, TypeError, ValueError) as exc:
+    shape = config_shapes.UPLOAD
+    checked = config_shapes.check(shape, data)
+    if checked.problems:
         raise ValidationError(
-            f"{config.UPLOAD_CONFIG_FILENAME} is missing or misshapes a required field ({exc}). "
-            f"Expected: {expected}"
-        ) from exc
-
-    if max_bytes <= 0:
-        raise ValidationError(f"{config.UPLOAD_CONFIG_FILENAME}: max_bytes must be positive.")
-    if submissions_per_day <= 0 or bytes_per_day <= 0:
-        raise ValidationError(f"{config.UPLOAD_CONFIG_FILENAME}: quota values must be positive.")
-    if not extensions:
-        raise ValidationError(f"{config.UPLOAD_CONFIG_FILENAME}: allowed_extensions must not be empty.")
-    not_dotted = [ext for ext in extensions if not ext.startswith(".")]
-    if not_dotted:
-        raise ValidationError(
-            f"{config.UPLOAD_CONFIG_FILENAME}: allowed_extensions must each start with '.': {not_dotted}"
+            f"{config.UPLOAD_CONFIG_FILENAME}: {checked.message()}. "
+            f"Expected: {config_shapes.render_expected(shape)}"
         )
-
+    values = checked.values
     return UploadConfig(
-        identity_header=identity_header,
-        max_bytes=max_bytes,
-        allowed_extensions=extensions,
-        submissions_per_day=submissions_per_day,
-        bytes_per_day=bytes_per_day,
+        identity_header=values["identity_header"],
+        max_bytes=values["max_bytes"],
+        allowed_extensions=values["allowed_extensions"],
+        submissions_per_day=values["quota"]["submissions_per_day"],
+        bytes_per_day=values["quota"]["bytes_per_day"],
     )