From 55f65c1ab12c24f845407116b016169338d368a4 Mon Sep 17 00:00:00 2001 From: Torben Nehmer Date: Tue, 15 Sep 2026 19:45:33 +0200 Subject: [PATCH] fix: types/type-spec.schema.yaml enforced against real type-spec frontmatter, doc-pull-through.md docs/-page count corrected (closes #105) Files changed: - CHANGES.md - VERSION - instructions/dev/doc-pull-through.md - tools/CONTRACT.md - tools/chemenu/commands/docs_verify.py - tools/chemenu/tests/test_docs_verify.py - types/type-spec.md - types/type-spec.schema.yaml --- CHANGES.md | 37 ++++++++++++++++++- VERSION | 2 +- instructions/dev/doc-pull-through.md | 2 +- tools/CONTRACT.md | 4 +-- tools/chemenu/commands/docs_verify.py | 46 +++++++++++++++++++++++- tools/chemenu/tests/test_docs_verify.py | 48 ++++++++++++++++++++++++- types/type-spec.md | 2 ++ types/type-spec.schema.yaml | 27 ++++++++++++++ 8 files changed, 161 insertions(+), 7 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index babff48..d9a9185 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -59,7 +59,7 @@ concern - readable here, never shipped as something to parse. --- -## 6.0.0-beta.6 - 2026-09-15 - types/: Seiten-Type-Spec-Anleitungsprosa in stackeigene guidance-Datei ausgelagert +## 6.0.0-beta.7 - 2026-09-15 - types/type-spec.schema.yaml enforced against real type-spec frontmatter **Author:** Torben Nehmer @@ -79,6 +79,7 @@ concern - readable here, never shipped as something to parse. **Medium impact** - docs verify: ein nur als .template ausgeliefertes Linkziel gilt als aufgeloest - Control-Plane-Sprache universell: Achse ist das Publikum, kein Instanz-Schalter +- types/type-spec.schema.yaml enforced against real type-spec frontmatter **Low impact** - gates.md/session-setup.md: die Budget-Ausnahme von version regrade haengt an der Aufrufform @@ -307,6 +308,40 @@ ein `dist upgrade` schreibt verbesserte Guidance-Prosa in eine adoptierte Instan Type-Spec selbst nie im Stamp stand, `types describe` komponiert beide Haelften in JSON und Textausgabe getrennt nachweisbar). +### types/type-spec.schema.yaml enforced against real type-spec frontmatter + +Bei der Vorbereitung der Aenderung oben fiel auf: `types/type-spec.schema.yaml` traegt +`additionalProperties: false`, kannte aber `root:` und `capture_fields:` nicht, obwohl +`types/instruction.md` bzw. `types/source.md` beide Felder tragen und `type_resolver.get_root()`/ +`get_capture_fields()` sie lesen. Gegen das Schema validiert waeren beide Type-Specs ungueltig +gewesen. Dass es niemandem auffiel, war der eigentliche Befund: Type-Spec-Frontmatter wurde +nirgends gegen sein eigenes Schema validiert - `resolver.validate_frontmatter()` lief nur ueber +`kb/`-Seiten, neu erzeugte Seiten und Instruktionsdateien, nie ueber die Type-Specs selbst. +`TypeResolver._validate_type_spec()`, der einzige Weg, den der Selbstbezug `type: types/type-spec.md` +nimmt, prueft ausschliesslich, ob `type`/`name`/`description` vorhanden sind. + +Beide fehlenden Felder ergaenzt (`root:` als Enum `kb`/`repo`, `capture_fields:` als Liste wie +`page_ref_fields:`), dazu `guidance:` (seit der Aenderung oben real benutzt, aber noch nie im +Schema). `docs verify` bekommt eine neue Pruefung: jede Datei unter `types/` mit +`type: types/type-spec.md` validiert jetzt gegen `types/type-spec.schema.yaml` +(`check_type_spec_frontmatter()`, wiederverwendet `resolver.list_type_specs()` statt eines zweiten +Parse-Durchlaufs). `types/type-spec.md` § Validation Contract und die beiden `docs verify`-Zeilen +in `tools/CONTRACT.md` nennen das jetzt. + +Daneben ein zweiter, unabhaengiger Befund derselben Aufraeumrunde behoben: +`instructions/dev/doc-pull-through.md` verwies fuer `docs/`-Seiten weiter auf "AGENTS.md § File +naming lists all four" - der Zaehler in AGENTS.md selbst war beim vorigen Bump schon auf fuenf +(plus eine sechste, nur von `CLAUDE.md` aus verlinkte) korrigiert worden, diese eine verbliebene +Stelle nicht. + +Grenzuebertritt-Frage geprueft und verneint: additiv in beide Richtungen - eine bestehende Instanz +validiert bereits (0 Befunde gegen den realen Baum), und ein Type-Spec ohne die drei neuen Felder +bleibt unveraendert gueltig. `--patch`, kein `--breaking`, keine neue Migration noetig. + +Verifiziert: `tools/wikitool docs verify`/`instructions verify` gruen, 1263 Tests gruen (2 neu: +alle Type-Specs dieses Repos validieren gegen ihr eigenes Schema; ein Type-Spec mit einem dem +Schema unbekannten Feld wird gemeldet, mit Dateiname und Feldname in der Meldung). + ## 5.1.0 - 2026-09-12 - changelog: Kandidaten-Eintrag nach Impact gruppiert, version regrade zur Nachkorrektur, version release verlangt eine Zusammenfassung **Author:** Torben Nehmer diff --git a/VERSION b/VERSION index 9ef549a..09deee8 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -6.0.0-beta.6 +6.0.0-beta.7 diff --git a/instructions/dev/doc-pull-through.md b/instructions/dev/doc-pull-through.md index 83910e5..047bfc0 100644 --- a/instructions/dev/doc-pull-through.md +++ b/instructions/dev/doc-pull-through.md @@ -37,7 +37,7 @@ touched; a row that does not apply needs no action. | A stage's authoring rules (`raw/`, `kb/`, `types/`, `reports/`, `work/`, `tools/`, `instructions/`) | The touched `/CONTRACT.md` | | A rule, gate, or invariant `AGENTS.md` itself states | The relevant `AGENTS.md` section (Invariants, Gates, File naming, Routing, ...) | | A workflow, stage, or command a human operates by hand | Whichever of `README.md`, `EVALS.md`, `tools/README.md`, `INSTALL.md`, `DEVELOPMENT.md` names it - AGENTS.md § File naming says which document is for which reader | - | The reasoning behind a gate, boundary, or design decision | The `docs/` page that carries it, if one exists (AGENTS.md § File naming lists all four) | + | The reasoning behind a gate, boundary, or design decision | The `docs/` page that carries it, if one exists (AGENTS.md § File naming lists all five reached from AGENTS.md itself, plus a sixth reached only from CLAUDE.md) | | A skill's own step sequence or catalogue | The skill's `SKILL.md` source under `instructions//` or `instructions/dev//` | 3. **A heading you changed means a table of contents to regenerate - by the tool, never by diff --git a/tools/CONTRACT.md b/tools/CONTRACT.md index 7589fb7..e976ee5 100644 --- a/tools/CONTRACT.md +++ b/tools/CONTRACT.md @@ -170,7 +170,7 @@ tools/wikitool --help | `instructions sync [--force]` | Publish every `instructions//SKILL.md` into `.agents/skills/` and `.claude/skills/` as **copies**, and delete published skills whose source is gone. Both targets are gitignored, so a fresh clone runs this once - see `instructions/bootstrap.md`. Re-running is also how a drifted copy is repaired: the source always wins. `--force` is required only to replace a target directory that is not a published skill at all (no `SKILL.md` in it) | | `instructions verify` | Check the instruction layer: flat instructions validate against `types/instruction.schema.yaml`, each `SKILL.md` carries the frontmatter its harness reads, no `SKILL.md` carries a relative markdown link (`sync` copies it to a different depth than the source, so a `SKILL.md` references a target as a repo-root-relative plain path instead - see [instructions/CONTRACT.md](../instructions/CONTRACT.md) § "A skill's outbound reference is a plain path, not a link"), every published copy is byte-identical to its source, no instruction is left that nothing references, and nothing under `instructions/dev/` is referenced from outside it (a `` block is exempt - see [instructions/CONTRACT.md](../instructions/CONTRACT.md)). Missing *every* copy is reported as "run sync", not as drift - that is a clean checkout | | `instructions list [--json]` | List the flat instructions with their descriptions. This is how the layer is discovered; `search` deliberately covers `kb/` only | -| `docs verify` | Check the docs that mirror the code: every CLI command documented in this file's own § Commands table and, separately, in its § Error contracts table (both directions, checked per table, so a row dropped from one is not hidden by the same name surviving in the other, and only a name's presence in a row is checked, never the rest of that row's text), every directory under `kb/` has a `COLLECTION.md` and no directory outside it does, every collection declaring `profile:` and a `required_by_stack:` that agrees with the stack's own list, `kb/CONVENTIONS.md` naming all three tool-owned section headings if it exists at all, every stage contract present, no pre-migration `type: entity` blocks left in the contracts, the `.gitignore` canaries clear in both directions (nothing ignored under `raw/`/`kb/`, `incoming/` ignored, everything ignored under `reports/` and the published skill directories), and no `.md`/`.template` file `dist export` would ship citing an issue number - the tracker exists only in the origin repo, so such a number in a distributed instance is a reference its reader can neither resolve nor recognise as unresolvable (a `` region is exempt: it is already gone from the text the check reads, which is the export plan's, not the working tree's), every reference file `docs toc` covers carrying the current table-of-contents region for its own headings - missing and stale are one check, because the generator is idempotent - and every relative markdown link in one of those same reference files resolving to a file that actually exists (a target's `#anchor` suffix is stripped first; code fences and inline code spans are masked before scanning, so a passage showing link syntax as an example is not mistaken for a real reference). The name is about documentation parity, not about the `docs/` directory - it neither reads nor requires one, the same way `kb/` predates the collection it now checks | +| `docs verify` | Check the docs that mirror the code: every CLI command documented in this file's own § Commands table and, separately, in its § Error contracts table (both directions, checked per table, so a row dropped from one is not hidden by the same name surviving in the other, and only a name's presence in a row is checked, never the rest of that row's text), every directory under `kb/` has a `COLLECTION.md` and no directory outside it does, every collection declaring `profile:` and a `required_by_stack:` that agrees with the stack's own list, `kb/CONVENTIONS.md` naming all three tool-owned section headings if it exists at all, every stage contract present, every file under `types/` declaring `type: types/type-spec.md` validating against `types/type-spec.schema.yaml`, no pre-migration `type: entity` blocks left in the contracts, the `.gitignore` canaries clear in both directions (nothing ignored under `raw/`/`kb/`, `incoming/` ignored, everything ignored under `reports/` and the published skill directories), and no `.md`/`.template` file `dist export` would ship citing an issue number - the tracker exists only in the origin repo, so such a number in a distributed instance is a reference its reader can neither resolve nor recognise as unresolvable (a `` region is exempt: it is already gone from the text the check reads, which is the export plan's, not the working tree's), every reference file `docs toc` covers carrying the current table-of-contents region for its own headings - missing and stale are one check, because the generator is idempotent - and every relative markdown link in one of those same reference files resolving to a file that actually exists (a target's `#anchor` suffix is stripped first; code fences and inline code spans are masked before scanning, so a passage showing link syntax as an example is not mistaken for a real reference). The name is about documentation parity, not about the `docs/` directory - it neither reads nor requires one, the same way `kb/` predates the collection it now checks | | `docs toc [--apply]` | Create, refresh or remove the generated table-of-contents region (`` ... ``, placed after the title and before the first `##`) on every reference file over 100 lines, in the scope Anthropic's skill-authoring guidance names for a file previewed rather than read in full: `AGENTS.md`, every stage contract, `kb/CONVENTIONS.md`, every `kb/*/COLLECTION.md`, every flat `instructions/**.md` file, every `types/*.md` type-spec, and every `docs/` page. Computed from those categories rather than listed, so a file added later is in scope without a code change. `SKILL.md` is the one exception, and the same guidance is why: it places a skill body on the loading level that is read whole when the skill triggers, and aims its own TOC advice at the bundled reference files a skill points *at*. Human docs (`README.md`, `CHANGES.md`, `EVALS.md`, `INSTALL.md`, `tools/README.md`) are out of scope because AGENTS.md § File naming says no agent loads them as instruction. Dry-run by default (prints which files would change); `--apply` writes. `docs verify` checks the result stays current the same way it checks every other generated-from-code copy | ### Telemetry @@ -381,7 +381,7 @@ is atomic, and whether a retry is safe. | `instructions sync` | No skills found under `instructions/`, or a target directory is not a published skill (no `SKILL.md`) and `--force` was not passed | No - one directory copy per skill per target (`.agents/skills/`, `.claude/skills/`); each copy is idempotent, so a re-run converges even after a partial failure | Check whether the flagged target holds anything worth keeping, then re-run with `--force` if not; otherwise fix the named cause and retry | | `instructions verify` | Nothing found under `instructions/` at all, a malformed instruction or `SKILL.md`, a `SKILL.md` carrying a relative markdown link, a published copy that drifted from its source, an instruction nothing references (or, for `manual: true`, one that IS linked from AGENTS.md, CLAUDE.md, or a skill and so risks running implicitly), or something under `instructions/dev/` referenced from outside it and outside a `dist:strip` block | Read-only | Fix the flagged file, then re-run. For a relative link in a `SKILL.md`, rewrite it as a repo-root-relative plain path instead. For drift, re-run `sync` instead of hand-editing the published copy - the source under `instructions/` always wins | | `instructions list` | Never fails - an empty `instructions/` prints "No instructions found." | Read-only | Safe to retry freely | -| `docs verify` | A command, contract, or type-form mismatch was found, a shipped `.md`/`.template` cites an issue number, a reference file's table-of-contents region is missing or stale, or a reference file's relative markdown link does not resolve to an existing file | Read-only | Fix the documentation it names, then re-run. For an issue reference: say what was decided instead of pointing at where, or move the pointer behind a `` block. For a table of contents: run `docs toc --apply` - never hand-write the region. For a dead link: fix the `../` count or the target's name | +| `docs verify` | A command, contract, or type-form mismatch was found, a type-spec's own frontmatter fails its schema, a shipped `.md`/`.template` cites an issue number, a reference file's table-of-contents region is missing or stale, or a reference file's relative markdown link does not resolve to an existing file | Read-only | Fix the documentation it names, then re-run. For a type-spec's own frontmatter: fix the field, or add a matching line to `types/type-spec.schema.yaml` if the field is legitimately new. For an issue reference: say what was decided instead of pointing at where, or move the pointer behind a `` block. For a table of contents: run `docs toc --apply` - never hand-write the region. For a dead link: fix the `../` count or the target's name | | `docs toc` | Never fails on content: a file with no `##` heading, or one at or under the threshold, is simply left without a region | `--apply` rewrites each named file in place, one at a time and idempotently, so a re-run after an interruption converges rather than doubling a region; the dry-run form is read-only | Nothing to fix - re-run with `--apply` to write what the dry run listed. If `docs verify` still reports a stale region afterwards, the file's `##` headings changed in between; run it again | ### Telemetry diff --git a/tools/chemenu/commands/docs_verify.py b/tools/chemenu/commands/docs_verify.py index f745034..ecfc1bc 100644 --- a/tools/chemenu/commands/docs_verify.py +++ b/tools/chemenu/commands/docs_verify.py @@ -44,6 +44,15 @@ assume - is `instructions verify`'s job, not this one, since that module already owns the Skill/Instruction split (`skill_dirs()` vs `instruction_files()`). +An eighth checks the type layer against its own schema: every file under +`types/` declaring `type: types/type-spec.md` must validate against +`types/type-spec.schema.yaml`. Before this check existed the schema had +already drifted behind two fields real type-specs carry (`root:`, +`capture_fields:`) while `additionalProperties: false` sat there describing a +contract nothing enforced - the exact "checked or absent" failure this file's +opening paragraph names, just one level up, for the schema that describes the +type layer instead of a copy the type layer's code produces (Gitea #105). + Everything here is a hard oracle: a set comparison or a regex, no judgment. Content quality of the contracts themselves stays with the LLM. """ @@ -422,6 +431,37 @@ def check_stack_required_types() -> list[str]: return issues +def check_type_spec_frontmatter() -> list[str]: + """Every type-spec's own frontmatter must validate against + `types/type-spec.schema.yaml` - the schema that describes the type layer + gets the same enforcement any other type's schema gets (Gitea #105). + + Before this check nothing ever called `validate_frontmatter` against a + type-spec's own frontmatter, so the schema had quietly drifted behind two + fields real type-specs actually carry (`root:`, `capture_fields:`) + without anything failing - `additionalProperties: false` described a + contract that bound nothing. `resolver.list_type_specs()` already reads + every file's frontmatter once for `wikitool types list`; reusing it here + means this check costs no second parse pass. + """ + from chemenu.type_resolver import resolver + + issues: list[str] = [] + for type_path, frontmatter in resolver.list_type_specs(): + try: + resolver.validate_frontmatter( + frontmatter, "types/type-spec.md", source_file=config.ROOT / type_path + ) + except ValueError as exc: + # `validate_frontmatter`'s own message names the type path it + # validated *against* (always `types/type-spec.md` here, since + # every type-spec is validated against the same schema) rather + # than the specific file that failed - prefix that file's own + # path so two failures in one run stay distinguishable. + issues.append(f"{type_path}: {exc}") + return issues + + def check_legacy_type_blocks() -> list[str]: issues = [] guarded = [ @@ -857,11 +897,12 @@ def check_breaking_change_for_boundary() -> list[str]: @app.command("verify") def verify(): - """Check the CLI/README command tables, contract presence, type-form drift, ignore rules, version/changelog agreement, issue references, and link targets in shipped documents.""" + """Check the CLI/README command tables, contract presence, type-form drift, every type-spec's frontmatter against its own schema, ignore rules, version/changelog agreement, issue references, and link targets in shipped documents.""" issues = ( check_cli_readme() + check_readmes_have_no_command_table() + check_collection_contracts() + + check_type_spec_frontmatter() + check_legacy_type_blocks() + check_ignored_content() + check_version_changelog() @@ -875,10 +916,13 @@ def verify(): if issues: fail("Documentation issues found:\n" + "\n".join(f"- {i}" for i in issues)) + from chemenu.type_resolver import resolver + success( f"Docs verified: {len(registered_commands())} command(s) documented, " f"{len(kb_collections.iter_kb_collections())} collection(s) and " f"{len(STAGE_CONTRACTS)} stage contract(s) present, no legacy type blocks, " + f"{len(resolver.list_type_specs())} type-spec(s) validating against their own schema, " f"{len(IGNORE_CANARIES)} ignore canaries clear, " f"no issue references in {len(shipped_prose())} shipped document(s), " f"tables of contents current and every link resolving on " diff --git a/tools/chemenu/tests/test_docs_verify.py b/tools/chemenu/tests/test_docs_verify.py index 6c58762..0fd9c56 100644 --- a/tools/chemenu/tests/test_docs_verify.py +++ b/tools/chemenu/tests/test_docs_verify.py @@ -1,8 +1,9 @@ import pytest import typer -from chemenu import config +from chemenu import config, type_resolver from chemenu.commands import dist_cmd, docs_verify +from chemenu.type_resolver import TypeResolver def test_every_registered_command_is_documented(): @@ -289,6 +290,51 @@ def test_an_absent_listed_doc_is_skipped_not_reported(tmp_path, monkeypatch): assert docs_verify.check_readmes_have_no_command_table() == [] +def test_this_repos_type_specs_validate_against_their_own_schema(): + """Regression guard for Gitea #105: types/type-spec.schema.yaml declared + `additionalProperties: false` while real type-specs already carried + `root:` and `capture_fields:`, and nothing ever validated a type-spec's + own frontmatter against it - so the mismatch shipped silently.""" + assert docs_verify.check_type_spec_frontmatter() == [] + + +def test_an_unknown_type_spec_field_is_reported(tmp_path, monkeypatch): + """Once the schema is enforced, a type-spec frontmatter field its own + schema does not know about must fail loudly rather than validating + silently - the other direction of the #105 regression guard above.""" + types_dir = tmp_path / "types" + types_dir.mkdir() + (types_dir / "type-spec.md").write_text( + "---\n" + "type: types/type-spec.md\n" + "name: type-spec\n" + "description: Authoring and validation contract for type specs\n" + "schema: types/type-spec.schema.yaml\n" + "---\n", + encoding="utf-8", + ) + (types_dir / "type-spec.schema.yaml").write_text( + (config.ROOT / "types" / "type-spec.schema.yaml").read_text(encoding="utf-8"), + encoding="utf-8", + ) + (types_dir / "widget.md").write_text( + "---\n" + "type: types/type-spec.md\n" + "name: widget\n" + "description: A type-spec with a field its own schema does not know\n" + "schema: null\n" + "not_a_real_field: true\n" + "---\n", + encoding="utf-8", + ) + monkeypatch.setattr(config, "ROOT", tmp_path) + monkeypatch.setattr(config, "TYPES_DIR", types_dir) + monkeypatch.setattr(type_resolver, "resolver", TypeResolver(repo_root=tmp_path)) + + issues = docs_verify.check_type_spec_frontmatter() + assert any("widget.md" in issue and "not_a_real_field" in issue for issue in issues) + + def test_legacy_type_blocks_are_absent(): assert docs_verify.check_legacy_type_blocks() == [] diff --git a/types/type-spec.md b/types/type-spec.md index a8a347d..d1b572e 100644 --- a/types/type-spec.md +++ b/types/type-spec.md @@ -44,6 +44,8 @@ Two systems check conformance to a type: Do not restate a schema rule in body text. The validator already enforces it, so the restatement adds no protection — it only spends the reviewer's judgment re-confirming something already guaranteed, instead of on the properties only a reviewer can check. +A type-spec's own frontmatter is checked the same way a page's is: `docs verify` validates every file under `types/` declaring `type: types/type-spec.md` against `types/type-spec.schema.yaml`. A frontmatter field used here needs a matching line in that schema, or `docs verify` fails. + ## Writing Shape - Write the body as a **content contract**: checkable properties of a conforming instance, in a form the type-conformance gate can apply directly. Prefer "what a conforming instance contains or claims" over "how to produce one" — production process is invisible to the reviewer, so it belongs in a skill or instruction, linked rather than embedded. diff --git a/types/type-spec.schema.yaml b/types/type-spec.schema.yaml index 2a5c5cd..12db7c5 100644 --- a/types/type-spec.schema.yaml +++ b/types/type-spec.schema.yaml @@ -69,6 +69,33 @@ properties: items: type: string pattern: "^[a-z][a-z0-9_]*$" + capture_fields: + type: array + description: >- + Frontmatter fields on instances of this type that are fixed at capture + time and never correctable afterwards except by re-capturing the source + (e.g. `fidelity`, `authority` for a source page). `raw accept`, + `new source` and `touch --set` read this instead of a hardcoded field + list. Omit for types with no capture-time fields. + items: + type: string + pattern: "^[a-z][a-z0-9_]*$" + root: + type: string + description: >- + Which root a `base_dir:` resolves against - `kb` (the default, so + omitting this field is equivalent to `kb`) for a page this instance + writes, or `repo` for a type whose artifacts are legitimately not + knowledge (e.g. `instruction`). + enum: [kb, repo] + guidance: + type: string + description: >- + Repo-relative path to a `types/type-guidance.md`-shaped document + holding this type's stack-owned authoring prose. Meaningful only on a + `root: kb` type-spec; omit for a type an instance writes entirely for + itself. + pattern: "^types/.*\\.md$" required: - type - name