fix: types/type-spec.schema.yaml enforced against real type-spec frontmatter, doc-pull-through.md docs/-page count corrected (closes #105)
Files changed: - CHANGES.md - VERSION - instructions/dev/doc-pull-through.md - tools/CONTRACT.md - tools/chemenu/commands/docs_verify.py - tools/chemenu/tests/test_docs_verify.py - types/type-spec.md - types/type-spec.schema.yaml
This commit is contained in:
+2
-2
@@ -170,7 +170,7 @@ tools/wikitool <command> --help
|
||||
| `instructions sync [--force]` | Publish every `instructions/<name>/SKILL.md` into `.agents/skills/` and `.claude/skills/` as **copies**, and delete published skills whose source is gone. Both targets are gitignored, so a fresh clone runs this once - see `instructions/bootstrap.md`. Re-running is also how a drifted copy is repaired: the source always wins. `--force` is required only to replace a target directory that is not a published skill at all (no `SKILL.md` in it) |
|
||||
| `instructions verify` | Check the instruction layer: flat instructions validate against `types/instruction.schema.yaml`, each `SKILL.md` carries the frontmatter its harness reads, no `SKILL.md` carries a relative markdown link (`sync` copies it to a different depth than the source, so a `SKILL.md` references a target as a repo-root-relative plain path instead - see [instructions/CONTRACT.md](../instructions/CONTRACT.md) § "A skill's outbound reference is a plain path, not a link"), every published copy is byte-identical to its source, no instruction is left that nothing references, and nothing under `instructions/dev/` is referenced from outside it (a `<!-- dist:strip-start/end -->` block is exempt - see [instructions/CONTRACT.md](../instructions/CONTRACT.md)). Missing *every* copy is reported as "run sync", not as drift - that is a clean checkout |
|
||||
| `instructions list [--json]` | List the flat instructions with their descriptions. This is how the layer is discovered; `search` deliberately covers `kb/` only |
|
||||
| `docs verify` | Check the docs that mirror the code: every CLI command documented in this file's own § Commands table and, separately, in its § Error contracts table (both directions, checked per table, so a row dropped from one is not hidden by the same name surviving in the other, and only a name's presence in a row is checked, never the rest of that row's text), every directory under `kb/` has a `COLLECTION.md` and no directory outside it does, every collection declaring `profile:` and a `required_by_stack:` that agrees with the stack's own list, `kb/CONVENTIONS.md` naming all three tool-owned section headings if it exists at all, every stage contract present, no pre-migration `type: entity` blocks left in the contracts, the `.gitignore` canaries clear in both directions (nothing ignored under `raw/`/`kb/`, `incoming/` ignored, everything ignored under `reports/` and the published skill directories), and no `.md`/`.template` file `dist export` would ship citing an issue number - the tracker exists only in the origin repo, so such a number in a distributed instance is a reference its reader can neither resolve nor recognise as unresolvable (a `<!-- dist:strip-start/end -->` region is exempt: it is already gone from the text the check reads, which is the export plan's, not the working tree's), every reference file `docs toc` covers carrying the current table-of-contents region for its own headings - missing and stale are one check, because the generator is idempotent - and every relative markdown link in one of those same reference files resolving to a file that actually exists (a target's `#anchor` suffix is stripped first; code fences and inline code spans are masked before scanning, so a passage showing link syntax as an example is not mistaken for a real reference). The name is about documentation parity, not about the `docs/` directory - it neither reads nor requires one, the same way `kb/` predates the collection it now checks |
|
||||
| `docs verify` | Check the docs that mirror the code: every CLI command documented in this file's own § Commands table and, separately, in its § Error contracts table (both directions, checked per table, so a row dropped from one is not hidden by the same name surviving in the other, and only a name's presence in a row is checked, never the rest of that row's text), every directory under `kb/` has a `COLLECTION.md` and no directory outside it does, every collection declaring `profile:` and a `required_by_stack:` that agrees with the stack's own list, `kb/CONVENTIONS.md` naming all three tool-owned section headings if it exists at all, every stage contract present, every file under `types/` declaring `type: types/type-spec.md` validating against `types/type-spec.schema.yaml`, no pre-migration `type: entity` blocks left in the contracts, the `.gitignore` canaries clear in both directions (nothing ignored under `raw/`/`kb/`, `incoming/` ignored, everything ignored under `reports/` and the published skill directories), and no `.md`/`.template` file `dist export` would ship citing an issue number - the tracker exists only in the origin repo, so such a number in a distributed instance is a reference its reader can neither resolve nor recognise as unresolvable (a `<!-- dist:strip-start/end -->` region is exempt: it is already gone from the text the check reads, which is the export plan's, not the working tree's), every reference file `docs toc` covers carrying the current table-of-contents region for its own headings - missing and stale are one check, because the generator is idempotent - and every relative markdown link in one of those same reference files resolving to a file that actually exists (a target's `#anchor` suffix is stripped first; code fences and inline code spans are masked before scanning, so a passage showing link syntax as an example is not mistaken for a real reference). The name is about documentation parity, not about the `docs/` directory - it neither reads nor requires one, the same way `kb/` predates the collection it now checks |
|
||||
| `docs toc [--apply]` | Create, refresh or remove the generated table-of-contents region (`<!-- wikitool:toc -->` ... `<!-- /wikitool:toc -->`, placed after the title and before the first `##`) on every reference file over 100 lines, in the scope Anthropic's skill-authoring guidance names for a file previewed rather than read in full: `AGENTS.md`, every stage contract, `kb/CONVENTIONS.md`, every `kb/*/COLLECTION.md`, every flat `instructions/**.md` file, every `types/*.md` type-spec, and every `docs/` page. Computed from those categories rather than listed, so a file added later is in scope without a code change. `SKILL.md` is the one exception, and the same guidance is why: it places a skill body on the loading level that is read whole when the skill triggers, and aims its own TOC advice at the bundled reference files a skill points *at*. Human docs (`README.md`, `CHANGES.md`, `EVALS.md`, `INSTALL.md`, `tools/README.md`) are out of scope because AGENTS.md § File naming says no agent loads them as instruction. Dry-run by default (prints which files would change); `--apply` writes. `docs verify` checks the result stays current the same way it checks every other generated-from-code copy |
|
||||
|
||||
### Telemetry
|
||||
@@ -381,7 +381,7 @@ is atomic, and whether a retry is safe.
|
||||
| `instructions sync` | No skills found under `instructions/`, or a target directory is not a published skill (no `SKILL.md`) and `--force` was not passed | No - one directory copy per skill per target (`.agents/skills/`, `.claude/skills/`); each copy is idempotent, so a re-run converges even after a partial failure | Check whether the flagged target holds anything worth keeping, then re-run with `--force` if not; otherwise fix the named cause and retry |
|
||||
| `instructions verify` | Nothing found under `instructions/` at all, a malformed instruction or `SKILL.md`, a `SKILL.md` carrying a relative markdown link, a published copy that drifted from its source, an instruction nothing references (or, for `manual: true`, one that IS linked from AGENTS.md, CLAUDE.md, or a skill and so risks running implicitly), or something under `instructions/dev/` referenced from outside it and outside a `dist:strip` block | Read-only | Fix the flagged file, then re-run. For a relative link in a `SKILL.md`, rewrite it as a repo-root-relative plain path instead. For drift, re-run `sync` instead of hand-editing the published copy - the source under `instructions/` always wins |
|
||||
| `instructions list` | Never fails - an empty `instructions/` prints "No instructions found." | Read-only | Safe to retry freely |
|
||||
| `docs verify` | A command, contract, or type-form mismatch was found, a shipped `.md`/`.template` cites an issue number, a reference file's table-of-contents region is missing or stale, or a reference file's relative markdown link does not resolve to an existing file | Read-only | Fix the documentation it names, then re-run. For an issue reference: say what was decided instead of pointing at where, or move the pointer behind a `<!-- dist:strip-start/end -->` block. For a table of contents: run `docs toc --apply` - never hand-write the region. For a dead link: fix the `../` count or the target's name |
|
||||
| `docs verify` | A command, contract, or type-form mismatch was found, a type-spec's own frontmatter fails its schema, a shipped `.md`/`.template` cites an issue number, a reference file's table-of-contents region is missing or stale, or a reference file's relative markdown link does not resolve to an existing file | Read-only | Fix the documentation it names, then re-run. For a type-spec's own frontmatter: fix the field, or add a matching line to `types/type-spec.schema.yaml` if the field is legitimately new. For an issue reference: say what was decided instead of pointing at where, or move the pointer behind a `<!-- dist:strip-start/end -->` block. For a table of contents: run `docs toc --apply` - never hand-write the region. For a dead link: fix the `../` count or the target's name |
|
||||
| `docs toc` | Never fails on content: a file with no `##` heading, or one at or under the threshold, is simply left without a region | `--apply` rewrites each named file in place, one at a time and idempotently, so a re-run after an interruption converges rather than doubling a region; the dry-run form is read-only | Nothing to fix - re-run with `--apply` to write what the dry run listed. If `docs verify` still reports a stale region afterwards, the file's `##` headings changed in between; run it again |
|
||||
|
||||
### Telemetry
|
||||
|
||||
@@ -44,6 +44,15 @@ assume - is `instructions verify`'s job, not this one, since that module
|
||||
already owns the Skill/Instruction split (`skill_dirs()` vs
|
||||
`instruction_files()`).
|
||||
|
||||
An eighth checks the type layer against its own schema: every file under
|
||||
`types/` declaring `type: types/type-spec.md` must validate against
|
||||
`types/type-spec.schema.yaml`. Before this check existed the schema had
|
||||
already drifted behind two fields real type-specs carry (`root:`,
|
||||
`capture_fields:`) while `additionalProperties: false` sat there describing a
|
||||
contract nothing enforced - the exact "checked or absent" failure this file's
|
||||
opening paragraph names, just one level up, for the schema that describes the
|
||||
type layer instead of a copy the type layer's code produces (Gitea #105).
|
||||
|
||||
Everything here is a hard oracle: a set comparison or a regex, no judgment.
|
||||
Content quality of the contracts themselves stays with the LLM.
|
||||
"""
|
||||
@@ -422,6 +431,37 @@ def check_stack_required_types() -> list[str]:
|
||||
return issues
|
||||
|
||||
|
||||
def check_type_spec_frontmatter() -> list[str]:
|
||||
"""Every type-spec's own frontmatter must validate against
|
||||
`types/type-spec.schema.yaml` - the schema that describes the type layer
|
||||
gets the same enforcement any other type's schema gets (Gitea #105).
|
||||
|
||||
Before this check nothing ever called `validate_frontmatter` against a
|
||||
type-spec's own frontmatter, so the schema had quietly drifted behind two
|
||||
fields real type-specs actually carry (`root:`, `capture_fields:`)
|
||||
without anything failing - `additionalProperties: false` described a
|
||||
contract that bound nothing. `resolver.list_type_specs()` already reads
|
||||
every file's frontmatter once for `wikitool types list`; reusing it here
|
||||
means this check costs no second parse pass.
|
||||
"""
|
||||
from chemenu.type_resolver import resolver
|
||||
|
||||
issues: list[str] = []
|
||||
for type_path, frontmatter in resolver.list_type_specs():
|
||||
try:
|
||||
resolver.validate_frontmatter(
|
||||
frontmatter, "types/type-spec.md", source_file=config.ROOT / type_path
|
||||
)
|
||||
except ValueError as exc:
|
||||
# `validate_frontmatter`'s own message names the type path it
|
||||
# validated *against* (always `types/type-spec.md` here, since
|
||||
# every type-spec is validated against the same schema) rather
|
||||
# than the specific file that failed - prefix that file's own
|
||||
# path so two failures in one run stay distinguishable.
|
||||
issues.append(f"{type_path}: {exc}")
|
||||
return issues
|
||||
|
||||
|
||||
def check_legacy_type_blocks() -> list[str]:
|
||||
issues = []
|
||||
guarded = [
|
||||
@@ -857,11 +897,12 @@ def check_breaking_change_for_boundary() -> list[str]:
|
||||
|
||||
@app.command("verify")
|
||||
def verify():
|
||||
"""Check the CLI/README command tables, contract presence, type-form drift, ignore rules, version/changelog agreement, issue references, and link targets in shipped documents."""
|
||||
"""Check the CLI/README command tables, contract presence, type-form drift, every type-spec's frontmatter against its own schema, ignore rules, version/changelog agreement, issue references, and link targets in shipped documents."""
|
||||
issues = (
|
||||
check_cli_readme()
|
||||
+ check_readmes_have_no_command_table()
|
||||
+ check_collection_contracts()
|
||||
+ check_type_spec_frontmatter()
|
||||
+ check_legacy_type_blocks()
|
||||
+ check_ignored_content()
|
||||
+ check_version_changelog()
|
||||
@@ -875,10 +916,13 @@ def verify():
|
||||
if issues:
|
||||
fail("Documentation issues found:\n" + "\n".join(f"- {i}" for i in issues))
|
||||
|
||||
from chemenu.type_resolver import resolver
|
||||
|
||||
success(
|
||||
f"Docs verified: {len(registered_commands())} command(s) documented, "
|
||||
f"{len(kb_collections.iter_kb_collections())} collection(s) and "
|
||||
f"{len(STAGE_CONTRACTS)} stage contract(s) present, no legacy type blocks, "
|
||||
f"{len(resolver.list_type_specs())} type-spec(s) validating against their own schema, "
|
||||
f"{len(IGNORE_CANARIES)} ignore canaries clear, "
|
||||
f"no issue references in {len(shipped_prose())} shipped document(s), "
|
||||
f"tables of contents current and every link resolving on "
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
import pytest
|
||||
import typer
|
||||
|
||||
from chemenu import config
|
||||
from chemenu import config, type_resolver
|
||||
from chemenu.commands import dist_cmd, docs_verify
|
||||
from chemenu.type_resolver import TypeResolver
|
||||
|
||||
|
||||
def test_every_registered_command_is_documented():
|
||||
@@ -289,6 +290,51 @@ def test_an_absent_listed_doc_is_skipped_not_reported(tmp_path, monkeypatch):
|
||||
assert docs_verify.check_readmes_have_no_command_table() == []
|
||||
|
||||
|
||||
def test_this_repos_type_specs_validate_against_their_own_schema():
|
||||
"""Regression guard for Gitea #105: types/type-spec.schema.yaml declared
|
||||
`additionalProperties: false` while real type-specs already carried
|
||||
`root:` and `capture_fields:`, and nothing ever validated a type-spec's
|
||||
own frontmatter against it - so the mismatch shipped silently."""
|
||||
assert docs_verify.check_type_spec_frontmatter() == []
|
||||
|
||||
|
||||
def test_an_unknown_type_spec_field_is_reported(tmp_path, monkeypatch):
|
||||
"""Once the schema is enforced, a type-spec frontmatter field its own
|
||||
schema does not know about must fail loudly rather than validating
|
||||
silently - the other direction of the #105 regression guard above."""
|
||||
types_dir = tmp_path / "types"
|
||||
types_dir.mkdir()
|
||||
(types_dir / "type-spec.md").write_text(
|
||||
"---\n"
|
||||
"type: types/type-spec.md\n"
|
||||
"name: type-spec\n"
|
||||
"description: Authoring and validation contract for type specs\n"
|
||||
"schema: types/type-spec.schema.yaml\n"
|
||||
"---\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
(types_dir / "type-spec.schema.yaml").write_text(
|
||||
(config.ROOT / "types" / "type-spec.schema.yaml").read_text(encoding="utf-8"),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(types_dir / "widget.md").write_text(
|
||||
"---\n"
|
||||
"type: types/type-spec.md\n"
|
||||
"name: widget\n"
|
||||
"description: A type-spec with a field its own schema does not know\n"
|
||||
"schema: null\n"
|
||||
"not_a_real_field: true\n"
|
||||
"---\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
monkeypatch.setattr(config, "ROOT", tmp_path)
|
||||
monkeypatch.setattr(config, "TYPES_DIR", types_dir)
|
||||
monkeypatch.setattr(type_resolver, "resolver", TypeResolver(repo_root=tmp_path))
|
||||
|
||||
issues = docs_verify.check_type_spec_frontmatter()
|
||||
assert any("widget.md" in issue and "not_a_real_field" in issue for issue in issues)
|
||||
|
||||
|
||||
def test_legacy_type_blocks_are_absent():
|
||||
assert docs_verify.check_legacy_type_blocks() == []
|
||||
|
||||
|
||||
Reference in New Issue
Block a user