diff --git a/CHANGES.md b/CHANGES.md index 58faae6..6ae7775 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -35,13 +35,14 @@ dev-checkout concern - readable here, never shipped as something to parse. --- -## 4.8.0-beta.1 - 2026-09-04 - page move: eine kb-Seite folgt ihrem Subtype ins Verzeichnis, das ihr Type-Spec berechnet +## 4.8.0-beta.2 - 2026-09-04 - kb/CONTRACT.md: Tiefe 1 als Grenze - Katalog liest nur eine Area-Ebene **Author:** Torben Nehmer - status/incoming: menschliche Stubs werden ausgearbeitet, nie so umgesetzt - page move: eine kb-Seite folgt ihrem Subtype ins Verzeichnis, das ihr Type-Spec berechnet +- kb/CONTRACT.md: Tiefe 1 als Grenze - Katalog liest nur eine Area-Ebene Das Label `status/incoming` gibt es seit heute in Gitea: der Mensch legt einen @@ -151,6 +152,71 @@ move`), `tools/CONTRACT.md`, `instructions/page-lifecycle.md`, verschobene Seite unverändert (Identität ist der Titel, nicht der Ort), vorwärts reines Überkopieren. +**Katalogtiefe** (#57): `index_build.group_pages` las bislang genau zwei +Pfadsegmente unter `kb/` (`parts[0]` als Collection, `parts[1]` als Area) und +faltete alles darunter still in die Level-1-Area. Real betroffen waren die +drei Seiten aus #56s Befund, `kb/entities/projects/{kfchou,vanillaflava, +yugasun}/*.md` — im generierten Katalog nicht als eigener Ort sichtbar, +sondern als läge jede direkt in `entities/projects/`. `kb/CONTRACT.md` § +Collections beschrieb bis heute nur eine Ebene, ohne eine zweite +auszuschließen; der Baum hatte trotzdem eine, handplatziert, ohne +unterstützten Weg dorthin. + +Entscheidung war (b) aus dem Issue: die drei Verzeichnisse auflösen statt den +Katalog rekursiv zu machen. Die Gruppierungsachse dahinter — Owner +(`kfchou`/`vanillaflava`/`yugasun`) — kommt aus keinem Frontmatter-Feld und +aus keinem Type-Spec, sondern aus einer Ad-hoc-Entscheidung beim Anlegen; sie +verdient keine zweite Verzeichnisebene. Tiefe 1 ist jetzt geschriebene Regel +in `kb/CONTRACT.md` § Collections, mit dieser Begründung. + +Vier Stücke setzen das um: + +- **`kb_scan.find_nested_pages`** liefert `(title, page, depth)` für jede + Seite mehr als ein Verzeichnis unterhalb ihrer Collection — reine + Pfadtiefe, unabhängig davon, ob der Typ auflöst, damit auch eine Seite mit + kaputtem `type:` nicht durchrutscht. +- **`lint`** bekommt den Befund **Nested Pages**, und anders als + `misplaced_pages` **hart**: eine fehlplatzierte Seite katalogisiert noch + korrekt von der falschen Stelle aus, eine verschachtelte macht den + *generierten* Katalog selbst falsch, und es gibt keine Version, ab der das + toleriert würde. +- **`index rebuild`** lehnt eine verschachtelte Seite nicht ab, sondern warnt + (Entscheidung aus der Session: melden statt verweigern, damit ein + Fremdinstanz-Upgrade mit handverschachtelten Seiten nicht hart bricht) — + `group_pages` faltet weiterhin wie zuvor, die Warnung ist die neue + Sichtbarkeit, nicht eine Verhaltensänderung der Faltung selbst. +- **`TypeResolver.get_layout`** validiert `layout: {dir: ...}` jetzt auf + einen einzelnen Pfadabschnitt (kein `/`, kein `\`, kein `.`/`..`, nicht + leer) und schlägt fehl statt eine zweite Ebene über den einzigen + unterstützten Weg — ein Type-Spec — entstehen zu lassen. + +Ein fünftes Stück, das das Issue selbst nicht explizit forderte, aber die +`move`-Mechanik aus #56 sonst mit toten Verzeichnissen zurückgelassen hätte: +**`move`** (`--page` wie `--reconcile`) entfernt jetzt ein Verzeichnis, das es +durch den Wegzug seiner letzten Seite geleert hat — symmetrisch zum +`mkdir(parents=True)` auf der Zielseite. Ohne das hätten `kfchou/`, +`vanillaflava/`, `yugasun/` den eigenen Fix überlebt, leer und für git +unsichtbar, aber für einen verzeichnisbasierten Test sichtbar. + +Auf dieser Instanz angewendet: `wikitool move --reconcile` hat die drei +Seiten nach `kb/entities/projects/` gezogen und die drei leeren +Owner-Verzeichnisse mitentfernt. `migrate verify --from HEAD` bestätigt +`compared == 182, added == 0, removed == 0`, alle drei als `moved` markiert. +Titelkollision trat wie im Issue erwartet keine auf. + +Geändert: `tools/chemenu/kb_scan.py` (`find_nested_pages`), +`tools/chemenu/lint_core.py` (`nested_pages`, `HARD_ERROR_KEYS`), +`tools/chemenu/commands/index_build.py` (Rebuild-Warnung), +`tools/chemenu/type_resolver.py` (`get_layout`-Validierung), +`tools/chemenu/commands/page_ops.py` (`_rmdir_if_emptied`), `tools/CONTRACT.md`, +`kb/CONTRACT.md` § Collections, plus die drei realen Seiten unter +`kb/entities/projects/`. PATCH: reine Codeänderung ohne Schnittstellenwechsel, +gefaltet in den offenen `4.8.0`-Kandidaten (max-wins gegen die MINOR-Bewegung +aus #56); die drei bewegten Seiten sind Korpus dieser Instanz, kein +ausgelieferter Inhalt. + +Schließt #57. + --- ## 4.7.4 - 2026-09-04 - bootstrap.md nennt den session-id-WARN nach frischem Bootstrap explizit als erwartet diff --git a/VERSION b/VERSION index 9ccf6df..d841651 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -4.8.0-beta.1 +4.8.0-beta.2 diff --git a/kb/CONTRACT.md b/kb/CONTRACT.md index d5cdf58..7b62aac 100644 --- a/kb/CONTRACT.md +++ b/kb/CONTRACT.md @@ -46,6 +46,16 @@ resolves against it by name. - A subdirectory *inside* a collection is an **area**. It inherits the enclosing contract and must not carry a `COLLECTION.md` of its own - `kb/entities/systems/` is an area of `kb/entities/`. +- **An area is as deep as a page goes.** `kb//.md` and + `kb///.md` are the two depths a page may sit at; nothing goes a level + deeper. A further subdirectory is not a second-level area - it is invisible to the generated + catalog, which reads exactly two path segments below `kb/` and folds anything past them into + the area's own table silently, with no location of its own (Gitea #57). That is why + `wikitool lint`'s `nested_pages` finding is a hard error rather than an advisory one like + `misplaced_pages`: a misplaced page still catalogs correctly from the wrong place, a nested + one makes the catalog itself wrong. A grouping axis that does not come from a type-spec's + `layout:` - project owner was the case that surfaced this - does not earn a second directory + level; it goes into frontmatter instead. - A `COLLECTION.md` nested inside another collection is invalid. - `COLLECTION.md` appears **nowhere outside `kb/`**. `raw/`, `types/`, `tools/`, `reports/` and `instructions/` are not collections and carry a `CONTRACT.md` or a root type-spec diff --git a/kb/entities/projects/yugasun/llm-wiki-skills.md b/kb/entities/projects/llm-wiki-skills.md similarity index 100% rename from kb/entities/projects/yugasun/llm-wiki-skills.md rename to kb/entities/projects/llm-wiki-skills.md diff --git a/kb/entities/projects/vanillaflava/wiki-skills-vanillaflava.md b/kb/entities/projects/wiki-skills-vanillaflava.md similarity index 100% rename from kb/entities/projects/vanillaflava/wiki-skills-vanillaflava.md rename to kb/entities/projects/wiki-skills-vanillaflava.md diff --git a/kb/entities/projects/kfchou/wiki-skills.md b/kb/entities/projects/wiki-skills.md similarity index 100% rename from kb/entities/projects/kfchou/wiki-skills.md rename to kb/entities/projects/wiki-skills.md diff --git a/kb/log.md b/kb/log.md index 6b5b4bd..1e5c21d 100644 --- a/kb/log.md +++ b/kb/log.md @@ -149,3 +149,9 @@ Neue Source-Seite fuer Gitea-Issues #62/#63: Einfuehrung des dritten status/-Fla Drittes status/-Flag status/incoming ergaenzt (Gitea #63): status/-Tabelle auf drei Zeilen, Labelzahl 16 auf 17 korrigiert, neuer Kernpunkt zur Aussetzung der Vier-Achsen-Pflicht, Historie- und Beispiele-Abschnitt sowie Versions-/PATCH-Hinweis nachgezogen. Quelle: Source - Gitea Issues 62-63 - status-incoming Label Introduction 2026-09-04. --- + +## [2026-09-04] move | Der Katalog bildet nur Tiefe 1 ab, aber es liegen Seiten auf Tiefe 2 (#57) + +wikitool move --reconcile hat die drei nach kb/entities/projects/{kfchou,vanillaflava,yugasun}/*.md verschachtelten Seiten (wiki-skills, wiki-skills-vanillaflava, llm-wiki-skills) nach kb/entities/projects/ hochgezogen und die drei geleerten Owner-Verzeichnisse entfernt. index rebuild und sources rebuild-index liefen danach; lint meldet weder misplaced_pages noch nested_pages noch duplicate_titles; migrate verify --from HEAD bestaetigt compared=182, added=0, removed=0, alle drei als moved. + +--- diff --git a/tools/CONTRACT.md b/tools/CONTRACT.md index f47c670..2e7a98b 100644 --- a/tools/CONTRACT.md +++ b/tools/CONTRACT.md @@ -40,7 +40,7 @@ tools/wikitool --help | `touch --page "" [--summary "..."] [--provenance <v>] [--confidence-base <n>] [--date YYYY-MM-DD] [--set field=value ...] [--add field=value ...] [--remove field=value ...] [--no-date] [--dry-run]` | Update a page's own frontmatter: bump `modified:` and optionally rewrite any field its type declares. `--summary`/`--provenance`/`--confidence-base` are shorthands; `--set` reaches every other field and **replaces** its value, while `--add`/`--remove` change single elements of an array field (removing an absent element succeeds and says so). Repeating `--set` for one array field appends *within the call*, and `\,` is a literal comma - same rules as `new --set`. Refused with the command that owns them instead: `type:` (page-lifecycle), `confidence:` (derived - set `--confidence-base`), and the page-ref arrays `related:`/`sources:`/`entities:`/`concepts:` (`xref`). Everything else the schema declares is settable, and an unknown field lists what the page actually has. Schema-validates the fields it writes, and `raw_files:` entries must exist on disk. A source declares `date:` instead of `modified:`, and that is the *publication* date of the raw material - it is never bumped to today, and changes only when `--date` names a value explicitly. | | `rename --from "<Old>" --to "<New>" [--dry-run]` | Rename a page and repoint every reference to it: body `[[wikilinks]]` (aliases and anchors preserved), a `[^cite-id]` whose id was derived from the old title (refreshed to match the new one, both in its Footnotes definition and every reference to it), the page's own H1, and every page-ref frontmatter array declared by the type's `page_ref_fields:`. If `--from` is *not* a page but is referenced, it instead repoints those references onto the existing `--to` page and moves nothing - the fix for a reference spelled `act_runner` when the page is `Act Runner` | | `rm --page "<Title>" [--yes] [--dry-run]` | Delete a page and mechanically de-link it. Refuses without `--yes` while other pages still reference it. Strips ref-array entries and bare `- [[Title]]` / `- **label:** [[Title]]` bullets; leaves prose and inline citations in place and reports them | -| `move --page "<Title>"` \| `move --reconcile` `[--dry-run]` | Move a page to the directory its type-spec computes for its current frontmatter (`base_dir` + `layout` - the same rule `new` places a page by, via `TypeResolver.compute_target_dir`), never a hand-chosen destination - there is no `--to <dir>`. `--reconcile` applies it corpus-wide: every misplaced page moves in one call, and a second run reports nothing left to do (`lint`'s `Misplaced Pages` finding is the advisory that this fixes - see `lint`). Neither mode touches a body or a frontmatter field, and the page's title (its only identity in the wiki) never changes - only the file moves. A destination already occupied (a pre-existing duplicate-stem collision) is refused rather than silently skipped | +| `move --page "<Title>"` \| `move --reconcile` `[--dry-run]` | Move a page to the directory its type-spec computes for its current frontmatter (`base_dir` + `layout` - the same rule `new` places a page by, via `TypeResolver.compute_target_dir`), never a hand-chosen destination - there is no `--to <dir>`. `--reconcile` applies it corpus-wide: every misplaced page moves in one call, and a second run reports nothing left to do (`lint`'s `Misplaced Pages` finding is the advisory that this fixes, and its `Nested Pages` finding the hard one - see `lint`). Neither mode touches a body or a frontmatter field, and the page's title (its only identity in the wiki) never changes - only the file moves. A directory a move empties is removed along with it, so a page that was nested below its area (#57) leaves no leftover directory behind. A destination already occupied (a pre-existing duplicate-stem collision) is refused rather than silently skipped | | `xref add --a "<A>" --b "<B>" --rel <label>` | Declare **one** edge: `A <label> B`, written into A's `related:` as `- <label>: B` and rendered into A's generated links region. B is not touched and does not point back - its inbound view is rendered from the graph. Idempotent, and re-running with a different label *relabels* rather than appending, since one page asserts one thing about another. Refuses before writing when the type does not declare `related:` (a source page declares `entities:`/`concepts:` - the refusal names them and points at `link-source`), and when `<label>` is not authorised by the source collection's `outbound:` block for the target's collection; that refusal lists the authorised set and points at `instructions/link-taxonomy.md` | | `xref remove --a "<A>" --b "<B>" [--dry-run]` | Clears the reference in **both** directions - it is the cleanup command for a deleted or hand-renamed page rather than the strict inverse of a one-directional `add`. Clears `<B>` from every page-ref frontmatter field `<A>`'s type declares (`related:`, `sources:`, `entities:`, `concepts:`) plus the matching bullets. It also sweeps a field the type does *not* declare but some other type does, and drops that key outright once empty - a leftover written before the check above existed has to stay repairable, or the page is a dead end. `--b` need not still exist as a page, so this is how a reference left by a hand-deleted or hand-renamed page gets cleared without hand-editing frontmatter. Idempotent. | | `xref link-source --source "Source - X" --entities A,B,C` | Batch-link a source page to every entity/concept it mentions: each target gets `sources:`, and the source page records each target in its own `entities:`/`concepts:`. No body bullet is written on either side - `sources:` *is* the record, and the See Also bullet this used to add was the reciprocal half of a model that no longer exists. Which of the two is chosen follows the target's collection (`kb/entities/` -> `entities:`), so a new collection needs no code change here. A target whose collection matches no reference field the source type declares is linked one-way and named in the output. Idempotent in both directions | @@ -51,7 +51,7 @@ tools/wikitool <command> --help | `index rebuild [--dry-run]` | Regenerate the catalog from every page's frontmatter: `kb/index.md` becomes a map (statistics, one row per collection and per area, links to the shards) and the page tables are written to a generated `INDEX.md` in each collection. An area past 50 rows gets its own shard. Stale shards from removed collections/areas are deleted in the same pass | | `log append --op ingest\|query\|lint\|create\|update\|delete\|rename\|move --title "..." [--body "..."\|--body-file path]` | Append a formatted entry to `kb/log.md` | | `log status` | Read-only: count `ingest` entries logged since the last `lint` entry - the deterministic trigger behind the Maintenance Schedule's "every 10 sources" full-lint cadence | -| `lint [--json] [--markdown out.md] [--full] [--fail-on-error]` | Structural + provenance checks: broken wikilinks, dangling frontmatter references, orphan pages, index drift, schema gaps, duplicate titles, title mismatches, uncovered raw files, broken `raw_files:` refs, raw files claimed by more than one source page, unmarked provenance, citation/frontmatter drift, unbalanced generated-region markers, edges whose label is missing or not authorised by the source collection's `outbound:` (both hard once `kb_version` has reached the release that introduced labelled edges - advisory below it, so a corpus mid-migration is not refused by the check measuring it), `see-also` edges whose reverse direction already carries a specific label (advisory only - redundant rather than wrong, and never migration-gated, since no version turns the redundancy into an error), quote-limit overages (>2 blockquoted lines/page, advisory only). Prints only the sections that found something and always writes the full report to `reports/Lint Report <date>.md` (or `--markdown`), naming the path - `--full` prints everything, `--json` prints the findings and writes nothing | +| `lint [--json] [--markdown out.md] [--full] [--fail-on-error]` | Structural + provenance checks: broken wikilinks, dangling frontmatter references, orphan pages, index drift, schema gaps, duplicate titles, title mismatches, pages nested more than one directory below their collection (hard - the generated catalog folds these into their area silently rather than merely reading it, see #57), uncovered raw files, broken `raw_files:` refs, raw files claimed by more than one source page, unmarked provenance, citation/frontmatter drift, unbalanced generated-region markers, edges whose label is missing or not authorised by the source collection's `outbound:` (both hard once `kb_version` has reached the release that introduced labelled edges - advisory below it, so a corpus mid-migration is not refused by the check measuring it), `see-also` edges whose reverse direction already carries a specific label (advisory only - redundant rather than wrong, and never migration-gated, since no version turns the redundancy into an error), quote-limit overages (>2 blockquoted lines/page, advisory only). Prints only the sections that found something and always writes the full report to `reports/Lint Report <date>.md` (or `--markdown`), naming the path - `--full` prints everything, `--json` prints the findings and writes nothing | | `search ["<text>"] [--field <predicate> ...] [--kind/--subtype/--collection/--tag <v>] [--regex] [--limit N] [--sort [-]<field>] [--backend <name>] [--matches] [--json]` | Find pages in `kb/` without reading the index. Text search runs through a pluggable backend (`rg` today); `--field` predicates are evaluated on frontmatter - `f=v`, `f~substring`, `'f>=v'`, `'f:*'` (present), `'!f'` (absent), repeatable and ANDed. With no text this is a pure structured query. Results carry kind/summary/confidence so a hit can be judged without opening the page. A page whose frontmatter does not parse can match no positive predicate, so it is **named** rather than dropped: `--json` always carries an `unreadable` list of `{path, reason}` (usually empty), and the table form writes the same lines to stderr. `--regex` is applied by `rg` alone, whose engine is linear; the ranking boosts for title and summary are literal-containment only, so a non-literal pattern is ranked by match count. `rg` is killed after 30 s and reported as a failure. Read-only, and **exempt from the Iteration Budget Gate** | | `confidence decay [--apply]` | Recompute every page's derived `confidence` as `confidence_base * (1 - 0.01/month)`, floored at 0.2; dry-run by default | | `confidence init-base [--apply]` | One-time backfill: set `confidence_base` from the current `confidence` on pages that predate the derived-confidence model | diff --git a/tools/chemenu/commands/index_build.py b/tools/chemenu/commands/index_build.py index 8ec4dde..9067d9a 100644 --- a/tools/chemenu/commands/index_build.py +++ b/tools/chemenu/commands/index_build.py @@ -28,7 +28,7 @@ from chemenu import config from chemenu.commands._util import rel_path, success from chemenu.kb_collections import iter_kb_collections from chemenu.page import Page -from chemenu.kb_scan import GENERATED_INDEX, load_kb_pages +from chemenu.kb_scan import GENERATED_INDEX, find_nested_pages, load_kb_pages from chemenu.type_resolver import resolver app = typer.Typer(help="Manage the generated wiki catalog (kb/index.md + per-collection INDEX.md).") @@ -302,6 +302,17 @@ def index_rebuild( False, "--dry-run", help="Print what would be written instead of writing it" ), ): + # Reported, not refused (#57 decision): a nested page still gets a catalog + # written for it, just a wrong one (folded into its area, no distinct + # location of its own) - `wikitool lint`'s `nested_pages` is the hard + # finding this warning previews. + for title, page, depth in find_nested_pages(config.KB_DIR, load_kb_pages(config.KB_DIR)): + typer.echo( + f"WARNING: [[{title}]] is {depth} directories below its collection " + f"({rel_path(page.path.parent)}) - the catalog folds it into the area silently. " + "See `wikitool lint`'s Nested Pages finding." + ) + plan = plan_index(config.KB_DIR) stale = stale_shards(config.KB_DIR, plan) diff --git a/tools/chemenu/commands/page_ops.py b/tools/chemenu/commands/page_ops.py index 9d02919..89ae418 100644 --- a/tools/chemenu/commands/page_ops.py +++ b/tools/chemenu/commands/page_ops.py @@ -381,6 +381,24 @@ def rm_command( ) +def _rmdir_if_emptied(directory: Path) -> bool: + """Remove `directory` if the move that just vacated it left it empty. + + Symmetric with the `target_dir.mkdir(parents=True, exist_ok=True)` a move + does on the way in: without this, a hand-nested directory (#57) survives + its own fix, and a directory-shape test would still find it after every + page under it moved out. Never touches `kb/` itself, and only ever a + directory this move just emptied - never a pre-existing empty one it + happens to pass through. + """ + if directory.resolve() == config.KB_DIR.resolve(): + return False + if not directory.is_dir() or any(directory.iterdir()): + return False + directory.rmdir() + return True + + def move_command( page_title: Optional[str] = typer.Option( None, "--page", help="Exact title of the page to move to its computed location" @@ -431,7 +449,9 @@ def move_command( moved: list[str] = [] failed: list[str] = list(collisions) + removed_dirs = 0 for title, page, target_dir, new_path in planned: + old_dir = page.path.parent try: target_dir.mkdir(parents=True, exist_ok=True) page.path.rename(new_path) @@ -440,13 +460,17 @@ def move_command( continue moved.append(title) typer.echo(f" moved '{title}' -> {rel_path(new_path)}") + if _rmdir_if_emptied(old_dir): + removed_dirs += 1 + typer.echo(f" removed empty {rel_path(old_dir)}") if failed: fail( f"Moved {len(moved)}/{len(candidates)} page(s) before a failure: {', '.join(failed)}. " "Safe to retry - `move --reconcile` only re-moves what is still misplaced." ) - success(f"Moved {len(moved)} page(s). Run `wikitool index rebuild` next.") + removed_note = f", removed {removed_dirs} emptied dir(s)" if removed_dirs else "" + success(f"Moved {len(moved)} page(s){removed_note}. Run `wikitool index rebuild` next.") return target = pages.get(page_title) @@ -473,6 +497,10 @@ def move_command( typer.echo(f"[dry-run] would move {rel_path(target.path)} -> {rel_path(new_path)}") return + old_dir = target.path.parent target_dir.mkdir(parents=True, exist_ok=True) target.path.rename(new_path) - success(f"Moved '{page_title}' -> {rel_path(new_path)}. Run `wikitool index rebuild` next.") + removed_note = "" + if _rmdir_if_emptied(old_dir): + removed_note = f" Removed emptied {rel_path(old_dir)}." + success(f"Moved '{page_title}' -> {rel_path(new_path)}.{removed_note} Run `wikitool index rebuild` next.") diff --git a/tools/chemenu/kb_scan.py b/tools/chemenu/kb_scan.py index e3a4eeb..01a8315 100644 --- a/tools/chemenu/kb_scan.py +++ b/tools/chemenu/kb_scan.py @@ -120,6 +120,32 @@ def count_wikilinks(body: str) -> Counter[str]: return Counter(m.group(1).strip() for m in WIKILINK_RE.finditer(strip_code_spans(body))) +def find_nested_pages(kb_dir: Path, pages: dict[str, Page]) -> list[tuple[str, Page, int]]: + """(title, page, depth) for every page sitting more than one directory + below its collection. + + `kb/<collection>/<page>.md` and `kb/<collection>/<area>/<page>.md` are the + only two depths `kb/CONTRACT.md` § Collections describes. A third level is + not merely unconventional - it is invisible to the catalog: + `index_build.group_pages` reads exactly `parts[0]`/`parts[1]` and folds + anything past them into the area's table silently (Gitea #57), so a page + down here renders as if it sat directly in the area, under no name of its + own. `depth` is how many directories separate the page from its + collection root (1 = directly in an area, the deepest that is not this + finding). + """ + found = [] + for title, page in sorted(pages.items()): + try: + parts = page.path.relative_to(kb_dir).parts + except ValueError: + continue + depth = len(parts) - 2 # collection + filename are always present + if depth > 1: + found.append((title, page, depth)) + return found + + def build_link_graph(pages: dict[str, Page]) -> dict[str, set[str]]: """Map each page title to the set of titles it links to.""" return {title: extract_wikilinks(page.body) for title, page in pages.items()} diff --git a/tools/chemenu/lint_core.py b/tools/chemenu/lint_core.py index 417ae73..b7d2f38 100644 --- a/tools/chemenu/lint_core.py +++ b/tools/chemenu/lint_core.py @@ -34,6 +34,7 @@ from chemenu.kb_scan import ( WIKILINK_RE, build_link_graph, find_duplicate_title_paths, + find_nested_pages, inbound_links, load_kb_pages, ) @@ -123,6 +124,23 @@ def misplaced_pages(pages: dict[str, Page]) -> list[dict]: ] +def nested_pages(kb_dir: Path, pages: dict[str, Page]) -> list[dict]: + """Report form of `find_nested_pages`: `{"page", "at", "depth"}` per + finding. + + Hard rather than advisory, unlike `misplaced_pages` above: a hand-placed + page in the wrong area is still a real page the catalog lists correctly. + A page nested past an area is not - `group_pages` folds it into the area + silently, so the *generated* catalog itself becomes wrong, which is the + thing invariant 1 does not allow. There is also no version this becomes + wrong at (unlike the migration-gated findings below): a nested page was + always going to be misread by the catalog that reads it today.""" + return [ + {"page": title, "at": _display(page.path.parent), "depth": depth} + for title, page, depth in find_nested_pages(kb_dir, pages) + ] + + def run_lint(kb_dir: Path) -> dict: pages = load_kb_pages(kb_dir) duplicate_titles = find_duplicate_title_paths(kb_dir, config.ROOT) @@ -187,6 +205,7 @@ def run_lint(kb_dir: Path) -> dict: title_mismatches.append({"page": title, "h1": h1}) misplaced = misplaced_pages(pages) + nested = nested_pages(kb_dir, pages) unmarked_provenance = [] for title, page in sorted(pages.items()): @@ -368,6 +387,7 @@ def run_lint(kb_dir: Path) -> dict: "title_mismatches": title_mismatches, "duplicate_titles": duplicate_titles, "misplaced_pages": misplaced, + "nested_pages": nested, "uncovered_raw_files": find_uncovered_raw_files(config.RAW_DIR, pages), "broken_raw_refs": find_broken_raw_refs(pages), "duplicate_raw_file_owners": find_duplicate_raw_file_owners(pages), @@ -437,6 +457,13 @@ def render_markdown(report: dict) -> str: lambda i: f"[[{i['page']}]] is at `{i['at']}`, should be under `{i['should_be']}` " f"- `wikitool move --page \"{i['page']}\"`", ) + _section( + lines, "Nested Pages (more than one directory below their collection)", + report.get("nested_pages", []), + lambda i: f"[[{i['page']}]] is {i['depth']} directories below `kb/` at `{i['at']}` - " + "the catalog folds this into its area silently; `wikitool move --reconcile` fixes it " + "when the page's type resolves to a shallower directory, otherwise move it up by hand", + ) _section( lines, "Uncovered Raw Files (no source page)", report["uncovered_raw_files"], lambda i: f"`{i}`", @@ -593,6 +620,12 @@ def default_report_path(report: dict) -> Path: # `malformed_edges` and `unbalanced_markers` are hard from the start: neither # describes an unconverted page, only a broken one. # +# `nested_pages` is hard from the start too, and for the same reason as +# `malformed_edges`/`unbalanced_markers` rather than `misplaced_pages`'s: it is +# not a hand-placement habit some instances predate, it is a page the +# generated catalog (`index rebuild`) silently mis-describes today, on every +# instance, at every version - see `nested_pages()` above. +# # One definition, used by `lint --fail-on-error` and by the eval scorecard: if # the two disagreed, a run could pass its score while lint refused it. HARD_ERROR_KEYS = ( @@ -600,6 +633,7 @@ HARD_ERROR_KEYS = ( "broken_links", "dangling_index_entries", "duplicate_titles", + "nested_pages", "broken_raw_refs", "duplicate_raw_file_owners", "legacy_source_pages", diff --git a/tools/chemenu/tests/test_index_build.py b/tools/chemenu/tests/test_index_build.py index 13023bb..2c5fbaf 100644 --- a/tools/chemenu/tests/test_index_build.py +++ b/tools/chemenu/tests/test_index_build.py @@ -6,6 +6,7 @@ from chemenu.commands.index_build import ( SHARD_THRESHOLD, _anchor, build_index, + index_rebuild, plan_index, stale_shards, ) @@ -202,3 +203,25 @@ def test_scanner_ignores_generated_shards_at_any_depth(kb_dir, plan): def test_rebuild_is_idempotent(kb_dir): assert plan_index(kb_dir) == plan_index(kb_dir) + + +def test_rebuild_warns_about_a_nested_page_without_failing(kb_dir, capsys): + """#57: `group_pages` still folds a nested page into its area silently - + that part is unchanged, reporting is the fix - so `index rebuild` warns + rather than refusing, and must not raise.""" + write_page( + kb_dir / "entities/projects/someowner/nested-tool.md", + {"type": "types/entity.md", "entity_type": "project", "tags": [], + "created": "2026-07-25", "modified": "2026-07-25", + "related": [], "sources": [], "confidence": 0.8}, + "\n# nested-tool\n", + ) + index_rebuild(dry_run=False) + out = capsys.readouterr().out + assert "WARNING" in out + assert "nested-tool" in out + + +def test_rebuild_is_silent_about_a_healthy_tree(kb_dir, capsys): + index_rebuild(dry_run=False) + assert "WARNING" not in capsys.readouterr().out diff --git a/tools/chemenu/tests/test_kb_scan.py b/tools/chemenu/tests/test_kb_scan.py index 7226a90..927d834 100644 --- a/tools/chemenu/tests/test_kb_scan.py +++ b/tools/chemenu/tests/test_kb_scan.py @@ -1,4 +1,5 @@ -from chemenu.kb_scan import iter_kb_pages, load_kb_pages +from chemenu.frontmatter_io import write_page +from chemenu.kb_scan import find_nested_pages, iter_kb_pages, load_kb_pages def _names(kb_dir): @@ -43,3 +44,21 @@ def test_pages_are_keyed_by_filename_stem(kb_dir): pages = load_kb_pages(kb_dir) assert "aurora" in pages assert "Source - Aurora" in pages + + +def test_find_nested_pages_flags_a_page_below_its_area(kb_dir): + write_page( + kb_dir / "entities/projects/someowner/nested-tool.md", + {"type": "types/entity.md", "entity_type": "project", "tags": [], + "created": "2026-07-25", "modified": "2026-07-25", + "related": [], "sources": [], "confidence": 0.8}, + "\n# nested-tool\n", + ) + pages = load_kb_pages(kb_dir) + found = find_nested_pages(kb_dir, pages) + assert [title for title, _page, _depth in found] == ["nested-tool"] + assert found[0][2] == 2 + + +def test_find_nested_pages_is_silent_for_pages_directly_in_an_area(kb_dir): + assert find_nested_pages(kb_dir, load_kb_pages(kb_dir)) == [] diff --git a/tools/chemenu/tests/test_lint.py b/tools/chemenu/tests/test_lint.py index f17f126..67dc92a 100644 --- a/tools/chemenu/tests/test_lint.py +++ b/tools/chemenu/tests/test_lint.py @@ -213,6 +213,29 @@ def test_lint_is_silent_about_correctly_placed_pages(kb_dir): assert run_lint(kb_dir)["misplaced_pages"] == [] +def test_lint_detects_a_nested_page_as_a_hard_error(kb_dir): + """Gitea #57: a page sitting a level below its area (like the three real + `kb/entities/projects/<owner>/*.md` pages that prompted this) is not just + misplaced - the generated catalog folds it into the area silently, so + this is hard rather than advisory.""" + write_page( + kb_dir / "entities/projects/someowner/nested-tool.md", + {"type": "types/entity.md", "entity_type": "project", "tags": [], "created": "2026-07-25", + "modified": "2026-07-25", "related": [], "sources": [], "confidence": 0.8}, + "\n# nested-tool\n", + ) + report = run_lint(kb_dir) + entry = next(i for i in report["nested_pages"] if i["page"] == "nested-tool") + assert entry["at"].endswith("entities/projects/someowner") + assert entry["depth"] == 2 + assert "nested_pages" in HARD_ERROR_KEYS + assert has_hard_errors(report) + + +def test_lint_is_silent_about_pages_directly_in_an_area(kb_dir): + assert run_lint(kb_dir)["nested_pages"] == [] + + def test_lint_flags_legacy_citation_marker_as_hard_error(kb_dir): write_page( kb_dir / "concepts/Modbus.md", diff --git a/tools/chemenu/tests/test_page_ops.py b/tools/chemenu/tests/test_page_ops.py index 47307f9..6a5d613 100644 --- a/tools/chemenu/tests/test_page_ops.py +++ b/tools/chemenu/tests/test_page_ops.py @@ -377,4 +377,35 @@ def test_move_reconcile_dry_run_writes_nothing(patched_wiki): _write_misplaced(patched_wiki, "entities/systems", "misplaced-tool", "tool") page_ops.move_command(page_title=None, reconcile=True, dry_run=True) assert (patched_wiki / "entities/systems/misplaced-tool.md").exists() + + +def test_move_removes_the_directory_it_empties(patched_wiki): + """The real-world case (Gitea #57): a page nested a level below its area + (`entities/projects/<owner>/*.md`) is also misplaced by `find_misplaced` - + its computed directory is `entities/projects`, one level up from where it + sits - so `move` fixes the nesting as a side effect of fixing the + placement, and must not leave the now-empty `<owner>/` directory behind.""" + _write_misplaced(patched_wiki, "entities/projects/someowner", "nested-tool", "project") + page_ops.move_command(page_title="nested-tool", reconcile=False, dry_run=False) + assert (patched_wiki / "entities/projects/nested-tool.md").exists() + assert not (patched_wiki / "entities/projects/someowner").exists() + + +def test_move_does_not_remove_a_directory_that_still_holds_pages(patched_wiki): + _write_misplaced(patched_wiki, "entities/systems", "misplaced-tool", "tool") + page_ops.move_command(page_title="misplaced-tool", reconcile=False, dry_run=False) + assert (patched_wiki / "entities/systems").is_dir() + assert (patched_wiki / "entities/systems/aurora.md").exists() + + +def test_move_reconcile_removes_every_directory_it_empties(patched_wiki): + _write_misplaced(patched_wiki, "entities/projects/kfchou", "wiki-skills", "project") + _write_misplaced( + patched_wiki, "entities/projects/vanillaflava", "wiki-skills-vanillaflava", "project" + ) + page_ops.move_command(page_title=None, reconcile=True, dry_run=False) + assert (patched_wiki / "entities/projects/wiki-skills.md").exists() + assert (patched_wiki / "entities/projects/wiki-skills-vanillaflava.md").exists() + assert not (patched_wiki / "entities/projects/kfchou").exists() + assert not (patched_wiki / "entities/projects/vanillaflava").exists() assert not (patched_wiki / "entities/tools/misplaced-tool.md").exists() diff --git a/tools/chemenu/tests/test_type_resolver.py b/tools/chemenu/tests/test_type_resolver.py index 93a76df..6c0806f 100644 --- a/tools/chemenu/tests/test_type_resolver.py +++ b/tools/chemenu/tests/test_type_resolver.py @@ -92,6 +92,33 @@ def test_get_layout_is_none_for_types_without_one(): assert resolver.get_layout("types/source.md") is None +def test_get_layout_rejects_a_dir_with_a_path_separator(tmp_path): + """A `layout:` `dir:` is the one supported way to place a page below its + collection; a value with a separator would nest a page a level past what + `kb/CONTRACT.md` allows, invisibly to the catalog (Gitea #57). Refused at + load time rather than left for `lint`'s `nested_pages` finding to catch + after a page has already been written there.""" + from chemenu.type_resolver import TypeResolver + + types_dir = tmp_path / "types" + types_dir.mkdir() + (types_dir / "badtype.md").write_text( + "---\n" + "type: types/badtype.md\n" + "name: badtype\n" + "description: broken layout\n" + "base_dir: badthings\n" + "subtype_field: bad_type\n" + "layout:\n" + " owner: {dir: projects/owner, title: Owner}\n" + "---\n\n# badtype\n", + encoding="utf-8", + ) + bad_resolver = TypeResolver(repo_root=tmp_path) + with pytest.raises(ValueError, match="single path segment"): + bad_resolver.get_layout("types/badtype.md") + + def test_get_base_dir_is_wiki_root_relative(): """base_dir is deliberately relative to the wiki root (not the repo root) so callers resolve it against config.KB_DIR, which tests diff --git a/tools/chemenu/type_resolver.py b/tools/chemenu/type_resolver.py index 5a90ed2..8a42a85 100644 --- a/tools/chemenu/type_resolver.py +++ b/tools/chemenu/type_resolver.py @@ -316,10 +316,27 @@ class TypeResolver: Raises: ValueError: If the type path cannot be resolved (propagated from - `load_type_spec`) + `load_type_spec`), or a declared `dir:` is not a single path + segment. """ type_spec = self.load_type_spec(type_path, source_file) - return type_spec['frontmatter'].get('layout') + layout = type_spec['frontmatter'].get('layout') + if layout: + for subtype_value, entry in layout.items(): + directory = entry.get('dir') + # A `dir:` with a separator would place a page a level below + # what `kb/CONTRACT.md` § Collections allows (collection/area, + # nothing deeper) through the one supported path - `layout:` - + # rather than by hand. Caught here, once, rather than left for + # `wikitool lint`'s `nested_pages` to find after the fact. + if directory is not None and ( + not directory or directory in ('.', '..') or '/' in directory or '\\' in directory + ): + raise ValueError( + f"Type {type_path} layout[{subtype_value!r}].dir is {directory!r} - " + "must be a single path segment, not a nested path (see Gitea #57)" + ) + return layout def get_base_dir(self, type_path: str, source_file: Path = None) -> Optional[str]: """Return a type-spec's `base_dir:` frontmatter - the directory where