feat: path budget - a file's path stays at 160 characters or fewer; new, rename, move and raw accept refuse more, lint reports Long Paths (#163)
Files changed: - CHANGES.md - README.md - VERSION - instructions/page-lifecycle.md - kb/CONTRACT.md - tools/CONTRACT.md - tools/chemenu/commands/_util.py - tools/chemenu/commands/lint.py - tools/chemenu/commands/new_page.py - tools/chemenu/commands/page_ops.py - tools/chemenu/commands/raw_cmd.py - tools/chemenu/lint_core.py - tools/chemenu/tests/test_lint.py - tools/chemenu/tests/test_new_page.py - tools/chemenu/tests/test_page_ops.py - tools/chemenu/tests/test_raw_cmd.py - tools/chemenu/tests/test_titles.py - tools/chemenu/titles.py
This commit is contained in:
1 parent
03743ebbc0
commit
04aebdeccf
18 files changed
+394
-29
No files matched your search
+13
-10
@@ -180,7 +180,7 @@ Scaffold a new wiki page of any type.
|
||||
|
||||
- 0 success
|
||||
- 1 A page with this title already exists, the type is unknown, or a `--set` value is invalid
|
||||
- 1 The title is not a valid file name (forbidden character, control character, reserved name such as `CON` or `Index`, trailing dot or space, empty), collides with another page by case or Unicode normalization, or the target file already exists
|
||||
- 1 The title is not a valid file name (forbidden character, control character, reserved name such as `CON` or `Index`, trailing dot or space, empty), collides with another page by case or Unicode normalization, the target file already exists, or the target path is over the 160-character path budget
|
||||
- 1 A `raw_files` path does not exist
|
||||
- 1 A capture field the type-spec requires is missing, or set to `unknown`
|
||||
- 1 `--resume` with a type other than `project`
|
||||
@@ -192,7 +192,7 @@ Scaffold a new wiki page of any type.
|
||||
**ON FAILURE**
|
||||
|
||||
- A page with this title already exists, the type is unknown, or a `--set` value is invalid -> Not transient - fix the argument and retry once
|
||||
- The title is not a valid file name (forbidden character, control character, reserved name such as `CON` or `Index`, trailing dot or space, empty), collides with another page by case or Unicode normalization, or the target file already exists -> Not transient - choose another title and retry once. Nothing was created, and for `new project` no tracker project either
|
||||
- The title is not a valid file name (forbidden character, control character, reserved name such as `CON` or `Index`, trailing dot or space, empty), collides with another page by case or Unicode normalization, the target file already exists, or the target path is over the 160-character path budget -> Not transient - choose another (for the budget: a shorter) title and retry once. Nothing was created, and for `new project` no tracker project either
|
||||
- A `raw_files` path does not exist -> Not transient - fix the path and retry once
|
||||
- A capture field the type-spec requires is missing, or set to `unknown` -> Pass it explicitly (e.g. `--set fidelity=verbatim --set authority=reporting`), then retry once
|
||||
- `--resume` with a type other than `project` -> Drop `--resume` and retry once
|
||||
@@ -208,6 +208,7 @@ Scaffold a new wiki page of any type.
|
||||
**NOTES**
|
||||
|
||||
- A title becomes a file name, so it must be valid and unique on Windows and macOS as well as Linux, whichever platform runs the command and whichever root the type writes to. The rule is `kb/CONTRACT.md` § Titles are identifiers; it is checked on the full title, after `title_prefix`.
|
||||
- The target's path below the instance root may be at most 160 characters, counted in UTF-16 code units the way Windows counts MAX_PATH, so a Windows checkout without long paths keeps working. A longer one is refused, for every root, naming the length and how much shorter it has to get.
|
||||
- `new` never overwrites: a file already at the target - or one a case-insensitive file system would treat as the same file - is refused for every root, `instructions/` included.
|
||||
- The type-spec drives everything: fields, directory (`base_dir`/`layout`), title prefix, and template. `types list`/`types describe` show what a type requires.
|
||||
- A schema `default:` is materialized only for a field the schema also lists in `required:`.
|
||||
@@ -475,14 +476,14 @@ Rename a page, or repoint references that name a page that never existed.
|
||||
- 0 success
|
||||
- 1 `--from` equals `--to`
|
||||
- 1 Neither `--from` nor `--to` is a page
|
||||
- 1 The `--to` title is already taken - also by a page that differs only in case or Unicode normalization, or by a file in the page's directory - or is not a valid file name (see `kb/CONTRACT.md` § Titles are identifiers)
|
||||
- 1 The `--to` title is already taken - also by a page that differs only in case or Unicode normalization, or by a file in the page's directory - or is not a valid file name, or would put the page's path over the path budget (see `kb/CONTRACT.md` § Titles are identifiers)
|
||||
- 1 A page write failed partway; nothing was renamed on disk
|
||||
|
||||
**ON FAILURE**
|
||||
|
||||
- `--from` equals `--to` -> Fix the arguments and retry once
|
||||
- Neither `--from` nor `--to` is a page -> Create the page first with `wikitool new`, or drop the reference with `wikitool xref remove`
|
||||
- The `--to` title is already taken - also by a page that differs only in case or Unicode normalization, or by a file in the page's directory - or is not a valid file name (see `kb/CONTRACT.md` § Titles are identifiers) -> Choose another title and retry once. Checked under `--dry-run` too
|
||||
- The `--to` title is already taken - also by a page that differs only in case or Unicode normalization, or by a file in the page's directory - or is not a valid file name, or would put the page's path over the path budget (see `kb/CONTRACT.md` § Titles are identifiers) -> Choose another, or a shorter, title and retry once. Checked under `--dry-run` too
|
||||
- A page write failed partway; nothing was renamed on disk -> Check `git status`, resolve the write failure (permissions/disk), then re-run the full command - safe, since each page's rewrite is idempotent
|
||||
|
||||
**NEVER**
|
||||
@@ -495,7 +496,7 @@ Rename a page, or repoint references that name a page that never existed.
|
||||
- If `--from` is *not* a page but is referenced, it instead repoints those references onto the existing `--to` page and moves nothing - the fix for a reference spelled `act_runner` when the page is `Act Runner`.
|
||||
- Each page's rewrite is idempotent, so a re-run as-is is safe. If a write fails midway, nothing is renamed on disk and the error lists what was updated.
|
||||
- `--dry-run` lists every page it would change; run it first to see the blast radius.
|
||||
- Only `--to` is checked against the title rule. A page whose current title breaks it (`lint`'s Unportable Titles) can always be renamed away from it, and a title that differs from the page's own only by case (`Foo` to `FOO`) is allowed.
|
||||
- Only `--to` is checked against the title rule and the path budget (160 UTF-16 code units for the whole path below the instance root). A page whose current title breaks either (`lint`'s Unportable Titles and Long Paths) can always be renamed away from it, and a title that differs from the page's own only by case (`Foo` to `FOO`) is allowed.
|
||||
|
||||
**SEE ALSO**
|
||||
|
||||
@@ -583,14 +584,14 @@ Move a page (or every misplaced page) to the directory its type-spec computes.
|
||||
- 0 success
|
||||
- 1 Neither or both of `--page`/`--reconcile` given
|
||||
- 1 The named page is not found, or has no `type:` to compute a placement from
|
||||
- 1 The destination already holds an entry with the same name, or one that differs only in case or Unicode normalization (a pre-existing duplicate-stem collision) - refused rather than silently skipped
|
||||
- 1 The destination already holds an entry with the same name, or one that differs only in case or Unicode normalization (a pre-existing duplicate-stem collision), or its path would be over the path budget - refused rather than silently skipped
|
||||
- 1 `--reconcile` failed partway
|
||||
|
||||
**ON FAILURE**
|
||||
|
||||
- Neither or both of `--page`/`--reconcile` given -> Fix the arguments and retry once
|
||||
- The named page is not found, or has no `type:` to compute a placement from -> Fix the title, or give the page its `type:`, then retry once
|
||||
- The destination already holds an entry with the same name, or one that differs only in case or Unicode normalization (a pre-existing duplicate-stem collision) - refused rather than silently skipped -> Resolve the collision, then retry
|
||||
- The destination already holds an entry with the same name, or one that differs only in case or Unicode normalization (a pre-existing duplicate-stem collision), or its path would be over the path budget - refused rather than silently skipped -> Resolve the collision, or `wikitool rename` the page to a shorter title, then retry
|
||||
- `--reconcile` failed partway -> Safe to retry as-is - `--reconcile` only re-moves what is still misplaced
|
||||
|
||||
**NEVER**
|
||||
@@ -600,6 +601,7 @@ Move a page (or every misplaced page) to the directory its type-spec computes.
|
||||
**NOTES**
|
||||
|
||||
- Moves a page to the directory its type-spec computes for its current frontmatter (`base_dir` + `layout`, the same rule `new` places a page by) - never to a hand-chosen destination; there is no `--to <dir>`.
|
||||
- A destination whose path would be over the path budget (160 UTF-16 code units below the instance root) is refused, and `--reconcile` skips such a page and names it, as it does for an occupied destination - `wikitool rename` the page to a shorter title.
|
||||
- `--reconcile` applies it corpus-wide: every misplaced page moves in one call, and a second run reports nothing left to do. It fixes `lint`'s `Misplaced Pages` (advisory) and `Nested Pages` (hard) findings.
|
||||
- Neither mode touches a body or a frontmatter field, and the page's title - its only identity in the wiki - never changes; only the file moves.
|
||||
- A directory a move empties is removed with it, so a page nested below its area leaves no leftover directory behind.
|
||||
@@ -1116,6 +1118,7 @@ Run structural lint checks against kb/.
|
||||
- Advisory only: `see-also` edges whose reverse direction already carries a specific label - never migration-gated.
|
||||
- Advisory only: a collection past the catalog's per-area shard threshold that has no areas to shard, reported with the split its subtype field would produce, and only when that split puts every resulting area at or under the threshold.
|
||||
- Advisory only: source pages sitting in the `unclassified` catalog slot.
|
||||
- Advisory only: Long Paths - a file under `kb/` or `raw/` whose path below the instance root is over 160 UTF-16 code units, the budget that keeps a Windows checkout without long paths working. Reported as `{path, length}`; a corpus over the budget breaks no lint run. `wikitool rename` is the fix for a page.
|
||||
- Advisory only: quote-limit overages (>2 blockquotes/page).
|
||||
- Prints only the sections that found something and always writes the full report to `reports/Lint Report <date>.md` (or `--markdown`), naming the path. `--full` prints everything; `--json` prints the findings and writes nothing.
|
||||
- Exits 0 whatever it finds unless `--fail-on-error` is passed.
|
||||
@@ -1124,7 +1127,7 @@ Run structural lint checks against kb/.
|
||||
|
||||
- `wiki-lint` skill - the procedure that runs this
|
||||
- `wikitool move --reconcile` - fixes Misplaced and Nested Pages
|
||||
- `wikitool rename` - fixes Unportable Titles
|
||||
- `wikitool rename` - fixes Unportable Titles and, for a page, Long Paths
|
||||
- `wikitool log status` - whether a full lint is due
|
||||
|
||||
#### `search`
|
||||
@@ -1400,7 +1403,7 @@ Promote one or more files from `incoming/` into `raw/`.
|
||||
**EXIT STATUS**
|
||||
|
||||
- 0 success
|
||||
- 1 raw accept: A file does not exist, is not under `incoming/`, or is nested more than one level below it; two files in one call share a filename; or a target path already exists
|
||||
- 1 raw accept: A file does not exist, is not under `incoming/`, or is nested more than one level below it; two files in one call share a filename; a target path already exists; or a target path would be over the path budget (160 UTF-16 code units below the instance root)
|
||||
- 1 raw accept: `--fidelity`/`--authority` is missing, or names `unknown` or a value outside the schema's enum
|
||||
- 1 raw accept: The target name is already occupied anywhere under `raw/` by something the call does not own
|
||||
- 1 raw accept: `--page` names an unknown page or one with no `raw_files:` yet, an existing `raw_files:` entry is missing on disk, a file to be moved has more than one owning page, or `--page` would overwrite an already-set `fidelity`/`authority` with a different value
|
||||
@@ -1410,7 +1413,7 @@ Promote one or more files from `incoming/` into `raw/`.
|
||||
|
||||
**ON FAILURE**
|
||||
|
||||
- raw accept: A file does not exist, is not under `incoming/`, or is nested more than one level below it; two files in one call share a filename; or a target path already exists -> Fix the named argument and retry once
|
||||
- raw accept: A file does not exist, is not under `incoming/`, or is nested more than one level below it; two files in one call share a filename; a target path already exists; or a target path would be over the path budget (160 UTF-16 code units below the instance root) -> Fix the named argument and retry once. For a path over the budget, rename the file in `incoming/` to something shorter - the refusal comes before anything moves, so `incoming/` and `raw/` are unchanged
|
||||
- raw accept: `--fidelity`/`--authority` is missing, or names `unknown` or a value outside the schema's enum -> Pass both with a valid value, then retry once
|
||||
- raw accept: The target name is already occupied anywhere under `raw/` by something the call does not own -> Not fixed by retrying: the refusal names `--replaces` (same source, new edition) and renaming in `incoming/` (a separate source) as the two routes, and neither is the tool's to pick. Show the message to the user and wait
|
||||
- raw accept: `--page` names an unknown page or one with no `raw_files:` yet, an existing `raw_files:` entry is missing on disk, a file to be moved has more than one owning page, or `--page` would overwrite an already-set `fidelity`/`authority` with a different value -> Fix the named argument and retry once; a different capture value on an existing page is a new edition - `--replaces`
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
"""Shared helpers for wikitool subcommands."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from datetime import date
|
||||
@@ -231,6 +232,26 @@ def check_title(name: str) -> None:
|
||||
fail(escape(f"'{name}' cannot be a page title: " + "; ".join(problems) + "."))
|
||||
|
||||
|
||||
def path_budget_problem_for(path: Path) -> str | None:
|
||||
"""`chemenu.titles.path_budget_problem` for a path on disk, measured from the
|
||||
instance root."""
|
||||
from chemenu.titles import path_budget_problem
|
||||
|
||||
return path_budget_problem(rel_path(path).replace(os.sep, "/"))
|
||||
|
||||
|
||||
def check_path_budget(path: Path, remedy: str) -> None:
|
||||
"""Fail if `path` is over the path budget (see `chemenu.titles.PATH_BUDGET`).
|
||||
|
||||
Runs before any write, for every root. `remedy` is the sentence that tells
|
||||
the caller what to shorten, since the name comes from a title in one command
|
||||
and from a file in `incoming/` in another.
|
||||
"""
|
||||
problem = path_budget_problem_for(path)
|
||||
if problem:
|
||||
fail(escape(f"Cannot write {problem}. {remedy}"))
|
||||
|
||||
|
||||
def check_collision(name: str, *, ignore: Path | None = None) -> None:
|
||||
"""Fail if a page under kb/ already has a title that collides with `name`.
|
||||
|
||||
|
||||
@@ -79,6 +79,10 @@ __all__ = [
|
||||
"areas to shard, reported with the split its subtype field would produce, and only "
|
||||
"when that split puts every resulting area at or under the threshold.",
|
||||
"Advisory only: source pages sitting in the `unclassified` catalog slot.",
|
||||
"Advisory only: Long Paths - a file under `kb/` or `raw/` whose path below the "
|
||||
"instance root is over 160 UTF-16 code units, the budget that keeps a Windows checkout "
|
||||
"without long paths working. Reported as `{path, length}`; a corpus over the budget breaks "
|
||||
"no lint run. `wikitool rename` is the fix for a page.",
|
||||
"Advisory only: quote-limit overages (>2 blockquotes/page).",
|
||||
"Prints only the sections that found something and always writes the full report to "
|
||||
"`reports/Lint Report <date>.md` (or `--markdown`), naming the path. `--full` prints "
|
||||
@@ -101,7 +105,7 @@ __all__ = [
|
||||
see_also=(
|
||||
"`wiki-lint` skill - the procedure that runs this",
|
||||
"`wikitool move --reconcile` - fixes Misplaced and Nested Pages",
|
||||
"`wikitool rename` - fixes Unportable Titles",
|
||||
"`wikitool rename` - fixes Unportable Titles and, for a page, Long Paths",
|
||||
"`wikitool log status` - whether a full lint is due",
|
||||
),
|
||||
))
|
||||
|
||||
@@ -32,6 +32,7 @@ from chemenu import cli_contract, config, tasks
|
||||
from chemenu.commands._util import (
|
||||
check_collision,
|
||||
check_raw_files_exist,
|
||||
check_path_budget,
|
||||
check_target_free,
|
||||
check_title,
|
||||
fail,
|
||||
@@ -381,6 +382,10 @@ def _ensure_tracker_project(page_title: str, *, resume: bool) -> Optional[str]:
|
||||
"well as Linux, whichever platform runs the command and whichever root the type writes "
|
||||
"to. The rule is `kb/CONTRACT.md` § Titles are identifiers; it is checked on the full "
|
||||
"title, after `title_prefix`.",
|
||||
"The target's path below the instance root may be at most 160 characters, counted in "
|
||||
"UTF-16 code units the way Windows counts MAX_PATH, so a Windows checkout without "
|
||||
"long paths keeps working. A longer one is refused, for every root, naming the length "
|
||||
"and how much shorter it has to get.",
|
||||
"`new` never overwrites: a file already at the target - or one a case-insensitive file "
|
||||
"system would treat as the same file - is refused for every root, `instructions/` "
|
||||
"included.",
|
||||
@@ -429,9 +434,10 @@ def _ensure_tracker_project(page_title: str, *, resume: bool) -> Optional[str]:
|
||||
cli_contract.Failure(
|
||||
cause="The title is not a valid file name (forbidden character, control "
|
||||
"character, reserved name such as `CON` or `Index`, trailing dot or space, empty), "
|
||||
"collides with another page by case or Unicode normalization, or the target file "
|
||||
"already exists",
|
||||
reaction="Not transient - choose another title and retry once. Nothing was created, "
|
||||
"collides with another page by case or Unicode normalization, the target file "
|
||||
"already exists, or the target path is over the 160-character path budget",
|
||||
reaction="Not transient - choose another (for the budget: a shorter) title and retry "
|
||||
"once. Nothing was created, "
|
||||
"and for `new project` no tracker project either",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
@@ -609,6 +615,7 @@ def new_page_command(
|
||||
check_raw_files_exist(frontmatter["raw_files"])
|
||||
|
||||
path = target_dir / f"{page_title}.md"
|
||||
check_path_budget(path, "Choose a shorter title.")
|
||||
check_target_free(path)
|
||||
body = _apply_template_variables(
|
||||
template,
|
||||
|
||||
@@ -28,9 +28,11 @@ import typer
|
||||
from chemenu import cli_contract, config, links
|
||||
from chemenu.commands._util import (
|
||||
check_collision,
|
||||
check_path_budget,
|
||||
check_target_free,
|
||||
check_title,
|
||||
fail,
|
||||
path_budget_problem_for,
|
||||
rel_path,
|
||||
success,
|
||||
target_conflict,
|
||||
@@ -240,9 +242,10 @@ def inbound_pages(pages: dict[str, Page], title: str) -> list[str]:
|
||||
"Each page's rewrite is idempotent, so a re-run as-is is safe. If a write fails "
|
||||
"midway, nothing is renamed on disk and the error lists what was updated.",
|
||||
"`--dry-run` lists every page it would change; run it first to see the blast radius.",
|
||||
"Only `--to` is checked against the title rule. A page whose current title breaks it "
|
||||
"(`lint`'s Unportable Titles) can always be renamed away from it, and a title that "
|
||||
"differs from the page's own only by case (`Foo` to `FOO`) is allowed.",
|
||||
"Only `--to` is checked against the title rule and the path budget (160 UTF-16 code "
|
||||
"units for the whole path below the instance root). A page whose current title breaks "
|
||||
"either (`lint`'s Unportable Titles and Long Paths) can always be renamed away from it, "
|
||||
"and a title that differs from the page's own only by case (`Foo` to `FOO`) is allowed.",
|
||||
),
|
||||
failures=(
|
||||
cli_contract.Failure(
|
||||
@@ -257,8 +260,10 @@ def inbound_pages(pages: dict[str, Page], title: str) -> list[str]:
|
||||
cli_contract.Failure(
|
||||
cause="The `--to` title is already taken - also by a page that differs only in case "
|
||||
"or Unicode normalization, or by a file in the page's directory - or is not a valid "
|
||||
"file name (see `kb/CONTRACT.md` § Titles are identifiers)",
|
||||
reaction="Choose another title and retry once. Checked under `--dry-run` too",
|
||||
"file name, or would put the page's path over the path budget (see `kb/CONTRACT.md` "
|
||||
"§ Titles are identifiers)",
|
||||
reaction="Choose another, or a shorter, title and retry once. Checked under "
|
||||
"`--dry-run` too",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="A page write failed partway; nothing was renamed on disk",
|
||||
@@ -317,6 +322,7 @@ def rename_command(
|
||||
# rename the real run refuses is worse than none.
|
||||
check_title(new)
|
||||
check_collision(new, ignore=target.path)
|
||||
check_path_budget(target.path.parent / f"{new}.md", "Choose a shorter --to title.")
|
||||
check_target_free(target.path.parent / f"{new}.md", ignore=target.path)
|
||||
|
||||
touched: list[str] = []
|
||||
@@ -538,6 +544,9 @@ def _rmdir_if_emptied(directory: Path) -> bool:
|
||||
"Moves a page to the directory its type-spec computes for its current frontmatter "
|
||||
"(`base_dir` + `layout`, the same rule `new` places a page by) - never to a hand-chosen "
|
||||
"destination; there is no `--to <dir>`.",
|
||||
"A destination whose path would be over the path budget (160 UTF-16 code units below "
|
||||
"the instance root) is refused, and `--reconcile` skips such a page and names it, as it "
|
||||
"does for an occupied destination - `wikitool rename` the page to a shorter title.",
|
||||
"`--reconcile` applies it corpus-wide: every misplaced page moves in one call, and a "
|
||||
"second run reports nothing left to do. It fixes `lint`'s `Misplaced Pages` (advisory) "
|
||||
"and `Nested Pages` (hard) findings.",
|
||||
@@ -563,8 +572,10 @@ def _rmdir_if_emptied(directory: Path) -> bool:
|
||||
cli_contract.Failure(
|
||||
cause="The destination already holds an entry with the same name, or one that "
|
||||
"differs only in case or Unicode normalization (a pre-existing duplicate-stem "
|
||||
"collision) - refused rather than silently skipped",
|
||||
reaction="Resolve the collision, then retry",
|
||||
"collision), or its path would be over the path budget - refused rather than "
|
||||
"silently skipped",
|
||||
reaction="Resolve the collision, or `wikitool rename` the page to a shorter title, "
|
||||
"then retry",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="`--reconcile` failed partway",
|
||||
@@ -617,6 +628,10 @@ def move_command(
|
||||
collisions: list[str] = []
|
||||
for title, page, target_dir in candidates:
|
||||
new_path = target_dir / f"{title}.md"
|
||||
over_budget = path_budget_problem_for(new_path)
|
||||
if over_budget is not None:
|
||||
collisions.append(f"{title} ({over_budget})")
|
||||
continue
|
||||
clash = target_conflict(new_path)
|
||||
if clash is not None:
|
||||
collisions.append(f"{title} (target {rel_path(clash)} already exists)")
|
||||
@@ -628,7 +643,7 @@ def move_command(
|
||||
typer.echo(f"[dry-run] would move {rel_path(page.path)} -> {rel_path(new_path)}")
|
||||
if collisions:
|
||||
typer.echo("")
|
||||
typer.echo("Skipped (target already exists) - resolve with `wikitool rename` first:")
|
||||
typer.echo("Skipped (target already exists, or its path is over the budget) - resolve with `wikitool rename` first:")
|
||||
for collision in collisions:
|
||||
typer.echo(f" - {collision}")
|
||||
typer.echo(f"[dry-run] would move {len(planned)} page(s). No files written.")
|
||||
@@ -677,6 +692,7 @@ def move_command(
|
||||
return
|
||||
|
||||
new_path = target_dir / f"{page_title}.md"
|
||||
check_path_budget(new_path, "Rename the page to a shorter title first (`wikitool rename`).")
|
||||
check_target_free(new_path)
|
||||
|
||||
if dry_run:
|
||||
|
||||
@@ -72,7 +72,7 @@ from typing import Optional
|
||||
import typer
|
||||
|
||||
from chemenu import cli_contract, config
|
||||
from chemenu.commands._util import fail, rel_path, success
|
||||
from chemenu.commands._util import check_path_budget, fail, rel_path, success
|
||||
from chemenu.frontmatter_io import write_page
|
||||
from chemenu.kb_scan import load_kb_pages
|
||||
from chemenu.provenance import citing_pages, source_pages_by_raw_file, source_raw_files
|
||||
@@ -371,9 +371,12 @@ def _replace(
|
||||
cli_contract.Failure(
|
||||
label="raw accept",
|
||||
cause="A file does not exist, is not under `incoming/`, or is nested more than one "
|
||||
"level below it; two files in one call share a filename; or a target path already "
|
||||
"exists",
|
||||
reaction="Fix the named argument and retry once",
|
||||
"level below it; two files in one call share a filename; a target path already "
|
||||
"exists; or a target path would be over the path budget (160 UTF-16 code units "
|
||||
"below the instance root)",
|
||||
reaction="Fix the named argument and retry once. For a path over the budget, "
|
||||
"rename the file in `incoming/` to something shorter - the refusal comes before "
|
||||
"anything moves, so `incoming/` and `raw/` are unchanged",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
label="raw accept",
|
||||
@@ -562,6 +565,11 @@ def raw_accept_command(
|
||||
moves.append((new_path, dst))
|
||||
|
||||
for _src, dst in moves:
|
||||
check_path_budget(
|
||||
dst,
|
||||
"The name comes from the file in incoming/: rename it there to something shorter "
|
||||
"and accept it again.",
|
||||
)
|
||||
if dst.exists():
|
||||
fail(f"Cannot promote: {rel_path(dst)} already exists.")
|
||||
|
||||
|
||||
@@ -13,6 +13,7 @@ of the line because the markdown report is a data product (it is what
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
|
||||
@@ -42,7 +43,7 @@ from chemenu.kb_scan import (
|
||||
iter_kb_pages,
|
||||
load_kb_pages,
|
||||
)
|
||||
from chemenu.titles import collision_key, title_problems
|
||||
from chemenu.titles import PATH_BUDGET, collision_key, path_length, title_problems
|
||||
from chemenu.type_resolver import resolver
|
||||
|
||||
# Style guide's one mechanically-checkable rule (hard oracle: a plain count).
|
||||
@@ -270,6 +271,29 @@ def unportable_titles(kb_dir: Path) -> list[dict]:
|
||||
return sorted(entries, key=lambda e: (e["path"], e["problem"]))
|
||||
|
||||
|
||||
def long_paths(kb_dir: Path, raw_dir: Path) -> list[dict]:
|
||||
"""Files under `kb/` and `raw/` whose path below the instance root is over
|
||||
`PATH_BUDGET`, as `{path, length}` (length in UTF-16 code units).
|
||||
|
||||
Advisory, absent from `HARD_ERROR_KEYS`: a corpus over the budget breaks no
|
||||
lint run and needs no migration, only a checkout on a system with long paths
|
||||
switched off - and `wikitool rename` is the remedy for a page. Hidden files
|
||||
are skipped, as `git` would have them only by accident.
|
||||
"""
|
||||
entries: list[dict] = []
|
||||
for root in (kb_dir, raw_dir):
|
||||
if not root.is_dir():
|
||||
continue
|
||||
for path in root.rglob("*"):
|
||||
if not path.is_file() or path.name.startswith("."):
|
||||
continue
|
||||
rel = _repo_relative(path, root).replace(os.sep, "/")
|
||||
length = path_length(rel)
|
||||
if length > PATH_BUDGET:
|
||||
entries.append({"path": rel, "length": length})
|
||||
return sorted(entries, key=lambda e: e["path"])
|
||||
|
||||
|
||||
def _repo_relative(path: Path, kb_dir: Path) -> str:
|
||||
try:
|
||||
return str(path.relative_to(config.ROOT))
|
||||
@@ -523,6 +547,7 @@ def run_lint(kb_dir: Path) -> dict:
|
||||
"title_mismatches": title_mismatches,
|
||||
"duplicate_titles": duplicate_titles,
|
||||
"unportable_titles": unportable_titles(kb_dir),
|
||||
"long_paths": long_paths(kb_dir, config.RAW_DIR),
|
||||
"misplaced_pages": misplaced,
|
||||
"nested_pages": nested,
|
||||
"unsharded_collections": unsharded_collections(kb_dir, pages),
|
||||
@@ -592,6 +617,13 @@ def render_markdown(report: dict) -> str:
|
||||
lambda i: f"[[{i['title']}]] at `{i['path']}` - {i['problem']}; "
|
||||
f"`wikitool rename --from \"{i['title']}\" --to \"<new title>\"` fixes it",
|
||||
)
|
||||
_section(
|
||||
lines, f"Long Paths (over {PATH_BUDGET} characters below the instance root) "
|
||||
"- advisory, not an error",
|
||||
report.get("long_paths", []),
|
||||
lambda i: f"`{i['path']}` is {i['length']} long - a Windows checkout without long paths "
|
||||
f"fails on it; `wikitool rename --from \"<title>\" --to \"<shorter title>\"` fixes a page",
|
||||
)
|
||||
_section(
|
||||
lines, "Filename / H1 Title Mismatches", report["title_mismatches"],
|
||||
lambda i: f"[[{i['page']}]] H1 is '{i['h1']}'",
|
||||
@@ -812,6 +844,11 @@ def default_report_path(report: dict) -> Path:
|
||||
# `MIGRATION_GATED_KEYS`: the rule needs no migration, so `kb_version` never
|
||||
# advances for it and a gate would keep the finding advisory forever.
|
||||
#
|
||||
# `long_paths` is advisory, like `misplaced_pages`: a corpus over the path budget
|
||||
# is not broken, only unfit for a checkout where Windows long paths are off, and
|
||||
# it needs no migration - so failing a lint run on it would penalise an instance
|
||||
# that never asked for that platform.
|
||||
#
|
||||
# One definition, used by `lint --fail-on-error` and by the eval scorecard: if
|
||||
# the two disagreed, a run could pass its score while lint refused it.
|
||||
HARD_ERROR_KEYS = (
|
||||
|
||||
@@ -951,3 +951,47 @@ def test_rendered_report_names_the_remedy(kb_dir):
|
||||
text = render_markdown(run_lint(kb_dir))
|
||||
assert "## Unportable Titles" in text
|
||||
assert 'wikitool rename --from "CON"' in text
|
||||
|
||||
|
||||
def _long_page(kb_dir, units):
|
||||
title = "t" * (units - len("kb/entities/tools/") - len(".md"))
|
||||
_plain_page(kb_dir / "entities/tools" / f"{title}.md")
|
||||
return f"kb/entities/tools/{title}.md"
|
||||
|
||||
|
||||
def test_lint_reports_a_path_over_the_budget_as_advisory(kb_dir, raw_dir):
|
||||
over = _long_page(kb_dir, 161)
|
||||
_long_page(kb_dir, 160)
|
||||
report = run_lint(kb_dir)
|
||||
assert report["long_paths"] == [{"path": over, "length": 161}]
|
||||
assert "long_paths" not in HARD_ERROR_KEYS
|
||||
|
||||
|
||||
def test_lint_reports_a_raw_path_over_the_budget(kb_dir, raw_dir):
|
||||
name = "r" * (161 - len("raw/notes/"))
|
||||
(raw_dir / "notes").mkdir(exist_ok=True)
|
||||
(raw_dir / "notes" / name).write_text("x", encoding="utf-8")
|
||||
report = run_lint(kb_dir)
|
||||
assert {"path": f"raw/notes/{name}", "length": 161} in report["long_paths"]
|
||||
|
||||
|
||||
def test_lint_fail_on_error_ignores_long_paths(kb_dir, raw_dir, tmp_path, monkeypatch):
|
||||
import typer
|
||||
|
||||
monkeypatch.setattr(config, "ROOT", tmp_path)
|
||||
monkeypatch.setattr(config, "KB_DIR", kb_dir)
|
||||
monkeypatch.setattr(config, "RAW_DIR", raw_dir)
|
||||
baseline = has_hard_errors(run_lint(kb_dir))
|
||||
_long_page(kb_dir, 170)
|
||||
report = run_lint(kb_dir)
|
||||
assert report["long_paths"]
|
||||
assert has_hard_errors(report) == baseline
|
||||
if not baseline:
|
||||
lint_command(json_out=True, markdown_out=None, full=False, fail_on_error=True)
|
||||
|
||||
|
||||
def test_rendered_report_names_rename_for_a_long_path(kb_dir, raw_dir):
|
||||
_long_page(kb_dir, 161)
|
||||
text = render_markdown(run_lint(kb_dir))
|
||||
assert "## Long Paths" in text
|
||||
assert "wikitool rename" in text
|
||||
@@ -1038,3 +1038,60 @@ def test_new_instruction_applies_the_title_rule_too(monkeypatch, tmp_path):
|
||||
result = _invoke_new_instruction(monkeypatch, tmp_path, ["new", "instruction", "--name", "a:b"])
|
||||
assert result.exit_code == 1
|
||||
assert list((tmp_path / "instructions").iterdir()) == []
|
||||
|
||||
|
||||
def _entity_title(units: int) -> str:
|
||||
"""A title whose `kb/entities/tools/<title>.md` path is `units` long."""
|
||||
return "t" * (units - len("kb/entities/tools/") - len(".md"))
|
||||
|
||||
|
||||
def test_new_accepts_a_path_at_the_budget_and_refuses_one_past_it(monkeypatch, kb_dir):
|
||||
before = _tree_snapshot(kb_dir)
|
||||
refused = _invoke_new(monkeypatch, kb_dir, [
|
||||
"new", "entity", "--name", _entity_title(161), "--set", "entity_type=tool",
|
||||
])
|
||||
assert refused.exit_code == 1, refused.output
|
||||
assert "161" in refused.output and "160" in refused.output
|
||||
assert _tree_snapshot(kb_dir) == before
|
||||
|
||||
accepted = _invoke_new(monkeypatch, kb_dir, [
|
||||
"new", "entity", "--name", _entity_title(160), "--set", "entity_type=tool",
|
||||
])
|
||||
assert accepted.exit_code == 0, accepted.output
|
||||
assert (kb_dir / "entities/tools" / f"{_entity_title(160)}.md").exists()
|
||||
|
||||
|
||||
def test_new_counts_an_emoji_as_two_units_at_the_budget(monkeypatch, kb_dir):
|
||||
fits = "😀" + _entity_title(160)[:-2]
|
||||
assert len(f"kb/entities/tools/{fits}.md") == 159
|
||||
over = fits + "t"
|
||||
refused = _invoke_new(monkeypatch, kb_dir, [
|
||||
"new", "entity", "--name", over, "--set", "entity_type=tool",
|
||||
])
|
||||
assert refused.exit_code == 1, refused.output
|
||||
assert not list(kb_dir.rglob(f"{over}.md"))
|
||||
accepted = _invoke_new(monkeypatch, kb_dir, [
|
||||
"new", "entity", "--name", fits, "--set", "entity_type=tool",
|
||||
])
|
||||
assert accepted.exit_code == 0, accepted.output
|
||||
|
||||
|
||||
def test_new_refuses_a_path_over_the_budget_for_a_repo_root_type(monkeypatch, tmp_path):
|
||||
name = "n" * (161 - len("instructions/") - len(".md"))
|
||||
result = _invoke_new_instruction(monkeypatch, tmp_path, ["new", "instruction", "--name", name])
|
||||
assert result.exit_code == 1, result.output
|
||||
assert "160" in result.output
|
||||
assert not list((tmp_path / "instructions").iterdir())
|
||||
|
||||
|
||||
def test_new_project_over_the_path_budget_never_reaches_the_tracker(monkeypatch, kb_dir):
|
||||
root = kb_dir.parent
|
||||
name = "p" * 200
|
||||
with _api_server([]) as (server, handler_cls):
|
||||
_write_tasks_config(root, _base_url(server))
|
||||
result = _invoke_new(monkeypatch, kb_dir, [
|
||||
"new", "project", "--name", name, "--set", "responsibility=haus",
|
||||
])
|
||||
assert result.exit_code == 1, result.output
|
||||
assert "160" in result.output
|
||||
assert handler_cls.posted is False
|
||||
@@ -470,3 +470,56 @@ def test_move_reconcile_skips_a_destination_that_differs_only_by_case(patched_wi
|
||||
page_ops.move_command(page_title=None, reconcile=True, dry_run=False)
|
||||
assert (patched_wiki / "entities/zzz-wrong/dup.md").exists()
|
||||
assert (patched_wiki / "entities/tools/Dup.md").exists()
|
||||
|
||||
|
||||
def _tools_title(units: int) -> str:
|
||||
"""A title whose `kb/entities/tools/<title>.md` path is `units` long."""
|
||||
return "t" * (units - len("kb/entities/tools/") - len(".md"))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("dry_run", [False, True])
|
||||
def test_rename_refuses_a_target_over_the_path_budget(patched_wiki, dry_run):
|
||||
_page_at(patched_wiki, "entities/tools", "Short")
|
||||
before = {p: p.read_bytes() for p in patched_wiki.rglob("*.md")}
|
||||
with pytest.raises(typer.Exit):
|
||||
page_ops.rename_command(old="Short", new=_tools_title(161), dry_run=dry_run)
|
||||
assert {p: p.read_bytes() for p in patched_wiki.rglob("*.md")} == before
|
||||
|
||||
|
||||
def test_rename_accepts_a_target_at_the_path_budget(patched_wiki):
|
||||
_page_at(patched_wiki, "entities/tools", "Short")
|
||||
page_ops.rename_command(old="Short", new=_tools_title(160), dry_run=False)
|
||||
assert (patched_wiki / "entities/tools" / f"{_tools_title(160)}.md").exists()
|
||||
|
||||
|
||||
def test_rename_away_from_a_page_over_the_budget_is_allowed(patched_wiki):
|
||||
"""`--from` is never checked: the rename is the remedy for a long page."""
|
||||
_page_at(patched_wiki, "entities/tools", _tools_title(170))
|
||||
page_ops.rename_command(old=_tools_title(170), new="Short", dry_run=False)
|
||||
assert (patched_wiki / "entities/tools/Short.md").exists()
|
||||
|
||||
|
||||
def test_move_refuses_a_destination_over_the_path_budget(patched_wiki):
|
||||
# kb/entities/systems/<t>.md is 2 longer than kb/entities/tools/<t>.md, so a
|
||||
# title that fits in tools/ is over the budget once the page moves to systems/.
|
||||
title = "t" * (160 - len("kb/entities/tools/") - len(".md"))
|
||||
_write_misplaced(patched_wiki, "entities/tools", title, "system")
|
||||
with pytest.raises(typer.Exit):
|
||||
page_ops.move_command(page_title=title, reconcile=False, dry_run=False)
|
||||
assert (patched_wiki / "entities/tools" / f"{title}.md").exists()
|
||||
assert not (patched_wiki / "entities/systems" / f"{title}.md").exists()
|
||||
|
||||
|
||||
def test_move_reconcile_skips_a_destination_over_the_budget_and_names_it(patched_wiki, capsys):
|
||||
title = "t" * (160 - len("kb/entities/tools/") - len(".md"))
|
||||
_write_misplaced(patched_wiki, "entities/tools", title, "system")
|
||||
_write_misplaced(patched_wiki, "entities/tools", "fine-system", "system")
|
||||
page_ops.move_command(page_title=None, reconcile=True, dry_run=True)
|
||||
out = capsys.readouterr().out
|
||||
assert "fine-system" in out and title in out and "characters long" in out
|
||||
assert "would move 1 page(s)" in out
|
||||
|
||||
with pytest.raises(typer.Exit):
|
||||
page_ops.move_command(page_title=None, reconcile=True, dry_run=False)
|
||||
assert (patched_wiki / "entities/systems/fine-system.md").exists()
|
||||
assert (patched_wiki / "entities/tools" / f"{title}.md").exists()
|
||||
@@ -627,3 +627,30 @@ def test_replaces_rejects_combination_with_page(tree):
|
||||
|
||||
assert target.read_text(encoding="utf-8") == "old"
|
||||
assert new.exists()
|
||||
|
||||
|
||||
def _incoming_name_for(units: int) -> str:
|
||||
"""A file name whose accepted path `raw/YYYY/MM/<name>` is `units` long."""
|
||||
return "f" * (units - len(_shard()) - 1 - len(".md")) + ".md"
|
||||
|
||||
|
||||
def _tree_files(root):
|
||||
return {str(p.relative_to(root)): p.read_bytes() for p in sorted(root.rglob("*")) if p.is_file()}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("dry_run", [False, True])
|
||||
def test_accept_refuses_a_target_over_the_path_budget(tree, dry_run):
|
||||
src = tree / "incoming" / _incoming_name_for(161)
|
||||
src.write_text("x", encoding="utf-8")
|
||||
before = _tree_files(tree)
|
||||
with pytest.raises(typer.Exit):
|
||||
_accept(src, dry_run=dry_run)
|
||||
assert _tree_files(tree) == before
|
||||
|
||||
|
||||
def test_accept_takes_a_target_at_the_path_budget(tree):
|
||||
src = tree / "incoming" / _incoming_name_for(160)
|
||||
src.write_text("x", encoding="utf-8")
|
||||
_accept(src)
|
||||
assert (tree / _shard() / src.name).exists()
|
||||
assert not src.exists()
|
||||
@@ -69,3 +69,23 @@ def test_collision_key_folds_case_and_normalization():
|
||||
assert collision_key("Foo") == collision_key("foo") == collision_key("FOO")
|
||||
assert collision_key("Straße") == collision_key("STRASSE")
|
||||
assert collision_key("Foo") != collision_key("Foo ")
|
||||
|
||||
|
||||
def test_path_budget_counts_utf16_code_units():
|
||||
from chemenu.titles import PATH_BUDGET, path_budget_problem, path_length
|
||||
|
||||
assert PATH_BUDGET == 160
|
||||
assert path_length("abc") == 3
|
||||
assert path_length("😀") == 2
|
||||
assert path_length("é") == 1
|
||||
assert path_budget_problem("a" * 160) is None
|
||||
over = path_budget_problem("a" * 161)
|
||||
assert over is not None and "161" in over and "160" in over and "1 shorter" in over
|
||||
|
||||
|
||||
def test_path_budget_counts_an_emoji_twice_at_the_boundary():
|
||||
from chemenu.titles import path_budget_problem
|
||||
|
||||
# 159 characters, 160 code units: fits. One more ASCII character tips it.
|
||||
assert path_budget_problem("a" * 158 + "😀") is None
|
||||
assert path_budget_problem("a" * 159 + "😀") is not None
|
||||
@@ -12,6 +12,12 @@ import unicodedata
|
||||
|
||||
FORBIDDEN_CHARS = frozenset('<>:"/\\|?*')
|
||||
|
||||
# Windows MAX_PATH is 259 usable characters for the whole path, install folder
|
||||
# included, and long paths are off on the target system. A file's path below the
|
||||
# instance root may use 160 of them; the folder limit that `doctor` and preflight
|
||||
# enforce is the other half of the same sum, so change one only together with it.
|
||||
PATH_BUDGET = 160
|
||||
|
||||
# Windows device names, matched on the part before the first dot. The superscript
|
||||
# forms are reserved by some Windows versions and not others; refusing all of
|
||||
# them costs nothing.
|
||||
@@ -61,3 +67,22 @@ def title_problems(title: str) -> list[str]:
|
||||
problems.append("ends with a dot or a space, which Windows strips from file names")
|
||||
|
||||
return problems
|
||||
|
||||
|
||||
def path_length(rel_path: str) -> int:
|
||||
"""`rel_path`'s length in UTF-16 code units, which is how Windows counts
|
||||
MAX_PATH - a character outside the BMP (an emoji) counts twice."""
|
||||
return len(rel_path.encode("utf-16-le")) // 2
|
||||
|
||||
|
||||
def path_budget_problem(rel_path: str) -> str | None:
|
||||
"""Why the path of a file, relative to the instance root and written with
|
||||
`/`, is over `PATH_BUDGET`; None when it fits."""
|
||||
length = path_length(rel_path)
|
||||
if length <= PATH_BUDGET:
|
||||
return None
|
||||
excess = length - PATH_BUDGET
|
||||
return (
|
||||
f"{rel_path} is {length} characters long (counted in UTF-16 code units, as Windows "
|
||||
f"counts MAX_PATH) and the budget is {PATH_BUDGET}: the path has to get {excess} shorter"
|
||||
)
|
||||
Reference in new issue
Block a user