feat: export guidelines - the guideline pages as a generated GUIDELINES.md, pushed into the captured repositories behind the Guideline Push Gate (#179)
Files changed: - AGENTS.md - CHANGES.md - README.md - VERSION - docs/why-gates-are-code.md - instructions/gates.md - instructions/kb-profiles.md - kb/CONTRACT.md - kb/CONVENTIONS.md - kb/CONVENTIONS.md.template - raw/CONTRACT.md - tools/CONTRACT.md - tools/README.md - tools/chemenu/cli.py - tools/chemenu/cli_contract.py - tools/chemenu/commands/export_cmd.py - tools/chemenu/commands/page_ops.py - tools/chemenu/commands/raw_cmd.py - tools/chemenu/commands/search.py - tools/chemenu/guideline_export.py - tools/chemenu/kb_scan.py - tools/chemenu/repo_capture.py - tools/chemenu/search/filters.py - tools/chemenu/tests/test_cli.py - tools/chemenu/tests/test_export_guidelines.py Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
torbenandClaude Opus 5.5 committed
1 parent
ccede04b97
commit
283cdae8be
25 files changed
+1753
-68
No files matched your search
@@ -37,6 +37,7 @@ file end to end is for changing the CLI itself.
|
||||
- [Provenance](#provenance)
|
||||
- [Raw material and uploads](#raw-material-and-uploads)
|
||||
- [Git](#git)
|
||||
- [Guideline export](#guideline-export)
|
||||
- [Workshop runs and session budget](#workshop-runs-and-session-budget)
|
||||
- [Types, instructions and docs](#types-instructions-and-docs)
|
||||
- [Telemetry](#telemetry)
|
||||
@@ -125,6 +126,7 @@ upload accept write non-idempotent budget:counted exit:0,1,
|
||||
upload reject write non-idempotent budget:counted exit:0,1 Delete a submission's material, keeping only its ledger trail.
|
||||
sync write idempotent budget:counted exit:0,1,42 Fetch `<remote>/<branch>` and bring the local branch up to date with it.
|
||||
publish write non-idempotent budget:counted exit:0,1,42 Reconcile with `<remote>/<branch>`, then stage all changes, commit, and push.
|
||||
export guidelines write idempotent budget:counted exit:0,1,42 Render the guideline pages into one generated `GUIDELINES.md`, or push it into the captured repositories that opted in (**Guideline Push Gate**).
|
||||
work new write non-idempotent budget:counted exit:0,1 Scaffold `work/<runkey>/` for one workshop run.
|
||||
work close write non-idempotent budget:counted exit:0,1 Delete a finished workshop.
|
||||
budget status read idempotent budget:exempt exit:0 Show the current session's `wikitool` call count and recent command history.
|
||||
@@ -1643,6 +1645,7 @@ Promote one or more files, or one folder, from `incoming/` into `raw/`.
|
||||
- 0 success
|
||||
- 1 raw accept: A file does not exist or is not directly in `incoming/`; two files in one call share a filename; a target path already exists; or a target path would be over the path budget (160 UTF-16 code units below the instance root)
|
||||
- 1 raw accept incoming/<folder>: The folder is not directly in `incoming/`, is combined with another argument, `--page` or `--replaces`, holds no file, or holds a hidden entry, a symlink or a special file; a target path is over the budget; or the folder name is already occupied under `raw/`
|
||||
- 1 A file - given directly, inside a folder, or as `--replaces`/`--replaces-bundle` material - is a guideline export: its first line, after an optional BOM, starts with `<!-- wikitool:export`
|
||||
- 1 raw accept: `--fidelity`/`--authority` is missing, or names `unknown` or a value outside the schema's enum
|
||||
- 1 raw accept: The target name is already occupied anywhere under `raw/` by something the call does not own
|
||||
- 1 raw accept: `--page` names an unknown page or one with no `raw_files:` yet, an existing `raw_files:` entry is missing on disk, a file to be moved has more than one owning page, or `--page` would overwrite an already-set `fidelity`/`authority` with a different value
|
||||
@@ -1657,6 +1660,7 @@ Promote one or more files, or one folder, from `incoming/` into `raw/`.
|
||||
|
||||
- raw accept: A file does not exist or is not directly in `incoming/`; two files in one call share a filename; a target path already exists; or a target path would be over the path budget (160 UTF-16 code units below the instance root) -> Fix the named argument and retry once. A file inside a subdirectory of `incoming/` is accepted with its whole folder (`raw accept incoming/<folder>`) or moved up into `incoming/` first. For a path over the budget, rename the file in `incoming/` to something shorter - the refusal comes before anything moves, so `incoming/` and `raw/` are unchanged
|
||||
- raw accept incoming/<folder>: The folder is not directly in `incoming/`, is combined with another argument, `--page` or `--replaces`, holds no file, or holds a hidden entry, a symlink or a special file; a target path is over the budget; or the folder name is already occupied under `raw/` -> Nothing moved. Fix what the message names and retry once. For an occupied name held by a captured bundle, the folder is its new edition: `raw accept incoming/<bundle> --replaces-bundle <raw-bundle>`. Any other occupied name: rename the folder in `incoming/` - such a folder has no replacement form
|
||||
- A file - given directly, inside a folder, or as `--replaces`/`--replaces-bundle` material - is a guideline export: its first line, after an optional BOM, starts with `<!-- wikitool:export` -> Not fixed by retrying: it is generated from a wiki's `kb/` and never goes back into `raw/`. Remove it from `incoming/`; nothing was moved
|
||||
- raw accept: `--fidelity`/`--authority` is missing, or names `unknown` or a value outside the schema's enum -> Pass both with a valid value, then retry once
|
||||
- raw accept: The target name is already occupied anywhere under `raw/` by something the call does not own -> Not fixed by retrying: the refusal names `--replaces` (same source, new edition) and renaming in `incoming/` (a separate source) as the two routes, and neither is the tool's to pick. Show the message to the user and wait
|
||||
- raw accept: `--page` names an unknown page or one with no `raw_files:` yet, an existing `raw_files:` entry is missing on disk, a file to be moved has more than one owning page, or `--page` would overwrite an already-set `fidelity`/`authority` with a different value -> Fix the named argument and retry once; a different capture value on an existing page is a new edition - `--replaces`
|
||||
@@ -2002,6 +2006,74 @@ Reconcile with `<remote>/<branch>`, then stage all changes, commit, and push.
|
||||
- `instructions/gates.md` - the gate procedure
|
||||
- `instructions/setup-instance.md` - the first publish of a new instance
|
||||
|
||||
### Guideline export
|
||||
|
||||
#### `export guidelines`
|
||||
|
||||
Render the guideline pages into one generated `GUIDELINES.md`, or push it into the captured repositories that opted in (**Guideline Push Gate**).
|
||||
|
||||
**SYNOPSIS**
|
||||
|
||||
- `wikitool export guidelines [--field <predicate> ...] [--kind/--subtype/--collection/--tag <v>]` - Prints the file to stdout; no network.
|
||||
- `wikitool export guidelines <predicates> --push [--bundle <raw-bundle> ...] [--confirm TOKEN]` - Writes it into the captured repositories, behind the gate.
|
||||
|
||||
**PROPERTIES**
|
||||
|
||||
- effect: write
|
||||
- idempotent: yes
|
||||
- atomic: Per target - one commit pushed without force, or nothing; targets already written stay written when a later one fails
|
||||
- budget: counted
|
||||
- network: yes
|
||||
- gates: guideline-push
|
||||
|
||||
**EXAMPLES**
|
||||
|
||||
- `tools/wikitool export guidelines --tag guideline`
|
||||
- `tools/wikitool export guidelines --tag guideline --push`
|
||||
- `tools/wikitool export guidelines --confirm <token> --push --field tags=guideline # re-run after exit 42, once the user approved every diff`
|
||||
|
||||
**EXIT STATUS**
|
||||
|
||||
- 0 success
|
||||
- 0 A target was skipped - tag rule, not reachable, not opted in, hand-written file, another instance's file
|
||||
- 1 No predicate, no page matches, a malformed predicate or an unknown field
|
||||
- 1 A page under `kb/` has unreadable frontmatter, `kb/` or `types/` has uncommitted changes, or the instance is not a git checkout
|
||||
- 1 `--push` with no `user.name` or `user.email` in the instance checkout's git configuration - checked before any target is fetched
|
||||
- 1 `--bundle` names no captured bundle under `raw/`, or `--bundle`/`--confirm` was given without `--push`
|
||||
- 1 A target rejected the push (its branch moved since the fetch) or a git call failed - the other targets were still served
|
||||
- 42 Guideline Push Gate: `--push` without `--confirm`, or with a token that does not match the current set of targets
|
||||
|
||||
**ON FAILURE**
|
||||
|
||||
- No predicate, no page matches, a malformed predicate or an unknown field -> Use the filter `kb/CONVENTIONS.md` names for this instance's guidelines, and retry once
|
||||
- A page under `kb/` has unreadable frontmatter, `kb/` or `types/` has uncommitted changes, or the instance is not a git checkout -> Fix the named pages, or publish the changes first, then retry once
|
||||
- `--push` with no `user.name` or `user.email` in the instance checkout's git configuration - checked before any target is fetched -> Show the message to the user; the identity is theirs to set, never yours
|
||||
- `--bundle` names no captured bundle under `raw/`, or `--bundle`/`--confirm` was given without `--push` -> Fix the argument and retry once
|
||||
- A target rejected the push (its branch moved since the fetch) or a git call failed - the other targets were still served -> Show the lines to the user. A rejected target is served by running the whole command again, which asks for clearance again
|
||||
- Guideline Push Gate: `--push` without `--confirm`, or with a token that does not match the current set of targets -> Show the user the command's full output verbatim and stop. Once they have approved it, run the re-run line the output prints, which carries `--confirm <token>`. Without that token, or with a wrong, invented or superseded one, it exits 42 again with the current state
|
||||
|
||||
**NEVER**
|
||||
|
||||
- Never pass a `--confirm` token the user has not seen and approved.
|
||||
- Never edit a `GUIDELINES.md` in a target repository by hand to make it match - the next export overwrites it; change the guideline page instead.
|
||||
|
||||
**NOTES**
|
||||
|
||||
- Selects pages with `search`'s own predicates and no text: `--field` (repeatable, AND), `--kind`, `--subtype`, `--collection`, `--tag`. Which filter selects this instance's guidelines is written down in `kb/CONVENTIONS.md`.
|
||||
- Renders mechanically: line 1 is `<!-- wikitool:export kind=guidelines instance=<origin URL without userinfo, or local> commit=<newest commit touching a selected page> - generated, do not edit by hand -->`; then each page sorted by title, as `# <title>` and its text without frontmatter, the generated links and footnote blocks, citation markers (`[^id]`, legacy `^[[...]]`), with `[[Title|Text]]` as `Text` and `[[Title]]` as `Title`. A leading H1 of the page is replaced by the title heading. Code is never rewritten. LF endings, no timestamp: the same pages give the same bytes.
|
||||
- Refuses before printing or pushing anything: no predicate, no match, an unknown field, any page under `kb/` with unreadable frontmatter, uncommitted changes in `kb/` or `types/` (untracked files included), or no git checkout.
|
||||
- `--push` targets every captured repository - each `(repo, ref)` from a `_capture.json` under `raw/`, `--bundle` narrowing it. A target is skipped with its reason when its ref rule is a tag pattern, it cannot be reached or has no such branch, its root has no `GUIDELINES.md` (not opted in), its `GUIDELINES.md` has no export header (hand-written), or the header names another `instance=`. A header line alone is the opt-in.
|
||||
- Guideline Push Gate: `--push` without a matching `--confirm` fetches and builds everything, pushes nothing, and exits 42 with each target's status (`new`, `changed`, `unchanged`, `skipped`), the diff for every target it would write, and the re-run line. The token digests URL, branch, old tip and new blob of every target to write.
|
||||
- With a matching token, each target gets exactly one commit on the fetched tip changing only `GUIDELINES.md`, authored as the instance checkout's `user.name`/`user.email`, pushed without force; one line per target: `unchanged`, `written <old> -> <new>`, `skipped (<reason>)`, `rejected (<reason>)` or `failed (<reason>)`.
|
||||
- Nothing to write (every target unchanged or skipped) ends with exit 0 and no gate. A second run after a successful one finds every target unchanged.
|
||||
- The target repository's own `AGENTS.md` points at the file (Claude Code: `@GUIDELINES.md`); that belongs to the repository, not to this command.
|
||||
|
||||
**SEE ALSO**
|
||||
|
||||
- `wikitool search` - the same predicates, to preview the selection
|
||||
- `wikitool raw capture` - how a repository becomes a target
|
||||
- `instructions/gates.md` - the gate procedure
|
||||
|
||||
### Workshop runs and session budget
|
||||
|
||||
#### `work new`
|
||||
|
||||
+5
-4
@@ -106,7 +106,8 @@ tools/
|
||||
kb_state.py the KB version (.wikitool-kb.json) and the migration chain
|
||||
corpus_diff.py invariant comparison of kb/ between two revisions
|
||||
web_capture.py `raw fetch`'s core: fetch a page, decide its charset, derive Markdown-like text from the HTML - standard library only, deterministic
|
||||
repo_capture.py `raw capture`/`raw status`'s core: resolve a ref rule, fetch it into a bare cache, select files by glob as blobs, the `_capture.json` manifest - git with the host's credentials, never a prompt
|
||||
repo_capture.py `raw capture`/`raw status`'s core: resolve a ref rule, fetch it into a bare cache, select files by glob as blobs, the `_capture.json` manifest - git with the host's credentials, never a prompt; plus the one write back, a single-file commit pushed without force for `export guidelines`
|
||||
guideline_export.py `export guidelines`' core: select pages by `search` predicates, render them into one deterministic GUIDELINES.md, and recognise an export again (`EXPORT_MARKER`) - with no CLI attached
|
||||
search/ pluggable search backends, plus service.py - the search core
|
||||
tasks/ the task-tracker provider layer: protocol.py (TaskReader/TaskWriter), config.py (.wikitool-tasks.json), one module per adapter - no instruction ever learns which provider it is
|
||||
commands/ one module per command or command group: the terminal adapters
|
||||
@@ -132,7 +133,7 @@ interpreter the way the preflight does - `python3`, `python` on `PATH`; on Windo
|
||||
and says why.
|
||||
|
||||
**Two consumers, one core.** The CLI is not the only caller any more. The cores
|
||||
(`search/service.py`, `lint_core.py`, `types_core.py`, `catalog.py`) hold what
|
||||
(`search/service.py`, `lint_core.py`, `types_core.py`, `catalog.py`, `guideline_export.py`) hold what
|
||||
decides an answer and import no `typer` and no `rich`; the modules under `commands/` turn
|
||||
those values into terminal output and those exceptions into exit codes.
|
||||
`api.Corpus` is the in-process entry point over the same functions - it takes a
|
||||
@@ -201,8 +202,8 @@ not from a list of its own. Only read-only retrieval earns it.
|
||||
frontmatter, index statistics, cross-reference bookkeeping, log formatting,
|
||||
version arithmetic - is computed here so it comes out the same every time.
|
||||
|
||||
**Gates are code, not prompts.** The Mass-Update Gate (`git_publish.py`) and the
|
||||
Iteration Budget Gate (`run_budget.py`) refuse in-process, because a
|
||||
**Gates are code, not prompts.** The Mass-Update Gate (`git_publish.py`), the Guideline Push
|
||||
Gate (`export_cmd.py`) and the Iteration Budget Gate (`run_budget.py`) refuse in-process, because a
|
||||
prompt-level limit is one an agent can talk itself past. Exemption lists are
|
||||
constants, never flags.
|
||||
|
||||
|
||||
@@ -31,6 +31,7 @@ try:
|
||||
doctor,
|
||||
docs_verify,
|
||||
eval_cmd,
|
||||
export_cmd,
|
||||
git_publish,
|
||||
index_build,
|
||||
instructions_cmd,
|
||||
@@ -178,6 +179,7 @@ app.add_typer(dist_cmd.app, name="dist")
|
||||
app.add_typer(version_cmd.app, name="version")
|
||||
app.add_typer(migrate_cmd.app, name="migrate")
|
||||
app.add_typer(task_cmd.app, name="task")
|
||||
app.add_typer(export_cmd.app, name="export")
|
||||
app.command("new")(new_page.new_page_command)
|
||||
app.command("touch")(touch_module.touch_command)
|
||||
app.command("rename")(page_ops.rename_command)
|
||||
|
||||
@@ -265,6 +265,9 @@ GROUPS: tuple[tuple[str, tuple[str, ...]], ...] = (
|
||||
("Git", (
|
||||
"sync", "publish",
|
||||
)),
|
||||
("Guideline export", (
|
||||
"export guidelines",
|
||||
)),
|
||||
("Workshop runs and session budget", (
|
||||
"work new", "work close", "budget status", "budget reset",
|
||||
)),
|
||||
|
||||
@@ -0,0 +1,429 @@
|
||||
"""`wikitool export guidelines` - the instance's guideline pages as one
|
||||
generated `GUIDELINES.md`, printed, or written into the captured repositories
|
||||
behind the Guideline Push Gate.
|
||||
|
||||
The core - selection, rendering, the header - is `chemenu/guideline_export.py`;
|
||||
the git write is `repo_capture.commit_file`/`push_commit`. This module decides
|
||||
which captured repositories are targets, which of them take part, and when the
|
||||
gate opens.
|
||||
|
||||
**Targets are the captured repositories, and nothing else.** The `repo` and
|
||||
`ref` of every `_capture.json` under `raw/` - the one declaration that this
|
||||
instance follows a repository, now read in both directions. A second list of
|
||||
targets would be a second declaration of the same thing to drift.
|
||||
|
||||
**A repository takes part by carrying the file.** Only a branch whose root
|
||||
`GUIDELINES.md` already starts with the guideline export header is written to;
|
||||
a hand-written `GUIDELINES.md` is never replaced, and one whose header names a
|
||||
different `instance=` belongs to that instance. A bare header line is the
|
||||
opt-in.
|
||||
|
||||
**Why a gate, and why this shape.** A push into another repository is the most
|
||||
irreversible thing the stack does, and a public target publishes the content -
|
||||
the `instance=` URL included - the moment it lands. So it is held to the same
|
||||
token form as the Mass-Update and Upload Review gates: the first run fetches,
|
||||
checks and builds everything, pushes nothing and exits 42 with every target's
|
||||
status and diff; the token digests exactly that set (URL, branch, old tip, new
|
||||
blob), so a moved branch, a changed guideline or a new target asks again. The
|
||||
Publish-Remote Gate is deliberately not applied: the targets are declared by
|
||||
committed manifests already, and listing the same URLs in
|
||||
`.wikitool-remotes.json` would be the second declaration again.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import difflib
|
||||
import hashlib
|
||||
import json
|
||||
import shlex
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
import typer
|
||||
from rich.markup import escape
|
||||
|
||||
from chemenu import cli_contract, config, guideline_export, repo_capture
|
||||
from chemenu.commands._util import EXIT_NEEDS_CLEARANCE, console, fail, rel_path
|
||||
from chemenu.errors import BackendError, ValidationError
|
||||
from chemenu.search import filters
|
||||
|
||||
app = typer.Typer(help="Generate files from the wiki for other repositories.")
|
||||
|
||||
GATE = "guideline-push"
|
||||
|
||||
|
||||
@dataclass
|
||||
class Target:
|
||||
"""One `(repo, ref)` pair - several bundles captured from one branch are
|
||||
one target. `error` is set when a manifest could not be read."""
|
||||
repo: str
|
||||
ref: str
|
||||
bundles: list[str] = field(default_factory=list)
|
||||
error: str = ""
|
||||
|
||||
@property
|
||||
def label(self) -> str:
|
||||
return f"{self.repo} {self.ref}"
|
||||
|
||||
|
||||
@dataclass
|
||||
class Plan:
|
||||
target: Target
|
||||
status: str # new | changed | unchanged | skipped
|
||||
reason: str = ""
|
||||
tip: str = ""
|
||||
blob: str = ""
|
||||
old: bytes = b""
|
||||
|
||||
@property
|
||||
def writes(self) -> bool:
|
||||
return self.status in ("new", "changed")
|
||||
|
||||
|
||||
def _targets(only: list[Path]) -> list[Target]:
|
||||
manifests = repo_capture.captured_manifests(config.RAW_DIR)
|
||||
if only:
|
||||
wanted = {p.resolve() for p in only}
|
||||
known = {m.parent.resolve() for m in manifests}
|
||||
unknown = sorted(rel_path(p) for p in only if p.resolve() not in known)
|
||||
if unknown:
|
||||
raise ValidationError(
|
||||
f"--bundle {', '.join(unknown)}: not a captured bundle under raw/ (a directory "
|
||||
f"holding {repo_capture.MANIFEST_NAME})."
|
||||
)
|
||||
manifests = [m for m in manifests if m.parent.resolve() in wanted]
|
||||
targets: dict[tuple[str, str], Target] = {}
|
||||
broken: list[Target] = []
|
||||
for path in manifests:
|
||||
bundle = rel_path(path.parent)
|
||||
try:
|
||||
manifest = repo_capture.read_manifest(path)
|
||||
except ValidationError as exc:
|
||||
broken.append(Target(bundle, "?", [bundle], f"manifest unreadable: {exc}"))
|
||||
continue
|
||||
target = targets.setdefault((manifest.repo, manifest.ref), Target(manifest.repo, manifest.ref))
|
||||
target.bundles.append(bundle)
|
||||
return [*targets.values(), *broken]
|
||||
|
||||
|
||||
def _plan(target: Target, export: guideline_export.Export) -> Plan:
|
||||
"""Where this target stands - fetched, never written to."""
|
||||
if target.error:
|
||||
return Plan(target, "skipped", target.error)
|
||||
if repo_capture.is_tag_pattern(target.ref):
|
||||
return Plan(target, "skipped", "captured by a tag pattern - a tag takes no commit")
|
||||
try:
|
||||
with repo_capture.cache_repo(target.repo) as repo:
|
||||
resolved = repo_capture.resolve_ref(target.repo, target.ref, repo)
|
||||
tip = repo_capture.fetch(target.repo, resolved.refname, repo)
|
||||
existing = repo_capture.read_path(repo, tip, guideline_export.TARGET_PATH)
|
||||
blob = repo_capture.blob_id(repo, export.data)
|
||||
except BackendError as exc:
|
||||
return Plan(target, "skipped", f"not reachable: {exc}")
|
||||
except ValidationError as exc: # the branch does not exist
|
||||
return Plan(target, "skipped", str(exc))
|
||||
if existing is None:
|
||||
return Plan(target, "skipped", f"no {guideline_export.TARGET_PATH} - not opted in", tip)
|
||||
fields = guideline_export.guidelines_header(existing)
|
||||
if fields is None:
|
||||
return Plan(
|
||||
target, "skipped",
|
||||
f"{guideline_export.TARGET_PATH} has no export header - hand-written, never replaced", tip,
|
||||
)
|
||||
owner = fields.get("instance")
|
||||
if owner is not None and owner != export.instance:
|
||||
return Plan(target, "skipped", f"delivered by another instance ({owner})", tip)
|
||||
if existing == export.data:
|
||||
return Plan(target, "unchanged", tip=tip)
|
||||
return Plan(target, "changed" if owner else "new", tip=tip, blob=blob, old=existing)
|
||||
|
||||
|
||||
def confirm_token(plans: list[Plan]) -> str:
|
||||
"""sha256 over every target the run would write - URL, branch, the tip it
|
||||
builds on and the blob it writes - cut to 12 hex chars, the shape of
|
||||
`git_publish.changeset_token`. The new commit's own id is not part of it:
|
||||
`commit-tree` stamps the time, so it differs on every run."""
|
||||
payload = json.dumps(
|
||||
sorted([p.target.repo, p.target.ref, p.tip, p.blob] for p in plans if p.writes),
|
||||
sort_keys=True,
|
||||
)
|
||||
return hashlib.sha256(payload.encode("utf-8")).hexdigest()[:12]
|
||||
|
||||
|
||||
def _rerun(token: str, predicates: list[str], bundles: list[Path]) -> str:
|
||||
parts = ["tools/wikitool", "export", "guidelines", "--confirm", token, "--push"]
|
||||
for raw in predicates:
|
||||
parts += ["--field", raw]
|
||||
for bundle in bundles:
|
||||
parts += ["--bundle", rel_path(bundle)]
|
||||
return " ".join(shlex.quote(part) for part in parts)
|
||||
|
||||
|
||||
def _diff(plan: Plan, export: guideline_export.Export) -> str:
|
||||
old = plan.old.decode("utf-8", "replace").splitlines(keepends=True)
|
||||
new = export.text.splitlines(keepends=True)
|
||||
name = guideline_export.TARGET_PATH
|
||||
return "".join(difflib.unified_diff(old, new, f"a/{name}", f"b/{name}"))
|
||||
|
||||
|
||||
def _status_line(plan: Plan) -> str:
|
||||
detail = f" ({plan.reason})" if plan.reason else ""
|
||||
return f" {plan.target.label}: {plan.status}{detail}"
|
||||
|
||||
|
||||
def _clearance(plans: list[Plan], export, token: str, rerun: str, stale: Optional[str]) -> str:
|
||||
writes = [p for p in plans if p.writes]
|
||||
lines = [
|
||||
f"This run would push one commit changing only {guideline_export.TARGET_PATH} into "
|
||||
f"{len(writes)} repositor{'y' if len(writes) == 1 else 'ies'}. A push there is visible at "
|
||||
"once - publicly, for a public repository - and not cheaply reversible. Nothing was pushed.",
|
||||
"",
|
||||
]
|
||||
if stale:
|
||||
lines += [
|
||||
f"The token you passed ({stale}) does not match this set - a branch moved, a guideline "
|
||||
"changed, a target came or went, or the token was invented. Here is the current state.",
|
||||
"",
|
||||
]
|
||||
lines += [
|
||||
"THE USER CANNOT SEE THIS OUTPUT. It went to your context, not to their screen.",
|
||||
"Reproduce every target line and every diff below in your reply, and stop there. Run no "
|
||||
"further commands in this turn.",
|
||||
"",
|
||||
f"TARGETS ({len(plans)}) - reproduce this in your reply:",
|
||||
"",
|
||||
*(_status_line(p) for p in plans),
|
||||
]
|
||||
for plan in writes:
|
||||
lines += ["", f"DIFF {plan.target.label}:", "", _diff(plan, export).rstrip("\n")]
|
||||
lines += [
|
||||
"",
|
||||
"Once they have replied approving exactly this, the line that pushes it is:",
|
||||
"",
|
||||
f" {rerun}",
|
||||
"",
|
||||
f"The token {token} covers each target's branch tip and the exact file; if either moves, "
|
||||
"clearance is asked again.",
|
||||
]
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _write(plan: Plan, export: guideline_export.Export, identity: tuple[str, str]) -> tuple[str, bool]:
|
||||
"""Commit and push one target. Returns its result line and whether it
|
||||
counts as a failure (rejected, or a git error)."""
|
||||
message = f"Update {guideline_export.TARGET_PATH} from {export.instance} at {export.commit[:12]}"
|
||||
label = plan.target.label
|
||||
try:
|
||||
with repo_capture.cache_repo(plan.target.repo) as repo:
|
||||
commit = repo_capture.commit_file(
|
||||
repo, plan.tip, guideline_export.TARGET_PATH, export.data, message, identity
|
||||
)
|
||||
result = repo_capture.push_commit(plan.target.repo, repo, commit, plan.target.ref)
|
||||
except BackendError as exc:
|
||||
return f" {label}: failed ({exc})", True
|
||||
if not result.accepted:
|
||||
return f" {label}: rejected ({result.detail}) - the branch moved; run again", True
|
||||
return f" {label}: written {plan.tip[:12]} -> {commit[:12]}", False
|
||||
|
||||
|
||||
@app.command("guidelines")
|
||||
@cli_contract.record(cli_contract.CommandRecord(
|
||||
path="export guidelines",
|
||||
summary="Render the guideline pages into one generated `GUIDELINES.md`, or push it into the "
|
||||
"captured repositories that opted in (**Guideline Push Gate**).",
|
||||
synopsis=(
|
||||
cli_contract.Variant(
|
||||
usage="export guidelines [--field <predicate> ...] [--kind/--subtype/--collection/"
|
||||
"--tag <v>]",
|
||||
notes="Prints the file to stdout; no network.",
|
||||
),
|
||||
cli_contract.Variant(
|
||||
usage="export guidelines <predicates> --push [--bundle <raw-bundle> ...] "
|
||||
"[--confirm TOKEN]",
|
||||
notes="Writes it into the captured repositories, behind the gate.",
|
||||
),
|
||||
),
|
||||
properties=cli_contract.Properties(
|
||||
effect=cli_contract.Effect.WRITE,
|
||||
idempotent=cli_contract.Idempotent.YES,
|
||||
atomic="Per target - one commit pushed without force, or nothing; targets already "
|
||||
"written stay written when a later one fails",
|
||||
budget=cli_contract.Budget.COUNTED,
|
||||
network=cli_contract.Network.YES,
|
||||
gates=(GATE,),
|
||||
),
|
||||
notes=(
|
||||
"Selects pages with `search`'s own predicates and no text: `--field` (repeatable, AND), "
|
||||
"`--kind`, `--subtype`, `--collection`, `--tag`. Which filter selects this instance's "
|
||||
"guidelines is written down in `kb/CONVENTIONS.md`.",
|
||||
"Renders mechanically: line 1 is `<!-- wikitool:export kind=guidelines instance=<origin "
|
||||
"URL without userinfo, or local> commit=<newest commit touching a selected page> - "
|
||||
"generated, do not edit by hand -->`; then each page sorted by title, as `# <title>` and "
|
||||
"its text without frontmatter, the generated links and footnote blocks, citation markers "
|
||||
"(`[^id]`, legacy `^[[...]]`), with `[[Title|Text]]` as `Text` and `[[Title]]` as "
|
||||
"`Title`. A leading H1 of the page is replaced by the title heading. Code is never "
|
||||
"rewritten. LF endings, no timestamp: the same pages give the same bytes.",
|
||||
"Refuses before printing or pushing anything: no predicate, no match, an unknown field, "
|
||||
"any page under `kb/` with unreadable frontmatter, uncommitted changes in `kb/` or "
|
||||
"`types/` (untracked files included), or no git checkout.",
|
||||
"`--push` targets every captured repository - each `(repo, ref)` from a `_capture.json` "
|
||||
"under `raw/`, `--bundle` narrowing it. A target is skipped with its reason when its ref "
|
||||
"rule is a tag pattern, it cannot be reached or has no such branch, its root has no "
|
||||
"`GUIDELINES.md` (not opted in), its `GUIDELINES.md` has no export header (hand-written), "
|
||||
"or the header names another `instance=`. A header line alone is the opt-in.",
|
||||
"Guideline Push Gate: `--push` without a matching `--confirm` fetches and builds "
|
||||
"everything, pushes nothing, and exits 42 with each target's status (`new`, `changed`, "
|
||||
"`unchanged`, `skipped`), the diff for every target it would write, and the re-run line. "
|
||||
"The token digests URL, branch, old tip and new blob of every target to write.",
|
||||
"With a matching token, each target gets exactly one commit on the fetched tip changing "
|
||||
"only `GUIDELINES.md`, authored as the instance checkout's `user.name`/`user.email`, "
|
||||
"pushed without force; one line per target: `unchanged`, `written <old> -> <new>`, "
|
||||
"`skipped (<reason>)`, `rejected (<reason>)` or `failed (<reason>)`.",
|
||||
"Nothing to write (every target unchanged or skipped) ends with exit 0 and no gate. A "
|
||||
"second run after a successful one finds every target unchanged.",
|
||||
"The target repository's own `AGENTS.md` points at the file (Claude Code: "
|
||||
"`@GUIDELINES.md`); that belongs to the repository, not to this command.",
|
||||
),
|
||||
failures=(
|
||||
cli_contract.Failure(
|
||||
cause="No predicate, no page matches, a malformed predicate or an unknown field",
|
||||
reaction="Use the filter `kb/CONVENTIONS.md` names for this instance's guidelines, and "
|
||||
"retry once",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="A page under `kb/` has unreadable frontmatter, `kb/` or `types/` has "
|
||||
"uncommitted changes, or the instance is not a git checkout",
|
||||
reaction="Fix the named pages, or publish the changes first, then retry once",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="`--push` with no `user.name` or `user.email` in the instance checkout's git "
|
||||
"configuration - checked before any target is fetched",
|
||||
reaction="Show the message to the user; the identity is theirs to set, never yours",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="`--bundle` names no captured bundle under `raw/`, or `--bundle`/`--confirm` "
|
||||
"was given without `--push`",
|
||||
reaction="Fix the argument and retry once",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="A target rejected the push (its branch moved since the fetch) or a git call "
|
||||
"failed - the other targets were still served",
|
||||
reaction="Show the lines to the user. A rejected target is served by running the "
|
||||
"whole command again, which asks for clearance again",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
code=0,
|
||||
cause="A target was skipped - tag rule, not reachable, not opted in, hand-written "
|
||||
"file, another instance's file",
|
||||
reaction="",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="Guideline Push Gate: `--push` without `--confirm`, or with a token that does "
|
||||
"not match the current set of targets",
|
||||
reaction=cli_contract.token_gate_reaction("--confirm"),
|
||||
code=42,
|
||||
),
|
||||
),
|
||||
examples=(
|
||||
"tools/wikitool export guidelines --tag guideline",
|
||||
"tools/wikitool export guidelines --tag guideline --push",
|
||||
"tools/wikitool export guidelines --confirm <token> --push --field tags=guideline # "
|
||||
"re-run after exit 42, once the user approved every diff",
|
||||
),
|
||||
never=(
|
||||
"Never pass a `--confirm` token the user has not seen and approved.",
|
||||
"Never edit a `GUIDELINES.md` in a target repository by hand to make it match - the "
|
||||
"next export overwrites it; change the guideline page instead.",
|
||||
),
|
||||
see_also=(
|
||||
"`wikitool search` - the same predicates, to preview the selection",
|
||||
"`wikitool raw capture` - how a repository becomes a target",
|
||||
"`instructions/gates.md` - the gate procedure",
|
||||
),
|
||||
))
|
||||
def export_guidelines_command(
|
||||
field_: Optional[list[str]] = typer.Option(
|
||||
None, "--field", "-f",
|
||||
help="Frontmatter predicate, repeatable (AND) - the same forms as `search --field`.",
|
||||
),
|
||||
kind: Optional[str] = typer.Option(None, "--kind", help="Shorthand for --field kind=<value>."),
|
||||
subtype: Optional[str] = typer.Option(None, "--subtype", help="Shorthand for --field subtype=<value>."),
|
||||
collection: Optional[str] = typer.Option(
|
||||
None, "--collection", help="Shorthand for --field collection=<value>."
|
||||
),
|
||||
tag: Optional[str] = typer.Option(None, "--tag", help="Shorthand for --field tags=<value>."),
|
||||
push: bool = typer.Option(
|
||||
False, "--push", help="Write the file into the captured repositories (Guideline Push Gate)."
|
||||
),
|
||||
bundle: Optional[list[Path]] = typer.Option(
|
||||
None, "--bundle", help="Only the repository of this captured bundle under raw/; repeatable."
|
||||
),
|
||||
confirm: Optional[str] = typer.Option(
|
||||
None, "--confirm", help="The token from a prior exit 42, once a human has approved it."
|
||||
),
|
||||
):
|
||||
"""Render the guideline pages into GUIDELINES.md - printed, or pushed into
|
||||
the captured repositories that carry one."""
|
||||
bundles = [b if b.is_absolute() else config.ROOT / b for b in (bundle or [])]
|
||||
if not push and (bundles or confirm):
|
||||
fail("--bundle and --confirm only apply with --push.")
|
||||
raw = filters.raw_predicates(field_, kind, subtype, collection, tag)
|
||||
try:
|
||||
predicates = tuple(filters.parse_predicate(r) for r in raw)
|
||||
export = guideline_export.build(predicates)
|
||||
except ValidationError as exc:
|
||||
fail(escape(str(exc)))
|
||||
return
|
||||
|
||||
if not push:
|
||||
typer.echo(export.text, nl=False)
|
||||
return
|
||||
|
||||
identity = guideline_export.git_identity()
|
||||
if identity is None:
|
||||
fail(
|
||||
"No git identity in this instance checkout - `git config user.name` and `git config "
|
||||
"user.email` must both be set, since the commit in each target repository carries them. "
|
||||
"Nothing was fetched or pushed."
|
||||
)
|
||||
return
|
||||
try:
|
||||
targets = _targets(bundles)
|
||||
except ValidationError as exc:
|
||||
fail(escape(str(exc)))
|
||||
return
|
||||
if not targets:
|
||||
typer.echo("No captured repository under raw/ - nothing to write.")
|
||||
return
|
||||
|
||||
plans = [_plan(t, export) for t in targets]
|
||||
if not any(p.writes for p in plans):
|
||||
typer.echo(f"{len(plans)} target(s), nothing to write:")
|
||||
for plan in plans:
|
||||
typer.echo(_status_line(plan))
|
||||
return
|
||||
|
||||
token = confirm_token(plans)
|
||||
if confirm != token:
|
||||
console.print("[bold yellow]NEEDS USER CLEARANCE[/bold yellow] Guideline Push Gate")
|
||||
typer.echo(_clearance(plans, export, token, _rerun(token, raw, bundles), confirm))
|
||||
raise typer.Exit(code=EXIT_NEEDS_CLEARANCE)
|
||||
|
||||
failed = 0
|
||||
typer.echo(f"{len(plans)} target(s):")
|
||||
for plan in plans:
|
||||
if not plan.writes:
|
||||
typer.echo(_status_line(plan))
|
||||
continue
|
||||
line, bad = _write(plan, export, identity)
|
||||
failed += bad
|
||||
typer.echo(line)
|
||||
written = sum(1 for p in plans if p.writes) - failed
|
||||
typer.echo(f"{written} written, {failed} rejected or failed, {len(plans) - written - failed} "
|
||||
"unchanged or skipped.")
|
||||
if failed:
|
||||
console.print(
|
||||
f"[bold red]ERROR[/bold red] {failed} target(s) not written - what was written stays "
|
||||
"written; run the command again for the rest."
|
||||
)
|
||||
raise typer.Exit(code=1)
|
||||
@@ -25,7 +25,7 @@ from typing import Optional
|
||||
|
||||
import typer
|
||||
|
||||
from chemenu import cli_contract, config, links
|
||||
from chemenu import cli_contract, config, kb_scan, links
|
||||
from chemenu.commands._util import (
|
||||
check_collision,
|
||||
check_path_budget,
|
||||
@@ -50,14 +50,10 @@ from chemenu.provenance import (
|
||||
)
|
||||
from chemenu.type_resolver import resolver
|
||||
|
||||
# `[[Target]]`, `[[Target|alias]]`, `[[Target#anchor]]` - including the
|
||||
# `[[Target]]` inside a `[^cite-id]: [[Target]]` Footnotes definition, which
|
||||
# is exactly what lets retarget_body() repoint a citation's link target on a
|
||||
# rename. Group 1 is the target title; group 2 keeps any alias/anchor suffix
|
||||
# untouched. The id itself is a separate concern - see retarget_cite_ids().
|
||||
# Group 1 may span a line break; it is compared through
|
||||
# kb_scan.normalize_link_target(), the same reading `lint` gives it.
|
||||
LINK_RE = re.compile(r"\[\[([^\[\]|#]+)((?:[|#][^\[\]]*)?)\]\]")
|
||||
# `kb_scan.LINK_RE` - the whole wikilink, defined there so the guideline export
|
||||
# reads links the same way without importing a command module. The cite id is a
|
||||
# separate concern from the link target - see retarget_cite_ids().
|
||||
LINK_RE = kb_scan.LINK_RE
|
||||
|
||||
|
||||
def page_ref_fields(page: Page) -> list[str]:
|
||||
|
||||
@@ -87,7 +87,7 @@ from typing import Annotated, Optional
|
||||
import typer
|
||||
from rich.markup import escape
|
||||
|
||||
from chemenu import cli_contract, config, repo_capture, web_capture
|
||||
from chemenu import cli_contract, config, guideline_export, repo_capture, web_capture
|
||||
from chemenu.commands._util import fail, path_budget_problem_for, rel_path, success
|
||||
from chemenu.errors import BackendError, ChemenuError, ValidationError
|
||||
from chemenu.frontmatter_io import write_page
|
||||
@@ -226,11 +226,27 @@ def _check_files(resolved: list[Path]) -> None:
|
||||
_check_directly_in_incoming(path, incoming)
|
||||
if path.name == repo_capture.MANIFEST_NAME:
|
||||
raise _Refused(_reserved_manifest_message(path))
|
||||
_check_no_export(resolved)
|
||||
names = [path.name for path in resolved]
|
||||
if len(names) != len(set(names)):
|
||||
raise _Refused("Two files share a filename; rename one before promoting.")
|
||||
|
||||
|
||||
def _check_no_export(paths: list[Path]) -> None:
|
||||
"""Refuse a file a wiki exported (`export guidelines`): it is generated
|
||||
from `kb/`, so taking it into `raw/` would feed the wiki its own output
|
||||
back as a source. `raw capture` already leaves such a file out; this
|
||||
closes the way in through `incoming/`."""
|
||||
exports = [p for p in paths if guideline_export.file_is_export(p)]
|
||||
if exports:
|
||||
listed = "\n".join(f" - {rel_path(p)}" for p in exports)
|
||||
raise _Refused(
|
||||
"These files are a wiki's guideline export - their first line starts with "
|
||||
"<!-- wikitool:export. They are generated from kb/ and never go back into raw/:\n"
|
||||
f"{listed}\n Remove them from incoming/ and accept again; nothing was moved."
|
||||
)
|
||||
|
||||
|
||||
def _check_target(dst: Path, remedy: str) -> None:
|
||||
problem = path_budget_problem_for(dst)
|
||||
if problem:
|
||||
@@ -409,6 +425,7 @@ def _plan_folder(folder: Path) -> tuple[Path, list[tuple[Path, Path]], list[Path
|
||||
stray = [f for f in files if f.name == repo_capture.MANIFEST_NAME and f.parent != folder]
|
||||
if stray:
|
||||
raise _Refused(_reserved_manifest_message(stray[0]))
|
||||
_check_no_export(files)
|
||||
|
||||
holder = _occupied_stems(config.RAW_DIR).get(folder.name)
|
||||
if holder is not None:
|
||||
@@ -490,6 +507,7 @@ def _replace(
|
||||
if not incoming_path.is_file():
|
||||
fail(f"{rel_path(incoming_path)} does not exist or is not a file.")
|
||||
_validate_under_incoming(incoming_path, _incoming_dir())
|
||||
_refusals_fail(_check_no_export, [incoming_path])
|
||||
|
||||
target = _resolve(replaces)
|
||||
try:
|
||||
@@ -784,6 +802,7 @@ def _plan_bundle_replacement(folder: Path, bundle: Path, pages) -> _BundlePlan:
|
||||
_check_matches_manifest(folder, manifest)
|
||||
|
||||
new_files = repo_capture.bundle_files(folder)
|
||||
_check_no_export(list(new_files.values()))
|
||||
old_files = repo_capture.bundle_files(bundle)
|
||||
changes = repo_capture.diff(old_files, {r: p.read_bytes() for r, p in new_files.items()})
|
||||
changed = {r for _s, r in changes}
|
||||
@@ -1061,6 +1080,13 @@ def _replace_bundle(
|
||||
"incoming/<bundle> --replaces-bundle <raw-bundle>`. Any other occupied name: rename "
|
||||
"the folder in `incoming/` - such a folder has no replacement form",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
cause="A file - given directly, inside a folder, or as `--replaces`/`--replaces-bundle` "
|
||||
"material - is a guideline export: its first line, after an optional BOM, starts with "
|
||||
"`<!-- wikitool:export`",
|
||||
reaction="Not fixed by retrying: it is generated from a wiki's `kb/` and never goes "
|
||||
"back into `raw/`. Remove it from `incoming/`; nothing was moved",
|
||||
),
|
||||
cli_contract.Failure(
|
||||
label="raw accept",
|
||||
cause="`--fidelity`/`--authority` is missing, or names `unknown` or a value outside "
|
||||
@@ -2155,13 +2181,7 @@ def raw_status_command(
|
||||
):
|
||||
"""Report which captured bundles have fallen behind their repository.
|
||||
See raw/CONTRACT.md "Getting a repository in: `raw capture`"."""
|
||||
manifests = []
|
||||
if config.RAW_DIR.is_dir():
|
||||
manifests = [
|
||||
p for p in sorted(config.RAW_DIR.rglob(repo_capture.MANIFEST_NAME))
|
||||
if p.is_file() and not any(part.startswith(".") for part in p.relative_to(config.RAW_DIR).parts)
|
||||
]
|
||||
rows = [_bundle_status(p) for p in manifests]
|
||||
rows = [_bundle_status(p) for p in repo_capture.captured_manifests(config.RAW_DIR)]
|
||||
|
||||
if json_out:
|
||||
typer.echo(json.dumps(rows, indent=2, ensure_ascii=False))
|
||||
|
||||
@@ -240,12 +240,7 @@ def search_command(
|
||||
json_out: bool = typer.Option(False, "--json", help="Print the results as JSON."),
|
||||
):
|
||||
"""Search kb/ by text, by frontmatter, or by both."""
|
||||
raw_predicates = list(field or [])
|
||||
for value, name in ((kind, "kind"), (subtype, "subtype"), (collection, "collection")):
|
||||
if value:
|
||||
raw_predicates.append(f"{name}={value}")
|
||||
if tag:
|
||||
raw_predicates.append(f"tags={tag}")
|
||||
raw_predicates = filters.raw_predicates(field, kind, subtype, collection, tag)
|
||||
|
||||
if not text and not raw_predicates:
|
||||
fail("Nothing to search for: give a query, or at least one --field predicate.")
|
||||
|
||||
@@ -0,0 +1,296 @@
|
||||
"""The guideline export, with no CLI attached: select pages by frontmatter,
|
||||
render them mechanically into one `GUIDELINES.md`, and recognise such a file
|
||||
again when it comes back.
|
||||
|
||||
`wikitool export guidelines` is the terminal adapter over this module; the git
|
||||
side - writing the file into a captured repository's branch - lives beside
|
||||
`fetch` in `repo_capture.py`. Nothing here imports `typer` or anything under
|
||||
`chemenu.commands`, and nothing here imports `repo_capture`, which imports
|
||||
`EXPORT_MARKER` from here: that is the direction the dependency runs.
|
||||
|
||||
Three properties carry the design, and each has a test:
|
||||
|
||||
- **Deterministic.** The output depends only on the selected pages and the
|
||||
commit that last touched one of them - no timestamp, no `HEAD`. A run on an
|
||||
unchanged corpus, or after an instance commit that touched no guideline, is
|
||||
byte-identical, and so writes nothing into any target repository.
|
||||
- **One filter semantics.** Selection is `search`'s own predicate machinery
|
||||
(`search.filters`), with no text argument: a guideline is a declared choice in
|
||||
frontmatter, not a full-text hit.
|
||||
- **Code is never rewritten.** Every transformation of the page text runs on a
|
||||
copy with code masked out (`markdown_code.strip_code_spans`, which keeps
|
||||
offsets), so inline code and fenced blocks reach the output byte for byte.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from chemenu import blocks, config, corpus_cache, provenance, toolpaths
|
||||
from chemenu.errors import ValidationError
|
||||
from chemenu.kb_scan import LINK_RE, normalize_link_target
|
||||
from chemenu.markdown_code import strip_code_spans
|
||||
from chemenu.page import Page
|
||||
from chemenu.search import filters
|
||||
from chemenu.search.service import load_pages_by_path, unreadable_pages
|
||||
from chemenu.search.types import Predicate
|
||||
|
||||
# The first bytes of every file this stack generates for another repository.
|
||||
# Defined once, here: `repo_capture` excludes a file carrying it from a
|
||||
# capture, `raw accept` refuses one, and the export writes it.
|
||||
EXPORT_MARKER = b"<!-- wikitool:export"
|
||||
GUIDELINES_KIND = "guidelines"
|
||||
GUIDELINES_PREFIX = f"<!-- wikitool:export kind={GUIDELINES_KIND}"
|
||||
TARGET_PATH = "GUIDELINES.md"
|
||||
LOCAL_INSTANCE = "local"
|
||||
|
||||
_BOM = b"\xef\xbb\xbf"
|
||||
_HEADER_FIELD = re.compile(r"\b(instance|commit)=(\S+)")
|
||||
_H1 = re.compile(r"#(?:[ \t]|$)")
|
||||
_SCHEME_USERINFO = re.compile(r"^([A-Za-z][A-Za-z0-9+.-]*://)[^/]*@")
|
||||
_SCP_USERINFO = re.compile(r"^[^/@:]+@(?=[^/:]+:)")
|
||||
|
||||
|
||||
# --- recognising an export ------------------------------------------------------
|
||||
|
||||
|
||||
def _first_line(data: bytes) -> bytes:
|
||||
data = data[len(_BOM):] if data.startswith(_BOM) else data
|
||||
return data.split(b"\n", 1)[0]
|
||||
|
||||
|
||||
def is_export(data: bytes) -> bool:
|
||||
"""Whether `data` is a file this stack exported: its first line, after an
|
||||
optional BOM, starts with `EXPORT_MARKER`. Reads no more than the first
|
||||
line, so a caller may pass only a file's first bytes."""
|
||||
return _first_line(data).startswith(EXPORT_MARKER)
|
||||
|
||||
|
||||
def guidelines_header(data: bytes) -> Optional[dict[str, str]]:
|
||||
"""The fields of a guideline export's first line - `instance`, `commit`,
|
||||
whichever it carries - or None when `data` is not a guideline export at
|
||||
all. A header with no fields (`{}`) is the bare opt-in stub."""
|
||||
line = _first_line(data).decode("utf-8", "replace")
|
||||
if not line.startswith(GUIDELINES_PREFIX):
|
||||
return None
|
||||
return dict(_HEADER_FIELD.findall(line[len(GUIDELINES_PREFIX):]))
|
||||
|
||||
|
||||
def file_is_export(path: Path) -> bool:
|
||||
"""`is_export` for a file on disk, reading only its first line's worth."""
|
||||
with open(path, "rb") as handle:
|
||||
return is_export(handle.read(len(_BOM) + len(EXPORT_MARKER)))
|
||||
|
||||
|
||||
# --- the instance it comes from -----------------------------------------------
|
||||
|
||||
|
||||
def _git(args: list[str], root: Path) -> Optional[str]:
|
||||
"""`git <args>` in the instance checkout: stdout on success, None on any
|
||||
failure. Only local reads - nothing here reaches a remote."""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[toolpaths.git(), *args], cwd=root, capture_output=True, text=True,
|
||||
encoding="utf-8", timeout=30, check=False,
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return None
|
||||
return result.stdout if result.returncode == 0 else None
|
||||
|
||||
|
||||
def strip_userinfo(url: str) -> str:
|
||||
"""`url` without a user or token in front of the host -
|
||||
`https://user:token@host/x` becomes `https://host/x`, `git@host:x` becomes
|
||||
`host:x`. The header lands in other repositories, some of them public."""
|
||||
stripped = _SCHEME_USERINFO.sub(r"\1", url, count=1)
|
||||
if stripped != url:
|
||||
return stripped
|
||||
return _SCP_USERINFO.sub("", url, count=1)
|
||||
|
||||
|
||||
def instance_id(root: Optional[Path] = None) -> str:
|
||||
"""The `instance=` value: `origin`'s fetch URL without userinfo, or
|
||||
`local` for a checkout without an `origin`."""
|
||||
url = (_git(["remote", "get-url", "origin"], root or config.ROOT) or "").strip()
|
||||
if not url:
|
||||
return LOCAL_INSTANCE
|
||||
return "".join(strip_userinfo(url).split()) or LOCAL_INSTANCE
|
||||
|
||||
|
||||
def git_identity(root: Optional[Path] = None) -> Optional[tuple[str, str]]:
|
||||
"""`(user.name, user.email)` as the instance checkout resolves them - its
|
||||
local configuration before the global one - or None if either is unset.
|
||||
The commit in a target repository is made in a bare cache repository, which
|
||||
would see only the global configuration; this is the identity the
|
||||
instance's own commits carry."""
|
||||
root = root or config.ROOT
|
||||
name = (_git(["config", "user.name"], root) or "").strip()
|
||||
email = (_git(["config", "user.email"], root) or "").strip()
|
||||
return (name, email) if name and email else None
|
||||
|
||||
|
||||
# --- selection ----------------------------------------------------------------
|
||||
|
||||
|
||||
def select(predicates: tuple[Predicate, ...], kb_dir: Optional[Path] = None,
|
||||
root: Optional[Path] = None) -> list[Page]:
|
||||
"""The pages the predicates select, sorted by `(title.casefold(), title)`.
|
||||
|
||||
Raises `ValidationError` instead of returning something incomplete: with
|
||||
no predicate (a forgotten filter would export the whole wiki), on an
|
||||
unknown field (as `search` does), on any page under `kb/` whose
|
||||
frontmatter does not parse (it could be a guideline that silently drops
|
||||
out), and on zero hits (an empty file would delete every repository's
|
||||
guidelines)."""
|
||||
if not predicates:
|
||||
raise ValidationError(
|
||||
"No predicate - the export takes the guidelines by frontmatter, and without a filter "
|
||||
"it would take the whole wiki. kb/CONVENTIONS.md names this instance's filter."
|
||||
)
|
||||
kb_dir = kb_dir or config.KB_DIR
|
||||
pages = load_pages_by_path(kb_dir, root or config.ROOT)
|
||||
unreadable = unreadable_pages(pages)
|
||||
if unreadable:
|
||||
listed = "\n".join(f" - {u['path']} ({u['reason']})" for u in unreadable)
|
||||
raise ValidationError(
|
||||
"These pages have frontmatter that does not parse, so the export cannot tell whether "
|
||||
f"they are guidelines:\n{listed}\n Fix them first; nothing was exported."
|
||||
)
|
||||
filters.validate_fields(predicates, pages)
|
||||
selected = filters.apply_predicates(pages, predicates, kb_dir)
|
||||
if not selected:
|
||||
rendered = " ".join(p.render() for p in predicates)
|
||||
raise ValidationError(
|
||||
f"No page matches {rendered} - an empty export would delete every repository's "
|
||||
"guidelines, so nothing was exported."
|
||||
)
|
||||
return sorted(selected.values(), key=lambda p: (p.title.casefold(), p.title))
|
||||
|
||||
|
||||
def check_clean(root: Optional[Path] = None) -> None:
|
||||
"""Refuse a checkout that is not a git repository, or whose `kb/` or
|
||||
`types/` has uncommitted changes, untracked files included - otherwise
|
||||
`commit=` would name a state the export was not made from. `types/` counts
|
||||
because `kind` and `subtype` are resolved through the type-specs."""
|
||||
root = root or config.ROOT
|
||||
if corpus_cache.head_commit(root) is None:
|
||||
raise ValidationError(
|
||||
f"{root} is not a git checkout with a commit - the export names the commit its pages "
|
||||
"come from."
|
||||
)
|
||||
dirty = [
|
||||
Path(path).relative_to(root).as_posix() if Path(path).is_relative_to(root) else str(path)
|
||||
for path in (config.KB_DIR, config.TYPES_DIR)
|
||||
if corpus_cache.is_dirty(root, path)
|
||||
]
|
||||
if dirty:
|
||||
raise ValidationError(
|
||||
f"{' and '.join(dirty)} have uncommitted changes (untracked files count). Publish them "
|
||||
"first - the export names the commit its pages come from."
|
||||
)
|
||||
|
||||
|
||||
def last_commit(pages: list[Page], root: Optional[Path] = None) -> str:
|
||||
"""The newest commit touching one of `pages`' files - not `HEAD`, so an
|
||||
instance commit that touched no guideline leaves the output unchanged."""
|
||||
root = root or config.ROOT
|
||||
paths = [Path(p.path).resolve().relative_to(Path(root).resolve()).as_posix() for p in pages]
|
||||
sha = (_git(["log", "-1", "--format=%H", "--", *paths], root) or "").strip()
|
||||
if not sha:
|
||||
raise ValidationError("None of the selected pages is in a commit yet - publish them first.")
|
||||
return sha
|
||||
|
||||
|
||||
# --- rendering ----------------------------------------------------------------
|
||||
|
||||
|
||||
def _inline(text: str) -> str:
|
||||
"""Wikilinks to their display text; `[^cite-id]` and legacy `^[[...]]`
|
||||
citations removed - all of it outside code only. Matches are found on the
|
||||
masked copy and applied to the original, which share offsets."""
|
||||
masked = strip_code_spans(text)
|
||||
edits: list[tuple[int, int, str]] = []
|
||||
for m in provenance.LEGACY_CITE_RE.finditer(masked):
|
||||
edits.append((m.start(), m.end(), ""))
|
||||
for m in provenance.CITE_REF_RE.finditer(masked):
|
||||
edits.append((m.start(), m.end(), ""))
|
||||
for m in LINK_RE.finditer(masked):
|
||||
rest = text[m.start(2):m.end(2)]
|
||||
display = (
|
||||
rest.split("|", 1)[1] if "|" in rest else normalize_link_target(text[m.start(1):m.end(1)])
|
||||
)
|
||||
edits.append((m.start(), m.end(), display))
|
||||
# A legacy citation contains a wikilink; the one starting first wins.
|
||||
edits.sort(key=lambda e: (e[0], -e[1]))
|
||||
out: list[str] = []
|
||||
pos = 0
|
||||
for start, end, replacement in edits:
|
||||
if start < pos:
|
||||
continue
|
||||
out.append(text[pos:start])
|
||||
out.append(replacement)
|
||||
pos = end
|
||||
out.append(text[pos:])
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def _trim_blank(lines: list[str]) -> list[str]:
|
||||
start, end = 0, len(lines)
|
||||
while start < end and not lines[start].strip():
|
||||
start += 1
|
||||
while end > start and not lines[end - 1].strip():
|
||||
end -= 1
|
||||
return lines[start:end]
|
||||
|
||||
|
||||
def render_page(page: Page) -> str:
|
||||
"""One page: `# <title>`, then its text without the generated links and
|
||||
footnote blocks, citations removed and wikilinks resolved to text. A
|
||||
leading H1 of the page's own is replaced by the title heading; every other
|
||||
heading keeps its level."""
|
||||
body = page.body.replace("\r\n", "\n")
|
||||
body = blocks.strip(body, blocks.LINKS)
|
||||
body, _definitions = provenance.split_cite_block(body)
|
||||
lines = _trim_blank(_inline(body).split("\n"))
|
||||
if lines and _H1.match(lines[0]):
|
||||
lines = _trim_blank(lines[1:])
|
||||
return "\n".join([f"# {page.title}", "", *lines]) if lines else f"# {page.title}"
|
||||
|
||||
|
||||
def header(instance: str, commit: str) -> str:
|
||||
return (
|
||||
f"{GUIDELINES_PREFIX} instance={instance} commit={commit} "
|
||||
"- generated, do not edit by hand -->"
|
||||
)
|
||||
|
||||
|
||||
def render(pages: list[Page], instance: str, commit: str) -> str:
|
||||
"""The whole file: the header line, then each page, one blank line
|
||||
between. LF line endings, exactly one trailing newline."""
|
||||
parts = [header(instance, commit), *(render_page(p) for p in pages)]
|
||||
return "\n\n".join(parts) + "\n"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Export:
|
||||
text: str
|
||||
pages: tuple[Page, ...]
|
||||
instance: str
|
||||
commit: str
|
||||
|
||||
@property
|
||||
def data(self) -> bytes:
|
||||
return self.text.encode("utf-8")
|
||||
|
||||
|
||||
def build(predicates: tuple[Predicate, ...], root: Optional[Path] = None) -> Export:
|
||||
"""Select, check and render in one go. Raises `ValidationError`."""
|
||||
root = root or config.ROOT
|
||||
pages = select(predicates, root=root)
|
||||
check_clean(root)
|
||||
commit = last_commit(pages, root)
|
||||
instance = instance_id(root)
|
||||
return Export(render(pages, instance, commit), tuple(pages), instance, commit)
|
||||
@@ -12,6 +12,14 @@ from chemenu.page import Page
|
||||
|
||||
WIKILINK_RE = re.compile(r"\[\[([^\]|#]+)")
|
||||
|
||||
# The whole link: `[[Target]]`, `[[Target|alias]]`, `[[Target#anchor]]` -
|
||||
# including the `[[Target]]` inside a `[^cite-id]: [[Target]]` Footnotes
|
||||
# definition, which is exactly what lets `page_ops.retarget_body()` repoint a
|
||||
# citation's link target on a rename. Group 1 is the target title; group 2 keeps
|
||||
# any alias/anchor suffix untouched. Group 1 may span a line break; it is
|
||||
# compared through `normalize_link_target()`, the same reading `lint` gives it.
|
||||
LINK_RE = re.compile(r"\[\[([^\[\]|#]+)((?:[|#][^\[\]]*)?)\]\]")
|
||||
|
||||
# A line break inside `[[...]]`, with the indentation around it. The target
|
||||
# class above admits a newline, so a link someone wrapped at a fixed column -
|
||||
# `[[Foo Bar\n Target]]` - captures the break as part of the title, matches no
|
||||
|
||||
+135
-15
@@ -1,10 +1,12 @@
|
||||
"""Capturing documentation from a git repository for `raw/`: resolve a ref
|
||||
rule to one commit, read the files the globs select straight out of that
|
||||
commit, and describe the result in a manifest - with no CLI attached.
|
||||
commit, and describe the result in a manifest - with no CLI attached. And the
|
||||
one write in the other direction: a single commit that changes one file on a
|
||||
captured repository's branch, for `export guidelines --push`.
|
||||
|
||||
`wikitool raw capture`, `raw status` and `raw accept --replaces-bundle` are the
|
||||
terminal adapters over this module; it decides nothing about `incoming/` or
|
||||
`raw/` paths beyond the manifest's own name.
|
||||
`wikitool raw capture`, `raw status`, `raw accept --replaces-bundle` and
|
||||
`export guidelines` are the terminal adapters over this module; it decides
|
||||
nothing about `incoming/` or `raw/` paths beyond the manifest's own name.
|
||||
|
||||
Three properties carry the design, and each has a test:
|
||||
|
||||
@@ -39,6 +41,7 @@ from typing import Iterator, Optional
|
||||
|
||||
from chemenu import config, filelock, toolpaths, web_capture
|
||||
from chemenu.errors import BackendError, ValidationError
|
||||
from chemenu.guideline_export import EXPORT_MARKER
|
||||
|
||||
MANIFEST_NAME = "_capture.json"
|
||||
SCHEMA = 1
|
||||
@@ -52,7 +55,6 @@ ALLOWED_SCHEMES: tuple[str, ...] = ("ssh", "https")
|
||||
|
||||
GIT_TIMEOUT_SECONDS = 120.0
|
||||
|
||||
EXPORT_MARKER = b"<!-- wikitool:export"
|
||||
LFS_MARKER = b"version https://git-lfs.github.com/spec/v1"
|
||||
_BOM = b"\xef\xbb\xbf"
|
||||
|
||||
@@ -199,21 +201,36 @@ def _git_env() -> dict[str, str]:
|
||||
return env
|
||||
|
||||
|
||||
def run_git(
|
||||
@dataclass(frozen=True)
|
||||
class GitResult:
|
||||
returncode: int
|
||||
stdout: bytes
|
||||
stderr: bytes
|
||||
|
||||
def last_error_line(self) -> str:
|
||||
message = self.stderr.decode("utf-8", "replace").strip().splitlines()
|
||||
return message[-1] if message else f"exit {self.returncode}"
|
||||
|
||||
|
||||
def run_git_result(
|
||||
args: list[str],
|
||||
git_dir: Path,
|
||||
*,
|
||||
stdin: Optional[bytes] = None,
|
||||
env: Optional[dict[str, str]] = None,
|
||||
timeout: float = GIT_TIMEOUT_SECONDS,
|
||||
) -> bytes:
|
||||
"""Run `git --git-dir=<git_dir> <args>` and return its stdout.
|
||||
) -> GitResult:
|
||||
"""Run `git --git-dir=<git_dir> <args>` and return its exit code and both
|
||||
streams, whatever the exit code - for a caller that reads stdout on a
|
||||
failure too (`push --porcelain` reports a rejected ref there, with exit 1).
|
||||
|
||||
No call can prompt: no terminal prompt, no askpass program (an empty
|
||||
`core.askPass` stops git from falling back to `SSH_ASKPASS`), and on POSIX a
|
||||
session of its own, so `ssh` has no controlling terminal to ask on either.
|
||||
A timeout kills the whole process group, `ssh` included - killing `git`
|
||||
alone would leave the pipe open and the read below hanging. Raises
|
||||
`BackendError` on a non-zero exit, a timeout or a git that cannot start."""
|
||||
alone would leave the pipe open and the read below hanging. `env` is laid
|
||||
over that environment. Raises `BackendError` on a timeout or a git that
|
||||
cannot start."""
|
||||
argv = [toolpaths.git(), "-c", "core.askPass=", f"--git-dir={git_dir}", *args]
|
||||
posix = os.name == "posix"
|
||||
try:
|
||||
@@ -222,7 +239,7 @@ def run_git(
|
||||
stdin=subprocess.PIPE if stdin is not None else subprocess.DEVNULL,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
env=_git_env(),
|
||||
env={**_git_env(), **(env or {})},
|
||||
start_new_session=posix,
|
||||
)
|
||||
except OSError as exc:
|
||||
@@ -239,10 +256,23 @@ def run_git(
|
||||
proc.kill()
|
||||
proc.communicate()
|
||||
raise BackendError(f"git {args[0]} did not finish within {timeout:.0f} s") from None
|
||||
if proc.returncode != 0:
|
||||
message = err.decode("utf-8", "replace").strip().splitlines()
|
||||
raise BackendError(f"git {args[0]} failed: {message[-1] if message else f'exit {proc.returncode}'}")
|
||||
return out
|
||||
return GitResult(proc.returncode, out, err)
|
||||
|
||||
|
||||
def run_git(
|
||||
args: list[str],
|
||||
git_dir: Path,
|
||||
*,
|
||||
stdin: Optional[bytes] = None,
|
||||
env: Optional[dict[str, str]] = None,
|
||||
timeout: float = GIT_TIMEOUT_SECONDS,
|
||||
) -> bytes:
|
||||
"""`run_git_result`, returning stdout and raising `BackendError` on a
|
||||
non-zero exit as well."""
|
||||
result = run_git_result(args, git_dir, stdin=stdin, env=env, timeout=timeout)
|
||||
if result.returncode != 0:
|
||||
raise BackendError(f"git {args[0]} failed: {result.last_error_line()}")
|
||||
return result.stdout
|
||||
|
||||
|
||||
def cache_root() -> Path:
|
||||
@@ -470,6 +500,84 @@ def remote_commit(url: str, rule: str) -> ResolvedRef:
|
||||
return resolve_ref(url, rule, repo)
|
||||
|
||||
|
||||
# --- writing one file back ----------------------------------------------------
|
||||
#
|
||||
# `export guidelines --push` - the only write into another repository. It needs
|
||||
# no working tree and touches none: blob, tree and commit are built with
|
||||
# plumbing in the bare cache repository, on top of the tip `fetch` brought in,
|
||||
# and pushed by commit id. The cache's shallow history is enough, because the
|
||||
# server already has the parent.
|
||||
|
||||
|
||||
def read_path(repo: Path, commit: str, path: str) -> Optional[bytes]:
|
||||
"""The bytes of the file at repository path `path` in `commit`, or None
|
||||
if there is no regular file there."""
|
||||
out = run_git(["ls-tree", "-z", commit, "--", path], repo)
|
||||
entry = out.split(b"\0", 1)[0]
|
||||
if not entry:
|
||||
return None
|
||||
meta, _, _name = entry.partition(b"\t")
|
||||
mode, kind, sha = meta.decode("ascii").split()
|
||||
if kind != "blob" or mode == _MODE_SYMLINK:
|
||||
return None
|
||||
return read_blobs(repo, [sha])[sha]
|
||||
|
||||
|
||||
def blob_id(repo: Path, data: bytes) -> str:
|
||||
"""The object id `data` has as a blob in `repo`, without writing it."""
|
||||
return run_git(["hash-object", "--no-filters", "--stdin"], repo, stdin=data).decode().strip()
|
||||
|
||||
|
||||
def commit_file(
|
||||
repo: Path, parent: str, path: str, data: bytes, message: str, identity: tuple[str, str]
|
||||
) -> str:
|
||||
"""A new commit on `parent` whose tree differs from the parent's only in
|
||||
`path`, now holding `data` - built in a temporary index, so nothing else of
|
||||
the tree can change. The file keeps an executable bit it already had.
|
||||
`identity` is `(name, email)` for author and committer alike. Returns the
|
||||
new commit's id; nothing is pushed."""
|
||||
blob = run_git(["hash-object", "-w", "--no-filters", "--stdin"], repo, stdin=data).decode().strip()
|
||||
existing = run_git(["ls-tree", "-z", parent, "--", path], repo).split(b"\0", 1)[0]
|
||||
mode = "100755" if existing.startswith(b"100755 ") else "100644"
|
||||
name, email = identity
|
||||
index = repo / f"wikitool-export-{os.getpid()}.index"
|
||||
env = {
|
||||
"GIT_INDEX_FILE": str(index),
|
||||
"GIT_AUTHOR_NAME": name, "GIT_AUTHOR_EMAIL": email,
|
||||
"GIT_COMMITTER_NAME": name, "GIT_COMMITTER_EMAIL": email,
|
||||
}
|
||||
try:
|
||||
run_git(["read-tree", parent], repo, env=env)
|
||||
run_git(["update-index", "--add", "--cacheinfo", f"{mode},{blob},{path}"], repo, env=env)
|
||||
tree = run_git(["write-tree"], repo, env=env).decode().strip()
|
||||
return run_git(["commit-tree", tree, "-p", parent, "-m", message], repo, env=env).decode().strip()
|
||||
finally:
|
||||
index.unlink(missing_ok=True)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PushResult:
|
||||
accepted: bool
|
||||
detail: str # git's own summary for the ref, e.g. "[rejected] (fetch first)"
|
||||
|
||||
|
||||
def push_commit(url: str, repo: Path, commit: str, branch: str) -> PushResult:
|
||||
"""Push `commit` to `refs/heads/<branch>` at `url` - never forced, so a
|
||||
branch that moved since the fetch rejects it and stays where it is.
|
||||
|
||||
Accepted or rejected is read from `--porcelain`'s status flag for the ref
|
||||
(`!` is rejected), never from the wording on stderr, which git translates
|
||||
into the host's locale. Raises `BackendError` when there is no status for
|
||||
the ref at all - the remote was not reached."""
|
||||
ref = f"refs/heads/{branch}"
|
||||
result = run_git_result(["push", "--porcelain", "--", url, f"{commit}:{ref}"], repo)
|
||||
for line in result.stdout.decode("utf-8", "replace").splitlines():
|
||||
fields = line.split("\t")
|
||||
if len(fields) >= 3 and fields[1].endswith(f":{ref}") and fields[0] in (" ", "*", "=", "!", "+", "-"):
|
||||
return PushResult(fields[0] != "!", fields[2].strip())
|
||||
raise BackendError(f"git push failed: {result.last_error_line()}")
|
||||
|
||||
|
||||
# --- manifest -----------------------------------------------------------------
|
||||
|
||||
|
||||
@@ -537,6 +645,18 @@ def read_manifest(path: Path) -> Manifest:
|
||||
)
|
||||
|
||||
|
||||
def captured_manifests(raw_dir: Path) -> list[Path]:
|
||||
"""Every captured bundle's `_capture.json` under `raw_dir`, sorted, none
|
||||
below a hidden directory - the one list both `raw status` and
|
||||
`export guidelines` work from."""
|
||||
if not raw_dir.is_dir():
|
||||
return []
|
||||
return [
|
||||
p for p in sorted(raw_dir.rglob(MANIFEST_NAME))
|
||||
if p.is_file() and not any(part.startswith(".") for part in p.relative_to(raw_dir).parts)
|
||||
]
|
||||
|
||||
|
||||
def bundle_files(bundle: Path) -> dict[str, Path]:
|
||||
"""Every file of a captured bundle on disk, keyed by its repository path -
|
||||
the manifest itself left out."""
|
||||
|
||||
@@ -70,6 +70,26 @@ def parse_predicate(raw: str) -> Predicate:
|
||||
)
|
||||
|
||||
|
||||
def raw_predicates(
|
||||
fields: Optional[list[str]] = None,
|
||||
kind: Optional[str] = None,
|
||||
subtype: Optional[str] = None,
|
||||
collection: Optional[str] = None,
|
||||
tag: Optional[str] = None,
|
||||
) -> list[str]:
|
||||
"""The `--field` arguments a call's shorthand options stand for, after its
|
||||
own `--field` values: `--kind/--subtype/--collection X` is `<name>=X`,
|
||||
`--tag X` is `tags=X`. Every command taking these options goes through
|
||||
here, so the shorthands mean the same thing everywhere."""
|
||||
out = list(fields or [])
|
||||
for value, name in ((kind, "kind"), (subtype, "subtype"), (collection, "collection")):
|
||||
if value:
|
||||
out.append(f"{name}={value}")
|
||||
if tag:
|
||||
out.append(f"tags={tag}")
|
||||
return out
|
||||
|
||||
|
||||
def known_fields(pages: dict[str, Page]) -> set[str]:
|
||||
"""Every field name a predicate may legitimately name: the union of all
|
||||
frontmatter keys actually present in the corpus, plus the virtual ones."""
|
||||
|
||||
@@ -340,13 +340,15 @@ def test_network_yes_is_exactly_the_commands_that_can_reach_outside_this_checkou
|
||||
not only the two `version_cmd.py` used to claim exclusivity for. `dist upgrade` is on the list
|
||||
since `--latest`, which asks the release feed and downloads from it, and `raw fetch`
|
||||
since it exists (Gitea #120) - its `--html` form stays offline, which does not turn the
|
||||
command back to `no`. `raw capture` and `raw status` reach a git remote (Gitea #177). Pinned as an explicit
|
||||
command back to `no`. `raw capture` and `raw status` reach a git remote (Gitea #177), and so
|
||||
does `export guidelines --push` (Gitea #179). Pinned as an explicit
|
||||
set so a command gaining or losing that reach is a deliberate edit here, not a silent
|
||||
drift between the property and what the command actually does."""
|
||||
expected = {
|
||||
"raw fetch",
|
||||
"raw capture",
|
||||
"raw status",
|
||||
"export guidelines",
|
||||
"sync",
|
||||
"publish",
|
||||
"version check",
|
||||
|
||||
@@ -0,0 +1,553 @@
|
||||
"""`export guidelines` and the Guideline Push Gate, against local repositories
|
||||
over `file://` - never the network.
|
||||
|
||||
The instance is the `kb_dir` fixture tree made into a git checkout with its own
|
||||
identity and a copy of the shipped `types/`, so the cleanliness check sees the
|
||||
same two directories it sees in a real instance. Target repositories are bare
|
||||
repositories, each fed from a work clone, because a push into a checked-out
|
||||
branch is refused by git itself.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from chemenu import config, guideline_export, repo_capture
|
||||
from chemenu.cli import app
|
||||
from chemenu.commands import export_cmd
|
||||
from chemenu.commands.raw_cmd import raw_accept_command, raw_capture_command, raw_status_command
|
||||
from chemenu.frontmatter_io import write_page
|
||||
from chemenu.markdown_code import strip_code_spans
|
||||
from chemenu.search import filters
|
||||
from chemenu.search.registry import resolve
|
||||
from chemenu.search.service import load_pages_by_path, run_search
|
||||
from chemenu.search.types import SearchQuery
|
||||
from chemenu.type_resolver import resolver
|
||||
|
||||
runner = CliRunner()
|
||||
|
||||
TAG = ["--tag", "guideline"]
|
||||
|
||||
|
||||
def _git(cwd: Path, *args: str, check: bool = True) -> str:
|
||||
return subprocess.run(
|
||||
["git", "-c", "user.name=Fixture", "-c", "user.email=f@example.org", *args],
|
||||
cwd=cwd, check=check, capture_output=True, text=True,
|
||||
).stdout.strip()
|
||||
|
||||
|
||||
def _bare(bare: Path, *args: str) -> str:
|
||||
return subprocess.run(
|
||||
["git", f"--git-dir={bare}", *args], check=True, capture_output=True, text=True
|
||||
).stdout.strip()
|
||||
|
||||
|
||||
class Target:
|
||||
"""A bare repository - what the export pushes into - fed from a work
|
||||
clone that plays the project's own developers."""
|
||||
|
||||
def __init__(self, base: Path, guidelines: bytes | None = None) -> None:
|
||||
self.work = base / "work"
|
||||
self.bare = base / "remote.git"
|
||||
self.work.mkdir(parents=True)
|
||||
_git(self.work, "init", "-q", "-b", "main")
|
||||
(self.work / "README.md").write_text("# Project\n", encoding="utf-8")
|
||||
(self.work / "docs").mkdir()
|
||||
(self.work / "docs" / "a.md").write_text("# A\n", encoding="utf-8")
|
||||
if guidelines is not None:
|
||||
(self.work / "GUIDELINES.md").write_bytes(guidelines)
|
||||
self.commit("initial")
|
||||
_git(base, "clone", "-q", "--bare", str(self.work), str(self.bare))
|
||||
_git(self.work, "remote", "add", "origin", self.url)
|
||||
|
||||
@property
|
||||
def url(self) -> str:
|
||||
return self.bare.as_uri()
|
||||
|
||||
def commit(self, message: str = "change") -> None:
|
||||
_git(self.work, "add", "-A")
|
||||
_git(self.work, "commit", "-q", "--allow-empty", "-m", message)
|
||||
|
||||
def advance(self, message: str = "upstream change") -> str:
|
||||
"""A commit by someone else, pushed to the remote branch."""
|
||||
_git(self.work, "pull", "-q", "--ff-only", "origin", "main", check=False)
|
||||
(self.work / "docs" / "a.md").write_text(f"# A\n\n{message}\n", encoding="utf-8")
|
||||
self.commit(message)
|
||||
_git(self.work, "push", "-q", "origin", "HEAD:main")
|
||||
return self.tip()
|
||||
|
||||
def tip(self) -> str:
|
||||
return _bare(self.bare, "rev-parse", "refs/heads/main")
|
||||
|
||||
def refs(self) -> str:
|
||||
return _bare(self.bare, "for-each-ref", "--format=%(refname) %(objectname)")
|
||||
|
||||
def file(self, rev: str = "refs/heads/main") -> bytes:
|
||||
return subprocess.run(
|
||||
["git", f"--git-dir={self.bare}", "show", f"{rev}:GUIDELINES.md"],
|
||||
check=True, capture_output=True,
|
||||
).stdout
|
||||
|
||||
|
||||
STUB = b"<!-- wikitool:export kind=guidelines -->\n"
|
||||
|
||||
ALPHA_BODY = """
|
||||
# Alpha Rule
|
||||
|
||||
Keep secrets in `.env` only[^s-handbook]. See [[Beta Rule]] and [[Beta Rule|the beta]],
|
||||
also [[Beta Rule#Scope]] and [[Beta Rule#Scope|its scope]]. Legacy^[[Source - Handbook]] too.
|
||||
|
||||
## Details
|
||||
|
||||
Inline `[[Not A Link]]` and `[^not-a-cite]` stay as written.
|
||||
|
||||
```
|
||||
[[Fenced|kept]] and [^fenced] and ^[[Fenced]]
|
||||
```
|
||||
|
||||
<!-- wikitool:links -->
|
||||
## Beziehungen
|
||||
|
||||
- **related:** [[Beta Rule]]
|
||||
<!-- /wikitool:links -->
|
||||
|
||||
<!-- wikitool:footnotes -->
|
||||
## Fußnoten
|
||||
|
||||
[^s-handbook]: [[Source - Handbook]]
|
||||
<!-- /wikitool:footnotes -->
|
||||
"""
|
||||
|
||||
BETA_BODY = """
|
||||
Beta has no heading of its own.
|
||||
|
||||
### Scope
|
||||
|
||||
Everything.
|
||||
|
||||
## Footnotes
|
||||
|
||||
[^s-handbook]: [[Source - Handbook]]
|
||||
"""
|
||||
|
||||
|
||||
def _guideline(kb: Path, title: str, body: str, tags=("guideline",)) -> Path:
|
||||
path = kb / "concepts" / f"{title}.md"
|
||||
write_page(path, {"type": "types/concept.md", "tags": list(tags), "summary": f"{title}."}, body)
|
||||
return path
|
||||
|
||||
|
||||
def _commit_instance(root: Path, message: str = "instance change") -> None:
|
||||
_git(root, "add", "-A")
|
||||
_git(root, "commit", "-q", "-m", message)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def instance(kb_dir, monkeypatch):
|
||||
root = kb_dir.parent
|
||||
shutil.copytree(config._PACKAGE_ROOT / "types", root / "types")
|
||||
monkeypatch.setattr(config, "TYPES_DIR", root / "types")
|
||||
monkeypatch.setattr(resolver, "_repo_root", root)
|
||||
monkeypatch.setattr(repo_capture, "ALLOWED_SCHEMES", ("ssh", "https", "file"))
|
||||
(root / "raw").mkdir(exist_ok=True)
|
||||
(root / "incoming").mkdir(exist_ok=True)
|
||||
(root / ".gitignore").write_text("tools/\n", encoding="utf-8")
|
||||
_guideline(kb_dir, "Alpha Rule", ALPHA_BODY)
|
||||
_guideline(kb_dir, "beta rule", BETA_BODY)
|
||||
_git(root, "init", "-q", "-b", "main")
|
||||
_git(root, "config", "user.name", "Instance Author")
|
||||
_git(root, "config", "user.email", "author@example.org")
|
||||
_commit_instance(root, "instance")
|
||||
return root
|
||||
|
||||
|
||||
def _manifest(root: Path, name: str, repo: str, ref: str = "main") -> Path:
|
||||
bundle = root / "raw" / "2026" / "10" / name
|
||||
bundle.mkdir(parents=True)
|
||||
data = {
|
||||
"schema": 1, "repo": repo, "ref": ref, "commit": "0" * 40, "paths": ["docs/**"],
|
||||
"captured": "2026-10-05T00:00:00Z", "fidelity": "verbatim", "authority": "normative",
|
||||
"files": [],
|
||||
}
|
||||
(bundle / repo_capture.MANIFEST_NAME).write_text(json.dumps(data), encoding="utf-8")
|
||||
return bundle
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def target(instance, tmp_path):
|
||||
t = Target(tmp_path / "targets" / "one", STUB)
|
||||
_manifest(instance, "one", t.url)
|
||||
_commit_instance(instance, "capture one")
|
||||
return t
|
||||
|
||||
|
||||
def _run(*args: str):
|
||||
return runner.invoke(app, ["export", "guidelines", *args])
|
||||
|
||||
|
||||
def _token(output: str) -> str:
|
||||
match = re.search(r"--confirm (\w+) --push", output)
|
||||
assert match, output
|
||||
return match.group(1)
|
||||
|
||||
|
||||
def _push(*extra: str):
|
||||
"""Gate run, then the confirmed run - what an approved push looks like."""
|
||||
gated = _run(*TAG, "--push", *extra)
|
||||
assert gated.exit_code == 42, gated.output
|
||||
return _run(*TAG, "--push", "--confirm", _token(gated.output), *extra)
|
||||
|
||||
|
||||
# --- selection ----------------------------------------------------------------
|
||||
|
||||
|
||||
def test_selection_is_searchs_own(instance, kb_dir):
|
||||
_guideline(kb_dir, "Gamma Elsewhere", "\nNot a guideline.\n", tags=("other",))
|
||||
_commit_instance(instance)
|
||||
result = _run(*TAG)
|
||||
assert result.exit_code == 0, result.output
|
||||
exported = re.findall(r"^# (.+)$", result.output, re.MULTILINE)
|
||||
predicates = tuple(filters.parse_predicate(r) for r in filters.raw_predicates(tag="guideline"))
|
||||
search = run_search(SearchQuery(text=None, predicates=predicates, limit=0),
|
||||
load_pages_by_path(), resolve(None))
|
||||
expected = sorted((h.title for h in search.hits), key=lambda t: (t.casefold(), t))
|
||||
assert exported == expected == ["Alpha Rule", "beta rule"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("args, needle", [
|
||||
((), "No predicate"),
|
||||
(("--tag", "nothing-has-this"), "No page matches"),
|
||||
(("--field", "no_such_field=x"), "unknown field"),
|
||||
])
|
||||
def test_refusals_print_no_export(instance, target, args, needle):
|
||||
before = target.refs()
|
||||
for extra in ((), ("--push",)):
|
||||
result = _run(*args, *extra)
|
||||
assert result.exit_code == 1, result.output
|
||||
assert needle in result.output
|
||||
assert "<!-- wikitool:export" not in result.output
|
||||
assert target.refs() == before
|
||||
|
||||
|
||||
def test_unreadable_frontmatter_anywhere_refuses(instance, kb_dir, target):
|
||||
(kb_dir / "concepts" / "Broken.md").write_text("---\ntype: [unclosed\n---\nBody\n", encoding="utf-8")
|
||||
_commit_instance(instance)
|
||||
before = target.refs()
|
||||
result = _run(*TAG, "--push")
|
||||
assert result.exit_code == 1
|
||||
assert "Broken.md" in result.output and "<!-- wikitool:export" not in result.output
|
||||
assert target.refs() == before
|
||||
|
||||
|
||||
@pytest.mark.parametrize("change", ["modified", "untracked-kb", "untracked-types"])
|
||||
def test_dirty_kb_or_types_refuses(instance, kb_dir, target, change):
|
||||
if change == "modified":
|
||||
page = kb_dir / "concepts" / "Alpha Rule.md"
|
||||
page.write_text(page.read_text(encoding="utf-8") + "\nmore\n", encoding="utf-8")
|
||||
elif change == "untracked-kb":
|
||||
(kb_dir / "concepts" / "draft.txt").write_text("x\n", encoding="utf-8")
|
||||
else:
|
||||
(instance / "types" / "draft.md").write_text("x\n", encoding="utf-8")
|
||||
before = target.refs()
|
||||
for extra in ((), ("--push",)):
|
||||
result = _run(*TAG, *extra)
|
||||
assert result.exit_code == 1, result.output
|
||||
assert "uncommitted changes" in result.output
|
||||
assert "<!-- wikitool:export" not in result.output
|
||||
assert target.refs() == before
|
||||
|
||||
|
||||
# --- rendering ----------------------------------------------------------------
|
||||
|
||||
|
||||
def _export(*args: str) -> str:
|
||||
result = _run(*(args or TAG))
|
||||
assert result.exit_code == 0, result.output
|
||||
return result.output
|
||||
|
||||
|
||||
def test_rendering_resolves_links_and_drops_citations_outside_code(instance):
|
||||
out = _export()
|
||||
assert out.endswith("\n") and not out.endswith("\n\n") and "\r" not in out
|
||||
assert "See Beta Rule and the beta,\nalso Beta Rule and its scope. Legacy too." in out
|
||||
assert "Keep secrets in `.env` only." in out
|
||||
assert "Inline `[[Not A Link]]` and `[^not-a-cite]` stay as written." in out
|
||||
assert "```\n[[Fenced|kept]] and [^fenced] and ^[[Fenced]]\n```" in out
|
||||
prose = strip_code_spans(out)
|
||||
for forbidden in ("[[", "[^", "^[[", "wikitool:links", "wikitool:footnotes", "Beziehungen",
|
||||
"Fußnoten", "Footnotes", "type: types/", "---\n"):
|
||||
assert forbidden not in prose, forbidden
|
||||
|
||||
|
||||
def test_headings(instance):
|
||||
out = _export()
|
||||
lines = out.split("\n")
|
||||
assert lines[1] == "" and lines[2] == "# Alpha Rule" and lines[3] == ""
|
||||
assert out.count("# Alpha Rule") == 1
|
||||
assert "\n## Details\n" in out and "\n### Scope\n" in out
|
||||
assert "\n\n# beta rule\n\nBeta has no heading of its own." in out
|
||||
|
||||
|
||||
def test_header_line(instance):
|
||||
first = _export().split("\n", 1)[0]
|
||||
commit = _git(instance, "log", "-1", "--format=%H", "--", "kb/concepts/Alpha Rule.md",
|
||||
"kb/concepts/beta rule.md")
|
||||
assert first == (
|
||||
f"<!-- wikitool:export kind=guidelines instance=local commit={commit} "
|
||||
"- generated, do not edit by hand -->"
|
||||
)
|
||||
|
||||
|
||||
def test_instance_is_origin_without_userinfo(instance):
|
||||
_git(instance, "remote", "add", "origin", "https://user:s3cret@git.example.org/team/wiki.git")
|
||||
first = _export().split("\n", 1)[0]
|
||||
assert "instance=https://git.example.org/team/wiki.git " in first
|
||||
assert "s3cret" not in first and "user" not in first
|
||||
|
||||
|
||||
@pytest.mark.parametrize("url, expected", [
|
||||
("https://user:tok@host/x.git", "https://host/x.git"),
|
||||
("ssh://git@host:2222/x.git", "ssh://host:2222/x.git"),
|
||||
("git@host:team/x.git", "host:team/x.git"),
|
||||
("https://host/x.git", "https://host/x.git"),
|
||||
])
|
||||
def test_strip_userinfo(url, expected):
|
||||
assert guideline_export.strip_userinfo(url) == expected
|
||||
|
||||
|
||||
def test_output_is_deterministic_and_ignores_unrelated_commits(instance, kb_dir):
|
||||
first = _export()
|
||||
assert _export() == first
|
||||
(kb_dir / "concepts" / "Unrelated.md").write_text(
|
||||
"---\ntype: types/concept.md\ntags: [other]\n---\n\nx\n", encoding="utf-8"
|
||||
)
|
||||
_commit_instance(instance, "unrelated")
|
||||
assert _export() == first
|
||||
|
||||
|
||||
# --- the gate and the push ----------------------------------------------------
|
||||
|
||||
|
||||
def test_push_without_identity_refuses_before_any_target(instance, target):
|
||||
_git(instance, "config", "--unset", "user.email")
|
||||
before = target.refs()
|
||||
result = _run(*TAG, "--push")
|
||||
assert result.exit_code == 1, result.output
|
||||
assert "user.email" in result.output
|
||||
assert not re.search(r"--confirm \w{12}", result.output)
|
||||
assert "NEEDS USER CLEARANCE" not in result.output
|
||||
assert target.refs() == before
|
||||
|
||||
|
||||
def test_push_without_token_is_gated(instance, target, tmp_path):
|
||||
hand = Target(tmp_path / "targets" / "hand", b"# Our own rules\n")
|
||||
_manifest(instance, "hand", hand.url)
|
||||
_commit_instance(instance)
|
||||
before = (target.refs(), hand.refs())
|
||||
result = _run(*TAG, "--push")
|
||||
assert result.exit_code == 42, result.output
|
||||
assert (target.refs(), hand.refs()) == before
|
||||
assert f"{target.url} main: new" in result.output
|
||||
assert f"{hand.url} main: skipped" in result.output
|
||||
assert "+# Alpha Rule" in result.output and "-<!-- wikitool:export kind=guidelines -->" in result.output
|
||||
assert re.search(r"tools/wikitool export guidelines --confirm \w{12} --push --field tags=guideline",
|
||||
result.output)
|
||||
|
||||
|
||||
def test_confirmed_push_writes_one_commit_with_only_guidelines(instance, target):
|
||||
old = target.tip()
|
||||
stdout = _export()
|
||||
result = _push()
|
||||
assert result.exit_code == 0, result.output
|
||||
new = target.tip()
|
||||
assert f"written {old[:12]} -> {new[:12]}" in result.output
|
||||
assert _bare(target.bare, "rev-parse", f"{new}^") == old
|
||||
assert _bare(target.bare, "diff", "--name-only", old, new) == "GUIDELINES.md"
|
||||
assert target.file() == stdout.encode("utf-8")
|
||||
assert _bare(target.bare, "log", "-1", "--format=%an <%ae>|%cn <%ce>", new) == (
|
||||
"Instance Author <author@example.org>|Instance Author <author@example.org>"
|
||||
)
|
||||
commit = stdout.split("commit=", 1)[1][:12]
|
||||
assert _bare(target.bare, "log", "-1", "--format=%s", new) == (
|
||||
f"Update GUIDELINES.md from local at {commit}"
|
||||
)
|
||||
|
||||
again = _run(*TAG, "--push")
|
||||
assert again.exit_code == 0, again.output
|
||||
assert f"{target.url} main: unchanged" in again.output
|
||||
assert "--confirm" not in again.output
|
||||
assert target.tip() == new
|
||||
|
||||
|
||||
def test_stale_tokens_are_gated_again(instance, kb_dir, target, tmp_path):
|
||||
token = _token(_run(*TAG, "--push").output)
|
||||
|
||||
target.advance()
|
||||
moved = _run(*TAG, "--push", "--confirm", token)
|
||||
assert moved.exit_code == 42 and "does not match" in moved.output
|
||||
token = _token(moved.output)
|
||||
|
||||
page = kb_dir / "concepts" / "beta rule.md"
|
||||
page.write_text(page.read_text(encoding="utf-8") + "\nAmended.\n", encoding="utf-8")
|
||||
_commit_instance(instance, "amend beta")
|
||||
changed = _run(*TAG, "--push", "--confirm", token)
|
||||
assert changed.exit_code == 42 and "+Amended." in changed.output
|
||||
token = _token(changed.output)
|
||||
|
||||
second = Target(tmp_path / "targets" / "two", STUB)
|
||||
_manifest(instance, "two", second.url)
|
||||
_commit_instance(instance, "capture two")
|
||||
added = _run(*TAG, "--push", "--confirm", token)
|
||||
assert added.exit_code == 42 and f"{second.url} main: new" in added.output
|
||||
|
||||
before = (target.refs(), second.refs())
|
||||
invented = _run(*TAG, "--push", "--confirm", "000000000000")
|
||||
assert invented.exit_code == 42
|
||||
assert (target.refs(), second.refs()) == before
|
||||
|
||||
|
||||
def test_targets_that_do_not_take_part_are_left_alone(instance, target, tmp_path):
|
||||
hand = Target(tmp_path / "targets" / "hand", b"# Our own rules\n")
|
||||
none = Target(tmp_path / "targets" / "none")
|
||||
foreign = Target(
|
||||
tmp_path / "targets" / "foreign",
|
||||
b"<!-- wikitool:export kind=guidelines instance=https://other.example/wiki commit=abc -->\n",
|
||||
)
|
||||
tagged = Target(tmp_path / "targets" / "tagged", STUB)
|
||||
for name, t, ref in (("hand", hand, "main"), ("none", none, "main"),
|
||||
("foreign", foreign, "main"), ("tagged", tagged, "v*")):
|
||||
_manifest(instance, name, t.url, ref)
|
||||
_commit_instance(instance)
|
||||
before = {t.url: t.refs() for t in (hand, none, foreign, tagged)}
|
||||
result = _push()
|
||||
assert result.exit_code == 0, result.output
|
||||
assert "hand-written" in result.output
|
||||
assert "not opted in" in result.output
|
||||
assert "another instance (https://other.example/wiki)" in result.output
|
||||
assert "tag pattern" in result.output
|
||||
assert {t.url: t.refs() for t in (hand, none, foreign, tagged)} == before
|
||||
assert "written" in result.output # the participating target was still served
|
||||
|
||||
|
||||
def test_a_branch_that_moved_is_rejected_not_forced(instance, target, monkeypatch):
|
||||
# Translated git messages must not change the verdict: it is read from the
|
||||
# porcelain status flag, not from the text.
|
||||
monkeypatch.setenv("LANGUAGE", "de")
|
||||
monkeypatch.setenv("LANG", "de_DE.UTF-8")
|
||||
token = _token(_run(*TAG, "--push").output)
|
||||
real_commit_file = repo_capture.commit_file
|
||||
foreign: list[str] = []
|
||||
|
||||
def commit_then_race(*args, **kwargs):
|
||||
foreign.append(target.advance("raced in"))
|
||||
return real_commit_file(*args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(repo_capture, "commit_file", commit_then_race)
|
||||
result = _run(*TAG, "--push", "--confirm", token)
|
||||
assert result.exit_code == 1, result.output
|
||||
assert f"{target.url} main: rejected" in result.output
|
||||
assert target.tip() == foreign[0]
|
||||
|
||||
|
||||
def test_an_unreachable_target_does_not_stop_the_others(instance, target, tmp_path):
|
||||
_manifest(instance, "gone", (tmp_path / "does-not-exist.git").as_uri())
|
||||
_commit_instance(instance)
|
||||
result = _push()
|
||||
assert result.exit_code == 0, result.output
|
||||
assert "does-not-exist.git main: skipped (not reachable" in result.output
|
||||
assert f"{target.url} main: written" in result.output
|
||||
|
||||
|
||||
def test_bundle_narrows_the_targets(instance, target, tmp_path):
|
||||
other = Target(tmp_path / "targets" / "other", STUB)
|
||||
_manifest(instance, "other", other.url)
|
||||
_commit_instance(instance)
|
||||
before = other.refs()
|
||||
result = _push("--bundle", "raw/2026/10/one")
|
||||
assert result.exit_code == 0, result.output
|
||||
assert other.url not in result.output
|
||||
assert other.refs() == before
|
||||
unknown = _run(*TAG, "--push", "--bundle", "raw/2026/10/nope")
|
||||
assert unknown.exit_code == 1 and "not a captured bundle" in unknown.output
|
||||
|
||||
|
||||
def test_bundle_and_confirm_need_push(instance):
|
||||
assert _run(*TAG, "--confirm", "abc").exit_code == 1
|
||||
assert _run(*TAG, "--bundle", "raw/x").exit_code == 1
|
||||
|
||||
|
||||
def test_raw_status_does_not_report_a_pushed_export(instance, tmp_path, capsys):
|
||||
t = Target(tmp_path / "targets" / "captured", STUB)
|
||||
raw_capture_command(repo_url=t.url, ref="main", paths=["**/*.md"], name="captured",
|
||||
fidelity="verbatim", authority="normative", update=None)
|
||||
raw_accept_command(files=[instance / "incoming" / "captured"], fidelity=None, authority=None,
|
||||
page=None, replaces=None, dry_run=False)
|
||||
_commit_instance(instance, "capture")
|
||||
capsys.readouterr()
|
||||
result = _push()
|
||||
assert result.exit_code == 0 and "written" in result.output, result.output
|
||||
raw_status_command(json_out=True)
|
||||
rows = json.loads(capsys.readouterr().out)
|
||||
assert len(rows) == 1 and rows[0]["error"] is None and rows[0]["changed"] is False
|
||||
|
||||
|
||||
# --- no way back in -----------------------------------------------------------
|
||||
|
||||
|
||||
def test_export_marker_is_defined_once():
|
||||
package = Path(guideline_export.__file__).parent
|
||||
definitions = [
|
||||
path for path in package.rglob("*.py")
|
||||
if "tests" not in path.parts and re.search(r'= b"<!-- wikitool:export"', path.read_text("utf-8"))
|
||||
]
|
||||
assert definitions == [Path(guideline_export.__file__)]
|
||||
|
||||
|
||||
def test_a_real_export_is_excluded_from_capture(instance):
|
||||
data = _export().encode("utf-8")
|
||||
assert repo_capture._content_exclusion(data) is not None
|
||||
assert repo_capture._content_exclusion(b"\xef\xbb\xbf" + data) is not None
|
||||
|
||||
|
||||
def _snapshot(*dirs: Path) -> dict[str, bytes]:
|
||||
return {p.as_posix(): p.read_bytes() for d in dirs for p in sorted(d.rglob("*")) if p.is_file()}
|
||||
|
||||
|
||||
def test_raw_accept_refuses_an_export_file_and_a_folder_holding_one(instance):
|
||||
incoming, raw = instance / "incoming", instance / "raw"
|
||||
export = _export().encode("utf-8")
|
||||
(incoming / "GUIDELINES.md").write_bytes(export)
|
||||
(incoming / "pack").mkdir()
|
||||
(incoming / "pack" / "notes.md").write_text("# Notes\n", encoding="utf-8")
|
||||
(incoming / "pack" / "sub").mkdir()
|
||||
(incoming / "pack" / "sub" / "rules.md").write_bytes(b"\xef\xbb\xbf" + export)
|
||||
before = _snapshot(incoming, raw)
|
||||
for path in (incoming / "GUIDELINES.md", incoming / "pack"):
|
||||
result = runner.invoke(app, ["raw", "accept", str(path), "--fidelity", "verbatim",
|
||||
"--authority", "normative"])
|
||||
assert result.exit_code == 1, result.output
|
||||
assert "guideline export" in result.output
|
||||
assert _snapshot(incoming, raw) == before
|
||||
|
||||
|
||||
def test_is_export_reads_the_first_line_only():
|
||||
assert guideline_export.is_export(b"<!-- wikitool:export kind=x -->\nbody")
|
||||
assert not guideline_export.is_export(b"# Title\n<!-- wikitool:export kind=x -->\n")
|
||||
assert guideline_export.guidelines_header(STUB) == {}
|
||||
assert guideline_export.guidelines_header(b"<!-- wikitool:export kind=other -->") is None
|
||||
|
||||
|
||||
def test_confirm_token_ignores_targets_it_does_not_write():
|
||||
t = export_cmd.Target("u", "main")
|
||||
writes = export_cmd.Plan(t, "new", tip="a", blob="b")
|
||||
skipped = export_cmd.Plan(export_cmd.Target("v", "main"), "skipped", "x")
|
||||
assert export_cmd.confirm_token([writes, skipped]) == export_cmd.confirm_token([writes])
|
||||
assert export_cmd.confirm_token([writes]) != export_cmd.confirm_token(
|
||||
[export_cmd.Plan(t, "new", tip="a2", blob="b")]
|
||||
)
|
||||
Reference in new issue
Block a user