export run: commit= im Marker je Seite statt je Ziel - jede Datei nennt den letzten Commit ihrer eigenen Seite, ermittelt in einem git log über alle ausgewählten Pfade (Pfade über stdin und wörtlich), das Manifest behält den neuesten Commit über alle; ein Commit schreibt nur die Dateien der Seiten neu, die er berührt hat, plus Dateien, deren Links sich ändern. Erster Lauf nach dem Update schreibt jede Datei einmal neu (#221); 10.1.0-beta.12
CI / verify (push) Successful in 7m54s
CI / pwsh (push) Successful in 2m42s
Release / converters (push) Successful in 26s
Release / release (push) Successful in 36s
Release / wiki-mirror (push) Skipped

Files changed:
- CHANGES.md
- VERSION
- docs/how-to/export-to-a-synced-folder.md
- docs/reference/commands/exports.md
- docs/reference/configuration.md
- tools/CONTRACT.md
- tools/chemenu/commands/export_cmd.py
- tools/chemenu/guideline_export.py
- tools/chemenu/page_export.py
- tools/chemenu/tests/test_export_run.py

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
torbenandClaude Opus 5.5 committed 2026-10-15 22:53:29 +02:00
1 parent ca12c54200
commit 52078bb4b0
10 files changed
+161 -21

No files matched your search

+1 -1
View File
@@ -2456,7 +2456,7 @@ Write the derived, read-only reading copy of `kb/` into the folders the `export`
**NOTES**
- An export is derived: regenerated from `kb/` on every run, never edited, never read back into the wiki. Each target names its pages with an allowlist of `search --field` predicates (`include`: a list of selections, AND within one, OR across them), a handler (`markdown`, or `command` for the operator's converter) and the folder (`path`, outside the instance). Nothing is exported that no selection names.
- Every Markdown file starts with a frontmatter block whose `wikitool_export` line says `kind=pages instance=<origin URL without userinfo, or local> commit=<newest commit touching a selected page>`, followed by `title`, `type`, `summary`, `modified` where the page has them (`frontmatter: summary`, the default; `minimal` keeps only the marker, `full` the whole frontmatter), then the page text. A file keeps its path below `kb/`. A wikilink to an exported page becomes a relative Markdown link (`links: relative`, the default) or plain text (`links: text`); a link to a page that is not exported is always plain text. Footnote markers stay; code is never rewritten. LF endings, no timestamp: the same pages give the same bytes.
- Every Markdown file starts with a frontmatter block whose `wikitool_export` line says `kind=pages instance=<origin URL without userinfo, or local> commit=<newest commit touching this page>`, followed by `title`, `type`, `summary`, `modified` where the page has them (`frontmatter: summary`, the default; `minimal` keeps only the marker, `full` the whole frontmatter), then the page text. A file keeps its path below `kb/`. A wikilink to an exported page becomes a relative Markdown link (`links: relative`, the default) or plain text (`links: text`); a link to a page that is not exported is always plain text. Footnote markers stay; code is never rewritten. LF endings, no timestamp: the same pages give the same bytes, and a commit that touches one page rewrites that page's file alone - plus any file whose links change because a page entered or left the selection. The manifest's `commit` is the newest commit touching any selected page; all of them come from one `git log`, however many pages are selected.
- Handler `command`: the Markdown is rendered into a staging directory, the program of `export.commands` runs with no shell, stdin closed and a timeout of 300 s, and receives the Markdown directory and an empty output directory as its last two arguments. On exit 0 the regular files in the output directory (recursively) become the target's content; any other end leaves the target unchanged and prints the end of its stderr. The staging directory is always removed.
- The folder is only ever touched by what it holds in its manifest, `.wikitool-export.json`: the files an earlier run wrote, each with its sha256. A run removes only manifest files that are no longer exported (and folders it created, once empty), rewrites a manifest file that is missing, leaves every other file byte for byte as it was, and writes nothing - not even a modification time - for a file that did not change. A run that changes nothing writes nothing at all.
- Refuses a target, before writing anything into it: a non-empty folder with no manifest (it is somebody's, not an export), a manifest file that was changed in the folder since the last run (naming it), a foreign file where an exported one belongs, an empty selection (it would delete the earlier export), an unknown field in a predicate. A missing folder is created.
+5 -2
View File
@@ -460,13 +460,16 @@ def export_guidelines_command(
"the instance). Nothing is exported that no selection names.",
"Every Markdown file starts with a frontmatter block whose `wikitool_export` line says "
"`kind=pages instance=<origin URL without userinfo, or local> commit=<newest commit "
"touching a selected page>`, followed by `title`, `type`, `summary`, `modified` where the "
"touching this page>`, followed by `title`, `type`, `summary`, `modified` where the "
"page has them (`frontmatter: summary`, the default; `minimal` keeps only the marker, "
"`full` the whole frontmatter), then the page text. A file keeps its path below `kb/`. A "
"wikilink to an exported page becomes a relative Markdown link (`links: relative`, the "
"default) or plain text (`links: text`); a link to a page that is not exported is always "
"plain text. Footnote markers stay; code is never rewritten. LF endings, no timestamp: "
"the same pages give the same bytes.",
"the same pages give the same bytes, and a commit that touches one page rewrites that "
"page's file alone - plus any file whose links change because a page entered or left the "
"selection. The manifest's `commit` is the newest commit touching any selected page; all "
"of them come from one `git log`, however many pages are selected.",
"Handler `command`: the Markdown is rendered into a staging directory, the program of "
"`export.commands` runs with no shell, stdin closed and a timeout of 300 s, and receives "
"the Markdown directory and an empty output directory as its last two arguments. On exit "
+39 -5
View File
@@ -122,12 +122,12 @@ def file_is_export(path: Path) -> bool:
# --- the instance it comes from -----------------------------------------------
def _git(args: list[str], root: Path) -> Optional[str]:
def _git(args: list[str], root: Path, stdin: Optional[str] = None) -> Optional[str]:
"""`git <args>` in the instance checkout: stdout on success, None on any
failure. Only local reads - nothing here reaches a remote."""
try:
result = subprocess.run(
[toolpaths.git(), *args], cwd=root, capture_output=True, text=True,
[toolpaths.git(), *args], cwd=root, input=stdin, capture_output=True, text=True,
encoding="utf-8", timeout=30, check=False,
)
except (OSError, subprocess.SubprocessError):
@@ -227,17 +227,51 @@ def check_clean(root: Optional[Path] = None) -> None:
)
_NOT_COMMITTED = "None of the selected pages is in a commit yet - publish them first."
def _repo_paths(pages: list[Page], root: Path) -> list[str]:
return [Path(p.path).resolve().relative_to(Path(root).resolve()).as_posix() for p in pages]
def last_commit(pages: list[Page], root: Optional[Path] = None) -> str:
"""The newest commit touching one of `pages`' files - not `HEAD`, so an
instance commit that touched no guideline leaves the output unchanged."""
root = root or config.ROOT
paths = [Path(p.path).resolve().relative_to(Path(root).resolve()).as_posix() for p in pages]
sha = (_git(["log", "-1", "--format=%H", "--", *paths], root) or "").strip()
sha = (_git(["log", "-1", "--format=%H", "--", *_repo_paths(pages, root)], root) or "").strip()
if not sha:
raise ValidationError("None of the selected pages is in a commit yet - publish them first.")
raise ValidationError(_NOT_COMMITTED)
return sha
def page_commits(pages: list[Page], root: Optional[Path] = None) -> tuple[str, dict[Path, str]]:
"""`last_commit(pages)`, and for each page's `path` the newest commit
touching that one file - from a single `git log`, however many pages there
are, since every process start costs on Windows. The paths go in on stdin,
so a whole corpus does not run into the command-line limit, and literally,
so a `[` in a title is not a glob. Without `--follow`, a page `rename` moved
gets the commit that moved it. A page no listed commit names - one that
only a merge's own resolution touched - falls back to the newest commit."""
root = root or config.ROOT
paths = _repo_paths(pages, root)
out = _git(["--literal-pathspecs", "log", "--stdin", "-z", "--format=commit %H", "--name-only"], root,
stdin="--\n" + "".join(f"{path}\n" for path in paths)) or ""
newest = ""
current = ""
first: dict[str, str] = {}
# `-z` ends each header and each name with NUL; a commit's first name
# follows its header after a newline.
for token in out.split("\0"):
if token.startswith("commit "):
current = token[len("commit "):]
newest = newest or current
elif token.strip("\n"):
first.setdefault(token.lstrip("\n"), current)
if not newest:
raise ValidationError(_NOT_COMMITTED)
return newest, {page.path: first.get(path, newest) for page, path in zip(pages, paths)}
# --- rendering ----------------------------------------------------------------
+11 -8
View File
@@ -10,9 +10,10 @@ An export is never a working copy: it is regenerated from `kb/` and what a
person changes in the folder is overwritten or refused, never merged back. The
properties that carry that, each with a test:
- **Deterministic.** The output depends on the selected pages and the commit
that last touched one of them - no timestamp, no `HEAD` - so a run on an
unchanged corpus writes nothing, not even a file's modification time.
- **Deterministic.** A file depends on its page, the selection it links into
and the commit that last touched that page - no timestamp, no `HEAD` - so a
run on an unchanged corpus writes nothing, not even a file's modification
time, and a commit to one page rewrites that page's file alone.
- **Only its own files are touched.** What an earlier run wrote is recorded in
a manifest in the target (`.wikitool-export.json`, with a digest per file).
A run removes only manifest files that are no longer exported, leaves every
@@ -290,8 +291,10 @@ def render_page(page: Page, here: str, destinations: dict[str, Optional[str]], t
return text.encode("utf-8")
def render(pages: list[Page], target: Target, kb_dir: Path, instance: str, commit: str) -> dict[str, bytes]:
"""The files of a target for `pages`, by path below the target. A page
def render(pages: list[Page], target: Target, kb_dir: Path, instance: str,
commits: dict[Path, str]) -> dict[str, bytes]:
"""The files of a target for `pages`, by path below the target, each
marked with its own page's commit from `commits`. A page
that cannot be written on every platform, or two that would share a file
name in a case-insensitive folder, is an `ExportError`."""
paths = {id(page): file_path(page, kb_dir) for page in pages}
@@ -312,7 +315,7 @@ def render(pages: list[Page], target: Target, kb_dir: Path, instance: str, commi
files: dict[str, bytes] = {}
for page in pages:
rel = paths[id(page)]
files[rel] = render_page(page, rel, by_title, target, instance, commit)
files[rel] = render_page(page, rel, by_title, target, instance, commits[page.path])
return files
@@ -563,8 +566,8 @@ def run(targets: Sequence[Target], root: "Path | str | None" = None) -> list[Tar
if not selected:
raise ExportError("no page matches the selection - an empty export would delete the "
"earlier one, so nothing was changed")
result.commit = guideline_export.last_commit(selected, root)
files = render(selected, target, kb_dir, instance, result.commit)
result.commit, commits = guideline_export.page_commits(selected, root)
files = render(selected, target, kb_dir, instance, commits)
result.titles = {file_path(p, kb_dir): p.title for p in selected}
if target.handler == config_shapes.EXPORT_HANDLER_COMMAND:
files = convert(target, files, result)
+94
View File
@@ -304,6 +304,100 @@ def test_an_instance_commit_that_touches_no_selected_page_changes_nothing(instan
assert _mtimes(folder) == stamps
def _marker_commit(path: Path) -> str:
line = path.read_text(encoding="utf-8").split("\n")[1]
return line.rsplit("commit=", 1)[1].rstrip('"')
def test_a_commit_to_one_page_rewrites_that_file_alone(instance):
_configure(instance, pages={"include": [["tags=docs"], ["tags=private"]]})
assert _run().exit_code == 0
folder = _out(instance, "pages")
before, stamps = _snapshot(folder), _mtimes(folder)
_page(instance, "concepts/Alpha.md", ALPHA + "\nMore.\n")
commit = _commit(instance, "edit alpha")
result = _run()
assert result.exit_code == 0, result.output
out = _flat(result.output)
assert "changed (1): Alpha" in out and "unchanged: 2" in out and "new (" not in out
after, after_stamps = _snapshot(folder), _mtimes(folder)
for rel in ("concepts/protocols/Beta.md", "concepts/Gamma.md"):
assert after[rel] == before[rel] and after_stamps[rel] == stamps[rel]
assert _marker_commit(folder / "concepts" / "Alpha.md") == commit
def test_each_file_carries_its_own_pages_commit_and_the_manifest_the_newest(instance):
_page(instance, "concepts/Löwe [1].md", "\n# Löwe\n\nText.\n")
first = _commit(instance, "lion")
_page(instance, "concepts/protocols/Beta.md", BETA + "\nMore.\n")
newest = _commit(instance, "edit beta")
(instance / "README.md").write_text("other\n", encoding="utf-8")
_commit(instance, "unrelated")
_configure(instance)
assert _run().exit_code == 0
folder = _out(instance, "pages")
init = _git(instance, "rev-list", "--max-parents=0", "HEAD")
assert _marker_commit(folder / "concepts" / "Alpha.md") == init
assert _marker_commit(folder / "concepts" / "Löwe [1].md") == first
assert _marker_commit(folder / "concepts" / "protocols" / "Beta.md") == newest
assert _manifest(folder)["commit"] == newest
def test_a_page_entering_the_selection_rewrites_only_the_files_whose_text_changes(instance):
_configure(instance, pages={"include": [["tags=docs"], ["summary~Gamma"]]})
_page(instance, "concepts/Gamma.md", "\n# Gamma\n\nPrivate.\n", tags=["private"], summary="Hidden.")
_commit(instance, "hide gamma")
assert _run().exit_code == 0
folder = _out(instance, "pages")
before, stamps = _snapshot(folder), _mtimes(folder)
assert "and Gamma for more" in before["concepts/Alpha.md"].decode("utf-8")
_page(instance, "concepts/Gamma.md", "\n# Gamma\n\nPrivate.\n", tags=["private"])
_commit(instance, "show gamma")
result = _run()
assert result.exit_code == 0, result.output
out = _flat(result.output)
assert "new (1): Gamma" in out and "changed (1): Alpha" in out and "unchanged: 1" in out
after, after_stamps = _snapshot(folder), _mtimes(folder)
assert "[Gamma](Gamma.md)" in after["concepts/Alpha.md"].decode("utf-8")
beta = "concepts/protocols/Beta.md"
assert after[beta] == before[beta] and after_stamps[beta] == stamps[beta]
def test_a_renamed_page_carries_the_commit_of_the_rename(instance):
(instance / "README.md").write_text("other\n", encoding="utf-8")
_commit(instance, "unrelated")
result = runner.invoke(app, ["rename", "--from", "Beta", "--to", "Epsilon"])
assert result.exit_code == 0, result.output
renamed = _commit(instance, "rename")
_configure(instance)
assert _run().exit_code == 0
folder = _out(instance, "pages")
assert _marker_commit(folder / "concepts" / "protocols" / "Epsilon.md") == renamed
def test_the_git_calls_do_not_grow_with_the_pages(instance, monkeypatch):
_configure(instance)
calls = []
real = guideline_export._git
def counted(args, root, stdin=None):
calls.append(args)
return real(args, root, stdin)
monkeypatch.setattr(guideline_export, "_git", counted)
assert _run().exit_code == 0
few = len(calls)
for index in range(30):
_page(instance, f"concepts/Many{index}.md", f"\n# Many{index}\n\nText.\n")
_commit(instance, "many")
calls.clear()
result = _run()
assert result.exit_code == 0, result.output
assert "new (30)" in _flat(result.output)
assert len(calls) == few
assert sum(1 for args in calls if "log" in args) == 1
def test_a_changed_page_a_new_page_and_a_removed_page(instance):
_configure(instance)
assert _run().exit_code == 0