fix: publish --path commits the uncommitted generated files outside its path, so the pushed catalog and log match its pages (#182)

Files changed:
- CHANGES.md
- VERSION
- tools/CONTRACT.md
- tools/chemenu/commands/git_publish.py
- tools/chemenu/tests/test_git_publish.py

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2

CHANGES.md

@@ -59,7 +59,7 @@ concern - readable here, never shipped as something to parse.
 
 ---
 
-## 8.0.0-beta.47 - 2026-10-06 - sync record: the autostash note states the behaviour, not the change
+## 8.0.0-beta.48 - 2026-10-06 - publish --path nimmt ungespeicherte generierte Dateien außerhalb des Pfads mit
 
 **Author:** Torben Nehmer
 
@@ -112,6 +112,7 @@ concern - readable here, never shipped as something to parse.
 - wiki-ingest: updating the captured repositories is a step-1 branch over raw status
 - export guidelines: the guideline pages as a generated GUIDELINES.md, pushed into the captured repositories behind the Guideline Push Gate
 - publish/sync merge generated files mechanically and carry non-overlapping uncommitted work through a rebase
+- publish --path nimmt ungespeicherte generierte Dateien außerhalb des Pfads mit
 
 **Low impact**
 - version bump no longer points at version release in its output
@@ -160,6 +161,32 @@ concern - readable here, never shipped as something to parse.
 - sync record: the autostash note states the behaviour, not the change
 <!-- /wikitool:bumps -->
 
+### publish --path nimmt ungespeicherte generierte Dateien außerhalb des Pfads mit
+
+`publish --path <X>` staged and committed only `<X>`. When the catalog and log it had written -
+or the ones a reconcile had just merged and regenerated - lay outside `<X>`, the push carried
+the new page with a catalog that did not list it and a log without its entries, and left the
+generated files behind uncommitted until a `publish` without `--path` took them along. Nothing was
+lost, but the remote stayed inconsistent in between, and another session's `sync` worked against
+that state.
+
+- **Uncommitted generated files outside `--path` now join the commit** (`kb/index.md`,
+  `kb/log.md`, `kb/provenance.md`, every `kb/**/INDEX.md`), and a line names them:
+  `Including N generated file(s) outside --path: ...`. Git decides what lies under `<X>`, by the
+  same pathspec the staging uses; files already under it are neither added nor named.
+- **Regardless of whether this run's reconcile regenerated anything.** A run the Mass-Update
+  Gate refuses has already reconciled; the confirmed re-run finds nothing left to regenerate,
+  and a condition on the regeneration would have pushed the inconsistent state on exactly that
+  path - the same goes for a `sync` followed by `publish --path`.
+- **The gate is unchanged.** The added files stay out of its count, and the `--confirm` token
+  and the re-run line cover the `--path` given, so a clearance issued for `<X>` stays valid.
+- **Every other file outside `--path`** is left alone, as before.
+
+Where uncommitted pages also lie outside `<X>`, the catalog that goes out with `<X>` already
+lists them until the next `publish` sends them - the reverse of before, where the page went out
+and the catalog did not know it. Drop-in: no command, flag, file format or state file changes.
+`publish`'s record follows (Gitea #182).
+
 ### sync record: the autostash note states the behaviour, not the change
 
 One note in `sync`'s command record, written with 8.0.0-beta.46, said that uncommitted changes no

VERSION

@@ -1 +1 @@
-8.0.0-beta.47
+8.0.0-beta.48

tools/CONTRACT.md

@@ -1993,7 +1993,7 @@ Reconcile with `<remote>/<branch>`, then stage all changes, commit, and push.
 - Order: branch check and Publish-Remote Gate, then the reconcile with `<remote>/<branch>`, then the Mass-Update Gate, then `git add -A`, commit and push. `--no-push` skips all but the Mass-Update Gate and the commit.
 - Without `--no-push`, a remote that is not configured or cannot be reached ends the call with exit 1 at the reconcile - before the gate, `git add` and the commit, and also on a clean tree. Nothing is committed, the index and the working tree are unchanged, and the message names `--no-push` as the way to a local commit. The next `publish` that reaches the remote pushes that commit along with whatever is new. A remote that answers but has no `<branch>` yet (a new, empty repository) is not this case: the first publish of an instance commits and pushes as before.
 - Reconcile: fetches `<remote>/<branch>`, fast-forwards when only the remote moved, rebases the local commits on top when both sides moved but touched disjoint files, and exits 42 (rebase-review gate) when both sides touched the same file. A refused reconcile performs no rebase attempt. The `--confirm-rebase` token covers the exact upstream state and the set of files touched on both sides.
-- Generated files are never an overlap: where they are all that stops the reconcile, it merges them as `sync` does (both sides' log entries kept, the catalog and `kb/provenance.md` regenerated) and carries uncommitted changes no incoming commit touches through the rebase. The proactive reconcile runs before staging, so this publish commits the regenerated files; on the retry after a rejected push they get a commit of their own before the second push.
+- Generated files are never an overlap: where they are all that stops the reconcile, it merges them as `sync` does (both sides' log entries kept, the catalog and `kb/provenance.md` regenerated) and carries uncommitted changes no incoming commit touches through the rebase. The proactive reconcile runs before staging, so this publish commits the regenerated files - with `--path` too, see there; on the retry after a rejected push they get a commit of their own before the second push.
 - The push target must be the checked-out branch; this is checked before anything is staged. The unborn branch of a fresh `git init -b main` counts as checked out, so the first publish of a new instance works; a real detached HEAD is refused.
 - With nothing new to stage, a local commit the remote lacks is still pushed: one left behind by an earlier publish whose push failed, or every commit when the remote answers but does not have the branch yet (a new, empty remote repository).
 - A rejected push that finds the remote unreachable on its one retry reports the original push error.
@@ -2002,7 +2002,7 @@ Reconcile with `<remote>/<branch>`, then stage all changes, commit, and push.
 - Never counted and never shown for approval, but committed like everything else: anything under `work/`, and the files `wikitool` generates itself (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md` under `kb/`). The refusal line accounts for both, by reason.
 - The `--confirm` token covers each counted path, the blob id of its contents and the publish target: a different file list or edited contents need a new clearance.
 - Publish-Remote Gate: when the checkout carries `.wikitool-remotes.json` and the push URL of `--remote` is not listed in it, exits 42 before the reconcile fetches anything. The URL is read with `git remote get-url --push`, so a repointed remote does not pass on its name. An absent file means unrestricted; a malformed one is an error, not permission.
-- `--path` (repeatable) scopes the whole operation - gate count, staging and commit - to that subtree.
+- `--path` (repeatable) scopes the whole operation - gate count, staging and commit - to that subtree, with one exception: uncommitted generated files outside `--path` (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md` under `kb/`) are staged and committed too, and a line names them - whether a reconcile or the change itself wrote them. They stay out of the gate count, and the `--confirm` token and the re-run line cover the `--path` given. Every other file outside `--path` is left as it is.
 - Commit message: `--message`, then a `Files changed:` paragraph listing every committed path. When `--message` ends in a paragraph git reads as a trailer block (`Co-Authored-By:` and the like), that block stays the last paragraph and the list goes in front of it, since git reads trailers from the last paragraph only. `git interpret-trailers` decides whether there is such a block, and the list moves only when git reads the same trailers from the result; otherwise it is appended at the end. `--message` is not part of any gate token.
 - After a successful commit or push whose changed files include `tools/`, `types/`, `instructions/`, `AGENTS.md` or a path ending in `CONTRACT.md`, prints one reminder line: the phase past this point (an issue-body rewrite, `docs/` staleness, a changelog entry's accuracy) is not covered by `docs verify`, `instructions verify` or `pytest`. It is not a gate: no exit code change, nothing to clear, and silent for an ordinary content publish.
 

tools/chemenu/commands/git_publish.py

@@ -816,10 +816,11 @@ def _git_bytes(*args: str) -> Optional[bytes]:
     return result.stdout if result.returncode == 0 else None
 
 
-def _dirty_paths() -> tuple[list[str], list[str]]:
-    """(tracked paths whose index or working copy differs from HEAD, untracked paths). Ignored
-    files are neither - nothing here ever touches one."""
-    result = _run(["git", "status", "--porcelain=v1", "-z", "--untracked-files=all", "--no-renames"])
+def _dirty_paths(pathspec: tuple[str, ...] = ()) -> tuple[list[str], list[str]]:
+    """(tracked paths whose index or working copy differs from HEAD, untracked paths), limited
+    to `pathspec` when one is given. Ignored files are neither - nothing here ever touches one."""
+    result = _run(["git", "status", "--porcelain=v1", "-z", "--untracked-files=all", "--no-renames",
+                   "--", *pathspec])
     tracked: list[str] = []
     untracked: list[str] = []
     for record in result.stdout.split("\0"):
@@ -1466,6 +1467,14 @@ def commit_message(message: str, changed_files: list[str]) -> str:
     return candidate if _trailers(candidate) == trailers else plain
 
 
+def generated_outside(paths: list[str]) -> list[str]:
+    """The uncommitted generated files `git add -A -- <paths>` would leave behind. Git decides
+    what lies under `paths`, by the same pathspec the staging uses."""
+    tracked, untracked = _dirty_paths()
+    inside = set(sum(_dirty_paths(tuple(paths)), []))
+    return sorted(path for path in tracked + untracked if is_generated(path) and path not in inside)
+
+
 def commit_regenerated(remote: str, branch: str) -> None:
     """Commit the generated files a reconcile just rewrote - and only those, whatever else is
     staged - with a message of their own."""
@@ -1522,8 +1531,8 @@ def commit_regenerated(remote: str, branch: str) -> None:
         "merges them as `sync` does (both sides' log entries kept, the catalog and "
         "`kb/provenance.md` regenerated) and carries uncommitted changes no incoming commit "
         "touches through the rebase. The proactive reconcile runs before staging, so this "
-        "publish commits the regenerated files; on the retry after a rejected push they get a "
-        "commit of their own before the second push.",
+        "publish commits the regenerated files - with `--path` too, see there; on the retry "
+        "after a rejected push they get a commit of their own before the second push.",
         "The push target must be the checked-out branch; this is checked before anything is "
         "staged. The unborn branch of a fresh `git init -b main` counts as checked out, so the "
         "first publish of a new instance works; a real detached HEAD is refused.",
@@ -1555,7 +1564,12 @@ def commit_regenerated(remote: str, branch: str) -> None:
         "on its name. An absent file means unrestricted; a malformed one is an error, not "
         "permission.",
         "`--path` (repeatable) scopes the whole operation - gate count, staging and commit - "
-        "to that subtree.",
+        "to that subtree, with one exception: uncommitted generated files outside `--path` "
+        "(`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md` under `kb/`) are "
+        "staged and committed too, and a line names them - whether a reconcile or the change "
+        "itself wrote them. They stay out of the gate count, and the `--confirm` token and the "
+        "re-run line cover the `--path` given. Every other file outside `--path` is left as it "
+        "is.",
         "Commit message: `--message`, then a `Files changed:` paragraph listing every committed "
         "path. When `--message` ends in a paragraph git reads as a trailer block "
         "(`Co-Authored-By:` and the like), that block stays the last paragraph and the list "
@@ -1693,6 +1707,9 @@ def publish_command(
         fail(YES_REMOVED_MESSAGE)
 
     paths = list(path or [])
+    # What is staged and committed: `paths` plus any uncommitted generated file outside them.
+    # The gate token and its re-run line keep `paths`.
+    staging = paths
 
     # Checked before anything is staged, for the same reason as the gate below:
     # a refused publish must leave the working tree exactly as it was found.
@@ -1737,9 +1754,20 @@ def publish_command(
         if summary:
             typer.echo(summary)
 
+    # `--path` would otherwise push a page with a catalog that does not list it and a log
+    # without its entries, and leave the generated files behind uncommitted. They carry no
+    # decision - the gate leaves them out of its count for the same reason - so they join the
+    # commit rather than stop it. Not only after a regenerating reconcile: a run the gate
+    # refused has already reconciled, and the confirmed re-run finds nothing left to regenerate.
+    if paths:
+        extra = generated_outside(paths)
+        if extra:
+            staging = [*paths, *extra]
+            typer.echo(f"Including {len(extra)} generated file(s) outside --path: {', '.join(extra)}")
+
     # The gate is evaluated *before* anything is staged, so a refused publish
     # leaves the working tree exactly as it was found.
-    changes = collect_changes(paths)
+    changes = collect_changes(staging)
     local_ahead = push and _local_ahead_of_remote(remote, branch)
     if not changes:
         if not local_ahead:
@@ -1790,7 +1818,7 @@ def publish_command(
         )
 
     if changes:
-        add_result = _run(["git", "add", "-A", "--", *paths])
+        add_result = _run(["git", "add", "-A", "--", *staging])
         if add_result.returncode != 0:
             fail(f"git add failed:\n{add_result.stderr}")
 
@@ -1799,8 +1827,8 @@ def publish_command(
         # With a pathspec, `git commit -- <paths>` commits exactly those paths and
         # ignores anything else that happens to be staged, so batches stay disjoint.
         commit_args = ["git", "commit", "-m", full_message]
-        if paths:
-            commit_args += ["--", *paths]
+        if staging:
+            commit_args += ["--", *staging]
         commit_result = _run(commit_args)
         if commit_result.returncode != 0:
             fail(f"git commit failed:\n{commit_result.stderr}")

tools/chemenu/tests/test_git_publish.py

@@ -1898,3 +1898,123 @@ def test_an_index_md_under_raw_still_goes_to_review_when_both_sides_change_it(re
 
     assert outcome.status == "needs-review"
     assert outcome.overlap_files == ["raw/repos/project/INDEX.md"]
+
+
+# --- publish --path and the uncommitted generated files outside it ---
+
+PROTOCOLS = "kb/concepts/protocols"
+
+
+def test_publish_path_commits_the_generated_files_the_reconcile_regenerated(wiki_repo, capsys):
+    """The pushed tip is consistent: its catalog and provenance are what a rebuild of it writes,
+    and its log carries every local entry once. Nothing generated stays behind uncommitted."""
+    writer = _clone_writer(wiki_repo)
+    _writer_ingest(writer, "Writer")
+    _ingest(wiki_repo, "Local")
+
+    _publish(message="ingest: Local", path=[PROTOCOLS])
+
+    check = _check_tree(wiki_repo)
+    assert (check / f"{PROTOCOLS}/Local.md").is_file()
+    assert _logged_titles(check) == ["Base", "Writer", "Local"]
+    _assert_generated_current(check)
+    assert _status(wiki_repo) == ""
+    out = capsys.readouterr().out
+    assert "generated file(s) outside --path" in out
+    assert "kb/index.md" in out and "kb/log.md" in out
+
+
+def test_publish_path_commits_the_generated_files_without_a_reconcile_too(wiki_repo, capsys):
+    """Nothing came in, so nothing was regenerated - the catalog and log the ingest itself wrote
+    still go out with its page, not one batch later."""
+    _ingest(wiki_repo, "Local")
+
+    _publish(message="ingest: Local", path=[PROTOCOLS])
+
+    check = _check_tree(wiki_repo)
+    assert _logged_titles(check) == ["Base", "Local"]
+    _assert_generated_current(check)
+    assert _status(wiki_repo) == ""
+    assert "generated file(s) outside --path" in capsys.readouterr().out
+
+
+def test_publish_path_after_sync_commits_what_the_sync_regenerated(wiki_repo):
+    writer = _clone_writer(wiki_repo)
+    _writer_ingest(writer, "Writer")
+    _ingest(wiki_repo, "Local")
+    _sync()
+
+    _publish(message="ingest: Local", path=[PROTOCOLS])
+
+    check = _check_tree(wiki_repo)
+    assert _logged_titles(check) == ["Base", "Writer", "Local"]
+    _assert_generated_current(check)
+    assert _status(wiki_repo) == ""
+
+
+def test_publish_path_leaves_every_other_file_outside_it_alone(wiki_repo):
+    _ingest(wiki_repo, "Local")
+    (wiki_repo / "README.md").write_text("init\nedited, not in scope\n", encoding="utf-8")
+
+    _publish(message="ingest: Local", path=[PROTOCOLS])
+
+    committed = _git(wiki_repo, "show", "--name-only", "--format=", "HEAD").stdout.split()
+    assert "README.md" not in committed and "kb/index.md" in committed
+    assert _status(wiki_repo) == " M README.md\n"
+
+
+def test_publish_path_names_nothing_when_the_generated_files_are_already_in_scope(
+    wiki_repo, capsys
+):
+    writer = _clone_writer(wiki_repo)
+    _writer_ingest(writer, "Writer")
+    _ingest(wiki_repo, "Local")
+
+    _publish(message="ingest: Local", path=["kb"])
+
+    check = _check_tree(wiki_repo)
+    assert _logged_titles(check) == ["Base", "Writer", "Local"]
+    _assert_generated_current(check)
+    assert _status(wiki_repo) == ""
+    assert "outside --path" not in capsys.readouterr().out
+
+
+def test_publish_path_keeps_its_token_and_rerun_line_when_generated_files_join(
+    wiki_repo, monkeypatch
+):
+    """The added files are never counted, so the clearance a user gives for `--path <X>` is the
+    one the gate asks for - with or without them - and it publishes."""
+    from chemenu.telemetry import reader
+
+    writer = _clone_writer(wiki_repo)
+    _writer_ingest(writer, "Writer")
+    for i in range(9):
+        _write_concept(wiki_repo, f"Local{i}")
+    _ingest(wiki_repo, "Local")
+    token_before = _token_for(wiki_repo, paths=[PROTOCOLS])
+    shown = {}
+    real_message = git_publish.clearance_message
+
+    def spy(changes, threshold, token, rerun, *args, **kwargs):
+        shown.update(changes=[c.path for c in changes], rerun=rerun)
+        return real_message(changes, threshold, token, rerun, *args, **kwargs)
+
+    monkeypatch.setattr(git_publish, "clearance_message", spy)
+
+    with pytest.raises(typer.Exit) as excinfo:
+        _publish(message="ingest: Local", path=[PROTOCOLS])
+
+    assert excinfo.value.exit_code == EXIT_NEEDS_CLEARANCE
+    assert "kb/index.md" in shown["changes"]  # the generated files did join the changeset
+    refused = [r for r in reader.read_trace("test-session") if r["event"] == "gate.refused"]
+    token = refused[-1]["attrs"]["token"]
+    assert token == token_before == _token_for(wiki_repo, paths=[PROTOCOLS])
+    assert shown["rerun"] == rerun_command(token, "ingest: Local", True, 10, "origin", "main",
+                                           [PROTOCOLS])
+
+    _publish(message="ingest: Local", path=[PROTOCOLS], confirm=token)
+
+    check = _check_tree(wiki_repo)
+    assert _logged_titles(check) == ["Base", "Writer", "Local"]
+    _assert_generated_current(check)
+    assert _status(wiki_repo) == ""