diff --git a/CHANGES.md b/CHANGES.md index 508f049..b02f273 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -59,7 +59,7 @@ concern - readable here, never shipped as something to parse. --- -## 8.0.0-beta.47 - 2026-10-06 - sync record: the autostash note states the behaviour, not the change +## 8.0.0-beta.48 - 2026-10-06 - publish --path nimmt ungespeicherte generierte Dateien außerhalb des Pfads mit **Author:** Torben Nehmer @@ -112,6 +112,7 @@ concern - readable here, never shipped as something to parse. - wiki-ingest: updating the captured repositories is a step-1 branch over raw status - export guidelines: the guideline pages as a generated GUIDELINES.md, pushed into the captured repositories behind the Guideline Push Gate - publish/sync merge generated files mechanically and carry non-overlapping uncommitted work through a rebase +- publish --path nimmt ungespeicherte generierte Dateien außerhalb des Pfads mit **Low impact** - version bump no longer points at version release in its output @@ -160,6 +161,32 @@ concern - readable here, never shipped as something to parse. - sync record: the autostash note states the behaviour, not the change +### publish --path nimmt ungespeicherte generierte Dateien außerhalb des Pfads mit + +`publish --path ` staged and committed only ``. When the catalog and log it had written - +or the ones a reconcile had just merged and regenerated - lay outside ``, the push carried +the new page with a catalog that did not list it and a log without its entries, and left the +generated files behind uncommitted until a `publish` without `--path` took them along. Nothing was +lost, but the remote stayed inconsistent in between, and another session's `sync` worked against +that state. + +- **Uncommitted generated files outside `--path` now join the commit** (`kb/index.md`, + `kb/log.md`, `kb/provenance.md`, every `kb/**/INDEX.md`), and a line names them: + `Including N generated file(s) outside --path: ...`. Git decides what lies under ``, by the + same pathspec the staging uses; files already under it are neither added nor named. +- **Regardless of whether this run's reconcile regenerated anything.** A run the Mass-Update + Gate refuses has already reconciled; the confirmed re-run finds nothing left to regenerate, + and a condition on the regeneration would have pushed the inconsistent state on exactly that + path - the same goes for a `sync` followed by `publish --path`. +- **The gate is unchanged.** The added files stay out of its count, and the `--confirm` token + and the re-run line cover the `--path` given, so a clearance issued for `` stays valid. +- **Every other file outside `--path`** is left alone, as before. + +Where uncommitted pages also lie outside ``, the catalog that goes out with `` already +lists them until the next `publish` sends them - the reverse of before, where the page went out +and the catalog did not know it. Drop-in: no command, flag, file format or state file changes. +`publish`'s record follows (Gitea #182). + ### sync record: the autostash note states the behaviour, not the change One note in `sync`'s command record, written with 8.0.0-beta.46, said that uncommitted changes no diff --git a/VERSION b/VERSION index 21429a9..2798d14 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -8.0.0-beta.47 +8.0.0-beta.48 diff --git a/tools/CONTRACT.md b/tools/CONTRACT.md index 250342c..f51d3a8 100644 --- a/tools/CONTRACT.md +++ b/tools/CONTRACT.md @@ -1993,7 +1993,7 @@ Reconcile with `/`, then stage all changes, commit, and push. - Order: branch check and Publish-Remote Gate, then the reconcile with `/`, then the Mass-Update Gate, then `git add -A`, commit and push. `--no-push` skips all but the Mass-Update Gate and the commit. - Without `--no-push`, a remote that is not configured or cannot be reached ends the call with exit 1 at the reconcile - before the gate, `git add` and the commit, and also on a clean tree. Nothing is committed, the index and the working tree are unchanged, and the message names `--no-push` as the way to a local commit. The next `publish` that reaches the remote pushes that commit along with whatever is new. A remote that answers but has no `` yet (a new, empty repository) is not this case: the first publish of an instance commits and pushes as before. - Reconcile: fetches `/`, fast-forwards when only the remote moved, rebases the local commits on top when both sides moved but touched disjoint files, and exits 42 (rebase-review gate) when both sides touched the same file. A refused reconcile performs no rebase attempt. The `--confirm-rebase` token covers the exact upstream state and the set of files touched on both sides. -- Generated files are never an overlap: where they are all that stops the reconcile, it merges them as `sync` does (both sides' log entries kept, the catalog and `kb/provenance.md` regenerated) and carries uncommitted changes no incoming commit touches through the rebase. The proactive reconcile runs before staging, so this publish commits the regenerated files; on the retry after a rejected push they get a commit of their own before the second push. +- Generated files are never an overlap: where they are all that stops the reconcile, it merges them as `sync` does (both sides' log entries kept, the catalog and `kb/provenance.md` regenerated) and carries uncommitted changes no incoming commit touches through the rebase. The proactive reconcile runs before staging, so this publish commits the regenerated files - with `--path` too, see there; on the retry after a rejected push they get a commit of their own before the second push. - The push target must be the checked-out branch; this is checked before anything is staged. The unborn branch of a fresh `git init -b main` counts as checked out, so the first publish of a new instance works; a real detached HEAD is refused. - With nothing new to stage, a local commit the remote lacks is still pushed: one left behind by an earlier publish whose push failed, or every commit when the remote answers but does not have the branch yet (a new, empty remote repository). - A rejected push that finds the remote unreachable on its one retry reports the original push error. @@ -2002,7 +2002,7 @@ Reconcile with `/`, then stage all changes, commit, and push. - Never counted and never shown for approval, but committed like everything else: anything under `work/`, and the files `wikitool` generates itself (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md` under `kb/`). The refusal line accounts for both, by reason. - The `--confirm` token covers each counted path, the blob id of its contents and the publish target: a different file list or edited contents need a new clearance. - Publish-Remote Gate: when the checkout carries `.wikitool-remotes.json` and the push URL of `--remote` is not listed in it, exits 42 before the reconcile fetches anything. The URL is read with `git remote get-url --push`, so a repointed remote does not pass on its name. An absent file means unrestricted; a malformed one is an error, not permission. -- `--path` (repeatable) scopes the whole operation - gate count, staging and commit - to that subtree. +- `--path` (repeatable) scopes the whole operation - gate count, staging and commit - to that subtree, with one exception: uncommitted generated files outside `--path` (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md` under `kb/`) are staged and committed too, and a line names them - whether a reconcile or the change itself wrote them. They stay out of the gate count, and the `--confirm` token and the re-run line cover the `--path` given. Every other file outside `--path` is left as it is. - Commit message: `--message`, then a `Files changed:` paragraph listing every committed path. When `--message` ends in a paragraph git reads as a trailer block (`Co-Authored-By:` and the like), that block stays the last paragraph and the list goes in front of it, since git reads trailers from the last paragraph only. `git interpret-trailers` decides whether there is such a block, and the list moves only when git reads the same trailers from the result; otherwise it is appended at the end. `--message` is not part of any gate token. - After a successful commit or push whose changed files include `tools/`, `types/`, `instructions/`, `AGENTS.md` or a path ending in `CONTRACT.md`, prints one reminder line: the phase past this point (an issue-body rewrite, `docs/` staleness, a changelog entry's accuracy) is not covered by `docs verify`, `instructions verify` or `pytest`. It is not a gate: no exit code change, nothing to clear, and silent for an ordinary content publish. diff --git a/tools/chemenu/commands/git_publish.py b/tools/chemenu/commands/git_publish.py index 6d0904f..ba7607f 100644 --- a/tools/chemenu/commands/git_publish.py +++ b/tools/chemenu/commands/git_publish.py @@ -816,10 +816,11 @@ def _git_bytes(*args: str) -> Optional[bytes]: return result.stdout if result.returncode == 0 else None -def _dirty_paths() -> tuple[list[str], list[str]]: - """(tracked paths whose index or working copy differs from HEAD, untracked paths). Ignored - files are neither - nothing here ever touches one.""" - result = _run(["git", "status", "--porcelain=v1", "-z", "--untracked-files=all", "--no-renames"]) +def _dirty_paths(pathspec: tuple[str, ...] = ()) -> tuple[list[str], list[str]]: + """(tracked paths whose index or working copy differs from HEAD, untracked paths), limited + to `pathspec` when one is given. Ignored files are neither - nothing here ever touches one.""" + result = _run(["git", "status", "--porcelain=v1", "-z", "--untracked-files=all", "--no-renames", + "--", *pathspec]) tracked: list[str] = [] untracked: list[str] = [] for record in result.stdout.split("\0"): @@ -1466,6 +1467,14 @@ def commit_message(message: str, changed_files: list[str]) -> str: return candidate if _trailers(candidate) == trailers else plain +def generated_outside(paths: list[str]) -> list[str]: + """The uncommitted generated files `git add -A -- ` would leave behind. Git decides + what lies under `paths`, by the same pathspec the staging uses.""" + tracked, untracked = _dirty_paths() + inside = set(sum(_dirty_paths(tuple(paths)), [])) + return sorted(path for path in tracked + untracked if is_generated(path) and path not in inside) + + def commit_regenerated(remote: str, branch: str) -> None: """Commit the generated files a reconcile just rewrote - and only those, whatever else is staged - with a message of their own.""" @@ -1522,8 +1531,8 @@ def commit_regenerated(remote: str, branch: str) -> None: "merges them as `sync` does (both sides' log entries kept, the catalog and " "`kb/provenance.md` regenerated) and carries uncommitted changes no incoming commit " "touches through the rebase. The proactive reconcile runs before staging, so this " - "publish commits the regenerated files; on the retry after a rejected push they get a " - "commit of their own before the second push.", + "publish commits the regenerated files - with `--path` too, see there; on the retry " + "after a rejected push they get a commit of their own before the second push.", "The push target must be the checked-out branch; this is checked before anything is " "staged. The unborn branch of a fresh `git init -b main` counts as checked out, so the " "first publish of a new instance works; a real detached HEAD is refused.", @@ -1555,7 +1564,12 @@ def commit_regenerated(remote: str, branch: str) -> None: "on its name. An absent file means unrestricted; a malformed one is an error, not " "permission.", "`--path` (repeatable) scopes the whole operation - gate count, staging and commit - " - "to that subtree.", + "to that subtree, with one exception: uncommitted generated files outside `--path` " + "(`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md` under `kb/`) are " + "staged and committed too, and a line names them - whether a reconcile or the change " + "itself wrote them. They stay out of the gate count, and the `--confirm` token and the " + "re-run line cover the `--path` given. Every other file outside `--path` is left as it " + "is.", "Commit message: `--message`, then a `Files changed:` paragraph listing every committed " "path. When `--message` ends in a paragraph git reads as a trailer block " "(`Co-Authored-By:` and the like), that block stays the last paragraph and the list " @@ -1693,6 +1707,9 @@ def publish_command( fail(YES_REMOVED_MESSAGE) paths = list(path or []) + # What is staged and committed: `paths` plus any uncommitted generated file outside them. + # The gate token and its re-run line keep `paths`. + staging = paths # Checked before anything is staged, for the same reason as the gate below: # a refused publish must leave the working tree exactly as it was found. @@ -1737,9 +1754,20 @@ def publish_command( if summary: typer.echo(summary) + # `--path` would otherwise push a page with a catalog that does not list it and a log + # without its entries, and leave the generated files behind uncommitted. They carry no + # decision - the gate leaves them out of its count for the same reason - so they join the + # commit rather than stop it. Not only after a regenerating reconcile: a run the gate + # refused has already reconciled, and the confirmed re-run finds nothing left to regenerate. + if paths: + extra = generated_outside(paths) + if extra: + staging = [*paths, *extra] + typer.echo(f"Including {len(extra)} generated file(s) outside --path: {', '.join(extra)}") + # The gate is evaluated *before* anything is staged, so a refused publish # leaves the working tree exactly as it was found. - changes = collect_changes(paths) + changes = collect_changes(staging) local_ahead = push and _local_ahead_of_remote(remote, branch) if not changes: if not local_ahead: @@ -1790,7 +1818,7 @@ def publish_command( ) if changes: - add_result = _run(["git", "add", "-A", "--", *paths]) + add_result = _run(["git", "add", "-A", "--", *staging]) if add_result.returncode != 0: fail(f"git add failed:\n{add_result.stderr}") @@ -1799,8 +1827,8 @@ def publish_command( # With a pathspec, `git commit -- ` commits exactly those paths and # ignores anything else that happens to be staged, so batches stay disjoint. commit_args = ["git", "commit", "-m", full_message] - if paths: - commit_args += ["--", *paths] + if staging: + commit_args += ["--", *staging] commit_result = _run(commit_args) if commit_result.returncode != 0: fail(f"git commit failed:\n{commit_result.stderr}") diff --git a/tools/chemenu/tests/test_git_publish.py b/tools/chemenu/tests/test_git_publish.py index 3156827..81a62ca 100644 --- a/tools/chemenu/tests/test_git_publish.py +++ b/tools/chemenu/tests/test_git_publish.py @@ -1898,3 +1898,123 @@ def test_an_index_md_under_raw_still_goes_to_review_when_both_sides_change_it(re assert outcome.status == "needs-review" assert outcome.overlap_files == ["raw/repos/project/INDEX.md"] + + +# --- publish --path and the uncommitted generated files outside it --- + +PROTOCOLS = "kb/concepts/protocols" + + +def test_publish_path_commits_the_generated_files_the_reconcile_regenerated(wiki_repo, capsys): + """The pushed tip is consistent: its catalog and provenance are what a rebuild of it writes, + and its log carries every local entry once. Nothing generated stays behind uncommitted.""" + writer = _clone_writer(wiki_repo) + _writer_ingest(writer, "Writer") + _ingest(wiki_repo, "Local") + + _publish(message="ingest: Local", path=[PROTOCOLS]) + + check = _check_tree(wiki_repo) + assert (check / f"{PROTOCOLS}/Local.md").is_file() + assert _logged_titles(check) == ["Base", "Writer", "Local"] + _assert_generated_current(check) + assert _status(wiki_repo) == "" + out = capsys.readouterr().out + assert "generated file(s) outside --path" in out + assert "kb/index.md" in out and "kb/log.md" in out + + +def test_publish_path_commits_the_generated_files_without_a_reconcile_too(wiki_repo, capsys): + """Nothing came in, so nothing was regenerated - the catalog and log the ingest itself wrote + still go out with its page, not one batch later.""" + _ingest(wiki_repo, "Local") + + _publish(message="ingest: Local", path=[PROTOCOLS]) + + check = _check_tree(wiki_repo) + assert _logged_titles(check) == ["Base", "Local"] + _assert_generated_current(check) + assert _status(wiki_repo) == "" + assert "generated file(s) outside --path" in capsys.readouterr().out + + +def test_publish_path_after_sync_commits_what_the_sync_regenerated(wiki_repo): + writer = _clone_writer(wiki_repo) + _writer_ingest(writer, "Writer") + _ingest(wiki_repo, "Local") + _sync() + + _publish(message="ingest: Local", path=[PROTOCOLS]) + + check = _check_tree(wiki_repo) + assert _logged_titles(check) == ["Base", "Writer", "Local"] + _assert_generated_current(check) + assert _status(wiki_repo) == "" + + +def test_publish_path_leaves_every_other_file_outside_it_alone(wiki_repo): + _ingest(wiki_repo, "Local") + (wiki_repo / "README.md").write_text("init\nedited, not in scope\n", encoding="utf-8") + + _publish(message="ingest: Local", path=[PROTOCOLS]) + + committed = _git(wiki_repo, "show", "--name-only", "--format=", "HEAD").stdout.split() + assert "README.md" not in committed and "kb/index.md" in committed + assert _status(wiki_repo) == " M README.md\n" + + +def test_publish_path_names_nothing_when_the_generated_files_are_already_in_scope( + wiki_repo, capsys +): + writer = _clone_writer(wiki_repo) + _writer_ingest(writer, "Writer") + _ingest(wiki_repo, "Local") + + _publish(message="ingest: Local", path=["kb"]) + + check = _check_tree(wiki_repo) + assert _logged_titles(check) == ["Base", "Writer", "Local"] + _assert_generated_current(check) + assert _status(wiki_repo) == "" + assert "outside --path" not in capsys.readouterr().out + + +def test_publish_path_keeps_its_token_and_rerun_line_when_generated_files_join( + wiki_repo, monkeypatch +): + """The added files are never counted, so the clearance a user gives for `--path ` is the + one the gate asks for - with or without them - and it publishes.""" + from chemenu.telemetry import reader + + writer = _clone_writer(wiki_repo) + _writer_ingest(writer, "Writer") + for i in range(9): + _write_concept(wiki_repo, f"Local{i}") + _ingest(wiki_repo, "Local") + token_before = _token_for(wiki_repo, paths=[PROTOCOLS]) + shown = {} + real_message = git_publish.clearance_message + + def spy(changes, threshold, token, rerun, *args, **kwargs): + shown.update(changes=[c.path for c in changes], rerun=rerun) + return real_message(changes, threshold, token, rerun, *args, **kwargs) + + monkeypatch.setattr(git_publish, "clearance_message", spy) + + with pytest.raises(typer.Exit) as excinfo: + _publish(message="ingest: Local", path=[PROTOCOLS]) + + assert excinfo.value.exit_code == EXIT_NEEDS_CLEARANCE + assert "kb/index.md" in shown["changes"] # the generated files did join the changeset + refused = [r for r in reader.read_trace("test-session") if r["event"] == "gate.refused"] + token = refused[-1]["attrs"]["token"] + assert token == token_before == _token_for(wiki_repo, paths=[PROTOCOLS]) + assert shown["rerun"] == rerun_command(token, "ingest: Local", True, 10, "origin", "main", + [PROTOCOLS]) + + _publish(message="ingest: Local", path=[PROTOCOLS], confirm=token) + + check = _check_tree(wiki_repo) + assert _logged_titles(check) == ["Base", "Writer", "Local"] + _assert_generated_current(check) + assert _status(wiki_repo) == ""