feat: raw capture / raw status / --replaces-bundle - documentation from git repositories as a bundle, with drift reporting (#177)

New repo_capture module: resolve a branch or tag-pattern ref rule, fetch it
shallowly by name into a bare cache, read the glob-selected files as blobs,
and record repo/ref/commit/globs/capture fields in _capture.json. raw status
reports changed captured bundles as A/M/D; raw accept --replaces-bundle swaps
a captured bundle for its new edition at the same address. iter_raw_files now
skips _capture.json and anchors the CONTRACT.md exclusion to raw/CONTRACT.md.

Files changed:
- .gitignore
- CHANGES.md
- README.md
- VERSION
- instructions/wiki-ingest/SKILL.md
- raw/CONTRACT.md
- tools/CONTRACT.md
- tools/README.md
- tools/chemenu/cli_contract.py
- tools/chemenu/commands/dist_cmd.py
- tools/chemenu/commands/docs_verify.py
- tools/chemenu/commands/raw_cmd.py
- tools/chemenu/config.py
- tools/chemenu/repo_capture.py
- tools/chemenu/tests/test_cli.py
- tools/chemenu/tests/test_raw_capture.py

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2

tools/CONTRACT.md

@@ -115,6 +115,8 @@ sources coverage      read  idempotent     budget:counted              exit:0
 sources trace         read  idempotent     budget:counted              exit:0,1    Trace provenance in either direction: raw file, or page.
 sources rebuild-index write idempotent     budget:counted              exit:0,1    Regenerate the `kb/provenance.md` reverse index.
 raw fetch             write non-idempotent budget:counted              exit:0,1    Capture a web page the user names into `incoming/`: the HTML as received plus a derived text, for `raw accept` to promote.
+raw capture           write non-idempotent budget:counted              exit:0,1    Capture documentation from a git repository into `incoming/<bundle>/`, with a manifest naming the repository, ref rule and commit, for `raw accept` to promote.
+raw status            read  idempotent     budget:counted              exit:0      Report which captured bundles under `raw/` have fallen behind their repository, with each changed file as `A`/`M`/`D`.
 raw pending           read  idempotent     budget:counted              exit:0      List what waits in `incoming/`, oldest first, and name the entry an ingest without an argument takes next.
 raw accept            write non-idempotent budget:counted              exit:0,1    Promote one or more files, or one folder, from `incoming/` into `raw/`.
 upload list           read  idempotent     budget:counted              exit:0      List every MCP submission currently waiting in the quarantine (`mcp-upload/`).
@@ -1455,6 +1457,120 @@ Capture a web page the user names into `incoming/`: the HTML as received plus a
 - `wikitool raw accept` - promotes the written files into `raw/`
 - `instructions/wiki-ingest/SKILL.md` - where a URL to ingest starts
 
+#### `raw capture`
+
+Capture documentation from a git repository into `incoming/<bundle>/`, with a manifest naming the repository, ref rule and commit, for `raw accept` to promote.
+
+**SYNOPSIS**
+
+- `wikitool raw capture <repo-url> --ref <branch|tag-pattern> --path <glob> [--path <glob> ...] --name <bundle> --fidelity <v> --authority <v>` - First capture of a repository
+- `wikitool raw capture --update <raw-bundle> [--fidelity <v>] [--authority <v>]` - Capture the current state of an already-accepted bundle, for `raw accept --replaces-bundle`
+
+**PROPERTIES**
+
+- effect: write
+- idempotent: no
+- atomic: Yes for what it leaves in `incoming/` - the bundle is written into a hidden staging directory and renamed into place in one step; a failure removes the staging directory. The git cache under `tools/.wikitool_capture/` is updated either way
+- budget: counted
+- network: yes
+
+**EXAMPLES**
+
+- `tools/wikitool raw capture ssh://git@example.org/team/service.git --ref main --path 'docs/**/*.md' --path README.md --name service-docs --fidelity verbatim --authority normative`
+- `tools/wikitool raw capture https://example.org/team/lib.git --ref 'v*' --path docs --name lib-docs --fidelity verbatim --authority reporting`
+- `tools/wikitool raw capture --update raw/2026/10/service-docs`
+
+**EXIT STATUS**
+
+- 0    success
+- 1    The URL is not `ssh://`, `https://` or `user@host:path`, or carries a password; the `--ref` rule, a `--path` glob or `--name` is unusable; or an argument of the other variant was mixed in
+- 1    `--fidelity`/`--authority` is missing on a first capture, or names `unknown` or a value outside the schema's enum
+- 1    The repository could not be reached, asked for credentials, or timed out; or no branch or tag matches `--ref`
+- 1    `incoming/<bundle>` already exists, the name is already taken under `raw/`, nothing matches the globs, or a path would be over the budget
+- 1    raw capture --update: The path is not a bundle under `raw/` with a readable `_capture.json`, or its manifest names a URL that is refused
+
+**ON FAILURE**
+
+- The URL is not `ssh://`, `https://` or `user@host:path`, or carries a password; the `--ref` rule, a `--path` glob or `--name` is unusable; or an argument of the other variant was mixed in -> Fix the call and retry once. A credential goes into git's credential helper, never into the URL
+- `--fidelity`/`--authority` is missing on a first capture, or names `unknown` or a value outside the schema's enum -> Pass both with a valid value, then retry once
+- The repository could not be reached, asked for credentials, or timed out; or no branch or tag matches `--ref` -> Nothing was written to `incoming/`. Check the URL, the ref rule and the host's git credentials with the user; an unreachable host may be retried once
+- `incoming/<bundle>` already exists, the name is already taken under `raw/`, nothing matches the globs, or a path would be over the budget -> Nothing was written. Accept or remove what is in `incoming/`; for a name taken by a captured bundle use `--update`; widen or narrow the globs; then retry once
+- raw capture --update: The path is not a bundle under `raw/` with a readable `_capture.json`, or its manifest names a URL that is refused -> Show the message to the user - a manifest under `raw/` is never edited by hand to make this pass
+
+**NEVER**
+
+- Never put a password or token into the repository URL.
+- Never edit a `_capture.json`, or the files of a captured bundle, by hand.
+- Never capture a repository the user has not named in this session.
+
+**NOTES**
+
+- Writes into `incoming/<bundle>/` only, never into `raw/`: `raw accept incoming/<bundle>` promotes it, as one folder, to `raw/<YYYY>/<MM>/<bundle>/`.
+- `--ref` is a branch name (`main`) or a tag pattern (`v*`, any of `*?[`), which names the newest matching tag by version order. It resolves to one commit - for an annotated tag, the commit it points at.
+- The commit is fetched by ref name, shallowly, into a bare cache repository per URL under `tools/.wikitool_capture/` (gitignored). Files are read from it as blobs, never through a checkout, so they are byte-identical to the repository's - no line-ending conversion, no filter.
+- `--path` globs (repeatable) select files by their path in the repository, as git's `:(glob)` pathspec does: `*`, `?` and `[...]` do not cross `/`; `**` as a whole segment spans any number of directories; a glob without wildcards also takes everything below it as a directory. Files keep their repository paths inside the bundle.
+- Never captured, each named in the output with its reason: a file whose first line starts with `<!-- wikitool:export`, a symlink, a submodule, a file over 25 MiB, a path with a segment starting with `.`, a file named `_capture.json`, and a Git LFS pointer.
+- `incoming/<bundle>/_capture.json` records `schema`, `repo`, `ref`, `commit`, `paths`, `captured` (UTC), `fidelity`, `authority` and `files`. It is the only declaration that and how this instance follows the repository.
+- Only `ssh://`, `https://` and the scp form `user@host:path`; a URL with a password or token in it is refused, since the manifest is committed. Git runs with the host's own credentials and configuration, restricted to those protocols, and never prompts: a repository that asks for credentials is reported as unreachable. Each git call times out after 120 s.
+- `--update <raw-bundle>` takes URL, ref rule and globs from that bundle's manifest and writes the current state to `incoming/<same bundle name>/`; `--fidelity`/`--authority` default to the manifest's values, and passing one is the way to correct it.
+- Refused before anything is written: `incoming/<bundle>` already exists, the name is already taken under `raw/` (a first capture), nothing matches the globs, or a target path - measured at its later place under `raw/` - is over the path budget.
+- Success prints the commit and the `raw accept` line that comes next.
+
+**SEE ALSO**
+
+- `raw/CONTRACT.md` "Getting a repository in: `raw capture`" - the rules and why
+- `wikitool raw accept` - promotes the bundle; `--replaces-bundle` for a new edition
+- `wikitool raw status` - which captured bundles have fallen behind their repository
+
+#### `raw status`
+
+Report which captured bundles under `raw/` have fallen behind their repository, with each changed file as `A`/`M`/`D`.
+
+**SYNOPSIS**
+
+- `wikitool raw status [--json]`
+
+**PROPERTIES**
+
+- effect: read
+- idempotent: yes
+- atomic: Read-only for the tree - writes nothing but the git cache under `tools/.wikitool_capture/`
+- budget: counted
+- network: yes
+
+**EXAMPLES**
+
+- `tools/wikitool raw status`
+- `tools/wikitool raw status --json`
+
+**EXIT STATUS**
+
+- 0    success
+- 0    A repository could not be reached, its manifest or URL was refused, or no ref matches its rule
+
+**ON FAILURE**
+
+- A repository could not be reached, its manifest or URL was refused, or no ref matches its rule -> Reported on that bundle's line, the others are still checked; check the URL and the host's git credentials with the user
+
+**NEVER**
+
+- Never edit a `_capture.json` to silence a line - a new edition goes through `raw capture --update` and `raw accept --replaces-bundle`.
+
+**NOTES**
+
+- Reads every `_capture.json` under `raw/` and resolves its ref rule with `git ls-remote`. Only where the commit moved is the new commit fetched, and compared - with the same globs and exclusions `raw capture` applies - against the bundle's files under `raw/`.
+- A bundle whose commit moved without a change inside its globs is reported as unchanged, not listed. A tag pattern follows new matching tags only, never new commits on a branch.
+- Each changed bundle prints `old -> new` and its files as `A`/`M`/`D`, grouped by owning source page with its citing pages, followed by the `raw capture --update` and `raw accept --replaces-bundle` lines that take the new edition in.
+- An unreachable repository, a refused URL or an unreadable manifest is one line beside the others, never an abort; the exit status is 0 as long as the command itself ran.
+- `--json` prints one object per bundle: `bundle`, `repo`, `ref`, `old`, `new`, `changed`, `files` (`[{path, status}]`), `error` (or null).
+- Git runs exactly as for `raw capture`: the host's credentials, only `ssh`/`https`, never a prompt, 120 s per call.
+
+**SEE ALSO**
+
+- `wikitool raw capture` - `--update` captures the new edition
+- `wikitool raw accept` - `--replaces-bundle` takes it in
+- `raw/CONTRACT.md` "Getting a repository in: `raw capture`"
+
 #### `raw pending`
 
 List what waits in `incoming/`, oldest first, and name the entry an ingest without an argument takes next.
@@ -1504,12 +1620,13 @@ Promote one or more files, or one folder, from `incoming/` into `raw/`.
 - `wikitool raw accept <file> [<file> ...] --fidelity <v> --authority <v> [--page "<Title>"] [--dry-run]` - Promote one or more files from `incoming/` into today's `raw/<YYYY>/<MM>/` shard
 - `wikitool raw accept incoming/<folder> --fidelity <v> --authority <v> [--dry-run]` - Promote a whole folder as one source, its structure kept, into `raw/<YYYY>/<MM>/<folder>/`
 - `wikitool raw accept <file> --replaces <raw-path> [--fidelity <v>] [--authority <v>] [--dry-run]` - Overwrite one existing raw file in place with a new edition
+- `wikitool raw accept incoming/<bundle> --replaces-bundle <raw-bundle> [--dry-run]` - Replace a captured bundle as a whole with the new edition `raw capture --update` wrote
 
 **PROPERTIES**
 
 - effect: write
 - idempotent: no
-- atomic: `raw accept`: No - one filesystem move per file, then (with `--page`) one page write. With a folder: No - one move per file, then one `rmdir` per emptied directory; a half-accepted folder is not resumed. `raw accept --replaces`: No - one `unlink()` + one `rename()`, plus (if `--fidelity`/`--authority` was given) one page write
+- atomic: `raw accept`: No - one filesystem move per file, then (with `--page`) one page write. With a folder: No - one move per file, then one `rmdir` per emptied directory; a half-accepted folder is not resumed. `raw accept --replaces`: No - one `unlink()` + one `rename()`, plus (if `--fidelity`/`--authority` was given) one page write. `raw accept --replaces-bundle`: No - one `unlink()` per removed file, one `rename()` per added or modified file, then the manifest, then one `rmdir` per emptied directory, plus one page write per owning page whose capture fields change; a half-replaced bundle is not resumed
 - budget: counted
 - network: no
 
@@ -1519,6 +1636,7 @@ Promote one or more files, or one folder, from `incoming/` into `raw/`.
 - `tools/wikitool raw accept incoming/part-2.md --fidelity verbatim --authority reporting --page "Source - Docker Cheatsheet"`
 - `tools/wikitool raw accept incoming/projekt-x --fidelity verbatim --authority reporting`
 - `tools/wikitool raw accept incoming/cluster.md --replaces raw/documents/cluster.md`
+- `tools/wikitool raw accept incoming/chemenu --replaces-bundle raw/2026/10/chemenu`
 
 **EXIT STATUS**
 
@@ -1531,6 +1649,9 @@ Promote one or more files, or one folder, from `incoming/` into `raw/`.
 - 1    raw accept --replaces: More than one incoming file, or `--page` also given
 - 1    raw accept --replaces: The incoming file does not exist or is not directly in `incoming/`, its filename differs from the target's, or the target does not lie under `raw/` or does not exist
 - 1    raw accept --replaces: `--fidelity`/`--authority` names `unknown` or a value outside the schema's enum, or the target has more than one owning source page
+- 1    raw accept --replaces / --page: The target, or a file the page already has, lies in a captured bundle
+- 1    raw accept incoming/<bundle>: A captured folder is given `--fidelity`/`--authority`, its `_capture.json` cannot be read or names an invalid value, or its files differ from the ones the manifest lists
+- 1    raw accept --replaces-bundle: Either side has no `_capture.json`, the two manifests name different repositories, the folder's name differs from the bundle's, `--page`, `--replaces`, `--fidelity` or `--authority` was also given, the folder does not match its manifest, a new path is over the budget, or a file of the bundle has more than one owning source page
 
 **ON FAILURE**
 
@@ -1542,6 +1663,9 @@ Promote one or more files, or one folder, from `incoming/` into `raw/`.
 - raw accept --replaces: More than one incoming file, or `--page` also given -> A replacement is one file for one file - fix the call and retry once
 - raw accept --replaces: The incoming file does not exist or is not directly in `incoming/`, its filename differs from the target's, or the target does not lie under `raw/` or does not exist -> Fix the named argument and retry once - every check runs before the filesystem is touched, so both files are exactly as they were
 - raw accept --replaces: `--fidelity`/`--authority` names `unknown` or a value outside the schema's enum, or the target has more than one owning source page -> Fix the named argument and retry once; nothing was touched
+- raw accept --replaces / --page: The target, or a file the page already has, lies in a captured bundle -> Not fixed by retrying: a captured bundle changes only as a whole - `raw capture --update <raw-bundle>`, then `raw accept --replaces-bundle`
+- raw accept incoming/<bundle>: A captured folder is given `--fidelity`/`--authority`, its `_capture.json` cannot be read or names an invalid value, or its files differ from the ones the manifest lists -> Nothing moved. Drop the two flags and retry once; a manifest that does not match its folder is fixed by removing the folder and capturing it again, never by editing either
+- raw accept --replaces-bundle: Either side has no `_capture.json`, the two manifests name different repositories, the folder's name differs from the bundle's, `--page`, `--replaces`, `--fidelity` or `--authority` was also given, the folder does not match its manifest, a new path is over the budget, or a file of the bundle has more than one owning source page -> Nothing under `raw/` or in `incoming/` changed. Fix what the message names and retry once; different repositories or a different name are a separate source, not a new edition - show the message to the user
 
 **NEVER**
 
@@ -1563,12 +1687,17 @@ Promote one or more files, or one folder, from `incoming/` into `raw/`.
 - `--replaces` refuses a target with more than one owning source page; with none, it replaces anyway and says so. It cannot be combined with `--page` or with more than one incoming file.
 - `--replaces` prints the source page (if any) and its citing pages, so their update lands in the same commit as the replacement.
 - A file already at its computed destination is what "already exists" reports, not a partial prior run to resume - safe to retry as-is once a cause is fixed.
+- A folder carrying `_capture.json` at its top is a captured bundle (`raw capture`): `--fidelity`/`--authority` come from that manifest only and are refused on the command line; the folder must hold exactly the files the manifest lists. `_capture.json` goes in no `raw_files:`.
+- `--replaces-bundle <raw-bundle>` replaces a captured bundle under `raw/` with the new edition in `incoming/<bundle>` at the same address: files gone from the new edition are removed, emptied directories `rmdir`ed, and afterwards the bundle holds exactly the new manifest's files plus `_capture.json`. `raw_files:` is left untouched; an owning source page's `fidelity`/`authority` is overwritten where the new manifest differs.
+- `--replaces-bundle` prints each changed file as `A`/`M`/`D`, grouped by owning source page with its citing pages, and the `touch --page ... --add/--remove raw_files=` lines that follow; `git diff` on the bundle shows the edition's changes.
+- `--replaces` and `--page` refuse a target inside a captured bundle; the refusal names `raw capture --update` and `--replaces-bundle`. A file named `_capture.json` anywhere but at a captured folder's top is refused - the name is reserved.
 - `--dry-run` reports the moves without making them.
 
 **SEE ALSO**
 
 - `raw/CONTRACT.md` "Getting a file in: incoming/" - the rules and why
 - `wikitool raw pending` - what is waiting in `incoming/`, and which entry is next
+- `wikitool raw capture` - writes a captured bundle, or its new edition, into `incoming/`
 - `wikitool types describe source` - the capture field values
 - `wikitool new source` - the source page for a promoted file