feat: wiki-ingest takes updating the captured repositories as a step-1 branch over raw status (#178)
CI / verify (push) Successful in 5m37s
CI / pwsh (push) Successful in 2m2s
Release / release (push) Successful in 35s

Files changed:
- CHANGES.md
- README.md
- VERSION
- instructions/wiki-ingest/SKILL.md
- raw/CONTRACT.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
torbenandClaude Opus 5.5 committed 2026-10-05 19:21:12 +02:00
1 parent 8ce202a34e
commit 8fec406463
5 files changed
+74 -11

No files matched your search

+23 -1
View File
@@ -59,7 +59,7 @@ concern - readable here, never shipped as something to parse.
--- ---
## 8.0.0-beta.42 - 2026-10-05 - raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle ## 8.0.0-beta.43 - 2026-10-05 - wiki-ingest: updating the captured repositories is a step-1 branch over raw status
**Author:** Torben Nehmer **Author:** Torben Nehmer
@@ -109,6 +109,7 @@ concern - readable here, never shipped as something to parse.
- lint: Unfilled Template Sections - a section still holding only its template's TODO placeholders (advisory) - lint: Unfilled Template Sections - a section still holding only its template's TODO placeholders (advisory)
- Comparison and source pages accept the sources: that cite add writes; sources may cite sources - Comparison and source pages accept the sources: that cite add writes; sources may cite sources
- raw capture / raw status / --replaces-bundle: documentation from git repositories as a bundle, with drift reporting - raw capture / raw status / --replaces-bundle: documentation from git repositories as a bundle, with drift reporting
- wiki-ingest: updating the captured repositories is a step-1 branch over raw status
**Low impact** **Low impact**
- version bump no longer points at version release in its output - version bump no longer points at version release in its output
@@ -155,6 +156,27 @@ concern - readable here, never shipped as something to parse.
- raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle - raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle
<!-- /wikitool:bumps --> <!-- /wikitool:bumps -->
### wiki-ingest: updating the captured repositories is a step-1 branch over raw status
`raw status`, `raw capture --update` and `raw accept --replaces-bundle` existed, but nothing told
a session what to do when asked to update the captured repositories: the decision point for a new
edition began with "`raw status` reported it" and nobody ran `raw status`. Step 1 of `wiki-ingest`
now has that entry, built like the existing branch over `raw pending`. It runs `raw status`, ends
the run when nothing changed (naming every unreachable or refused repository with its reason),
otherwise takes one changed bundle per run (the one the user named, else the first in the output),
announces it with `old -> new` and how many others changed, and runs the `raw capture --update`
line `raw status` printed. From step 5 the existing decision point carries the run. The user never
names a commit or a path. The skill's description and example triggers name the request ("update
the captured repositories", "pull the repo docs").
The decision point now says what the edition diff is: `git diff -- <raw-bundle>` for changed and
removed files, plus the `A` list `raw accept` prints, because added files are still untracked and
never show up in `git diff`. `raw/CONTRACT.md` § "Getting a repository in" and `README.md` say how
an update is started.
Instruction text only, no interface changed - drop-in in both directions; an older version simply
lacks the branch (Gitea #178).
### raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle ### raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle
The ON FAILURE reaction `raw accept incoming/<folder>` prints for an occupied name still said The ON FAILURE reaction `raw accept incoming/<folder>` prints for an occupied name still said
+3 -1
View File
@@ -222,7 +222,9 @@ Documentation that lives in a git repository is captured, not copied: `tools/wik
for byte, into `incoming/<bundle>/` together with a `_capture.json` manifest naming the repository, for byte, into `incoming/<bundle>/` together with a `_capture.json` manifest naming the repository,
the ref rule and the commit. Later, `tools/wikitool raw status` tells you which captured bundles the ref rule and the commit. Later, `tools/wikitool raw status` tells you which captured bundles
have fallen behind their repository, file by file; `raw capture --update` and `raw accept have fallen behind their repository, file by file; `raw capture --update` and `raw accept
--replaces-bundle` take the new edition in as a whole. Git uses your own keys and credential --replaces-bundle` take the new edition in as a whole. You need not run any of that yourself:
tell the LLM `Update the captured repositories`, and it checks them, takes one changed bundle per
run into the wiki and says how many are still behind. Git uses your own keys and credential
helpers - nothing is stored in the repository. helpers - nothing is stored in the repository.
### Querying Knowledge ### Querying Knowledge
+1 -1
View File
@@ -1 +1 @@
8.0.0-beta.42 8.0.0-beta.43
+41 -7
View File
@@ -1,6 +1,6 @@
--- ---
name: wiki-ingest name: wiki-ingest
description: Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. Use when the user drops a file or folder into incoming/ or raw/, names a URL to ingest, or says "ingest <file>", "ingest <url>", "process this source", "add this to the wiki" - or just "ingest" with nothing named, which takes the oldest entry waiting in incoming/. description: Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. Use when the user drops a file or folder into incoming/ or raw/, names a URL to ingest, or says "ingest <file>", "ingest <url>", "process this source", "add this to the wiki" - or just "ingest" with nothing named, which takes the oldest entry waiting in incoming/ - or asks to "update the captured repositories" / "pull the repo docs", which takes the next bundle `raw status` reports as changed.
--- ---
# Wiki Ingest # Wiki Ingest
@@ -10,7 +10,8 @@ description: Processes a new source file into the LLM wiki - extracts entities a
**Trigger:** User drops a file or a folder into `incoming/` (the normal path - see step 5) or **Trigger:** User drops a file or a folder into `incoming/` (the normal path - see step 5) or
directly into `raw/`, names a URL to ingest (step 1 fetches it into `incoming/` first), or directly into `raw/`, names a URL to ingest (step 1 fetches it into `incoming/` first), or
explicitly requests ingestion - with or without naming what (step 1 picks the entry when nothing explicitly requests ingestion - with or without naming what (step 1 picks the entry when nothing
is named). **One run is one source:** one file, one bundle or one folder. is named) - or asks to update the captured repositories (step 1 asks `raw status` which one moved).
**One run is one source:** one file, one bundle or one folder.
**Before the first `wikitool` call:** `instructions/session-setup.md`. **Before the first `wikitool` call:** `instructions/session-setup.md`.
@@ -59,6 +60,35 @@ validator complains - and the ticked list is the only record that they happened.
nothing acceptable is waiting, the run ends here: say so, and name each entry the listing nothing acceptable is waiting, the run ends here: say so, and name each entry the listing
marked as not acceptable, with its reason - those need the user, not a guess. marked as not acceptable, with its reason - those need the user, not a guess.
**The user asks to update the captured repositories** ("update the captured repositories",
"pull the repo docs"), **or names one of them?** The entry comes from the repositories, not
from `incoming/`:
```bash
tools/wikitool raw status
```
It resolves every captured bundle's ref rule against its repository and prints, per bundle that
changed, `old -> new` as short commits, its files as `A`/`M`/`D` grouped by the source page that
owns them, and the two `Next:` lines that take the new edition in. Everything this run needs
comes out of that output: never ask the user for a commit id or a path.
- **Nothing changed?** The run ends here. Say how many bundles are unchanged, and name every
error line - not reachable, manifest or URL refused, no matching ref - with its reason: those
need the user, not a workaround.
- **One bundle per run:** the one the user named, otherwise the first changed one in the output.
Announce it and carry on - which bundle, `old -> new` from its line, how many others changed,
and any error lines as above; an error on one repository never stops the run for another. Ask
nothing here - step 4 is the halt before anything is written. Each further bundle is a run of
its own, started by the same request, so every run stays within the iteration budget and has
its own `publish`. A bundle the user named that is unchanged, or on an error line, ends the
run with that said - no other bundle is taken in its place unasked.
- **Run the `raw capture --update <raw-bundle>` line printed under it.** `incoming/<bundle>/` is
this run's source from here on, and step 1 continues as for any folder, the size check below
included. A refusal - an `incoming/<bundle>` left over from an earlier run, for instance - is
shown to the user, not worked around.
- From step 5 on, the decision point "new edition of a captured bundle" below carries the run.
**The source is a folder** (`incoming/<folder>/`, named or picked)? It is one source - read **The source is a folder** (`incoming/<folder>/`, named or picked)? It is one source - read
every file in it. Whether it is ingested here or by the large-tree procedure is decided by every file in it. Whether it is ingested here or by the large-tree procedure is decided by
the size check below, by its thresholds, not by its being a folder: three notes in a folder the size check below, by its thresholds, not by its being a folder: three notes in a folder
@@ -408,10 +438,13 @@ validator complains - and the ticked list is the only record that they happened.
- **One source names far more subjects than usual?** That is breadth, not volume. It is not - **One source names far more subjects than usual?** That is breadth, not volume. It is not
split into several sources - it cannot be - and it does not get a page per name either: split into several sources - it cannot be - and it does not get a page per name either:
`instructions/ingest-large-tree.md` § A broad source is not cut. `instructions/ingest-large-tree.md` § A broad source is not cut.
- **The entry is a new edition of a captured bundle?** `raw status` reported it, `raw capture - **The entry is a new edition of a captured bundle?** Step 1's branch for updating the captured
--update` wrote it, and `raw accept incoming/<bundle> --replaces-bundle <raw-bundle>` takes it repositories is how a run gets here: `raw status` reported it, `raw capture --update` wrote it,
in - the name refusal above does not apply to it. Read the edition diff with `git diff` on the and `raw accept incoming/<bundle> --replaces-bundle <raw-bundle>` takes it in - the name refusal
bundle, update every page the output lists under the changed files' source page, and carry the above does not apply to it. The edition diff has two halves: `git diff -- <raw-bundle>` for the
`M` and `D` files, and the `A` list the accept prints for the new ones, which are still
untracked and so never appear in `git diff`. Update every page the output lists under the
changed files' source page, and carry the
`A`/`D` lines out with the `touch --page "<Source page>" --add/--remove raw_files=<path>` lines `A`/`D` lines out with the `touch --page "<Source page>" --add/--remove raw_files=<path>` lines
it prints - a new file may instead earn a source page of its own. A source page left with no it prints - a new file may instead earn a source page of its own. A source page left with no
raw file is retired by `instructions/page-lifecycle.md` § Delete. All of it goes into the one raw file is retired by `instructions/page-lifecycle.md` § Delete. All of it goes into the one
@@ -438,4 +471,5 @@ validator complains - and the ticked list is the only record that they happened.
Updated wiki with the source's knowledge integrated, published to `origin/main`. Updated wiki with the source's knowledge integrated, published to `origin/main`.
**Example triggers:** "Ingest raw/articles/my-article.md", "Ingest https://example.org/post", **Example triggers:** "Ingest raw/articles/my-article.md", "Ingest https://example.org/post",
"Ingest incoming/projekt-x", "Ingest" (the oldest entry waiting in `incoming/`) "Ingest incoming/projekt-x", "Ingest" (the oldest entry waiting in `incoming/`), "Update the
captured repositories" / "Pull the repo docs" (the next bundle `raw status` reports as changed)
+6 -1
View File
@@ -311,7 +311,12 @@ the folder carries a different name, or when either side is not a captured bundl
Afterwards the bundle holds exactly the new manifest's files and `_capture.json`. Like Afterwards the bundle holds exactly the new manifest's files and `_capture.json`. Like
`--replaces`, it leaves every `raw_files:` as it was: which source page takes a new file, and `--replaces`, it leaves every `raw_files:` as it was: which source page takes a new file, and
whether a source page left with no file is retired, is the ingest's judgment, and the command whether a source page left with no file is retired, is the ingest's judgment, and the command
prints the `touch` lines that carry it out. `git diff` on the bundle is the edition diff. prints the `touch` lines that carry it out. The edition diff is `git diff` on the bundle for the
changed and removed files, plus the added ones the accept lists - those are still untracked.
**When to update is the user's call; the rest is not.** Asked to update the captured
repositories, `wiki-ingest` runs `raw status` itself and takes one changed bundle per run through
these two commands - the user never names a commit or a path.
**A captured bundle changes only as a whole.** `raw accept --replaces` and `--page` refuse a **A captured bundle changes only as a whole.** `raw accept --replaces` and `--page` refuse a
target inside one: replacing or adding a single file would leave the bundle's content no longer target inside one: replacing or adding a single file would leave the bundle's content no longer