From 8fec406463f434b852417bb9a17c8934b2a4ad17 Mon Sep 17 00:00:00 2001 From: Torben Nehmer Date: Mon, 5 Oct 2026 19:21:12 +0200 Subject: [PATCH] feat: wiki-ingest takes updating the captured repositories as a step-1 branch over raw status (#178) Files changed: - CHANGES.md - README.md - VERSION - instructions/wiki-ingest/SKILL.md - raw/CONTRACT.md Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2 --- CHANGES.md | 24 +++++++++++++++- README.md | 4 ++- VERSION | 2 +- instructions/wiki-ingest/SKILL.md | 48 ++++++++++++++++++++++++++----- raw/CONTRACT.md | 7 ++++- 5 files changed, 74 insertions(+), 11 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index d277427..48d461f 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -59,7 +59,7 @@ concern - readable here, never shipped as something to parse. --- -## 8.0.0-beta.42 - 2026-10-05 - raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle +## 8.0.0-beta.43 - 2026-10-05 - wiki-ingest: updating the captured repositories is a step-1 branch over raw status **Author:** Torben Nehmer @@ -109,6 +109,7 @@ concern - readable here, never shipped as something to parse. - lint: Unfilled Template Sections - a section still holding only its template's TODO placeholders (advisory) - Comparison and source pages accept the sources: that cite add writes; sources may cite sources - raw capture / raw status / --replaces-bundle: documentation from git repositories as a bundle, with drift reporting +- wiki-ingest: updating the captured repositories is a step-1 branch over raw status **Low impact** - version bump no longer points at version release in its output @@ -155,6 +156,27 @@ concern - readable here, never shipped as something to parse. - raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle +### wiki-ingest: updating the captured repositories is a step-1 branch over raw status + +`raw status`, `raw capture --update` and `raw accept --replaces-bundle` existed, but nothing told +a session what to do when asked to update the captured repositories: the decision point for a new +edition began with "`raw status` reported it" and nobody ran `raw status`. Step 1 of `wiki-ingest` +now has that entry, built like the existing branch over `raw pending`. It runs `raw status`, ends +the run when nothing changed (naming every unreachable or refused repository with its reason), +otherwise takes one changed bundle per run (the one the user named, else the first in the output), +announces it with `old -> new` and how many others changed, and runs the `raw capture --update` +line `raw status` printed. From step 5 the existing decision point carries the run. The user never +names a commit or a path. The skill's description and example triggers name the request ("update +the captured repositories", "pull the repo docs"). + +The decision point now says what the edition diff is: `git diff -- ` for changed and +removed files, plus the `A` list `raw accept` prints, because added files are still untracked and +never show up in `git diff`. `raw/CONTRACT.md` § "Getting a repository in" and `README.md` say how +an update is started. + +Instruction text only, no interface changed - drop-in in both directions; an older version simply +lacks the branch (Gitea #178). + ### raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle The ON FAILURE reaction `raw accept incoming/` prints for an occupied name still said diff --git a/README.md b/README.md index da8cdd7..7c58667 100644 --- a/README.md +++ b/README.md @@ -222,7 +222,9 @@ Documentation that lives in a git repository is captured, not copied: `tools/wik for byte, into `incoming//` together with a `_capture.json` manifest naming the repository, the ref rule and the commit. Later, `tools/wikitool raw status` tells you which captured bundles have fallen behind their repository, file by file; `raw capture --update` and `raw accept ---replaces-bundle` take the new edition in as a whole. Git uses your own keys and credential +--replaces-bundle` take the new edition in as a whole. You need not run any of that yourself: +tell the LLM `Update the captured repositories`, and it checks them, takes one changed bundle per +run into the wiki and says how many are still behind. Git uses your own keys and credential helpers - nothing is stored in the repository. ### Querying Knowledge diff --git a/VERSION b/VERSION index 17e8140..3385b23 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -8.0.0-beta.42 +8.0.0-beta.43 diff --git a/instructions/wiki-ingest/SKILL.md b/instructions/wiki-ingest/SKILL.md index 489d7e6..b9f26e0 100644 --- a/instructions/wiki-ingest/SKILL.md +++ b/instructions/wiki-ingest/SKILL.md @@ -1,6 +1,6 @@ --- name: wiki-ingest -description: Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. Use when the user drops a file or folder into incoming/ or raw/, names a URL to ingest, or says "ingest ", "ingest ", "process this source", "add this to the wiki" - or just "ingest" with nothing named, which takes the oldest entry waiting in incoming/. +description: Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. Use when the user drops a file or folder into incoming/ or raw/, names a URL to ingest, or says "ingest ", "ingest ", "process this source", "add this to the wiki" - or just "ingest" with nothing named, which takes the oldest entry waiting in incoming/ - or asks to "update the captured repositories" / "pull the repo docs", which takes the next bundle `raw status` reports as changed. --- # Wiki Ingest @@ -10,7 +10,8 @@ description: Processes a new source file into the LLM wiki - extracts entities a **Trigger:** User drops a file or a folder into `incoming/` (the normal path - see step 5) or directly into `raw/`, names a URL to ingest (step 1 fetches it into `incoming/` first), or explicitly requests ingestion - with or without naming what (step 1 picks the entry when nothing -is named). **One run is one source:** one file, one bundle or one folder. +is named) - or asks to update the captured repositories (step 1 asks `raw status` which one moved). +**One run is one source:** one file, one bundle or one folder. **Before the first `wikitool` call:** `instructions/session-setup.md`. @@ -59,6 +60,35 @@ validator complains - and the ticked list is the only record that they happened. nothing acceptable is waiting, the run ends here: say so, and name each entry the listing marked as not acceptable, with its reason - those need the user, not a guess. + **The user asks to update the captured repositories** ("update the captured repositories", + "pull the repo docs"), **or names one of them?** The entry comes from the repositories, not + from `incoming/`: + + ```bash + tools/wikitool raw status + ``` + + It resolves every captured bundle's ref rule against its repository and prints, per bundle that + changed, `old -> new` as short commits, its files as `A`/`M`/`D` grouped by the source page that + owns them, and the two `Next:` lines that take the new edition in. Everything this run needs + comes out of that output: never ask the user for a commit id or a path. + + - **Nothing changed?** The run ends here. Say how many bundles are unchanged, and name every + error line - not reachable, manifest or URL refused, no matching ref - with its reason: those + need the user, not a workaround. + - **One bundle per run:** the one the user named, otherwise the first changed one in the output. + Announce it and carry on - which bundle, `old -> new` from its line, how many others changed, + and any error lines as above; an error on one repository never stops the run for another. Ask + nothing here - step 4 is the halt before anything is written. Each further bundle is a run of + its own, started by the same request, so every run stays within the iteration budget and has + its own `publish`. A bundle the user named that is unchanged, or on an error line, ends the + run with that said - no other bundle is taken in its place unasked. + - **Run the `raw capture --update ` line printed under it.** `incoming//` is + this run's source from here on, and step 1 continues as for any folder, the size check below + included. A refusal - an `incoming/` left over from an earlier run, for instance - is + shown to the user, not worked around. + - From step 5 on, the decision point "new edition of a captured bundle" below carries the run. + **The source is a folder** (`incoming//`, named or picked)? It is one source - read every file in it. Whether it is ingested here or by the large-tree procedure is decided by the size check below, by its thresholds, not by its being a folder: three notes in a folder @@ -408,10 +438,13 @@ validator complains - and the ticked list is the only record that they happened. - **One source names far more subjects than usual?** That is breadth, not volume. It is not split into several sources - it cannot be - and it does not get a page per name either: `instructions/ingest-large-tree.md` § A broad source is not cut. -- **The entry is a new edition of a captured bundle?** `raw status` reported it, `raw capture - --update` wrote it, and `raw accept incoming/ --replaces-bundle ` takes it - in - the name refusal above does not apply to it. Read the edition diff with `git diff` on the - bundle, update every page the output lists under the changed files' source page, and carry the +- **The entry is a new edition of a captured bundle?** Step 1's branch for updating the captured + repositories is how a run gets here: `raw status` reported it, `raw capture --update` wrote it, + and `raw accept incoming/ --replaces-bundle ` takes it in - the name refusal + above does not apply to it. The edition diff has two halves: `git diff -- ` for the + `M` and `D` files, and the `A` list the accept prints for the new ones, which are still + untracked and so never appear in `git diff`. Update every page the output lists under the + changed files' source page, and carry the `A`/`D` lines out with the `touch --page "" --add/--remove raw_files=` lines it prints - a new file may instead earn a source page of its own. A source page left with no raw file is retired by `instructions/page-lifecycle.md` § Delete. All of it goes into the one @@ -438,4 +471,5 @@ validator complains - and the ticked list is the only record that they happened. Updated wiki with the source's knowledge integrated, published to `origin/main`. **Example triggers:** "Ingest raw/articles/my-article.md", "Ingest https://example.org/post", -"Ingest incoming/projekt-x", "Ingest" (the oldest entry waiting in `incoming/`) +"Ingest incoming/projekt-x", "Ingest" (the oldest entry waiting in `incoming/`), "Update the +captured repositories" / "Pull the repo docs" (the next bundle `raw status` reports as changed) diff --git a/raw/CONTRACT.md b/raw/CONTRACT.md index b4ff13e..67fdec2 100644 --- a/raw/CONTRACT.md +++ b/raw/CONTRACT.md @@ -311,7 +311,12 @@ the folder carries a different name, or when either side is not a captured bundl Afterwards the bundle holds exactly the new manifest's files and `_capture.json`. Like `--replaces`, it leaves every `raw_files:` as it was: which source page takes a new file, and whether a source page left with no file is retired, is the ingest's judgment, and the command -prints the `touch` lines that carry it out. `git diff` on the bundle is the edition diff. +prints the `touch` lines that carry it out. The edition diff is `git diff` on the bundle for the +changed and removed files, plus the added ones the accept lists - those are still untracked. + +**When to update is the user's call; the rest is not.** Asked to update the captured +repositories, `wiki-ingest` runs `raw status` itself and takes one changed bundle per run through +these two commands - the user never names a commit or a path. **A captured bundle changes only as a whole.** `raw accept --replaces` and `--page` refuse a target inside one: replacing or adding a single file would leave the bundle's content no longer