feat: wiki-ingest takes updating the captured repositories as a step-1 branch over raw status (#178)
CI / verify (push) Successful in 5m37s
CI / pwsh (push) Successful in 2m2s
Release / release (push) Successful in 35s

Files changed:
- CHANGES.md
- README.md
- VERSION
- instructions/wiki-ingest/SKILL.md
- raw/CONTRACT.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
torbenandClaude Opus 5.5 committed 2026-10-05 19:21:12 +02:00
1 parent 8ce202a34e
commit 8fec406463
5 files changed
+74 -11

No files matched your search

+41 -7
View File
@@ -1,6 +1,6 @@
---
name: wiki-ingest
description: Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. Use when the user drops a file or folder into incoming/ or raw/, names a URL to ingest, or says "ingest <file>", "ingest <url>", "process this source", "add this to the wiki" - or just "ingest" with nothing named, which takes the oldest entry waiting in incoming/.
description: Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. Use when the user drops a file or folder into incoming/ or raw/, names a URL to ingest, or says "ingest <file>", "ingest <url>", "process this source", "add this to the wiki" - or just "ingest" with nothing named, which takes the oldest entry waiting in incoming/ - or asks to "update the captured repositories" / "pull the repo docs", which takes the next bundle `raw status` reports as changed.
---
# Wiki Ingest
@@ -10,7 +10,8 @@ description: Processes a new source file into the LLM wiki - extracts entities a
**Trigger:** User drops a file or a folder into `incoming/` (the normal path - see step 5) or
directly into `raw/`, names a URL to ingest (step 1 fetches it into `incoming/` first), or
explicitly requests ingestion - with or without naming what (step 1 picks the entry when nothing
is named). **One run is one source:** one file, one bundle or one folder.
is named) - or asks to update the captured repositories (step 1 asks `raw status` which one moved).
**One run is one source:** one file, one bundle or one folder.
**Before the first `wikitool` call:** `instructions/session-setup.md`.
@@ -59,6 +60,35 @@ validator complains - and the ticked list is the only record that they happened.
nothing acceptable is waiting, the run ends here: say so, and name each entry the listing
marked as not acceptable, with its reason - those need the user, not a guess.
**The user asks to update the captured repositories** ("update the captured repositories",
"pull the repo docs"), **or names one of them?** The entry comes from the repositories, not
from `incoming/`:
```bash
tools/wikitool raw status
```
It resolves every captured bundle's ref rule against its repository and prints, per bundle that
changed, `old -> new` as short commits, its files as `A`/`M`/`D` grouped by the source page that
owns them, and the two `Next:` lines that take the new edition in. Everything this run needs
comes out of that output: never ask the user for a commit id or a path.
- **Nothing changed?** The run ends here. Say how many bundles are unchanged, and name every
error line - not reachable, manifest or URL refused, no matching ref - with its reason: those
need the user, not a workaround.
- **One bundle per run:** the one the user named, otherwise the first changed one in the output.
Announce it and carry on - which bundle, `old -> new` from its line, how many others changed,
and any error lines as above; an error on one repository never stops the run for another. Ask
nothing here - step 4 is the halt before anything is written. Each further bundle is a run of
its own, started by the same request, so every run stays within the iteration budget and has
its own `publish`. A bundle the user named that is unchanged, or on an error line, ends the
run with that said - no other bundle is taken in its place unasked.
- **Run the `raw capture --update <raw-bundle>` line printed under it.** `incoming/<bundle>/` is
this run's source from here on, and step 1 continues as for any folder, the size check below
included. A refusal - an `incoming/<bundle>` left over from an earlier run, for instance - is
shown to the user, not worked around.
- From step 5 on, the decision point "new edition of a captured bundle" below carries the run.
**The source is a folder** (`incoming/<folder>/`, named or picked)? It is one source - read
every file in it. Whether it is ingested here or by the large-tree procedure is decided by
the size check below, by its thresholds, not by its being a folder: three notes in a folder
@@ -408,10 +438,13 @@ validator complains - and the ticked list is the only record that they happened.
- **One source names far more subjects than usual?** That is breadth, not volume. It is not
split into several sources - it cannot be - and it does not get a page per name either:
`instructions/ingest-large-tree.md` § A broad source is not cut.
- **The entry is a new edition of a captured bundle?** `raw status` reported it, `raw capture
--update` wrote it, and `raw accept incoming/<bundle> --replaces-bundle <raw-bundle>` takes it
in - the name refusal above does not apply to it. Read the edition diff with `git diff` on the
bundle, update every page the output lists under the changed files' source page, and carry the
- **The entry is a new edition of a captured bundle?** Step 1's branch for updating the captured
repositories is how a run gets here: `raw status` reported it, `raw capture --update` wrote it,
and `raw accept incoming/<bundle> --replaces-bundle <raw-bundle>` takes it in - the name refusal
above does not apply to it. The edition diff has two halves: `git diff -- <raw-bundle>` for the
`M` and `D` files, and the `A` list the accept prints for the new ones, which are still
untracked and so never appear in `git diff`. Update every page the output lists under the
changed files' source page, and carry the
`A`/`D` lines out with the `touch --page "<Source page>" --add/--remove raw_files=<path>` lines
it prints - a new file may instead earn a source page of its own. A source page left with no
raw file is retired by `instructions/page-lifecycle.md` § Delete. All of it goes into the one
@@ -438,4 +471,5 @@ validator complains - and the ticked list is the only record that they happened.
Updated wiki with the source's knowledge integrated, published to `origin/main`.
**Example triggers:** "Ingest raw/articles/my-article.md", "Ingest https://example.org/post",
"Ingest incoming/projekt-x", "Ingest" (the oldest entry waiting in `incoming/`)
"Ingest incoming/projekt-x", "Ingest" (the oldest entry waiting in `incoming/`), "Update the
captured repositories" / "Pull the repo docs" (the next bundle `raw status` reports as changed)