feat: wiki-ingest takes updating the captured repositories as a step-1 branch over raw status (#178)
Files changed: - CHANGES.md - README.md - VERSION - instructions/wiki-ingest/SKILL.md - raw/CONTRACT.md Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
1 parent
8ce202a34e
commit
8fec406463
5 files changed
+74
-11
No files matched your search
+23
-1
@@ -59,7 +59,7 @@ concern - readable here, never shipped as something to parse.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 8.0.0-beta.42 - 2026-10-05 - raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle
|
## 8.0.0-beta.43 - 2026-10-05 - wiki-ingest: updating the captured repositories is a step-1 branch over raw status
|
||||||
|
|
||||||
**Author:** Torben Nehmer
|
**Author:** Torben Nehmer
|
||||||
|
|
||||||
@@ -109,6 +109,7 @@ concern - readable here, never shipped as something to parse.
|
|||||||
- lint: Unfilled Template Sections - a section still holding only its template's TODO placeholders (advisory)
|
- lint: Unfilled Template Sections - a section still holding only its template's TODO placeholders (advisory)
|
||||||
- Comparison and source pages accept the sources: that cite add writes; sources may cite sources
|
- Comparison and source pages accept the sources: that cite add writes; sources may cite sources
|
||||||
- raw capture / raw status / --replaces-bundle: documentation from git repositories as a bundle, with drift reporting
|
- raw capture / raw status / --replaces-bundle: documentation from git repositories as a bundle, with drift reporting
|
||||||
|
- wiki-ingest: updating the captured repositories is a step-1 branch over raw status
|
||||||
|
|
||||||
**Low impact**
|
**Low impact**
|
||||||
- version bump no longer points at version release in its output
|
- version bump no longer points at version release in its output
|
||||||
@@ -155,6 +156,27 @@ concern - readable here, never shipped as something to parse.
|
|||||||
- raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle
|
- raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle
|
||||||
<!-- /wikitool:bumps -->
|
<!-- /wikitool:bumps -->
|
||||||
|
|
||||||
|
### wiki-ingest: updating the captured repositories is a step-1 branch over raw status
|
||||||
|
|
||||||
|
`raw status`, `raw capture --update` and `raw accept --replaces-bundle` existed, but nothing told
|
||||||
|
a session what to do when asked to update the captured repositories: the decision point for a new
|
||||||
|
edition began with "`raw status` reported it" and nobody ran `raw status`. Step 1 of `wiki-ingest`
|
||||||
|
now has that entry, built like the existing branch over `raw pending`. It runs `raw status`, ends
|
||||||
|
the run when nothing changed (naming every unreachable or refused repository with its reason),
|
||||||
|
otherwise takes one changed bundle per run (the one the user named, else the first in the output),
|
||||||
|
announces it with `old -> new` and how many others changed, and runs the `raw capture --update`
|
||||||
|
line `raw status` printed. From step 5 the existing decision point carries the run. The user never
|
||||||
|
names a commit or a path. The skill's description and example triggers name the request ("update
|
||||||
|
the captured repositories", "pull the repo docs").
|
||||||
|
|
||||||
|
The decision point now says what the edition diff is: `git diff -- <raw-bundle>` for changed and
|
||||||
|
removed files, plus the `A` list `raw accept` prints, because added files are still untracked and
|
||||||
|
never show up in `git diff`. `raw/CONTRACT.md` § "Getting a repository in" and `README.md` say how
|
||||||
|
an update is started.
|
||||||
|
|
||||||
|
Instruction text only, no interface changed - drop-in in both directions; an older version simply
|
||||||
|
lacks the branch (Gitea #178).
|
||||||
|
|
||||||
### raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle
|
### raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle
|
||||||
|
|
||||||
The ON FAILURE reaction `raw accept incoming/<folder>` prints for an occupied name still said
|
The ON FAILURE reaction `raw accept incoming/<folder>` prints for an occupied name still said
|
||||||
|
|||||||
@@ -222,7 +222,9 @@ Documentation that lives in a git repository is captured, not copied: `tools/wik
|
|||||||
for byte, into `incoming/<bundle>/` together with a `_capture.json` manifest naming the repository,
|
for byte, into `incoming/<bundle>/` together with a `_capture.json` manifest naming the repository,
|
||||||
the ref rule and the commit. Later, `tools/wikitool raw status` tells you which captured bundles
|
the ref rule and the commit. Later, `tools/wikitool raw status` tells you which captured bundles
|
||||||
have fallen behind their repository, file by file; `raw capture --update` and `raw accept
|
have fallen behind their repository, file by file; `raw capture --update` and `raw accept
|
||||||
--replaces-bundle` take the new edition in as a whole. Git uses your own keys and credential
|
--replaces-bundle` take the new edition in as a whole. You need not run any of that yourself:
|
||||||
|
tell the LLM `Update the captured repositories`, and it checks them, takes one changed bundle per
|
||||||
|
run into the wiki and says how many are still behind. Git uses your own keys and credential
|
||||||
helpers - nothing is stored in the repository.
|
helpers - nothing is stored in the repository.
|
||||||
|
|
||||||
### Querying Knowledge
|
### Querying Knowledge
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: wiki-ingest
|
name: wiki-ingest
|
||||||
description: Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. Use when the user drops a file or folder into incoming/ or raw/, names a URL to ingest, or says "ingest <file>", "ingest <url>", "process this source", "add this to the wiki" - or just "ingest" with nothing named, which takes the oldest entry waiting in incoming/.
|
description: Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. Use when the user drops a file or folder into incoming/ or raw/, names a URL to ingest, or says "ingest <file>", "ingest <url>", "process this source", "add this to the wiki" - or just "ingest" with nothing named, which takes the oldest entry waiting in incoming/ - or asks to "update the captured repositories" / "pull the repo docs", which takes the next bundle `raw status` reports as changed.
|
||||||
---
|
---
|
||||||
|
|
||||||
# Wiki Ingest
|
# Wiki Ingest
|
||||||
@@ -10,7 +10,8 @@ description: Processes a new source file into the LLM wiki - extracts entities a
|
|||||||
**Trigger:** User drops a file or a folder into `incoming/` (the normal path - see step 5) or
|
**Trigger:** User drops a file or a folder into `incoming/` (the normal path - see step 5) or
|
||||||
directly into `raw/`, names a URL to ingest (step 1 fetches it into `incoming/` first), or
|
directly into `raw/`, names a URL to ingest (step 1 fetches it into `incoming/` first), or
|
||||||
explicitly requests ingestion - with or without naming what (step 1 picks the entry when nothing
|
explicitly requests ingestion - with or without naming what (step 1 picks the entry when nothing
|
||||||
is named). **One run is one source:** one file, one bundle or one folder.
|
is named) - or asks to update the captured repositories (step 1 asks `raw status` which one moved).
|
||||||
|
**One run is one source:** one file, one bundle or one folder.
|
||||||
|
|
||||||
**Before the first `wikitool` call:** `instructions/session-setup.md`.
|
**Before the first `wikitool` call:** `instructions/session-setup.md`.
|
||||||
|
|
||||||
@@ -59,6 +60,35 @@ validator complains - and the ticked list is the only record that they happened.
|
|||||||
nothing acceptable is waiting, the run ends here: say so, and name each entry the listing
|
nothing acceptable is waiting, the run ends here: say so, and name each entry the listing
|
||||||
marked as not acceptable, with its reason - those need the user, not a guess.
|
marked as not acceptable, with its reason - those need the user, not a guess.
|
||||||
|
|
||||||
|
**The user asks to update the captured repositories** ("update the captured repositories",
|
||||||
|
"pull the repo docs"), **or names one of them?** The entry comes from the repositories, not
|
||||||
|
from `incoming/`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
tools/wikitool raw status
|
||||||
|
```
|
||||||
|
|
||||||
|
It resolves every captured bundle's ref rule against its repository and prints, per bundle that
|
||||||
|
changed, `old -> new` as short commits, its files as `A`/`M`/`D` grouped by the source page that
|
||||||
|
owns them, and the two `Next:` lines that take the new edition in. Everything this run needs
|
||||||
|
comes out of that output: never ask the user for a commit id or a path.
|
||||||
|
|
||||||
|
- **Nothing changed?** The run ends here. Say how many bundles are unchanged, and name every
|
||||||
|
error line - not reachable, manifest or URL refused, no matching ref - with its reason: those
|
||||||
|
need the user, not a workaround.
|
||||||
|
- **One bundle per run:** the one the user named, otherwise the first changed one in the output.
|
||||||
|
Announce it and carry on - which bundle, `old -> new` from its line, how many others changed,
|
||||||
|
and any error lines as above; an error on one repository never stops the run for another. Ask
|
||||||
|
nothing here - step 4 is the halt before anything is written. Each further bundle is a run of
|
||||||
|
its own, started by the same request, so every run stays within the iteration budget and has
|
||||||
|
its own `publish`. A bundle the user named that is unchanged, or on an error line, ends the
|
||||||
|
run with that said - no other bundle is taken in its place unasked.
|
||||||
|
- **Run the `raw capture --update <raw-bundle>` line printed under it.** `incoming/<bundle>/` is
|
||||||
|
this run's source from here on, and step 1 continues as for any folder, the size check below
|
||||||
|
included. A refusal - an `incoming/<bundle>` left over from an earlier run, for instance - is
|
||||||
|
shown to the user, not worked around.
|
||||||
|
- From step 5 on, the decision point "new edition of a captured bundle" below carries the run.
|
||||||
|
|
||||||
**The source is a folder** (`incoming/<folder>/`, named or picked)? It is one source - read
|
**The source is a folder** (`incoming/<folder>/`, named or picked)? It is one source - read
|
||||||
every file in it. Whether it is ingested here or by the large-tree procedure is decided by
|
every file in it. Whether it is ingested here or by the large-tree procedure is decided by
|
||||||
the size check below, by its thresholds, not by its being a folder: three notes in a folder
|
the size check below, by its thresholds, not by its being a folder: three notes in a folder
|
||||||
@@ -408,10 +438,13 @@ validator complains - and the ticked list is the only record that they happened.
|
|||||||
- **One source names far more subjects than usual?** That is breadth, not volume. It is not
|
- **One source names far more subjects than usual?** That is breadth, not volume. It is not
|
||||||
split into several sources - it cannot be - and it does not get a page per name either:
|
split into several sources - it cannot be - and it does not get a page per name either:
|
||||||
`instructions/ingest-large-tree.md` § A broad source is not cut.
|
`instructions/ingest-large-tree.md` § A broad source is not cut.
|
||||||
- **The entry is a new edition of a captured bundle?** `raw status` reported it, `raw capture
|
- **The entry is a new edition of a captured bundle?** Step 1's branch for updating the captured
|
||||||
--update` wrote it, and `raw accept incoming/<bundle> --replaces-bundle <raw-bundle>` takes it
|
repositories is how a run gets here: `raw status` reported it, `raw capture --update` wrote it,
|
||||||
in - the name refusal above does not apply to it. Read the edition diff with `git diff` on the
|
and `raw accept incoming/<bundle> --replaces-bundle <raw-bundle>` takes it in - the name refusal
|
||||||
bundle, update every page the output lists under the changed files' source page, and carry the
|
above does not apply to it. The edition diff has two halves: `git diff -- <raw-bundle>` for the
|
||||||
|
`M` and `D` files, and the `A` list the accept prints for the new ones, which are still
|
||||||
|
untracked and so never appear in `git diff`. Update every page the output lists under the
|
||||||
|
changed files' source page, and carry the
|
||||||
`A`/`D` lines out with the `touch --page "<Source page>" --add/--remove raw_files=<path>` lines
|
`A`/`D` lines out with the `touch --page "<Source page>" --add/--remove raw_files=<path>` lines
|
||||||
it prints - a new file may instead earn a source page of its own. A source page left with no
|
it prints - a new file may instead earn a source page of its own. A source page left with no
|
||||||
raw file is retired by `instructions/page-lifecycle.md` § Delete. All of it goes into the one
|
raw file is retired by `instructions/page-lifecycle.md` § Delete. All of it goes into the one
|
||||||
@@ -438,4 +471,5 @@ validator complains - and the ticked list is the only record that they happened.
|
|||||||
Updated wiki with the source's knowledge integrated, published to `origin/main`.
|
Updated wiki with the source's knowledge integrated, published to `origin/main`.
|
||||||
|
|
||||||
**Example triggers:** "Ingest raw/articles/my-article.md", "Ingest https://example.org/post",
|
**Example triggers:** "Ingest raw/articles/my-article.md", "Ingest https://example.org/post",
|
||||||
"Ingest incoming/projekt-x", "Ingest" (the oldest entry waiting in `incoming/`)
|
"Ingest incoming/projekt-x", "Ingest" (the oldest entry waiting in `incoming/`), "Update the
|
||||||
|
captured repositories" / "Pull the repo docs" (the next bundle `raw status` reports as changed)
|
||||||
+6
-1
@@ -311,7 +311,12 @@ the folder carries a different name, or when either side is not a captured bundl
|
|||||||
Afterwards the bundle holds exactly the new manifest's files and `_capture.json`. Like
|
Afterwards the bundle holds exactly the new manifest's files and `_capture.json`. Like
|
||||||
`--replaces`, it leaves every `raw_files:` as it was: which source page takes a new file, and
|
`--replaces`, it leaves every `raw_files:` as it was: which source page takes a new file, and
|
||||||
whether a source page left with no file is retired, is the ingest's judgment, and the command
|
whether a source page left with no file is retired, is the ingest's judgment, and the command
|
||||||
prints the `touch` lines that carry it out. `git diff` on the bundle is the edition diff.
|
prints the `touch` lines that carry it out. The edition diff is `git diff` on the bundle for the
|
||||||
|
changed and removed files, plus the added ones the accept lists - those are still untracked.
|
||||||
|
|
||||||
|
**When to update is the user's call; the rest is not.** Asked to update the captured
|
||||||
|
repositories, `wiki-ingest` runs `raw status` itself and takes one changed bundle per run through
|
||||||
|
these two commands - the user never names a commit or a path.
|
||||||
|
|
||||||
**A captured bundle changes only as a whole.** `raw accept --replaces` and `--page` refuse a
|
**A captured bundle changes only as a whole.** `raw accept --replaces` and `--page` refuse a
|
||||||
target inside one: replacing or adding a single file would leave the bundle's content no longer
|
target inside one: replacing or adding a single file would leave the bundle's content no longer
|
||||||
|
|||||||
Reference in new issue
Block a user