feat: wiki-ingest takes updating the captured repositories as a step-1 branch over raw status (#178)
Files changed: - CHANGES.md - README.md - VERSION - instructions/wiki-ingest/SKILL.md - raw/CONTRACT.md Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
1 parent
8ce202a34e
commit
8fec406463
5 files changed
+74
-11
No files matched your search
+23
-1
@@ -59,7 +59,7 @@ concern - readable here, never shipped as something to parse.
|
||||
|
||||
---
|
||||
|
||||
## 8.0.0-beta.42 - 2026-10-05 - raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle
|
||||
## 8.0.0-beta.43 - 2026-10-05 - wiki-ingest: updating the captured repositories is a step-1 branch over raw status
|
||||
|
||||
**Author:** Torben Nehmer
|
||||
|
||||
@@ -109,6 +109,7 @@ concern - readable here, never shipped as something to parse.
|
||||
- lint: Unfilled Template Sections - a section still holding only its template's TODO placeholders (advisory)
|
||||
- Comparison and source pages accept the sources: that cite add writes; sources may cite sources
|
||||
- raw capture / raw status / --replaces-bundle: documentation from git repositories as a bundle, with drift reporting
|
||||
- wiki-ingest: updating the captured repositories is a step-1 branch over raw status
|
||||
|
||||
**Low impact**
|
||||
- version bump no longer points at version release in its output
|
||||
@@ -155,6 +156,27 @@ concern - readable here, never shipped as something to parse.
|
||||
- raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle
|
||||
<!-- /wikitool:bumps -->
|
||||
|
||||
### wiki-ingest: updating the captured repositories is a step-1 branch over raw status
|
||||
|
||||
`raw status`, `raw capture --update` and `raw accept --replaces-bundle` existed, but nothing told
|
||||
a session what to do when asked to update the captured repositories: the decision point for a new
|
||||
edition began with "`raw status` reported it" and nobody ran `raw status`. Step 1 of `wiki-ingest`
|
||||
now has that entry, built like the existing branch over `raw pending`. It runs `raw status`, ends
|
||||
the run when nothing changed (naming every unreachable or refused repository with its reason),
|
||||
otherwise takes one changed bundle per run (the one the user named, else the first in the output),
|
||||
announces it with `old -> new` and how many others changed, and runs the `raw capture --update`
|
||||
line `raw status` printed. From step 5 the existing decision point carries the run. The user never
|
||||
names a commit or a path. The skill's description and example triggers name the request ("update
|
||||
the captured repositories", "pull the repo docs").
|
||||
|
||||
The decision point now says what the edition diff is: `git diff -- <raw-bundle>` for changed and
|
||||
removed files, plus the `A` list `raw accept` prints, because added files are still untracked and
|
||||
never show up in `git diff`. `raw/CONTRACT.md` § "Getting a repository in" and `README.md` say how
|
||||
an update is started.
|
||||
|
||||
Instruction text only, no interface changed - drop-in in both directions; an older version simply
|
||||
lacks the branch (Gitea #178).
|
||||
|
||||
### raw accept: an occupied folder name held by a captured bundle points at --replaces-bundle
|
||||
|
||||
The ON FAILURE reaction `raw accept incoming/<folder>` prints for an occupied name still said
|
||||
|
||||
@@ -222,7 +222,9 @@ Documentation that lives in a git repository is captured, not copied: `tools/wik
|
||||
for byte, into `incoming/<bundle>/` together with a `_capture.json` manifest naming the repository,
|
||||
the ref rule and the commit. Later, `tools/wikitool raw status` tells you which captured bundles
|
||||
have fallen behind their repository, file by file; `raw capture --update` and `raw accept
|
||||
--replaces-bundle` take the new edition in as a whole. Git uses your own keys and credential
|
||||
--replaces-bundle` take the new edition in as a whole. You need not run any of that yourself:
|
||||
tell the LLM `Update the captured repositories`, and it checks them, takes one changed bundle per
|
||||
run into the wiki and says how many are still behind. Git uses your own keys and credential
|
||||
helpers - nothing is stored in the repository.
|
||||
|
||||
### Querying Knowledge
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: wiki-ingest
|
||||
description: Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. Use when the user drops a file or folder into incoming/ or raw/, names a URL to ingest, or says "ingest <file>", "ingest <url>", "process this source", "add this to the wiki" - or just "ingest" with nothing named, which takes the oldest entry waiting in incoming/.
|
||||
description: Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. Use when the user drops a file or folder into incoming/ or raw/, names a URL to ingest, or says "ingest <file>", "ingest <url>", "process this source", "add this to the wiki" - or just "ingest" with nothing named, which takes the oldest entry waiting in incoming/ - or asks to "update the captured repositories" / "pull the repo docs", which takes the next bundle `raw status` reports as changed.
|
||||
---
|
||||
|
||||
# Wiki Ingest
|
||||
@@ -10,7 +10,8 @@ description: Processes a new source file into the LLM wiki - extracts entities a
|
||||
**Trigger:** User drops a file or a folder into `incoming/` (the normal path - see step 5) or
|
||||
directly into `raw/`, names a URL to ingest (step 1 fetches it into `incoming/` first), or
|
||||
explicitly requests ingestion - with or without naming what (step 1 picks the entry when nothing
|
||||
is named). **One run is one source:** one file, one bundle or one folder.
|
||||
is named) - or asks to update the captured repositories (step 1 asks `raw status` which one moved).
|
||||
**One run is one source:** one file, one bundle or one folder.
|
||||
|
||||
**Before the first `wikitool` call:** `instructions/session-setup.md`.
|
||||
|
||||
@@ -59,6 +60,35 @@ validator complains - and the ticked list is the only record that they happened.
|
||||
nothing acceptable is waiting, the run ends here: say so, and name each entry the listing
|
||||
marked as not acceptable, with its reason - those need the user, not a guess.
|
||||
|
||||
**The user asks to update the captured repositories** ("update the captured repositories",
|
||||
"pull the repo docs"), **or names one of them?** The entry comes from the repositories, not
|
||||
from `incoming/`:
|
||||
|
||||
```bash
|
||||
tools/wikitool raw status
|
||||
```
|
||||
|
||||
It resolves every captured bundle's ref rule against its repository and prints, per bundle that
|
||||
changed, `old -> new` as short commits, its files as `A`/`M`/`D` grouped by the source page that
|
||||
owns them, and the two `Next:` lines that take the new edition in. Everything this run needs
|
||||
comes out of that output: never ask the user for a commit id or a path.
|
||||
|
||||
- **Nothing changed?** The run ends here. Say how many bundles are unchanged, and name every
|
||||
error line - not reachable, manifest or URL refused, no matching ref - with its reason: those
|
||||
need the user, not a workaround.
|
||||
- **One bundle per run:** the one the user named, otherwise the first changed one in the output.
|
||||
Announce it and carry on - which bundle, `old -> new` from its line, how many others changed,
|
||||
and any error lines as above; an error on one repository never stops the run for another. Ask
|
||||
nothing here - step 4 is the halt before anything is written. Each further bundle is a run of
|
||||
its own, started by the same request, so every run stays within the iteration budget and has
|
||||
its own `publish`. A bundle the user named that is unchanged, or on an error line, ends the
|
||||
run with that said - no other bundle is taken in its place unasked.
|
||||
- **Run the `raw capture --update <raw-bundle>` line printed under it.** `incoming/<bundle>/` is
|
||||
this run's source from here on, and step 1 continues as for any folder, the size check below
|
||||
included. A refusal - an `incoming/<bundle>` left over from an earlier run, for instance - is
|
||||
shown to the user, not worked around.
|
||||
- From step 5 on, the decision point "new edition of a captured bundle" below carries the run.
|
||||
|
||||
**The source is a folder** (`incoming/<folder>/`, named or picked)? It is one source - read
|
||||
every file in it. Whether it is ingested here or by the large-tree procedure is decided by
|
||||
the size check below, by its thresholds, not by its being a folder: three notes in a folder
|
||||
@@ -408,10 +438,13 @@ validator complains - and the ticked list is the only record that they happened.
|
||||
- **One source names far more subjects than usual?** That is breadth, not volume. It is not
|
||||
split into several sources - it cannot be - and it does not get a page per name either:
|
||||
`instructions/ingest-large-tree.md` § A broad source is not cut.
|
||||
- **The entry is a new edition of a captured bundle?** `raw status` reported it, `raw capture
|
||||
--update` wrote it, and `raw accept incoming/<bundle> --replaces-bundle <raw-bundle>` takes it
|
||||
in - the name refusal above does not apply to it. Read the edition diff with `git diff` on the
|
||||
bundle, update every page the output lists under the changed files' source page, and carry the
|
||||
- **The entry is a new edition of a captured bundle?** Step 1's branch for updating the captured
|
||||
repositories is how a run gets here: `raw status` reported it, `raw capture --update` wrote it,
|
||||
and `raw accept incoming/<bundle> --replaces-bundle <raw-bundle>` takes it in - the name refusal
|
||||
above does not apply to it. The edition diff has two halves: `git diff -- <raw-bundle>` for the
|
||||
`M` and `D` files, and the `A` list the accept prints for the new ones, which are still
|
||||
untracked and so never appear in `git diff`. Update every page the output lists under the
|
||||
changed files' source page, and carry the
|
||||
`A`/`D` lines out with the `touch --page "<Source page>" --add/--remove raw_files=<path>` lines
|
||||
it prints - a new file may instead earn a source page of its own. A source page left with no
|
||||
raw file is retired by `instructions/page-lifecycle.md` § Delete. All of it goes into the one
|
||||
@@ -438,4 +471,5 @@ validator complains - and the ticked list is the only record that they happened.
|
||||
Updated wiki with the source's knowledge integrated, published to `origin/main`.
|
||||
|
||||
**Example triggers:** "Ingest raw/articles/my-article.md", "Ingest https://example.org/post",
|
||||
"Ingest incoming/projekt-x", "Ingest" (the oldest entry waiting in `incoming/`)
|
||||
"Ingest incoming/projekt-x", "Ingest" (the oldest entry waiting in `incoming/`), "Update the
|
||||
captured repositories" / "Pull the repo docs" (the next bundle `raw status` reports as changed)
|
||||
+6
-1
@@ -311,7 +311,12 @@ the folder carries a different name, or when either side is not a captured bundl
|
||||
Afterwards the bundle holds exactly the new manifest's files and `_capture.json`. Like
|
||||
`--replaces`, it leaves every `raw_files:` as it was: which source page takes a new file, and
|
||||
whether a source page left with no file is retired, is the ingest's judgment, and the command
|
||||
prints the `touch` lines that carry it out. `git diff` on the bundle is the edition diff.
|
||||
prints the `touch` lines that carry it out. The edition diff is `git diff` on the bundle for the
|
||||
changed and removed files, plus the added ones the accept lists - those are still untracked.
|
||||
|
||||
**When to update is the user's call; the rest is not.** Asked to update the captured
|
||||
repositories, `wiki-ingest` runs `raw status` itself and takes one changed bundle per run through
|
||||
these two commands - the user never names a commit or a path.
|
||||
|
||||
**A captured bundle changes only as a whole.** `raw accept --replaces` and `--page` refuse a
|
||||
target inside one: replacing or adding a single file would leave the bundle's content no longer
|
||||
|
||||
Reference in new issue
Block a user