feat: wiki-ingest takes updating the captured repositories as a step-1 branch over raw status (#178)
Files changed: - CHANGES.md - README.md - VERSION - instructions/wiki-ingest/SKILL.md - raw/CONTRACT.md Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
1 parent
8ce202a34e
commit
8fec406463
5 files changed
+74
-11
No files matched your search
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: wiki-ingest
|
||||
description: Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. Use when the user drops a file or folder into incoming/ or raw/, names a URL to ingest, or says "ingest <file>", "ingest <url>", "process this source", "add this to the wiki" - or just "ingest" with nothing named, which takes the oldest entry waiting in incoming/.
|
||||
description: Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. Use when the user drops a file or folder into incoming/ or raw/, names a URL to ingest, or says "ingest <file>", "ingest <url>", "process this source", "add this to the wiki" - or just "ingest" with nothing named, which takes the oldest entry waiting in incoming/ - or asks to "update the captured repositories" / "pull the repo docs", which takes the next bundle `raw status` reports as changed.
|
||||
---
|
||||
|
||||
# Wiki Ingest
|
||||
@@ -10,7 +10,8 @@ description: Processes a new source file into the LLM wiki - extracts entities a
|
||||
**Trigger:** User drops a file or a folder into `incoming/` (the normal path - see step 5) or
|
||||
directly into `raw/`, names a URL to ingest (step 1 fetches it into `incoming/` first), or
|
||||
explicitly requests ingestion - with or without naming what (step 1 picks the entry when nothing
|
||||
is named). **One run is one source:** one file, one bundle or one folder.
|
||||
is named) - or asks to update the captured repositories (step 1 asks `raw status` which one moved).
|
||||
**One run is one source:** one file, one bundle or one folder.
|
||||
|
||||
**Before the first `wikitool` call:** `instructions/session-setup.md`.
|
||||
|
||||
@@ -59,6 +60,35 @@ validator complains - and the ticked list is the only record that they happened.
|
||||
nothing acceptable is waiting, the run ends here: say so, and name each entry the listing
|
||||
marked as not acceptable, with its reason - those need the user, not a guess.
|
||||
|
||||
**The user asks to update the captured repositories** ("update the captured repositories",
|
||||
"pull the repo docs"), **or names one of them?** The entry comes from the repositories, not
|
||||
from `incoming/`:
|
||||
|
||||
```bash
|
||||
tools/wikitool raw status
|
||||
```
|
||||
|
||||
It resolves every captured bundle's ref rule against its repository and prints, per bundle that
|
||||
changed, `old -> new` as short commits, its files as `A`/`M`/`D` grouped by the source page that
|
||||
owns them, and the two `Next:` lines that take the new edition in. Everything this run needs
|
||||
comes out of that output: never ask the user for a commit id or a path.
|
||||
|
||||
- **Nothing changed?** The run ends here. Say how many bundles are unchanged, and name every
|
||||
error line - not reachable, manifest or URL refused, no matching ref - with its reason: those
|
||||
need the user, not a workaround.
|
||||
- **One bundle per run:** the one the user named, otherwise the first changed one in the output.
|
||||
Announce it and carry on - which bundle, `old -> new` from its line, how many others changed,
|
||||
and any error lines as above; an error on one repository never stops the run for another. Ask
|
||||
nothing here - step 4 is the halt before anything is written. Each further bundle is a run of
|
||||
its own, started by the same request, so every run stays within the iteration budget and has
|
||||
its own `publish`. A bundle the user named that is unchanged, or on an error line, ends the
|
||||
run with that said - no other bundle is taken in its place unasked.
|
||||
- **Run the `raw capture --update <raw-bundle>` line printed under it.** `incoming/<bundle>/` is
|
||||
this run's source from here on, and step 1 continues as for any folder, the size check below
|
||||
included. A refusal - an `incoming/<bundle>` left over from an earlier run, for instance - is
|
||||
shown to the user, not worked around.
|
||||
- From step 5 on, the decision point "new edition of a captured bundle" below carries the run.
|
||||
|
||||
**The source is a folder** (`incoming/<folder>/`, named or picked)? It is one source - read
|
||||
every file in it. Whether it is ingested here or by the large-tree procedure is decided by
|
||||
the size check below, by its thresholds, not by its being a folder: three notes in a folder
|
||||
@@ -408,10 +438,13 @@ validator complains - and the ticked list is the only record that they happened.
|
||||
- **One source names far more subjects than usual?** That is breadth, not volume. It is not
|
||||
split into several sources - it cannot be - and it does not get a page per name either:
|
||||
`instructions/ingest-large-tree.md` § A broad source is not cut.
|
||||
- **The entry is a new edition of a captured bundle?** `raw status` reported it, `raw capture
|
||||
--update` wrote it, and `raw accept incoming/<bundle> --replaces-bundle <raw-bundle>` takes it
|
||||
in - the name refusal above does not apply to it. Read the edition diff with `git diff` on the
|
||||
bundle, update every page the output lists under the changed files' source page, and carry the
|
||||
- **The entry is a new edition of a captured bundle?** Step 1's branch for updating the captured
|
||||
repositories is how a run gets here: `raw status` reported it, `raw capture --update` wrote it,
|
||||
and `raw accept incoming/<bundle> --replaces-bundle <raw-bundle>` takes it in - the name refusal
|
||||
above does not apply to it. The edition diff has two halves: `git diff -- <raw-bundle>` for the
|
||||
`M` and `D` files, and the `A` list the accept prints for the new ones, which are still
|
||||
untracked and so never appear in `git diff`. Update every page the output lists under the
|
||||
changed files' source page, and carry the
|
||||
`A`/`D` lines out with the `touch --page "<Source page>" --add/--remove raw_files=<path>` lines
|
||||
it prints - a new file may instead earn a source page of its own. A source page left with no
|
||||
raw file is retired by `instructions/page-lifecycle.md` § Delete. All of it goes into the one
|
||||
@@ -438,4 +471,5 @@ validator complains - and the ticked list is the only record that they happened.
|
||||
Updated wiki with the source's knowledge integrated, published to `origin/main`.
|
||||
|
||||
**Example triggers:** "Ingest raw/articles/my-article.md", "Ingest https://example.org/post",
|
||||
"Ingest incoming/projekt-x", "Ingest" (the oldest entry waiting in `incoming/`)
|
||||
"Ingest incoming/projekt-x", "Ingest" (the oldest entry waiting in `incoming/`), "Update the
|
||||
captured repositories" / "Pull the repo docs" (the next bundle `raw status` reports as changed)
|
||||
Reference in new issue
Block a user