chemenu/docs/reference/commands/raw-material-and-uploads.md

# Raw material and uploads

Bringing new material into `raw/`: fetching and capturing it, seeing what is waiting, accepting it, and reviewing what an MCP client uploaded.

<!-- wikitool:toc -->
## Contents

- [`raw fetch`](#raw-fetch)
- [`raw derive`](#raw-derive)
- [`raw capture`](#raw-capture)
- [`raw status`](#raw-status)
- [`raw pending`](#raw-pending)
- [`raw accept`](#raw-accept)
- [`upload list`](#upload-list)
- [`upload show`](#upload-show)
- [`upload accept`](#upload-accept)
- [`upload reject`](#upload-reject)
<!-- /wikitool:toc -->

<!-- wikitool:reference-commands -->
## `raw fetch`

Capture a web page the user names into `incoming/`: the HTML as received plus a derived text, for `raw accept` to promote. Pages of a known website are fetched in the website's standard form and cleaned of its page furniture.

**Synopsis**

- `wikitool raw fetch <url> [--name <stem>]` - Fetch the page and write `incoming/<stem>.html` and `incoming/<stem>.md`
- `wikitool raw fetch <url> --scope [--name <stem>]` - Fetch the page and the subpages its site handler offers, and write the folder `incoming/<stem>/` - a `.html` + `.md` pair per page and the manifest `_scope.json`
- `wikitool raw fetch --html incoming/<file>.html --url <url>` - Derive the `.md` from a page a human saved from their browser - no network access

**Options**

```text
Arguments:
  url  The http(s) URL to fetch

Options:
  --html <path>  Derive the text from this HTML file under incoming/ (saved from
                 a browser) instead of fetching - no network access
  --url <str>    With --html: the page's URL, for the header and relative links
  --name <str>   File stem in incoming/ instead of the one computed from the URL
  --scope        Also fetch the subpages the site handler offers for this page,
                 into the folder incoming/<stem>/
  -h, --help     Show this message and exit.
```

**Examples**

```bash
tools/wikitool raw fetch https://example.org/blog/post
tools/wikitool raw fetch https://example.org/ --name example-start
tools/wikitool raw fetch https://learn.microsoft.com/en-us/powershell/module/oslicense/ --scope
tools/wikitool raw fetch --html incoming/post.html --url https://example.org/blog/post
```

**Behaviour**

- Writes into `incoming/` only, never into `raw/`: `raw accept` promotes both files afterwards, in one call, as one bundle under `raw/<YYYY>/<MM>/<stem>/`.
- The `.html` holds the response body byte for byte. The `.md` starts with a fixed header block (`fetched_by`, `url`, `final_url`, `retrieved`, `http_status`, `content_type`, `charset`, `title`, `derived_from`) followed by the derived text. A page of a website with a site handler adds `site`, and `requested_url` when the URL fetched is not the one typed.
- Character set, in this order: byte-order mark, the HTTP `Content-Type` charset, a `<meta>` declaration in the first 4 KiB, UTF-8. Undecodable bytes are replaced with U+FFFD and reported; the header names the charset and where it came from.
- Derived text: the content root is `<main>`, else the single `<article>`, else `<body>`; `script`, `style`, `noscript`, `nav`, `header`, `footer`, `aside`, `form`, `template` and `svg` are dropped below it. Headings, lists, code, tables and links (made absolute) become Markdown. The same bytes always give the same text.
- Site handlers: for the pages of a website the stack or the instance knows (the stack's today: `learn.microsoft.com`; the instance's are JSON files in `sites/`), `raw fetch <url>` fetches the standard form of the URL (the locale segment becomes `en-us`, the `view=` query stays) and `<url>` as typed goes into the header as `requested_url`; `url` is the form fetched. The handler also removes the page furniture the website renders around the article from the derived text - never from the `.html` - and prints hints that apply to this page as `CHECK` lines: act on one before accepting. `--html --url <url>` removes the same furniture but keeps the URL as typed. A URL no handler claims is handled as before. The header names an instance's handler as `<name> (instance)`. A handler file of the instance that cannot be used, and names the URL's host or no readable host, stops the fetch before anything is fetched or written. The `sites` section of the settings switches handlers off, sets the locale and adds hint lines of the instance's own per host.
- `--scope` is for a page whose site handler offers subpages - the stack's today: a PowerShell module's overview page on `learn.microsoft.com`. The handler names them from the page's own HTML: the links it selects that lead to the same host, under the path of the page itself (or the prefix the handler sets); no other link is followed, and a subpage's own links never are. Without `--scope` the one page is fetched as before and a `CHECK` line says how many subpages it offers. With it, `incoming/<stem>/` holds the page and every subpage as a `<name>.html` + `<name>.md` pair - the `.html` the response byte for byte, the `.md` derived as for a single page, `<name>` from the subpage's URL - and `_scope.json`: the URL, the retrieval time and the `sha256` of the `.html` of each page. `raw accept incoming/<stem>` takes the folder as one bundle.
- `--scope` is bounded. The `robots.txt` of the host is honoured for the subpages (not for the page itself): one it forbids refuses the whole fetch and names the URL. More subpages than `sites.max_pages` (200) refuses it before the first subpage is requested; `sites.pause_seconds` (1) is the pause before each subpage request. Both are settings of the `instance` tier. Everything is checked before the first write: the path budget under `raw/<YYYY>/<MM>/`, a folder of that name already in `incoming/` or `raw/`. A failing request leaves `incoming/` as it was.
- The stem is the URL's last path segment without its extension, else its host name - ASCII, lowercase, `-`-separated, at most 60 characters; `--name` overrides it.
- A `text/plain` or `text/markdown` response is stored as received as `.txt`/`.md`; any other non-HTML response (PDF, image, ...) as received under the extension of its content type. Neither gets a derived file or a header.
- Only `http`/`https`, on every redirect hop too. 30 s for the whole transfer, 25 MiB at most. No cookies, no JavaScript: a derived text under 200 characters is written, with a warning to read it before accepting.
- `--html` derives the `.md` beside a page saved to `incoming/` from a logged-in browser - the way past a paywall or a script-rendered page. Its header carries `derived` (when the text was derived) instead of `retrieved`, `final_url`, `http_status` and `content_type`, and the `.html` is left untouched.
- Success prints the `raw accept` line for the written files and the `source_url` for the source page.

**Properties**

- Changes files: yes
- Idempotent: no
- Atomic: Yes for what it leaves behind - one or two new files in `incoming/`, created exclusively; a failure on the second removes the first. With `--scope`: all of the folder or none of it - the pages are fetched into a temporary place and moved in as a whole
- Counts toward the session call budget: yes
- Can reach the network: yes

**Exit codes**

- `0` - success
- `1` - The URL is not `http`/`https` (also after a redirect), or neither or both of `<url>` and `--html` were given
- `1` - The `sites` section of the settings is malformed, `disabled` names a handler that does not exist, or a file in `sites/` cannot be used and names this URL's host (or none can be read)
- `1` - A target file already exists in `incoming/`
- `1` - The server answered with an HTTP error, could not be reached, took longer than 30 s, or sent more than 25 MiB
- `1` - raw fetch --scope: No site handler offers subpages for the URL, the page offers none or more than `sites.max_pages`, `robots.txt` forbids a subpage or cannot be read, a subpage cannot be fetched, answers something other than HTML or leaves the host, the folder `incoming/<stem>/` exists, or a path would be over the budget
- `1` - raw fetch --html: `--url` is missing, the file is not directly in `incoming/` or does not exist, or `incoming/<stem>.md` already exists

**See also**

- `raw/CONTRACT.md` "Getting a URL in: `raw fetch`" - the rules and why
- `sites/CONTRACT.md` - the instance's own handler files
- `docs/reference/configuration.md` - the `sites` section, which tunes the site handlers
- `wikitool raw accept` - promotes the written files into `raw/`
- `instructions/wiki-ingest/SKILL.md` - where a URL to ingest starts

## `raw derive`

Derive reading text from a PDF, e-mail, Office file or web page in `incoming/`, or unpack a ZIP, and leave the original beside it, for `raw accept` to promote.

**Synopsis**

- `wikitool raw derive incoming/<file> [--name <stem>]` - Move the file into the new folder `incoming/<stem>/` as `original.<ext>` and write `incoming/<stem>/derived.md`
- `wikitool raw derive incoming/<folder>` - Write a `<stem>.md` beside every file in the folder that has a filter

**Options**

```text
Arguments:
  path  A file or a folder directly in incoming/  [required]

Options:
  --name <str>  Folder stem for a single file instead of the file's own stem
  -h, --help    Show this message and exit.
```

**Examples**

```bash
tools/wikitool raw derive incoming/report.docx
tools/wikitool raw derive incoming/Angebot-XYZ.xlsx --name angebot-xyz
tools/wikitool raw derive incoming/meeting-files
tools/wikitool raw derive incoming/scans.zip
```

**Behaviour**

- Writes into `incoming/` only, never into `raw/`, and reads no network: `raw accept` promotes the folder afterwards as one source under `raw/<YYYY>/<MM>/<stem>/`.
- A single file directly in `incoming/` becomes a folder named after its stem (or `--name`) holding `original.<ext>` - the original bytes, moved - and `derived.md`. The fixed names keep the path inside the path budget; the header of `derived.md` records the file's own name as `original_name`.
- A folder is derived in place: each file with a filter gets `<stem>.md` beside it, nothing is renamed, and the files without a filter are named in the output.
- `derived.md` starts with a fixed header (`derived_by`, `filter`, `engine`, `derived`, `derived_from`, `original_name` for a renamed file, `title`, then the fields of the format) followed by the derived text. The same bytes always give the same text; only `derived` differs between runs. `engine` names the tool version and, where a package reads the format, its version.
- Formats: web pages (`.html`), Word (`.docx`), PowerPoint (`.pptx`), Excel (`.xlsx`, `.xlsm`), e-mail (`.eml`) and ZIP archives (`.zip`), read with the standard library only; PDF (`.pdf`, `pypdf`) and Outlook messages (`.msg`, `olefile`), each read by a package in `tools/requirements.txt` that is imported only when such a file is derived. A missing package refuses that format alone, with the preflight hint; `wikitool doctor` shows one `import-filter-<name>` line per package-backed format. A PDF gets `<!-- page N -->` markers and a WARN when it yields almost no text (a scan; there is no OCR); an encrypted PDF is refused. A `.msg` is a container like an `.eml`; an embedded message inside it is named, not read. The text of a mail (`.eml` and `.msg` alike) is its plain text, unless that is under 10% of the length of the HTML text - a "plain text version not available" placeholder - in which case it is the HTML text and a WARN names both lengths. The other version of the text in a `multipart/alternative` is not an attachment.
- A workbook (`.xlsx`, `.xlsm`) gets its cells as files: beside its `.md` a folder `<md stem>.sheets/` (`derived.sheets/` for a single file) holds one `NN-<sheet>.tsv` per sheet - tab-separated per RFC 4180, UTF-8, row 1 of the file is row 1 of the sheet - and, for a sheet with formulas, `NN-<sheet>.formulas.tsv` listing cell, formula and last stored value. The `.md` is the overview: one section per sheet with its range, the file names and a preview. Formulas are not recalculated; a shared formula is written out per cell, and one that cannot be is named `shared:<master cell>` with a WARN. The folder counts against the path budget and is accepted with the rest; an existing one is never overwritten.
- A ZIP and a mail are containers. A ZIP at the top of `incoming/` becomes the folder `incoming/<stem>/` holding its files under their own names, each with a `<stem>.md` where a filter reads it; the ZIP is removed and the output names it with its sha256. A mail becomes a folder with `original.eml`, `derived.md` and its attachments under their own names; a ZIP attached to it is unpacked once into a folder of its own, and an attached mail is quoted in `derived.md`, not unpacked. A ZIP inside a ZIP, an absolute or `..` path, a symbolic link, an encrypted entry, more than 1000 entries, a file over 25 MiB or more than 250 MiB in total refuse the whole call. Hidden entries (`__MACOSX/`, `.DS_Store`, anything starting with a dot) are named, not written. A name taken twice, or reserved (`original.*` and `derived.md` at the top of a mail's folder, `_capture.json` anywhere), gets `-2`, `-3` before its extension, and the output lists each rename.
- A name already taken under `raw/` is a WARN, not a refusal: whether the folder is a later edition of that source or a separate one is the human's call. A later edition of a derived source is replaced file by file with `raw accept --replaces`; there is no `--replaces-bundle` for it.
- After a success the output prints the warnings the filters raised, at most three `CHECK` lines per format that was derived in this run - what to look at before the text is trusted - and the `raw accept` line for the result.
- Never edit `derived.md` by hand: a better derivation is a new `raw derive` of the original.

**Properties**

- Changes files: yes
- Idempotent: no
- Atomic: Yes for what it leaves behind - everything is read, derived and checked before the first write; the files are created exclusively, and a failure part-way puts back what this call changed
- Counts toward the session call budget: yes
- Can reach the network: no

**Exit codes**

- `0` - success
- `1` - The path is not directly in `incoming/`, does not exist, or no filter reads the file's extension
- `1` - The target folder, a `.md` or a workbook's `.sheets` folder already exists, or the path would be over the path budget
- `1` - The document cannot be read (damaged, encrypted), a container breaks one of its rules (a ZIP inside a ZIP, an unsafe path, a limit) or the filter's package is not installed

**See also**

- `raw/CONTRACT.md` "Getting a document in: `raw derive`" - the rules and why
- `wikitool raw accept` - promotes the folder into `raw/`
- `instructions/wiki-ingest/SKILL.md` - where a PDF, mail or Office file to ingest starts

## `raw capture`

Capture documentation from a git repository into `incoming/<bundle>/`, with a manifest naming the repository, ref rule and commit, for `raw accept` to promote.

**Synopsis**

- `wikitool raw capture <repo-url> --ref <branch|tag-pattern> --path <glob> [--path <glob> ...] --name <bundle> --fidelity <v> --authority <v>` - First capture of a repository
- `wikitool raw capture --update <raw-bundle> [--fidelity <v>] [--authority <v>]` - Capture the current state of an already-accepted bundle, for `raw accept --replaces-bundle`

**Options**

```text
Arguments:
  repo_url  The repository URL (ssh://, https:// or user@host:path)

Options:
  --ref <str>        A branch name (main) or a tag pattern (v*)
  --path <str>       A glob over the repository's paths; repeatable
  --name <str>       The bundle name - the folder in incoming/ and under raw/
  --fidelity <str>   How faithful the capture is (see `wikitool types describe
                     source`)
  --authority <str>  What the material may claim about its subject (see
                     `wikitool types describe source`)
  --update <path>    Capture the current state of this captured bundle under
                     raw/, from its own manifest
  -h, --help         Show this message and exit.
```

**Examples**

```bash
tools/wikitool raw capture ssh://git@example.org/team/service.git --ref main --path 'docs/**/*.md' --path README.md --name service-docs --fidelity verbatim --authority normative
tools/wikitool raw capture https://example.org/team/lib.git --ref 'v*' --path docs --name lib-docs --fidelity verbatim --authority reporting
tools/wikitool raw capture --update raw/2026/10/service-docs
```

**Behaviour**

- Writes into `incoming/<bundle>/` only, never into `raw/`: `raw accept incoming/<bundle>` promotes it, as one folder, to `raw/<YYYY>/<MM>/<bundle>/`.
- `--ref` is a branch name (`main`) or a tag pattern (`v*`, any of `*?[`), which names the newest matching tag by version order. It resolves to one commit - for an annotated tag, the commit it points at.
- The commit is fetched by ref name, shallowly, into a bare cache repository per URL under `tools/.wikitool_capture/` (gitignored). Files are read from it as blobs, never through a checkout, so they are byte-identical to the repository's - no line-ending conversion, no filter.
- `--path` globs (repeatable) select files by their path in the repository, as git's `:(glob)` pathspec does: `*`, `?` and `[...]` do not cross `/`; `**` as a whole segment spans any number of directories; a glob without wildcards also takes everything below it as a directory. Files keep their repository paths inside the bundle.
- Never captured, each named in the output with its reason: a file whose first line starts with `<!-- wikitool:export`, a symlink, a submodule, a file over 25 MiB, a path with a segment starting with `.`, a file named `_capture.json`, and a Git LFS pointer.
- `incoming/<bundle>/_capture.json` records `schema`, `repo`, `ref`, `commit`, `paths`, `captured` (UTC), `fidelity`, `authority` and `files`. It is the only declaration that and how this instance follows the repository.
- Only `ssh://`, `https://` and the scp form `user@host:path`; a URL with a password or token in it is refused, since the manifest is committed. Git runs with the host's own credentials and configuration, restricted to those protocols, and never prompts: a repository that asks for credentials is reported as unreachable. Each git call times out after 120 s.
- `--update <raw-bundle>` takes URL, ref rule and globs from that bundle's manifest and writes the current state to `incoming/<same bundle name>/`; `--fidelity`/`--authority` default to the manifest's values, and passing one is the way to correct it.
- Refused before anything is written: `incoming/<bundle>` already exists, the name is already taken under `raw/` (a first capture), nothing matches the globs, or a target path - measured at its later place under `raw/` - is over the path budget.
- Success prints the commit and the `raw accept` line that comes next.

**Properties**

- Changes files: yes
- Idempotent: no
- Atomic: Yes for what it leaves in `incoming/` - the bundle is written into a hidden staging directory and renamed into place in one step; a failure removes the staging directory. The git cache under `tools/.wikitool_capture/` is updated either way
- Counts toward the session call budget: yes
- Can reach the network: yes

**Exit codes**

- `0` - success
- `1` - The URL is not `ssh://`, `https://` or `user@host:path`, or carries a password; the `--ref` rule, a `--path` glob or `--name` is unusable; or an argument of the other variant was mixed in
- `1` - `--fidelity`/`--authority` is missing on a first capture, or names `unknown` or a value outside the schema's enum
- `1` - The repository could not be reached, asked for credentials, or timed out; or no branch or tag matches `--ref`
- `1` - `incoming/<bundle>` already exists, the name is already taken under `raw/`, nothing matches the globs, or a path would be over the budget
- `1` - raw capture --update: The path is not a bundle under `raw/` with a readable `_capture.json`, or its manifest names a URL that is refused

**See also**

- `raw/CONTRACT.md` "Getting a repository in: `raw capture`" - the rules and why
- `wikitool raw accept` - promotes the bundle; `--replaces-bundle` for a new edition
- `wikitool raw status` - which captured bundles have fallen behind their repository

## `raw status`

Report which captured bundles under `raw/` have fallen behind their repository, with each changed file as `A`/`M`/`D`.

**Synopsis**

- `wikitool raw status [--json]`

**Options**

```text
  --json      Print one JSON object per captured bundle
  -h, --help  Show this message and exit.
```

**Examples**

```bash
tools/wikitool raw status
tools/wikitool raw status --json
```

**Behaviour**

- Reads every `_capture.json` under `raw/` and resolves its ref rule with `git ls-remote`. Only where the commit moved is the new commit fetched, and compared - with the same globs and exclusions `raw capture` applies - against the bundle's files under `raw/`.
- A bundle whose commit moved without a change inside its globs is reported as unchanged, not listed. A tag pattern follows new matching tags only, never new commits on a branch.
- Each changed bundle prints `old -> new` and its files as `A`/`M`/`D`, grouped by owning source page with its citing pages, followed by the `raw capture --update` and `raw accept --replaces-bundle` lines that take the new edition in.
- An unreachable repository, a refused URL or an unreadable manifest is one line beside the others, never an abort; the exit status is 0 as long as the command itself ran.
- `--json` prints one object per bundle: `bundle`, `repo`, `ref`, `old`, `new`, `changed`, `files` (`[{path, status}]`), `error` (or null).
- Git runs exactly as for `raw capture`: the host's credentials, only `ssh`/`https`, never a prompt, 120 s per call.

**Properties**

- Changes files: no
- Idempotent: yes
- Atomic: Read-only for the tree - writes nothing but the git cache under `tools/.wikitool_capture/`
- Counts toward the session call budget: yes
- Can reach the network: yes

**Exit codes**

- `0` - success
- `0` - A repository could not be reached, its manifest or URL was refused, or no ref matches its rule

**See also**

- `wikitool raw capture` - `--update` captures the new edition
- `wikitool raw accept` - `--replaces-bundle` takes it in
- `raw/CONTRACT.md` "Getting a repository in: `raw capture`"

## `raw pending`

List what waits in `incoming/`, oldest first, and name the entry an ingest without an argument takes next.

**Synopsis**

- `wikitool raw pending [--json]`

**Options**

```text
  --json      Print the candidates as JSON
  -h, --help  Show this message and exit.
```

**Examples**

```bash
tools/wikitool raw pending
tools/wikitool raw pending --json
```

**Behaviour**

- A candidate is a top-level entry of `incoming/`: a single `file`, a `bundle` of top-level files sharing a stem (a `raw fetch` pair), or a `folder` with every file below it. Dotfiles, empty directories and their contents are none; `mcp-upload/` is outside `incoming/` and never listed.
- Order: oldest first by modification time. A bundle or folder counts as new as its newest file; a tie goes by name. The mtime is when a document last changed only if it was copied with its timestamps kept (`cp -p`, `rsync -a`, an unpacked archive) - for a download or a `raw fetch` it is merely when it was dropped.
- Each candidate shows its path(s), kind, file count and mtime, and whether `raw accept` would take it as it stands - the same checks, minus `--fidelity`/`--authority`. One it would refuse is listed with the reason and skipped: it needs a human.
- The default is the first candidate `raw accept` would take, and the output names it.
- `--json` prints the same candidates in the same order: `kind`, `paths`, `files`, `mtime`, `acceptable`, `reason`, `default`, `hint`.
- A lone file that a `raw derive` filter reads (a PDF, mail, Office file or web page) is listed with the `raw derive` line for it, in text output and as `hint` in JSON (`null` for every other candidate). The hint changes nothing about the order or the default.
- Reads directory listings and `lstat` only, never a file's content; an empty `incoming/` is exit 0 with nothing to do.

**Properties**

- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no

**Exit codes**

- `0` - success

**See also**

- `wikitool raw accept` - promotes the chosen candidate
- `instructions/wiki-ingest/SKILL.md` - ingest without an argument starts here
- `raw/CONTRACT.md` "Getting a file in: incoming/" - candidates and order, and why

## `raw accept`

Promote one or more files, or one folder, from `incoming/` into `raw/`.

**Synopsis**

- `wikitool raw accept <file> [<file> ...] --fidelity <v> --authority <v> [--page "<Title>"] [--dry-run]` - Promote one or more files from `incoming/` into today's `raw/<YYYY>/<MM>/` shard
- `wikitool raw accept incoming/<folder> --fidelity <v> --authority <v> [--dry-run]` - Promote a whole folder as one source, its structure kept, into `raw/<YYYY>/<MM>/<folder>/`
- `wikitool raw accept <file> --replaces <raw-path> [--fidelity <v>] [--authority <v>] [--dry-run]` - Overwrite one existing raw file in place with a new edition
- `wikitool raw accept incoming/<bundle> --replaces-bundle <raw-bundle> [--dry-run]` - Replace a captured bundle as a whole with the new edition `raw capture --update` wrote

**Options**

```text
Arguments:
  files...  One or more files directly in incoming/, all belonging to the same
            source - or one folder in incoming/, accepted whole as one source
            [required]

Options:
  --fidelity <str>          How faithful the capture is (see `wikitool types
                            describe source`). Required unless --replaces is
                            given, where it is the optional, sanctioned way to
                            correct an already-set value
  --authority <str>         What the material may claim about its subject (see
                            `wikitool types describe source`). Required unless
                            --replaces is given, where it is the optional,
                            sanctioned way to correct an already-set value
  --page <str>              Extend this existing source page's raw_files: with
                            the promoted file(s), folding in its already-
                            promoted file if this raises it past one
  --replaces <path>         Overwrite this existing raw/ file in place with the
                            single incoming/ file passed alongside it - the only
                            sanctioned way past the stem-uniqueness rule below
  --dry-run                 List what would move without writing
  --replaces-bundle <path>  Replace this captured bundle under raw/ as a whole
                            with the new edition in the incoming/ folder passed
                            alongside it (written by raw capture --update)
  -h, --help                Show this message and exit.
```

**Examples**

```bash
tools/wikitool raw accept incoming/docker-cheatsheet.md --fidelity verbatim --authority reporting
tools/wikitool raw accept incoming/part-2.md --fidelity verbatim --authority reporting --page "Source - Docker Cheatsheet"
tools/wikitool raw accept incoming/projekt-x --fidelity verbatim --authority reporting
tools/wikitool raw accept incoming/cluster.md --replaces raw/documents/cluster.md
tools/wikitool raw accept incoming/chemenu --replaces-bundle raw/2026/10/chemenu
```

**Behaviour**

- Promotes files from `incoming/` into `raw/<YYYY>/<MM>/`, computed from the accept date rather than chosen by hand. A file argument must sit directly in `incoming/`; a file inside a subdirectory is refused, since a subdirectory is a source of its own.
- A folder argument (`incoming/<folder>`) is one source: every file below it moves to `raw/<YYYY>/<MM>/<folder>/` at the same relative path, and the directories left empty are removed - `incoming/<folder>` no longer exists afterwards. The folder name is the bundle name; two `README.md` in different subfolders are no conflict.
- A folder is accepted alone - no other argument, no `--page`, no `--replaces` - with one `--fidelity`/`--authority` pair for all of it. It is refused, before anything moves, if it is empty, or if a hidden entry (name starting with `.`), a symlink or a special file sits anywhere below it; the refusal names each one.
- One file promoted alone lands with no directory of its own; several files in one call nest under `raw/<YYYY>/<MM>/<stem>/`, named after the first file's stem.
- `--fidelity`/`--authority` are required on a plain accept (`types describe source` lists the values); `unknown` is refused - it is backfill-only.
- `--page "<Title>"` additionally extends that existing source page's `raw_files:` in the same call and writes both capture fields onto it - refused if it already carries a different value, since a capture field is fixed once.
- If `--page` raises the page past one file, its already-promoted file is folded into a bundle at *its own* parent directory, not today's shard, so a bundle never mixes an old and a new capture date - after checking that file has no other owner.
- Every name occupied anywhere under `raw/` - file stems and bundle directory names alike, old type directories and date shards together - stays unique: a promote whose target name already belongs to something this call does not itself own is refused, naming `--replaces` and renaming in `incoming/` as the two routes, without recommending either.
- `--replaces <raw-path>` overwrites that file in place with the single incoming file (same filename required), leaves every page's `raw_files:` untouched and writes no `kb/` page; the previous edition survives only in `git log --follow <raw-path>`.
- With `--replaces`, `--fidelity`/`--authority` are optional, and passing one overwrites the owning page's already-set value - the one path the fixed-once rule does not block.
- `--replaces` refuses a target with more than one owning source page; with none, it replaces anyway and says so. It cannot be combined with `--page` or with more than one incoming file.
- `--replaces` prints the source page (if any) and its citing pages, so their update lands in the same commit as the replacement.
- A file already at its computed destination is what "already exists" reports, not a partial prior run to resume - safe to retry as-is once a cause is fixed.
- A folder carrying `_capture.json` at its top is a captured bundle (`raw capture`): `--fidelity`/`--authority` come from that manifest only and are refused on the command line; the folder must hold exactly the files the manifest lists. `_capture.json` goes in no `raw_files:`.
- `--replaces-bundle <raw-bundle>` replaces a captured bundle under `raw/` with the new edition in `incoming/<bundle>` at the same address: files gone from the new edition are removed, emptied directories `rmdir`ed, and afterwards the bundle holds exactly the new manifest's files plus `_capture.json`. `raw_files:` is left untouched; an owning source page's `fidelity`/`authority` is overwritten where the new manifest differs.
- `--replaces-bundle` prints each changed file as `A`/`M`/`D`, grouped by owning source page with its citing pages, and the `touch --page ... --add/--remove raw_files=` lines that follow; `git diff` on the bundle shows the `M` and `D` files, the `A` files appear only in that list because they are still untracked.
- `--replaces` and `--page` refuse a target inside a captured bundle; the refusal names `raw capture --update` and `--replaces-bundle`. A file named `_capture.json` anywhere but at a captured folder's top is refused - the name is reserved.
- `--dry-run` reports the moves without making them.

**Properties**

- Changes files: yes
- Idempotent: no
- Atomic: `raw accept`: No - one filesystem move per file, then (with `--page`) one page write. With a folder: No - one move per file, then one `rmdir` per emptied directory; a half-accepted folder is not resumed. `raw accept --replaces`: No - one `unlink()` + one `rename()`, plus (if `--fidelity`/`--authority` was given) one page write. `raw accept --replaces-bundle`: No - one `unlink()` per removed file, one `rename()` per added or modified file, then the manifest, then one `rmdir` per emptied directory, plus one page write per owning page whose capture fields change; a half-replaced bundle is not resumed
- Counts toward the session call budget: yes
- Can reach the network: no

**Exit codes**

- `0` - success
- `1` - raw accept: A file does not exist or is not directly in `incoming/`; two files in one call share a filename; a target path already exists; or a target path would be over the path budget (160 UTF-16 code units below the instance root)
- `1` - raw accept incoming/<folder>: The folder is not directly in `incoming/`, is combined with another argument, `--page` or `--replaces`, holds no file, or holds a hidden entry, a symlink or a special file; a target path is over the budget; or the folder name is already occupied under `raw/`
- `1` - A file - given directly, inside a folder, or as `--replaces`/`--replaces-bundle` material - is a guideline export: its first line, after an optional BOM, starts with `<!-- wikitool:export`
- `1` - raw accept: `--fidelity`/`--authority` is missing, or names `unknown` or a value outside the schema's enum
- `1` - raw accept: The target name is already occupied anywhere under `raw/` by something the call does not own
- `1` - raw accept: `--page` names an unknown page or one with no `raw_files:` yet, an existing `raw_files:` entry is missing on disk, a file to be moved has more than one owning page, or `--page` would overwrite an already-set `fidelity`/`authority` with a different value
- `1` - raw accept --replaces: More than one incoming file, or `--page` also given
- `1` - raw accept --replaces: The incoming file does not exist or is not directly in `incoming/`, its filename differs from the target's, or the target does not lie under `raw/` or does not exist
- `1` - raw accept --replaces: `--fidelity`/`--authority` names `unknown` or a value outside the schema's enum, or the target has more than one owning source page
- `1` - raw accept --replaces / --page: The target, or a file the page already has, lies in a captured bundle
- `1` - raw accept incoming/<bundle>: A captured folder is given `--fidelity`/`--authority`, its `_capture.json` cannot be read or names an invalid value, or its files differ from the ones the manifest lists
- `1` - raw accept --replaces-bundle: Either side has no `_capture.json`, the two manifests name different repositories, the folder's name differs from the bundle's, `--page`, `--replaces`, `--fidelity` or `--authority` was also given, the folder does not match its manifest, a new path is over the budget, or a file of the bundle has more than one owning source page

**See also**

- `raw/CONTRACT.md` "Getting a file in: incoming/" - the rules and why
- `wikitool raw pending` - what is waiting in `incoming/`, and which entry is next
- `wikitool raw capture` - writes a captured bundle, or its new edition, into `incoming/`
- `wikitool types describe source` - the capture field values
- `wikitool new source` - the source page for a promoted file

## `upload list`

List every MCP submission currently waiting in the quarantine (`mcp-upload/`).

**Synopsis**

- `wikitool upload list [--json]`

**Options**

```text
  --json      Print every waiting submission as JSON
  -h, --help  Show this message and exit.
```

**Examples**

```bash
tools/wikitool upload list
tools/wikitool upload list --json
```

**Behaviour**

- Lists every submission waiting in `mcp-upload/`, oldest id first: id, filename, size, submitter.
- Only ever non-empty when the `upload` section of `.wikitool-local.json` opts the checkout into the MCP server's `submit` tool.
- A submission directory with a corrupt manifest is silently skipped.
- Never fails; read-only and safe to retry freely.

**Properties**

- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no

**Exit codes**

- `0` - success

**See also**

- `wikitool upload show` - one submission's manifest
- `docs/how-to/set-up-the-mcp-server.md` § "Optional: the submit path"

## `upload show`

Print one submission's manifest in full.

**Synopsis**

- `wikitool upload show <id> [--json]`

**Options**

```text
Arguments:
  submission_id  A submission id from `upload list`  [required]

Options:
  --json
  -h, --help  Show this message and exit.
```

**Examples**

```bash
tools/wikitool upload show <id>
```

**Behaviour**

- Prints one submission's manifest in full: filename, size, sha256, submitter, submitter source (the header name, not a claim the header was honest), submission time.
- What a reviewer reads before `upload accept`.
- Read-only.

**Properties**

- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no

**Exit codes**

- `0` - success
- `1` - Unknown or malformed submission id

**See also**

- `wikitool upload accept` - promotes it after review
- `wikitool upload reject` - declines it

## `upload accept`

**Upload Review Gate:** promote a submission's file from quarantine into `incoming/`.

**Synopsis**

- `wikitool upload accept <id> [--confirm TOKEN]`

**Options**

```text
Arguments:
  submission_id  A submission id from `upload list`  [required]

Options:
  --confirm <str>  The token from a prior refusal, once a human has reviewed it
  -h, --help       Show this message and exit.
```

**Examples**

```bash
tools/wikitool upload accept <id>
tools/wikitool upload accept <id> --confirm <token>    # re-run after exit 42, once the user approved the manifest
```

**Behaviour**

- Promotes a submission's file from `mcp-upload/<id>/` into `incoming/`, deletes the quarantine directory, and appends an `accepted` event to `mcp-upload/ledger.jsonl`.
- Upload Review Gate: without a matching `--confirm`, exits 42 and prints the manifest in full plus the exact re-run line - one submission at a time. The gate check runs before anything is moved.
- The token digests id, filename, size, sha256 and submitter, so an edited or superseded manifest invalidates it.
- An occupied `incoming/<filename>` is an ordinary validation error, not the gate.

**Properties**

- Changes files: yes
- Idempotent: no
- Atomic: No - one filesystem move, one directory delete, one ledger append; the gate check runs first, before any of them
- Counts toward the session call budget: yes
- Can reach the network: no
- Gates: upload-review - see [exit codes and gates](../exit-codes.md)

**Exit codes**

- `0` - success
- `1` - Unknown or malformed submission id, or the submission's file is missing from `mcp-upload/<id>/`
- `1` - `incoming/<filename>` already exists
- `42` - Upload Review Gate: `--confirm` is absent or does not match the manifest's current token

**See also**

- `wikitool upload show` - the manifest to review
- `wikitool raw accept` - the next step for the file in `incoming/`
- `instructions/gates.md` - the gate procedure

## `upload reject`

Delete a submission's material, keeping only its ledger trail.

**Synopsis**

- `wikitool upload reject <id> --reason "<why>"`

**Options**

```text
Arguments:
  submission_id  A submission id from `upload list`  [required]

Options:
  --reason <str>  Why this submission was declined  [required]
  -h, --help      Show this message and exit.
```

**Examples**

```bash
tools/wikitool upload reject <id> --reason "duplicate of raw/articles/llm-wiki.md"
```

**Behaviour**

- Appends an append-only `rejected` event naming the reason and the sha256 of what was declined, then deletes the submission's material.
- The ledger write happens first, so an interruption still leaves the reason on record.
- No gate - rejecting needs no clearance, only accepting does.
- Not idempotent: a second call with the same id reports "unknown id", which confirms the first call worked.

**Properties**

- Changes files: yes
- Idempotent: no
- Atomic: No - one ledger append, then one recursive delete; the ledger write happens first, so an interruption still leaves the reason on record
- Counts toward the session call budget: yes
- Can reach the network: no

**Exit codes**

- `0` - success
- `1` - Unknown or malformed submission id, or an empty `--reason`

**See also**

- `wikitool upload show` - the manifest to review
<!-- /wikitool:reference-commands -->