feat: incoming/ as a queue - raw pending picks the next entry, raw accept takes a whole folder, a file in a subdirectory of incoming/ is refused (#112)

Files changed: - CHANGES.md - README.md - VERSION - instructions/ingest-large-tree.md - instructions/wiki-ingest/SKILL.md - raw/CONTRACT.md - tools/CONTRACT.md - tools/chemenu/cli_contract.py - tools/chemenu/commands/raw_cmd.py - tools/chemenu/tests/test_raw_cmd.py - tools/chemenu/tests/test_raw_fetch.py Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
2026-10-03 09:49:20 +02:00
1 parent c4dcff76e6
commit 59c06e5ddc
@@ -59,7 +59,7 @@ concern - readable here, never shipped as something to parse.
 ---
-## 8.0.0-beta.26 - 2026-10-02 - kb/CONTRACT.md: an external article's raw_files point under raw/, a raw fetch capture names both files
+## 8.0.0-beta.27 - 2026-10-03 - incoming/ as a queue: raw pending picks the next entry, raw accept takes a whole folder
 **Author:** Torben Nehmer
@@ -69,6 +69,7 @@ concern - readable here, never shipped as something to parse.
 - new, rename, move and raw accept refuse a target whose path below the instance root is over 160 characters (UTF-16 code units); lint reports existing files over it as Long Paths (advisory) - rename each affected page with tools/wikitool rename, and shorten an incoming/ file name before raw accept
 - tools/wikitool now refuses to start (exit 42) until tools/preflight.sh (PowerShell 7: tools/preflight.ps1) has passed in the checkout - after updating, run it once: it checks Python, git and ripgrep, records their paths in .wikitool-tools.json and sets up tools/.venv
 - An instance is installed only from a release, into an empty folder (instructions/setup-instance.md); dist export, a clone of the origin repo and a private clone with the origin as upstream are no install paths any more, and wikitool upstream merge, upstream verify and instructions/private-instance.md are gone - an instance built one of those ways is reinstalled from a release into an empty folder and its kb/, raw/ and personal files are copied over. The preflight release asset installs into its own folder, which must be empty apart from the script and a .git, instead of creating a chemenu/ subfolder
 - A file in a subdirectory of incoming/ is no longer accepted - raw accept and raw fetch --html refuse it; a subdirectory is now one source, accepted whole with raw accept incoming/<folder>. Drop files directly into incoming/ and adjust any script that writes to incoming/<type>/; files still waiting in such a subdirectory are moved up into incoming/, or, if they belong together, accepted as one folder
 **Migration:** none required - No page format changes: the title and path rules only refuse names, each affected page is renamed with tools/wikitool rename, and the removed install paths touch no page
@@ -79,6 +80,7 @@ concern - readable here, never shipped as something to parse.
 - Page titles must form valid, unique file names on Windows and macOS
 - Preflight: prerequisites checked and tool paths recorded before wikitool runs (#151, POSIX half)
 - Installation only from a release, into an empty folder; upstream merge/verify and the other install paths removed (#153)
 - incoming/ as a queue: raw pending picks the next entry, raw accept takes a whole folder
 **Medium impact**
 - CalDAV task-tracker provider (Nextcloud Tasks, iOS Reminders); review reports unknown-value findings instead of skipping them
@@ -138,6 +140,40 @@ concern - readable here, never shipped as something to parse.
 - kb/CONTRACT.md: an external article's raw_files point under raw/, a raw fetch capture names both files
 <!-- /wikitool:bumps -->
 ### incoming/ as a queue: raw pending picks the next entry, raw accept takes a whole folder
 **Breaking.** A subdirectory of `incoming/` was tolerated and ignored since the date shard
 replaced the type directories, so an old `incoming/documents/` habit kept working. It protected
 nothing - the kind of source comes from its content now, as `source_type:` - and it left a folder
 of files that belong together with no way in: `raw accept` took files at most one level down,
 bundled them flat under the first file's stem, lost the folder's name and left the emptied
 directory behind, and a folder with subfolders could not be accepted at all. Now a file argument
 of `raw accept` (also with `--replaces`) and of `raw fetch --html` must sit directly in
 `incoming/`; a file inside a subdirectory is refused, and the message names both ways out.
 **A folder is one source.** `raw accept incoming/<folder> --fidelity ... --authority ...` moves
 every file below it to `raw/<YYYY>/<MM>/<folder>/` at the same relative path and removes the
 directories left empty. The folder name is the bundle name and falls under the existing name
 rule; the files inside do not, so several `README.md` in one tree are no conflict. It is accepted
 alone (no other argument, no `--page`, no `--replaces`), and every check runs before anything
 moves: an empty folder, or a hidden entry, a symlink or a special file anywhere below it, is
 refused with each entry named - which is what guarantees that the clean-up, an `rmdir` per
 directory and never a recursive delete, cannot take a file with it. A folder that passes the
 large-tree thresholds continues with `work new --input raw/<YYYY>/<MM>/<folder>`, which the
 success message prints.
 **`tools/wikitool raw pending`** reads `incoming/` as a queue without changing it: the
 top-level entries, with top-level files sharing a stem as one bundle (a `raw fetch` pair) and a
 folder as one candidate, oldest first by modification time - a bundle or folder as new as its
 newest file - each marked with whether `raw accept` would take it, by the same checks, and why
 not. The default is the first acceptable one. `wiki-ingest` uses it when the user names nothing:
 it announces the entry it took, ingests exactly that one, and says at the end how many still
 wait. `raw/CONTRACT.md` § Getting a file in describes candidates, order and the limit of an
 mtime, which is a document's last change only when it was copied with its timestamps kept.
 No migration: no page changes and `raw/` is untouched. An instance whose scripts write to
 `incoming/<type>/` changes them to write directly into `incoming/`.
 ### kb/CONTRACT.md: an external article's raw_files point under raw/, a raw fetch capture names both files
 `kb/CONTRACT.md` still told a source page for an external article to point `raw_files:` at "the
@@ -90,7 +90,7 @@ chemenu/
 ├── mcp-upload/         # QUARANTINE (optional): the MCP `submit` tool's write path, gitignored -
 │                       #   read by no command in the ordinary pipeline; a human reviews it with
 │                       #   `wikitool upload list/show/accept/reject`
-├── incoming/           # INBOX: flat, content gitignored - drop a file here, `raw accept` promotes it
+├── incoming/           # INBOX: content gitignored - drop a file or one folder per source here, `raw accept` promotes it
 ├── raw/                # INPUT: immutable, untrusted source material
 │   ├── CONTRACT.md     # Date shard, capture fields, immutability, untrusted content
 │   ├── 2026/09/        # Where `raw accept` puts a file: the month it was accepted
@@ -185,12 +185,14 @@ working *with* it.
 ### Adding Knowledge (Ingest)
-1. Drop a file into `incoming/` - flat, no classification to make. Everything past that
+1. Drop a file into `incoming/` - directly, no classification to make. Files that belong
-   (the destination in `raw/`, which is a `YYYY/MM` shard of the day it was accepted,
+   together go into one folder there instead: a folder is one source, accepted whole with
-   and whether several files of one source get bundled) is computed by
+   its structure kept. Everything past that (the destination in `raw/`, which is a `YYYY/MM`
-   `tools/wikitool raw accept`, never chosen by hand
+   shard of the day it was accepted, and whether several files of one source get bundled)
-2. Tell the LLM: `Ingest incoming/my-article.md`. It will ask you two things before
+   is computed by `tools/wikitool raw accept`, never chosen by hand
-   promoting: how faithful the capture is (`fidelity`) and what the material may claim
+2. Tell the LLM: `Ingest incoming/my-article.md` - or just `Ingest`, which takes the oldest
   entry waiting in `incoming/` (`tools/wikitool raw pending` lists them). It will ask you
   two things before promoting: how faithful the capture is (`fidelity`) and what the material may claim
   about its subject (`authority`). Both are recorded once and never guessed - they are
   knowable now and unrecoverable later
 3. The LLM will:
@@ -285,7 +287,8 @@ tools/wikitool types describe entity
 ### For You (Human)
-1. **Curate sources** - Drop files you want processed into `incoming/` (flat)
+1. **Curate sources** - Drop files you want processed into `incoming/` - directly, or one folder
   per source
 2. **Ask questions** - Query the wiki naturally
 3. **Review changes** - Check `kb/log.md` and `kb/index.md`
 4. **Direct the LLM** - Guide it on what to emphasize or investigate
@@ -1 +1 @@
-8.0.0-beta.26
+8.0.0-beta.27
@@ -109,7 +109,15 @@ session.
   tools/wikitool work new --input <input path>
   ```
-   This derives the run key, refuses a collision instead of working around it, and writes
+   `--input` must lie under `raw/`, so a tree still waiting in `incoming/` is promoted first, as
   one source and with its structure kept - `raw accept` prints the path to pass on:
   ```bash
   tools/wikitool raw accept --fidelity <value> --authority <value> incoming/<folder>
   tools/wikitool work new --input raw/<YYYY>/<MM>/<folder>
   ```
   `work new` derives the run key, refuses a collision instead of working around it, and writes
   `README.md` + `plan.md`. Never create the directory by hand -
   [work/CONTRACT.md](../work/CONTRACT.md) explains why the run key is not a free choice.
@@ -1,15 +1,16 @@
 ---
 name: wiki-ingest
-description: Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. Use when the user drops a file into incoming/ or raw/, names a URL to ingest, or says "ingest <file>", "ingest <url>", "process this source", "add this to the wiki".
+description: Processes a new source file into the LLM wiki - extracts entities and concepts, creates a source summary page, files a tracker item for any commitment the source also carries, cross-references, rebuilds indexes, and publishes. Use when the user drops a file or folder into incoming/ or raw/, names a URL to ingest, or says "ingest <file>", "ingest <url>", "process this source", "add this to the wiki" - or just "ingest" with nothing named, which takes the oldest entry waiting in incoming/.
 ---
 # Wiki Ingest
 **Purpose:** Process a new source file and integrate its knowledge into the wiki.
-**Trigger:** User drops a file into `incoming/` (the normal path - see step 5) or directly into
+**Trigger:** User drops a file or a folder into `incoming/` (the normal path - see step 5) or
-`raw/`, names a URL to ingest (step 1 fetches it into `incoming/` first), or explicitly requests
+directly into `raw/`, names a URL to ingest (step 1 fetches it into `incoming/` first), or
-ingestion.
+explicitly requests ingestion - with or without naming what (step 1 picks the entry when nothing
 is named). **One run is one source:** one file, one bundle or one folder.
 **Before the first `wikitool` call:** `instructions/session-setup.md`.
@@ -45,6 +46,24 @@ validator complains - and the ticked list is the only record that they happened.
   just wrote, for instance, which skips step 5 entirely). If it is binary or an image, note its
   presence and what it shows.
   **The user named nothing** ("ingest", "process the inbox")? Pick the entry from the queue:
   ```bash
   tools/wikitool raw pending
   ```
   It lists what waits in `incoming/`, oldest first, and marks the default - the oldest entry
   `raw accept` would take as it stands. **Announce it and carry on with it:** which entry, why
   this one (the oldest that can be accepted), and how many wait after it. Ask nothing here -
   step 4 is the halt before anything is written. One run takes exactly that one entry. If
   nothing acceptable is waiting, the run ends here: say so, and name each entry the listing
   marked as not acceptable, with its reason - those need the user, not a guess.
   **The source is a folder** (`incoming/<folder>/`, named or picked)? It is one source - read
   every file in it. Whether it is ingested here or by the large-tree procedure is decided by
   the size check below, by its thresholds, not by its being a folder: three notes in a folder
   do not earn a workshop.
   **The user named a URL instead of a file?** Fetch it into `incoming/` first - never with
   `curl` or the harness's own web fetch, which returns a model's summary rather than the page:
@@ -202,8 +221,14 @@ validator complains - and the ticked list is the only record that they happened.
   List every file this one source produced (e.g. an uploaded PDF plus its converted Markdown)
   in the same call, so they land bundled together rather than as two independent promotions. A
-   file already in `raw/` skips this step entirely. A subdirectory under `incoming/` (an old
+   file already in `raw/` skips this step entirely. A folder is accepted as a whole, on its own:
-   `incoming/<type>/` habit) is tolerated and ignored - it carries no meaning any more.
+
   ```bash
   tools/wikitool raw accept --fidelity <value> --authority <value> incoming/<folder>
   ```
   A file inside a subdirectory of `incoming/` is refused on its own - the subdirectory is the
   source.
   **A file that arrived through the MCP `submit` tool is not yet in `incoming/`** - it sits in
   `mcp-upload/<id>/`, a quarantine no command in this step reads. A reviewer promotes it first
@@ -343,6 +368,9 @@ validator complains - and the ticked list is the only record that they happened.
    deterministic count behind the "every 10 sources" cadence. If the threshold is reached,
    tell the user a full lint is due and offer to run `wiki-lint` next.
    Then say how many entries still wait in `incoming/` (`tools/wikitool raw pending`), so the
    user knows whether another run is due.
 ## Decision points
 - **Subject already has a page?** Update it (step 7, `touch`) instead of creating a second one.
@@ -374,7 +402,7 @@ validator complains - and the ticked list is the only record that they happened.
 ## wikitool commands used
-`raw fetch`, `raw accept`, `search`, `types describe`, `task new`, `task list`, `task close`,
+`raw pending`, `raw fetch`, `raw accept`, `search`, `types describe`, `task new`, `task list`, `task close`,
 `new project`, `new source`, `new entity`, `new concept`, `touch`, `cite add`, `xref add`,
 `xref link-source`, `sources coverage`, `sources rebuild-index`, `index rebuild`, `log append`,
 `log status`, `publish`
@@ -383,4 +411,5 @@ validator complains - and the ticked list is the only record that they happened.
 Updated wiki with the source's knowledge integrated, published to `origin/main`.
-**Example triggers:** "Ingest raw/articles/my-article.md", "Ingest https://example.org/post"
+**Example triggers:** "Ingest raw/articles/my-article.md", "Ingest https://example.org/post",
 "Ingest incoming/projekt-x", "Ingest" (the oldest entry waiting in `incoming/`)
@@ -66,19 +66,57 @@ walks `raw/` recursively and works unchanged either way.
 `raw/` is never chosen by hand. A file to be ingested is dropped into the top-level `incoming/`
 - gitignored content, so a fresh clone finds the directory itself already there but never
-anything dropped into it - flat: no subdirectory carries any classification any more. A
+anything dropped into it - **directly**, not into a subdirectory of it:
 subdirectory is still tolerated if one is used out of habit or by an older script (so an upgrade
 never has to touch a caller), but it is **ignored**, never inspected:
 ```bash
 tools/wikitool raw accept --fidelity verbatim --authority reporting incoming/handbuch.pdf
 # -> raw/2026/09/handbuch.pdf
 ```
 **A subdirectory of `incoming/` is a source of its own, accepted as a whole.** Several files
 that belong together - a folder of notes, an unpacked export - keep their structure:
 ```bash
 tools/wikitool raw accept --fidelity verbatim --authority reporting incoming/projekt-x
 # incoming/projekt-x/plan.md          -> raw/2026/09/projekt-x/plan.md
 # incoming/projekt-x/docs/README.md   -> raw/2026/09/projekt-x/docs/README.md
 ```
 The folder name is the bundle name, so the name rule below applies to it and not to the files
 inside: two `README.md` in different subfolders are no conflict. A folder is accepted alone, with
 one `--fidelity`/`--authority` pair for all of it, and `incoming/projekt-x` is gone afterwards.
 Every check runs before anything moves: an empty folder is refused, and so is one with a hidden
 entry (a name starting with `.`), a symlink or a special file anywhere below it - each is named.
 That is what keeps the clean-up safe: only directories the moves emptied are removed, so no file
 can go with them. A file *inside* a subdirectory is never accepted on its own; the refusal names
 both ways out - the whole folder, or the file moved up into `incoming/`.
 A subdirectory used to be tolerated and ignored, for the old `incoming/<type>/` habit. It carries
 no type any more - the kind of source comes from its content, as `source_type:` on the source
 page (§ Directory routing above) - so the tolerance protected nothing, and a folder that belongs
 together had no way in at all.
 The file's name becomes part of its path under `raw/`, and that path has the same budget as a
 page's - [kb/CONTRACT.md § Titles are identifiers](../kb/CONTRACT.md#titles-are-identifiers).
 `raw accept` refuses a target over it before anything moves; the fix is a shorter name in
-`incoming/`.
+`incoming/` - for a folder, a shorter folder name or shorter names inside it.
 **`incoming/` is a queue, and `tools/wikitool raw pending` reads it** - what an ingest without an
 argument works through, one entry per run:
 - **Candidates** are the top-level entries only. A single file is one; top-level files sharing a
  stem are one `bundle` (a `raw fetch` pair, a PDF and its converted text - the files one
  `raw accept` call takes together); a folder is one, with every file below it. Dotfiles and empty
  directories are none.
 - **Order:** oldest first by modification time, so that newer material builds on what the wiki
  already took from older, or corrects it. A bundle or folder is as new as its newest file; a tie
  goes by name. The limit of this: the mtime is when a document last changed only if it was
  copied with its timestamps kept (`cp -p`, `rsync -a`, an unpacked archive) - for a download or
  a `raw fetch` it is merely when it was dropped. Reading each candidate for a date of its own
  would not be mechanical, and a name says nothing about age.
 - **The default** is the first candidate `raw accept` would take as it stands - the same checks,
  run without moving anything. One it would refuse is listed with the reason and skipped: it
  needs a human, not a guess.
 **A bundle directory is created only from the second file onward.** One file promoted alone needs
 no directory of its own and lands as `raw/<YYYY>/<MM>/<name>`; promoting several files of one
@@ -115,7 +115,8 @@ sources coverage      read  idempotent     budget:counted              exit:0
 sources trace         read  idempotent     budget:counted              exit:0,1    Trace provenance in either direction: raw file, or page.
 sources rebuild-index write idempotent     budget:counted              exit:0,1    Regenerate the `kb/provenance.md` reverse index.
 raw fetch             write non-idempotent budget:counted              exit:0,1    Capture a web page the user names into `incoming/`: the HTML as received plus a derived text, for `raw accept` to promote.
-raw accept            write non-idempotent budget:counted              exit:0,1    Promote one or more files from `incoming/` into `raw/`.
+raw pending           read  idempotent     budget:counted              exit:0      List what waits in `incoming/`, oldest first, and name the entry an ingest without an argument takes next.
 raw accept            write non-idempotent budget:counted              exit:0,1    Promote one or more files, or one folder, from `incoming/` into `raw/`.
 upload list           read  idempotent     budget:counted              exit:0      List every MCP submission currently waiting in the quarantine (`mcp-upload/`).
 upload show           read  idempotent     budget:counted              exit:0,1    Print one submission's manifest in full.
 upload accept         write non-idempotent budget:counted              exit:0,1,42 **Upload Review Gate:** promote a submission's file from quarantine into `incoming/`.
@@ -1446,20 +1447,61 @@ Capture a web page the user names into `incoming/`: the HTML as received plus a
 - `wikitool raw accept` - promotes the written files into `raw/`
 - `instructions/wiki-ingest/SKILL.md` - where a URL to ingest starts
 #### `raw pending`
 List what waits in `incoming/`, oldest first, and name the entry an ingest without an argument takes next.
 **SYNOPSIS**
 - `wikitool raw pending [--json]`
 **PROPERTIES**
 - effect: read
 - idempotent: yes
 - atomic: Read-only
 - budget: counted
 - network: no
 **EXAMPLES**
 - `tools/wikitool raw pending`
 - `tools/wikitool raw pending --json`
 **EXIT STATUS**
 - 0    success
 **NOTES**
 - A candidate is a top-level entry of `incoming/`: a single `file`, a `bundle` of top-level files sharing a stem (a `raw fetch` pair, a PDF and its converted text), or a `folder` with every file below it. Dotfiles, empty directories and their contents are none; `mcp-upload/` is outside `incoming/` and never listed.
 - Order: oldest first by modification time. A bundle or folder counts as new as its newest file; a tie goes by name. The mtime is when a document last changed only if it was copied with its timestamps kept (`cp -p`, `rsync -a`, an unpacked archive) - for a download or a `raw fetch` it is merely when it was dropped.
 - Each candidate shows its path(s), kind, file count and mtime, and whether `raw accept` would take it as it stands - the same checks, minus `--fidelity`/`--authority`. One it would refuse is listed with the reason and skipped: it needs a human.
 - The default is the first candidate `raw accept` would take, and the output names it.
 - `--json` prints the same candidates in the same order: `kind`, `paths`, `files`, `mtime`, `acceptable`, `reason`, `default`.
 - Reads directory listings and `lstat` only, never a file's content; an empty `incoming/` is exit 0 with nothing to do.
 **SEE ALSO**
 - `wikitool raw accept` - promotes the chosen candidate
 - `instructions/wiki-ingest/SKILL.md` - ingest without an argument starts here
 - `raw/CONTRACT.md` "Getting a file in: incoming/" - candidates and order, and why
 #### `raw accept`
-Promote one or more files from `incoming/` into `raw/`.
+Promote one or more files, or one folder, from `incoming/` into `raw/`.
 **SYNOPSIS**
 - `wikitool raw accept <file> [<file> ...] --fidelity <v> --authority <v> [--page "<Title>"] [--dry-run]` - Promote one or more files from `incoming/` into today's `raw/<YYYY>/<MM>/` shard
 - `wikitool raw accept incoming/<folder> --fidelity <v> --authority <v> [--dry-run]` - Promote a whole folder as one source, its structure kept, into `raw/<YYYY>/<MM>/<folder>/`
 - `wikitool raw accept <file> --replaces <raw-path> [--fidelity <v>] [--authority <v>] [--dry-run]` - Overwrite one existing raw file in place with a new edition
 **PROPERTIES**
 - effect: write
 - idempotent: no
- atomic: `raw accept`: No - one filesystem move per file, then (with `--page`) one page write. `raw accept --replaces`: No - one `unlink()` + one `rename()`, plus (if `--fidelity`/`--authority` was given) one page write
+- atomic: `raw accept`: No - one filesystem move per file, then (with `--page`) one page write. With a folder: No - one move per file, then one `rmdir` per emptied directory; a half-accepted folder is not resumed. `raw accept --replaces`: No - one `unlink()` + one `rename()`, plus (if `--fidelity`/`--authority` was given) one page write
 - budget: counted
 - network: no
@@ -1467,27 +1509,30 @@ Promote one or more files from `incoming/` into `raw/`.
 - `tools/wikitool raw accept incoming/docker-cheatsheet.md --fidelity verbatim --authority reporting`
 - `tools/wikitool raw accept incoming/part-2.md --fidelity verbatim --authority reporting --page "Source - Docker Cheatsheet"`
 - `tools/wikitool raw accept incoming/projekt-x --fidelity verbatim --authority reporting`
 - `tools/wikitool raw accept incoming/cluster.md --replaces raw/documents/cluster.md`
 **EXIT STATUS**
 - 0    success
- 1    raw accept: A file does not exist, is not under `incoming/`, or is nested more than one level below it; two files in one call share a filename; a target path already exists; or a target path would be over the path budget (160 UTF-16 code units below the instance root)
+- 1    raw accept: A file does not exist or is not directly in `incoming/`; two files in one call share a filename; a target path already exists; or a target path would be over the path budget (160 UTF-16 code units below the instance root)
 - 1    raw accept incoming/<folder>: The folder is not directly in `incoming/`, is combined with another argument, `--page` or `--replaces`, holds no file, or holds a hidden entry, a symlink or a special file; a target path is over the budget; or the folder name is already occupied under `raw/`
 - 1    raw accept: `--fidelity`/`--authority` is missing, or names `unknown` or a value outside the schema's enum
 - 1    raw accept: The target name is already occupied anywhere under `raw/` by something the call does not own
 - 1    raw accept: `--page` names an unknown page or one with no `raw_files:` yet, an existing `raw_files:` entry is missing on disk, a file to be moved has more than one owning page, or `--page` would overwrite an already-set `fidelity`/`authority` with a different value
 - 1    raw accept --replaces: More than one incoming file, or `--page` also given
- 1    raw accept --replaces: The incoming file does not exist or is not under `incoming/` (or is nested more than one level below it), its filename differs from the target's, or the target does not lie under `raw/` or does not exist
+- 1    raw accept --replaces: The incoming file does not exist or is not directly in `incoming/`, its filename differs from the target's, or the target does not lie under `raw/` or does not exist
 - 1    raw accept --replaces: `--fidelity`/`--authority` names `unknown` or a value outside the schema's enum, or the target has more than one owning source page
 **ON FAILURE**
- raw accept: A file does not exist, is not under `incoming/`, or is nested more than one level below it; two files in one call share a filename; a target path already exists; or a target path would be over the path budget (160 UTF-16 code units below the instance root) -> Fix the named argument and retry once. For a path over the budget, rename the file in `incoming/` to something shorter - the refusal comes before anything moves, so `incoming/` and `raw/` are unchanged
+- raw accept: A file does not exist or is not directly in `incoming/`; two files in one call share a filename; a target path already exists; or a target path would be over the path budget (160 UTF-16 code units below the instance root) -> Fix the named argument and retry once. A file inside a subdirectory of `incoming/` is accepted with its whole folder (`raw accept incoming/<folder>`) or moved up into `incoming/` first. For a path over the budget, rename the file in `incoming/` to something shorter - the refusal comes before anything moves, so `incoming/` and `raw/` are unchanged
 - raw accept incoming/<folder>: The folder is not directly in `incoming/`, is combined with another argument, `--page` or `--replaces`, holds no file, or holds a hidden entry, a symlink or a special file; a target path is over the budget; or the folder name is already occupied under `raw/` -> Nothing moved. Fix what the message names and retry once; for an occupied name, rename the folder in `incoming/` - there is no `--replaces` for a folder
 - raw accept: `--fidelity`/`--authority` is missing, or names `unknown` or a value outside the schema's enum -> Pass both with a valid value, then retry once
 - raw accept: The target name is already occupied anywhere under `raw/` by something the call does not own -> Not fixed by retrying: the refusal names `--replaces` (same source, new edition) and renaming in `incoming/` (a separate source) as the two routes, and neither is the tool's to pick. Show the message to the user and wait
 - raw accept: `--page` names an unknown page or one with no `raw_files:` yet, an existing `raw_files:` entry is missing on disk, a file to be moved has more than one owning page, or `--page` would overwrite an already-set `fidelity`/`authority` with a different value -> Fix the named argument and retry once; a different capture value on an existing page is a new edition - `--replaces`
 - raw accept --replaces: More than one incoming file, or `--page` also given -> A replacement is one file for one file - fix the call and retry once
- raw accept --replaces: The incoming file does not exist or is not under `incoming/` (or is nested more than one level below it), its filename differs from the target's, or the target does not lie under `raw/` or does not exist -> Fix the named argument and retry once - every check runs before the filesystem is touched, so both files are exactly as they were
+- raw accept --replaces: The incoming file does not exist or is not directly in `incoming/`, its filename differs from the target's, or the target does not lie under `raw/` or does not exist -> Fix the named argument and retry once - every check runs before the filesystem is touched, so both files are exactly as they were
 - raw accept --replaces: `--fidelity`/`--authority` names `unknown` or a value outside the schema's enum, or the target has more than one owning source page -> Fix the named argument and retry once; nothing was touched
 **NEVER**
@@ -1497,7 +1542,9 @@ Promote one or more files from `incoming/` into `raw/`.
 **NOTES**
- Promotes files from `incoming/` into `raw/<YYYY>/<MM>/`, computed from the accept date rather than chosen by hand. A subdirectory under `incoming/` is tolerated and ignored, not inspected - `raw/` does not address by type.
+- Promotes files from `incoming/` into `raw/<YYYY>/<MM>/`, computed from the accept date rather than chosen by hand. A file argument must sit directly in `incoming/`; a file inside a subdirectory is refused, since a subdirectory is a source of its own.
 - A folder argument (`incoming/<folder>`) is one source: every file below it moves to `raw/<YYYY>/<MM>/<folder>/` at the same relative path, and the directories left empty are removed - `incoming/<folder>` no longer exists afterwards. The folder name is the bundle name; two `README.md` in different subfolders are no conflict.
 - A folder is accepted alone - no other argument, no `--page`, no `--replaces` - with one `--fidelity`/`--authority` pair for all of it. It is refused, before anything moves, if it is empty, or if a hidden entry (name starting with `.`), a symlink or a special file sits anywhere below it; the refusal names each one.
 - One file promoted alone lands with no directory of its own; several files in one call nest under `raw/<YYYY>/<MM>/<stem>/`, named after the first file's stem.
 - `--fidelity`/`--authority` are required on a plain accept (`types describe source` lists the values); `unknown` is refused - it is backfill-only.
 - `--page "<Title>"` additionally extends that existing source page's `raw_files:` in the same call and writes both capture fields onto it - refused if it already carries a different value, since a capture field is fixed once.
@@ -1513,6 +1560,7 @@ Promote one or more files from `incoming/` into `raw/`.
 **SEE ALSO**
 - `raw/CONTRACT.md` "Getting a file in: incoming/" - the rules and why
 - `wikitool raw pending` - what is waiting in `incoming/`, and which entry is next
 - `wikitool types describe source` - the capture field values
 - `wikitool new source` - the source page for a promoted file
@@ -259,7 +259,8 @@ GROUPS: tuple[tuple[str, tuple[str, ...]], ...] = (
        "sources coverage", "sources trace", "sources rebuild-index",
    )),
    ("Raw material and uploads", (
-        "raw fetch", "raw accept", "upload list", "upload show", "upload accept", "upload reject",
+        "raw fetch", "raw pending", "raw accept",
        "upload list", "upload show", "upload accept", "upload reject",
    )),
    ("Git", (
        "sync", "publish",
@@ -2,17 +2,22 @@
 `raw/`, with the destination computed rather than chosen by hand (Gitea #58),
 sharded by the accept date rather than by a hand-picked type (Gitea #67).
-A human no longer classifies a file at all: `incoming/` is flat, and a
+A human no longer classifies a file at all - `raw/` is an address computed
-subdirectory dropped under it (an old `incoming/documents/` habit, a script
+purely from *when* the file was accepted, and the kind of source comes from
-that still writes one) is accepted and ignored rather than inspected -
+its content (`source_type:` on the source page). So a subdirectory of
-promoting `raw/` from a routing decision to an address computed purely from
+`incoming/` carries no type any more, and since Gitea #112 it is not tolerated
-*when* the file was accepted:
+as one either: it **is** a source, accepted as a whole. A file argument must
 sit directly in `incoming/`:
 - **Single file, no bundle.** One file promoted alone lands as
  `raw/<YYYY>/<MM>/<name>` - no directory of its own.
 - **Bundle from the second file on.** Several files of one source promoted in
  the same call land under `raw/<YYYY>/<MM>/<stem>/`, named after the first
  file's stem.
 - **A folder is one source** (Gitea #112). `raw accept incoming/<folder>`
  moves every file below it to `raw/<YYYY>/<MM>/<folder>/` at the same
  relative path, then removes the directories that are left empty. The folder
  name is the bundle name.
 - **Growing an existing single file into a bundle.** `--page` extends an
  existing source page's `raw_files:`. If that raises the page from one file
  to more than one, the file it already had is folded into a bundle at its
@@ -20,6 +25,11 @@ promoting `raw/` from a routing decision to an address computed purely from
  today's shard, so a bundle never mixes an old capture date with today's
  (Gitea #67 decision, "Datums-Shard" § "Bündelort").
 `raw pending` reads the same queue without changing it: the candidates in
 `incoming/` oldest first, each judged by the very checks `raw accept` runs
 before it moves anything (`_Refused` is how those checks report without
 exiting).
 Existing files under `raw/` are never moved by this change (Gitea #67
 "Altbestand bleibt stehen"): `raw/articles/`, `raw/documents/`, `raw/notes/`
 and `raw/assets/` keep whatever they already held, and stay valid promotion
@@ -65,7 +75,10 @@ and names both routes rather than choosing one (Gitea #64 decision 2).
 from __future__ import annotations
 import datetime
 import json
 import os
 import re
 from dataclasses import dataclass
 from pathlib import Path
 from typing import Optional
@@ -73,7 +86,7 @@ import typer
 from rich.markup import escape
 from chemenu import cli_contract, config, web_capture
-from chemenu.commands._util import check_path_budget, fail, rel_path, success
+from chemenu.commands._util import fail, path_budget_problem_for, rel_path, success
 from chemenu.errors import ChemenuError
 from chemenu.frontmatter_io import write_page
 from chemenu.kb_scan import load_kb_pages
@@ -107,33 +120,51 @@ def _shard_dir(today: Optional[datetime.date] = None) -> Path:
    return config.RAW_DIR / f"{d.year:04d}" / f"{d.month:02d}"
-def _validate_under_incoming(path: Path, incoming: Path) -> None:
+class _Refused(Exception):
-    """`path` must sit directly in `incoming/`, or exactly one level below it.
+    """A check that runs before anything moves has failed; the message is what
    `fail()` prints. Raised rather than failing on the spot so `raw pending`
    can ask the same checks of every candidate without exiting - one set of
    checks, two callers (Gitea #112)."""
-    Unlike before #67, that one optional level carries no meaning any more -
+
-    it is accepted and ignored, kept only so an old `incoming/<type>/` habit
+def _refusals_fail(fn, *args):
-    or script does not have to change to keep working (the MINOR condition
+    try:
-    named in Gitea #67 "Versionsteil"). Nesting deeper than that is still
+        return fn(*args)
-    refused: it was never meaningful and silently accepting it would hide a
+    except _Refused as exc:
-    typo'd path.
+        fail(escape(str(exc)))
 def _check_directly_in_incoming(path: Path, incoming: Path) -> None:
    """`path` must sit directly in `incoming/`.
    Up to Gitea #112 one subdirectory level was tolerated and ignored, so an
    old `incoming/<type>/` habit kept working after #67 stopped reading the
    type from it. A subdirectory is a source of its own now - accepted as a
    whole - so a file inside one is refused with both ways out named.
    """
    try:
        rel = path.relative_to(incoming)
    except ValueError:
-        fail(
+        raise _Refused(
            f"{rel_path(path)} is not under incoming/ - `raw accept` and `raw fetch --html` "
            "only take files from there. See raw/CONTRACT.md."
-        )
+        ) from None
    if len(rel.parts) < 1:
-        fail(f"incoming/{rel.as_posix()} names no file.")
+        raise _Refused(f"incoming/{rel.as_posix()} names no file.")
-    if len(rel.parts) > 2:
+    if len(rel.parts) > 1:
-        fail(
+        top = rel.parts[0]
-            f"incoming/{rel.as_posix()} is nested more than one level below incoming/ - place it "
+        raise _Refused(
-            "directly in incoming/, or in at most one subdirectory of it (the "
+            f"incoming/{rel.as_posix()} lies in a subdirectory of incoming/ - a file is taken "
-            "subdirectory itself is ignored, see raw/CONTRACT.md)."
+            "only from directly inside incoming/, and a subdirectory is a source of its own. "
            f"Either accept the whole folder as one source (raw accept incoming/{top}), or move "
            "the file up into incoming/ and accept it there. See raw/CONTRACT.md."
        )
 def _validate_under_incoming(path: Path, incoming: Path) -> None:
    _refusals_fail(_check_directly_in_incoming, path, incoming)
 _YEAR_DIR = re.compile(r"\d{4}")
@@ -185,6 +216,180 @@ def _stem_collision_message(claimed_name: str, holder: Path) -> str:
    )
 def _check_files(resolved: list[Path]) -> None:
    incoming = _incoming_dir()
    for path in resolved:
        if not path.is_file():
            raise _Refused(f"{rel_path(path)} does not exist or is not a file.")
        _check_directly_in_incoming(path, incoming)
    names = [path.name for path in resolved]
    if len(names) != len(set(names)):
        raise _Refused("Two files share a filename; rename one before promoting.")
 def _check_target(dst: Path, remedy: str) -> None:
    problem = path_budget_problem_for(dst)
    if problem:
        raise _Refused(f"Cannot write {problem}. {remedy}")
    if dst.exists():
        raise _Refused(f"Cannot promote: {rel_path(dst)} already exists.")
 def _plan_file_moves(
    resolved: list[Path], existing_raw_paths: list[Path], page: Optional[str]
 ) -> list[tuple[Path, Path]]:
    """Where each file goes - the bundle decision, the path budget, an existing
    target and the name rule, all checked before anything moves. `page` and
    `existing_raw_paths` are the `--page` case; `raw pending` asks with
    neither."""
    # A bundle directory forms once two or more files belong to the source
    # (Gitea #58 decision 3): from the second file on, never before. Whenever
    # --page targets an existing page it always has >=1 raw file already
    # (types/source.md requires raw_files:), so bundling always applies there.
    total = len(existing_raw_paths) + len(resolved)
    bundle_dir: Optional[Path] = None
    if len(existing_raw_paths) >= 2:
        parents = {p.parent for p in existing_raw_paths}
        if len(parents) != 1:
            raise _Refused(
                f"'{page}' raw_files: are not all in one directory - fix them by hand first "
                "(see `sources coverage`)."
            )
        bundle_dir = parents.pop()
    elif total >= 2:
        if existing_raw_paths:
            # Growing a bundle out of a single already-promoted file (Gitea
            # #67 decision): the bundle forms at that file's own parent
            # directory, never at today's shard - the file's capture date is
            # whatever it always was, and a bundle mixing an old and a new
            # shard would have no single correct address.
            primary = existing_raw_paths[0]
            bundle_dir = primary.parent / primary.stem
        else:
            primary = resolved[0]
            bundle_dir = _shard_dir() / primary.stem
    moves: list[tuple[Path, Path]] = []
    for existing in existing_raw_paths:
        if bundle_dir is not None and existing.parent != bundle_dir:
            moves.append((existing, bundle_dir / existing.name))
    for new_path in resolved:
        dst = (bundle_dir / new_path.name) if bundle_dir is not None else (_shard_dir() / new_path.name)
        moves.append((new_path, dst))
    for _src, dst in moves:
        _check_target(
            dst,
            "The name comes from the file in incoming/: rename it there to something shorter "
            "and accept it again.",
        )
    # Stem uniqueness across raw/ (Gitea #64, widened by #67): the name this
    # call is about to claim - the bundle's name, or the lone file's stem when
    # no bundle forms - must not already belong to something this call does
    # not itself own. "Owns" means: one of the page's already-registered raw
    # files (the pitfall from the module docstring - a single file growing
    # into a bundle of its own name momentarily still occupies that name), or,
    # once a bundle already has >=2 registered files, the bundle directory
    # itself.
    claimed_name = bundle_dir.name if bundle_dir is not None else resolved[0].stem
    occupied = _occupied_stems(config.RAW_DIR)
    owned = set(existing_raw_paths)
    if len(existing_raw_paths) >= 2:
        owned.add(bundle_dir)
    holder = occupied.get(claimed_name)
    if holder is not None and holder not in owned:
        raise _Refused(_stem_collision_message(claimed_name, holder))
    return moves
 def _walk_folder(folder: Path) -> tuple[list[Path], list[Path], list[str]]:
    """Every file below `folder`, every directory (`folder` included), and every
    entry `raw accept` refuses to take - hidden or a symlink anywhere, or
    neither a file nor a directory. Never descends into a refused entry, and
    never follows a link."""
    files: list[Path] = []
    dirs: list[Path] = [folder]
    refused: list[str] = []
    for dirpath, dirnames, filenames in os.walk(folder, followlinks=False):
        base = Path(dirpath)
        descend = []
        for name in sorted(dirnames + filenames):
            entry = base / name
            if name.startswith("."):
                refused.append(f"{rel_path(entry)} (hidden)")
            elif entry.is_symlink():
                refused.append(f"{rel_path(entry)} (symlink)")
            elif entry.is_dir():
                dirs.append(entry)
                descend.append(name)
            elif entry.is_file():
                files.append(entry)
            else:
                refused.append(f"{rel_path(entry)} (neither a file nor a directory)")
        dirnames[:] = descend
    return sorted(files), dirs, refused
 def _folder_collision_message(name: str, holder: Path) -> str:
    return (
        f'{rel_path(holder)} already claims the name "{name}" under raw/.\n'
        "  Rename the folder in incoming/ (add a distinguishing suffix) and accept it again.\n"
        "  A folder has no --replaces: a later edition of one file in it is replaced file by file."
    )
 def _plan_folder(folder: Path) -> tuple[Path, list[tuple[Path, Path]], list[Path]]:
    """The bundle directory for `raw accept incoming/<folder>`, every move into
    it, and the directories to remove afterwards, deepest first - or
    `_Refused`, before anything moves.
    The invariant the cleanup rests on (Gitea #112): after the moves no file
    is left below `folder`, so removing the emptied directories can never
    take one with it. That holds only because every entry the move would
    skip - hidden, a symlink, a special file - is refused here, up front."""
    incoming = _incoming_dir()
    try:
        rel = folder.relative_to(incoming)
    except ValueError:
        raise _Refused(
            f"{rel_path(folder)} is not under incoming/ - `raw accept` only takes a folder from "
            "there. See raw/CONTRACT.md."
        ) from None
    if len(rel.parts) != 1:
        raise _Refused(
            f"incoming/{rel.as_posix()} is not directly in incoming/ - a folder is accepted only "
            f"as a whole, from the top: raw accept incoming/{rel.parts[0]}"
        )
    if folder.name.startswith(".") or folder.is_symlink():
        raise _Refused(f"{rel_path(folder)} is hidden or a symlink - raw accept does not take it.")
    files, dirs, refused = _walk_folder(folder)
    if refused:
        listed = "\n".join(f"  - {entry}" for entry in refused)
        raise _Refused(
            f"{rel_path(folder)}/ holds entries raw accept does not take - hidden entries, "
            f"symlinks and special files are refused, so nothing is left behind:\n{listed}\n"
            "  Remove or replace them in incoming/, then accept the folder again."
        )
    if not files:
        raise _Refused(f"{rel_path(folder)}/ holds no file - nothing to accept.")
    holder = _occupied_stems(config.RAW_DIR).get(folder.name)
    if holder is not None:
        raise _Refused(_folder_collision_message(folder.name, holder))
    bundle_dir = _shard_dir() / folder.name
    moves = [(src, bundle_dir / src.relative_to(folder)) for src in files]
    for _src, dst in moves:
        _check_target(
            dst,
            "The path comes from the folder in incoming/: shorten the folder's name or the "
            "names inside it, and accept it again.",
        )
    return bundle_dir, moves, sorted(dirs, key=lambda d: len(d.parts), reverse=True)
 def _capture_choices() -> tuple[list[str], list[str]]:
    """Allowed `--fidelity`/`--authority` values, straight from the schema
    (single source of truth) - `unknown` excluded, since it is backfill-only
@@ -312,9 +517,55 @@ def _replace(
    success("Review them in this same run: the replacement and their update belong in one commit.")
 def _accept_folder(folder: Path, fidelity: str, authority: str, dry_run: bool) -> None:
    """`raw accept incoming/<folder>` - one folder, one source (Gitea #112).
    Not atomic: one move per file, then the emptied directories. A failure
    part-way leaves a half-accepted folder, which is reported, not resumed -
    the call is not repeated (tool error contract, case 4)."""
    bundle_dir, moves, dirs = _refusals_fail(_plan_folder, folder)
    if dry_run:
        for src, dst in moves:
            typer.echo(f"[dry-run] would move {rel_path(src)} -> {rel_path(dst)}")
        typer.echo(f"[dry-run] would remove {rel_path(folder)}/ once it is empty")
        typer.echo(f"[dry-run] would move {len(moves)} file(s). No files written.")
        return
    for src, dst in moves:
        dst.parent.mkdir(parents=True, exist_ok=True)
        src.rename(dst)
        typer.echo(f"  moved {rel_path(src)} -> {rel_path(dst)}")
    # rmdir, never a recursive delete: it refuses a directory that still holds
    # anything, so this step can only ever remove what the moves emptied.
    left = []
    for directory in dirs:
        try:
            directory.rmdir()
        except OSError:
            left.append(directory)
    if left:
        typer.echo(
            f"WARN {', '.join(rel_path(d) for d in left)} could not be removed - something appeared "
            "in it during the move. Every file of the folder was moved; look at what is left."
        )
    raw_files_arg = ",".join(rel_path(dst) for _src, dst in moves)
    success(
        f"Promoted {rel_path(folder)}/ ({len(moves)} file(s)) to {rel_path(bundle_dir)}/.\n"
        "  Next:\n"
        "  tools/wikitool new source --name \"<Title>\" \\\n"
        f"    --set raw_files={raw_files_arg} \\\n"
        f"    --set fidelity={fidelity} --set authority={authority} \\\n"
        "    --set source_type=<category>\n"
        "  Past the thresholds in instructions/ingest-large-tree.md, continue there instead:\n"
        f"  tools/wikitool work new --input {rel_path(bundle_dir)}"
    )
@cli_contract.record(cli_contract.CommandRecord(
    path="raw accept",
-    summary="Promote one or more files from `incoming/` into `raw/`.",
+    summary="Promote one or more files, or one folder, from `incoming/` into `raw/`.",
    synopsis=(
        cli_contract.Variant(
            usage='raw accept <file> [<file> ...] --fidelity <v> --authority <v> '
@@ -322,6 +573,11 @@ def _replace(
            notes="Promote one or more files from `incoming/` into today's `raw/<YYYY>/<MM>/` "
            "shard",
        ),
        cli_contract.Variant(
            usage="raw accept incoming/<folder> --fidelity <v> --authority <v> [--dry-run]",
            notes="Promote a whole folder as one source, its structure kept, into "
            "`raw/<YYYY>/<MM>/<folder>/`",
        ),
        cli_contract.Variant(
            usage='raw accept <file> --replaces <raw-path> [--fidelity <v>] [--authority <v>] '
            "[--dry-run]",
@@ -332,14 +588,23 @@ def _replace(
        effect=cli_contract.Effect.WRITE,
        idempotent=cli_contract.Idempotent.NO,
        atomic="`raw accept`: No - one filesystem move per file, then (with `--page`) one page "
-        "write. `raw accept --replaces`: No - one `unlink()` + one `rename()`, plus (if "
+        "write. With a folder: No - one move per file, then one `rmdir` per emptied directory; "
-        "`--fidelity`/`--authority` was given) one page write",
+        "a half-accepted folder is not resumed. `raw accept --replaces`: No - one `unlink()` + "
        "one `rename()`, plus (if `--fidelity`/`--authority` was given) one page write",
        budget=cli_contract.Budget.COUNTED,
    ),
    notes=(
        "Promotes files from `incoming/` into `raw/<YYYY>/<MM>/`, computed from the accept "
-        "date rather than chosen by hand. A subdirectory under `incoming/` is tolerated and "
+        "date rather than chosen by hand. A file argument must sit directly in `incoming/`; a "
-        "ignored, not inspected - `raw/` does not address by type.",
+        "file inside a subdirectory is refused, since a subdirectory is a source of its own.",
        "A folder argument (`incoming/<folder>`) is one source: every file below it moves to "
        "`raw/<YYYY>/<MM>/<folder>/` at the same relative path, and the directories left empty "
        "are removed - `incoming/<folder>` no longer exists afterwards. The folder name is the "
        "bundle name; two `README.md` in different subfolders are no conflict.",
        "A folder is accepted alone - no other argument, no `--page`, no `--replaces` - with "
        "one `--fidelity`/`--authority` pair for all of it. It is refused, before anything "
        "moves, if it is empty, or if a hidden entry (name starting with `.`), a symlink or a "
        "special file sits anywhere below it; the refusal names each one.",
        "One file promoted alone lands with no directory of its own; several files in one call "
        "nest under `raw/<YYYY>/<MM>/<stem>/`, named after the first file's stem.",
        "`--fidelity`/`--authority` are required on a plain accept (`types describe source` "
@@ -373,13 +638,23 @@ def _replace(
    failures=(
        cli_contract.Failure(
            label="raw accept",
-            cause="A file does not exist, is not under `incoming/`, or is nested more than one "
+            cause="A file does not exist or is not directly in `incoming/`; two files in one "
-            "level below it; two files in one call share a filename; a target path already "
+            "call share a filename; a target path already exists; or a target path would be "
-            "exists; or a target path would be over the path budget (160 UTF-16 code units "
+            "over the path budget (160 UTF-16 code units below the instance root)",
-            "below the instance root)",
+            reaction="Fix the named argument and retry once. A file inside a subdirectory of "
-            reaction="Fix the named argument and retry once. For a path over the budget, "
+            "`incoming/` is accepted with its whole folder (`raw accept incoming/<folder>`) or "
-            "rename the file in `incoming/` to something shorter - the refusal comes before "
+            "moved up into `incoming/` first. For a path over the budget, rename the file in "
-            "anything moves, so `incoming/` and `raw/` are unchanged",
+            "`incoming/` to something shorter - the refusal comes before anything moves, so "
            "`incoming/` and `raw/` are unchanged",
        ),
        cli_contract.Failure(
            label="raw accept incoming/<folder>",
            cause="The folder is not directly in `incoming/`, is combined with another argument, "
            "`--page` or `--replaces`, holds no file, or holds a hidden entry, a symlink or a "
            "special file; a target path is over the budget; or the folder name is already "
            "occupied under `raw/`",
            reaction="Nothing moved. Fix what the message names and retry once; for an occupied "
            "name, rename the folder in `incoming/` - there is no `--replaces` for a folder",
        ),
        cli_contract.Failure(
            label="raw accept",
@@ -411,9 +686,9 @@ def _replace(
        ),
        cli_contract.Failure(
            label="raw accept --replaces",
-            cause="The incoming file does not exist or is not under `incoming/` (or is nested "
+            cause="The incoming file does not exist or is not directly in `incoming/`, its "
-            "more than one level below it), its filename differs from the target's, or the "
+            "filename differs from the target's, or the target does not lie under `raw/` or "
-            "target does not lie under `raw/` or does not exist",
+            "does not exist",
            reaction="Fix the named argument and retry once - every check runs before the "
            "filesystem is touched, so both files are exactly as they were",
        ),
@@ -429,6 +704,7 @@ def _replace(
        "--authority reporting",
        'tools/wikitool raw accept incoming/part-2.md --fidelity verbatim --authority reporting '
        '--page "Source - Docker Cheatsheet"',
        "tools/wikitool raw accept incoming/projekt-x --fidelity verbatim --authority reporting",
        "tools/wikitool raw accept incoming/cluster.md --replaces raw/documents/cluster.md",
    ),
    never=(
@@ -439,6 +715,7 @@ def _replace(
    ),
    see_also=(
        "`raw/CONTRACT.md` \"Getting a file in: incoming/\" - the rules and why",
        "`wikitool raw pending` - what is waiting in `incoming/`, and which entry is next",
        "`wikitool types describe source` - the capture field values",
        "`wikitool new source` - the source page for a promoted file",
    ),
@@ -447,7 +724,8 @@ def _replace(
 def raw_accept_command(
    files: list[Path] = typer.Argument(
        ...,
-        help="One or more files under incoming/, all belonging to the same source",
+        help="One or more files directly in incoming/, all belonging to the same source - or "
        "one folder in incoming/, accepted whole as one source",
    ),
    fidelity: Optional[str] = typer.Option(
        None,
@@ -476,13 +754,19 @@ def raw_accept_command(
    ),
    dry_run: bool = typer.Option(False, "--dry-run", help="List what would move without writing"),
 ):
-    """Promote file(s) from incoming/ into raw/, computing the destination
+    """Promote file(s) or one folder from incoming/ into raw/, computing the
-    (date shard, bundle or not, bundle name) instead of taking it as an
+    destination (date shard, bundle or not, bundle name) instead of taking it
-    argument. See raw/CONTRACT.md "Getting a file in: incoming/"."""
+    as an argument. See raw/CONTRACT.md "Getting a file in: incoming/"."""
    if not files:
        fail("Pass at least one file to promote.")
-    incoming = _incoming_dir()
+    resolved = [_resolve(f) for f in files]
    folders = [p for p in resolved if p.is_dir()]
    if folders and (len(files) != 1 or page is not None or replaces is not None):
        fail(escape(
            f"{rel_path(folders[0])} is a folder, and a folder is accepted alone: one source, "
            "with no other argument, no --page and no --replaces. Accept it in a call of its own."
        ))
    if replaces is not None:
        _replace(files, replaces, page, fidelity, authority, dry_run)
@@ -498,16 +782,11 @@ def raw_accept_command(
    _check_capture_value("fidelity", fidelity, fidelity_choices)
    _check_capture_value("authority", authority, authority_choices)
-    resolved = [_resolve(f) for f in files]
+    if folders:
        _accept_folder(folders[0], fidelity, authority, dry_run)
        return
-    for path in resolved:
+    _refusals_fail(_check_files, resolved)
        if not path.is_file():
            fail(f"{rel_path(path)} does not exist or is not a file.")
        _validate_under_incoming(path, incoming)
    names = [path.name for path in resolved]
    if len(names) != len(set(names)):
        fail("Two files share a filename; rename one before promoting.")
    pages = None
    target_page = None
@@ -532,66 +811,7 @@ def raw_accept_command(
                "before promoting more."
            )
-    # A bundle directory forms once two or more files belong to the source
+    moves = _refusals_fail(_plan_file_moves, resolved, existing_raw_paths, page)
    # (Gitea #58 decision 3): from the second file on, never before. Whenever
    # --page targets an existing page it always has >=1 raw file already
    # (types/source.md requires raw_files:), so bundling always applies there.
    total = len(existing_raw_paths) + len(resolved)
    bundle_dir: Optional[Path] = None
    if len(existing_raw_paths) >= 2:
        parents = {p.parent for p in existing_raw_paths}
        if len(parents) != 1:
            fail(
                f"'{page}' raw_files: are not all in one directory - fix them by hand first "
                "(see `sources coverage`)."
            )
        bundle_dir = parents.pop()
    elif total >= 2:
        if existing_raw_paths:
            # Growing a bundle out of a single already-promoted file (Gitea
            # #67 decision): the bundle forms at that file's own parent
            # directory, never at today's shard - the file's capture date is
            # whatever it always was, and a bundle mixing an old and a new
            # shard would have no single correct address.
            primary = existing_raw_paths[0]
            bundle_dir = primary.parent / primary.stem
        else:
            primary = resolved[0]
            bundle_dir = _shard_dir() / primary.stem
    moves: list[tuple[Path, Path]] = []
    for existing in existing_raw_paths:
        if bundle_dir is not None and existing.parent != bundle_dir:
            moves.append((existing, bundle_dir / existing.name))
    for new_path in resolved:
        dst = (bundle_dir / new_path.name) if bundle_dir is not None else (_shard_dir() / new_path.name)
        moves.append((new_path, dst))
    for _src, dst in moves:
        check_path_budget(
            dst,
            "The name comes from the file in incoming/: rename it there to something shorter "
            "and accept it again.",
        )
        if dst.exists():
            fail(f"Cannot promote: {rel_path(dst)} already exists.")
    # Stem uniqueness across raw/ (Gitea #64, widened by #67): the name this
    # call is about to claim - the bundle's name, or the lone file's stem when
    # no bundle forms - must not already belong to something this call does
    # not itself own. "Owns" means: one of the page's already-registered raw
    # files (the pitfall from the module docstring - a single file growing
    # into a bundle of its own name momentarily still occupies that name), or,
    # once a bundle already has >=2 registered files, the bundle directory
    # itself.
    claimed_name = bundle_dir.name if bundle_dir is not None else resolved[0].stem
    occupied = _occupied_stems(config.RAW_DIR)
    owned = set(existing_raw_paths)
    if len(existing_raw_paths) >= 2:
        owned.add(bundle_dir)
    holder = occupied.get(claimed_name)
    if holder is not None and holder not in owned:
        fail(_stem_collision_message(claimed_name, holder))
    moving_existing = [src for src, _dst in moves if src in existing_raw_paths]
    if moving_existing:
@@ -663,6 +883,181 @@ def raw_accept_command(
    )
 # --- raw pending --------------------------------------------------------------
 #
 # `incoming/` read as a queue (Gitea #112): what `wiki-ingest` without an
 # argument works through, one candidate per run, oldest first so that newer
 # material builds on - or corrects - what the wiki already took from older.
@dataclass(frozen=True)
 class _Candidate:
    kind: str  # file | bundle | folder
    paths: tuple[Path, ...]
    files: int
    mtime_ns: int
    reason: Optional[str]  # None when `raw accept` would take it as it stands
    def label(self) -> str:
        return ", ".join(rel_path(p) + ("/" if self.kind == "folder" else "") for p in self.paths)
    def mtime_iso(self) -> str:
        stamp = datetime.datetime.fromtimestamp(self.mtime_ns / 1e9).astimezone()
        return stamp.isoformat(timespec="seconds")
 def _refusal(fn, *args) -> Optional[str]:
    try:
        fn(*args)
    except _Refused as exc:
        return str(exc)
    return None
 def _folder_contents(folder: Path) -> tuple[int, int]:
    """How many non-directory entries sit below `folder`, and the newest
    mtime among them - `lstat` only, no content read and no link followed."""
    count, newest = 0, 0
    for dirpath, dirnames, filenames in os.walk(folder, followlinks=False):
        base = Path(dirpath)
        for name in filenames + [d for d in dirnames if (base / d).is_symlink()]:
            count += 1
            newest = max(newest, (base / name).lstat().st_mtime_ns)
    return count, newest
 def _plain_files(paths: list[Path]) -> None:
    _check_files(paths)
    _plan_file_moves(paths, [], None)
 def _pending_candidates() -> list[_Candidate]:
    """The candidates in `incoming/`, oldest first (Gitea #112 E1/E5).
    Only top-level entries count; dotfiles and empty directories are none.
    Top-level files sharing a stem are one `bundle` - a `raw fetch` pair, a PDF
    and its converted text. A top-level folder is one `folder`, with every
    file below it. A unit is as new as its newest part, so a bundle or folder
    sorts by the newest mtime it holds; a tie goes by the path's bytes."""
    incoming = _incoming_dir()
    if not incoming.is_dir():
        return []
    candidates: list[_Candidate] = []
    by_stem: dict[str, list[Path]] = {}
    for entry in incoming.iterdir():
        if entry.name.startswith("."):
            continue
        if entry.is_dir():
            if entry.is_symlink():
                count, newest = 1, entry.lstat().st_mtime_ns
            else:
                count, newest = _folder_contents(entry)
            if count:
                candidates.append(_Candidate(
                    "folder", (entry,), count, newest, _refusal(_plan_folder, entry)
                ))
        else:
            by_stem.setdefault(entry.stem, []).append(entry)
    for paths in by_stem.values():
        paths.sort(key=lambda p: os.fsencode(p.name))
        candidates.append(_Candidate(
            "file" if len(paths) == 1 else "bundle",
            tuple(paths),
            len(paths),
            max(p.lstat().st_mtime_ns for p in paths),
            _refusal(_plain_files, paths),
        ))
    candidates.sort(key=lambda c: (c.mtime_ns, os.fsencode(c.paths[0].name)))
    return candidates
@cli_contract.record(cli_contract.CommandRecord(
    path="raw pending",
    summary="List what waits in `incoming/`, oldest first, and name the entry an ingest without "
    "an argument takes next.",
    synopsis=(cli_contract.Variant(usage="raw pending [--json]"),),
    properties=cli_contract.Properties(
        effect=cli_contract.Effect.READ,
        idempotent=cli_contract.Idempotent.YES,
        atomic="Read-only",
        budget=cli_contract.Budget.COUNTED,
    ),
    notes=(
        "A candidate is a top-level entry of `incoming/`: a single `file`, a `bundle` of "
        "top-level files sharing a stem (a `raw fetch` pair, a PDF and its converted text), or "
        "a `folder` with every file below it. Dotfiles, empty directories and their contents "
        "are none; `mcp-upload/` is outside `incoming/` and never listed.",
        "Order: oldest first by modification time. A bundle or folder counts as new as its "
        "newest file; a tie goes by name. The mtime is when a document last changed only if "
        "it was copied with its timestamps kept (`cp -p`, `rsync -a`, an unpacked archive) - "
        "for a download or a `raw fetch` it is merely when it was dropped.",
        "Each candidate shows its path(s), kind, file count and mtime, and whether `raw "
        "accept` would take it as it stands - the same checks, minus `--fidelity`/"
        "`--authority`. One it would refuse is listed with the reason and skipped: it needs a "
        "human.",
        "The default is the first candidate `raw accept` would take, and the output names it.",
        "`--json` prints the same candidates in the same order: `kind`, `paths`, `files`, "
        "`mtime`, `acceptable`, `reason`, `default`.",
        "Reads directory listings and `lstat` only, never a file's content; an empty "
        "`incoming/` is exit 0 with nothing to do.",
    ),
    failures=(),
    examples=(
        "tools/wikitool raw pending",
        "tools/wikitool raw pending --json",
    ),
    see_also=(
        "`wikitool raw accept` - promotes the chosen candidate",
        "`instructions/wiki-ingest/SKILL.md` - ingest without an argument starts here",
        "`raw/CONTRACT.md` \"Getting a file in: incoming/\" - candidates and order, and why",
    ),
 ))
@app.command("pending")
 def raw_pending_command(
    json_out: bool = typer.Option(False, "--json", help="Print the candidates as JSON"),
 ):
    """List the candidates waiting in incoming/, oldest first, and name the
    default. See raw/CONTRACT.md "Getting a file in: incoming/"."""
    candidates = _pending_candidates()
    default = next((c for c in candidates if c.reason is None), None)
    if json_out:
        typer.echo(json.dumps([
            {
                "kind": c.kind,
                "paths": [rel_path(p) for p in c.paths],
                "files": c.files,
                "mtime": c.mtime_iso(),
                "acceptable": c.reason is None,
                "reason": c.reason,
                "default": c is default,
            }
            for c in candidates
        ], indent=2, ensure_ascii=False))
        return
    if not candidates:
        typer.echo("Nothing is waiting in incoming/.")
        return
    typer.echo(f"{len(candidates)} candidate(s) in incoming/, oldest first:")
    for number, c in enumerate(candidates, 1):
        marker = "*" if c is default else " "
        typer.echo(f"{marker} {number}. {c.kind:<6}  {c.label()}  ({c.files} file(s), {c.mtime_iso()})")
        if c.reason is not None:
            first, *rest = c.reason.splitlines()
            typer.echo(f"       not acceptable: {first}")
            for line in rest:
                typer.echo(f"       {line}")
    if default is None:
        typer.echo("No candidate can be accepted as it stands - each needs a human (reasons above).")
        return
    waiting = len(candidates) - 1
    typer.echo(
        f"Default (*): {default.label()} - the oldest candidate raw accept would take; "
        f"{waiting} more waiting after it."
    )
 # --- raw fetch ----------------------------------------------------------------
 #
 # The sanctioned intake for a URL the user names. It ends in `incoming/`, never
@@ -1,10 +1,13 @@
 import datetime
 import json
 import os
 import re
 import pytest
 import typer
 from chemenu import config
-from chemenu.commands.raw_cmd import raw_accept_command
+from chemenu.commands.raw_cmd import raw_accept_command, raw_pending_command
 from chemenu.frontmatter_io import read_page, write_page
 from chemenu.provenance import uncovered_raw_files
 from chemenu.kb_scan import load_kb_pages
@@ -85,40 +88,21 @@ def test_file_directly_in_incoming_is_accepted(tree):
    assert not src.exists()
-def test_subdirectory_under_incoming_is_ignored_not_inspected(tree):
+@pytest.mark.parametrize("rel", ["documents/a.pdf", "documents/sub/a.pdf"])
-    """Gitea #67: a subdirectory of incoming/ - old habit, old script - is
+def test_file_in_a_subdirectory_is_refused_naming_both_routes(tree, capsys, rel):
-    tolerated and ignored rather than read as a type classification. This is
+    """Gitea #112: a subdirectory of incoming/ is a source of its own, no
-    the MINOR condition named in the issue's Versionsteil."""
+    longer a tolerated, ignored type directory - a file inside one is refused,
-    (tree / "incoming/videos").mkdir()
+    and the message names accepting the folder and moving the file up."""
-    src = tree / "incoming/videos/clip.mp4"
+    src = tree / "incoming" / rel
-    src.write_bytes(b"x")
+    src.parent.mkdir(parents=True)
    _accept(src)
    assert (tree / _shard() / "clip.mp4").is_file()
    assert not src.exists()
 def test_nested_too_deep_is_rejected(tree):
    nested = tree / "incoming/documents/sub"
    nested.mkdir(parents=True)
    src = nested / "deep.pdf"
    src.write_bytes(b"x")
    before = _tree_files(tree)
    with pytest.raises(typer.Exit):
        _accept(src)
-    assert src.exists()
+    assert _tree_files(tree) == before
-
+    out = " ".join(capsys.readouterr().out.split())
-
+    assert "raw accept incoming/documents" in out
-def test_files_from_different_ignored_subdirs_bundle_together(tree):
+    assert "move the file up into incoming/" in out
    """No more per-call type agreement to enforce (Gitea #67): which ignored
    subdirectory each file happened to sit under is irrelevant now."""
    a = tree / "incoming/documents/a.pdf"
    b = tree / "incoming/notes/a.md"
    a.parent.mkdir(parents=True)
    b.parent.mkdir(parents=True)
    a.write_bytes(b"a")
    b.write_text("b", encoding="utf-8")
    _accept(a, b)
    bundle = tree / _shard() / "a"
    assert (bundle / "a.pdf").exists() and (bundle / "a.md").exists()
 def test_same_file_passed_twice_is_rejected(tree):
@@ -573,15 +557,13 @@ def test_replaces_rejects_filename_mismatch(tree):
    assert new.exists()
-def test_replaces_across_legacy_directories_is_now_allowed(tree):
+def test_replaces_writes_back_into_a_legacy_directory(tree):
-    """Gitea #67 removes the type-directory-match check `--replaces` used to
+    """Gitea #67 removed the type-directory-match check `--replaces` used to
-    enforce: a subdirectory of incoming/ carries no meaning any more, so
+    enforce: a raw/notes/ file is replaced by a file dropped flat into
-    replacing a raw/notes/ file with an incoming file dropped under an
+    incoming/ like any other."""
    unrelated incoming/documents/ works exactly like one dropped flat."""
    target = tree / "raw/notes/handbuch.md"
    target.write_text("old", encoding="utf-8")
-    new = tree / "incoming/documents/handbuch.md"
+    new = tree / "incoming/handbuch.md"
    new.parent.mkdir(parents=True)
    new.write_text("new", encoding="utf-8")
    _accept(new, replaces=target, fidelity=None, authority=None)
@@ -590,6 +572,21 @@ def test_replaces_across_legacy_directories_is_now_allowed(tree):
    assert not new.exists()
 def test_replaces_refuses_an_incoming_file_in_a_subdirectory(tree):
    """Gitea #112: the subdirectory tolerance is gone for --replaces too."""
    target = tree / "raw/notes/handbuch.md"
    target.write_text("old", encoding="utf-8")
    new = tree / "incoming/documents/handbuch.md"
    new.parent.mkdir(parents=True)
    new.write_text("new", encoding="utf-8")
    with pytest.raises(typer.Exit):
        _accept(new, replaces=target, fidelity=None, authority=None)
    assert target.read_text(encoding="utf-8") == "old"
    assert new.read_text(encoding="utf-8") == "new"
 def test_replaces_rejects_nonexistent_target(tree):
    new = tree / "incoming/handbuch.md"
    new.write_text("new", encoding="utf-8")
@@ -654,3 +651,263 @@ def test_accept_takes_a_target_at_the_path_budget(tree):
    _accept(src)
    assert (tree / _shard() / src.name).exists()
    assert not src.exists()
 # --- A folder is one source (Gitea #112) ---
 def _make_folder(tree, name, files):
    folder = tree / "incoming" / name
    for rel, data in files.items():
        (folder / rel).parent.mkdir(parents=True, exist_ok=True)
        (folder / rel).write_bytes(data)
    return folder
 _BAUM = {"a.md": b"A", "sub/b.md": b"B", "sub/deep/README.md": b"R"}
 def test_folder_moves_every_file_byte_identical_and_leaves_no_folder(tree):
    folder = _make_folder(tree, "baum", _BAUM)
    _accept(folder)
    bundle = tree / _shard() / "baum"
    assert {str(p.relative_to(bundle)): p.read_bytes() for p in bundle.rglob("*") if p.is_file()} == _BAUM
    assert not folder.exists()
    assert sorted(p.name for p in (tree / "incoming").iterdir()) == []
 def test_folder_takes_same_named_files_in_different_subfolders(tree):
    folder = _make_folder(tree, "doku", {"x/README.md": b"x", "y/README.md": b"y"})
    _accept(folder)
    bundle = tree / _shard() / "doku"
    assert (bundle / "x/README.md").read_bytes() == b"x"
    assert (bundle / "y/README.md").read_bytes() == b"y"
 def test_folder_prints_raw_files_and_capture_pair(tree, capsys):
    folder = _make_folder(tree, "baum", _BAUM)
    _accept(folder, fidelity="secondhand", authority="opinion")
    out = "".join(capsys.readouterr().out.split())  # success() wraps long lines
    assert f"--setraw_files={_shard()}/baum/a.md,{_shard()}/baum/sub/b.md,{_shard()}/baum/sub/deep/README.md" in out
    assert "--setfidelity=secondhand--setauthority=opinion" in out
@pytest.mark.parametrize("extra", ["sub/.hidden", ".git/config", "link"])
 def test_folder_with_hidden_entry_or_symlink_is_refused_and_named(tree, capsys, extra):
    folder = _make_folder(tree, "baum", _BAUM)
    if extra == "link":
        (folder / "sub/link").symlink_to(folder / "a.md")
        named = "baum/sub/link"
    else:
        (folder / extra).parent.mkdir(parents=True, exist_ok=True)
        (folder / extra).write_bytes(b"h")
        named = f"baum/{extra.split('/')[0]}" if extra.startswith(".") else f"baum/{extra}"
    before = _tree_files(tree)
    links_before = sorted(str(p) for p in tree.rglob("*") if p.is_symlink())
    with pytest.raises(typer.Exit):
        _accept(folder)
    assert _tree_files(tree) == before
    assert sorted(str(p) for p in tree.rglob("*") if p.is_symlink()) == links_before
    assert named in capsys.readouterr().out
 def test_empty_folder_is_refused(tree):
    (tree / "incoming/leer/sub").mkdir(parents=True)
    with pytest.raises(typer.Exit):
        _accept(tree / "incoming/leer")
    assert (tree / "incoming/leer/sub").is_dir()
@pytest.mark.parametrize("combination", ["other-file", "page", "replaces"])
 def test_folder_with_anything_else_is_refused(tree, combination):
    folder = _make_folder(tree, "baum", _BAUM)
    other = tree / "incoming/other.md"
    other.write_text("o", encoding="utf-8")
    (tree / "raw/notes/a.md").write_bytes(b"old")
    _write_source(tree / "kb", "Source - Old", ["raw/notes/a.md"])
    before = _tree_files(tree)
    with pytest.raises(typer.Exit):
        if combination == "other-file":
            _accept(folder, other)
        elif combination == "page":
            _accept(folder, page="Source - Old")
        else:
            _accept(folder, replaces=tree / "raw/notes/a.md", fidelity=None, authority=None)
    assert _tree_files(tree) == before
 def test_folder_needs_both_capture_fields(tree):
    folder = _make_folder(tree, "baum", _BAUM)
    with pytest.raises(typer.Exit):
        raw_accept_command(files=[folder], fidelity=None, authority="reporting", page=None, replaces=None, dry_run=False)
    assert folder.is_dir()
 def test_folder_name_already_occupied_is_refused_with_the_rename_route(tree, capsys):
    (tree / "raw/documents/baum.pdf").write_bytes(b"pdf")
    folder = _make_folder(tree, "baum", _BAUM)
    before = _tree_files(tree)
    with pytest.raises(typer.Exit):
        _accept(folder)
    assert _tree_files(tree) == before
    out = " ".join(capsys.readouterr().out.split())
    assert "Rename the folder in incoming/" in out
    assert "--replaces raw/" not in out
 def test_folder_target_over_the_path_budget_is_refused_before_any_move(tree):
    long_name = "n" * (161 - len(f"{_shard()}/baum/sub/") - len(".md")) + ".md"
    folder = _make_folder(tree, "baum", {"a.md": b"A", f"sub/{long_name}": b"L"})
    before = _tree_files(tree)
    with pytest.raises(typer.Exit):
        _accept(folder)
    assert _tree_files(tree) == before
 def test_folder_dry_run_lists_every_move_and_the_folder_and_changes_nothing(tree, capsys):
    folder = _make_folder(tree, "baum", _BAUM)
    before = _tree_files(tree)
    _accept(folder, dry_run=True)
    assert _tree_files(tree) == before
    out = capsys.readouterr().out
    for rel in _BAUM:
        assert f"incoming/baum/{rel} -> {_shard()}/baum/{rel}" in out
    assert "would remove incoming/baum/" in out
 def test_folder_files_are_uncovered_not_broken_afterwards(tree):
    folder = _make_folder(tree, "baum", _BAUM)
    _accept(folder)
    from chemenu.provenance import broken_raw_refs
    pages = load_kb_pages(tree / "kb")
    assert sorted(f for f in uncovered_raw_files(tree / "raw", pages) if "/baum/" in f) == sorted(
        f"{_shard()}/baum/{rel}" for rel in _BAUM
    )
    assert [r for r in broken_raw_refs(pages) if "baum" in str(r)] == []
 def test_file_and_bundle_at_the_top_still_work(tree):
    single = tree / "incoming/a.pdf"
    single.write_bytes(b"a")
    _accept(single)
    assert (tree / _shard() / "a.pdf").read_bytes() == b"a"
    html, md = tree / "incoming/post.html", tree / "incoming/post.md"
    html.write_bytes(b"h")
    md.write_bytes(b"m")
    _accept(html, md)
    assert sorted(p.name for p in (tree / _shard() / "post").iterdir()) == ["post.html", "post.md"]
 # --- raw pending (Gitea #112) ---
 def _pending(capsys):
    capsys.readouterr()
    raw_pending_command(json_out=True)
    return json.loads(capsys.readouterr().out)
 def _age(path, seconds):
    for p in [path, *path.rglob("*")] if path.is_dir() else [path]:
        os.utime(p, ns=(seconds * 10**9, seconds * 10**9), follow_symlinks=False)
 def test_pending_on_an_empty_inbox_is_nothing(tree, capsys):
    (tree / "incoming/.gitkeep").write_bytes(b"")
    (tree / "incoming/documents").mkdir()
    (tree / "incoming/notes/sub").mkdir(parents=True)
    assert _pending(capsys) == []
    raw_pending_command(json_out=False)
    assert "Nothing is waiting" in capsys.readouterr().out
 def test_pending_groups_a_fetch_pair_into_one_bundle(tree, capsys):
    (tree / "incoming/post.html").write_bytes(b"h")
    (tree / "incoming/post.md").write_bytes(b"m")
    [candidate] = _pending(capsys)
    assert candidate["kind"] == "bundle"
    assert candidate["paths"] == ["incoming/post.html", "incoming/post.md"]
    assert candidate["files"] == 2
 def test_pending_counts_a_folder_once_with_all_its_files(tree, capsys):
    _make_folder(tree, "baum", _BAUM)
    [candidate] = _pending(capsys)
    assert candidate["kind"] == "folder"
    assert candidate["paths"] == ["incoming/baum"]
    assert candidate["files"] == 3
 def test_pending_orders_oldest_first_by_the_newest_part_then_by_name(tree, capsys):
    a = tree / "incoming/a.md"
    a.write_bytes(b"a")
    _age(a, 3000)
    b_html, b_md = tree / "incoming/b.html", tree / "incoming/b.md"
    b_html.write_bytes(b"h")
    b_md.write_bytes(b"m")
    _age(b_html, 1000)
    _age(b_md, 4000)  # the bundle is as new as its newest part
    folder = _make_folder(tree, "c", {"x.md": b"x", "y/z.md": b"z"})
    _age(folder, 1000)
    _age(folder / "y/z.md", 2000)
    tie = tree / "incoming/0-tie.md"
    tie.write_bytes(b"t")
    _age(tie, 2000)
    order = [c["paths"][0] for c in _pending(capsys)]
    assert order == ["incoming/0-tie.md", "incoming/c", "incoming/a.md", "incoming/b.html"]
 def test_pending_marks_what_raw_accept_would_refuse_and_defaults_to_the_first_acceptable(tree, capsys):
    hidden = _make_folder(tree, "hidden", {"a.md": b"a", ".DS_Store": b"x"})
    _age(hidden, 1000)
    (tree / "raw/documents/taken.pdf").write_bytes(b"pdf")
    taken = tree / "incoming/taken.md"
    taken.write_bytes(b"t")
    _age(taken, 2000)
    long = tree / "incoming" / _incoming_name_for(161)
    long.write_bytes(b"l")
    _age(long, 3000)
    fine = tree / "incoming/fine.md"
    fine.write_bytes(b"f")
    _age(fine, 4000)
    candidates = _pending(capsys)
    assert [c["acceptable"] for c in candidates] == [False, False, False, True]
    assert ".DS_Store" in candidates[0]["reason"]
    assert "already claims" in candidates[1]["reason"]
    assert "160" in candidates[2]["reason"] or "budget" in candidates[2]["reason"].lower()
    assert [c["default"] for c in candidates] == [False, False, False, True]
    raw_pending_command(json_out=False)
    out = capsys.readouterr().out
    assert "Default (*): incoming/fine.md" in out
    assert "not acceptable" in out
 def test_pending_changes_nothing_not_even_mtimes(tree, capsys):
    _make_folder(tree, "baum", _BAUM)
    (tree / "incoming/post.html").write_bytes(b"h")
    (tree / "raw/documents/old.pdf").write_bytes(b"o")
    def snapshot():
        return {
            str(p.relative_to(tree)): (p.read_bytes() if p.is_file() else None, p.lstat().st_mtime_ns)
            for root in ("incoming", "raw", "kb") for p in sorted((tree / root).rglob("*"))
        }
    before = snapshot()
    _pending(capsys)
    raw_pending_command(json_out=False)
    assert snapshot() == before
 def test_pending_json_and_text_list_the_same_candidates_in_the_same_order(tree, capsys):
    for name, age in (("a.md", 3000), ("b.md", 1000), ("c.md", 2000)):
        (tree / "incoming" / name).write_bytes(b"x")
        _age(tree / "incoming" / name, age)
    paths = [c["paths"][0] for c in _pending(capsys)]
    raw_pending_command(json_out=False)
    text = capsys.readouterr().out
    listed = [m.group(1) for m in re.finditer(r"^[* ] \d+\. \w+\s+(\S+)", text, re.MULTILINE)]
    assert listed == paths
@@ -371,6 +371,18 @@ def test_html_outside_incoming_is_refused(tree):
    assert list((tree / "incoming").iterdir()) == []
 def test_html_in_a_subdirectory_of_incoming_is_refused(tree, capsys):
    """A subdirectory of incoming/ is a source of its own (Gitea #112), so a
    saved page inside one is not derived in place."""
    saved = tree / "incoming/articles/post.html"
    saved.parent.mkdir()
    saved.write_bytes(ARTICLE)
    before = _snapshot(tree)
    _refused(html=saved, source_url="https://example.org/post")
    assert _snapshot(tree) == before
    assert "raw accept incoming/articles" in " ".join(capsys.readouterr().out.split())
 def test_html_without_url_is_refused(tree):
    saved = tree / "incoming/post.html"
    saved.write_bytes(ARTICLE)