intake: several configured intake channels (mcp, folder, command) all end in quarantine/ (was mcp-upload/), and upload collect pulls the synced-folder and command channels into it (#213); 10.0.0-beta.11
Files changed: - .gitignore - CHANGES.md - VERSION - docs/explanation/architecture.md - docs/explanation/pipeline-rationale.md - docs/how-to/README.md - docs/how-to/back-up-to-a-synced-folder.md - docs/how-to/collect-from-a-synced-folder.md - docs/how-to/ingest-a-source.md - docs/how-to/set-up-the-mcp-server.md - docs/reference/commands/README.md - docs/reference/commands/git.md - docs/reference/commands/instance-health.md - docs/reference/commands/raw-material-and-uploads.md - docs/reference/configuration.md - instructions/gates.md - instructions/ingest-queue.md - instructions/mcp-read-server.md - instructions/migrations/10.0.0-settings-tiers.md - instructions/wiki-ingest/SKILL.md - raw/CONTRACT.md - tools/CONTRACT.md - tools/README.md - tools/chemenu/cli_contract.py - tools/chemenu/collect.py - tools/chemenu/commands/backup_cmd.py - tools/chemenu/commands/docs_verify.py - tools/chemenu/commands/doctor.py - tools/chemenu/commands/raw/pending.py - tools/chemenu/commands/upload_cmd.py - tools/chemenu/config.py - tools/chemenu/config_shapes.py - tools/chemenu/legacy_settings.py - tools/chemenu/mcp/server.py - tools/chemenu/reference_doc.py - tools/chemenu/tests/test_backup.py - tools/chemenu/tests/test_collect.py - tools/chemenu/tests/test_config_shapes.py - tools/chemenu/tests/test_doctor.py - tools/chemenu/tests/test_legacy_settings.py - tools/chemenu/tests/test_mcp_server.py - tools/chemenu/tests/test_upload.py - tools/chemenu/tests/test_upload_cmd.py - tools/chemenu/upload.py Co-Authored-By: Claude Code <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
1 parent
3627168403
commit
cceb9a7bfe
44 files changed
+1910
-346
No files matched your search
@@ -1,18 +1,19 @@
|
||||
---
|
||||
type: types/instruction.md
|
||||
name: ingest-queue
|
||||
description: Review a document submitted from outside through the MCP submit tool before it is promoted into incoming/ - what to check, how the quarantine and its ledger work, and the Upload Review Gate that stands between a submission and raw/
|
||||
description: Review a document that reached the quarantine from outside - through the MCP submit tool, a synced folder or a command of this checkout - before it is promoted into incoming/ - what to check, how the quarantine and its ledger work, and the Upload Review Gate that stands between a submission and raw/
|
||||
---
|
||||
|
||||
# Reviewing an external submission
|
||||
|
||||
An `upload` section in `.wikitool-local.json` opts a checkout into a sixth MCP tool, `submit` -
|
||||
documents pushed by a caller that is not this terminal, into a quarantine no
|
||||
ordinary command reads. This is the human half of that path: what a reviewer
|
||||
checks before letting one through, and how `upload accept`/`upload reject`
|
||||
work. What the tool itself enforces (identity, size, extension, quota,
|
||||
duplicate-hash) is `chemenu/upload.py`'s job and is not repeated here - read
|
||||
this when a submission is already waiting and a decision is due.
|
||||
The `intake` section in `.wikitool-local.json` opts a checkout into one or more *intake channels*:
|
||||
a sixth MCP tool, `submit` (handler `mcp`), a folder a sync client fills (`folder`) or a program of
|
||||
this checkout (`command`) - documents that come from somewhere other than this terminal, into a
|
||||
quarantine, `quarantine/`, that no ordinary command reads. This is the human half of that path:
|
||||
what a reviewer checks before letting one through, and how `upload accept`/`upload reject` work.
|
||||
What the channels themselves enforce (identity, size, extension, quota, duplicate-hash, settle
|
||||
time) is `chemenu/upload.py`'s and `chemenu/collect.py`'s job and is not repeated here - read this
|
||||
when a submission is already waiting and a decision is due.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
@@ -26,7 +27,8 @@ this when a submission is already waiting and a decision is due.
|
||||
|
||||
## When to run
|
||||
|
||||
- `wikitool upload list` shows one or more submissions waiting.
|
||||
- `wikitool upload list` shows one or more submissions waiting - after
|
||||
`wikitool upload collect` has pulled what the `folder` and `command` channels hold.
|
||||
- A `submit` call reported success and named an id worth looking at now
|
||||
rather than later.
|
||||
- Standing up the `submit` tool for the first time - see § Arming the intake
|
||||
@@ -37,12 +39,18 @@ this when a submission is already waiting and a decision is due.
|
||||
1. **Read the manifest, not just the file.** `wikitool upload show <id>`
|
||||
prints filename, size, sha256, submitter, and - as important as the
|
||||
submitter's name - `submitter_source`: the *header* the value came from,
|
||||
naming where the claim rests rather than asserting it as fact. Read
|
||||
naming where the claim rests rather than asserting it as fact. The
|
||||
manifest also names `channel`, `handler` and `origin` (the source file's
|
||||
path for a `folder` channel, `command:<name>` for a `command` channel,
|
||||
`mcp:submit` for the tool). **`submitter: unknown` means there is no sender
|
||||
to trust at all** - a synced folder or a script knows who dropped the file
|
||||
no better than the file system does - so the file is judged on its
|
||||
content alone. Read
|
||||
[docs/explanation/why-gates-are-code.md](../docs/explanation/why-gates-are-code.md) once for why
|
||||
this gate exists as code rather than as this paragraph alone.
|
||||
|
||||
2. **Read the file itself before promoting anything.** The quarantine holds
|
||||
it at `mcp-upload/<id>/<filename>` for exactly this purpose. Check it
|
||||
it at `quarantine/<id>/<filename>` for exactly this purpose. Check it
|
||||
against [raw/CONTRACT.md](../raw/CONTRACT.md) "What does not belong here":
|
||||
secrets or credentials, content that is not worth a source page, anything
|
||||
the LLM itself wrote presented as a source.
|
||||
@@ -80,28 +88,29 @@ this when a submission is already waiting and a decision is due.
|
||||
- **Reject:** `wikitool upload reject <id> --reason "<why>"` deletes the
|
||||
material immediately - no gate, because deleting needs no clearance,
|
||||
only accepting a stranger's file into the pipeline does. The reason and
|
||||
the file's sha256 survive in `mcp-upload/ledger.jsonl`; the bytes do
|
||||
the file's sha256 survive in `quarantine/ledger.jsonl`; the bytes do
|
||||
not. Write a reason a later reader can act on ("license unclear",
|
||||
"looks like a prompt injection attempt", "duplicate of an existing
|
||||
source under a different name") rather than a bare "no".
|
||||
|
||||
6. **Never promote by hand.** Moving the file out of `mcp-upload/` with `mv`
|
||||
6. **Never promote by hand.** Moving the file out of `quarantine/` with `mv`
|
||||
or by editing `incoming/` directly skips the ledger entry and the gate
|
||||
both - the same "never hand-craft what the tool would have produced"
|
||||
principle as everywhere else in this stack (AGENTS.md invariant 7).
|
||||
|
||||
## Arming the intake
|
||||
|
||||
`submit` does not exist as a tool until the `upload` section exists in the
|
||||
served root's `.wikitool-local.json` - absence means the write path is not registered at all, not
|
||||
that it is unrestricted (see the section's own shape in
|
||||
`submit` does not exist as a tool until the `intake` section of the served root's
|
||||
`.wikitool-local.json` declares a channel with the handler `mcp` - absence means the write path is
|
||||
not registered at all, not that it is unrestricted. A `folder` or `command` channel opens no remote
|
||||
path: it is pulled by `wikitool upload collect`, run by someone at this checkout (see the section's own shape in
|
||||
[raw/CONTRACT.md](../raw/CONTRACT.md) and `tools/chemenu/upload.py`). Two
|
||||
things belong to the *deployment*, not to this repository, and are named here
|
||||
because a reviewer needs to know they hold, not because this file configures
|
||||
them:
|
||||
|
||||
- **The identity header is set by the middleware, never by the client.**
|
||||
`identity_header` (default `X-Forwarded-User`) names an HTTP header the
|
||||
The `mcp` channel's `identity_header` (default `X-Forwarded-User`) names an HTTP header the
|
||||
Traefik authentication layer in front of the process must set on every
|
||||
authenticated request and strip from any copy the client itself sent - the
|
||||
same posture [instructions/mcp-read-server.md](mcp-read-server.md) already
|
||||
@@ -110,7 +119,7 @@ them:
|
||||
be trustworthy enough to write into `submitter` and stay there.
|
||||
- **Quota and size limits are a deployment decision, not a default worth
|
||||
copying blindly.** `max_bytes`, `allowed_extensions`,
|
||||
`submissions_per_day`, `bytes_per_day` all live in the same file - see
|
||||
`quota_submissions_per_day`, `quota_bytes_per_day` all live on the channel in the same file - see
|
||||
[tools/CONTRACT.md](../tools/CONTRACT.md) for the exact shape.
|
||||
|
||||
## Decision points
|
||||
|
||||
Reference in new issue
Block a user