tools: Export- und Eingangsordner im Backup-Ziel - Überschneidungsregel nur noch auf <ziel>/git und den eigenen Maschinenordner, Laufzeitprüfung auf Vorfahren in upload collect und export run (#216); 10.1.0-beta.16
Files changed: - CHANGES.md - VERSION - docs/how-to/back-up-to-a-synced-folder.md - docs/how-to/collect-from-a-synced-folder.md - docs/how-to/export-to-a-synced-folder.md - docs/reference/commands/exports.md - docs/reference/commands/raw-material-and-uploads.md - docs/reference/configuration.md - tools/CONTRACT.md - tools/chemenu/backup.py - tools/chemenu/collect.py - tools/chemenu/commands/export_cmd.py - tools/chemenu/commands/upload_cmd.py - tools/chemenu/config_shapes.py - tools/chemenu/page_export.py - tools/chemenu/tests/test_collect.py - tools/chemenu/tests/test_export_run.py - tools/chemenu/upload.py Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
1 parent
b48a35b224
commit
02bf3b0403
18 files changed
+312
-41
No files matched your search
+2
-1
@@ -1913,6 +1913,7 @@ Pull material from the configured `folder` and `command` intake channels into th
|
||||
- Everything a channel delivers lands in `quarantine/<id>/` unchanged, with a manifest that names `channel`, `handler` and `origin`; the submitter is `unknown`. Nothing reaches `incoming/` or `raw/` - that is `upload accept`, behind its gate, and an agent may run `collect` because collecting gives no trust.
|
||||
- `folder`: only the top level of `path`. A file is taken once its modification time and, where the platform has it, its creation time are older than `settle_seconds`, its extension and size fit, and its name passes the filename rules. It is removed from the folder only after its copy is fully written and the source still has the size and sha256 that were read; if it changed meanwhile, nothing is kept and it stays. A file whose sha256 already waits in the quarantine is reported as a duplicate and removed from the folder, because its content is kept. Subdirectories, hidden files (names starting with `.` or `~$`, or the hidden or system attribute on Windows), symbolic links, other extensions and oversized files are left untouched and reported.
|
||||
- `command`: the program of `commands.<name>` runs without a shell, with stdin closed, a 300 s timeout and an empty staging directory under `quarantine/.staging/` as its last argument. On exit 0 each regular file at the top of that directory with an allowed extension and size becomes a submission; on anything else nothing is taken and the end of stderr is printed. The staging directory is always removed.
|
||||
- `folder` never empties backup or export data: a `path` that holds a `.chemenu-backup.json` (a machine's folder of the backup, any machine's) or a `.wikitool-export.json` (an export), is a bare git repository, or lies in one of these, is that channel's error - nothing is read and nothing removed. A `path` above such folders is fine, since only its top level is read.
|
||||
- Channels are independent: when one fails the others still run, and the exit status is 1 if any channel or file failed. With no pulled channel it prints `nothing to collect` and exits 0.
|
||||
- Exits 1, before it does anything, while an `mcp-upload/` of an earlier version still holds submissions, and when the `intake` section is malformed.
|
||||
- Not idempotent: a second run finds the folder emptied.
|
||||
@@ -2459,7 +2460,7 @@ Write the derived, read-only reading copy of `kb/` into the folders the `export`
|
||||
- Every Markdown file starts with a frontmatter block whose `wikitool_export` line says `kind=pages instance=<origin URL without userinfo, or local> commit=<newest commit touching this page>`, followed by `title`, `type`, `summary`, `modified` where the page has them (`frontmatter: summary`, the default; `minimal` keeps only the marker, `full` the whole frontmatter), then the page text. A file keeps its path below `kb/`. A wikilink to an exported page becomes a relative Markdown link (`links: relative`, the default) or plain text (`links: text`); a link to a page that is not exported is always plain text. Footnote markers stay; code is never rewritten. LF endings, no timestamp: the same pages give the same bytes, and a commit that touches one page rewrites that page's file alone - plus any file whose links change because a page entered or left the selection. The manifest's `commit` is the newest commit touching any selected page; all of them come from one `git log`, however many pages are selected.
|
||||
- Handler `command`: the Markdown is rendered into a staging directory, the program of `export.commands` runs with no shell, stdin closed and a timeout of 300 s, and receives the Markdown directory and an empty output directory as its last two arguments. On exit 0 the regular files in the output directory (recursively) become the target's content; any other end leaves the target unchanged and prints the end of its stderr. The staging directory is always removed.
|
||||
- The folder is only ever touched by what it holds in its manifest, `.wikitool-export.json`: the files an earlier run wrote, each with its sha256. A run removes only manifest files that are no longer exported (and folders it created, once empty), rewrites a manifest file that is missing, leaves every other file byte for byte as it was, and writes nothing - not even a modification time - for a file that did not change. A run that changes nothing writes nothing at all.
|
||||
- Refuses a target, before writing anything into it: a non-empty folder with no manifest (it is somebody's, not an export), a manifest file that was changed in the folder since the last run (naming it), a foreign file where an exported one belongs, an empty selection (it would delete the earlier export), an unknown field in a predicate. A missing folder is created.
|
||||
- Refuses a target, before writing anything into it: a non-empty folder with no manifest (it is somebody's, not an export), a folder that holds or lies in backup data (a `.chemenu-backup.json` - any machine's, also one this checkout does not know - or a bare git repository), a manifest file that was changed in the folder since the last run (naming it), a foreign file where an exported one belongs, an empty selection (it would delete the earlier export), an unknown field in a predicate. A missing folder is created.
|
||||
- Refuses everything, before any target: uncommitted changes in `kb/` or `types/` (untracked files included), no git checkout, a page under `kb/` with unreadable frontmatter, an unknown target name, a broken `export` section.
|
||||
- Output per target: `new` (with the page titles), `changed`, `removed`, the count `unchanged`, or `failed` with the reason. A page that newly enters a selection is reported as new, not gated. Exit 0 when every target ran; exit 1 when any failed - the others were still served.
|
||||
- A target with `after_publish: true` is also refreshed by every `publish` that committed, `--no-push` included; a failure there is a warning, the commit stands, and this command is the repeat.
|
||||
|
||||
@@ -209,6 +209,47 @@ def read_config(root: "Path | str | None" = None) -> Optional[BackupConfig]:
|
||||
machine_name=machine_name, extra=tuple(section["extra"]))
|
||||
|
||||
|
||||
def written_folders(root: "Path | str | None" = None) -> list[tuple[str, Path]]:
|
||||
"""The folders this instance's backup writes into - `<target>/git` and
|
||||
`<target>/<own machine>` - each with a label for a message. Whatever else
|
||||
lies in `<target>` is the sync folder's, not the backup's, so an export or
|
||||
an intake folder may sit beside these two. Read as leniently as the other
|
||||
features read each other's folders: a broken section yields none, it is
|
||||
reported where it is read (`doctor`'s `backup` check)."""
|
||||
try:
|
||||
cfg = read_config(root)
|
||||
except ValidationError:
|
||||
return []
|
||||
if cfg is None:
|
||||
return []
|
||||
return [(f"the backup's git folder ({cfg.git_dir})", cfg.git_dir),
|
||||
(f"the backup's machine folder ({cfg.machine_dir})", cfg.machine_dir)]
|
||||
|
||||
|
||||
def _is_bare_repo_dir(folder: Path) -> bool:
|
||||
return (folder / "HEAD").is_file() and (folder / "objects").is_dir()
|
||||
|
||||
|
||||
def data_marker(folder: Path, extra_manifests: tuple[str, ...] = ()) -> Optional[str]:
|
||||
"""What marks `folder`, or one of its ancestors, as data that belongs to the
|
||||
backup - a machine folder (the backup manifest), a bare repository - or to
|
||||
a manifest named in `extra_manifests`; None when nothing does. The
|
||||
configuration cannot see another machine's folder, or one a machine starts
|
||||
to fill later, so a run asks the folder itself; the ancestors cover a
|
||||
folder inside one. Looked at through links, and without a `git` call."""
|
||||
resolved = real_path(folder)
|
||||
for candidate in (resolved, *resolved.parents):
|
||||
try:
|
||||
for name in (MANIFEST_NAME, *extra_manifests):
|
||||
if (candidate / name).is_file():
|
||||
return f"{candidate} holds {name}"
|
||||
if _is_bare_repo_dir(candidate):
|
||||
return f"{candidate} is a git repository"
|
||||
except OSError:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def require_config(root: "Path | str | None" = None) -> BackupConfig:
|
||||
found = read_config(root)
|
||||
if found is None:
|
||||
|
||||
@@ -19,6 +19,9 @@ What a pull guarantees:
|
||||
in the channel's `path`. It writes nothing there, deletes no directory and
|
||||
follows no symbolic link.
|
||||
- Whatever it does not take stays where it is and is reported with its reason.
|
||||
- It never empties data of the backup or of an export: a folder that holds a
|
||||
backup or export manifest or is a bare repository, or lies in one, is the
|
||||
channel's error - nothing read, nothing removed.
|
||||
|
||||
No `typer`, no `rich`: the CLI renders a `ChannelResult`.
|
||||
"""
|
||||
@@ -33,7 +36,7 @@ from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Callable, Optional
|
||||
|
||||
from chemenu import upload
|
||||
from chemenu import backup, page_export, upload
|
||||
from chemenu.errors import ValidationError
|
||||
|
||||
COMMAND_TIMEOUT_SECONDS = 300
|
||||
@@ -170,6 +173,11 @@ def collect_folder(root: "Path | str", channel: upload.Channel, now: Optional[fl
|
||||
if folder is None or not folder.is_dir():
|
||||
result.error = f"the folder {folder} does not exist or is not a directory"
|
||||
return result
|
||||
marker = backup.data_marker(folder, (page_export.MANIFEST_NAME,))
|
||||
if marker:
|
||||
result.error = (f"the folder {folder} lies in backup or export data ({marker}) - a channel empties "
|
||||
"its folder, so nothing was read or removed")
|
||||
return result
|
||||
clock = time.time() if now is None else now
|
||||
try:
|
||||
entries = sorted(os.scandir(folder), key=lambda entry: entry.name)
|
||||
|
||||
@@ -483,7 +483,9 @@ def export_guidelines_command(
|
||||
"nothing - not even a modification time - for a file that did not change. A run that "
|
||||
"changes nothing writes nothing at all.",
|
||||
"Refuses a target, before writing anything into it: a non-empty folder with no manifest "
|
||||
"(it is somebody's, not an export), a manifest file that was changed in the folder since "
|
||||
"(it is somebody's, not an export), a folder that holds or lies in backup data (a "
|
||||
"`.chemenu-backup.json` - any machine's, also one this checkout does not know - or a "
|
||||
"bare git repository), a manifest file that was changed in the folder since "
|
||||
"the last run (naming it), a foreign file where an exported one belongs, an empty "
|
||||
"selection (it would delete the earlier export), an unknown field in a predicate. A "
|
||||
"missing folder is created.",
|
||||
|
||||
@@ -322,6 +322,11 @@ def _result_lines(result: collect_module.ChannelResult) -> list[str]:
|
||||
"argument. On exit 0 each regular file at the top of that directory with an allowed "
|
||||
"extension and size becomes a submission; on anything else nothing is taken and the end of "
|
||||
"stderr is printed. The staging directory is always removed.",
|
||||
"`folder` never empties backup or export data: a `path` that holds a `.chemenu-backup.json` "
|
||||
"(a machine's folder of the backup, any machine's) or a `.wikitool-export.json` (an "
|
||||
"export), is a bare git repository, or lies in one of these, is that channel's error - "
|
||||
"nothing is read and nothing removed. A `path` above such folders is fine, since only "
|
||||
"its top level is read.",
|
||||
"Channels are independent: when one fails the others still run, and the exit status is 1 "
|
||||
"if any channel or file failed. With no pulled channel it prints `nothing to collect` "
|
||||
"and exits 0.",
|
||||
|
||||
@@ -494,7 +494,10 @@ INTAKE = Shape(
|
||||
f"submitter's identity, default `{INTAKE_DEFAULT_IDENTITY_HEADER}`) and, required, "
|
||||
"`quota_submissions_per_day` and `quota_bytes_per_day` (per submitter, over any "
|
||||
"rolling 24 hours); `folder` takes `path` (required: an absolute path outside the "
|
||||
"instance, read without descending into subdirectories) and `settle_seconds` (default "
|
||||
"instance, read without descending into subdirectories; it must not be or lie in "
|
||||
"`<backup.target>/git` or `<backup.target>/<machine>`, and `upload collect` refuses "
|
||||
"a folder that holds or lies in any machine folder, bare repository or export) "
|
||||
"and `settle_seconds` (default "
|
||||
f"{INTAKE_SETTLE_SECONDS}: a file whose modification or creation time is younger is "
|
||||
"left alone); `command` takes `command` (required: the name of an entry of "
|
||||
"`commands`). At most one channel has the handler `mcp`. A field the handler does not "
|
||||
@@ -775,7 +778,9 @@ BACKUP = Shape(
|
||||
Field(
|
||||
"target", "string", "The backup folder: an absolute path outside the instance. "
|
||||
"`backup init` creates `<target>/git` in it, `backup run` writes only to "
|
||||
"`<target>/<machine>/`.",
|
||||
"`<target>/<machine>/`. Everything else in the folder is the sync folder's: an "
|
||||
"`export` target or an `intake` folder may sit beside `git` and the machine folders "
|
||||
"(`<target>/Export`, `<target>/Inbox`), but none may be, or lie in, one of them.",
|
||||
required=True, json_type="string", coerce=exact_str, check=absolute_path_problem,
|
||||
tier=CHECKOUT, gate=True,
|
||||
),
|
||||
@@ -913,7 +918,9 @@ EXPORT = Shape(
|
||||
"the name is the argument of `export run`. Every target has `handler` (`markdown`: the "
|
||||
"pages as Markdown files; `command`: the same Markdown run through a converter of this "
|
||||
"checkout), `path` (an absolute path outside the instance; it must not lie in or "
|
||||
"contain another target, an `intake` folder or `backup.target`) and `include` (an "
|
||||
"contain another target or an `intake` folder, and not be, lie in or contain "
|
||||
"`<backup.target>/git` or `<backup.target>/<machine>`, nor lie in any other "
|
||||
"machine's folder or a bare repository) and `include` (an "
|
||||
"array of selections, each an array of `search --field` predicates: all predicates of "
|
||||
"a selection must hold, and a page that fits any selection is exported; there is no "
|
||||
"\"everything\"). Optional: `frontmatter` (`minimal`: only the export marker; "
|
||||
|
||||
@@ -81,8 +81,8 @@ class Target:
|
||||
|
||||
def _others_paths(root: Path) -> list[tuple[str, Path]]:
|
||||
"""The folders other features of this checkout write into or read from -
|
||||
the `intake` folders and the backup target - for the conflict check. A
|
||||
section that is itself broken is that feature's own problem and is skipped
|
||||
the `intake` folders and the two folders the backup writes into - for the
|
||||
conflict check. A section that is itself broken is that feature's own problem and is skipped
|
||||
here: it is reported where it is read."""
|
||||
from chemenu import upload
|
||||
|
||||
@@ -93,12 +93,7 @@ def _others_paths(root: Path) -> list[tuple[str, Path]]:
|
||||
found.append((f"the intake channel {name!r} ({channel.path})", channel.path))
|
||||
except ValidationError:
|
||||
pass
|
||||
try:
|
||||
backup_cfg = backup.read_config(root)
|
||||
except ValidationError:
|
||||
backup_cfg = None
|
||||
if backup_cfg is not None:
|
||||
found.append((f"backup.target ({backup_cfg.target})", backup_cfg.target))
|
||||
found.extend(backup.written_folders(root))
|
||||
return found
|
||||
|
||||
|
||||
@@ -121,7 +116,7 @@ def read_config(root: "Path | str | None" = None) -> Optional[tuple[Target, ...]
|
||||
that is broken - including what the record cannot judge without the
|
||||
instance and the rest of the settings: a predicate that does not parse, a
|
||||
path on this machine that is not absolute, inside the instance, or in the
|
||||
way of another target, an `intake` folder or the backup target."""
|
||||
way of another target, an `intake` folder or a folder the backup writes into."""
|
||||
root = Path(config.ROOT if root is None else root)
|
||||
key = config_shapes.EXPORT.key
|
||||
section = settings.load(root, only=(key,)).section(key)
|
||||
@@ -406,6 +401,10 @@ def mirror(folder: Path, files: dict[str, bytes], result: TargetResult, *, name:
|
||||
"so nothing was changed")
|
||||
if folder.exists() and not folder.is_dir():
|
||||
raise ExportError(f"{folder} exists and is not a folder")
|
||||
marker = backup.data_marker(folder)
|
||||
if marker:
|
||||
raise ExportError(f"{folder} lies in backup data ({marker}) - an export belongs in a folder of its "
|
||||
"own, beside the backup's. Nothing was written.")
|
||||
manifest = read_manifest(folder) if folder.is_dir() else None
|
||||
if folder.is_dir() and manifest is None and any(folder.iterdir()):
|
||||
raise ExportError(
|
||||
|
||||
@@ -14,6 +14,7 @@ import pytest
|
||||
import typer
|
||||
|
||||
from chemenu import collect, config, upload
|
||||
from chemenu.errors import ValidationError
|
||||
from chemenu.commands.upload_cmd import upload_collect_command, upload_list_command
|
||||
|
||||
OLD = time.time() - 3600
|
||||
@@ -295,3 +296,90 @@ def test_collected_material_shows_in_no_git_status_even_without_a_root_entry(roo
|
||||
collect.collect(root, _command_channel(_script("(out / 'm.md').write_bytes(b'mail')")))
|
||||
assert (root / "quarantine" / ".gitignore").read_text().strip() == "*"
|
||||
assert _git(root, "status", "--porcelain", "--untracked-files=all") == ""
|
||||
|
||||
|
||||
# --- backup and export data -------------------------------------------------
|
||||
|
||||
def _backup_target(root: Path, tmp_path: Path) -> Path:
|
||||
shared = tmp_path / "shared"
|
||||
(shared / "git").mkdir(parents=True)
|
||||
body = {"backup": {"target": str(shared), "machine_data": False, "machine_name": "WORK-LAPTOP"}}
|
||||
(root / config.LOCAL_SETTINGS_FILENAME).write_text(json.dumps({"schema": 1, **body}), encoding="utf-8")
|
||||
return shared
|
||||
|
||||
|
||||
def _channel_at(root: Path, folder: Path) -> None:
|
||||
path = root / config.LOCAL_SETTINGS_FILENAME
|
||||
data = json.loads(path.read_text(encoding="utf-8")) if path.exists() else {"schema": 1}
|
||||
data["intake"] = {"channels": {"inbox": {
|
||||
"handler": "folder", "path": str(folder), "max_bytes": 1024, "allowed_extensions": [".md"]}}}
|
||||
path.write_text(json.dumps(data), encoding="utf-8")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("where", ["git", "git/inner", "WORK-LAPTOP", "WORK-LAPTOP/inner"])
|
||||
def test_an_intake_folder_equal_to_or_in_a_folder_the_backup_writes_is_refused(root, tmp_path, where):
|
||||
shared = _backup_target(root, tmp_path)
|
||||
_channel_at(root, shared / where)
|
||||
with pytest.raises(ValidationError, match="the backup's"):
|
||||
upload.read_channels(root)
|
||||
|
||||
|
||||
def test_an_intake_folder_beside_or_over_the_backup_folders_is_valid(root, tmp_path):
|
||||
shared = _backup_target(root, tmp_path)
|
||||
for folder in (shared / "Inbox", shared, shared.parent):
|
||||
_channel_at(root, folder)
|
||||
assert upload.read_channels(root)["inbox"].path == folder
|
||||
|
||||
|
||||
def test_a_broken_backup_section_does_not_break_the_intake_section(root, tmp_path):
|
||||
_backup_target(root, tmp_path)
|
||||
path = root / config.LOCAL_SETTINGS_FILENAME
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
data["backup"] = {"target": "relative/path"}
|
||||
path.write_text(json.dumps(data), encoding="utf-8")
|
||||
_channel_at(root, tmp_path / "shared" / "git")
|
||||
assert upload.read_channels(root)["inbox"].path == tmp_path / "shared" / "git"
|
||||
|
||||
|
||||
def _backup_manifest(folder: Path) -> None:
|
||||
(folder / ".chemenu-backup.json").write_text("{}", encoding="utf-8")
|
||||
|
||||
|
||||
def _bare_repository(folder: Path) -> None:
|
||||
(folder / "objects").mkdir()
|
||||
(folder / "HEAD").write_text("ref: refs/heads/main\n", encoding="utf-8")
|
||||
|
||||
|
||||
def _export_manifest(folder: Path) -> None:
|
||||
(folder / ".wikitool-export.json").write_text("{}", encoding="utf-8")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("mark, nested", [
|
||||
(_backup_manifest, False), (_backup_manifest, True),
|
||||
(_bare_repository, False), (_bare_repository, True),
|
||||
(_export_manifest, False), (_export_manifest, True),
|
||||
], ids=["machine", "in-machine", "bare", "in-bare", "export", "in-export"])
|
||||
def test_a_pull_from_backup_or_export_data_fails_its_channel_and_leaves_the_folder_as_it_was(
|
||||
root, drop, mark, nested):
|
||||
base = drop / "inner" if nested else drop
|
||||
base.mkdir(exist_ok=True)
|
||||
mark(drop)
|
||||
_put(base, "ENVIRONMENT.md", b"machine data\n")
|
||||
before = _tree(drop)
|
||||
result = collect.collect(root, _folder_channel(base))
|
||||
assert result.error and "lies in" in result.error
|
||||
assert not (result.taken or result.failed or result.skipped)
|
||||
assert _tree(drop) == before
|
||||
assert _manifests(root) == []
|
||||
|
||||
|
||||
def test_the_other_channels_still_run_when_one_lies_in_backup_data(root, drop, tmp_path, capsys):
|
||||
_two_channels(root, drop)
|
||||
_backup_manifest(drop)
|
||||
_put(drop, "a.md", b"alpha\n")
|
||||
before = _tree(drop)
|
||||
with pytest.raises(typer.Exit) as excinfo:
|
||||
upload_collect_command(names=None)
|
||||
assert excinfo.value.exit_code == 1
|
||||
assert _tree(drop) == before
|
||||
assert {m["channel"] for m in _manifests(root)} == {"mail"}
|
||||
@@ -191,21 +191,97 @@ def test_targets_lying_in_one_another_are_refused(instance):
|
||||
assert "lie in one another" in _flat(result.output)
|
||||
|
||||
|
||||
def test_a_target_in_the_backup_target_or_an_intake_folder_is_refused(instance):
|
||||
backup_target = _out(instance, "backup")
|
||||
path = instance / config.LOCAL_SETTINGS_FILENAME
|
||||
_configure(instance, pages={"path": str(backup_target / "pages")})
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
data["backup"] = {"target": str(backup_target)}
|
||||
path.write_text(json.dumps(data), encoding="utf-8")
|
||||
assert _run().exit_code == 1
|
||||
data.pop("backup")
|
||||
def _local(root: Path) -> dict:
|
||||
return json.loads((root / config.LOCAL_SETTINGS_FILENAME).read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
def _save(root: Path, data: dict) -> None:
|
||||
(root / config.LOCAL_SETTINGS_FILENAME).write_text(json.dumps(data), encoding="utf-8")
|
||||
|
||||
|
||||
def _backup_section(root: Path) -> Path:
|
||||
"""A backup whose target is the shared folder itself; returns that folder."""
|
||||
shared = _out(root, "Chemenu")
|
||||
data = _local(root)
|
||||
data["backup"] = {"target": str(shared), "machine_data": False, "machine_name": "WORK-LAPTOP"}
|
||||
_save(root, data)
|
||||
return shared
|
||||
|
||||
|
||||
def _channel(root: Path, path: Path) -> None:
|
||||
data = _local(root)
|
||||
data["intake"] = {"channels": {"inbox": {
|
||||
"handler": "folder", "path": str(_out(instance, "pages") / "in"), "max_bytes": 1024,
|
||||
"allowed_extensions": [".md"]}}}
|
||||
data["export"]["targets"]["pages"]["path"] = str(_out(instance, "pages"))
|
||||
path.write_text(json.dumps(data), encoding="utf-8")
|
||||
"handler": "folder", "path": str(path), "max_bytes": 1024, "allowed_extensions": [".md"]}}}
|
||||
_save(root, data)
|
||||
|
||||
|
||||
def test_export_intake_and_backup_may_share_one_synced_folder(instance):
|
||||
shared = _out(instance, "Chemenu")
|
||||
_configure(instance, pages={"path": str(shared / "Export")})
|
||||
(shared / "Inbox").mkdir(parents=True)
|
||||
_backup_section(instance)
|
||||
_channel(instance, shared / "Inbox")
|
||||
result = _run()
|
||||
assert result.exit_code == 0, result.output
|
||||
assert (shared / "Export" / ".wikitool-export.json").is_file()
|
||||
for check in (doctor.check_export(), doctor.check_upload_intake(), doctor.check_backup()):
|
||||
assert check.status != "FAIL", (check.name, check.detail)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("where", ["git", "git/inner", "WORK-LAPTOP", "WORK-LAPTOP/inner", ""])
|
||||
def test_an_export_equal_to_in_or_over_a_folder_the_backup_writes_is_refused(instance, where):
|
||||
shared = _out(instance, "Chemenu")
|
||||
_configure(instance, pages={"path": str(shared / where) if where else str(shared)})
|
||||
_backup_section(instance)
|
||||
result = _run()
|
||||
assert result.exit_code == 1
|
||||
assert "lie in one another" in _flat(result.output)
|
||||
assert "the backup's" in _flat(result.output)
|
||||
assert not (shared / "git").exists()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("where", ["git", "git/inner", "WORK-LAPTOP", "WORK-LAPTOP/inner"])
|
||||
def test_doctor_fails_an_intake_folder_in_a_folder_the_backup_writes(instance, where):
|
||||
shared = _out(instance, "Chemenu")
|
||||
_configure(instance, pages={"path": str(shared / "Export")})
|
||||
_backup_section(instance)
|
||||
_channel(instance, shared / where)
|
||||
check = doctor.check_upload_intake()
|
||||
assert check.status == "FAIL" and "the backup's" in check.detail
|
||||
|
||||
|
||||
def test_a_target_in_an_intake_folder_is_refused(instance):
|
||||
_configure(instance, pages={"path": str(_out(instance, "pages"))})
|
||||
_channel(instance, _out(instance, "pages") / "in")
|
||||
assert _run().exit_code == 1
|
||||
_channel(instance, _out(instance, "pages").parent)
|
||||
assert _run().exit_code == 1
|
||||
|
||||
|
||||
def test_a_target_below_another_machines_folder_is_refused_before_anything_is_written(instance):
|
||||
shared = _out(instance, "Chemenu")
|
||||
foreign = shared / "OTHER-PC"
|
||||
foreign.mkdir(parents=True)
|
||||
(foreign / ".chemenu-backup.json").write_text("{}", encoding="utf-8")
|
||||
_configure(instance, pages={"path": str(foreign / "deeper" / "Export")})
|
||||
_backup_section(instance)
|
||||
before = {p: p.read_bytes() for p in shared.rglob("*") if p.is_file()}
|
||||
result = _run()
|
||||
assert result.exit_code == 1
|
||||
assert "backup data" in _flat(result.output)
|
||||
assert {p: p.read_bytes() for p in shared.rglob("*") if p.is_file()} == before
|
||||
assert not (foreign / "deeper").exists()
|
||||
|
||||
|
||||
def test_a_target_in_a_bare_repository_is_refused(instance):
|
||||
bare = _out(instance, "repo.git")
|
||||
(bare / "objects").mkdir(parents=True)
|
||||
(bare / "HEAD").write_text("ref: refs/heads/main\n", encoding="utf-8")
|
||||
_configure(instance, pages={"path": str(bare / "Export")})
|
||||
result = _run()
|
||||
assert result.exit_code == 1
|
||||
assert "is a git repository" in _flat(result.output)
|
||||
assert not (bare / "Export").exists()
|
||||
|
||||
|
||||
def test_the_operator_only_fields_cannot_be_set_by_the_agent():
|
||||
|
||||
+12
-3
@@ -102,9 +102,18 @@ def _folder_problem(name: str, raw: str, root: "Path | str") -> Optional[str]:
|
||||
try:
|
||||
resolved.relative_to(real_root)
|
||||
except ValueError:
|
||||
return None
|
||||
return (f"intake.channels.{name}.path {raw!r} lies inside the instance - a folder the channel "
|
||||
"empties belongs outside it")
|
||||
pass
|
||||
else:
|
||||
return (f"intake.channels.{name}.path {raw!r} lies inside the instance - a folder the channel "
|
||||
"empties belongs outside it")
|
||||
# Imported here: the MCP server process imports this module, and only a folder channel needs it.
|
||||
from chemenu import backup
|
||||
|
||||
for label, written in backup.written_folders(root):
|
||||
if backup.is_under(resolved, backup.real_path(written)):
|
||||
return (f"intake.channels.{name}.path {raw!r} is, or lies in, {label} - a folder the channel "
|
||||
"empties belongs beside the backup's folders, not in them")
|
||||
return None
|
||||
|
||||
|
||||
def read_channels(root: "Path | str") -> dict[str, Channel]:
|
||||
|
||||
Reference in new issue
Block a user