fix: Testisolation - kb_dir repointet config.ROOT, lint loest Kollektionen gegen den uebergebenen Baum auf (4.1.1, #44)
CI / verify (push) Successful in 55s
Release / release (push) Successful in 38s

Files changed:
- CHANGES.md
- VERSION
- instructions/dev/testing-conventions.md
- tools/chemenu/lint_core.py
- tools/chemenu/tests/conftest.py
- tools/chemenu/tests/test_hermetic_env.py
- tools/chemenu/tests/test_lint.py
This commit is contained in:
2026-09-03 06:33:38 +02:00
parent cfe925a76c
commit 23307c3c5f
7 changed files with 263 additions and 33 deletions
+49 -2
View File
@@ -40,6 +40,48 @@ resolved paths and `conventions`' parsed `kb/CONVENTIONS.md`. A test that *rewri
conventions file mid-test calls `conventions.reset_cache()` itself - the fixture answers for the
boundary between tests, not for one inside a test.
## Which tree a test writes into
The environment is one half of the isolation; `config.ROOT` is the other. With `CHEMENU_ROOT`
cleared, `ROOT` falls back to the checkout pytest is running from - deliberately, because most
tests want the shipped `types/`. It also means that any code path resolving a file through
`config.ROOT` or `config.KB_DIR` reaches **the real repository**, no matter which tree the
fixture built.
Both corpus fixtures therefore repoint it: `raw_dir` and `kb_dir` each set
`config.ROOT` to their `tmp_path` and re-declare the shipped `types/` through
`use_shipped_type_specs()`. `config`'s module `__getattr__` resolves the derived paths on
access, so repointing `ROOT` carries `KB_DIR`, `RAW_DIR` and the rest with it. A new fixture
that builds a tree does the same thing - that is the rule here, not a per-test judgment.
`kb_dir` did not, until Gitea #44. Two things came of that. A test calling
`kb_state.write_kb_state()` overwrote the real `.wikitool-kb.json`, which `git status` made
visible within the minute. Quieter and worse: `lint`'s collection lookup resolved a page
against `config.KB_DIR`, so every fixture page read back as "no collection" and the
`unauthorised_labels` check skipped every edge in silence - the finding had no working test at
all, and its green run read like an assurance.
Two guards came out of it, both in `conftest.py`:
| Guard | Default | Cost |
|---|---|---|
| `repository_tree_guard` (session) | on | two `git status --porcelain` calls per run |
| `per_test_tree_guard` | off, `CHEMENU_TREE_GUARD=each` turns it on | one `git status` per test |
The session guard compares the working tree before against after and fails the run if anything
moved, so it says nothing about uncommitted work a developer already had. It cannot name the
test that did it; `CHEMENU_TREE_GUARD=each` can, and is the way to bisect once it fires. Where
git is unavailable or the checkout is not a repository, both are silently inert.
Neither guard sees the second, quieter half: a check that silently *does nothing* under test
writes no file. That one is only caught by a test that asserts the finding actually fires -
which is why `test_unauthorised_label_is_judged_in_a_tree_that_is_not_the_configured_kb`
lints a tree `ROOT` deliberately points away from.
**A function that takes a directory resolves against that directory.** `run_lint(kb_dir)`
reading `config.KB_DIR` for one of its own lookups was the defect behind the quiet half, and
no fixture can fix that shape from the outside.
## When to run
Whenever you add or change a test under `tools/chemenu/tests/`.
@@ -80,7 +122,12 @@ Whenever you add or change a test under `tools/chemenu/tests/`.
`conftest.py` in the same change. A variable the tool reads and the fixture does not clear
is the exact hole this whole file is about, reopened.
5. **Verify against an empty machine before publishing**, not only in your own shell:
5. **Writing a fixture that builds a tree?** Repoint `config.ROOT` at it and call
`use_shipped_type_specs(monkeypatch)`, as `raw_dir` and `kb_dir` do - see
[Which tree a test writes into](#which-tree-a-test-writes-into). A fixture that returns a
path without repointing hands the code under test the real repository.
6. **Verify against an empty machine before publishing**, not only in your own shell:
```bash
cd tools && env -i PATH="$PATH" HOME="$(mktemp -d)" \
@@ -92,7 +139,7 @@ Whenever you add or change a test under `tools/chemenu/tests/`.
`.venv/bin/python -m pytest -q`. A difference between the two is a leak, and the leaking
variable belongs in step 4's list.
6. **Check the coverage report when adding tests to close a gap**, rather than guessing which
7. **Check the coverage report when adding tests to close a gap**, rather than guessing which
lines were uncovered:
```bash