fix: Testisolation - kb_dir repointet config.ROOT, lint loest Kollektionen gegen den uebergebenen Baum auf (4.1.1, #44)
Files changed: - CHANGES.md - VERSION - instructions/dev/testing-conventions.md - tools/chemenu/lint_core.py - tools/chemenu/tests/conftest.py - tools/chemenu/tests/test_hermetic_env.py - tools/chemenu/tests/test_lint.py
This commit is contained in:
@@ -40,6 +40,48 @@ resolved paths and `conventions`' parsed `kb/CONVENTIONS.md`. A test that *rewri
|
||||
conventions file mid-test calls `conventions.reset_cache()` itself - the fixture answers for the
|
||||
boundary between tests, not for one inside a test.
|
||||
|
||||
## Which tree a test writes into
|
||||
|
||||
The environment is one half of the isolation; `config.ROOT` is the other. With `CHEMENU_ROOT`
|
||||
cleared, `ROOT` falls back to the checkout pytest is running from - deliberately, because most
|
||||
tests want the shipped `types/`. It also means that any code path resolving a file through
|
||||
`config.ROOT` or `config.KB_DIR` reaches **the real repository**, no matter which tree the
|
||||
fixture built.
|
||||
|
||||
Both corpus fixtures therefore repoint it: `raw_dir` and `kb_dir` each set
|
||||
`config.ROOT` to their `tmp_path` and re-declare the shipped `types/` through
|
||||
`use_shipped_type_specs()`. `config`'s module `__getattr__` resolves the derived paths on
|
||||
access, so repointing `ROOT` carries `KB_DIR`, `RAW_DIR` and the rest with it. A new fixture
|
||||
that builds a tree does the same thing - that is the rule here, not a per-test judgment.
|
||||
|
||||
`kb_dir` did not, until Gitea #44. Two things came of that. A test calling
|
||||
`kb_state.write_kb_state()` overwrote the real `.wikitool-kb.json`, which `git status` made
|
||||
visible within the minute. Quieter and worse: `lint`'s collection lookup resolved a page
|
||||
against `config.KB_DIR`, so every fixture page read back as "no collection" and the
|
||||
`unauthorised_labels` check skipped every edge in silence - the finding had no working test at
|
||||
all, and its green run read like an assurance.
|
||||
|
||||
Two guards came out of it, both in `conftest.py`:
|
||||
|
||||
| Guard | Default | Cost |
|
||||
|---|---|---|
|
||||
| `repository_tree_guard` (session) | on | two `git status --porcelain` calls per run |
|
||||
| `per_test_tree_guard` | off, `CHEMENU_TREE_GUARD=each` turns it on | one `git status` per test |
|
||||
|
||||
The session guard compares the working tree before against after and fails the run if anything
|
||||
moved, so it says nothing about uncommitted work a developer already had. It cannot name the
|
||||
test that did it; `CHEMENU_TREE_GUARD=each` can, and is the way to bisect once it fires. Where
|
||||
git is unavailable or the checkout is not a repository, both are silently inert.
|
||||
|
||||
Neither guard sees the second, quieter half: a check that silently *does nothing* under test
|
||||
writes no file. That one is only caught by a test that asserts the finding actually fires -
|
||||
which is why `test_unauthorised_label_is_judged_in_a_tree_that_is_not_the_configured_kb`
|
||||
lints a tree `ROOT` deliberately points away from.
|
||||
|
||||
**A function that takes a directory resolves against that directory.** `run_lint(kb_dir)`
|
||||
reading `config.KB_DIR` for one of its own lookups was the defect behind the quiet half, and
|
||||
no fixture can fix that shape from the outside.
|
||||
|
||||
## When to run
|
||||
|
||||
Whenever you add or change a test under `tools/chemenu/tests/`.
|
||||
@@ -80,7 +122,12 @@ Whenever you add or change a test under `tools/chemenu/tests/`.
|
||||
`conftest.py` in the same change. A variable the tool reads and the fixture does not clear
|
||||
is the exact hole this whole file is about, reopened.
|
||||
|
||||
5. **Verify against an empty machine before publishing**, not only in your own shell:
|
||||
5. **Writing a fixture that builds a tree?** Repoint `config.ROOT` at it and call
|
||||
`use_shipped_type_specs(monkeypatch)`, as `raw_dir` and `kb_dir` do - see
|
||||
[Which tree a test writes into](#which-tree-a-test-writes-into). A fixture that returns a
|
||||
path without repointing hands the code under test the real repository.
|
||||
|
||||
6. **Verify against an empty machine before publishing**, not only in your own shell:
|
||||
|
||||
```bash
|
||||
cd tools && env -i PATH="$PATH" HOME="$(mktemp -d)" \
|
||||
@@ -92,7 +139,7 @@ Whenever you add or change a test under `tools/chemenu/tests/`.
|
||||
`.venv/bin/python -m pytest -q`. A difference between the two is a leak, and the leaking
|
||||
variable belongs in step 4's list.
|
||||
|
||||
6. **Check the coverage report when adding tests to close a gap**, rather than guessing which
|
||||
7. **Check the coverage report when adding tests to close a gap**, rather than guessing which
|
||||
lines were uncovered:
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user