diff --git a/.gitea/workflows/ci.yml b/.gitea/workflows/ci.yml index 022b9db..4de8641 100644 --- a/.gitea/workflows/ci.yml +++ b/.gitea/workflows/ci.yml @@ -138,6 +138,12 @@ jobs: # v3, not v4 - v4 is restricted on this Gitea instance; v3 is what is # proven here (torben/gitea-mcp@ci-build, ci-build.yaml, runs # 42-45). + # + # The artifact is downloadable from the run page, but the Actions + # artifact REST endpoints report `total_count: 0` for it - v3 writes + # through the older artifact API, which those endpoints do not read. + # An empty list is not a failed upload. See EVALS.md ยง "How much of the + # stack the suite reaches"; do not re-derive this. if: always() uses: actions/upload-artifact@v3 with: diff --git a/EVALS.md b/EVALS.md index c01a22d..3927676 100644 --- a/EVALS.md +++ b/EVALS.md @@ -251,27 +251,43 @@ created but not yet written reports broken links. That is the scaffold saying it Coverage is measured in CI and reported, never enforced - `pytest --cov`, config in `tools/.coveragerc`, HTML and XML uploaded as the `coverage-` artifact of every run. +**Fetch that artifact from the run's own page, not from the API**: `upload-artifact@v3` writes +through the older artifact API, and the Actions artifact REST endpoints answer `total_count: 0` +for a run whose artifact the run page offers for download. The upload works; only the listing +does not see it. Do not re-derive this, and do not read the empty list as a failed upload. + There is no `--cov-fail-under`: a threshold is owed (Gitea #10), in its own commit, once the number has been watched long enough to freeze the state it actually reached. -**First measurement, 2026-08-31, stack 1.8.1: 86.9% of 5105 statements across `chemenu/`, -730 tests** - as reported by CI run 87, not by the local run that preceded the last commit of -that release. Reproduce it with `cd tools && .venv/bin/python -m pytest -q --cov` (needs -`pytest-cov`, which is CI-only and deliberately absent from `tools/requirements.txt` - an -instance runs the wiki, it does not measure this suite). +**Measured 2026-09-04, stack 4.7.1: 87.0% of 6498 statements across `chemenu/`, 975 tests** - +CI run 163. The first measurement, at stack 1.8.1 on 2026-08-31, was 86.9% of 5105 statements +over 730 tests (CI run 87). Both are what CI reported, never a local run: the local number +preceding a release measures a tree that is one commit short of the published one. -The total is the least interesting number here. What the report is for is *which* modules sit +The pair says more than either number does. Between them the measured code grew by a quarter +and the suite by a third, and the quota moved by a tenth of a point - which is the observation a +threshold was waiting for, rather than the total itself. Reproduce either with +`cd tools && .venv/bin/python -m pytest -q --cov` (needs `pytest-cov`, which is CI-only and +deliberately absent from `tools/requirements.txt` - an instance runs the wiki, it does not +measure this suite). + +The total stays the least interesting number here. What the report is for is *which* modules sit low, and three kinds have to be told apart before any of it turns into work: - **Thin Typer wrappers**, where the logic lives beside them and is tested there: - `eval_cmd.py` (36%), `types_cmd.py` (52%), `cli.py` (52%). Low coverage on a wrapper is - evidence of a good cut, not of a missing test. + `eval_cmd.py` (36%), `types_cmd.py` (40%), `search.py` (49%), `cli.py` (54%), + `links_cmd.py` (61%). Low coverage on a wrapper is evidence of a good cut, not of a missing + test - `search.py`'s uncovered block is its command body alone, while the backends under + `chemenu/search/` that do the work sit between 91% and 98%. - **Code that reaches the network or the filesystem's outside**, where the interesting half is already injectable and tested through the seam: `version.py`'s `fetch_latest()` takes a `fetcher` parameter for exactly that, and the real network line stays uncovered on purpose. - **Genuine gaps**, where uncovered lines are logic nobody exercises: `provenance_cmd.py` - (44%), `migrate_cmd.py` (71%), `type_resolver.py` (79%). This is the list worth reading, and - the reason step 2 of #10 is not a formality. + (44%), `migrate_cmd.py` (65%), `type_resolver.py` (79%). This is the list worth reading, and + the reason step 2 of #10 is not a formality. It is also the list that has not moved while + everything around it did: `provenance_cmd.py` sits where it sat, and `migrate_cmd.py` fell + from 71% because it grew and its new lines arrived untested. A threshold freezes this; it does + not close it. ## Scoring a session