torbenandClaude Opus 5.5
53383e7e9b
Files changed: - CHANGES.md - README.md - VERSION - docs/reference/README.md - docs/reference/commands/README.md - docs/reference/commands/catalog-and-log.md - docs/reference/commands/content-migrations.md - docs/reference/commands/distribution-and-versioning.md - docs/reference/commands/finding-and-checking.md - docs/reference/commands/git.md - docs/reference/commands/guideline-export.md - docs/reference/commands/instance-health.md - docs/reference/commands/links-and-citations.md - docs/reference/commands/pages.md - docs/reference/commands/provenance.md - docs/reference/commands/raw-material-and-uploads.md - docs/reference/commands/telemetry.md - docs/reference/commands/types-instructions-and-docs.md - docs/reference/commands/workshop-runs-and-session-budget.md - docs/reference/exit-codes.md - instructions/dev/doc-pull-through.md - tools/CONTRACT.md - tools/README.md - tools/chemenu/cli.py - tools/chemenu/cli_contract.py - tools/chemenu/commands/docs_verify.py - tools/chemenu/reference_doc.py - tools/chemenu/tests/test_reference_doc.py - tools/chemenu/toc.py Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
3.1 KiB
3.1 KiB
Telemetry
Reading and scoring the traces that sessions leave behind.
Contents
eval sessions
List the sessions that have a trace under reports/telemetry/.
Synopsis
wikitool eval sessions [--json]
Options
--json Print the list as JSON
-h, --help Show this message and exit.
Examples
tools/wikitool eval sessions
tools/wikitool eval sessions --json
Behaviour
- Lists the sessions that have a trace under
reports/telemetry/, most recent first. - Never fails; an empty list is a valid answer.
- Read-only and exempt from the Iteration Budget Gate.
Properties
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
Exit codes
0- success
See also
wikitool eval score- scores one of themEVALS.md- how telemetry and evaluation work
eval score
Score one traced session.
Synopsis
wikitool eval score [--session <id>] [--json] [--markdown out.md] [--save] [--fail-on-error]
Options
--session <str> Session to score. Defaults to this shell's session, the
same id the budget gate uses.
--json Print the scorecard as JSON
--markdown <path> Write a markdown scorecard here, conventionally under
reports/evals/.
--save Write the scorecard to
reports/evals/<date>/<session>.{json,md}.
--fail-on-error Exit non-zero when the tree has hard errors or an invariant
was violated.
-h, --help Show this message and exit.
Examples
tools/wikitool eval score
tools/wikitool eval score --session wiki-1727330000 --save
Behaviour
- Scores one traced session: structural state from
lint's own checks (L1) plus trajectory rules over the trace (L2) - was a refused call repeated unchanged, was a gate flag passed without that gate having refused anything, did a publish ofkb/pages go unlogged. - Defaults to the current session;
--session <id>picks another. --savewritesreports/evals/<date>/<session>.{json,md};--markdownwrites the report to the named file.- A session records nothing when telemetry is off -
WIKI_TRACE=0, or a distributed instance with no opt-in (wikitool doctorsays which) - so an absent trace is not necessarily a fault. - Read-only over
kb/, safe to retry, and exempt from the Iteration Budget Gate.
Properties
- Changes files: no
- Idempotent: yes
- Atomic: Read-only, apart from the files
--save/--markdownwrite - Counts toward the session call budget: no
- Can reach the network: no
Exit codes
0- success1- No trace exists for the named session1- Only with--fail-on-error: the tree has hard errors or an invariant was violated
See also
wikitool eval sessions- which session ids existEVALS.md- the scoring levels