chemenu/docs/reference/commands/telemetry.md

Telemetry

Reading and scoring the traces that sessions leave behind.

Contents

• eval sessions
• eval score

eval sessions

List the sessions that have a trace under reports/telemetry/.

Synopsis

• wikitool eval sessions [--json]

Options

  --json      Print the list as JSON
  -h, --help  Show this message and exit.

Examples

tools/wikitool eval sessions
tools/wikitool eval sessions --json

Behaviour

• Lists the sessions that have a trace under reports/telemetry/, most recent first.
• Never fails; an empty list is a valid answer.
• Read-only and exempt from the Iteration Budget Gate.

Properties

• Changes files: no
• Idempotent: yes
• Atomic: Read-only
• Counts toward the session call budget: no
• Can reach the network: no

Exit codes

• 0 - success

See also

• wikitool eval score - scores one of them
• docs/explanation/telemetry-and-evaluation.md - how telemetry and evaluation work

eval score

Score one traced session.

Synopsis

• wikitool eval score [--session <id>] [--json] [--markdown out.md] [--save] [--fail-on-error]

Options

  --session <str>    Session to score. Defaults to this shell's session, the
                     same id the budget gate uses.
  --json             Print the scorecard as JSON
  --markdown <path>  Write a markdown scorecard here, conventionally under
                     reports/evals/.
  --save             Write the scorecard to
                     reports/evals/<date>/<session>.{json,md}.
  --fail-on-error    Exit non-zero when the tree has hard errors or an invariant
                     was violated.
  -h, --help         Show this message and exit.

Examples

tools/wikitool eval score
tools/wikitool eval score --session wiki-1727330000 --save

Behaviour

• Scores one traced session: structural state from lint's own checks (L1) plus trajectory rules over the trace (L2) - was a refused call repeated unchanged, was a gate flag passed without that gate having refused anything, did a publish of kb/ pages go unlogged.
• Defaults to the current session; --session <id> picks another.
• --save writes reports/evals/<date>/<session>.{json,md}; --markdown writes the report to the named file.
• A session records nothing when telemetry is off - WIKI_TRACE=0, or a distributed instance with no opt-in (wikitool doctor says which) - so an absent trace is not necessarily a fault.
• Read-only over kb/, safe to retry, and exempt from the Iteration Budget Gate.

Properties

• Changes files: no
• Idempotent: yes
• Atomic: Read-only, apart from the files --save/--markdown write
• Counts toward the session call budget: no
• Can reach the network: no

Exit codes

• 0 - success
• 1 - No trace exists for the named session
• 1 - Only with --fail-on-error: the tree has hard errors or an invariant was violated

See also

• wikitool eval sessions - which session ids exist
• docs/explanation/telemetry-and-evaluation.md - the scoring levels