3

Reference Commands Telemetry

Wiki Mirror edited this page 2026-10-16 14:43:37 +00:00

Table of Contents

• Telemetry
• Contents
• eval sessions
• eval score

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Generated from docs/reference/commands/telemetry.md at v10.1.0 – edit the repository, not this wiki.

Telemetry

Reading and scoring the traces that sessions leave behind.

Contents

• eval sessions
• eval score

eval sessions

List the sessions that have a trace under reports/telemetry/.

Synopsis

• wikitool eval sessions [--json]

Options

  --json      Print the list as JSON
  -h, --help  Show this message and exit.

Examples

tools/wikitool eval sessions
tools/wikitool eval sessions --json

Behaviour

• Lists the sessions that have a trace under reports/telemetry/, most recent first.
• Never fails; an empty list is a valid answer.
• Read-only and exempt from the Iteration Budget Gate.

Properties

• Changes files: no
• Idempotent: yes
• Atomic: Read-only
• Counts toward the session call budget: no
• Can reach the network: no

Exit codes

• 0 - success

See also

• wikitool eval score - scores one of them
• docs/explanation/telemetry-and-evaluation.md - how telemetry and evaluation work

eval score

Score one traced session.

Synopsis

• wikitool eval score [--session <id>] [--json] [--markdown out.md] [--save] [--fail-on-error]

Options

  --session <str>    Session to score. Defaults to this shell's session, the
                     same id the budget gate uses.
  --json             Print the scorecard as JSON
  --markdown <path>  Write a markdown scorecard here, conventionally under
                     reports/evals/.
  --save             Write the scorecard to
                     reports/evals/<date>/<session>.{json,md}.
  --fail-on-error    Exit non-zero when the tree has hard errors or an invariant
                     was violated.
  -h, --help         Show this message and exit.

Examples

tools/wikitool eval score
tools/wikitool eval score --session wiki-1727330000 --save

Behaviour

• Scores one traced session: structural state from lint's own checks (L1) plus trajectory rules over the trace (L2) - was a refused call repeated unchanged, was a gate flag passed without that gate having refused anything, did a publish of kb/ pages go unlogged.
• Defaults to the current session; --session <id> picks another.
• --save writes reports/evals/<date>/<session>.{json,md}; --markdown writes the report to the named file.
• A session records nothing when telemetry is off - WIKI_TRACE=0, or a distributed instance with no opt-in (wikitool doctor says which) - so an absent trace is not necessarily a fault.
• Read-only over kb/, safe to retry, and exempt from the Iteration Budget Gate.

Properties

• Changes files: no
• Idempotent: yes
• Atomic: Read-only, apart from the files --save/--markdown write
• Counts toward the session call budget: no
• Can reach the network: no

Exit codes

• 0 - success
• 1 - No trace exists for the named session
• 1 - Only with --fail-on-error: the tree has hard errors or an invariant was violated

See also

• wikitool eval sessions - which session ids exist
• docs/explanation/telemetry-and-evaluation.md - the scoring levels

• Chemenu - Personal IT Knowledge Base
• Install a Chemenu instance
• Tutorials
  • From your first source to your first answer
  • Home lab backups - how they work now
• How-to guides
  • Back up to a synced folder
  • Browse the wiki in Obsidian
  • Capture documentation from a git repository
  • Collect documents from a synced folder
  • Export guidelines to captured repositories
  • Export to a synced folder
  • Fetch pages behind a login
  • Ingest a source
  • Maintain the wiki
  • Query the wiki
  • Run the weekly review
  • Set up the MCP read server
  • Trace and score agent sessions
  • Troubleshoot an instance
  • Update an instance
  • Use 1Password for settings
  • Write a site handler
• Reference
  • Command reference
    • Catalog and log
    • Configuration
    • Content migrations
    • Distribution and versioning
    • Exports
    • Finding and checking
    • Git
    • Instance health
    • Links and citations
    • Pages
    • Provenance
    • Raw material and uploads
    • Telemetry
    • Types, instructions and docs
    • Workshop runs and session budget
  • Configuration
  • Exit codes and gates
  • Page types
  • Skill catalogue
• Explanation
  • Architecture
  • Configuration tiers
  • Why knowledge and commitments are two layers
  • Language Boundaries
  • Choosing a Claude Code model and effort level
  • Ownership and Templates
  • Why the pipeline has four stages
  • Telemetry and evaluation
  • Why the stack version splits compatibility from migration
  • Why gates are code
• Dev
  • Developing this stack