feat: Windows portability - path separators, LF line endings, UTF-8 decoding and output, msvcrt lock fallback (#152)
CI / verify (push) Successful in 2m42s
CI / pwsh (push) Successful in 1m55s
Release / release (push) Successful in 36s

Files changed:
- .gitattributes
- CHANGES.md
- VERSION
- raw/CONTRACT.md
- tools/README.md
- tools/chemenu/commands/_util.py
- tools/chemenu/commands/dist_cmd.py
- tools/chemenu/commands/docs_verify.py
- tools/chemenu/commands/doctor.py
- tools/chemenu/commands/eval_cmd.py
- tools/chemenu/commands/git_publish.py
- tools/chemenu/commands/index_build.py
- tools/chemenu/commands/lint.py
- tools/chemenu/commands/log_append.py
- tools/chemenu/commands/migrate_cmd.py
- tools/chemenu/commands/provenance_cmd.py
- tools/chemenu/commands/raw_cmd.py
- tools/chemenu/commands/run_budget.py
- tools/chemenu/commands/upstream_cmd.py
- tools/chemenu/commands/version_cmd.py
- tools/chemenu/commands/work_cmd.py
- tools/chemenu/config.py
- tools/chemenu/corpus_cache.py
- tools/chemenu/filelock.py
- tools/chemenu/frontmatter_io.py
- tools/chemenu/kb_scan.py
- tools/chemenu/kb_state.py
- tools/chemenu/lint_core.py
- tools/chemenu/prerequisites.py
- tools/chemenu/provenance.py
- tools/chemenu/search/base.py
- tools/chemenu/search/ripgrep.py
- tools/chemenu/telemetry/writer.py
- tools/chemenu/tests/test_dist_cmd.py
- tools/chemenu/tests/test_portability.py
- tools/chemenu/tests/test_search.py
- tools/chemenu/tests/test_trace_ingest.py
- tools/chemenu/type_resolver.py
- tools/chemenu/upload.py
- tools/chemenu/version.py
- tools/run_wikitool.py
- tools/trace_ingest.py
This commit is contained in:
torben committed 2026-10-01 21:15:03 +02:00
1 parent 9d06050338
commit d0f08d1fba
42 files changed
+780 -89

No files matched your search

+53 -1
View File
@@ -59,7 +59,7 @@ concern - readable here, never shipped as something to parse.
---
## 8.0.0-beta.18 - 2026-10-01 - trace-hook.ps1: Copilot hooks no longer open Windows' choose-an-app dialog
## 8.0.0-beta.19 - 2026-10-01 - Windows-Portabilität: Pfadtrenner, Zeilenenden, Encoding und Locks
**Author:** Torben Nehmer
@@ -92,6 +92,7 @@ concern - readable here, never shipped as something to parse.
- PowerShell 7 preflight and launcher: tools/preflight.ps1, tools/wikitool.ps1, doctor checks for execution policy and Mark of the Web
- Preflight as a release asset: download, verify and unpack the stack, then run the tree preflight
- trace-hook.ps1: Copilot hooks no longer open Windows' choose-an-app dialog
- Windows-Portabilität: Pfadtrenner, Zeilenenden, Encoding und Locks
**Low impact**
- version bump no longer points at version release in its output
@@ -129,6 +130,57 @@ concern - readable here, never shipped as something to parse.
- preflight.ps1: the asset-mode error helper is Exit-Asset, so PSScriptAnalyzer passes
<!-- /wikitool:bumps -->
### Windows-Portabilität: Pfadtrenner, Zeilenenden, Encoding und Locks
The Python package assumed POSIX in several places that nothing on Linux would ever reveal
(Gitea #152, part of #140). In the run analysed in #140, `instructions verify` failed on all 23
instructions on Windows. This changeset makes the package behave the same on Windows, and holds
it there with guards that run in the ordinary Linux CI.
- **Path separators.** Every `str(<path>.relative_to(...))` is now `.as_posix()`, as are the
error messages that printed a relative path. On Windows these strings came out as
`kb\x.md`. They were then compared with POSIX keys or stored. The type-spec
self-reference check in `type_resolver.py` is one of them, and it failed every validation.
- **ripgrep paths.** `rg --json` writes `\` on Windows, and `--path-separator /` does not
reach its JSON output (measured on the target system, T3). `search/ripgrep.py` converts the
separator where it parses a match, and does so only when `os.sep` is `\`. Nothing is lost:
no Windows path component can contain `\`, and since #155 no page title can either.
- **Line endings.** A new `.gitattributes` (`* text=auto eol=lf`) keeps every text file LF in
a checkout with `core.autocrlf=true`. Without it the sh launcher gets CRLF and Git Bash fails
with `env: 'bash\r'`. `raw/` and `incoming/` are `-text`, so a source is stored byte for
byte as it arrived. `dist export` ships the file. The index was LF throughout already, so
renormalizing changes nothing. Every text write now passes `newline="\n"`, so the byte
comparison of published skill copies and the per-file sha256 in `dist upgrade` agree on
Windows too.
- **Decoding.** Every `subprocess` call with `text=True` names `encoding="utf-8"`. Without it,
Windows decodes `git` and `rg` output in the locale's code page (cp1252).
- **wikitool's own output.** `tools/run_wikitool.py`, the file both launchers run, sets
stdout and stderr to UTF-8. Python on Windows writes into a pipe in cp1252. Measured on
the target system, PowerShell decodes the output of a child process with
`[Console]::OutputEncoding`. Under Copilot that is UTF-8, and Git Bash passes bytes through
unchanged. Before this change `doctor` showed `Fu�noten` under both harnesses. A console is
unaffected either way.
- **Hook payloads.** `trace_ingest.py` reads its stdin as UTF-8 bytes. A locale-decoded read
failed outright on Windows when a payload contained a character such as `Ł`, whose UTF-8
form holds a byte that cp1252 leaves undefined.
- **Locks.** The budget counter and the telemetry writer locked with `fcntl` and silently
skipped the lock where it does not exist. Parallel calls on Windows could then lose a
budget increment. The new module `chemenu/filelock.py` is the only one allowed to import
`fcntl` or `msvcrt`. On Windows it locks one byte far past the file's data with
`msvcrt.locking`, because a Windows lock is mandatory and a lock on the data would block
`trace_ingest.py` from reading a trace. It waits for a contended lock the way `flock` does.
New tests: `tests/test_portability.py` reads the source of `tools/chemenu` and the scripts
beside it. It fails on a stringified `relative_to`, on a text open/read/write without
`encoding=`, on a text write without `newline=`, on `text=True` without `encoding=`, and on an
`fcntl`/`msvcrt` import outside `filelock.py`. Each detector also gets the defect it exists
for, so a guard that matches nothing cannot pass. The `fcntl` fallback is tested with the
import hidden and a fake `msvcrt`. A real Windows lock is not exercised, because CI runs only
Linux. The UTF-8 output and stdin are tested under `PYTHONIOENCODING=cp1252`, which simulates
the Windows pipe. Further tests feed the T3 JSON line through `search/ripgrep.py` with a
simulated Windows separator and check `.gitattributes` with `git check-attr`. Whether `doctor`
shows `Fußnoten` under Copilot and Claude Code is checked by hand on the target machine.
### trace-hook.ps1: Copilot hooks no longer open Windows' choose-an-app dialog
On the Windows target machine, Copilot opened Windows' "choose an app" dialog for `trace-hook`