feat: Windows portability - path separators, LF line endings, UTF-8 decoding and output, msvcrt lock fallback (#152)
Files changed: - .gitattributes - CHANGES.md - VERSION - raw/CONTRACT.md - tools/README.md - tools/chemenu/commands/_util.py - tools/chemenu/commands/dist_cmd.py - tools/chemenu/commands/docs_verify.py - tools/chemenu/commands/doctor.py - tools/chemenu/commands/eval_cmd.py - tools/chemenu/commands/git_publish.py - tools/chemenu/commands/index_build.py - tools/chemenu/commands/lint.py - tools/chemenu/commands/log_append.py - tools/chemenu/commands/migrate_cmd.py - tools/chemenu/commands/provenance_cmd.py - tools/chemenu/commands/raw_cmd.py - tools/chemenu/commands/run_budget.py - tools/chemenu/commands/upstream_cmd.py - tools/chemenu/commands/version_cmd.py - tools/chemenu/commands/work_cmd.py - tools/chemenu/config.py - tools/chemenu/corpus_cache.py - tools/chemenu/filelock.py - tools/chemenu/frontmatter_io.py - tools/chemenu/kb_scan.py - tools/chemenu/kb_state.py - tools/chemenu/lint_core.py - tools/chemenu/prerequisites.py - tools/chemenu/provenance.py - tools/chemenu/search/base.py - tools/chemenu/search/ripgrep.py - tools/chemenu/telemetry/writer.py - tools/chemenu/tests/test_dist_cmd.py - tools/chemenu/tests/test_portability.py - tools/chemenu/tests/test_search.py - tools/chemenu/tests/test_trace_ingest.py - tools/chemenu/type_resolver.py - tools/chemenu/upload.py - tools/chemenu/version.py - tools/run_wikitool.py - tools/trace_ingest.py
This commit is contained in:
1 parent
9d06050338
commit
d0f08d1fba
42 files changed
+780
-89
No files matched your search
+53
-1
@@ -59,7 +59,7 @@ concern - readable here, never shipped as something to parse.
|
||||
|
||||
---
|
||||
|
||||
## 8.0.0-beta.18 - 2026-10-01 - trace-hook.ps1: Copilot hooks no longer open Windows' choose-an-app dialog
|
||||
## 8.0.0-beta.19 - 2026-10-01 - Windows-Portabilität: Pfadtrenner, Zeilenenden, Encoding und Locks
|
||||
|
||||
**Author:** Torben Nehmer
|
||||
|
||||
@@ -92,6 +92,7 @@ concern - readable here, never shipped as something to parse.
|
||||
- PowerShell 7 preflight and launcher: tools/preflight.ps1, tools/wikitool.ps1, doctor checks for execution policy and Mark of the Web
|
||||
- Preflight as a release asset: download, verify and unpack the stack, then run the tree preflight
|
||||
- trace-hook.ps1: Copilot hooks no longer open Windows' choose-an-app dialog
|
||||
- Windows-Portabilität: Pfadtrenner, Zeilenenden, Encoding und Locks
|
||||
|
||||
**Low impact**
|
||||
- version bump no longer points at version release in its output
|
||||
@@ -129,6 +130,57 @@ concern - readable here, never shipped as something to parse.
|
||||
- preflight.ps1: the asset-mode error helper is Exit-Asset, so PSScriptAnalyzer passes
|
||||
<!-- /wikitool:bumps -->
|
||||
|
||||
### Windows-Portabilität: Pfadtrenner, Zeilenenden, Encoding und Locks
|
||||
|
||||
The Python package assumed POSIX in several places that nothing on Linux would ever reveal
|
||||
(Gitea #152, part of #140). In the run analysed in #140, `instructions verify` failed on all 23
|
||||
instructions on Windows. This changeset makes the package behave the same on Windows, and holds
|
||||
it there with guards that run in the ordinary Linux CI.
|
||||
|
||||
- **Path separators.** Every `str(<path>.relative_to(...))` is now `.as_posix()`, as are the
|
||||
error messages that printed a relative path. On Windows these strings came out as
|
||||
`kb\x.md`. They were then compared with POSIX keys or stored. The type-spec
|
||||
self-reference check in `type_resolver.py` is one of them, and it failed every validation.
|
||||
- **ripgrep paths.** `rg --json` writes `\` on Windows, and `--path-separator /` does not
|
||||
reach its JSON output (measured on the target system, T3). `search/ripgrep.py` converts the
|
||||
separator where it parses a match, and does so only when `os.sep` is `\`. Nothing is lost:
|
||||
no Windows path component can contain `\`, and since #155 no page title can either.
|
||||
- **Line endings.** A new `.gitattributes` (`* text=auto eol=lf`) keeps every text file LF in
|
||||
a checkout with `core.autocrlf=true`. Without it the sh launcher gets CRLF and Git Bash fails
|
||||
with `env: 'bash\r'`. `raw/` and `incoming/` are `-text`, so a source is stored byte for
|
||||
byte as it arrived. `dist export` ships the file. The index was LF throughout already, so
|
||||
renormalizing changes nothing. Every text write now passes `newline="\n"`, so the byte
|
||||
comparison of published skill copies and the per-file sha256 in `dist upgrade` agree on
|
||||
Windows too.
|
||||
- **Decoding.** Every `subprocess` call with `text=True` names `encoding="utf-8"`. Without it,
|
||||
Windows decodes `git` and `rg` output in the locale's code page (cp1252).
|
||||
- **wikitool's own output.** `tools/run_wikitool.py`, the file both launchers run, sets
|
||||
stdout and stderr to UTF-8. Python on Windows writes into a pipe in cp1252. Measured on
|
||||
the target system, PowerShell decodes the output of a child process with
|
||||
`[Console]::OutputEncoding`. Under Copilot that is UTF-8, and Git Bash passes bytes through
|
||||
unchanged. Before this change `doctor` showed `Fu�noten` under both harnesses. A console is
|
||||
unaffected either way.
|
||||
- **Hook payloads.** `trace_ingest.py` reads its stdin as UTF-8 bytes. A locale-decoded read
|
||||
failed outright on Windows when a payload contained a character such as `Ł`, whose UTF-8
|
||||
form holds a byte that cp1252 leaves undefined.
|
||||
- **Locks.** The budget counter and the telemetry writer locked with `fcntl` and silently
|
||||
skipped the lock where it does not exist. Parallel calls on Windows could then lose a
|
||||
budget increment. The new module `chemenu/filelock.py` is the only one allowed to import
|
||||
`fcntl` or `msvcrt`. On Windows it locks one byte far past the file's data with
|
||||
`msvcrt.locking`, because a Windows lock is mandatory and a lock on the data would block
|
||||
`trace_ingest.py` from reading a trace. It waits for a contended lock the way `flock` does.
|
||||
|
||||
New tests: `tests/test_portability.py` reads the source of `tools/chemenu` and the scripts
|
||||
beside it. It fails on a stringified `relative_to`, on a text open/read/write without
|
||||
`encoding=`, on a text write without `newline=`, on `text=True` without `encoding=`, and on an
|
||||
`fcntl`/`msvcrt` import outside `filelock.py`. Each detector also gets the defect it exists
|
||||
for, so a guard that matches nothing cannot pass. The `fcntl` fallback is tested with the
|
||||
import hidden and a fake `msvcrt`. A real Windows lock is not exercised, because CI runs only
|
||||
Linux. The UTF-8 output and stdin are tested under `PYTHONIOENCODING=cp1252`, which simulates
|
||||
the Windows pipe. Further tests feed the T3 JSON line through `search/ripgrep.py` with a
|
||||
simulated Windows separator and check `.gitattributes` with `git check-attr`. Whether `doctor`
|
||||
shows `Fußnoten` under Copilot and Claude Code is checked by hand on the target machine.
|
||||
|
||||
### trace-hook.ps1: Copilot hooks no longer open Windows' choose-an-app dialog
|
||||
|
||||
On the Windows target machine, Copilot opened Windows' "choose an app" dialog for `trace-hook`
|
||||
|
||||
Reference in new issue
Block a user