docs: lock remote delegation contract and WS8 tracking

AGENTS.md

@@ -42,7 +42,8 @@ Only modify these paths when the task explicitly requires it.
 
 - Treat OTA/recovery reliability as a product-level contract.
 - Treat partition layout as a fixed contract unless explicitly requested otherwise.
-- Do not introduce platform-specific compile-time branches when runtime/config solutions are sufficient.
+- Do not introduce platform-specific compile-time branches when
+  runtime/config solutions are sufficient.
 - Avoid unrelated churn; keep patches tightly scoped to the request.
 
 ## Standard Workflow
@@ -52,13 +53,45 @@ Only modify these paths when the task explicitly requires it.
 3. Implement the smallest coherent patch that preserves contracts.
 4. Run targeted validation commands.
 5. Report changes, validation evidence, and residual risks.
+6. Commit hygiene: make small commits and push frequently when progress
+   needs to be traceable by commit SHA across sessions/roles.
+
+Commit hygiene scope:
+- each role Codex runtime is responsible for commit hygiene in its own
+  repository (product repo for `arch`, infra repo for `infra`, runner
+  repo for `runner`)
+
+## Ownership Contract (Arch Closure Rule)
+
+When `arch:codex` is explicitly asked to "take ownership" of a task or
+workstream, `arch:codex` is accountable for driving it to closure:
+
+- closure means either:
+  - `done` with acceptance met and evidence recorded, or
+  - `blocked` with concrete blockers, owners, operator gates, and next
+    actions recorded (not vague "waiting")
+- `arch:codex` may delegate execution to `infra:codex` / `runner:codex`,
+  but must actively steer until closure (handoffs updated, blockers
+  cleared or escalated, and the workstream status stabilized)
+- do not stop at "prepared a packet"; prepared packets are inputs to
+  closure, not closure themselves
+
+Exception:
+- `arch:codex` may stop early when a pivot is required or a problem
+  solving loop is stalled, but only after recording a stable closure
+  state:
+  - `action_type=replan` with the pivot proposal and updated next
+    actions, or
+  - `blocked` with the specific stalled hypothesis, what was tried, and
+    what new information is needed to proceed
 
 ## Validation Commands
 
 - Style check: `build-scripts/lint_style.sh check`
 - Style normalize: `build-scripts/lint_style.sh format`
 - Firmware build (example): `idf.py build`
-- Recovery footprint (platform-specific): `build-scripts/build_recovery_size.sh <platform|defaults-file> [build-dir]`
+- Recovery footprint (platform-specific):
+  `build-scripts/build_recovery_size.sh <platform|defaults-file> [build-dir]`
 
 Use targeted checks first; avoid full-matrix builds unless requested.
 
@@ -66,29 +99,86 @@ Use targeted checks first; avoid full-matrix builds unless requested.
 
 - Tests should validate behavior contracts and invariants, not implementation details.
 - Prefer module-boundary and regression tests over private-internal assertions.
-- Every bug fix should include a regression test at the highest stable boundary possible.
+- Every bug fix should include a regression test at the highest stable
+  boundary possible.
 - Use:
   - `documentation/TESTING_CHARTER.md`
   - `documentation/CONTRACT_TEST_TEMPLATE.md`
   - `documentation/HARDWARE_TEST_MATRIX.md`
+  - `documentation/agents/ci_lane_contract.md`
 
 ## Style Baseline
 
 - Use repository formatting/lint settings as the source of truth.
 - Do not introduce style-only churn outside task scope unless requested.
 - During refactors, normalization is acceptable when intentionally planned.
+- For doc-only cleanup tasks, follow
+  `documentation/agents/document_gardening.md`.
 
 ## Frontend Context (Required)
 
-- For any UI/API work involving `components/wifi-manager/webapp/` or `components/wifi-manager/http_server_handlers.c`, read:
+- For any UI/API work involving `components/wifi-manager/webapp/` or
+  `components/wifi-manager/http_server_handlers.c`, read:
   - `documentation/agents/frontend_requirements_context.md`
 - Treat that file as requirements-first context and keep it current.
 - When frontend payload, routes, or protobuf contract changes, run:
   - `build-scripts/ui_footprint_snapshot.sh`
-- Then update `documentation/agents/frontend_requirements_context.md` in the same change.
+- Then update `documentation/agents/frontend_requirements_context.md` in
+  the same change.
 
 ## Agent Docs Layout
 
 - Keep this file concise and policy-focused.
 - Put detailed playbooks under `documentation/agents/`.
-- Add module-specific guidance in local `AGENTS.md` files when needed (closest file to changed code takes precedence).
+- For progressive context discovery, start with:
+  - `documentation/agents/start_here.md`
+- For remote-role delegation (`infra`/`runner`), follow:
+  - `documentation/agents/remote_delegation_contract.md` (autonomous
+    delegation + handshake + `thread_ref` correlation)
+- Add module-specific guidance in local `AGENTS.md` files when needed
+  (closest file to changed code takes precedence).
+
+## Short-Term Goal Tracking (Ephemeral)
+
+- Active multi-agent implementation goals live under:
+  - `documentation/short-term/active/`
+- Coordination files live under:
+  - `documentation/short-term/coordination/workstream_board.md`
+  - `documentation/short-term/coordination/handoff_log.md`
+- When prompted to continue pending multi-agent work, read those files
+  first and update them before ending the session.
+- Use role-based owner values in short-term boards:
+  - `arch:<agent-id>` for architecture/control-plane work
+  - `infra:<agent-id>` for infrastructure/host/VM work
+  - `runner:<agent-id>` for VM CI/CD execution work
+- Handoff entries for short-term goals should include:
+  - `context` (`arch-local` | `infra-live` | `runner-live`)
+  - `action_type` (`scaffold` | `delegate` | `execute` | `replan`)
+  - `operator_required` (`yes` | `no`)
+- Role-origin evidence naming should use prefixes where feasible:
+  - `arch_*`, `infra_*`, `runner_*`
+- User/operator reprioritization or path-shift requests (e.g. \"we
+  missed...\", \"easier way...\", \"we should prioritize...\") should
+  trigger a formal short-term replan update before continuing execution.
+- For ad-hoc user requests, proactively ask whether to track as a
+  ticket (`yes`/`no`) and, when accepted, record it in
+  `documentation/short-term/coordination/ad_hoc_ticket_queue.md`.
+- For local GitLab usage (`git.lecsys.net`), keep role secrets only in
+  `${XDG_CONFIG_HOME:-$HOME/.config}/codex/credentials/gitlab/git.lecsys.net/`
+  with directory mode `0700` and file mode `0600`; never commit
+  credential files or paste secret values in handoff logs.
+- Preserve CI lane boundary:
+  - upstream GitHub lane for build/non-hardware tests
+  - local LXD lane for hardware-in-loop execution
+- Repository boundary enforcement (strict):
+  - never edit or run role execution from `arch` against runner/infra
+    repositories on this machine (for example
+    `/workspaces/codex-runner-agent`, `/workspaces/codex-infra-agent`)
+  - all `runner` and `infra` workstreams must be executed by their
+    remote Codex runtimes and reported back via handoff logs
+- Completion gate:
+  - Workstream `done` means that workstream acceptance is met.
+  - Goal completion is separate and requires every item in the goal
+    deliverables checklist to be checked with evidence.
+  - Do not mark a goal complete or archive it while any deliverable
+    checklist item remains unchecked.

build-scripts/lxd_remote.sh

@@ -0,0 +1,64 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ENV_FILE="${LXD_ENV_FILE:-.lxd.env}"
+if [[ -f "$ENV_FILE" ]]; then
+  # shellcheck disable=SC1090
+  source "$ENV_FILE"
+fi
+
+: "${LXD_HOST:?Missing LXD_HOST (set in .lxd.env or environment)}"
+: "${LXD_SSH_USER:?Missing LXD_SSH_USER (set in .lxd.env or environment)}"
+
+LXD_SSH_PORT="${LXD_SSH_PORT:-22}"
+LXD_SSH_AUTH="${LXD_SSH_AUTH:-key}"
+LXD_SSH_PRIVATE_KEY_PATH="${LXD_SSH_PRIVATE_KEY_PATH:-}"
+LXD_WORKDIR="${LXD_WORKDIR:-}"
+
+expand_home() {
+  local p="$1"
+  if [[ "$p" == ~* ]]; then
+    printf '%s\n' "${HOME}${p:1}"
+  else
+    printf '%s\n' "$p"
+  fi
+}
+
+ssh_opts=(
+  -p "$LXD_SSH_PORT"
+  -o BatchMode=yes
+  -o ConnectTimeout=10
+  -o StrictHostKeyChecking=accept-new
+)
+
+if [[ "$LXD_SSH_AUTH" == "key" ]]; then
+  : "${LXD_SSH_PRIVATE_KEY_PATH:?Missing LXD_SSH_PRIVATE_KEY_PATH for key auth}"
+  key_path="$(expand_home "$LXD_SSH_PRIVATE_KEY_PATH")"
+  ssh_opts+=( -i "$key_path" )
+fi
+
+if [[ "${1:-}" == "heartbeat" ]]; then
+  shift
+  cmd='hostname && whoami && uptime && df -h /'
+else
+  if [[ "$#" -eq 0 ]]; then
+    cat <<'USAGE' >&2
+Usage:
+  build-scripts/lxd_remote.sh heartbeat
+  build-scripts/lxd_remote.sh <remote command...>
+
+Environment source: .lxd.env by default (override with LXD_ENV_FILE)
+USAGE
+    exit 2
+  fi
+
+  cmd="$*"
+fi
+
+if [[ -n "$LXD_WORKDIR" ]]; then
+  remote_cmd="cd $LXD_WORKDIR && ( $cmd )"
+else
+  remote_cmd="$cmd"
+fi
+
+exec ssh "${ssh_opts[@]}" "${LXD_SSH_USER}@${LXD_HOST}" "$remote_cmd"

documentation/agents/README.md

@@ -1,20 +1,44 @@
 # Agent Documentation
 
-Store long-form guidance for coding agents here.
+Use this folder for operational docs that agents can discover progressively.
 
-## Suggested Files
+## Start Path (minimal context first)
 
-- `documentation/agents/architecture.md`: subsystem boundaries, data flow, invariants.
-- `documentation/agents/build-and-test.md`: canonical commands, fast checks, CI mapping.
-- `documentation/agents/style-and-lint.md`: formatting policy, lint severity, suppression rules.
-- `documentation/agents/refactor-playbook.md`: safe refactor steps, rollout strategy, risk controls.
-- `documentation/agents/frontend_requirements_context.md`: requirements-first UI context, size budgets, and migration constraints.
-- `documentation/agents/module-notes/<module>.md`: module-level constraints and edge cases.
+1. Read `documentation/agents/start_here.md`.
+2. Load exactly one route document based on task type.
+3. Load additional documents only when the active route requires them.
+
+## Route Documents
+
+- `documentation/agents/ci_lane_contract.md`
+  - lane boundary and SHA handoff between upstream and local HIL
+- `documentation/agents/integration_test_worklist.md`
+  - integration/hardware execution tracker and handoff protocol
+- `documentation/agents/document_gardening.md`
+  - docs lint and docs maintenance workflow for CI/LXD sessions
+- `documentation/agents/frontend_requirements_context.md`
+  - requirements-first UI/API context and update contract
+
+## Short-Term Execution Docs
+
+- `documentation/short-term/active/*.md`
+  - active, time-boxed goals
+- `documentation/short-term/active/GOAL_TEMPLATE.md`
+  - standardized template for creating new goals
+- `documentation/short-term/coordination/workstream_board.md`
+  - goal/workstream owner and status board (role-based owner governance)
+- `documentation/short-term/coordination/handoff_log.md`
+  - session handoff trail
+- `documentation/short-term/coordination/ad_hoc_ticket_queue.md`
+  - user-approved ad-hoc ticket tracking queue
 
 ## Writing Rules
 
 - Keep files operational and concrete.
 - Prefer checklists and explicit commands.
 - Record known pitfalls and non-obvious invariants.
-- Link back from `AGENTS.md` only to docs that are actively maintained.
-- When frontend payload/routes/contracts change, refresh the snapshot with `build-scripts/ui_footprint_snapshot.sh` and update `frontend_requirements_context.md`.
+- Keep documents pointer-heavy so agents can discover context on demand.
+- Link from `AGENTS.md` only to maintained docs.
+- Archive completed short-term goals out of active rotation.
+- Do not mark a goal complete or archive it while its deliverables
+  checklist has unchecked items.

documentation/agents/ci_lane_contract.md

@@ -0,0 +1,83 @@
+# CI Lane Contract
+
+Define and preserve the split between upstream CI and local hardware CI.
+
+## Purpose
+
+Maintain seamless developer flow while acknowledging hardware constraints:
+
+- GitHub CI cannot access physical hardware
+- Local LXD/HUT CI can run physical integration tests
+
+## Lane Definitions
+
+## 1) Upstream GitHub Lane (hardware-agnostic)
+
+Allowed:
+
+- compile/build
+- packaging
+- lint/static checks
+- unit tests and non-hardware integration tests
+
+Not allowed:
+
+- direct serial/USB assumptions
+- relay/home-automation dependencies
+- any test requiring physical board availability
+
+## 2) Local LXD Hardware Lane (HIL)
+
+Allowed:
+
+- flash/monitor cycles
+- physical integration and regression tests
+- hard off/on recovery with relay control
+- multi-HUT scheduling and isolation
+
+## Seamless Handoff Rules
+
+1. Hardware runs should map to the same commit SHA validated upstream.
+2. Artifact format and naming must be lane-neutral and SHA-addressable.
+3. Hardware result summaries must include:
+   - commit SHA
+   - HUT slot
+   - pass/fail
+   - recovery/power-cycle events
+4. Upstream checks must not block on hardware-runner availability.
+
+## Operational Guidance
+
+1. Treat upstream lane as fast feedback gate.
+2. Treat local HIL lane as higher-fidelity system validation.
+3. Keep trigger policy explicit:
+   - automatic for upstream lane
+   - explicit/controlled for local HIL lane
+4. Minimize drift between lanes by reusing the same
+   container/toolchain baseline where feasible.
+
+## Documentation Lint Contract
+
+1. Upstream GitHub lane must run markdown lint checks for:
+
+   - `AGENTS.md`
+   - `documentation/**/*.md`
+2. Local LXD lane should run agent-driven document gardening for:
+
+   - lint fixes
+   - pointer refresh
+   - short-term board/handoff updates
+3. Upstream lint failures block merge for documentation changes.
+4. Local gardening output is evidence, not a replacement for upstream checks.
+
+## Lane Vs Role Boundary
+
+CI lanes and agent roles are different dimensions:
+
+1. Lanes define where validation runs (`upstream` vs `local-hil`).
+2. Roles define who executes/coordinates (`arch`, `infra`, `runner`).
+3. Do not infer ownership from lane alone; use
+   `documentation/short-term/coordination/workstream_board.md` owner
+   field as source of truth.
+4. Use handoff `context` to distinguish control plane vs live remote
+   execution (`arch-local`, `infra-live`, `runner-live`).

documentation/agents/document_gardening.md

@@ -0,0 +1,80 @@
+# Document Gardening Playbook
+
+Use document gardening to keep agent-facing and implementation docs current,
+lint-clean, and actionable during active development.
+
+## What It Includes
+
+- fix markdown lint/style issues
+- refresh stale links and pointers
+- sync status boards and handoff logs
+- archive completed short-term goals
+- remove drift between policy docs and current workflow
+
+## What It Does Not Include
+
+- changing product behavior
+- changing source code unless explicitly requested
+- rewriting long-form architecture without a scoped request
+
+## CI Flow Integration
+
+Use the lane split from `documentation/agents/ci_lane_contract.md`.
+
+1. Upstream GitHub lane:
+   - run markdown/doc lint checks
+   - validate docs can be rendered and linked
+   - fail fast on malformed docs
+2. Local LXD lane:
+   - run agent-driven gardening tasks
+   - update short-term coordination docs
+   - produce handoff notes for next session
+
+## Standard Gardening Task
+
+1. Read:
+   - `AGENTS.md`
+   - `documentation/agents/README.md`
+   - `documentation/short-term/coordination/workstream_board.md`
+2. Lint target docs.
+3. Apply minimal, scoped fixes.
+4. Re-run lint.
+5. Update handoff/status docs if task context changed.
+6. Report:
+   - files touched
+   - lint evidence
+   - residual risks
+
+## Baseline Commands
+
+```bash
+npx -y markdownlint-cli2 "documentation/**/*.md" AGENTS.md
+```
+
+Optional targeted run:
+
+```bash
+npx -y markdownlint-cli2 \
+  AGENTS.md \
+  documentation/agents/**/*.md \
+  documentation/short-term/**/*.md
+```
+
+## Prompt Template
+
+Use this when assigning gardening in CI/LXD:
+
+```text
+Run a document gardening pass for the current goal.
+Scope: AGENTS.md, documentation/agents/, documentation/short-term/.
+Requirements:
+1) lint markdown and fix only in-scope issues,
+2) keep behavior/policy intent unchanged,
+3) update workstream board + handoff log if status changed,
+4) report lint before/after and residual issues.
+```
+
+## Related Active Goal
+
+- `documentation/short-term/active/GOAL-003-agent-doc-lint-ci.md`
+  tracks short-term implementation for agent-driven doc linting in CI/CD.

documentation/agents/integration_test_worklist.md

@@ -0,0 +1,265 @@
+# Integration Test Worklist
+
+Date: 2026-02-12
+Scope: Shared progress tracker for integration and hardware-boundary tests in `squeezelite-esp32`.
+
+## Related Active Goal
+
+- `documentation/short-term/active/GOAL-002-hut-surface-first-test.md`
+  drives `HW-BOOT-001` across all available HUT slots on the target
+  system.
+
+## How To Use
+
+- Update this file in every integration-test PR that changes status.
+- Keep entries ordered by priority (`P0`, then `P1`, then `P2`).
+- Status values: `todo`, `in_progress`, `blocked`, `done`.
+- Add `Owner`, `Last Update`, and `Evidence` (PR, CI run, or log path) when status changes.
+- Do not remove completed rows; keep history visible.
+
+## Agent Contract
+
+Use this contract at the start of any new conversation so execution is consistent.
+
+```text
+MODE: guided+freeform
+GOAL: implement test roadmap execution using documentation/agents/integration_test_worklist.md
+START_ITEM: <ID or auto>      # e.g. UT-CHUNK-001 or auto
+CONTROL: stepwise              # one step at a time
+VALIDATION: fast|full          # default validation level
+CONSTRAINTS:
+- short answers
+- precise control points
+- update worklist status/evidence/handoff on every step
+FIRST_ACTION: propose next step with A/B/C + freeform option
+```
+
+Minimal kickoff:
+
+```text
+Use documentation/agents/integration_test_worklist.md as orchestrator.
+Run guided+freeform, short responses, one step at a time.
+Start with UT-CHUNK-001, validation=fast.
+Give A/B/C plus freeform each step.
+```
+
+### Short-Hand Hints
+
+- `kickoff auto fast` -> start from highest-priority unclaimed item with fast checks
+- `kickoff <ID> full` -> start from specific item with full checks
+- `pick A|B|C` -> choose one proposed option
+- `do: <plain instruction>` -> freeform instruction instead of multiple choice
+- `switch <ID>` -> change active item
+- `pause` -> stop changes and wait
+- `continue` -> proceed with current plan
+- `tighten` -> stricter done criteria and evidence bar
+- `status` -> one-screen summary of active item, blockers, next action
+- `handoff` -> force handoff update now (status, evidence, next)
+
+### `idf.py` Usage Hints
+
+- Baseline test-build invocation in this repo:
+  - `source /opt/esp/idf/export.sh >/tmp/idf_export.log 2>&1 && idf.py -C test build`
+- Why this is appropriate:
+  - `test/CMakelists.txt` defines a standalone ESP-IDF test project, so `-C test` is the expected entry point.
+
+For long/chatty builds, redirect to a temporary log to avoid context overload:
+
+```bash
+build_log="$(mktemp /tmp/idf_test_build.XXXXXX.log)"
+source /opt/esp/idf/export.sh >/tmp/idf_export.log 2>&1 && idf.py -C test build >"$build_log" 2>&1
+tail -n 200 "$build_log"
+```
+
+Log retention and cleanup rule:
+
+- Keep temp log files only while actively analyzing a failure.
+- Remove when no longer needed:
+  - `rm -f "$build_log" /tmp/idf_export.log`
+
+## Agent Handoff Protocol
+
+Use this protocol so any agent can continue work with minimal context loading.
+
+### Claiming
+
+1. Pick one `todo` item with highest priority and no unresolved dependency.
+2. Set `Status` to `in_progress`, set `Owner`, set `Last Update` (YYYY-MM-DD).
+3. In `Notes`, add:
+- `Context:` short current state (1 line)
+- `Next:` single next action
+- `Blockers:` `none` or short blocker text
+
+### During Work
+
+1. Keep updates compact and factual.
+2. If scope expands, add new IDs instead of rewriting existing IDs.
+3. If blocked, set `Status` to `blocked` and state unblock condition in `Notes`.
+
+### Handoff
+
+Before stopping work on an item, update:
+
+1. `Evidence`: latest PR/commit/CI/log reference.
+2. `Notes`:
+- `Done:` what was completed
+- `Next:` exact next action for the next agent
+- `Risks:` any known regression risk or uncertainty
+3. Add a one-line entry in `Activity Log`.
+
+### Done Criteria For Any Agent-Closed Item
+
+- Contract tested at stable boundary (`documentation/TESTING_CHARTER.md`).
+- Regression case included for a realistic failure mode.
+- Runnable command listed and passing evidence attached.
+- Handoff `Next` is either `none` or a linked follow-up ID.
+
+## Dependency Keys
+
+Use these keys in `Notes` when a task depends on another:
+
+- `DEP:HW-*` for hardware matrix dependencies
+- `DEP:UT-*` for unit chunk dependencies
+- `DEP:CI-*` for CI/workflow dependencies
+- `DEP:DOC-*` for required documentation updates
+
+## Activity Log
+
+Append-only, newest first.
+
+| Date | Agent | Item ID | Change | Evidence |
+|---|---|---|---|---|
+| 2026-02-12 | codex | HW-BOOT-001 | GOAL-002 parked by request; execution deferred until GOAL-001 is implemented and LXD backend is available | `documentation/short-term/coordination/workstream_board.md` |
+| 2026-02-12 | codex | HW-BOOT-001 | GOAL-002 WS1 claimed and inventory probe executed; current workspace has no serial devices, so slot mapping remains blocked pending run on LXD HIL host | `test/build/log/hut_slot_inventory_20260212.log` |
+| 2026-02-12 | codex | HW-BOOT-001 | Retried with updated IDF instructions; `idf.py -C test build` passed after sourcing `/opt/esp/idf/export.sh`; blocker narrowed to pending HIL execution | `test/build/log/idf_py_stdout_output_20260212_2.log` |
+| 2026-02-12 | codex | HW-BOOT-001 | Auto-fast kickoff claimed top P0 hardware item; fast validation blocked by missing local `idf.py` toolchain | `test/build/log/idf_py_missing_20260212.txt` |
+| 2026-02-12 | codex | UT-CHUNK-001 | Unblocked test-build path for current IDF and recorded passing fast validation | `test/build/log/idf_py_stdout_output_20413` |
+| 2026-02-12 | codex | UT-CHUNK-001 | Added bootstate regression tests; fixed test harness recovery path typo; fast build now blocked on missing `mdns` dependency | `components/tools/test/test_bootstate.cpp`, `test/CMakelists.txt`, `test/build/log/idf_py_stderr_output_2477` |
+| 2026-02-12 | codex | UT-CHUNK-001 | Claimed item and initiated guided+freeform fast kickoff | `documentation/agents/integration_test_worklist.md` |
+| 2026-02-12 | codex | DOC-TEST-ROADMAP-001 | Added no-prune roadmap and unit-test chunk structure for multi-agent execution | `documentation/agents/integration_test_worklist.md` |
+
+## Comprehensive Roadmap (No-Prune)
+
+This roadmap is intentionally exhaustive. No subsystem is excluded at this stage.
+
+### Layer Definitions
+
+- `U`: unit tests (contract-level logic and error semantics)
+- `I`: integration tests (cross-component behavior)
+- `H`: hardware/HIL tests (real device and peripheral behavior)
+- `S`: soak/endurance and recovery testing
+
+### Full Component Coverage Map
+
+| Component | Required Layers | Must-Hold Contracts (Minimum) | Priority Wave |
+|---|---|---|---|
+| `audio` | U, I, H | init/play/stop lifecycle stability; no panic on format changes | Wave 1 |
+| `codecs` | U, I | decode errors are bounded and recoverable; no invalid memory access on malformed frames | Wave 2 |
+| `display` | U, I, H | rendering bounds safety; device init/update robustness | Wave 1 |
+| `driver_bt` | U, I, H, S | pair/connect/disconnect stability; recoverable stack restart | Wave 2 |
+| `esp_http_server` | U, I | route registration/error handling remains stable under malformed requests | Wave 2 |
+| `led_strip` | U, I, H | LED state transitions deterministic; invalid config handled safely | Wave 3 |
+| `metrics` | U, I | telemetry payload correctness; metrics publication never blocks critical paths | Wave 2 |
+| `platform_config` | U, I | config defaulting and schema validation; malformed payload rejection | Wave 1 |
+| `platform_console` | U, I, H | command behavior contracts stable; failure paths return deterministic errors | Wave 2 |
+| `raop` | U, I, H | session lifecycle and stream control resilience; error recovery on network churn | Wave 3 |
+| `services` | U, I, H | queue/event/state contracts deterministic; no deadlock under pressure | Wave 1 |
+| `spotify` | U, I, H | connect/playback lifecycle and error handling remain recoverable | Wave 3 |
+| `squeezelite` | U, I, H, S | stream/decode/output stability; underrun/rebuffer recovery | Wave 1 |
+| `squeezelite-ota` | U, I, H, S | OTA success/failure/rollback safety; never brick | Wave 1 |
+| `targets` | U, I, H | target-specific init and mapping correctness (`i2s`, `muse`, `squeezeamp`) | Wave 2 |
+| `telnet` | U, I | command channel lifecycle and invalid input handling | Wave 3 |
+| `tjpgd` | U, I | image decode bounds and failure safety | Wave 3 |
+| `tools` | U, I | utility and storage helper correctness; safe error handling | Wave 1 |
+| `wifi-manager` | U, I, H, S | connection/reconnect/credential flow stability; bounded retries | Wave 1 |
+| `_override` | I, H | override behavior compatibility with base driver contracts | Wave 3 |
+| `esp-dsp` (vendor) | I, H | integration compatibility and runtime stability only | Wave 3 |
+| `spotify/cspot` (vendor) | I, H | integration compatibility and runtime stability only | Wave 3 |
+| `telnet/libtelnet` (vendor) | I, H | integration compatibility and runtime stability only | Wave 3 |
+
+### Execution Waves
+
+| Wave | Scope | Exit Criteria |
+|---|---|---|
+| Wave 1 | Release-critical contracts (`services`, `wifi-manager`, `squeezelite-ota`, `squeezelite`, `platform_config`, `display`, `tools`, `audio`) | Required `P0` chunks complete; no unresolved `P0` regressions |
+| Wave 2 | Stability amplification (`metrics`, `platform_console`, `driver_bt`, `targets`, `esp_http_server`, `codecs`) | `P1` chunks for these modules complete; nightly pass signal stable |
+| Wave 3 | Extended and compatibility coverage (`raop`, `spotify`, `telnet`, `tjpgd`, `led_strip`, `_override`, vendor integrations) | `P2` chunks and targeted soak coverage complete |
+
+### Required Artifacts Per Completed Chunk
+
+- test file(s) and contract statement
+- run command(s) and CI job reference
+- pass/fail evidence (logs, run link, or artifact path)
+- regression linkage (bug/issue/incident id if applicable)
+
+## Priority Work Queue
+
+| ID | Priority | Status | Test | Platforms | Owner | Last Update | Evidence | Notes |
+|---|---|---|---|---|---|---|---|---|
+| HW-BOOT-001 | P0 | blocked | Cold boot to operational state | all | - | 2026-02-12 | `documentation/short-term/coordination/workstream_board.md`, `test/build/log/hut_slot_inventory_20260212.log`, `test/build/log/idf_py_stdout_output_20260212_2.log` | Context: GOAL-002 is intentionally parked after accidental kickoff. Done: preserved prior inventory/build evidence and cleared active owner. Next: resume when GOAL-001 is complete and LXD hardware backend is available; then rerun slot inventory and continue WS2/WS3. Risks: none beyond explicit dependency delay. Blockers: DEP:GOAL-001 backend availability prerequisite. |
+| HW-BOOT-002 | P0 | todo | Warm reboot loop x50 | all | - | - | - | |
+| HW-BOOT-003 | P0 | todo | Platform profile/GPIO sanity | all | - | - | - | |
+| HW-STOR-001 | P0 | todo | NVS read/write/reset cycle | all | - | - | - | |
+| HW-STOR-003 | P0 | todo | SPIFFS mount + required defaults | all | - | - | - | |
+| HW-NET-001 | P0 | todo | Wi-Fi connect + DHCP + DNS | all | - | - | - | |
+| HW-NET-002 | P0 | todo | Wi-Fi AP loss/recovery reconnect | all | - | - | - | |
+| HW-AUD-001 | P0 | todo | Playback start/stop lifecycle | all | - | - | - | |
+| HW-OTA-001 | P0 | todo | OTA happy path | all | - | - | - | |
+| HW-OTA-002 | P0 | todo | OTA interrupted update recovery | all | - | - | - | |
+| HW-OTA-003 | P0 | todo | Recovery partition entry/exit | all | - | - | - | |
+| HW-PWRF-001 | P0 | todo | Power-cut/brownout recovery | all | - | - | - | |
+| HW-STOR-002 | P1 | todo | Corrupt/partial NVS recovery | all | - | - | - | |
+| HW-NET-003 | P1 | todo | mDNS announce/discover | all | - | - | - | |
+| HW-NET-004 | P1 | todo | Ethernet link up/down + DHCP traffic | ethernet-capable | - | - | - | |
+| HW-AUD-002 | P1 | todo | Format/rate transitions | all | - | - | - | |
+| HW-AUD-003 | P1 | todo | Underrun/rebuffer recovery | all | - | - | - | |
+| HW-AUD-004 | P1 | todo | Volume/mute/jack/speaker controls | platform-specific | - | - | - | |
+| HW-UI-001 | P1 | todo | Button/rotary/IR input mapping | platform-specific | - | - | - | |
+| HW-UI-002 | P1 | todo | Display init + update loop | display-capable | - | - | - | |
+| HW-PWR-001 | P1 | todo | Battery telemetry/status logic | battery-capable | - | - | - | |
+| HW-BT-001 | P1 | todo | Bluetooth pair/connect/disconnect cycles | bt-enabled | - | - | - | |
+| HW-BT-002 | P2 | todo | Bluetooth stack restart/recovery | bt-enabled | - | - | - | |
+| HW-SOAK-001 | P2 | todo | 12h playback + periodic reconnect | all | - | - | - | |
+| HW-SOAK-002 | P2 | todo | 24h mixed load soak | all | - | - | - | |
+
+## Needed Unit Test Chunks (Short-Lived Backlog)
+
+Purpose: define the minimum unit-test chunks needed now to de-risk integration work. Remove this section once all rows are `done`.
+
+| Chunk ID | Priority | Status | Required Tests | Target Component(s) | Suggested Command | Owner | Last Update | Evidence | Notes |
+|---|---|---|---|---|---|---|---|---|---|
+| UT-CHUNK-001 | P0 | in_progress | Boot/partition decision logic: normal boot, forced recovery, invalid state fallback | `services`, `bootstate` path in `test_main` | `idf.py -C test build` | codex | 2026-02-12 | `test/build/log/idf_py_stdout_output_20413` | Contract: never enters non-recoverable boot loop. Context: added `components/tools/test/test_bootstate.cpp` and updated test-build compatibility for current IDF/CMake tooling. Done: regression tests for normal counter path, forced recovery threshold boundary (`5`), invalid-state counter normalization (`>100`), and recovery reset semantics; fast validation build now passes. Next: execute/collect runtime Unity test evidence on target for chunk closure. Risks: current evidence is build-pass in fast mode; runtime execution evidence still pending. Blockers: none. |
+| UT-CHUNK-002 | P0 | todo | OTA decision and error mapping: success path, transport failure, invalid image metadata | `squeezelite-ota`, `platform_console/cmd_ota` | `idf.py -C test -T tools build` | - | - | - | Contract: failed OTA remains recoverable |
+| UT-CHUNK-003 | P0 | todo | Messaging queue contracts: publish/subscribe ordering, timeout behavior, overflow handling | `services/messaging` | `idf.py -C test -T tools build` | - | - | - | Contract: no crash or deadlock on queue pressure |
+| UT-CHUNK-004 | P1 | todo | Wi-Fi manager state transitions: connect, reconnect backoff, credential update, failure exhaustion | `wifi-manager` | `idf.py -C test -T wifi-manager build` | - | - | - | Contract: bounded retries and deterministic state |
+| UT-CHUNK-005 | P1 | todo | Display text/render boundaries: clipping, wrapping, out-of-bounds coordinates, null font/data guards | `display/core` (`gds_text`, `gds_draw`, `gds_font`) | `idf.py -C test -T tools build` | - | - | - | Contract: renderer never writes outside target buffer |
+| UT-CHUNK-006 | P1 | todo | Platform config schema handling: defaulting, unknown fields, malformed payload rejection | `platform_config` | `idf.py -C test -T platform_config build` | - | - | - | Extend existing `components/platform_config/test/` coverage |
+| UT-CHUNK-007 | P2 | todo | Input event normalization: button/rotary/IR debounce and duplicate suppression | `services/buttons`, `services/rotary_encoder`, `services/infrared` | `idf.py -C test -T tools build` | - | - | - | Contract: no event storm from bounce/repeat |
+| UT-CHUNK-008 | P2 | todo | Battery/telemetry bounds: invalid sensor values, low-battery transitions, status publication | `services/battery`, `metrics` | `idf.py -C test -T tools build` | - | - | - | Contract: invalid telemetry never triggers invalid state loops |
+
+### Chunk Completion Rule
+
+- Each chunk must add at least one regression test for a realistic failure mode.
+- Each chunk must reference contract text from `documentation/CONTRACT_TEST_TEMPLATE.md` in PR notes.
+- Mark chunk `done` only after test pass evidence is attached.
+
+## Agent Startup Checklist
+
+1. Read only these sections first: `Activity Log`, `Needed Unit Test Chunks`, `Priority Work Queue`.
+2. Choose one highest-priority unclaimed item.
+3. Claim it using `Agent Handoff Protocol`.
+4. Execute targeted tests first; avoid full-matrix runs unless required by the item.
+5. Leave a complete handoff entry before ending session.
+
+## Definition Of Done
+
+- Test case exists at a stable contract boundary and follows `documentation/TESTING_CHARTER.md`.
+- Platform scope is explicit (`all` or constrained target set).
+- Execution path is documented (local command or CI job).
+- Pass evidence is linked in `Evidence`.
+
+## Update Example
+
+| ID | Priority | Status | Test | Platforms | Owner | Last Update | Evidence | Notes |
+|---|---|---|---|---|---|---|---|---|
+| HW-NET-001 | P0 | done | Wi-Fi connect + DHCP + DNS | all | @agent-name | 2026-02-12 | PR #123, CI run #456 | Added regression for reconnect timeout handling |

documentation/agents/remote_delegation_contract.md

@@ -0,0 +1,120 @@
+# Remote Delegation + Threading Contract
+
+This document makes remote delegation predictable across sessions and
+reduces "arch has to re-steer every time" drift.
+
+Scope: how `arch:codex` delegates to `runner:codex` / `infra:codex` as
+autonomous executors, including a lightweight handshake and a stable
+thread reference mechanism.
+
+## Definitions
+
+- `delegation packet`: a Markdown prompt intended to be fed to a remote
+  role Codex runtime (typically via SSH transport).
+- `thread_ref`: an opaque identifier chosen by `arch` to correlate a
+  multi-step remote effort across sessions (example:
+  `thread_ref=RUNNER-WS8-20260213A`).
+- `codex_thread_id` (optional): if the remote runtime exposes an
+  internal conversation/thread ID, record it, but do not depend on it.
+- `executor=remote_codex`: work is executed by the Codex runtime inside
+  the role environment/repository.
+- `executor=ssh_direct`: bootstrap/emergency only; direct commands run
+  without prompting a remote Codex runtime.
+
+Authoritative state is always in repos + evidence + commit SHAs, not in
+chat history.
+
+## Delegation Is Not Micromanagement
+
+When `arch` delegates to a remote role, `arch` provides:
+
+- objective and constraints (what outcome, what invariants)
+- acceptance criteria and evidence format (what "done" looks like)
+- reporting requirements (commit SHA(s), evidence paths, next action)
+- escalation gates (what requires operator, what must bounce to another
+  role)
+
+Remote roles decide implementation details and may adjust the plan if
+they can still satisfy the acceptance criteria. If they deviate, they
+must explain the deviation in their handoff summary.
+
+Commit hygiene rule:
+- each role Codex runtime owns commit hygiene in its own role repository
+  (small commits, frequent pushes, SHA-based reporting). `arch` must not
+  "paper over" missing remote commits by committing into role repos
+  locally.
+
+## Handshake (HANDSHAKE-001)
+
+Problem this solves: starting a new session without relying on implicit
+context, while still enabling a stable multi-step "thread".
+
+Rule:
+- For any multi-step remote effort (or any time context is uncertain),
+  start with a 30-90 second handshake request before deeper execution.
+
+Handshake request must include:
+- `thread_ref=<value>`
+- target scope (workstream/ticket)
+- the single most important success condition for the next step
+
+Handshake response must include (in the role repo handoff log entry):
+- `thread_ref=<same value>`
+- `executor=remote_codex` (or `executor=ssh_direct` if forced)
+- current `HEAD` commit SHA and branch
+- 1-3 bullets: what the role believes is the best next step and why
+- any blockers with `operator_required=yes|no`
+
+If the role can provide a `codex_thread_id`, include it as
+`codex_thread_id=<value>` in the summary, but treat it as advisory.
+
+## Parallel Threads
+
+Parallelism is allowed and expected:
+
+- `arch` may run multiple simultaneous delegations, each with its own
+  `thread_ref`.
+- Do not try to "multiplex" multiple unrelated objectives into one
+  thread.
+- If two threads touch the same stateful target (for example the same
+  HUT slot, the same VM configuration, or the same infra resource),
+  the remote role must serialize safely (locks or explicit sequencing).
+
+Transport note:
+- SSH is the transport. It does not need to be persistent to enable
+  parallel threads. Correlation happens via `thread_ref` + evidence +
+  commit SHA(s), not via a single interactive SSH session.
+
+## Where To Record `thread_ref`
+
+`arch` must record the `thread_ref` in:
+
+- the delegation packet header (or first lines), and
+- the `documentation/short-term/coordination/handoff_log.md` summary
+  field for `action_type=delegate` and subsequent related entries.
+
+Remote roles must record the same `thread_ref` in their role-local
+handoff log for related entries.
+
+## Closure Contract (When `arch` Owns It)
+
+If `arch:codex` is assigned as the owner (or is explicitly asked to "take
+ownership") for a task/workstream that requires remote-role execution,
+`arch` must drive the work to a stable closure state:
+
+- `done`: acceptance met and evidence + commit SHA(s) recorded, or
+- `blocked`: concrete blocker(s) recorded with:
+  - correct role owner(s) for the next action (`infra:*` / `runner:*` /
+    `operator`)
+  - `operator_required=yes|no`
+  - the exact next action (packet or command family), not a vague
+    placeholder
+
+Delegation packets are necessary but not sufficient: `arch` remains
+responsible for follow-through and for keeping the short-term board and
+handoff logs aligned with reality.
+
+Allowed stop condition:
+- `arch` may stop when a pivot is required or a problem-solving loop is
+  stalled, but must first record a closure state (`replan` or `blocked`)
+  with concrete next actions and the information needed to resume.

documentation/agents/remote_transport_lock.md

@@ -0,0 +1,41 @@
+# Remote Transport Lock Record
+
+This file is the single source of truth for the remote-role prompting
+transport mechanism (`arch` -> `runner` / `infra`).
+
+Once `status: locked`, follow-on work must implement the locked
+mechanism and must not re-litigate transport per session.
+
+## Status
+
+status: evaluating
+locked_at_utc: (unset)
+
+## Current Mechanism
+
+primary_transport: ssh
+fallback_transport: ssh
+
+## Candidate Mechanisms
+
+- codex_app_server
+- codex_mcp_server
+
+## Decision Criteria (v1)
+
+- supports parallel, multi-step threads without relying on implicit chat
+  context
+- supports stable correlation (`thread_ref`) across sessions
+- preserves repo boundary contract (role repos remain authoritative)
+- does not require secrets to be written into repositories
+- has a clear break-glass path (SSH) during rollout
+
+## Notes / Next Actions
+
+- Populate evaluation notes and prototype plan under GOAL-001/WS12.
+- When ready to lock:
+  - set `status: locked`
+  - set `primary_transport: <chosen>`
+  - record `locked_at_utc`
+  - record rollback/fallback expectations
+

documentation/agents/start_here.md

@@ -0,0 +1,67 @@
+# Agent Start Here
+
+Use this file to avoid loading large context up front.
+
+## 90-Second Startup
+
+1. Read `AGENTS.md` for repository policy.
+2. Read `documentation/agents/ci_lane_contract.md` to select lane:
+   - upstream GitHub lane (no hardware)
+   - local LXD lane (hardware/HIL)
+3. If the request requires remote-role work (`infra`/`runner`), read:
+   - `documentation/agents/remote_delegation_contract.md` (handshake +
+     `thread_ref` correlation)
+4. If the request references a short-term goal, read:
+   - `documentation/short-term/coordination/workstream_board.md`
+   - the referenced file under `documentation/short-term/active/`
+   - `documentation/short-term/coordination/handoff_log.md`
+   - owner-governance rules in the board (`Owner Governance` section)
+   - role activation tracker/state in the board
+5. Read one task route doc only:
+   - hardware/integration execution:
+     `documentation/agents/integration_test_worklist.md`
+   - documentation linting/gardening:
+     `documentation/agents/document_gardening.md`
+   - frontend payload/route/API changes:
+     `documentation/agents/frontend_requirements_context.md`
+
+## Load-On-Demand Rule
+
+- Start with the smallest possible doc set.
+- Pull in more docs only when the current task hits a dependency.
+- Keep updates local to the active goal/workstream.
+
+## Session Close-Out
+
+1. Update active goal/workstream status if changed.
+2. Add evidence pointers (logs, run IDs, artifacts, commands).
+3. Add a handoff line in
+   `documentation/short-term/coordination/handoff_log.md` when work remains.
+4. If you are the owner (especially `arch`) and work is not `done`,
+   ensure the workstream is either:
+   - `blocked` with concrete blockers + owners + operator gate + next
+     action, or
+   - explicitly delegated with a `thread_ref` and a scheduled follow-up
+     check (do not leave work as "implicitly pending")
+   - if stopping due to a required pivot or a stalled loop, log
+     `action_type=replan` (pivot) or `blocked` (stall) with the new
+     decision/unknowns captured
+5. If closing a goal: verify all deliverables checklist items in the goal
+   file are checked. If any are unchecked, keep goal status active and do
+   not archive.
+6. For remote-role work:
+   - do not set `in_progress` when owner is `tbd`
+   - include `context`, `action_type`, and `operator_required` fields in
+     handoff entries
+   - update role activation tracker (`unassigned`/`assigned`/
+     `first_heartbeat`/`active`) when state changes
+7. For user/operator course-correction signals (for example: \"we
+   missed...\", \"easier way...\", \"we should prioritize...\"):
+   - treat as formal `action_type=replan`
+   - update goal workstreams + board dependencies before resuming
+     execution
+8. For ad-hoc user requests:
+   - proactively ask if user wants ticket tracking (`yes`/`no`)
+   - if `yes`, create/update
+     `documentation/short-term/coordination/ad_hoc_ticket_queue.md`
+   - if `no`, proceed and log `ticket_tracking=declined` in handoff

documentation/short-term/README.md

@@ -0,0 +1,257 @@
+# Short-Term Goals (Ephemeral)
+
+Use this folder for time-boxed, multi-agent implementation goals.
+
+This area is intentionally operational and temporary:
+
+- track current work
+- coordinate handoffs
+- capture blockers and next actions
+- archive when complete
+
+## Structure
+
+- `documentation/short-term/active/`
+  - active goal documents (one file per goal)
+- `documentation/short-term/coordination/workstream_board.md`
+  - current ownership and status by workstream
+- `documentation/short-term/coordination/handoff_log.md`
+  - chronological handoffs across agents/sessions
+- `documentation/short-term/coordination/ad_hoc_ticket_queue.md`
+  - optional ticket tracking queue for user-approved ad-hoc requests
+- `documentation/short-term/archive/`
+  - closed goals moved out of active rotation
+
+## Operating Rules
+
+1. Keep one active goal file as the source of truth for scope and acceptance criteria.
+2. Update `workstream_board.md` when status, owner, or blocker changes.
+3. Add a handoff entry before ending a session when work is in progress.
+4. Keep entries concise and actionable; do not duplicate long-term
+   architecture docs here.
+5. Treat workstream completion and goal completion separately:
+   - workstream `done` requires that workstream acceptance is met
+   - goal completion requires every deliverable checklist item in the goal file checked with evidence
+6. Move completed goals to `archive/` only after all deliverable checklist
+   items are checked; leave a short completion note in the board.
+7. For docs-maintenance sessions, use
+   `documentation/agents/document_gardening.md`.
+8. Use role-based ownership format in workstream boards:
+   `<role>:<agent-id>` where role is one of:
+   - `arch` (Architecture Agent)
+   - `infra` (Infrastructure Agent)
+   - `runner` (Runner Agent)
+9. Enforce repository boundaries:
+   - `infra` and `runner` operate from dedicated repositories
+   - `infra` must never clone product repositories like
+     `squeezelite-esp32`
+   - `arch` must never execute `runner` or `infra` workstreams by
+     editing or running commands in their repositories locally (for
+     example `/workspaces/codex-runner-agent`); instead, delegate by
+     prompting the remote Codex runtime in that role environment
+10. Handoff entries must include execution metadata:
+   - `context`: `arch-local` | `infra-live` | `runner-live`
+   - `action_type`: `scaffold` | `delegate` | `execute` | `replan`
+   - `operator_required`: `yes` | `no`
+11. Use evidence naming prefixes by role:
+   - `arch_*`, `infra_*`, `runner_*`
+12. For remote-role workstreams (`infra`/`runner`):
+   - do not move to `in_progress` while owner is `tbd`
+   - complete delegation checklist before execution:
+     - owner assigned
+     - activation state at least `assigned`
+     - first handoff line logged in role context
+13. Owner semantics are execution semantics:
+   - `<role>:codex` means Codex is installed/authenticated in that role
+     environment and the work is executed by that remote Codex runtime.
+   - `<role>:agent` means the `arch:codex` control-plane is acting as a
+     temporary proxy executor for that role (typically via SSH transport).
+     This is allowed only for bootstrap/emergency and must be explicitly
+     called out in handoff summaries as `executor=ssh_direct`.
+   - avoid ambiguous ownership values like `infra:codex` unless the
+     remote Codex runtime is actually active and being prompted there.
+14. Commit hygiene (arch + runner):
+   - make small commits that correspond to workstream-sized progress
+   - push frequently so progress is traceable by remote commit SHA
+   - include commit SHA(s) in handoff summaries when work is executed
+   - do not batch unrelated changes; avoid “mega commits”
+   - do not commit secrets or generated/build outputs
+
+## Ad-hoc Ticket Reflex
+
+When user/operator gives an ad-hoc request, agents should proactively
+offer ticket tracking and let the user decide.
+
+Required reflex:
+1. Ask: "Track this as a ticket? (yes/no)" before major execution.
+2. If `yes`:
+   - create/update entry in
+     `documentation/short-term/coordination/ad_hoc_ticket_queue.md`
+   - reference ticket id in handoff summaries and relevant workstream
+     notes
+3. If `no`:
+   - continue execution without ticket creation
+   - add one line in handoff summary: `ticket_tracking=declined`
+4. Prefer `yes` recommendation when request affects:
+   - cross-session continuity
+   - cross-role delegation
+   - sequencing/prioritization
+   - non-trivial risk or scope
+
+## GitLab Credential Home Contract
+
+Use this contract for agent-repo bootstrap work targeting
+`git.lecsys.net`.
+
+Canonical local secret home:
+- `${XDG_CONFIG_HOME:-$HOME/.config}/codex/credentials/gitlab/git.lecsys.net/`
+
+Required files (one per account/role):
+- `codex.env`
+- `arch.env`
+- `infra.env`
+- `runner.env`
+
+Required file variables:
+- `GITLAB_HOST=git.lecsys.net`
+- `GITLAB_USER=<account>`
+- `GITLAB_PASSWORD=<bootstrap-only>`
+- `GITLAB_PAT=<preferred for non-interactive git/api>`
+
+## Remote Agent Credential + Invocation Contract
+
+Remote agents need two things:
+1. credentials stored locally in a non-repo location, and
+2. a documented transport/invocation mechanism.
+
+### Credential Storage (Do Not Commit)
+
+- SSH transport (example: LXD host access):
+  - configure connection variables in `.lxd.env` (see `build-scripts/lxd_remote.sh`)
+  - store private keys outside the repo (typically under `~/.ssh/`) and
+    point to them via `LXD_SSH_PRIVATE_KEY_PATH`
+- GitLab (agent repos):
+  - store role credentials in:
+    `${XDG_CONFIG_HOME:-$HOME/.config}/codex/credentials/gitlab/git.lecsys.net/<role>.env`
+    (mode `0600`; parent directory mode `0700`)
+  - prefer PAT for non-interactive git/API; never embed secrets in
+    remotes, scripts, or handoff logs
+- Codex login (remote Codex runtime):
+  - authentication state lives on the remote machine under the remote
+    user's Codex config/state (do not copy it into repos)
+
+### Invocation Mechanism (Prompting Remote Codex)
+
+When a workstream owner is `<role>:codex`, delegation means: prompt the
+Codex runtime in that role environment to do the work inside its role
+repository, then report back via commit SHA + evidence paths.
+
+Transport is typically SSH, but SSH is not the execution model:
+- `executor=remote_codex`: SSH is only used to reach the remote and run
+  `codex exec` (or open an interactive Codex session) on that machine.
+- `executor=ssh_direct`: bootstrap/emergency only (direct commands
+  without prompting a remote Codex runtime).
+
+Template (non-interactive delegation using a packet read from stdin):
+
+```bash
+ssh -i ~/.ssh/<role_key> <role_user>@<role_host> \\
+  'cd /path/to/<role-repo> && codex exec --sandbox workspace-write -C . -' \\
+  < documentation/short-term/coordination/delegations/<packet>.md
+```
+
+Security contract:
+1. Create root directory with mode `0700`.
+2. Create each `*.env` file with mode `0600`.
+3. Do not store secrets in repository files, commit messages, or
+   handoff summaries.
+4. Propagate only role-specific credential files to target runtime
+   hosts; never copy the entire secret home.
+
+Auth mode contract:
+1. Account password is bootstrap-only (initial sign-in/token creation).
+2. PAT is the default for non-interactive API/git operations.
+3. Git remotes use HTTPS hostnames without embedded credentials.
+
+Verification flow (record only redacted evidence):
+1. Verify permissions: `stat` on secret root and role file.
+2. Verify auth: `curl --fail -H "PRIVATE-TOKEN: $GITLAB_PAT"
+   https://git.lecsys.net/api/v4/user`.
+3. Verify repo reachability: `git ls-remote
+   https://git.lecsys.net/<namespace>/<repo>.git`.
+4. Log pass/fail and command context without printing token/password.
+
+## Agent Repo Bootstrap Flow (ADHOC-20260212-03)
+
+GitLab remotes created/pushed:
+- `http://git.lecsys.net/infra/infra-agent.git`
+- `http://git.lecsys.net/runner/runner-agent.git`
+
+Bootstrap checkout commands:
+
+```bash
+git clone http://git.lecsys.net/infra/infra-agent.git /workspaces/codex-infra-agent
+git -C /workspaces/codex-infra-agent checkout main
+
+git clone http://git.lecsys.net/runner/runner-agent.git /workspaces/codex-runner-agent
+git -C /workspaces/codex-runner-agent checkout main
+```
+
+Bootstrap verification:
+
+```bash
+git -C /workspaces/codex-infra-agent remote -v
+git -C /workspaces/codex-runner-agent remote -v
+git ls-remote http://git.lecsys.net/infra/infra-agent.git
+git ls-remote http://git.lecsys.net/runner/runner-agent.git
+```
+
+Transport note (2026-02-12):
+- `https://git.lecsys.net` was unreachable from this execution
+  environment (TCP/443 connect failure), while `http://git.lecsys.net`
+  was reachable and used for bootstrap push/clone.
+- After TLS is available, migrate remotes to `https://` and switch to
+  PAT-based auth for non-interactive operations.
+
+## Goal Adjustment Protocol
+
+Apply this protocol for both hard prerequisites and softer strategy
+changes (priority shifts, easier path proposals, intermediate quests).
+
+Trigger examples from operator/user input:
+- \"we missed ...\"
+- \"in order to achieve this in an easier way ...\"
+- \"we should prioritize ...\"
+
+Required actions:
+1. Add a `replan` handoff entry documenting the requested change.
+2. Freeze affected workstreams (`blocked`) if current sequencing is no
+   longer valid.
+3. Add/reshape workstreams in the active goal for:
+   - prerequisite fixes, and/or
+   - intermediate quest path(s) that improve delivery.
+4. Rewire dependencies/blockers and next actions in the board.
+5. Reconfirm owner assignment and activation state for impacted remote
+   roles.
+6. Resume execution only after the new plan is reflected in goal + board
+   + handoff docs.
+
+## Goal Template
+
+When creating a new short-term goal, start from:
+
+- `documentation/short-term/active/GOAL_TEMPLATE.md`
+
+## Current Active Goals
+
+- `GOAL-001`: LXD Codex orchestration and multi-HUT CI foundation
+- `GOAL-002`: HUT surfacing with first implemented hardware test (`HW-BOOT-001`)
+- `GOAL-003`: agent-driven documentation linting in CI/CD
+
+## Agent Pickup Order
+
+When asked to continue short-term work, read in this order:
+
+1. `documentation/short-term/coordination/workstream_board.md`
+2. relevant file in `documentation/short-term/active/`
+3. `documentation/short-term/coordination/handoff_log.md`

documentation/short-term/active/GOAL-001-lxd-codex-hardware-ci.md

@@ -0,0 +1,678 @@
+# GOAL-001: LXD Codex Orchestration + Multi-Hardware CI
+
+Status: active  
+Owner: arch:codex  
+Last updated: 2026-02-12
+
+## Objective
+
+Establish a durable workflow where:
+1. the Architecture Agent (`arch`) in this repository remains the
+   code-authoring and coordination control plane,
+2. the Architecture Agent can drive an Infrastructure Agent (`infra`)
+   for host/container/VM maintenance,
+3. the Architecture Agent can drive a Runner Agent (`runner`) inside
+   the provisioned VM for CI/CD and hardware execution,
+4. the LXD VM becomes the operational base for hardware-aware CI, and
+5. multiple hardware-under-test (HUT) nodes can be flashed/tested in
+   parallel with safe, locked power control.
+
+## Follow-On Goals
+
+- `documentation/short-term/active/GOAL-002-hut-surface-first-test.md`
+  for first cross-slot HUT surfacing execution.
+- `documentation/short-term/active/GOAL-003-agent-doc-lint-ci.md`
+  for documentation linting and gardening flow in CI/CD.
+
+## CI Environment Split (required contract)
+
+There are two CI lanes and they must remain distinct:
+
+1. Upstream GitHub lane (no physical hardware):
+   - compile, package, static checks, unit/integration tests that do not require physical devices
+   - never assume serial ports, relay access, or USB-attached targets
+2. Local LXD hardware lane (physical HUT available):
+   - flash/monitor, hardware integration tests, soak runs, power-cycle recovery flows
+
+Seamless operation requirement:
+- hardware lane consumes the same commit SHA/artifacts produced by upstream lane (or rebuilds deterministically from same SHA),
+- result reporting flows back to shared status artifacts and short-term board,
+- no hidden environment-only behavior differences outside explicitly documented hardware dependencies.
+
+Note:
+- CI lanes define where tests run.
+- Agent roles define which Codex agent controls which environment.
+- These are orthogonal and must not be conflated.
+
+## Agent Roles (required contract)
+
+Short names (use these in board ownership and reports):
+
+1. `arch`: Architecture Agent (main engineering/control plane)
+2. `infra`: Infrastructure Agent (platform/host/VM operations)
+3. `runner`: Runner Agent (VM CI/CD + test execution)
+
+Role responsibilities:
+
+1. `arch`
+   - owns product-repo development, task coordination, and status
+     reporting governance
+   - delegates remote work to role-local Codex runtimes when available
+     (`infra:codex`, `runner:codex`); SSH is treated as a transport
+     layer, not as the execution model
+   - mentors and improves `infra`/`runner` operating efficiency over
+     time; must propose improvements when asked or when major
+     inefficiencies degrade team throughput
+   - governs documentation quality, tracking hygiene, and baseline
+     environment decisions
+   - keeps governance documentation enabling and operational, not
+     artificially restrictive
+2. `infra`
+   - manages host/container/VM provisioning and maintenance
+   - never clones or edits this product repository
+   - maintains its own repository and AGENT documentation
+   - does not communicate directly with `runner` except when only local
+     host commands can reach the VM
+3. `runner`
+   - executes CI/CD and test workflows inside VM scope
+   - reports structured artifacts/status back to `arch`
+   - maintains its own repository and AGENT documentation
+
+## Owner Semantics (Prevent Drift)
+
+Owner values encode where execution happens:
+
+1. `<role>:codex`
+   - Codex is installed/authenticated in that role environment.
+   - Coordination means prompting that remote Codex runtime to execute
+     work in its role repository (not SSH-direct command execution).
+2. `<role>:agent`
+   - The `arch:codex` control-plane is acting as a temporary proxy
+     executor for that role (typically via SSH transport).
+   - Allowed only for bootstrap/emergency.
+   - Must be called out in handoff summaries as `executor=ssh_direct`.
+
+## Repository Boundary Contract (required target)
+
+1. Product repository (`squeezelite-esp32`) for `arch`.
+2. Infrastructure repository for `infra`.
+3. Runner orchestration repository for `runner`.
+
+Required behavior:
+- each role has a dedicated repository
+- each remote role (`infra`, `runner`) has independent `AGENTS.md`,
+  workstream board, and handoff log
+- strict execution boundary:
+  - `arch` must never implement `runner` workstreams by modifying or
+    running commands inside the runner repository on the development
+    machine (example local path: `/workspaces/codex-runner-agent`)
+  - `runner` workstreams are executed only by `runner:codex` inside the
+    runner VM, with results reported back via runner evidence + commit SHA
+- cross-agent handoffs include commit SHA, agent role, status, and
+  artifact/log pointers
+- optional local GitLab can be used as shared upstream for portability
+  and document access when the operator enables it
+
+## Operator Touchpoints
+
+1. Operator assistance is explicitly expected for:
+   - runner Codex interactive authentication
+   - optional local GitLab setup/onboarding for agent repos
+2. `arch` should request operator help when these gates are reached
+   instead of introducing brittle workarounds.
+
+## GitLab Credential Contract (ADHOC-20260212-02)
+
+This contract standardizes secret location, auth mode, and verification
+for agent-repo bootstrap on `git.lecsys.net`.
+
+Canonical secret root:
+- `${XDG_CONFIG_HOME:-$HOME/.config}/codex/credentials/gitlab/git.lecsys.net/`
+
+Expected role files:
+- `codex.env`, `arch.env`, `infra.env`, `runner.env`
+
+Minimum variables per role file:
+- `GITLAB_HOST`
+- `GITLAB_USER`
+- `GITLAB_PASSWORD` (bootstrap-only)
+- `GITLAB_PAT` (required for non-interactive flows)
+
+Required controls:
+1. permissions: root `0700`, files `0600`
+2. auth mode: PAT over HTTPS for automation; password only for bootstrap
+3. propagation: transfer role-local file only to its target runtime
+4. evidence hygiene: record command outcome, never raw secret values
+
+## Agent Repo Remote Bootstrap (ADHOC-20260212-03)
+
+Bootstrapped remotes and default branch:
+- `infra` repo: `http://git.lecsys.net/infra/infra-agent.git` (`main`)
+- `runner` repo: `http://git.lecsys.net/runner/runner-agent.git` (`main`)
+
+Bootstrap checkout contract:
+1. clone role repo to its canonical workspace path
+2. checkout `main`
+3. verify `origin` remote URL and branch tracking
+4. verify remote reachability with `git ls-remote`
+
+Evidence locations:
+- `/workspaces/codex-infra-agent/README.md`
+- `/workspaces/codex-runner-agent/README.md`
+- `documentation/short-term/coordination/handoff_log.md`
+
+## Container Agent Pickup (first 15 minutes)
+
+1. Read:
+   - `documentation/short-term/coordination/workstream_board.md`
+   - `documentation/short-term/coordination/handoff_log.md`
+2. Claim one workstream by updating board owner/status.
+3. Execute only the claimed workstream scope.
+4. Log handoff with concrete next step before stopping.
+
+## Workstreams
+
+- WS12: Codex App Server Transport (evaluate -> prototype -> implement)
+  - Objective: evaluate Codex App Server as a more structured transport
+    than SSH for prompting `runner:codex` / `infra:codex`, especially
+    for parallel multi-step threads and stable context reuse.
+  - Mechanism lock (required):
+    - WS12 must produce and maintain a single transport decision record
+      in long-lived docs:
+      - `documentation/agents/remote_transport_lock.md`
+    - Once `remote_transport_lock.md` sets `status: locked`, follow-on
+      WS12 work must implement that locked mechanism (no re-litigating
+      transport per session).
+    - SSH remains the break-glass fallback unless explicitly retired in
+      the lock record.
+  - Acceptance (v1):
+    - Evaluation:
+      - documented transport decision criteria and current status
+        (evaluating/locked) in `remote_transport_lock.md`
+    - Prototype:
+      - a minimal proof-of-viability plan that can be executed by the
+        appropriate role(s) without `arch` becoming the executor
+    - Implementation:
+      - documented invocation pattern(s) (no secrets) that preserve the
+        repo boundary contract and keep `thread_ref` correlation usable
+        across sessions and in parallel
+
+## Delegation Gate (required before remote-role execution)
+
+Before executing any `infra` or `runner` workstream:
+
+1. assign concrete owner (`infra:<id>` or `runner:<id>`)
+2. verify role activation state is at least `assigned`
+3. log first role-context handoff entry
+4. record operator gate for the action:
+   - `operator_required=yes` or `operator_required=no`
+5. record execution mechanism in the handoff summary:
+   - `executor=remote_codex` (prompt remote Codex runtime), or
+   - `executor=ssh_direct` (bootstrap/emergency only)
+
+Handoff lines should include:
+- `context`: `arch-local` | `infra-live` | `runner-live`
+- `action_type`: `scaffold` | `delegate` | `execute` | `replan`
+- `operator_required`: `yes` | `no`
+
+Ad-hoc request reflex:
+- ask user: `Track this as a ticket? (yes/no)`
+- if `yes`, create/update
+  `documentation/short-term/coordination/ad_hoc_ticket_queue.md`
+- if `no`, continue and include `ticket_tracking=declined` in handoff
+  summary
+
+Evidence naming convention:
+- `arch_*`, `infra_*`, `runner_*`
+
+## WS1: Infrastructure SSH + Codex passthrough
+
+### Outcome
+
+`arch` can reliably trigger/drive commands on the `infra` runtime over
+SSH.
+
+### Tasks
+
+1. Validate key-based SSH path and non-interactive command execution.
+2. Standardize connection variables in `.lxd.env` (host/user/key/path).
+3. Create a minimal remote control wrapper script (local side) for repeatable calls.
+4. Add heartbeat check command (`hostname`, `uptime`, `whoami`, disk availability).
+
+### Acceptance
+
+- One command from `arch` context can run remote shell on `infra`
+  without manual interaction.
+- Failures return non-zero status and readable diagnostics.
+
+## WS2: Multi-agent operating model
+
+### Outcome
+
+`arch` can delegate infrastructure operations to `infra` and CI/CD
+execution to `runner`, then collect structured status back without role
+ambiguity.
+
+### Tasks
+
+1. Define role-responsibility contract for `arch`/`infra`/`runner`
+   including explicit no-product-repo boundary for `infra`.
+2. Define contract for remote job invocation:
+   - target role (`infra` or `runner`)
+   - job id
+   - working directory
+   - command bundle
+   - expected artifacts/log paths
+3. Define report format (single machine-readable summary + human note)
+   including `agent_role`, `commit_sha`, and `artifact_paths`.
+4. Add guardrails:
+   - one remote job per lock file
+   - timeout policy
+   - cancellation behavior
+5. Define minimum required files for each agent-role repository:
+   - `AGENTS.md` with startup path and discovery-first checks
+   - workstream board/handoff format for cross-session continuation
+6. Smoke test one `infra` maintenance command through
+   `build-scripts/lxd_remote.sh` and log evidence.
+
+### Acceptance
+
+- Remote execution contract documents all three agent roles and their
+  boundaries.
+- `infra` instructions explicitly avoid assuming existing host/VM
+  resources.
+- At least one `infra` smoke task is executed with evidence.
+
+### WS2 Contract Draft (v1)
+
+#### Role Responsibility Contract
+
+1. `arch` (Architecture Agent, this repo):
+   - prepares job request and selects target role
+   - may edit product code
+   - collects reports and updates coordination docs
+2. `infra` (Infrastructure Agent):
+   - manages host/VM lifecycle and host maintenance only
+   - must not edit product code repositories directly
+   - must run discovery-first checks before mutating host state
+3. `runner` (Runner Agent inside VM):
+   - executes CI/CD and hardware workflows in VM scope
+   - may operate on product repo clones/worktrees inside VM
+   - does not own host virtualization lifecycle
+
+#### Invocation Schema (machine-readable)
+
+```json
+{
+  "schema_version": "ws2.invocation.v1",
+  "job_id": "GOAL-001-WS2-0001",
+  "target_role": "infra",
+  "requested_by": "arch",
+  "request_utc": "2026-02-12T15:30:00Z",
+  "commit_sha": "<git sha or none>",
+  "repo": "squeezelite-esp32",
+  "workdir": "/home/codexsvc",
+  "command": [
+    "hostname",
+    "whoami",
+    "uptime"
+  ],
+  "timeout_sec": 600,
+  "lock_key": "goal001-ws2-smoke",
+  "expected_artifacts": [
+    "test/build/log/lxd_ws2_smoke_YYYYMMDD.log"
+  ],
+  "notes": "WS2 role routing smoke test"
+}
+```
+
+Required fields:
+- `schema_version`, `job_id`, `target_role`, `requested_by`,
+  `request_utc`, `command`, `timeout_sec`, `lock_key`.
+
+Allowed `target_role` values:
+- `infra`
+- `runner`
+
+#### Report Schema (machine-readable + human note)
+
+```json
+{
+  "schema_version": "ws2.report.v1",
+  "job_id": "GOAL-001-WS2-0001",
+  "agent_role": "infra",
+  "status": "success",
+  "start_utc": "2026-02-12T15:30:10Z",
+  "end_utc": "2026-02-12T15:30:12Z",
+  "duration_sec": 2,
+  "exit_code": 0,
+  "lock_key": "goal001-ws2-smoke",
+  "executor": {
+    "hostname": "hpi5-2",
+    "user": "codexsvc"
+  },
+  "commit_sha": "<git sha or none>",
+  "artifact_paths": [
+    "test/build/log/lxd_ws2_smoke_YYYYMMDD.log"
+  ],
+  "human_summary": "Infra-role smoke command completed successfully.",
+  "next_action": "Proceed with role-specific AGENT repo skeletons."
+}
+```
+
+Required fields:
+- `schema_version`, `job_id`, `agent_role`, `status`, `start_utc`,
+  `end_utc`, `duration_sec`, `exit_code`, `lock_key`, `human_summary`.
+
+Allowed `status` values:
+- `success`
+- `failed`
+- `timeout`
+- `cancelled`
+
+#### Guardrail Defaults
+
+1. Locking:
+   - lock file path pattern:
+     `/tmp/codex-<target_role>-<lock_key>.lock`
+   - only one active job per `(target_role, lock_key)` tuple
+2. Timeout defaults:
+   - `infra`: 600 seconds default, 3600 seconds max
+   - `runner`: 1800 seconds default, 14400 seconds max
+3. Cancellation behavior:
+   - cancellation request writes
+     `/tmp/codex-cancel-<job_id>.flag`
+   - executor checks cancellation between command steps and returns
+     status `cancelled`
+4. Non-zero exits:
+   - any non-zero command exit returns report `status=failed`
+   - partial outputs are still published in `artifact_paths`
+
+#### Minimum Files For Each Agent-Role Repository
+
+1. `AGENTS.md`
+   - startup read order
+   - role boundaries
+   - discovery-first checks
+2. `documentation/coordination/workstream_board.md`
+   - owner/status/blocker/next action table
+3. `documentation/coordination/handoff_log.md`
+   - timestamped handoff lines with evidence pointers
+4. `scripts/`
+   - executable wrappers for invocation, reporting, and locks
+
+#### WS2 Evidence
+
+- Smoke command log:
+  - `test/build/log/lxd_ws2_smoke_20260212.log`
+- Structured report:
+  - `test/build/log/lxd_ws2_report_20260212.json`
+
+## WS3: Infrastructure agent bootstrap
+
+### Outcome
+
+`infra` has a dedicated repository, baseline `AGENTS.md`, and
+coordination artifacts so host/VM operations can run independently from
+this product repository.
+
+### Tasks
+
+1. Create/initialize the `infra` repository.
+2. Add role-specific `AGENTS.md` with discovery-first startup checks.
+3. Add `workstream_board.md` and `handoff_log.md` in the infra repo.
+4. Document explicit boundary: `infra` must never clone
+   `squeezelite-esp32`.
+
+### Acceptance
+
+- Infra repo exists with required files and startup path.
+- Role boundary rules are documented and testable.
+
+### WS3 Evidence
+
+- Infra repository path:
+  - `/workspaces/codex-infra-agent`
+- Infra governance files:
+  - `/workspaces/codex-infra-agent/AGENTS.md`
+  - `/workspaces/codex-infra-agent/documentation/coordination/workstream_board.md`
+  - `/workspaces/codex-infra-agent/documentation/coordination/handoff_log.md`
+
+## WS4: Runner VM provisioning
+
+### Outcome
+
+`infra` can provision the runner VM with deterministic naming and
+baseline dependencies.
+
+### Tasks
+
+1. Provision VM instance for `runner` workloads.
+2. Record VM identity (name/IP/image/resources) in infra tracking docs.
+3. Install baseline packages required for remote management.
+
+### Acceptance
+
+- VM is created and reachable from host.
+- Provisioning commands and outputs are captured in infra evidence logs.
+
+## WS5: Runner SSH reachability
+
+### Outcome
+
+`arch` can reach the runner VM over SSH using non-interactive key auth.
+
+### Tasks
+
+1. Configure SSH service/user/key on runner VM.
+2. Validate non-interactive SSH from `arch` context to runner VM.
+3. Capture heartbeat evidence (`hostname`, `whoami`, `uptime`, disk).
+
+### Acceptance
+
+- Non-interactive SSH to runner VM succeeds from `arch`.
+- Failed SSH attempts return clear non-zero diagnostics.
+
+## WS6: Runner Codex authentication (operator-assisted)
+
+### Outcome
+
+Runner VM Codex runtime is authenticated and ready for delegated tasks.
+
+### Tasks
+
+1. Execute Codex auth bootstrap on runner VM.
+2. Request operator assistance for interactive auth step when needed.
+3. Record completion evidence and residual access risks.
+
+### Acceptance
+
+- Runner Codex auth is confirmed and timestamped.
+- Operator-assist step is logged when used.
+
+## WS7: Runner agent bootstrap
+
+### Outcome
+
+`runner` has its own repository with AGENT contract and reporting
+format, ready to accept delegated CI/test tasks.
+
+### Tasks
+
+1. Initialize runner repository and baseline docs.
+2. Add `AGENTS.md`, workstream board, and handoff log.
+3. Define runner reporting contract back to `arch`.
+
+### Acceptance
+
+- Runner repo and governance docs exist.
+- A smoke report from runner to arch is captured.
+
+## WS8: Multi-HUT hardware CI topology
+
+### Outcome
+
+Runner-based CI can schedule and run hardware jobs across multiple
+attached boards in parallel with deterministic ownership.
+
+### Lane Boundary Rules
+
+1. Hardware job definitions stay out of mandatory upstream checks.
+2. Upstream jobs must succeed without any hardware runner availability.
+3. Hardware jobs are triggered from local lane and mapped back to same
+   commit/branch context.
+
+### Topology (recommended)
+
+1. Runner VM runs orchestration:
+   - GitHub self-hosted runners (or local runner service)
+   - artifact store path
+   - health checks
+2. One runner label per HUT slot, e.g.:
+   - `esp32-hut-01`
+   - `esp32-hut-02`
+3. Each HUT slot has stable serial identity:
+   - `/dev/serial/by-id/...`
+4. Job routing uses labels and lock files to prevent slot collision.
+
+### Acceptance
+
+- At least two HUT slots can run independent jobs without resource
+  conflict.
+
+### Required Runner Artifacts (WS8 completion evidence)
+
+Produced in runner repository (by `runner:codex` on the runner VM):
+- topology documentation describing labels + lock strategy
+- per-slot lock wrapper (`flock`) and a self-test proving:
+  - same-slot operations serialize
+  - different-slot operations can run concurrently
+- slot inventory evidence capturing `/dev/serial/by-id` visibility
+- a local (ignored) slot mapping config with at least two slots defined
+  (`hut-01`, `hut-02`) mapping to stable `/dev/serial/by-id/...`
+
+### Current Status (WS8)
+
+- Runner VM: two stable `/dev/serial/by-id/*` devices are visible.
+  - Evidence (runner repo): `runner_hil_topo_inventory_20260213_215705_utc.log`
+  - Evidence (runner repo): `runner_hil_topo_lock_selftest_20260213_215839_utc.log`
+- GitLab push gate: runner repo closeout commits exist locally
+  (`runner/runner-agent@2ebd4da`, `runner/runner-agent@c03a9a0`) but are
+  not yet pushed due to HTTP auth failure (runner PAT needs refresh with
+  `write_repository`).
+
+## WS9: Hard power-cycle via Home Assistant relay
+
+### Outcome
+
+Power control is scripted and lock-protected so recovery sequences are
+deterministic and safe.
+
+### Service Contract
+
+1. Inputs:
+   - relay entity id
+   - slot id
+   - on/off durations
+2. Safety:
+   - per-slot lock (`/var/lock/hut-<id>.lock`)
+   - max retry count
+   - cooldown interval
+3. Output:
+   - structured result (`ok`, `timeout`, `ha_error`)
+
+### Acceptance
+
+- CI job can request power-cycle for a slot and receive deterministic
+  status.
+- Concurrent power-cycle requests for the same slot are serialized.
+
+## WS10: Branch/agent parallelism model
+
+### Outcome
+
+High parallel throughput with low conflict/overhead across agents.
+
+### Recommended Pattern
+
+1. Keep GitHub as source of truth (no mandatory local git server).
+2. Use local bare mirror/cache only as accelerator (optional).
+3. Spawn one worktree per agent/task:
+   - `~/workspace/wt/<task-id>`
+4. Run isolated build/test output folders per worktree.
+5. Standardize artifact naming by task id and commit hash.
+
+### Seamless Lane Handoff
+
+1. Upstream lane publishes build artifacts keyed by commit SHA.
+2. Local lane pulls/uses artifact for that same SHA when possible.
+3. Hardware result summary references:
+   - commit SHA
+   - lane (`upstream` or `local-hil`)
+   - HUT slot id
+   - power-cycle count/recovery events
+
+### Acceptance
+
+- Multiple agents can run concurrently without workspace contamination.
+
+## Achievable "Dream" Environment (pragmatic target)
+
+## Base VM image
+
+- Ubuntu 24.04 LTS (LXD VM, not container, for better device/systemd behavior)
+
+## Core dependencies
+
+1. `git`, `openssh-client`, `curl`, `jq`, `python3`, `pip`
+2. Docker engine + buildx + compose plugin
+3. ESP-IDF toolchain usage via project container (`espressif/idf:release-v5.5` derivative)
+4. Optional observability:
+   - `tmux`, `htop`, `nvme-cli`/disk monitors
+
+## CI/Cd strategy
+
+1. Primary CI remains GitHub-hosted + self-hosted hardware jobs.
+2. Optional local preflight pipeline on LXD for fast feedback before push.
+3. Trigger model:
+   - on push to feature branches: local preflight + optional hardware smoke
+   - on PR: gated hardware jobs by label/comment trigger
+
+## Why no mandatory local git server?
+
+Local Git service adds operational overhead and split-source risk.
+Prefer:
+1. GitHub canonical origin
+2. optional local mirror for speed/caching only
+
+## Long-run autonomy target for role-based Codex
+
+1. `infra` has:
+   - persistent auth
+   - host/VM bootstrap scripts
+   - health checks
+   - clearly scoped sudo permissions
+2. `runner` has:
+   - persistent auth
+   - CI/CD task runner scripts
+   - artifact/report emission to `arch`
+3. `arch` can delegate operations to `infra`/`runner` and receive
+   structured reports without direct host-manual intervention.
+4. Human operator only needed for:
+   - policy decisions
+   - hardware maintenance
+   - credential rotation
+
+## Deliverables Checklist
+
+- [x] SSH passthrough and remote command wrapper verified
+- [x] Multi-agent contract documented and smoke-tested
+- [x] Infrastructure agent repository + governance bootstrap completed
+- [ ] Runner VM provisioned with reproducible baseline
+- [ ] Architecture-to-runner SSH connectivity verified
+- [ ] Runner Codex authentication completed (operator-assisted)
+- [ ] Runner agent repository + governance bootstrap completed
+- [ ] Multi-HUT runner topology implemented with label routing
+- [ ] Home Assistant relay power-cycle lock and retry logic implemented
+- [ ] Parallel worktree branch workflow documented and operational
+- [ ] Closure notes moved to archive when complete

documentation/short-term/active/GOAL-002-hut-surface-first-test.md

@@ -0,0 +1,106 @@
+# GOAL-002: Surface Every HUT with First Hardware Test
+
+Status: blocked  
+Owner: unassigned  
+Last updated: 2026-02-12
+
+## Objective
+
+Use the first implemented hardware test (`HW-BOOT-001`) as a surfacing pass
+across every hardware-under-test (HUT) slot on the target system.
+
+Reference source:
+
+- `documentation/agents/integration_test_worklist.md`
+
+## Why This Goal Exists
+
+- `HW-BOOT-001` is already defined and partially validated (build path).
+- HIL evidence is still incomplete.
+- Running it across all slots establishes a baseline for real hardware coverage.
+
+## Scope
+
+In scope:
+
+- local LXD hardware lane execution only
+- per-slot cold-boot to operational-state validation
+- standardized evidence capture for each slot
+
+Out of scope:
+
+- new test definitions beyond `HW-BOOT-001`
+- upstream GitHub lane changes requiring hardware
+
+## Inputs
+
+1. HUT slot inventory and stable serial mappings.
+2. Build artifact or deterministic same-SHA rebuild.
+3. Slot-level flash/monitor command contract.
+
+## Workstreams
+
+## WS1: Slot Inventory and Identity
+
+1. Assign one stable slot id per board (`hut-01`, `hut-02`, ...).
+2. Record serial path (`/dev/serial/by-id/...`) per slot.
+3. Record target type/platform profile per slot.
+
+Acceptance:
+
+- all active boards are mapped to stable slot ids and serial identities.
+
+WS1 execution snapshot (2026-02-12):
+
+- Commit: `bf2fff44`
+- Evidence: `test/build/log/hut_slot_inventory_20260212.log`
+- Result: current workspace has no visible `/dev/serial/by-id`, `ttyUSB*`, or `ttyACM*` devices; slot mapping must be collected on the LXD HIL host.
+
+| Slot ID | Serial Path | Platform Profile | Status | Notes |
+|---|---|---|---|---|
+| pending | pending | pending | blocked | Inventory capture must run on hardware-attached target host |
+
+## WS2: HW-BOOT-001 Execution Harness
+
+1. Define one canonical command sequence for:
+   - flash
+   - reboot (or hard power-cycle where required)
+   - boot-monitor evidence capture
+2. Use one log naming standard:
+   - `<sha>_<slot>_HW-BOOT-001_<timestamp>.log`
+
+Acceptance:
+
+- a single command path can run HW-BOOT-001 on any mapped slot.
+
+## WS3: Per-Slot Surfacing Runs
+
+1. Execute `HW-BOOT-001` on each mapped slot.
+2. Capture result and log path per slot.
+3. Mark slot as `pass`, `fail`, or `blocked` with reason.
+
+Acceptance:
+
+- every mapped slot has one recorded outcome and evidence pointer.
+
+## WS4: Tracker and Matrix Sync
+
+1. Update `HW-BOOT-001` row evidence in:
+   - `documentation/agents/integration_test_worklist.md`
+2. Update slot coverage status in:
+   - `documentation/HARDWARE_TEST_MATRIX.md`
+
+Acceptance:
+
+- worklist and hardware matrix show the same slot-level reality.
+
+## Exit Criteria
+
+1. Every mapped slot has one HW-BOOT-001 outcome with evidence.
+2. Remaining blockers are explicit, actionable, and assigned.
+3. Next hardware item is named for follow-up execution.
+
+## Park/Resume Note
+
+- GOAL-002 kickoff was accidental and is intentionally parked.
+- Resume condition: GOAL-001 is implemented and the LXD hardware backend is available.

documentation/short-term/active/GOAL-003-agent-doc-lint-ci.md

@@ -0,0 +1,91 @@
+# GOAL-003: Agent-Driven Documentation Linting in CI/CD
+
+Status: active  
+Owner: unassigned  
+Last updated: 2026-02-12
+
+## Objective
+
+Make documentation linting reliable and low-overhead by combining:
+
+1. upstream merge-gating markdown lint checks, and
+2. local LXD agent-driven document gardening loops.
+
+Reference docs:
+
+- `documentation/agents/ci_lane_contract.md`
+- `documentation/agents/document_gardening.md`
+- `documentation/agents/start_here.md`
+
+## Scope
+
+In scope:
+
+- markdown lint contract and ownership by lane
+- agent run instructions and evidence format
+- progressive discovery pointers for docs work
+
+Out of scope:
+
+- product/firmware behavior changes
+- non-doc CI jobs
+
+## Strategy
+
+1. Keep upstream lane authoritative for merge-gating doc lint.
+2. Use local lane for agent-powered doc cleanup and maintenance.
+3. Keep agent context small with route-based document discovery.
+
+## Workstreams
+
+## WS1: Upstream Doc Lint Gate
+
+1. Define lint target scope:
+   - `AGENTS.md`
+   - `documentation/**/*.md`
+2. Define failure policy:
+   - lint failure blocks merge when docs change.
+
+Acceptance:
+
+- lane contract documents required lint command and blocking policy.
+
+## WS2: Local Agent Gardening Loop
+
+1. Standardize gardening task contract (prompt template + commands).
+2. Require before/after lint evidence in session output.
+3. Require board/handoff updates when status changes.
+
+Acceptance:
+
+- agents can run one reusable gardening flow with consistent output.
+
+## WS3: Progressive Context Discovery
+
+1. Keep one `start_here` route document as first entry.
+2. Route agents to one task-specific doc at a time.
+3. Avoid large up-front context loads.
+
+Acceptance:
+
+- docs in `documentation/agents/` support stepwise discovery.
+
+## WS4: Reporting and Handoff Format
+
+1. Standardize lint report fields:
+   - scope
+   - command
+   - before errors
+   - after errors
+   - residual risk
+2. Record unresolved lint debt in short-term handoff when needed.
+
+Acceptance:
+
+- any agent can continue lint remediation from prior evidence.
+
+## Exit Criteria
+
+1. `documentation/agents` docs consistently point to progressive startup.
+2. Lint ownership is explicit per CI lane.
+3. Gardening sessions produce machine-usable and human-usable evidence.

documentation/short-term/active/GOAL_TEMPLATE.md

@@ -0,0 +1,124 @@
+# GOAL-XXX: <Short Goal Title>
+
+Status: active  
+Owner: arch:<agent-id|tbd>  
+Last updated: YYYY-MM-DD
+
+## Objective
+
+<One paragraph describing the intended outcome and constraints.>
+
+## Scope
+
+- In scope:
+  - <item>
+- Out of scope:
+  - <item>
+
+## Follow-On Goals (optional)
+
+- `documentation/short-term/active/GOAL-YYY-<slug>.md`
+
+## Workstreams
+
+| Workstream | Outcome | Owner | Status | Notes |
+|---|---|---|---|---|
+| WS1: <name> | <outcome> | arch:<agent-id|tbd> | pending | <notes> |
+| WS2: <name> | <outcome> | infra:<agent-id|tbd> | pending | <notes> |
+
+Status values: `pending`, `in_progress`, `blocked`, `done`.
+
+Owner format:
+- `<role>:<agent-id>`
+- roles: `arch`, `infra`, `runner`
+
+Role activation states:
+- `unassigned` -> `assigned` -> `first_heartbeat` -> `active`
+
+## Deliverables Checklist (Goal Completion Gate)
+
+- [ ] <deliverable 1 with objective evidence target>
+- [ ] <deliverable 2 with objective evidence target>
+- [ ] <deliverable 3 with objective evidence target>
+
+Goal completion rule:
+- A goal is complete only when every deliverable above is checked and
+  evidence is recorded in coordination artifacts.
+- Workstream `done` does not by itself mean goal complete.
+- Do not archive until all deliverables are checked.
+
+## Agent Governance
+
+- `arch` owns engineering coordination, status governance, and
+  delegation to remote agents; it should also propose role/process
+  improvements when asked or when major inefficiencies are evident.
+- governance docs should be enabling and operational, not restrictive.
+- `infra` owns host/container/VM maintenance and must never clone the
+  product repository.
+- `runner` owns VM CI/CD and test execution.
+- `infra` must not directly communicate with `runner` except when only
+  local infra commands can reach runner execution context.
+- each remote role (`infra`, `runner`) requires its own dedicated
+  repository and AGENT artifacts.
+
+## Delegation Checklist (Required For Remote Roles)
+
+- [ ] Owner assigned (`infra:<id>` or `runner:<id>`, not `tbd`)
+- [ ] Role activation state is at least `assigned`
+- [ ] First role-context handoff entry logged
+- [ ] Operator gate decision recorded (`operator_required=yes|no`)
+- [ ] Execution mechanism recorded (`executor=remote_codex|ssh_direct`)
+- [ ] Commit hygiene plan stated (expected commit cadence + push points)
+
+## Course-Correction Checklist (Required For Reprioritization)
+
+Use this when scope/sequence changes even without hard prerequisites.
+
+- [ ] Replan request captured in handoff (`action_type=replan`)
+- [ ] Impacted workstreams restated (`blocked`/`pending` updates)
+- [ ] New intermediate quest workstream(s) added where useful
+- [ ] Dependencies and next actions updated in board
+- [ ] Deliverables checklist still matches revised path
+
+## Ad-hoc Request Intake
+
+- Ask user: `Track this as a ticket? (yes/no)`
+- If `yes`: create/update ticket in
+  `documentation/short-term/coordination/ad_hoc_ticket_queue.md`
+- If `no`: proceed and log `ticket_tracking=declined` in handoff summary
+
+## Acceptance By Workstream
+
+### WS1: <name>
+
+Tasks:
+1. <task>
+2. <task>
+
+Acceptance:
+- <acceptance criterion>
+
+### WS2: <name>
+
+Tasks:
+1. <task>
+2. <task>
+
+Acceptance:
+- <acceptance criterion>
+
+## Evidence
+
+- Commands:
+  - `<command>`
+- Logs/artifacts:
+  - `<path>` (prefix with `arch_`, `infra_`, or `runner_`)
+- Related coordination updates:
+  - `documentation/short-term/coordination/workstream_board.md`
+  - `documentation/short-term/coordination/handoff_log.md`
+
+## Handoff Entry Format
+
+```text
+YYYY-MM-DD HH:MM UTC | agent/session | goal/workstream | context | action_type | operator_required | summary | next action | blocker (optional)
+```

documentation/short-term/archive/README.md

@@ -0,0 +1,12 @@
+# Archive
+
+Move completed short-term goal documents from `active/` to this folder.
+
+Recommended naming:
+- `YYYY-MM-DD-GOAL-<id>-<slug>.md`
+
+Keep only a short closure summary in the active board after archival.
+
+Archive gate:
+- Only archive goals whose deliverables checklist is fully checked.
+- If any deliverable remains unchecked, keep the goal in `active/`.

documentation/short-term/coordination/ad_hoc_ticket_queue.md

@@ -0,0 +1,33 @@
+# Ad-hoc Ticket Queue
+
+Use this queue when user/operator approves ticket tracking for ad-hoc
+requests.
+
+## Ticket ID Format
+
+- `ADHOC-YYYYMMDD-<NN>`
+- example: `ADHOC-20260212-01`
+
+## Status Values
+
+- `open`
+- `in_progress`
+- `blocked`
+- `done`
+- `declined` (used only when user explicitly declines tracking after
+  suggestion)
+
+## Queue
+
+| Ticket ID | Requested UTC | Requested By | Scope Summary | Linked Goal/WS | Owner | Status | Next Action | Evidence |
+|---|---|---|---|---|---|---|---|---|
+| ADHOC-20260212-01 | 2026-02-12 18:30 UTC | operator | Establish ad-hoc ticket reflex and replan handling protocol | GOAL-001/governance | arch:codex | done | monitor adoption in future sessions | `documentation/short-term/README.md`, `documentation/short-term/coordination/workstream_board.md`, `documentation/short-term/coordination/handoff_log.md` |
+| ADHOC-20260212-02 | 2026-02-12 20:04 UTC | operator | Standardize GitLab credential home + secure propagation contract for future agent bootstrap | GOAL-001/WS3-WS7 | arch:codex | done | execute ADHOC-20260212-03 remote/repo bootstrap against standardized credential contract | `AGENTS.md`, `documentation/short-term/README.md`, `documentation/short-term/active/GOAL-001-lxd-codex-hardware-ci.md`, `documentation/short-term/coordination/handoff_log.md`, `/workspaces/codex-infra-agent/AGENTS.md` |
+| ADHOC-20260212-03 | 2026-02-12 20:04 UTC | operator | Push new Codex agent repos to GitLab and define bootstrap checkout flow against those remotes | GOAL-001/WS3-WS7 | arch:codex | done | proceed with GOAL-001 role assignment/delegation gates for WS4+ using published remotes and bootstrap contract | `documentation/short-term/README.md`, `documentation/short-term/active/GOAL-001-lxd-codex-hardware-ci.md`, `documentation/short-term/coordination/handoff_log.md`, `/workspaces/codex-infra-agent/README.md`, `/workspaces/codex-runner-agent/README.md` |
+
+## Usage Rules
+
+1. Agent should ask user if ad-hoc request should be tracked.
+2. If user says `yes`, create/update a row here before major execution.
+3. If user says `no`, do not create a ticket row; log
+   `ticket_tracking=declined` in handoff instead.

documentation/short-term/coordination/delegations/GOAL-001_WS8_blocker_runner_push_retry_danger_packet.md

@@ -0,0 +1,51 @@
+# GOAL-001 / WS8 Blocker Packet: Push Retry (danger-full-access) (runner:codex)
+
+Purpose: retry pushing the runner repo local WS8 commits to origin from
+the runner VM **outside the restricted sandbox**.
+
+Notes:
+- Previous attempts ran under a restricted execution environment where:
+  - `sudo` was blocked by `no new privileges`
+  - network reachability tests failed
+- This retry must be executed with `codex exec --sandbox danger-full-access`
+  so the agent can access the VM’s real network and privileges.
+
+## Tasks
+
+Completion rule:
+- this blocker is not cleared until the runner repo changes are pushed
+  to `origin/main` and the pushed commit SHA is reported.
+
+1. Confirm network + DNS from the runner VM (evidence):
+   - `getent hosts git.lecsys.net || true`
+   - `ping -c 1 -W 1 git.lecsys.net || true`
+   - `curl -fsS http://git.lecsys.net/ >/dev/null && echo ok_http || echo fail_http`
+   - `git ls-remote http://git.lecsys.net/runner/runner-agent.git HEAD`
+2. Reconcile with `origin/main` before pushing (avoid unsafe force):
+   - `git -C /home/runner/workspaces/codex-runner-agent fetch origin`
+   - `git -C /home/runner/workspaces/codex-runner-agent status --porcelain`
+   - `git -C /home/runner/workspaces/codex-runner-agent log --oneline -n 15`
+   - `git -C /home/runner/workspaces/codex-runner-agent log --oneline --decorate origin/main..HEAD || true`
+   - `git -C /home/runner/workspaces/codex-runner-agent log --oneline --decorate HEAD..origin/main || true`
+   - If `HEAD..origin/main` is non-empty, prefer `git pull --rebase` and
+     re-run the checks above.
+   - Do not use `--force` unless explicitly required and safe; if it is
+     required, prefer `--force-with-lease` and record the justification
+     + before/after SHAs in evidence.
+3. Push all local commits on `main`:
+   - `git -C /home/runner/workspaces/codex-runner-agent status --porcelain`
+   - `git -C /home/runner/workspaces/codex-runner-agent log --oneline -n 10`
+   - `git -C /home/runner/workspaces/codex-runner-agent push origin main`
+   - if non-interactive auth is required, use the role PAT contract
+     (`runner.env`) and a PAT-backed push method; do not paste secrets
+     into logs
+4. Evidence + coordination:
+   - write `documentation/evidence/runner_ws8_push_retry_<timestamp>.log`
+     capturing the above command outputs and push result
+   - append a runner handoff line with:
+     - `context=runner-live`, `action_type=execute`
+     - pushed commit SHA range and evidence path
+
+## Completion
+
+- `git push origin main` succeeds and commits appear on `origin/main`

documentation/short-term/coordination/delegations/GOAL-001_WS8_blocker_runner_push_unblock_packet.md

@@ -0,0 +1,54 @@
+# GOAL-001 / WS8 Blocker Packet: Unblock Runner Push (runner:codex)
+
+Objective: unblock `git push origin main` from the runner VM so WS8
+topology work can be pushed and reported by commit SHA.
+
+This is not a separate workstream from WS8; it is a WS8 blocker-clearing
+packet.
+
+Context (from arch side):
+- `git.lecsys.net` resolves to `192.168.10.75` from the arch environment
+- runner VM error was: `Could not resolve host: git.lecsys.net`
+
+## Tasks (runner VM)
+
+Completion rule:
+- this blocker is not cleared until the runner repo changes are pushed
+  to `origin/main` and the pushed commit SHA is reported.
+
+1. Capture DNS diagnostics (evidence):
+   - `getent hosts git.lecsys.net || true`
+   - `cat /etc/resolv.conf || true`
+   - `ip -4 addr && ip route`
+2. Attempt to reach GitLab by IP (evidence):
+   - `curl -fsS http://192.168.10.75/ >/dev/null && echo ok_ip_http || echo fail_ip_http`
+3. Fix name resolution, preferring least-invasive:
+   - If `getent hosts git.lecsys.net` fails, try adding a host entry:
+     - idempotent edit: only add if missing
+     - line: `192.168.10.75 git.lecsys.net`
+     - command suggestion:
+       - `sudo sh -c 'grep -q \"^192\\.168\\.10\\.75[[:space:]]\\+git\\.lecsys\\.net$\" /etc/hosts || echo \"192.168.10.75 git.lecsys.net\" >> /etc/hosts'`
+   - If `sudo` is unavailable/non-functional:
+     - stop and report blocker `operator_required=yes` and recommend
+       delegating to `infra` to fix runner VM DNS.
+4. Verify name resolution works:
+   - `getent hosts git.lecsys.net`
+   - `curl -fsS http://git.lecsys.net/ >/dev/null && echo ok_name_http || echo fail_name_http`
+5. Push runner repo commits:
+   - `git -C /home/runner/workspaces/codex-runner-agent status --porcelain`
+   - `git -C /home/runner/workspaces/codex-runner-agent log --oneline -n 5`
+   - `git -C /home/runner/workspaces/codex-runner-agent push origin main`
+   - if non-interactive auth is required, use the role PAT contract
+     (`runner.env`) and a PAT-backed push method; do not paste secrets
+     into logs
+6. Evidence + coordination:
+   - write one evidence log capturing key outputs:
+     - `documentation/evidence/runner_ws8_push_unblock_<timestamp>.log`
+   - append runner handoff line:
+     - `context=runner-live`, `action_type=execute`
+     - include pushed commit SHA(s) and the evidence path
+
+## Completion Criteria
+
+- `git push origin main` succeeds from runner VM
+- runner handoff log includes the push success and evidence path

documentation/short-term/coordination/delegations/GOAL-001_WS8_infra_blockers_packet.md

@@ -0,0 +1,52 @@
+# GOAL-001 / WS8 Infra Blockers Packet (infra:codex)
+
+Goal: unblock `GOAL-001/WS8` by fixing infra-scope dependencies for the
+runner VM:
+
+- pass through at least 2 real USB serial devices so runner sees stable
+  `/dev/serial/by-id/*` identities
+- ensure runner VM can resolve and reach `git.lecsys.net` (DNS/routing)
+
+This is a blocker-clearing packet for WS8, not a separate workstream.
+
+Hard rules:
+- execute only in infra scope (host/LXD/VM/networking). Do not clone or
+  edit the product repo (`squeezelite-esp32`).
+- prefer least-invasive, reversible fixes; capture evidence.
+
+## Deliverables (infra repo)
+
+1. Evidence log (commit this) capturing:
+   - current runner VM network/DNS status relevant to `git.lecsys.net`
+   - USB passthrough configuration and the resulting `/dev/serial/by-id`
+     view inside the runner VM
+2. A short infra handoff entry (`context=infra-live`) summarizing:
+   - what changed
+   - whether runner VM can now push to GitLab
+   - whether runner VM now sees >=2 `/dev/serial/by-id/*` devices
+
+## Tasks (infra)
+
+1. Runner VM reachability for GitLab (evidence):
+   - verify runner VM can resolve and reach `git.lecsys.net`
+     (DNS + routing).
+   - if name resolution is broken, fix it at the correct layer
+     (VM DNS config, LXD network config, host resolver/DHCP), preferring
+     reversible changes.
+2. USB passthrough (evidence):
+   - identify at least 2 target USB serial devices on the host
+   - configure passthrough into the runner VM
+   - verify inside the runner VM that `/dev/serial/by-id/*` contains
+     stable identities for both devices
+
+## Evidence naming
+
+- `documentation/evidence/infra_ws8_blockers_<timestamp>.log`
+
+## Completion criteria
+
+- runner VM can resolve/reach `git.lecsys.net` (or infra reports the
+  remaining hard blocker and whether `operator_required=yes`)
+- runner VM sees >=2 serial devices with stable `/dev/serial/by-id/*`
+  identities
+

documentation/short-term/coordination/delegations/GOAL-001_WS8_runner_closeout_continue_packet.md

@@ -0,0 +1,39 @@
+# GOAL-001 / WS8 Runner Closeout Continue Packet (runner:codex)
+
+thread_ref=WS8-CLOSE-20260213A
+
+Use this only if the main closeout packet was interrupted mid-run.
+
+Goal: finish the remaining closeout steps, commit evidence/coordination,
+and push to `origin/main` using a PAT-backed non-interactive method.
+
+Hard rules:
+- execute inside runner VM repo only:
+  - `/home/runner/workspaces/codex-runner-agent`
+- do not commit `config/hut_slots.json`
+
+## Tasks
+
+1. Validate slot resolution:
+   - `p1=$(scripts/hut_slot_resolve.sh hut-01); test -e "$p1"`
+   - `p2=$(scripts/hut_slot_resolve.sh hut-02); test -e "$p2"`
+   - record `hut-01` and `hut-02` by-id strings in the handoff summary
+2. Lock selftest evidence (commit this):
+   - `scripts/hut_lock_selftest.sh | tee documentation/evidence/runner_hil_topo_lock_selftest_<timestamp>.log`
+3. Update runner coordination:
+   - set `HIL-TOPO-001` to `done` with evidence paths
+   - fix stale WS8/push row (credentials are now configured and pushes work)
+   - append runner handoff log entry:
+     - `context=runner-live`, `action_type=execute`, `operator_required=no`
+     - `thread_ref=...`
+     - evidence paths
+     - next action: `POWER-CTL-001`
+4. Commit + push:
+   - stage evidence + coordination only
+   - commit (small message)
+   - push via `/home/runner/.local/bin/gitlab_push_origin_main.sh`
+5. Report back:
+   - pushed commit SHA
+   - evidence paths
+   - selected hut-01/hut-02 by-id identities
+

documentation/short-term/coordination/delegations/GOAL-001_WS8_runner_closeout_packet.md

@@ -0,0 +1,60 @@
+# GOAL-001 / WS8 Runner Closeout Packet (runner:codex)
+
+thread_ref=WS8-CLOSE-20260213A
+
+Goal: bring `GOAL-001/WS8` to completion by finishing runner workstream
+`HIL-TOPO-001` with real slot identities (>=2 `/dev/serial/by-id/*`),
+updating runner coordination, and pushing the resulting commit(s) to
+`origin/main`.
+
+Hard rules:
+- execute inside runner VM repo only:
+  - `/home/runner/workspaces/codex-runner-agent`
+- do not commit local slot mapping file:
+  - `config/hut_slots.json` must remain ignored
+- completion requires:
+  - evidence committed, coordination updated, and a pushed SHA on
+    `origin/main` (local-only progress does not count)
+
+## Tasks
+
+1. Verify device count:
+   - list `/dev/serial/by-id/*`
+   - if fewer than 2 entries exist: stop and report `blocked` with
+     `by_id_count=<n>` and the exact missing dependency.
+2. Inventory evidence (commit this):
+   - run:
+     - `scripts/hut_inventory.sh | tee documentation/evidence/runner_hil_topo_inventory_<timestamp>.log`
+3. Slot mapping (local-only, do not commit):
+   - update `config/hut_slots.json` to map:
+     - `hut-01` -> one stable `/dev/serial/by-id/*`
+     - `hut-02` -> a different stable `/dev/serial/by-id/*`
+   - validate:
+     - `scripts/hut_slot_resolve.sh hut-01` resolves and target exists
+     - `scripts/hut_slot_resolve.sh hut-02` resolves and target exists
+4. Lock evidence (commit this):
+   - run:
+     - `scripts/hut_lock_selftest.sh | tee documentation/evidence/runner_hil_topo_lock_selftest_<timestamp>.log`
+5. Runner coordination:
+   - update `documentation/coordination/workstream_board.md`:
+     - set `HIL-TOPO-001` to `done` once the above is satisfied
+     - ensure any stale WS8/push rows reflect current state (push is no
+       longer blocked if PAT is configured)
+   - append `documentation/coordination/handoff_log.md` entry with:
+     - `context=runner-live`, `action_type=execute`, `operator_required=no`
+     - `thread_ref=...`
+     - evidence paths
+     - pushed SHA (after push)
+     - next action: `POWER-CTL-001`
+6. Commit + push:
+   - stage only tracked changes (evidence + coordination), not
+     `config/hut_slots.json`
+   - commit with a small message (WS8 closeout)
+   - push to `origin/main` using a PAT-backed non-interactive method
+     (do not paste secrets into logs). If available, use:
+     - `/home/runner/.local/bin/gitlab_push_origin_main.sh`
+7. Report back to arch:
+   - pushed commit SHA
+   - evidence paths
+   - by-id identities chosen for `hut-01` and `hut-02`
+

documentation/short-term/coordination/delegations/GOAL-001_WS8_runner_handshake_push_packet.md

@@ -0,0 +1,59 @@
+# GOAL-001 / WS8 Runner Handshake + Push Packet (runner:codex)
+
+thread_ref=WS8-PUSH-20260213A
+
+Goal: drive WS8 to a pushed state (commit SHA visible on `origin/main`)
+and report back with evidence. This packet is safe to run repeatedly.
+
+Hard rules:
+- execute inside runner VM repo only:
+  - `/home/runner/workspaces/codex-runner-agent`
+- do not leak secrets into logs, repo files, or handoff summaries
+- completion requires a pushed SHA on `origin/main` (not just local commits)
+
+## Tasks
+
+1. Handshake (record in runner handoff summary):
+   - `thread_ref=...`
+   - current `HEAD` SHA and branch
+   - what you believe is the best next step and why
+2. Collect non-secret diagnostics (evidence log):
+   - `getent hosts git.lecsys.net || true`
+   - `curl -fsS http://git.lecsys.net/ >/dev/null && echo ok_http || echo fail_http`
+   - `git -C /home/runner/workspaces/codex-runner-agent remote -v`
+   - `git -C /home/runner/workspaces/codex-runner-agent status --porcelain`
+   - `git -C /home/runner/workspaces/codex-runner-agent fetch origin`
+   - `git -C /home/runner/workspaces/codex-runner-agent log --oneline --decorate -n 10`
+   - `git -C /home/runner/workspaces/codex-runner-agent log --oneline --decorate origin/main..HEAD | head -n 50 || true`
+3. Credential gate (do not print secrets):
+   - expected credential file:
+     - `${XDG_CONFIG_HOME:-$HOME/.config}/codex/credentials/gitlab/git.lecsys.net/runner.env`
+   - if missing or `GITLAB_USER`/`GITLAB_PAT` is empty:
+     - create the directory + a template file with correct perms
+       (`0700` dir, `0600` file)
+     - stop and report `operator_required=yes` with the exact path that
+       needs to be filled
+4. Push:
+   - once `runner.env` has non-empty `GITLAB_USER` and `GITLAB_PAT`,
+     push non-interactively without storing credentials in git config.
+   - recommended approach:
+     - use an ephemeral `http.extraheader` basic auth header built in
+       memory and run:
+       - (optional) `git -c http.extraheader="Authorization: Basic <...>" fetch origin`
+       - `git push origin main`
+   - if available, you may use an approved local helper script that
+     performs a PAT-backed push without persisting credentials in git
+     config (for example `/home/runner/.local/bin/gitlab_push_origin_main.sh`)
+5. Evidence + coordination:
+   - write:
+     - `documentation/evidence/runner_ws8_push_handshake_<timestamp>.log`
+   - append a runner handoff log line with:
+     - `context=runner-live`, `action_type=execute`
+     - `thread_ref=...`
+     - pushed SHA(s) (or remaining blocker)
+     - evidence path
+
+## Completion Criteria
+
+- `origin/main` contains the WS8 commits (push succeeded), and runner
+  handoff includes pushed SHA(s) + evidence path.

documentation/short-term/coordination/delegations/GOAL-001_WS8_runner_inventory_refresh_packet.md

@@ -0,0 +1,50 @@
+# GOAL-001 / WS8 Runner Packet: Inventory Refresh After USB Passthrough (runner:codex)
+
+thread_ref=WS8-USB-INV-20260213A
+
+Goal: after infra USB passthrough changes, refresh WS8 inventory evidence
+and report whether runner now has >=2 stable `/dev/serial/by-id/*`
+devices for `hut-01` + `hut-02`.
+
+Hard rules:
+- execute inside runner VM repo only:
+  - `/home/runner/workspaces/codex-runner-agent`
+- do not commit local slot mapping file:
+  - `config/hut_slots.json` must remain ignored
+- commit hygiene: if any tracked files are changed (evidence/coordination),
+  commit and push them to `origin/main` from the runner VM and report the
+  pushed SHA; local-only progress does not count.
+
+## Tasks
+
+1. Inventory evidence:
+   - run `scripts/hut_inventory.sh` and capture a new evidence log:
+     - `documentation/evidence/runner_hil_topo_inventory_<timestamp>.log`
+2. Slot mapping status:
+   - if there are >=2 `/dev/serial/by-id/*` entries:
+     - populate local `config/hut_slots.json` with `hut-01` + `hut-02`
+       using stable by-id paths
+     - validate:
+       - `scripts/hut_slot_resolve.sh hut-01`
+       - `scripts/hut_slot_resolve.sh hut-02`
+   - else (only 0 or 1 device visible):
+     - do not fake identities
+     - record explicit blocker: "need >=2 serial devices passed through"
+3. Coordination:
+   - update runner `documentation/coordination/workstream_board.md`:
+     - keep `HIL-TOPO-001` status `blocked` until >=2 devices exist
+     - reference the new inventory evidence log
+   - append runner `documentation/coordination/handoff_log.md` entry with:
+     - `context=runner-live`, `action_type=execute`, `operator_required=no`
+     - `thread_ref=...`
+     - inventory evidence path
+     - current device count observed
+     - next action (infra/operator attach + pass through a 2nd device if missing)
+4. Commit + push:
+   - commit and push the new evidence + coordination updates (not `hut_slots.json`)
+   - report pushed commit SHA
+
+## Completion
+
+- new inventory evidence is committed+pushed, and runner coordination reflects
+  current device count and blocker/next action accurately.

documentation/short-term/coordination/delegations/GOAL-001_WS8_runner_packet.md

@@ -0,0 +1,65 @@
+# GOAL-001 / WS8 Delegation Packet (runner:codex)
+
+Scope: implement `GOAL-001/WS8` by completing runner workstream
+`HIL-TOPO-001` (multi-HUT topology and slot mapping).
+
+Hard rule: execute inside the runner VM runner repo only
+(`/home/runner/workspaces/codex-runner-agent`). Do not rely on any local
+`/workspaces/codex-runner-agent` copy on the dev machine.
+
+Completion rule (commit hygiene):
+- WS8 is not complete until changes are committed and pushed to
+  `origin/main` from the runner VM, and the pushed commit SHA is
+  reported back. Local-only progress does not count.
+
+## Deliverables (runner repo)
+
+1. Slot mapping schema + example:
+   - `config/hut_slots.example.json`
+   - `.gitignore` that ignores `config/hut_slots.json`
+2. Inventory + resolver:
+   - `scripts/hut_inventory.sh` (captures `/dev/serial/by-id`)
+   - `scripts/hut_slot_resolve.sh hut-XX` (reads `config/hut_slots.json`)
+3. Per-slot locking:
+   - `scripts/hut_lock_exec.sh hut-XX -- <cmd...>` using `flock`
+   - `scripts/hut_lock_selftest.sh` proving:
+     - same-slot serializes
+     - different-slots can run concurrently
+4. Topology doc:
+   - `documentation/hil_topology.md`
+5. Evidence logs (commit these):
+   - `documentation/evidence/runner_hil_topo_inventory_<timestamp>.log`
+   - `documentation/evidence/runner_hil_topo_lock_selftest_<timestamp>.log`
+6. Local config (do not commit):
+   - `config/hut_slots.json` with at least `hut-01` + `hut-02` mapping to
+     stable `/dev/serial/by-id/...` symlinks.
+
+## Execution Checklist (runner VM)
+
+1. Confirm repo is clean:
+   - `git -C /home/runner/workspaces/codex-runner-agent status --porcelain`
+2. Implement deliverables above; ensure scripts are executable.
+3. Run:
+   - `scripts/hut_inventory.sh | tee documentation/evidence/runner_hil_topo_inventory_<timestamp>.log`
+   - `scripts/hut_lock_selftest.sh | tee documentation/evidence/runner_hil_topo_lock_selftest_<timestamp>.log`
+4. Populate local slot mapping (ignored file):
+   - copy `config/hut_slots.example.json` -> `config/hut_slots.json`
+   - replace `serial_by_id` with real paths from `/dev/serial/by-id`
+   - validate:
+     - `scripts/hut_slot_resolve.sh hut-01`
+     - `scripts/hut_slot_resolve.sh hut-02`
+5. Commit + push all tracked changes (do not commit `config/hut_slots.json`):
+   - include commit SHA in runner handoff entry
+6. Update runner coordination:
+   - `documentation/coordination/workstream_board.md`: set `HIL-TOPO-001`
+     to `done` with evidence paths
+   - `documentation/coordination/handoff_log.md`: append one line with:
+     - `context=runner-live`, `action_type=execute`, `operator_required=no`
+     - commit SHA(s), evidence paths, and next action (`POWER-CTL-001`)
+
+## Report Back (to arch)
+
+Return:
+- pushed commit SHA
+- evidence paths
+- summary of discovered slots (`hut-01`, `hut-02`) and their `/dev/serial/by-id` identities

documentation/short-term/coordination/delegations/GOAL-001_WS8_runner_pushfix_packet.md

@@ -0,0 +1,24 @@
+# GOAL-001 / WS8 Delegation Packet: Runner Push Fix (shim) (runner:codex)
+
+This file exists to resolve a filename collision/ambiguity in WS8
+delegation references.
+
+Use the purpose-specific WS8 packets instead:
+
+- Primary WS8 implementation:
+  - `GOAL-001_WS8_runner_packet.md`
+- WS8 push blocker (DNS/name resolution + push):
+  - `GOAL-001_WS8_blocker_runner_push_unblock_packet.md`
+- WS8 push retry requiring real VM network/privileges:
+  - `GOAL-001_WS8_blocker_runner_push_retry_danger_packet.md`
+
+If you are the runner role and you received this shim:
+
+1. Pick the correct packet above based on current blocker symptoms.
+2. Execute it inside the runner VM repo:
+   - `/home/runner/workspaces/codex-runner-agent`
+3. In the runner handoff summary, include:
+   - `thread_ref=<value if provided by arch>`
+   - pushed commit SHA(s) (or remaining blocker)
+   - evidence path(s)
+

documentation/short-term/coordination/delegations/README.md

@@ -0,0 +1,21 @@
+# Delegation Packets
+
+Packets in this folder are prompts meant to be fed to `codex exec -` on
+remote role machines (typically via SSH transport).
+
+Naming conventions:
+- `GOAL-001_WS8_runner_packet.md`: primary WS8 implementation packet
+- `GOAL-001_WS8_runner_pushfix_packet.md`: shim for a past filename
+  collision; points at the real WS8 push blocker packets
+- `GOAL-001_WS8_infra_blockers_packet.md`: infra-scope packet for WS8
+  blockers (runner VM DNS/routing, USB passthrough)
+- `GOAL-001_WS8_runner_handshake_push_packet.md`: runner packet to
+  handshake, verify credentials gate, and push WS8 commits with evidence
+- `GOAL-001_WS8_runner_inventory_refresh_packet.md`: runner packet to
+  refresh inventory evidence after USB passthrough and report device count
+- `GOAL-001_WS8_runner_closeout_packet.md`: runner packet to finish WS8
+  (inventory + slot mapping validation + lock selftest), commit, and push
+- `GOAL-001_WS8_runner_closeout_continue_packet.md`: runner packet to
+  continue closeout if an earlier closeout run was interrupted
+- `GOAL-001_WS8_blocker_*.md`: WS8 blocker-clearing packets (do not
+  represent separate workstreams)

documentation/short-term/coordination/handoff_log.md

@@ -0,0 +1,68 @@
+# Handoff Log
+
+Use this log for short handoffs between agents/sessions.
+
+Format:
+
+```text
+YYYY-MM-DD HH:MM UTC | agent/session | goal/workstream | context | action_type | operator_required | summary | next action | blocker (optional)
+```
+
+Field values:
+
+- `context`: `arch-local` | `infra-live` | `runner-live`
+- `action_type`: `scaffold` | `delegate` | `execute` | `replan`
+- `operator_required`: `yes` | `no`
+
+Ticket tracking convention:
+
+- If request is tracked: include `ticket_id=<id>` in summary.
+- If user declines tracking: include `ticket_tracking=declined` in
+  summary.
+
+Legacy note:
+- Older lines without these fields are valid historical entries.
+- New entries must include all fields.
+
+Entries:
+
+```text
+2026-02-13 22:04 UTC | arch:codex | GOAL-001/WS8 | arch-local | execute | no | Runner VM now has >=2 stable `/dev/serial/by-id/*` devices (inventory evidence `runner_hil_topo_inventory_20260213_215705_utc.log`) and lock selftest evidence `runner_hil_topo_lock_selftest_20260213_215839_utc.log`; runner repo `main` has new closeout commits (`2ebd4da`, `c03a9a0`) but push/fetch to `git.lecsys.net` is blocked by HTTP Basic access denied using current runner PAT (`runner.env`) | operator: refresh runner PAT with `write_repository` and update runner VM `runner.env`; runner: run `~/.local/bin/gitlab_push_origin_main.sh` and report pushed SHA; then mark WS8 `done` | DEP:valid runner GitLab PAT
+2026-02-13 21:13 UTC | arch:codex | GOAL-001/WS8 | arch-local | execute | no | Cleared infra/runner execution path: fixed infra credential perms on LXD host (codexsvc now in `lxd` group), enabled USB passthrough for the first CP210x device (vendorid=10c4 productid=ea60), restarted runner VM and installed Ubuntu kernel 6.14.0-36 on runner so `/dev/serial/by-id/*` maps to `/dev/ttyUSB0`; runner produced inventory evidence `runner_hil_topo_inventory_20260213_211129_utc.log` showing by_id_count=1 and pushed refresh commit `runner/runner-agent@ce16c27` | infra/operator: attach + pass through a 2nd serial device; runner: rerun inventory + complete hut-01/hut-02 local mapping + lock selftest evidence if needed; then mark WS8 done | DEP:2nd USB serial device availability
+2026-02-13 20:26 UTC | arch:codex | GOAL-001/WS8 | arch-local | execute | no | Runner GitLab credentials updated (runner.env now has non-empty PAT) and runner successfully pushed WS8 commits to GitLab: `runner/runner-agent` `main` advanced `6255908..bbad5d5` (push output captured on runner). WS8 remains blocked only on infra USB passthrough (runner VM still lacks `/dev/serial/by-id`) | infra/operator: pass through >=2 serial devices; runner: rerun inventory + complete local slot mapping and evidence | DEP:infra USB passthrough + DEP:operator LXD permissions (codexsvc not in lxd group)
+2026-02-13 14:20 UTC | arch:codex | GOAL-001/WS8 | arch-local | replan | yes | Security hygiene: operator pasted GitLab account passwords into a chat transcript; treat them as compromised. Do not use or re-share; rotate passwords and revoke/rotate tokens/sessions as appropriate. WS8 remains blocked on provisioning a runner GitLab PAT in `/home/runner/.config/codex/credentials/gitlab/git.lecsys.net/runner.env` (no secrets in logs) | operator: rotate leaked GitLab passwords; create runner PAT and populate runner.env; rerun WS8 push packet | DEP:operator credential rotation + PAT provisioning
+2026-02-13 14:12 UTC | arch:codex | GOAL-001/WS8 | arch-local | execute | yes | Took ownership of WS8 closure: removed local role-repo collisions (repos quarantined), added infra blocker packet, and executed runner remote-codex handshake/push gate; runner VM resolves `git.lecsys.net` and produced evidence `/home/runner/workspaces/codex-runner-agent/documentation/evidence/runner_ws8_push_handshake_20260213T141012Z.log`, but push remains blocked because `/home/runner/.config/codex/credentials/gitlab/git.lecsys.net/runner.env` exists with empty `GITLAB_USER`/`GITLAB_PAT`; runner VM still has no `/dev/serial/by-id` (USB passthrough not done). Local runner `main` is now ahead in range `33ef496`..`bbad5d5` | operator: fill runner PAT file (no secrets logged); operator/infra: enable USB passthrough (>=2 serial devices) to runner VM; then rerun `GOAL-001_WS8_runner_handshake_push_packet.md` on runner and `git push origin main` | DEP:operator credentials + DEP:infra USB passthrough (codexsvc not in lxd group)
+2026-02-13 13:51 UTC | arch:codex | governance/ownership | arch-local | replan | no | Refined closure contract: `arch:codex` may stop early only when a pivot is required or a problem-solving loop is stalled, and must first record `replan` (pivot) or explicit `blocked` (stall) with concrete next actions and required new information | apply this exception consistently without weakening closure accountability | none
+2026-02-13 13:48 UTC | arch:codex | governance/ownership | arch-local | replan | no | Locked ownership closure rule: when `arch:codex` is asked to take ownership, it must drive the task/workstream to successful closure (`done`) or explicit `blocked` with concrete blockers/owners/operator gate/next action; "prepared a packet" is not closure | apply closure rule to GOAL-001/WS8 and future remote-role workstreams | none
+2026-02-13 13:30 UTC | arch:codex | GOAL-001/WS8 | arch-local | execute | no | Quarantined local non-authoritative role repo clones to prevent collisions: moved `/workspaces/codex-infra-agent` and `/workspaces/codex-runner-agent` into `/workspaces/.quarantine_remote_role_repos/20260213T132921Z/` (also stored tar snapshots); added `GOAL-001_WS8_infra_blockers_packet.md` for infra-scope VM DNS/USB passthrough and tightened runner push-retry packet to avoid unsafe force pushes | delegate infra packet to `infra:codex`, then delegate push unblock/retry to `runner:codex`; once pushed, resume WS8 inventory + slot mapping | none
+2026-02-13 13:12 UTC | arch:codex | GOAL-001/WS12 | arch-local | replan | no | Added new workstream WS12 to evaluate Codex App Server as a structured alternative to SSH transport for remote-role prompting, targeting better parallel multi-step threading and context reuse; current SSH remains bootstrap/break-glass transport | implement WS12: document App Server option and define minimal prototype plan + role boundaries | none
+2026-02-13 04:45 UTC | arch:codex | GOAL-001/WS8 | arch-local | execute | yes | executor=remote_codex Prompted `runner:codex` on runner VM to implement WS8/HIL-TOPO-001 and attempt push; runner created local commits `33ef496`..`1d1e412` (topology + evidence + coordination) and produced auth diagnostics evidence `/home/runner/workspaces/codex-runner-agent/documentation/evidence/runner_ws8_push_retry_20260213T044210Z.log`, but push remains blocked due to missing non-interactive GitLab credentials on runner VM (`runner.env` absent) and no `/dev/serial/by-id` HUT passthrough yet | operator: provision runner PAT file (`runner.env`) on runner VM; infra: pass through at least 2 USB serial devices; runner: `git push origin main` and re-run inventory to produce real slot mapping | DEP:operator credential provisioning + DEP:infra USB passthrough
+2026-02-13 04:28 UTC | codex | GOAL-001/WS11 | arch-local | replan | no | Added GOAL-001 WS11 to explicitly track long-run grounding of the remote delegation mechanism (credential storage + transport + `executor=remote_codex` prompting) so future tickets can reliably maintain the `arch:codex` <-> `runner:codex` link | implement WS11 by promoting the stable invocation contract into `documentation/agents/` (no secrets) and keeping the short-term version as a working draft | none
+2026-02-13 04:09 UTC | arch:codex | GOAL-001/WS8 | arch-local | replan | no | Strict boundary enforced: `arch` must not execute/modify runner repo locally; all runner workstreams execute via `runner:codex` on runner VM and report back by commit SHA + evidence; any local `/workspaces/codex-runner-agent` copy is non-authoritative | delegate WS8 packet to `runner:codex` and await runner-side implementation evidence | none
+2026-02-13 04:09 UTC | arch:codex | GOAL-001/WS8 | arch-local | delegate | no | executor=remote_codex Delegate WS8 to `runner:codex` using `documentation/short-term/coordination/delegations/GOAL-001_WS8_runner_packet.md` (HIL-TOPO-001: topology + slot mapping + `flock` locks + evidence); commit+push and report commit SHA + evidence paths back to arch | runner: execute delegation packet and append runner-context handoff entry with commit SHA | none
+2026-02-12 22:31 UTC | runner:codex | GOAL-001/WS7 | runner-live | execute | no | executor=ssh_direct Runner repo first commit/push completed from runner VM to GitLab `runner/runner-agent` main: `fed6ce9` (docs commit hygiene contract), `6255908` (WS7 evidence + handoff) | proceed to GOAL-001/WS8 by defining HUT topology/labels and evidence/lock strategy in runner repo | none
+2026-02-12 22:10 UTC | codex | GOAL-001/WS7 | arch-local | execute | no | executor=remote_codex Runner-side Codex completed WS7 bootstrap: cloned runner repo to `/home/runner/workspaces/codex-runner-agent`, captured first heartbeat, and wrote a runner smoke report JSON | proceed to GOAL-001/WS8 by defining HUT topology/labels and evidence/lock strategy in runner repo | none
+2026-02-12 22:01 UTC | codex | GOAL-001/WS7 | arch-local | delegate | no | executor=remote_codex Delegated WS7 to `runner:codex`: bootstrap runner workspace under `/home/runner/workspaces/codex-runner-agent`, clone runner repo, capture first heartbeat evidence, and append first runner-context handoff entry | run runner-side codex exec for WS7 bootstrap tasks and return evidence paths + next action | none
+2026-02-12 21:42 UTC | codex | GOAL-001/WS6 | arch-local | execute | yes | executor=ssh_direct Operator confirmed Codex login completed on runner VM; WS6 can be marked done and runner ownership can shift to `runner:codex` for remote execution | start GOAL-001/WS7 from runner repo on the runner VM (first heartbeat + first runner-context handoff entry) | none
+2026-02-12 21:34 UTC | codex | GOAL-001/WS6 | arch-local | execute | yes | executor=ssh_direct Installed Node/npm and `codex-cli` on runner VM `codex-runner-01` and verified `codex --version` runs over SSH; interactive login still required | operator: SSH to runner VM and run `codex login`, then report success so WS6 can be marked done and runner ownership can move to `runner:codex` | DEP:operator interactive login
+2026-02-12 21:26 UTC | codex | GOAL-001/WS4-WS5 | arch-local | execute | no | executor=ssh_direct Completed runner VM provisioning (codex-runner-01) and established non-interactive SSH reachability (LAN IP) with evidence captured under `/workspaces/codex-infra-agent/documentation/evidence/` | proceed to GOAL-001/WS6 runner Codex authentication (operator-assisted) using the now-reachable runner VM | none
+2026-02-12 20:45 UTC | codex | GOAL-001/WS4 | arch-local | execute | yes | Ran infra first-heartbeat discovery on LXD host `192.168.10.201` (evidence log captured) but cannot run `lxc` as `codexsvc` due to LXD unix socket permission denied and lack of sudo | operator: add `codexsvc` to `lxd` group (or grant sudo), re-login/refresh session, then retry `lxc list` and proceed with runner VM provisioning | DEP:operator permission fix on LXD host
+2026-02-12 20:26 UTC | codex | GOAL-001/WS4-delegation | arch-local | delegate | no | Delegated GOAL-001 WS4/WS5 ownership to `infra:agent` and advanced infra role activation to `assigned` (pending first heartbeat) | from `/workspaces/codex-infra-agent`, claim `VM-PROV-001` and produce first heartbeat evidence + runner VM spec/SSH identity | none
+2026-02-12 20:19 UTC | codex | GOAL-001/ad-hoc-gitlab-bootstrap | arch-local | execute | no | ticket_id=ADHOC-20260212-03 Created/pushed agent repos to GitLab remotes (`infra/infra-agent`, `runner/runner-agent`) and published bootstrap checkout/verification flow; ADHOC-20260212-03 marked done | continue GOAL-001 by assigning live `infra:<agent-id>` owner and starting WS4 runner VM provisioning with repo bootstrap contract | none
+2026-02-12 20:10 UTC | codex | GOAL-001/ad-hoc-credential-contract | arch-local | execute | no | ticket_id=ADHOC-20260212-02 Standardized GitLab credential-home/auth/verification contract for `git.lecsys.net` across AGENTS + short-term governance docs; unblocked follow-on ticket ADHOC-20260212-03 | execute ADHOC-20260212-03 by configuring/pushing infra + runner repos with documented bootstrap checkout flow | none
+2026-02-12 20:04 UTC | codex | GOAL-001/ad-hoc-intake | arch-local | replan | yes | ticket_id=ADHOC-20260212-02,ADHOC-20260212-03 Tracked user ad-hoc requests for GitLab credential-home standardization and subsequent repo push/bootstrap flow; linked second item as blocked on credential readiness | await operator credential setup details, then execute ADHOC-20260212-02 followed by ADHOC-20260212-03 | DEP:operator credential setup pending
+2026-02-12 18:32 UTC | codex | GOAL-001/governance-ad-hoc | arch-local | replan | no | ticket_id=ADHOC-20260212-01 Added mandatory ad-hoc ticket reflex: agents should suggest ticket tracking on user ad-hoc requests and follow user yes/no decision path across board/template/start-here/AGENTS + queue docs | use ad-hoc queue by default for user-approved tracking requests | none
+2026-02-12 18:30 UTC | codex | GOAL-001/governance-replan | arch-local | replan | no | Added formal course-correction protocol for user-driven reprioritization/easier-path/intermediate-quest changes, including trigger examples and required board/goal/handoff updates | apply this protocol when operator requests sequencing/path adjustments, even without hard prerequisites | none
+2026-02-12 18:16 UTC | codex | GOAL-001/governance | arch-local | scaffold | no | Added anti-confusion governance controls across board/template/startup docs: role activation state machine, concrete-owner gate for remote `in_progress`, mandatory handoff context/action/operator fields, delegation checklist, and role-prefixed evidence convention | use new handoff format for all future entries and update remote role activation states as owners are assigned | none
+2026-02-12 16:55 UTC | codex | GOAL-001/WS3-correction | Clarification: WS3 completion was performed by `arch:codex` as bootstrap scaffolding (creating the infra repo/governance skeleton), not by a live delegated `infra` session; delegation boundary remains intact | assign `infra:<agent-id>` and start WS4 from infra context | WS4 blocked until infra owner assignment
+2026-02-12 16:33 UTC | codex | GOAL-001/WS3 | Completed WS3 by creating dedicated infra repository at `/workspaces/codex-infra-agent`, initialized git repo, added platform-agnostic `AGENTS.md` with discovery-first startup and hard no-product-repo-clone boundary, and added infra coordination board/handoff files | assign an `infra:<agent-id>` owner and execute GOAL-001 WS4 runner VM provisioning | none
+2026-02-12 16:10 UTC | codex | GOAL-001/WS3-planning | Refactored governance to role-based ownership (`arch`/`infra`/`runner`), expanded GOAL-001 with readiness workstreams (infra bootstrap, runner VM provisioning, SSH, auth, runner bootstrap) before hardware topology, and updated board/template/start-here/ci-lane rules accordingly | execute GOAL-001 WS3 by bootstrapping infra repo + AGENTS/board/handoff; then delegate WS4 to infra role | OPERATOR support needed later for runner Codex auth and optional local GitLab onboarding
+2026-02-12 15:26 UTC | codex | GOAL-001/WS2 | Completed WS2: claimed board ownership, documented v1 layer-aware invocation/report contracts, ran Layer1 smoke via `build-scripts/lxd_remote.sh heartbeat`, and published evidence (`test/build/log/lxd_ws2_smoke_20260212.log`, `test/build/log/lxd_ws2_report_20260212.json`) | claim GOAL-001 WS3 and define runner labels, slot identity mapping, and lock routing | none
+2026-02-12 15:25 UTC | codex | GOAL-001/WS2 | Claimed WS2 on board and added concrete v1 contracts in GOAL-001 for layer responsibility, invocation schema, report schema, guardrail defaults, and minimum puppet-layer repo files | run one WS2 smoke job using the new invocation/report fields and attach resulting log + structured report | none
+2026-02-12 15:22 UTC | codex | GOAL-001/WS2 | Clarified GOAL-001 control model: CI lanes vs puppet layers are orthogonal; added explicit Layer 0/1/2 responsibilities, separate-repo portability target, and Layer 1 AGENT discovery-first/no-codebase-edit constraints in goal doc | claim GOAL-001 WS2 on board and draft concrete remote invocation/report schema + layer-specific AGENT skeletons in infra repos | none
+2026-02-12 15:04 UTC | codex | GOAL-001/WS1 | Completed WS1: validated key-based non-interactive SSH to LXD host, added `build-scripts/lxd_remote.sh` wrapper with strict failure semantics, and captured heartbeat evidence in `test/build/log/lxd_ws1_heartbeat_ok_20260212.log` and failure-path evidence in `test/build/log/lxd_ws1_heartbeat_fail_workdir_20260212.log` | claim GOAL-001 WS2 and define remote job invocation/reporting contract | none
+2026-02-12 14:49 UTC | codex | GOAL-002 | Marked GOAL-002 as parked after accidental kickoff; all GOAL-002 workstreams set to blocked with GOAL-001 dependency | resume GOAL-002 once GOAL-001 is implemented and LXD backend is available, then restart WS1 inventory | DEP:GOAL-001 backend unavailable
+2026-02-12 14:46 UTC | codex | GOAL-002/WS1 | Claimed WS1 and executed slot inventory probe; workspace has no visible serial devices (evidence: test/build/log/hut_slot_inventory_20260212.log) | run inventory probe on hardware-attached LXD HIL host and record `hut-XX` -> `/dev/serial/by-id/...` + platform mapping | no HUT serial devices exposed in current session
+2026-02-12 00:00 UTC | codex | GOAL-002/GOAL-003 | Added active goal specs and board workstreams for HUT surfacing and doc lint CI strategy | assign owners and execute WS1 in each goal | none
+2026-02-12 00:00 UTC | bootstrap | GOAL-001 | Goal scaffold created | assign WS owners and start WS1 | none
+```

documentation/short-term/coordination/workstream_board.md

@@ -0,0 +1,141 @@
+# Workstream Board
+
+Last updated: 2026-02-13
+
+Use this table as the live status board for active short-term goals.
+
+Mission critical focus:
+- `GOAL-001/WS8` Multi-HUT hardware CI topology (runner lane enablement)
+
+| Goal ID | Workstream | Owner | Status | Blockers | Next Action |
+|---|---|---|---|---|---|
+| GOAL-001 | WS1: Infrastructure SSH + Codex passthrough | arch:codex | done | none | none (completed) |
+| GOAL-001 | WS2: Multi-agent operating model | arch:codex | done | none | none (completed) |
+| GOAL-001 | WS3: Infrastructure agent bootstrap | arch:codex | done | none | none (completed) |
+| GOAL-001 | WS4: Runner VM provisioning | infra:agent | done | none | Evidence recorded in infra repo (VM-PROV-001). |
+| GOAL-001 | WS5: Runner SSH reachability | infra:agent | done | none | Evidence recorded in infra repo (RUNNER-SSH-001). |
+| GOAL-001 | WS6: Runner Codex authentication | arch:codex | done | none | Operator completed `codex login` on runner VM; ready to activate `runner:codex` execution |
+| GOAL-001 | WS7: Runner agent bootstrap | runner:codex | done | none | Evidence: `/home/runner/workspaces/codex-runner-agent/documentation/evidence/runner_first_heartbeat_20260212_2209_utc.log`, `/home/runner/workspaces/codex-runner-agent/documentation/evidence/runner_smoke_report_20260212_2209_utc.json`; GitLab: `runner/runner-agent@fed6ce9`, `runner/runner-agent@6255908` |
+| GOAL-001 | WS8: Multi-HUT hardware CI topology | runner:codex | blocked | MISSION_CRITICAL: runner VM now has >=2 stable `/dev/serial/by-id/*` devices and WS8 closeout commits exist locally (`runner/runner-agent@c03a9a0`), but push/fetch to GitLab is blocked by HTTP auth failure (runner PAT appears invalid or missing `write_repository`) | operator: refresh `runner` PAT with `write_repository` and update runner VM `runner.env`; runner: run `~/.local/bin/gitlab_push_origin_main.sh` to push `main`, then report final pushed SHA and mark WS8 `done` | Runner evidence (local): `/home/runner/workspaces/codex-runner-agent/documentation/evidence/runner_hil_topo_inventory_20260213_215705_utc.log`, `/home/runner/workspaces/codex-runner-agent/documentation/evidence/runner_hil_topo_lock_selftest_20260213_215839_utc.log` |
+| GOAL-001 | WS9: Power-cycle control with Home Assistant relay | runner:codex | blocked | DEP:GOAL-001/WS8 pushed to GitLab and slot mapping completed (hut-01 + hut-02 stable) | After WS8 push unblocked: define lock protocol and HA service contract | - |
+| GOAL-001 | WS10: Parallel branch/agent workflow | arch:codex | pending | none | Define worktree strategy, artifact routing, and branch policy |
+| GOAL-001 | WS11: Long-Run Remote Delegation Grounding | arch:codex | pending | none | Promote remote delegation + credential + transport invocation contract into long-lived docs (`documentation/agents/`) and keep an explicit “arch<->runner link” procedure (no secrets) so future tickets can reliably delegate to `runner:codex` |
+| GOAL-001 | WS12: Codex App Server Transport (evaluate -> prototype -> implement) | arch:codex | pending | none | Create `documentation/agents/remote_transport_lock.md` and run WS12 as a gated workflow; once locked, implement the locked mechanism and keep SSH as break-glass fallback unless retired |
+| GOAL-002 | WS1: HUT slot inventory + identity | unassigned | blocked | DEP:GOAL-001/WS8 runner topology not yet implemented | Resume after GOAL-001 WS8; run slot inventory on runner/HIL host |
+| GOAL-002 | WS2: HW-BOOT-001 execution harness | unassigned | blocked | DEP:GOAL-001/WS8 runner topology not yet implemented | Resume after GOAL-001 WS8; define canonical flash/reboot/monitor flow |
+| GOAL-002 | WS3: Per-slot surfacing runs | unassigned | blocked | DEP:GOAL-001/WS8 runner topology not yet implemented | Resume after GOAL-001 WS8; execute per-slot HW-BOOT-001 |
+| GOAL-002 | WS4: Worklist + matrix sync | unassigned | blocked | DEP:GOAL-001/WS8 runner topology not yet implemented | Resume after GOAL-001 WS8; sync worklist/matrix with slot outcomes |
+| GOAL-003 | WS1: Upstream docs lint gate | unassigned | pending | none | Define required markdown lint scope and failure policy |
+| GOAL-003 | WS2: Local agent gardening loop | unassigned | pending | none | Define agent task contract for docs maintenance runs |
+| GOAL-003 | WS3: Progressive doc discovery | unassigned | pending | none | Ensure agents can discover context without large upfront reads |
+| GOAL-003 | WS4: Lint evidence + handoff format | unassigned | pending | none | Standardize report fields for lint before/after and residuals |
+
+## Status Values
+
+- `pending`: not started
+- `in_progress`: actively being worked
+- `blocked`: waiting on dependency or decision
+- `done`: completed and verified for that workstream (does not by itself mean the parent goal is complete)
+
+## Owner Governance
+
+- Owner format: `<role>:<agent-id>` (examples: `arch:codex`,
+  `infra:tbd`, `runner:tbd`).
+- Role keys:
+  - `arch`: Architecture Agent (engineering/control/governance)
+  - `infra`: Infrastructure Agent (host/container/VM operations)
+  - `runner`: Runner Agent (VM CI/CD + test execution)
+- Governance rules:
+  - Owner values are execution semantics:
+    - `<role>:codex` means a Codex runtime is installed/authenticated in
+      that role environment and is being prompted there.
+    - `<role>:agent` means the `arch:codex` control-plane is acting as a
+      temporary proxy executor for that role (typically via SSH
+      transport). This is allowed only for bootstrap/emergency and must
+      be called out in handoff summaries as `executor=ssh_direct`.
+  - `arch` delegates work; SSH may be used as a transport layer, but
+    direct SSH command execution must not be conflated with prompting a
+    remote Codex runtime.
+  - Ownership implies closure accountability:
+    - when `arch:codex` is the owner (or is explicitly asked to "take
+      ownership"), `arch` must drive the workstream to closure:
+      - `done` with acceptance met and evidence recorded, or
+      - `blocked` with concrete blockers, correct owners, operator gate,
+        and explicit next action(s)
+    - "prepared a packet" is not closure; packets are inputs to closure
+  - Commit hygiene (all roles):
+    - each role Codex runtime (`arch`, `infra`, `runner`) is responsible
+      for commit hygiene in its own role repository
+    - make small commits frequently and push so progress is traceable by
+      commit SHA
+    - include commit SHA(s) in handoff summaries for executed work
+    - avoid batching unrelated changes
+  - `arch` is accountable for coordination quality and status reporting.
+  - `arch` should continuously improve agent efficiency and governance
+    quality when requested or when serious inefficiencies are observed.
+  - bootstrap scaffolding by `arch` for a remote role repository does
+    not mean that remote role is active; activation requires explicit
+    owner assignment and first role-context handoff entry.
+  - `infra` must never clone `squeezelite-esp32`.
+  - `infra` does not directly communicate with `runner` except when only
+    local host commands can reach `runner`.
+  - each remote agent role uses a dedicated repository with its own
+    AGENTS/board/handoff artifacts.
+  - evidence paths should use role prefixes for quick origin discovery:
+    `arch_*`, `infra_*`, `runner_*`.
+  - a remote-role workstream must not be set to `in_progress` while
+    owner is `infra:tbd` or `runner:tbd`.
+
+## Role Activation State Machine
+
+- States:
+  - `unassigned`
+  - `assigned`
+  - `first_heartbeat`
+  - `active`
+- Transition rules:
+  - `unassigned` -> `assigned`: concrete owner set (`infra:<id>` or
+    `runner:<id>`)
+  - `assigned` -> `first_heartbeat`: first remote heartbeat evidence
+    captured
+  - `first_heartbeat` -> `active`: first role-context handoff entry
+    logged
+- Enforcement:
+  - remote execution workstreams require role state `>= assigned`
+  - destructive or stateful remote changes require role state
+    `>= first_heartbeat`
+  - continuous delegated execution requires role state `active`
+
+## Change Intake (Prereq + Priority + Intermediate Quest)
+
+When operator/user input indicates plan change (for example: \"we
+missed...\", \"easier way...\", \"we should prioritize...\"), treat it as
+a formal replan event even if it is not a hard prerequisite.
+
+Required board updates:
+
+1. Set impacted workstreams to `blocked` or `pending` as appropriate.
+2. Add new prerequisite/intermediate-quest workstreams when needed.
+3. Update blockers/dependencies and next actions to match new sequence.
+4. Ensure owner + activation state are still valid for the new plan.
+5. Require a matching `action_type=replan` handoff entry.
+
+## Ad-hoc Ticket Reflex
+
+When user/operator gives an ad-hoc request, agents should recommend
+ticket tracking for continuity and let user decide.
+
+Rules:
+1. Ask user: `Track this as a ticket? (yes/no)`.
+2. If `yes`, add/update ticket in:
+   - `documentation/short-term/coordination/ad_hoc_ticket_queue.md`
+3. If `no`, continue and note `ticket_tracking=declined` in handoff.
+4. Default recommendation is `yes` when request spans sessions, roles,
+   or priority changes.
+
+## Role Activation Tracker
+
+| Role | Owner | Activation State | Evidence | Notes |
+|---|---|---|---|---|
+| `infra` | `infra:agent` | `active` | `/workspaces/codex-infra-agent/documentation/evidence/infra_first_heartbeat_20260212_2044_utc.log` | executor=ssh_direct; WS4/WS5 completed with evidence logs under `/workspaces/codex-infra-agent/documentation/evidence/` |
+| `runner` | `runner:codex` | `active` | `/home/runner/workspaces/codex-runner-agent/documentation/evidence/runner_first_heartbeat_20260212_2209_utc.log` | executor=remote_codex; runner repo bootstrap + first handoff entry captured |