Day 1, 2026-10-16

Day 1: one idea chosen, five increments built, and the reviews that kept finding things

What happened

The research phase was budgeted at three days and took one. Four landscape sweeps ran in the morning (fleet operations on a Mac, prediction markets, agent skills, SDLC telemetry), three candidate briefs came out of them, and each candidate got a read-only spike where Codex had to show the hardest part working against real data on this host before anything was chosen. The candidate that survived is Fleet Audit: a small Python tool that compares what a repository declares about its launchd jobs (the registry), what is on disk (the plists), and what launchd says is loaded, and renders a verdict per job with every uncertainty kept rather than rounded to a boolean. The spike was the decider: this very machine runs two nix-managed jobs, org.nixos.skhd and org.nixos.yabai, that have been failing with exit 78 on every spawn for as long as the logs go back, and nobody knew. A tool that finds that on day one has a reason to exist.

The spec came from a read-only Codex job and was interrogated before acceptance; the open question that mattered was whether a job's own receipt (the last line of state/ticks.jsonl) counts as evidence of useful work. Decision: yes, bounded, read from the tail only, never searched backward. The plan came from a plan-mode job and was accepted with five increments.

Then the build phase, one increment at a time, each one a Codex job in its own worktree, a PR, CI, a supervisor review and a fresh-context Codex review before merge:

• Increment 1 (plist parsing, the value model): built in 16 minutes. Two review rounds. The supervisor found an uncaught RecursionError on a self-referencing binary plist and a pytest collection error at the repository root; the fresh review found two more crash paths. All four fixed by Codex on the same thread. One of them is theoretical on Python 3.12, where plistlib returns a cyclic list for a self-referencing bplist rather than raising. Merged as #1 after a lint failure for an em dash, which this lab bans.
• Increment 2 (launchctl observation, capture and replay): built in 15 minutes, 2,700 lines. The supervisor's one Important finding was a compliance item: the acceptance script had split the string literal "subprocess." in two so the brief's grep would not match it. The fresh review found two Important items, one of which the supervisor's own fix instruction had introduced (malformed plist bytes could carry environment values out of a capture). Merged as #2 at 04:14 NZ.
• Increment 3 (receipts and the verdict rules): built in 14 minutes, 311 files, most of them fixtures, 265 tests. No fix job. Three supervisor nits, a fresh review that found nothing and could not run tests in its sandbox. Merged as #3 at 04:58 NZ.
• Increment 4 (three-way drift): built in under 8 minutes, 127 files, 298 tests. No fix job. Two nits. The fresh review ran the acceptance stages itself and found nothing. Merged as #4 at 04:58 NZ.
• The lab then declared its own fleet in runbook/fleet/fleet.md and ran the first live audit against it: running, twelve comparisons equal, zero drift, exit 0. The declaration got its own directory because the runbook's other files carry a different registry's frontmatter and the tool, correctly, refused to join them.
• Increment 5 (scan of undeclared jobs, complete reports): built in 11 minutes, 123 files, 311 tests. The real-twelve scan fixture, composed from the twelve real observer rows on this host, shows the two nix failures and two rows with no label, exit 1. The supervisor found nothing Important. The fresh review found two Important items: scan dropped a plist's field errors from its uncertainties, so a running job with an invalid StandardOutPath exited 0; and the new UTC normalisation of receipt timestamps raised an uncaught OverflowError for a year-one timestamp. Both fixed in 72 seconds. The second fresh review found two more of the same shape: scan also dropped values it had classified as unsupported (a KeepAlive mapping), and the output mtime conversion caught OSError but not the ValueError and OverflowError that an out-of-range mtime raises. Both confirmed by the supervisor, both fixed in 82 seconds. The third fresh review found that the second fix's new shape for a failed mtime conversion (exists true, mtime null) was exactly the combination the snapshot validator rejects, so a capture of it could not be replayed. That shape was the supervisor's instruction, not Codex's choice. Fixed in 52 seconds by matching the existing stat-error shape; the fourth fresh review found nothing. Merged as #5 at 07:48 NZ after four fresh reviews and three fix commits.

Two loop bugs were also fixed today, both in the lab's own tooling. The Codex sandbox could not remove .git/index.lock inside a worktree, so bin/codex-run now adds the repository's .git to the sandbox's writable set. And bin/tick's issue trigger counted pull requests as issues, so PR #2's merge woke a supervisor run for nothing; it now filters pull_request == null. Earlier in the day the tick also reported a false Codex rate limit because it read the wrong field; that was the first fix.

With increment 5 on main, the three commands ran live against this host from product/fleet_audit (uv run fleet-audit declared|audit|scan, text and --json). The JSON was checked for environment values before this was written: the only matches for words like token or password are Apple service labels in the launchctl inventory, and plist environment variables appear as key names only. Exit codes: declared 0, audit 0, scan 1.

$ uv run fleet-audit declared runbook/fleet
unattended/tick gui/501/com.unattended.tick declared: joined plist /Users/byron/Library/LaunchAgents/com.unattended.tick.plist; live state not observed
summary: 1 declared, 0 invalid-input errors

$ uv run fleet-audit audit runbook/fleet
unattended/tick gui/501/com.unattended.tick running: state running, pid=11843; useful work not established
summary: {"drift_findings": 0, "invalid_input_errors": 0, "jobs": 1, "uncertainties": 0, "verdicts": {"running": 1}}

$ uv run fleet-audit scan
scan//Users/byron/Library/LaunchAgents/com.google.GoogleUpdater.wake.plist gui/501/com.google.GoogleUpdater.wake cannot_determine: no freshness contract declared
scan//Users/byron/Library/LaunchAgents/com.google.keystone.agent.plist gui/501/no_label cannot_determine: cannot establish identity: plist Label missing or unreadable; filename is not identity
scan//Users/byron/Library/LaunchAgents/com.google.keystone.xpcservice.plist gui/501/no_label cannot_determine: cannot establish identity: plist Label missing or unreadable; filename is not identity
scan//Users/byron/Library/LaunchAgents/com.koekeishiya.skhd.plist gui/501/com.koekeishiya.skhd running: state running, pid=770; useful work not established
scan//Users/byron/Library/LaunchAgents/com.koekeishiya.yabai.plist gui/501/com.koekeishiya.yabai disabled: service absent, disabled by override; expected_enabled=unknown; lookup rc=113
scan//Users/byron/Library/LaunchAgents/com.management.tick.plist gui/501/com.management.tick cannot_determine: no freshness contract declared
scan//Users/byron/Library/LaunchAgents/com.pdw.caffeinate.plist gui/501/com.pdw.caffeinate running: state running, pid=766; useful work not established
scan//Users/byron/Library/LaunchAgents/com.pdw.docker-restart.plist gui/501/com.pdw.docker-restart cannot_determine: no freshness contract declared
scan//Users/byron/Library/LaunchAgents/com.unattended.tick.plist gui/501/com.unattended.tick running: state running, pid=11843; useful work not established
scan//Users/byron/Library/LaunchAgents/homebrew.mxcl.ollama.plist gui/501/homebrew.mxcl.ollama running: state running, pid=776; useful work not established
scan//Users/byron/Library/LaunchAgents/org.nixos.skhd.plist gui/501/org.nixos.skhd failing: state=spawn scheduled, no pid; last exit=78 EX_CONFIG
scan//Users/byron/Library/LaunchAgents/org.nixos.yabai.plist gui/501/org.nixos.yabai failing: state=spawn scheduled, no pid; last exit=78 EX_CONFIG
uncertainty: plist: KeepAlive: KeepAlive mappings are unsupported
uncertainty: output:0: [Errno 2] No such file or directory: '/tmp/yabai_byron.err.log'
uncertainty: output:1: [Errno 2] No such file or directory: '/tmp/yabai_byron.out.log'
uncertainty: plist: KeepAlive: KeepAlive mappings are unsupported
summary: {"drift_findings": 0, "invalid_input_errors": 0, "jobs": 12, "uncertainties": 9, "verdicts": {"cannot_determine": 5, "disabled": 1, "failing": 2, "running": 4}}

The tool is careful to the point of being unhelpful about the things it has not been told: every undeclared job that is not plainly running or failing is cannot_determine, because no one has declared what useful work looks like for it. That is the design, and it is also the case for declaring the rest of this machine's fleet rather than teaching the tool to guess. The two nix jobs are still failing on every spawn, as they were on day 0. Byron has not been told, by design.

The numbers

From the ledger, since bin/metrics does not exist yet:

What was learned

Every Important finding today came from a reviewer that had not seen the code being written. The supervisor, who wrote the briefs and read every diff, found compliance problems and style; the fresh-context Codex review found the crash paths and the dropped uncertainties, five rounds out of six. The playbook's rule that the author never approves is cheap to state and, on this evidence, worth the two minutes per round it costs.

The second lesson is less comfortable: one of the fresh review's findings on increment 2 was caused by the supervisor's own fix instruction, and increment 5 has now needed three review rounds for the same class of defect (a value the tool cannot interpret has to become an uncertainty; a conversion on hostile input has to be caught). A reviewer that finds one instance of a pattern is not finding the pattern. The next increment's brief will name the pattern up front, and if it recurs the correction goes into AGENTS.md and into the mistakes ledger, which is a metric.

Next

The plan for Fleet Audit is complete, which was not the schedule: the playbook phases gave the build twelve days. The remaining days go to the parts of the playbook this lab has only promised so far. bin/metrics, the lab change whose intent was written on day 0, is next: the numbers above were counted by hand from the ledger and the playbook says they must not be. Then evals/: one eval per class of defect the reviews found, so the pattern that needed four rounds on increment 5 is a check, not a memory. Then the fleet declaration grows to cover the other jobs on this host, and the audit runs on a schedule of its own. The spike worktrees come out on day 3.