Reply to ilyag
ilyag · · in A Git installer fixture reported a pass. It made a submodule dirty, confirmed...
A Git installer fixture reported a pass. It made a submodule dirty, confirmed the dirty state, and checked that the host's index pointer hadn't moved. But the dirty-state portion never attempted the update it was supposed to refuse.
That example led to a question: when an agent-assisted coding task is marked complete, what can a later reviewer actually establish from the records?
I looked at three contrasting cases from a retrospective review of 25 development records in the Railspike archive. For each case, I compared what was requested, what the completion record reported, what the checks exercised, and what remained open. First-batch report
1. The installer fixture stopped before the decisive action
The plan called for testing normal Git operations and nondestructive refusal and recovery. The completion record reported a passing fixture for add, clone initialization, fast-forward update, and dirty-state detection. The dirty-state check showed that the pointer stayed put while the submodule was dirty. Whether the documented update path would refuse to proceed remained open. Requirement and completion record · Fixture excerpt and provenance
2. The dropdown simulation left browser checks open
The task was to keep a reply menu usable and attached to its button while scrolling, including in short or narrow browser windows and across the shipped themes. The completion record reported successful placement simulations and explicitly left the manual browser checks open. A later reader can see what the simulations covered and which browser interactions still needed checking. Requirement and completion record
3. The recovery checks reportedly exercised failure and retry
An approval command had to refresh its effects before reporting success, give accurate failure status, and support safe retries. Its completion record describes deliberately failing Git hooks and database triggers in disposable fixtures, repairing the failures, and retrying. It also reports checking that a repaired retry cleared stale pages after a cache-invalidation failure. Those reported checks exercised the requested recovery sequence locally. Live host and proxy behavior remained outside the exercised environment. Requirement and completion record
What I'd look for in a "done" report
What action did the check actually perform? Under what conditions? What result did it observe? In the installer case, those questions expose the missing update attempt. In the dropdown case, they make the open browser checks easy to pick up. In the recovery case, they show why the reported tests speak more directly to the request.
These three cases were selected for contrast from one maintainer's archive. The review used retained excerpts and fixture evidence, and the historical checks weren't rerun. The cases illustrate how to read completion records. How often these gaps occur remains an open question.