The difficult rendering investigations in Ashes of the Unbound all began to follow the same choreography: load a state, apply a diagnostic mode, wait for the world to settle, capture a screenshot, write down what changed and hope the filename still made sense tomorrow.
On 11–12 May I turned that process into EvidenceRunner, an editor-only recipe runner for repeatable diagnostic cells. Its first dogfood recipe executed twenty steps in 11.64 seconds and produced thirteen artifacts.
The speed was nice. The durable result was a contract for what counts as evidence.
EvidenceRunner test cells
A recipe is divided into cells. Each cell declares the state it needs, the actions to perform, the condition that allows the run to continue and the artifacts it should produce.
That makes the cell the unit of work rather than the screenshot. A PNG alone cannot tell me which console variables were active, whether the world had finished updating or which comparison it belongs to. The cell carries that context.
The runner executes cells through an explicit state machine. Setup, action, wait, capture and completion are observable states, not timing guesses spread across editor scripts.
Waiting for the screenshot file
One of the first useful failures came from screenshot timing.
Unreal can accept a capture request before the output file has been written. If the runner advances at request time, a following step can hash a missing file, move on to the next state or even end the run while the artifact is still in flight.
EvidenceRunner treats file existence as the completion condition. The capture step waits until the expected file appears, then records it. This sounds tiny, but it is the difference between an action log and evidence.
The broader rule is that asynchronous tools need completion conditions at the boundary users care about. The user asked for a file, not a successful function return.
Artifact manifests and hashes
Every produced artifact is connected to a manifest containing its recipe, cell, state and provenance. Files receive SHA-256 hashes so later movement or ingestion does not erase their identity.
The manifest turns a directory of captures into a run. It becomes possible to answer which configuration produced a frame, whether two references are byte-identical and whether an artifact was changed after capture.
That does not make a screenshot scientifically conclusive. It makes the claim around it inspectable.
Keeping the vault outside the plugin
The project development record lives in an Obsidian vault, but EvidenceRunner does not know that.
The reusable plugin produces a clean run with manifests and artifacts. A project-side bridge can ingest that run into the vault layout. This prevents a generally useful editor tool from acquiring hard-coded knowledge of one repository, one note structure or one developer’s filesystem.
It also means a failed vault import cannot corrupt the original run. Evidence is produced first, then integrated.
The first dogfood run
The initial dogfood recipe completed all twenty planned steps in 11.64 seconds and wrote thirteen artifacts. More importantly, it exercised waits, captures, hashing and manifest production together rather than proving each in isolation.
That gave subsequent debugging work a repeatable vocabulary. A report could point to a recipe and cell instead of describing a series of manual editor actions from memory. Before/after comparisons could prove they used the intended modes. Missing artifacts became run failures rather than historical mysteries.
Running the investigation matrix
Automation does more than perform the same steps faster. It makes certain questions cheap enough to ask.
If capturing five controlled variants takes a minute and produces self-describing output, I can test a matrix instead of choosing the two states that feel most likely. If a control behaves unexpectedly, preserving it is automatic rather than optional. If the result is unresolved, the evidence can still be retained without forcing a conclusion.
That last point matters for the later first-connection SkyLight investigation. EvidenceRunner helped demonstrate causal interventions, including a control that became worse, without turning those results into a false production fix.
Next step: restoring a player
The runner was built in the editor, but the game still had to restore real players on a dedicated server. Once planetary coordinates entered the save payload, restoring “the same position” became much more complicated than deserializing a vector.
That is the next dispatch: Restoring a Player on Another Moon.