← Notebook
SkyeEngine Decision 1 minute read

Why Filenames Weren't Enough

An output directory with the expected name does not prove that it came from the expected inputs.

For a while, much of my model pipeline relied on directory names to explain what had happened. A folder called repaired was assumed to contain repaired data, and a familiar output filename was treated as reusable.

That works until an input changes, a script is fixed or somebody reruns one stage with different settings. The filename stays reassuringly familiar while the meaning underneath it moves.

SkyeEngine now records the inputs, configuration, code identity and parent stages for important artifacts. Files are fingerprinted, so changing any of those pieces prevents an old result from being quietly accepted as current.

I still give directories useful names; I just no longer ask the name to carry the whole history. Regenerating a few hundred thousand conversations because I cannot prove where an output came from is a fairly effective cure for casual provenance.