How does Crab know what to rerun?

Crab does not ask whether a file modification time looks recent. It asks whether a stage has the same declared identity as a previous successful run.

Conceptually, each key covers:

stage key = hash(command + dependency bytes + parameter values + selected environment)

The real implementation uses canonical workflow structures and content identities. The equation is a mental model, not a byte-level serialization contract.

Read the three records separately

A matching lockfile does not replace payload bytes. A cache entry without a matching stage key is not eligible. Both identity and materialization have to line up.

Preview the decision

Run a dry plan whenever compute is expensive:

crab run --dry
crab workflow status
crab status --workflow --why train

If train is stale and the reason is surprising, request the hash breakdown:

crab run train --dry --explain-miss

Typical causes are changed dependency bytes, a changed command, a selected parameter value, or an allowed environment variable. Fix an undeclared dependency in crab.yaml; do not hide it by forcing a run.

Choose the correct cache mode

Use --no-overwrite when restored outputs must not replace differing workspace files:

crab run --cache-only --no-overwrite

Understand downstream invalidation

Suppose src/train.py changes. ingest and features retain their keys. train changes because its dependency changed. evaluate must then reconsider its key because it consumes models/fraud-model.pkl.

crab run train --downstream

If only the target should run, without automatically adding upstream stages, use the DVC-compatible single-item mode:

crab repro --single-item train

That is an execution selection, not a promise that missing upstream data is safe. The stage still needs its declared inputs.

Share results through the Crab remote

Publish new entries as stages complete:

crab run --cache-push

Backfill all local entries that are absent remotely:

crab workflow push-cache --all --json

In a clean CI clone, require replay:

crab pull
crab run --pull --cache-only --jsonl | tee workflow-events.jsonl

--pull fetches missing dependencies or cache entries. --cache-only prevents a hidden recomputation and exits with code 3 on a miss.

Inspect crashes without trusting partial output

Crab records run trajectories under .crab/workflow/runs/. Inspect them with:

crab workflow journal ls
crab workflow journal show <run-id>

Resume handling is conservative because a partly written model can look plausible while being invalid. Use normal resume verification first. --resume-trust-outputs is an explicit operator decision for a case where you have independently proved the output files.

Prune old terminal journals without touching active runs:

crab workflow journal gc --keep 50

Next, run and queue a parameter sweep.