Promote and roll back ML artifacts safely

Promotion answers a narrow question: which immutable model version should a human-facing stage label select now?

It does not retrain, copy payload bytes, edit the version manifest, or prove model quality.

Choose a candidate, switch between staging and production, then simulate a concurrent label change. The lab previews the exact compare-and-swap boundary without changing a real registry.

1. Read current canonical state

crab artifacts show fraud-model --json
crab artifacts history fraud-model --json

Record both the candidate version ID and the version currently selected by staging or production. Do not infer them from a filename or experiment label.

2. Validate the candidate by immutable ID

crab artifacts get \
  fraud-model \
  --version <candidate-version> \
  --output validation/candidate.pkl

python3 src/smoke_model.py validation/candidate.pkl
python3 src/check_quality_gate.py metrics/evaluation.json

Add the organization's schema, latency, fairness, security, and canary checks. Hash verification is necessary, but it is not a deployment decision.

3. Promote staging with compare-and-swap

crab artifacts promote \
  fraud-model \
  <candidate-version> \
  staging \
  --expected <current-staging-version> \
  --json

If another operator changed staging after your read, the compare-and-swap fails. Reread the registry and decide whether their selection should remain. Do not retry using a guessed expected value.

If the label does not exist yet, there is no current version to compare. Omit --expected only for that first promotion, then read the new mapping and use compare-and-swap for every later move.

After staging tests pass, repeat the pattern for production:

crab artifacts promote \
  fraud-model \
  <candidate-version> \
  production \
  --expected <current-production-version> \
  --json

4. Verify the consumer boundary

Inspect mapping and history:

crab artifacts show fraud-model
crab artifacts history fraud-model

Then retrieve by stage from a clean client:

crab artifacts get \
  fraud-model \
  --stage production \
  --output runtime/fraud-model.pkl \
  --json

The consumer should record the immutable version ID it resolved. A later production promotion can move the label, but it cannot change the bytes of that recorded version.

5. Roll back by moving the label

Rollback selects a previously validated immutable version:

crab artifacts show fraud-model --json

crab artifacts promote \
  fraud-model \
  <previous-good-version> \
  production \
  --expected <failing-version> \
  --json

No model payload is rebuilt or copied. History records the prior mapping and timestamp, preserving the sequence of decisions.

6. Use the complete operational loop

# Producer
crab run --cache-push
crab metrics show
git commit -am "record approved fraud candidate"
crab artifacts version create fraud-model --json

# Release automation
crab artifacts show fraud-model --json
crab artifacts get fraud-model --version <candidate> --output validation/model.pkl
python3 src/smoke_model.py validation/model.pkl
crab artifacts promote fraud-model <candidate> staging --expected <old-staging>
crab artifacts promote fraud-model <candidate> production --expected <old-production>

# Independent consumer
crab artifacts get fraud-model --stage production --output runtime/model.pkl
crab artifacts history fraud-model --json

This series has now covered the main ML command families: stage authoring, DAG validation and execution, cache publication and replay, status and journals, experiments and queues, params/metrics/plots, and the complete artifact list/show/create/get/promote/history lifecycle.

Finish with the complete Crab ML workflow CLI cookbook, then use Workflow artifacts, Artifact promotions, and the crab workflow reference for command-level details.