DVC Migration

Crab Workflow intentionally keeps the familiar DVC shape: stages, deps, outs, params, metrics, plots, experiments, queues, and repro-style execution. The main difference is the storage model. Crab uses your Crab remote and object store rather than a separate DVC remote.

For the automated migration command, see the full Migrating from DVC guide.

Command Mapping

DVC command	Crab command
`dvc stage add`	`crab stage add`
`dvc repro`	`crab repro` or `crab run`
`dvc dag`	`crab workflow dag`
`dvc status`	`crab workflow status`
`dvc params show`	`crab params show`
`dvc params diff`	`crab params diff`
`dvc metrics show`	`crab metrics show`
`dvc metrics diff`	`crab metrics diff`
`dvc plots show`	`crab plots show`
`dvc plots diff`	`crab plots diff`
`dvc exp run`	`crab exp run`
`dvc exp show`	`crab exp show`
`dvc exp diff`	`crab exp diff`
`dvc exp apply`	`crab exp apply`
`dvc exp push`	`crab exp push`
`dvc exp pull`	`crab exp pull`
`dvc queue start`	`crab queue start`

Migration Checklist

Commit or stash current DVC work. Keep dvc.yaml, dvc.lock, .dvc/, .dvc/cache/, .dvc/config, and any tracked .dvc pointer files in place until the data and lock migration is independently verified.
Initialize Crab. Fresh configs enable Workflow; set workflow.enabled = false only when you explicitly want to opt out:
```
crab init crab://my-bucket/ml-project
# Optional explicit setting:
crab config set workflow.enabled true
```
Start with a read-only inventory and converted definition:
```
crab migrate from-dvc --plan --json
```
Review every pointer, cache object, lock record, remote, provenance record, and blocking finding. For a mapped DVC remote, pass an explicit credential-free destination only after deciding which Crab remote owns it:
```
crab migrate from-dvc --remote-map origin=crab://bucket/project
```
Run the repository-aware migration and validate:
```
crab migrate from-dvc
crab run --validate
crab workflow dag
```
The command writes .crab/workflow/migration/dvc.json, transfers verified sources through Crab staging, and writes canonical pointers and crab.lock only after the source and publication gates pass. A failed run retains the journal and source state; resume only after the inventory is unchanged:
```
crab migrate from-dvc --resume
```
Inspect safe_to_remove_dvc in the text or JSON report. It is false for missing/corrupt/unknown records, unverified remote mappings, unsupported checkpoints/providers, and artifacts awaiting lifecycle proof. It may be true only after a fresh Crab clone restores the migrated tree byte-for-byte (including modes) and the journal contains clean-clone evidence.

Push Git state and stage cache only after the report is understood:

git add crab.yaml crab.lock params.yaml
git commit -m "migrate workflow to crab"
crab push
crab workflow push-cache --all

Do not remove DVC files unless the report explicitly says safe_to_remove_dvc: true. The command never deletes or rewrites dvc.yaml, dvc.lock, .dvc/, cache objects, remotes, workspace data, or pointer files. A manual cleanup after a verified cutover is outside the migration command.

YAML Compatibility

Most DVC fields map directly:

DVC field	Crab support
`cmd`	Supported as string or command list.
`deps`	Supported for local paths and supported external deps.
`outs`	Supported for path entries and DVC-style path-key maps.
`params`	Supported with dotted keys.
`metrics`	Supported at top level and stage level.
`plots`	Supported at top level and stage level.
`wdir`	Supported.
`frozen`	Supported with `crab freeze` and `crab unfreeze`.
`foreach`	Supported.
`matrix`	Supported.
`vars` and `${...}`	Supported for templating.
`always_changed`	Supported as DVC-compatible spelling for nondeterministic stages.
`artifacts`	Preserved and validated as catalog metadata. `crab artifacts` list/show/get/version/promote/history use the primary Crab remote when configured (with a local mirror); migration remains cutover-blocked until clean-clone and GC evidence pass.

Behavior Differences

Crab Workflow is enabled by default for fresh configs; set workflow.enabled = false for an explicit opt-out.
Stage cache is shared through the Crab remote with crab run --cache-push or crab workflow push-cache --all.
Large files should be tracked with Crab's file layer, not a DVC remote.
checkpoint outputs are rejected with dvc_checkpoint_unsupported; they are never silently rewritten to persist. Hand-authored Crab checkpoints belong to crab exp run, not ordinary run/repro.
The migration command inventories and transfers supported local sources, but it does not infer or live-verify a DVC remote destination. Use --remote-map, then satisfy the provider and clean-clone gates before treating the result as safe.
Top-level artifacts: declarations are preserved and validated as catalog metadata. See the artifact lifecycle pages for remote-backed versioning and promotion; remote artifact GC reachability remains a release gate.
Local experiment metadata lives under .crab/workflow/exp/.
Run journals live under .crab/workflow/runs/.
Split workflow files can use workflow.discover recursive and workflow.lockfile split.

Hydra

Enable Hydra-style composition before migrating Hydra-heavy experiments:

crab config set hydra.enabled true
crab config set hydra.config_dir conf
crab config set hydra.config_name config.yaml

Then run experiments with group and scalar overrides:

crab exp run -S train/model=efficientnet -S train.optimizer.lr=0.0005

See Hydra Workflows for the recommended layout and precedence model.

CI Migration

Replace DVC cache commands with Crab cache commands:

crab run --validate
crab run --pull --allow-missing --cache-push --jsonl
crab workflow push-cache --all --json

For strict read-only verification:

crab run --cache-only --json

Cutover status

The converter is not a cutover gate. A clean Crab run, metrics check, and remote cache push are necessary but insufficient because DVC cache, remotes, pointer files, workspace data, and lock state are not imported yet. Keep the DVC inputs until a dedicated importer has verified a fresh-clone restore.

For now, use the following as validation only:

crab run
crab metrics show
crab plots show
crab workflow push-cache --all

Do not remove DVC-specific files or uninstall DVC based on this workflow converter. Removal becomes safe only after the data migration and restore qualification work is complete.

DVC Migration

For the automated migration command, see the full Migrating from DVC guide.

Command Mapping

DVC command	Crab command
`dvc stage add`	`crab stage add`
`dvc repro`	`crab repro` or `crab run`
`dvc dag`	`crab workflow dag`
`dvc status`	`crab workflow status`
`dvc params show`	`crab params show`
`dvc params diff`	`crab params diff`
`dvc metrics show`	`crab metrics show`
`dvc metrics diff`	`crab metrics diff`
`dvc plots show`	`crab plots show`
`dvc plots diff`	`crab plots diff`
`dvc exp run`	`crab exp run`
`dvc exp show`	`crab exp show`
`dvc exp diff`	`crab exp diff`
`dvc exp apply`	`crab exp apply`
`dvc exp push`	`crab exp push`
`dvc exp pull`	`crab exp pull`
`dvc queue start`	`crab queue start`

Migration Checklist

Commit or stash current DVC work. Keep dvc.yaml, dvc.lock, .dvc/, .dvc/cache/, .dvc/config, and any tracked .dvc pointer files in place until the data and lock migration is independently verified.
Initialize Crab. Fresh configs enable Workflow; set workflow.enabled = false only when you explicitly want to opt out:
```
crab init crab://my-bucket/ml-project
# Optional explicit setting:
crab config set workflow.enabled true
```
Start with a read-only inventory and converted definition:
```
crab migrate from-dvc --plan --json
```
Review every pointer, cache object, lock record, remote, provenance record, and blocking finding. For a mapped DVC remote, pass an explicit credential-free destination only after deciding which Crab remote owns it:
```
crab migrate from-dvc --remote-map origin=crab://bucket/project
```
Run the repository-aware migration and validate:
```
crab migrate from-dvc
crab run --validate
crab workflow dag
```
The command writes .crab/workflow/migration/dvc.json, transfers verified sources through Crab staging, and writes canonical pointers and crab.lock only after the source and publication gates pass. A failed run retains the journal and source state; resume only after the inventory is unchanged:
```
crab migrate from-dvc --resume
```
Inspect safe_to_remove_dvc in the text or JSON report. It is false for missing/corrupt/unknown records, unverified remote mappings, unsupported checkpoints/providers, and artifacts awaiting lifecycle proof. It may be true only after a fresh Crab clone restores the migrated tree byte-for-byte (including modes) and the journal contains clean-clone evidence.

Push Git state and stage cache only after the report is understood:

git add crab.yaml crab.lock params.yaml
git commit -m "migrate workflow to crab"
crab push
crab workflow push-cache --all

Do not remove DVC files unless the report explicitly says safe_to_remove_dvc: true. The command never deletes or rewrites dvc.yaml, dvc.lock, .dvc/, cache objects, remotes, workspace data, or pointer files. A manual cleanup after a verified cutover is outside the migration command.

YAML Compatibility

Most DVC fields map directly:

DVC field	Crab support
`cmd`	Supported as string or command list.
`deps`	Supported for local paths and supported external deps.
`outs`	Supported for path entries and DVC-style path-key maps.
`params`	Supported with dotted keys.
`metrics`	Supported at top level and stage level.
`plots`	Supported at top level and stage level.
`wdir`	Supported.
`frozen`	Supported with `crab freeze` and `crab unfreeze`.
`foreach`	Supported.
`matrix`	Supported.
`vars` and `${...}`	Supported for templating.
`always_changed`	Supported as DVC-compatible spelling for nondeterministic stages.
`artifacts`	Preserved and validated as catalog metadata. `crab artifacts` list/show/get/version/promote/history use the primary Crab remote when configured (with a local mirror); migration remains cutover-blocked until clean-clone and GC evidence pass.

Behavior Differences

Crab Workflow is enabled by default for fresh configs; set workflow.enabled = false for an explicit opt-out.
Stage cache is shared through the Crab remote with crab run --cache-push or crab workflow push-cache --all.
Large files should be tracked with Crab's file layer, not a DVC remote.
checkpoint outputs are rejected with dvc_checkpoint_unsupported; they are never silently rewritten to persist. Hand-authored Crab checkpoints belong to crab exp run, not ordinary run/repro.
The migration command inventories and transfers supported local sources, but it does not infer or live-verify a DVC remote destination. Use --remote-map, then satisfy the provider and clean-clone gates before treating the result as safe.
Top-level artifacts: declarations are preserved and validated as catalog metadata. See the artifact lifecycle pages for remote-backed versioning and promotion; remote artifact GC reachability remains a release gate.
Local experiment metadata lives under .crab/workflow/exp/.
Run journals live under .crab/workflow/runs/.
Split workflow files can use workflow.discover recursive and workflow.lockfile split.

Hydra

Enable Hydra-style composition before migrating Hydra-heavy experiments:

crab config set hydra.enabled true
crab config set hydra.config_dir conf
crab config set hydra.config_name config.yaml

Then run experiments with group and scalar overrides:

crab exp run -S train/model=efficientnet -S train.optimizer.lr=0.0005

See Hydra Workflows for the recommended layout and precedence model.

CI Migration

Replace DVC cache commands with Crab cache commands:

crab run --validate
crab run --pull --allow-missing --cache-push --jsonl
crab workflow push-cache --all --json

For strict read-only verification:

crab run --cache-only --json

Cutover status

For now, use the following as validation only:

crab run
crab metrics show
crab plots show
crab workflow push-cache --all

Do not remove DVC-specific files or uninstall DVC based on this workflow converter. Removal becomes safe only after the data migration and restore qualification work is complete.

DVC Migration

Command Mapping

Migration Checklist

YAML Compatibility

Behavior Differences

Hydra

CI Migration

Cutover status

On this page

DVC Migration

Command Mapping

Migration Checklist

YAML Compatibility

Behavior Differences

Hydra

CI Migration

Cutover status

On this page