crab exp

Run reproducible experiments against throwaway worktrees with parameter overrides, then inspect, compare, and manage experiment history.

Synopsis

crab exp run [OPTIONS]
crab exp show <ID> [--json]
crab exp diff <ID_A> <ID_B> [--json]
crab exp ls [--limit <N>] [--json]
crab exp promote <ID> --branch <BRANCH> [--json]
crab exp gc [--keep <N>] [--dry-run] [--json]

Description

crab exp manages content-addressed experiment runs. Each experiment branches off HEAD into a throwaway worktree, applies optional --set key=value parameter overrides, executes the DAG defined in crab.yaml, and persists metadata locally under .crab/workflow/exp/<uuid>.meta.json.

Experiments are keyed by UUIDv7 identifiers that sort chronologically, so listing and diffing operations work without relying on wall-clock timestamps.

Six subcommands are available:

• run — mint an experiment ID, create a temporary worktree from HEAD, execute the workflow DAG with optional parameter overrides, and persist metadata locally. The temporary worktree is cleaned up on completion.

• show — display the full metadata record for a given experiment, including parameter overrides, stage hashes, and collected metrics.

• diff — compare two experiments side-by-side, showing differences in parameter overrides, stage hashes, and metric values.

• ls — list all local experiments in reverse chronological order (newest first).

• promote — create a git branch pointing at an experiment's base commit, making it easy to share or build upon a successful experiment.

• gc — prune old experiment metadata beyond a configurable retention count. Orphan temporary directories are swept as well.

Subcommands

crab exp run

Execute a new experiment.

crab exp show

Display metadata for a specific experiment.

crab exp diff

Compare two experiments.

crab exp ls

List local experiments.

crab exp promote

Create a git branch from an experiment's base commit.

crab exp gc

Prune old experiment metadata.

Examples

Run an experiment with parameter overrides

crab exp run --set lr=0.01 --set epochs=50 --name "lower-lr-test"

Creates a throwaway worktree from HEAD, writes the parameter overrides into the declared params files, executes the full DAG, and persists the result metadata locally.

List recent experiments

crab exp ls --limit 5

Shows the five most recent experiments with their IDs, start times, stage counts, and base commits.

Compare two experiments

crab exp diff 019077a2-... 019077b3-...

Displays parameter, stage hash, and metric differences between the two experiments. Added, removed, and changed values are shown clearly.

Promote a successful experiment to a branch

crab exp promote 019077a2-... --branch feature/best-model

Creates a git branch feature/best-model pointing at the experiment's base commit, making it easy to share or continue work from that point.

Clean up old experiments

crab exp gc --keep 20

Removes metadata for all experiments beyond the 20 most recent. Orphan temporary directories are swept as well.

Preview garbage collection

crab exp gc --keep 10 --dry-run

Shows which experiments would be removed without actually deleting anything.

Prerequisites

• Workflow feature enabled with crab config set workflow.enabled true.
• A crab.yaml file at the repository root defining stages and metrics.
• A clean git HEAD (the experiment worktree branches from the current commit).

JSON Output

All subcommands support --json for structured output.

crab exp run --json

{
  "schema": "workflow.exp.run",
  "version": "1.0",
  "timestamp": "2026-06-01T10:15:42.000Z",
  "data": {
    "exp_id": "019077a2-1234-7000-8000-abcdef012345",
    "base_commit": "a7f3b2c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0",
    "stages": {
      "preprocess": "c4d5e6f7...",
      "train": "a1b2c3d4..."
    },
    "metrics_files": ["metrics/train.json"],
    "status": "success",
    "duration_ms": 34200,
    "started_at": "2026-06-01T10:15:08.000Z"
  }
}

crab exp show --json

{
  "schema": "workflow.exp.show",
  "version": "1.0",
  "timestamp": "2026-06-01T10:16:00.000Z",
  "data": {
    "metadata": {
      "exp_id": "019077a2-1234-7000-8000-abcdef012345",
      "base_commit": "a7f3b2c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0",
      "param_overrides": { "lr": "0.01", "epochs": "50" },
      "stages": { "preprocess": "c4d5e6f7...", "train": "a1b2c3d4..." },
      "metrics": { "metrics/train.json": { "loss": 0.023, "accuracy": 0.97 } },
      "started_at": "2026-06-01T10:15:08.000Z",
      "ended_at": "2026-06-01T10:15:42.000Z",
      "host_fingerprint": "macos-aarch64-crab-0.9.0"
    }
  }
}

crab exp diff --json

{
  "schema": "workflow.exp.diff",
  "version": "1.0",
  "timestamp": "2026-06-01T10:20:00.000Z",
  "data": {
    "id_a": "019077a2-...",
    "id_b": "019077b3-...",
    "params_added": { "warmup": "5" },
    "params_removed": {},
    "params_changed": { "lr": ["0.01", "0.001"] },
    "stages_changed": { "train": ["a1b2c3d4...", "e5f6a7b8..."] },
    "metrics_changed": { "metrics/train.json": [{ "loss": 0.023 }, { "loss": 0.019 }] }
  }
}

crab exp ls --json

{
  "schema": "workflow.exp.ls",
  "version": "1.0",
  "timestamp": "2026-06-01T10:20:00.000Z",
  "data": {
    "experiments": [
      {
        "id": "019077b3-...",
        "started_at": "2026-06-01T10:18:00.000Z",
        "base_commit": "a7f3b2c4d5e6...",
        "stages": 3,
        "metrics_keys": ["metrics/train.json"]
      }
    ]
  }
}

crab exp gc --json

{
  "schema": "workflow.exp.gc",
  "version": "1.0",
  "timestamp": "2026-06-01T10:25:00.000Z",
  "data": {
    "keep": 20,
    "dry_run": false,
    "removed": ["019070a1-...", "019070a0-..."],
    "kept": ["019077b3-...", "019077a2-..."]
  }
}

Related Commands

• crab run — execute workflow stages with content-addressed caching.
• crab exp-queue — queue experiments for batch execution.
• crab params — manage workflow parameters.
• crab metrics — track and query stage metrics.
• crab workflow — manage workflow definitions.
• crab dag — visualize the stage dependency graph.