Structured Output (--json / --jsonl)

Crab supports machine-readable output for automation, scripting, and UI integration. Two modes are available:

• --json — emit a single JSON object with the full result.
• --jsonl — emit a stream of newline-delimited JSON objects (progress events followed by a terminal result).

Default (text) output is unchanged. Structured output is opt-in.

Mode Selection

--json and --jsonl are mutually exclusive. Where --porcelain exists (e.g. crab status), it also conflicts with --json.

For streaming commands (add, hydrate, push, etc.), --json suppresses all intermediate events and emits only the final result.

Envelope Format

Every --json response is wrapped in a consistent envelope:

{
  "schema": "status",
  "version": "1.0",
  "timestamp": "2026-04-24T18:32:17.123Z",
  "data": {
    "total_tracked": 1420,
    "hydrated": 312,
    "pointer": 1108,
    "modified": 2,
    "files": [
      { "path": "models/bert.bin", "state": "hydrated", "bytes": 440401920 }
    ]
  }
}

Error Envelope

When a command fails in --json or --jsonl mode, the envelope contains error instead of data. The process exits with a non-zero code.

{
  "schema": "hydrate",
  "version": "1.0",
  "timestamp": "2026-04-24T18:32:17.456Z",
  "error": {
    "code": "CRAB-E0001",
    "category": "transient",
    "message": "network transient error: connection reset by peer",
    "retryable": true,
    "details": {},
    "source_chain": [
      { "message": "connection reset by peer" },
      { "message": "broken pipe" }
    ]
  }
}

Error Fields

Error Categories

Exit Codes

Exit codes are preserved from the existing CrabError::exit_code() mapping:

JSONL Streaming (--jsonl)

Long-running commands emit a stream of newline-delimited JSON objects. Each line is a complete JSON object (ndjson / RFC 7464). No trailing commas, no wrapping array.

Event Envelope

{
  "schema": "add.event",
  "version": "1.0",
  "timestamp": "2026-04-24T18:32:17.123Z",
  "type": "file_done",
  "data": {
    "path": "data/train.parquet",
    "bytes": 52428800,
    "duration_ms": 340,
    "status": "ok"
  }
}

The schema field uses the .event suffix (e.g. "hydrate.event", "push.event"). The type field identifies the event kind.

Event Types

The final event is always type: "result". Its data field matches what --json would emit for the same operation.

Example: crab add --jsonl

{"schema":"add.event","version":"1.0","timestamp":"2026-04-24T18:32:17.100Z","type":"progress","data":{"operation":"staging","current":3,"total":10,"bytes":15728640,"total_bytes":52428800,"rate_bytes_per_sec":45000000.0}}
{"schema":"add.event","version":"1.0","timestamp":"2026-04-24T18:32:17.450Z","type":"file_done","data":{"path":"data/train.parquet","bytes":52428800,"duration_ms":340,"status":"ok"}}
{"schema":"add.event","version":"1.0","timestamp":"2026-04-24T18:32:17.460Z","type":"file_done","data":{"path":"data/val.parquet","bytes":10485760,"duration_ms":120,"status":"ok"}}
{"schema":"add.event","version":"1.0","timestamp":"2026-04-24T18:32:17.500Z","type":"result","data":{"files_staged":10,"bytes_staged":524288000,"chunks_new":42,"chunks_deduped":8}}

Rate Limiting

progress events are rate-limited to at most one per 250ms (wall-clock). Completion events (file_done, xorb_done, warning, result) are never rate-limited. No heartbeat is emitted during idle periods.

Cancellation

On SIGINT, the command attempts to emit a final result event with "status": "cancelled" and error code CRAB-E0090 before exiting with code 10. A second SIGINT forces immediate exit and may truncate the stream. Consumers should tolerate incomplete final lines.

Command Table

Every command's structured output support at a glance.

Read Commands (--json)

Streaming Commands (--json and --jsonl)

These commands accept both flags. --json emits only the terminal result; --jsonl emits the full event stream.

Remote Helper Exception

The git-remote-crab remote helper (invoked by git push) cannot use stdout for structured output because git owns that channel. Instead, set:

export CRAB_PROGRESS_FORMAT=jsonl

When this env var is set, the remote helper emits JSONL events to stderr. The event schema is identical to crab push --jsonl (push.event).

UI applications spawning git push should read JSONL from the child's stderr. Applications spawning crab push directly should read from stdout as usual.

Commands Without Structured Output

The following are excluded from structured output:

• Protocol binaries: filter-process, diff-driver, lfs-transfer-agent (git owns their stdout)
• Stub commands: cache stats, sync refs, bench (not yet implemented)
• Daemon control: mount, unmount, daemon (foreground), daemon enable/disable/fetch/remount
• One-shot actions: install, uninstall, logs clear

Schema Versioning

Each command's output has an independent version string ("<major>.<minor>").

• Minor bump (e.g. 1.0 → 1.1): new optional fields added. Existing consumers are unaffected.
• Major bump (e.g. 1.1 → 2.0): fields removed, renamed, or type changed. Consumers must update.

JSON Schemas (draft-07) for every payload type are committed to crab/schemas/. They are generated from Rust types via schemars — not hand-written. A CI test asserts that emitted output validates against the committed schemas; drift fails the build.

To see all schema names and versions supported by your binary:

crab version --json

{
  "schema": "version",
  "version": "1.0",
  "timestamp": "2026-04-24T18:32:17.123Z",
  "data": {
    "crab_version": "0.15.0",
    "git_sha": "abc1234",
    "build_timestamp": "2026-04-24T12:00:00Z",
    "schemas": {
      "add": "1.0",
      "add.event": "1.0",
      "du": "1.1",
      "hydrate": "1.0",
      "status": "1.0"
    }
  }
}

Version changes are recorded in structured-output-changelog.md.

Quick Examples

Parse a single result

crab du --json | jq '.data.total_bytes'

Stream progress in a script

crab add --jsonl *.bin | while IFS= read -r line; do
  type=$(echo "$line" | jq -r '.type')
  case "$type" in
    progress)  echo "$(echo "$line" | jq -r '.data.current')/$(echo "$line" | jq -r '.data.total')" ;;
    file_done) echo "Done: $(echo "$line" | jq -r '.data.path')" ;;
    result)    echo "Finished: $(echo "$line" | jq '.data')" ;;
  esac
done

Handle errors programmatically

if ! output=$(crab hydrate --all --json 2>/dev/null); then
  code=$(echo "$output" | jq -r '.error.code')
  retryable=$(echo "$output" | jq -r '.error.retryable')
  if [ "$retryable" = "true" ]; then
    echo "Retryable error $code, trying again..."
    crab hydrate --all --json
  else
    echo "Fatal error: $code"
    exit 1
  fi
fi