Comparing Changes
Traditional git diff needs full file content to show byte-level differences. For a 10 GB model file, that can require reconstructing both versions before comparison. Crab instead compares tracked files at the chunk level from metadata, so the file payloads do not need to be downloaded for the summary.
Why Chunk-Level Diffs?
When Crab stores a file, it splits it into content-defined chunks. Each chunk has a hash. To compare two versions of a file, Crab only needs to compare their chunk lists — no need to download the actual data.
This means you can:
- Compare multi-gigabyte files from metadata: Report changed chunks and byte ranges without reconstructing both payloads.
- See exactly what changed — Know which byte ranges were modified, added, or removed.
- Work offline — Diffs use cached metadata, not file content.
- Understand dedup efficiency — See how many chunks are shared between versions.
How It Works
When you run a diff between two refs, Crab:
- Resolves both refs to their tree objects.
- Finds Crab-tracked files that differ between the two trees.
- Parses the pointer blobs to get file hashes and shard hints.
- Resolves chunk lists from shard metadata (lightweight, cached locally).
- Compares chunk lists to identify added, removed, and modified segments.
- Reports the differences without ever downloading file content.
Common Usage
Compare with the previous commit
crab diff HEAD~1Compare two branches
crab diff main feature/new-modelCompare two tags (release comparison)
crab diff v1.0 v2.0Summary view (like git diff --stat)
crab diff --stat HEAD~3 models/weights.bin | 423 chunks changed (+312, -111), +1.2 GB
data/train.bin | 12 chunks changed (+12, -0), +45 MB
2 files changed, 435 segments, +1.245 GB deltaShow file names only
crab diff --name-only v1.0 v2.0Restrict to specific paths
crab diff HEAD~1 -- models/Understanding the Output
The default output shows:
- Chunks changed — How many chunks differ between versions
- Chunks added/removed — Net change in chunk count
- Bytes delta — Approximate size difference
With --verbose, you also see individual xorb hashes and chunk ranges. With --byte-ranges, you see the exact byte offsets within each file that changed.
| View | Best for | Does not prove |
|---|---|---|
| Default | Changed chunks and approximate byte impact | Semantic meaning of binary changes |
--stat | Review summaries and release sizing | Exact changed offsets |
--name-only | Selecting files for another command | Amount of data changed |
--byte-ranges | Locating changed regions | That either full file is locally hydrated |
--verbose | Storage and reconstruction diagnosis | Application-level correctness |
Use Cases
Code review for data changes
Before merging a PR that updates training data:
crab diff main feature/updated-dataset --statUse the summary to estimate how much file data changed without downloading the payloads.
Release notes
Compare what changed between releases:
crab diff v1.0 v2.0 --name-onlyDebugging hydration issues
If a file hydrates differently than expected, compare the chunk lists:
crab diff HEAD~1 --verbose -- problematic-file.binCLI Reference
For complete command syntax, all output modes, and JSON format, see the crab diff reference.