How does Crab LFS work without a server?
Use standard Git LFS pointers with Crab's local transfer agent. Your clients talk directly to your bucket; no Git LFS server or gateway is required.
Crab LFS is the “I need the Git LFS contract, not a Git LFS server” path. Git keeps the normal LFS pointer in the commit. Crab runs locally on each developer machine or CI runner and reads or writes the object-storage bucket directly.
DIRECT STORAGE LAB / CLICK A BOUNDARY
One LFS file. No gateway.
Crab runs on the developer machine and talks to the bucket. The Git LFS pointer remains standard so existing tooling can keep reading it.
Git receives a pointer, not the 8 GB file
YOUR MACHINE
WORKTREE
full file bytes
GIT INDEX
LFS pointer · ~130 B
LOCAL CACHE
.git/lfs/objects/<oid>
YOUR BUCKET
BUCKET
no network write yet
GATEWAY
not required
$ git add models/encoder.safetensorsCrab's LFS clean filter keeps the full file in your worktree, writes a standard LFS pointer into Git, and saves the bytes in the local LFS cache.
Start with the one important choice
Large does not automatically mean LFS. Choose the pointer format that your repository and tools actually need:
| Need | Tracking command | Pointer in Git | Storage shape |
|---|---|---|---|
| Existing LFS history or standard LFS tooling | crab lfs track | Git LFS pointer | One whole-file LFS object per version |
| New Crab workflow with internal chunk reuse | crab track | Crab pointer | Reusable chunks, xorbs, and shards |
| Source code, config, or small text | No tracking rule | Normal Git blob | Git object database |
The two large-file modes can coexist when their path rules do not overlap:
models/releases/** filter=lfs diff=lfs merge=lfs -text
datasets/** filter=crab diff=crab merge=crab -textUse Crab LFS when a repository already contains lines like these:
version https://git-lfs.github.com/spec/v1
oid sha256:91ae...b72c
size 8589934592Use native Crab when you want content-defined chunking and deduplication across versions. Choosing LFS is a compatibility decision, not a requirement imposed by file size.
Configure the direct route
Run the setup once in a new or existing Git repository:
crab init --storage-provider s3 crab://team-bucket/model-repository
crab lfs install --local
crab lfs track '*.safetensors'
git add crab.toml .gitattributes
git commit -m "Configure direct Crab LFS"
git push origin mainThe URL names a bucket and repository prefix. The client uses your normal provider credential chain; never put credentials in the URL or committed configuration.
- 1Install Crab and grant the client scoped access to the bucket prefix.
- 2Track only the paths that must remain standard Git LFS pointers.
- 3Push through Git; Crab uploads the object before the ref advances.
- 4Clone the history, pull the object, and verify its SHA-256 identity.
crab lfs install configures:
- Crab's clean, smudge, and filter-process commands for
filter=lfs. - The
crabstandalone transfer agent for an unmodified Git LFS client. - A pre-push hook that uploads missing LFS objects before Git publishes the commit ref.
The hook and transfer settings live in local Git configuration. The
.gitattributes rule belongs in the repository and must be
committed.
The daily loop looks like Git
Once installed, contributors can use familiar commands:
cp /path/to/encoder-v2.safetensors models/encoder.safetensors
git add models/encoder.safetensors
git commit -m "Update encoder"
git push origin mainAt git add, the worktree keeps the full bytes while Git receives a
small pointer. At git push, Crab's local hook or transfer agent
uploads the missing object directly to the bucket. Git history still contains
the standard LFS pointer, so existing pointer-aware tools can inspect it.
For explicit operations and diagnostics:
crab lfs status
crab lfs push origin main
crab lfs fsck HEADThe direct LFS object is intentionally a whole-file object. If a model changes, the new pointer names a new object. Crab LFS preserves compatibility; native Crab is the choice when chunk-level reuse is the priority.
A collaborator can pull only what they need
Install the integration after cloning, then keep large files as pointers until the task needs them:
git clone crab://team-bucket/model-repository model-repository
cd model-repository
crab lfs install --local --skip-smudge
crab lfs fetch origin --include 'models/**'
crab lfs checkout 'models/**'If Git LFS is installed, this is the equivalent standard retrieval command:
git lfs pull origin mainIf Git LFS is not installed, use:
crab lfs pull originThe result is the same: a pointer in Git history, a verified object in the local LFS cache, and materialized bytes in the worktree.
Prove it from an empty cache
Do not treat a successful upload progress bar as the whole proof. A useful pilot crosses both write and read boundaries:
crab lfs env
git check-attr filter -- models/encoder.safetensors
git show HEAD:models/encoder.safetensors
crab lfs fsck HEADThen use a disposable bucket prefix and a separate clone:
- Add a representative model or dataset and confirm the staged Git object is a small LFS pointer.
- Push with
git lfs pushorcrab lfs push. - Confirm the object exists under the repository's
lfs/objects/prefix. - Clone with an empty local cache and run
git lfs pullorcrab lfs pull. - Compare the recovered file's SHA-256 with the pointer's OID.
- Run
crab lfs fsck HEADon the reader.
A clean clone proves more than local status: it checks Git pointer selection, credentials, remote object discovery, transfer, checkout, and byte identity.
What Crab LFS does not require
You do not need to:
- deploy a Git LFS HTTP server;
- configure an LFS Batch API endpoint;
- run a Crab gateway or repository database;
- rewrite existing commits just to change the transfer route.
You do need Crab installed on every client that pushes or materializes these
LFS paths, and each client needs permission to the repository prefix. The
supported direct route is a crab://bucket/repository remote plus
Crab's local standalone transfer agent.
If you want to convert existing LFS pointers into native Crab pointers, that is a separate history-rewrite decision. Read How do you move a Git LFS repository to Crab? after this guide.
KNOWLEDGE PROOF
Check the decision, not your memory.
Where does a Crab LFS client send the large-file object?