JSC Gitlab Server
https://gitlab.version.fz-juelich.de
Git Large File Storage (LFS)
LFS is a comfortable and efficient way to store large binaries.
Motivation
Without LFS
- every git repository contains the complete history of all files including every version of big binaries
- cloning such a repository requires to download all these binaries in all versions
- bad performance of Git repositories larger than 1GB
With LFS
- same git commands can be used, the LFS binaries are automatically downloaded
- only pointers to the binary data are stored in the repository, the binaries ares stored on a separate disk on the server
- only one version of large binaries will be downloaded, if the repository is cloned
On the JSC gitlab server, a single file may not exeed the size of 20Mb, otherwise the push will be rejected:
> git push origin master Counting objects: 5, done. Delta compression using up to 8 threads. Compressing objects: 100% (5/5), done. Writing objects: 100% (5/5), 553 bytes | 0 bytes/s, done. Total 5 (delta 3), reused 0 (delta 0) remote: b0fd97d641cf2db0129faee2c4eeee768a9169ac..b6e4e93d5ec12229d42dd120548afe42183801d9 1048576 remote: remote: ------------------------------------------------------------------------------------------------------- remote: Your push was rejected because it contains files larger than 20 MB. remote: You might want to consider using git-lfs instead (see https://trac.version.fz-juelich.de/gitlab/wiki/). remote: ------------------------------------------------------------------------------------------------------- remote: remote: Offending files: remote: - bigFile.avi remote: To ssh://git@gitlab.version.fz-juelich.de:10022/user/test.git ! [remote rejected] master -> master (pre-receive hook declined)
Please keep in mind, that deleting previously committed large file doesn't solve the problem. The file stays in history. You have to revert the change.
For future commits, you may want to install a pre-commit hook to prevent committing large files (see example).
Prequisites
- check if git lfs client is installed
git lfs version
otherwise install lfs client - setup git lfs:
git lfs install
First usage
This command specifies the files which should use LFS and also creates or updates .gitattributes.
git lfs track "*.bin" # or git lfs track myBigFile.bin
Important: Add .gitattributes to the repository.
git add .gitattributes
Now you may use git like before to add a file to the repository.
git add myBigFile.bin git commit -m "Add binary file using lfs" git push origin master
Further commands
If you have many LFS objects, cloning with git lfs clone is faster.
git lfs clone
If you have already checked out a repository and want to replace the object links (e.g. ExamplePointerFile) with the binary contents, use git lfs pull or
git lfs fetch # downloads objects with batch mode git lfs checkout # changes objects to binary files
Show files which use LFS
git lfs ls-files
Documentation
Support
For further questions, send a email to version-admin.jsc@fz-juelich.de
