gitlab at trac.fzj
wiki:WikiStart

JSC Gitlab Server

https://gitlab.version.fz-juelich.de

Git Large File Storage (LFS)

LFS is a comfortable and efficient way to store large binaries.

Motivation

Without LFS

  • every git repository contains the complete history of all files including every version of big binaries
  • cloning such a repository requires to download all these binaries in all versions
  • bad performance of Git repositories larger than 1GB

With LFS

  • same git commands can be used, the LFS binaries are automatically downloaded
  • only pointers to the binary data are stored in the repository, the binaries ares stored on a separate disk on the server
  • only one version of large binaries will be downloaded, if the repository is cloned

On the JSC gitlab server, a single file may not exeed the size of 20Mb, otherwise the push will be rejected:

> git push origin master
Counting objects: 5, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 553 bytes | 0 bytes/s, done.
Total 5 (delta 3), reused 0 (delta 0)
remote: b0fd97d641cf2db0129faee2c4eeee768a9169ac..b6e4e93d5ec12229d42dd120548afe42183801d9 1048576
remote: 
remote: -------------------------------------------------------------------------------------------------------
remote: Your push was rejected because it contains files larger than 20 MB.
remote: You might want to consider using git-lfs instead (see https://trac.version.fz-juelich.de/gitlab/wiki/).
remote: -------------------------------------------------------------------------------------------------------
remote: 
remote: Offending files:
remote:  - bigFile.avi
remote: 
To ssh://git@gitlab.version.fz-juelich.de:10022/user/test.git
 ! [remote rejected] master -> master (pre-receive hook declined)

Please keep in mind, that deleting previously committed large file doesn't solve the problem. The file stays in history. You have to revert the change.

For future commits, you may want to install a pre-commit hook to prevent committing large files (see example).

Prequisites

  • check if git lfs client is installed git lfs version
    otherwise install lfs client
  • setup git lfs: git lfs install

First usage

This command specifies the files which should use LFS and also creates or updates .gitattributes.

git lfs track "*.bin" # or git lfs track myBigFile.bin

Important: Add .gitattributes to the repository.

git add .gitattributes

Now you may use git like before to add a file to the repository.

git add myBigFile.bin
git commit -m "Add binary file using lfs"
git push origin master

Further commands

If you have many LFS objects, cloning with git lfs clone is faster.

git lfs clone

If you have already checked out a repository and want to replace the object links (e.g. ExamplePointerFile) with the binary contents, use git lfs pull or

git lfs fetch # downloads objects with batch mode
git lfs checkout # changes objects to binary files

Show files which use LFS

git lfs ls-files

Documentation

Support

For further questions, send a email to version-admin.jsc@fz-juelich.de

Last modified 5 months ago Last modified on 03/30/17 09:02:45