Using Git LFS for Large Files

Introduction

Git is a powerful version control system that excels at managing text files, but it can struggle with large binary files like images, videos, and compiled binaries. This can lead to bloated repositories, slow cloning times, and increased storage costs. Git Large File Storage (LFS) is an extension of Git designed specifically to handle these challenges by moving large files out of the repository's history and storing them in a separate location.

In this tutorial, we'll explore how to use Git LFS to manage large files efficiently in your repositories. We'll cover installation, configuration, usage, and best practices for optimizing performance with Git LFS.

Prerequisites

Before you begin, ensure you have the following:

• A working knowledge of Git.
• Access to a GitHub account (or any other Git hosting service that supports LFS).
• Basic understanding of command-line operations.

Installing Git LFS

Git LFS is available as a plugin for Git. You can install it using the following commands:

# Install Git LFS globally
git lfs install

# Verify installation
git lfs version

This will set up Git LFS on your system, and you should see the version number of Git LFS installed.

Configuring Git LFS

Once Git LFS is installed, you need to configure it to track specific file types. This is done by specifying patterns for files that should be managed by LFS.

Tracking Large Files

To start tracking large files with Git LFS, use the git lfs track command followed by a pattern matching the file type:

# Track all .psd files
git lfs track "*.psd"

# Track all .mp4 video files
git lfs track "*.mp4"

This will create a .gitattributes file in your repository with entries like:

*.psd filter=lfs diff=lfs merge=lfs -text
*.mp4 filter=lfs diff=lfs merge=lfs -text

These entries tell Git to use LFS for files matching the specified patterns.

Committing Configuration

After configuring tracking, commit the .gitattributes file:

# Add .gitattributes to staging area
git add .gitattributes

# Commit changes
git commit -m "Track large files with Git LFS"

Using Git LFS

Now that you have configured Git LFS, you can start using it to manage your large files.

Adding Large Files

When you add a file that matches the patterns specified in .gitattributes, Git will automatically use LFS:

# Add a large PSD file
git add path/to/largefile.psd

# Commit changes
git commit -m "Add large PSD file"

Git LFS will handle the storage of this file separately from your repository history.

Fetching Large Files

When you clone or pull a repository that uses Git LFS, the large files are not downloaded by default. You need to fetch them using:

# Fetch all LFS objects
git lfs fetch --all

# Checkout the latest commit (this will download LFS files)
git checkout master

Alternatively, you can use git pull with LFS support:

# Pull changes and fetch LFS files
git pull origin master

Best Practices for Using Git LFS

1. Track Only Necessary Files

Only track the file types that are truly necessary to be managed by LFS. Overtracking can lead to unnecessary storage costs.

2. Use Specific Patterns

Instead of using a wildcard (*), specify more precise patterns in .gitattributes. For example, *.psd is better than assets/*.

3. Regularly Clean Up Unnecessary Files

Use git lfs prune to remove local copies of LFS files that are no longer needed:

# Prune unused LFS files
git lfs prune

4. Monitor Storage Usage

Keep an eye on your repository's storage usage on GitHub (or your Git hosting service). Large repositories can incur significant costs.

5. Use Branch-Specific Tracking

If certain branches require different large file tracking, you can specify branch-specific .gitattributes files:

# Create a .gitattributes file for the 'dev' branch
echo "*.mp4 filter=lfs diff=lfs merge=lfs -text" > dev/.gitattributes

# Add and commit the new .gitattributes file
git checkout dev
git add dev/.gitattributes
git commit -m "Configure LFS tracking for dev branch"

6. Avoid Large Files in Commit History

If you accidentally commit a large file without using LFS, you can rewrite history to correct it:

# Identify the commit with the large file
git rev-list --objects --all | grep $(shasum -s <largefile.psd | cut -d' ' -f1)

# Remove the file from the repository history
BLOB=$(git rev-list --objects --all | grep $(shasum -s <largefile.psd | cut -d' ' -f1) | awk '{print $1}')
git filter-branch --index-filter "git rm --cached --ignore-unmatch path/to/largefile.psd" HEAD

# Force push the changes
git push origin master --force

7. Use LFS for Binary Files Only

Git LFS is designed for binary files. Avoid using it for text files, as this can lead to unnecessary overhead.

Conclusion

Git LFS is a powerful tool for managing large files in Git repositories. By following the steps outlined in this tutorial and adhering to best practices, you can optimize your repository's performance, reduce storage costs, and streamline collaboration with your team.

Remember that while Git LFS simplifies the management of large files, it's essential to use it judiciously and monitor its impact on your workflow and repository size.