Yarn Zero-Installs and git-lfs

If you are using Yarn’s PnP Zero-Installs feature you may find your repository growing quite large due to the Yarn package cache being tracked. Sometimes this can produce a warning from your git hosting provider or even prevent you from making further pushes. Usually this can be fixed by migrating the cache to git-lfs so let’s see how to do this.

1. Install git-lfs

Head over to https://git-lfs.com/ and download the correct package for your system. Follow the usual process for installing on your particular operating system.

2. Clone Your Repository

We need to make a bare clone of the repository in order to move all the cached files to git LFS. Make sure you have committed and pushed all of your changes first. Also ensure nobody else who is working on this repository has any un-pushed changes.

Now from a terminal in an empty directory run the following command:

git clone --mirror <repo url>

Hint: You can usually find your repo url by navigating to your git provider’s web interface, finding your repository and finding the “Clone” link or similar.

When this is done you should now have a directory named the same as your repository.

3. Backup the Clone

Important: Make an exact copy of the cloned repository directory in case of issues further on.

4. Download BFG

Head over to https://rtyley.github.io/bfg-repo-cleaner/ and download the latest version of BFG Repo-Cleaner. This will be a Java jar file so make sure you have an up-to-date version of the Java runtime on your system.

5. Convert git History

Using BFG we are going to rewrite the git history to point all of the Yarn cached files to LFS. Run the following command:

java -jar <path to>bfg-x.x.x.jar --convert-to-git-lfs "*.zip" --no-blob-protection <repo-name>.git

Here we are telling BFG to convert all the zip file paths but you can add any other files you want to store in LFS by changing the pattern. For example to include mp4 videos you can use "*.{zip,mp4}"
Depending on the amount of files and the size of your repository this may take some time.

Note: Using --no-blob-protection will also update your HEAD commit. This is likely to break your build and should be avoided if you want to ensure a CI pipeline still works. You can omit this option and fix the repository by making a manual change and commit after Step 8.

When complete you should see something like the following at the end of the output:

BFG run is complete! When ready, run: git reflog expire --expire=now --all && git gc --prune=now --aggressive

We can now run the suggested command to clean up the repository.

cd <repo-name>.git
git reflog expire --expire=now --all && git gc --prune=now --aggressive

6. Set up git LFS

Next we have to tell the converted repository to use LFS. Run the following in a terminal:

cd <repo name>.git
git lfs install

You should see the following:

Updated git hooks.
Git LFS initialized.

7. Push Converted Repository

Now we can push the converted repository back to the remote. In the terminal run the following command:

git push

This may take a while depending on the amount and size of the files and your connection speed.

8. Clone Converted Repository

In a fresh directory make a new clone of the repository. This will replace any previous clone you may have had. e.g.

git clone <repo url>

Now we need to set up git LFS tracking of the same files we converted in step 5 so any new files will be tracked by git-lfs:

git lfs track "*.zip"

Note: Use the same file pattern as you used previously.

9. Informing Collaborators

Now that the repository history has been updated it is important that anybody else who has an existing clone deletes this and makes a new clone.

10. Remote Garbage Collection

If your remote repository does not do automatic garbage collection you will most likely find the repository size has increased. To get things back to the right size you will need to ensure git gc is run at the remote end also. For example, Bitbucket requires a support email as this is a manual process.