Question

Somewhere along the line, some huge, huge file got added to the git repository of a project I own. When I go to clone this project on a new machine, the project appears to get “stuck” on 37% for a loooong time. This project should clone in just a couple of minutes.

How can I find out what object(s) are causing this lengthy cloning time?

I know how to “git rm” files. Will that remove it, even if it is some old object that only exists in the history?

I am not really clear if, once you “git rm” a file, if it is removed from the repository completely, or just going forward.

Any help is deeply appreciated!

Answer 1

Pro Git has a great, step-by-step explanation of how to do this. It takes a bit of work, but can be reliably done. I can’t take credit for the following explanation — I am just stealing it from that website (see the original website for a more detailed description):

1. Pack your repo:

   $ git gc
   

2. Find the largest items in the git database. The following command will list the three biggest ones, with the biggest one being the last line of output (the example below includes both the command you would type and also sample output):

   # In the following command, replace the pack*.idx filename
   # with whatever filename you find in the .git/objects/pack
   # directory:
   $ git verify-pack -v .git/objects/pack/pack-3f8c0...bb.idx | sort -k 3 -n | tail -3
   e3f094f522629ae358806b17daf78246c27c007b blob   1486 734 4667
   05408d195263d853f09dca71d55116663690c27c blob   12908 3478 1189
   7a9eb2fba2b1811321254ac360970fc169ba2330 blob   2056716 2056872 5401
   

3. Ask what filename is associated with that biggest blob:

   $ git rev-list --objects --all | grep 7a9eb2fb
   7a9eb2fba2b1811321254ac360970fc169ba2330 git.tbz2
   

4. Get the history of that file:

   $ git log --pretty=oneline -- git.tbz2
   da3f30d019005479c99eb4c3406225613985a1db oops - removed large tarball
   6df764092f3e7c8f5f94cbe08ee5cf42e92a0289 added git tarball
   

5. Use git filter-branch to remove all references to that file:

   $ git filter-branch --index-filter \
      'git rm --cached --ignore-unmatch git.tbz2' -- 6df7640^..
   Rewrite 6df764092f3e7c8f5f94cbe08ee5cf42e92a0289 (1/2)rm 'git.tbz2'
   Rewrite da3f30d019005479c99eb4c3406225613985a1db (2/2)
   Ref 'refs/heads/master' was rewritten
   

6. Clean up a few remaining references to this blob, and then gc again to re-pack:

   $ rm -Rf .git/refs/original
   $ rm -Rf .git/logs/
   $ git gc
   Counting objects: 19, done.
   Delta compression using 2 threads.
   Compressing objects: 100% (14/14), done.
   Writing objects: 100% (19/19), done.
   Total 19 (delta 3), reused 16 (delta 1)