git/t/t3210-pack-refs.sh

215 lines
5.7 KiB
Bash
Raw Normal View History

#!/bin/sh
#
# Copyright (c) 2005 Amos Waterland
# Copyright (c) 2006 Christian Couder
#
test_description='git pack-refs should not change the branch semantic
This test runs git pack-refs and git show-ref and checks that the branch
semantic is still the same.
'
. ./test-lib.sh
refs: speed up is_refname_available Our filesystem ref storage does not allow D/F conflicts; so if "refs/heads/a/b" exists, we do not allow "refs/heads/a" to exist (and vice versa). This falls out naturally for loose refs, where the filesystem enforces the condition. But for packed-refs, we have to make the check ourselves. We do so by iterating over the entire packed-refs namespace and checking whether each name creates a conflict. If you have a very large number of refs, this is quite inefficient, as you end up doing a large number of comparisons with uninteresting bits of the ref tree (e.g., we know that all of "refs/tags" is uninteresting in the example above, yet we check each entry in it). Instead, let's take advantage of the fact that we have the packed refs stored as a trie of ref_entry structs. We can find each component of the proposed refname as we walk through the trie, checking for D/F conflicts as we go. For a refname of depth N (i.e., 4 in the above example), we only have to visit N nodes. And at each visit, we can binary search the M names at that level, for a total complexity of O(N lg M). ("M" is different at each level, of course, but we can take the worst-case "M" as a bound). In a pathological case of fetching 30,000 fresh refs into a repository with 8.5 million refs, this dropped the time to run "git fetch" from tens of minutes to ~30s. This may also help smaller cases in which we check against loose refs (which we do when renaming a ref), as we may avoid a disk access for unrelated loose directories. Note that the tests we add appear at first glance to be redundant with what is already in t3210. However, the early tests are not robust; they are run with reflogs turned on, meaning that we are not actually testing is_refname_available at all! The operations will still fail because the reflogs will hit D/F conflicts in the filesystem. To get a true test, we must turn off reflogs (but we don't want to do so for the entire script, because the point of turning them on was to cover some other cases). Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-10 11:11:55 +00:00
test_expect_success 'enable reflogs' '
git config core.logallrefupdates true
'
test_expect_success \
'prepare a trivial repository' \
'echo Hello > A &&
git update-index --add A &&
git commit -m "Initial commit." &&
HEAD=$(git rev-parse --verify HEAD)'
SHA1=
test_expect_success \
'see if git show-ref works as expected' \
'git branch a &&
SHA1=$(cat .git/refs/heads/a) &&
echo "$SHA1 refs/heads/a" >expect &&
git show-ref a >result &&
test_cmp expect result'
test_expect_success \
'see if a branch still exists when packed' \
'git branch b &&
git pack-refs --all &&
rm -f .git/refs/heads/b &&
echo "$SHA1 refs/heads/b" >expect &&
git show-ref b >result &&
test_cmp expect result'
Sane use of test_expect_failure Originally, test_expect_failure was designed to be the opposite of test_expect_success, but this was a bad decision. Most tests run a series of commands that leads to the single command that needs to be tested, like this: test_expect_{success,failure} 'test title' ' setup1 && setup2 && setup3 && what is to be tested ' And expecting a failure exit from the whole sequence misses the point of writing tests. Your setup$N that are supposed to succeed may have failed without even reaching what you are trying to test. The only valid use of test_expect_failure is to check a trivial single command that is expected to fail, which is a minority in tests of Porcelain-ish commands. This large-ish patch rewrites all uses of test_expect_failure to use test_expect_success and rewrites the condition of what is tested, like this: test_expect_success 'test title' ' setup1 && setup2 && setup3 && ! this command should fail ' test_expect_failure is redefined to serve as a reminder that that test *should* succeed but due to a known breakage in git it currently does not pass. So if git-foo command should create a file 'bar' but you discovered a bug that it doesn't, you can write a test like this: test_expect_failure 'git-foo should create bar' ' rm -f bar && git foo && test -f bar ' This construct acts similar to test_expect_success, but instead of reporting "ok/FAIL" like test_expect_success does, the outcome is reported as "FIXED/still broken". Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-01 09:50:53 +00:00
test_expect_success 'git branch c/d should barf if branch c exists' '
git branch c &&
git pack-refs --all &&
Sane use of test_expect_failure Originally, test_expect_failure was designed to be the opposite of test_expect_success, but this was a bad decision. Most tests run a series of commands that leads to the single command that needs to be tested, like this: test_expect_{success,failure} 'test title' ' setup1 && setup2 && setup3 && what is to be tested ' And expecting a failure exit from the whole sequence misses the point of writing tests. Your setup$N that are supposed to succeed may have failed without even reaching what you are trying to test. The only valid use of test_expect_failure is to check a trivial single command that is expected to fail, which is a minority in tests of Porcelain-ish commands. This large-ish patch rewrites all uses of test_expect_failure to use test_expect_success and rewrites the condition of what is tested, like this: test_expect_success 'test title' ' setup1 && setup2 && setup3 && ! this command should fail ' test_expect_failure is redefined to serve as a reminder that that test *should* succeed but due to a known breakage in git it currently does not pass. So if git-foo command should create a file 'bar' but you discovered a bug that it doesn't, you can write a test like this: test_expect_failure 'git-foo should create bar' ' rm -f bar && git foo && test -f bar ' This construct acts similar to test_expect_success, but instead of reporting "ok/FAIL" like test_expect_success does, the outcome is reported as "FIXED/still broken". Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-01 09:50:53 +00:00
rm -f .git/refs/heads/c &&
test_must_fail git branch c/d
Sane use of test_expect_failure Originally, test_expect_failure was designed to be the opposite of test_expect_success, but this was a bad decision. Most tests run a series of commands that leads to the single command that needs to be tested, like this: test_expect_{success,failure} 'test title' ' setup1 && setup2 && setup3 && what is to be tested ' And expecting a failure exit from the whole sequence misses the point of writing tests. Your setup$N that are supposed to succeed may have failed without even reaching what you are trying to test. The only valid use of test_expect_failure is to check a trivial single command that is expected to fail, which is a minority in tests of Porcelain-ish commands. This large-ish patch rewrites all uses of test_expect_failure to use test_expect_success and rewrites the condition of what is tested, like this: test_expect_success 'test title' ' setup1 && setup2 && setup3 && ! this command should fail ' test_expect_failure is redefined to serve as a reminder that that test *should* succeed but due to a known breakage in git it currently does not pass. So if git-foo command should create a file 'bar' but you discovered a bug that it doesn't, you can write a test like this: test_expect_failure 'git-foo should create bar' ' rm -f bar && git foo && test -f bar ' This construct acts similar to test_expect_success, but instead of reporting "ok/FAIL" like test_expect_success does, the outcome is reported as "FIXED/still broken". Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-01 09:50:53 +00:00
'
test_expect_success \
'see if a branch still exists after git pack-refs --prune' \
'git branch e &&
git pack-refs --all --prune &&
echo "$SHA1 refs/heads/e" >expect &&
git show-ref e >result &&
test_cmp expect result'
Sane use of test_expect_failure Originally, test_expect_failure was designed to be the opposite of test_expect_success, but this was a bad decision. Most tests run a series of commands that leads to the single command that needs to be tested, like this: test_expect_{success,failure} 'test title' ' setup1 && setup2 && setup3 && what is to be tested ' And expecting a failure exit from the whole sequence misses the point of writing tests. Your setup$N that are supposed to succeed may have failed without even reaching what you are trying to test. The only valid use of test_expect_failure is to check a trivial single command that is expected to fail, which is a minority in tests of Porcelain-ish commands. This large-ish patch rewrites all uses of test_expect_failure to use test_expect_success and rewrites the condition of what is tested, like this: test_expect_success 'test title' ' setup1 && setup2 && setup3 && ! this command should fail ' test_expect_failure is redefined to serve as a reminder that that test *should* succeed but due to a known breakage in git it currently does not pass. So if git-foo command should create a file 'bar' but you discovered a bug that it doesn't, you can write a test like this: test_expect_failure 'git-foo should create bar' ' rm -f bar && git foo && test -f bar ' This construct acts similar to test_expect_success, but instead of reporting "ok/FAIL" like test_expect_success does, the outcome is reported as "FIXED/still broken". Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-01 09:50:53 +00:00
test_expect_success 'see if git pack-refs --prune remove ref files' '
git branch f &&
git pack-refs --all --prune &&
Sane use of test_expect_failure Originally, test_expect_failure was designed to be the opposite of test_expect_success, but this was a bad decision. Most tests run a series of commands that leads to the single command that needs to be tested, like this: test_expect_{success,failure} 'test title' ' setup1 && setup2 && setup3 && what is to be tested ' And expecting a failure exit from the whole sequence misses the point of writing tests. Your setup$N that are supposed to succeed may have failed without even reaching what you are trying to test. The only valid use of test_expect_failure is to check a trivial single command that is expected to fail, which is a minority in tests of Porcelain-ish commands. This large-ish patch rewrites all uses of test_expect_failure to use test_expect_success and rewrites the condition of what is tested, like this: test_expect_success 'test title' ' setup1 && setup2 && setup3 && ! this command should fail ' test_expect_failure is redefined to serve as a reminder that that test *should* succeed but due to a known breakage in git it currently does not pass. So if git-foo command should create a file 'bar' but you discovered a bug that it doesn't, you can write a test like this: test_expect_failure 'git-foo should create bar' ' rm -f bar && git foo && test -f bar ' This construct acts similar to test_expect_success, but instead of reporting "ok/FAIL" like test_expect_success does, the outcome is reported as "FIXED/still broken". Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-01 09:50:53 +00:00
! test -f .git/refs/heads/f
'
test_expect_success 'see if git pack-refs --prune removes empty dirs' '
git branch r/s/t &&
git pack-refs --all --prune &&
! test -e .git/refs/heads/r
'
test_expect_success \
'git branch g should work when git branch g/h has been deleted' \
'git branch g/h &&
git pack-refs --all --prune &&
git branch -d g/h &&
git branch g &&
git pack-refs --all &&
git branch -d g'
Sane use of test_expect_failure Originally, test_expect_failure was designed to be the opposite of test_expect_success, but this was a bad decision. Most tests run a series of commands that leads to the single command that needs to be tested, like this: test_expect_{success,failure} 'test title' ' setup1 && setup2 && setup3 && what is to be tested ' And expecting a failure exit from the whole sequence misses the point of writing tests. Your setup$N that are supposed to succeed may have failed without even reaching what you are trying to test. The only valid use of test_expect_failure is to check a trivial single command that is expected to fail, which is a minority in tests of Porcelain-ish commands. This large-ish patch rewrites all uses of test_expect_failure to use test_expect_success and rewrites the condition of what is tested, like this: test_expect_success 'test title' ' setup1 && setup2 && setup3 && ! this command should fail ' test_expect_failure is redefined to serve as a reminder that that test *should* succeed but due to a known breakage in git it currently does not pass. So if git-foo command should create a file 'bar' but you discovered a bug that it doesn't, you can write a test like this: test_expect_failure 'git-foo should create bar' ' rm -f bar && git foo && test -f bar ' This construct acts similar to test_expect_success, but instead of reporting "ok/FAIL" like test_expect_success does, the outcome is reported as "FIXED/still broken". Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-01 09:50:53 +00:00
test_expect_success 'git branch i/j/k should barf if branch i exists' '
git branch i &&
git pack-refs --all --prune &&
test_must_fail git branch i/j/k
Sane use of test_expect_failure Originally, test_expect_failure was designed to be the opposite of test_expect_success, but this was a bad decision. Most tests run a series of commands that leads to the single command that needs to be tested, like this: test_expect_{success,failure} 'test title' ' setup1 && setup2 && setup3 && what is to be tested ' And expecting a failure exit from the whole sequence misses the point of writing tests. Your setup$N that are supposed to succeed may have failed without even reaching what you are trying to test. The only valid use of test_expect_failure is to check a trivial single command that is expected to fail, which is a minority in tests of Porcelain-ish commands. This large-ish patch rewrites all uses of test_expect_failure to use test_expect_success and rewrites the condition of what is tested, like this: test_expect_success 'test title' ' setup1 && setup2 && setup3 && ! this command should fail ' test_expect_failure is redefined to serve as a reminder that that test *should* succeed but due to a known breakage in git it currently does not pass. So if git-foo command should create a file 'bar' but you discovered a bug that it doesn't, you can write a test like this: test_expect_failure 'git-foo should create bar' ' rm -f bar && git foo && test -f bar ' This construct acts similar to test_expect_success, but instead of reporting "ok/FAIL" like test_expect_success does, the outcome is reported as "FIXED/still broken". Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-01 09:50:53 +00:00
'
test_expect_success \
'test git branch k after branch k/l/m and k/lm have been deleted' \
'git branch k/l &&
git branch k/lm &&
git branch -d k/l &&
git branch k/l/m &&
git branch -d k/l/m &&
git branch -d k/lm &&
git branch k'
test_expect_success \
'test git branch n after some branch deletion and pruning' \
'git branch n/o &&
git branch n/op &&
git branch -d n/o &&
git branch n/o/p &&
git branch -d n/op &&
git pack-refs --all --prune &&
git branch -d n/o/p &&
git branch n'
test_expect_success \
'see if up-to-date packed refs are preserved' \
'git branch q &&
git pack-refs --all --prune &&
git update-ref refs/heads/q refs/heads/q &&
! test -f .git/refs/heads/q'
test_expect_success 'pack, prune and repack' '
git tag foo &&
git pack-refs --all --prune &&
git show-ref >all-of-them &&
git pack-refs &&
git show-ref >again &&
test_cmp all-of-them again
'
test_expect_success 'explicit pack-refs with dangling packed reference' '
git commit --allow-empty -m "soon to be garbage-collected" &&
git pack-refs --all &&
git reset --hard HEAD^ &&
git reflog expire --expire=all --all &&
git prune --expire=all &&
git pack-refs --all 2>result &&
test_cmp /dev/null result
'
test_expect_success 'delete ref with dangling packed version' '
git checkout -b lamb &&
git commit --allow-empty -m "future garbage" &&
git pack-refs --all &&
git reset --hard HEAD^ &&
git checkout master &&
git reflog expire --expire=all --all &&
git prune --expire=all &&
git branch -d lamb 2>result &&
test_cmp /dev/null result
'
repack_without_ref(): silence errors for dangling packed refs Stop emitting an error message when deleting a packed reference if we find another dangling packed reference that is overridden by a loose reference. See the previous commit for a longer explanation of the issue. We have to be careful to make sure that the invalid packed reference really *is* overridden by a loose reference; otherwise what we have found is repository corruption, which we *should* report. Please note that this approach is vulnerable to a race condition similar to the race conditions already known to affect packed references [1]: * Process 1 tries to peel packed reference X as part of deleting another packed reference. It discovers that X does not refer to a valid object (because the object that it referred to has been garbage collected). * Process 2 tries to delete reference X. It starts by deleting the loose reference X. * Process 1 checks whether there is a loose reference X. There is not (it has just been deleted by process 2), so process 1 reports a spurious error "X does not point to a valid object!" The worst case seems relatively harmless, and the fix is identical to the fix that will be needed for the other race conditions (namely holding a lock on the packed-refs file during *all* reference deletions), so we leave the cleaning up of all of them as a future project. [1] http://thread.gmane.org/gmane.comp.version-control.git/211956 Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-22 19:52:25 +00:00
test_expect_success 'delete ref while another dangling packed ref' '
git branch lamb &&
git commit --allow-empty -m "future garbage" &&
git pack-refs --all &&
git reset --hard HEAD^ &&
git reflog expire --expire=all --all &&
git prune --expire=all &&
git branch -d lamb 2>result &&
test_cmp /dev/null result
'
test_expect_success 'pack ref directly below refs/' '
git update-ref refs/top HEAD &&
git pack-refs --all --prune &&
grep refs/top .git/packed-refs &&
test_path_is_missing .git/refs/top
'
test_expect_success 'do not pack ref in refs/bisect' '
git update-ref refs/bisect/local HEAD &&
git pack-refs --all --prune &&
! grep refs/bisect/local .git/packed-refs >/dev/null &&
test_path_is_file .git/refs/bisect/local
'
refs: speed up is_refname_available Our filesystem ref storage does not allow D/F conflicts; so if "refs/heads/a/b" exists, we do not allow "refs/heads/a" to exist (and vice versa). This falls out naturally for loose refs, where the filesystem enforces the condition. But for packed-refs, we have to make the check ourselves. We do so by iterating over the entire packed-refs namespace and checking whether each name creates a conflict. If you have a very large number of refs, this is quite inefficient, as you end up doing a large number of comparisons with uninteresting bits of the ref tree (e.g., we know that all of "refs/tags" is uninteresting in the example above, yet we check each entry in it). Instead, let's take advantage of the fact that we have the packed refs stored as a trie of ref_entry structs. We can find each component of the proposed refname as we walk through the trie, checking for D/F conflicts as we go. For a refname of depth N (i.e., 4 in the above example), we only have to visit N nodes. And at each visit, we can binary search the M names at that level, for a total complexity of O(N lg M). ("M" is different at each level, of course, but we can take the worst-case "M" as a bound). In a pathological case of fetching 30,000 fresh refs into a repository with 8.5 million refs, this dropped the time to run "git fetch" from tens of minutes to ~30s. This may also help smaller cases in which we check against loose refs (which we do when renaming a ref), as we may avoid a disk access for unrelated loose directories. Note that the tests we add appear at first glance to be redundant with what is already in t3210. However, the early tests are not robust; they are run with reflogs turned on, meaning that we are not actually testing is_refname_available at all! The operations will still fail because the reflogs will hit D/F conflicts in the filesystem. To get a true test, we must turn off reflogs (but we don't want to do so for the entire script, because the point of turning them on was to cover some other cases). Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-10 11:11:55 +00:00
test_expect_success 'disable reflogs' '
git config core.logallrefupdates false &&
rm -rf .git/logs
'
test_expect_success 'create packed foo/bar/baz branch' '
git branch foo/bar/baz &&
git pack-refs --all --prune &&
test_path_is_missing .git/refs/heads/foo/bar/baz &&
test_must_fail git reflog exists refs/heads/foo/bar/baz
refs: speed up is_refname_available Our filesystem ref storage does not allow D/F conflicts; so if "refs/heads/a/b" exists, we do not allow "refs/heads/a" to exist (and vice versa). This falls out naturally for loose refs, where the filesystem enforces the condition. But for packed-refs, we have to make the check ourselves. We do so by iterating over the entire packed-refs namespace and checking whether each name creates a conflict. If you have a very large number of refs, this is quite inefficient, as you end up doing a large number of comparisons with uninteresting bits of the ref tree (e.g., we know that all of "refs/tags" is uninteresting in the example above, yet we check each entry in it). Instead, let's take advantage of the fact that we have the packed refs stored as a trie of ref_entry structs. We can find each component of the proposed refname as we walk through the trie, checking for D/F conflicts as we go. For a refname of depth N (i.e., 4 in the above example), we only have to visit N nodes. And at each visit, we can binary search the M names at that level, for a total complexity of O(N lg M). ("M" is different at each level, of course, but we can take the worst-case "M" as a bound). In a pathological case of fetching 30,000 fresh refs into a repository with 8.5 million refs, this dropped the time to run "git fetch" from tens of minutes to ~30s. This may also help smaller cases in which we check against loose refs (which we do when renaming a ref), as we may avoid a disk access for unrelated loose directories. Note that the tests we add appear at first glance to be redundant with what is already in t3210. However, the early tests are not robust; they are run with reflogs turned on, meaning that we are not actually testing is_refname_available at all! The operations will still fail because the reflogs will hit D/F conflicts in the filesystem. To get a true test, we must turn off reflogs (but we don't want to do so for the entire script, because the point of turning them on was to cover some other cases). Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-10 11:11:55 +00:00
'
test_expect_success 'notice d/f conflict with existing directory' '
test_must_fail git branch foo &&
test_must_fail git branch foo/bar
'
test_expect_success 'existing directory reports concrete ref' '
test_must_fail git branch foo 2>stderr &&
grep refs/heads/foo/bar/baz stderr
'
test_expect_success 'notice d/f conflict with existing ref' '
test_must_fail git branch foo/bar/baz/extra &&
test_must_fail git branch foo/bar/baz/lots/of/extra/components
'
test_expect_success 'timeout if packed-refs.lock exists' '
LOCK=.git/packed-refs.lock &&
>"$LOCK" &&
test_when_finished "rm -f $LOCK" &&
test_must_fail git pack-refs --all --prune
'
test_expect_success 'retry acquiring packed-refs.lock' '
LOCK=.git/packed-refs.lock &&
>"$LOCK" &&
test_when_finished "wait; rm -f $LOCK" &&
{
( sleep 1 ; rm -f $LOCK ) &
} &&
git -c core.packedrefstimeout=3000 pack-refs --all --prune
'
test_done