development/git
development/git
1
0
Fork 0
mirror of https://github.com/git/git synced 2024-10-31 08:44:08 +00:00
Code Issues Projects Releases Packages Wiki Activity
git/t/t7701-repack-unpack-unreachable.sh

154 lines
4.5 KiB
Bash
Raw Normal View History

repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 04:01:55 +00:00
#!/bin/sh
tests: use "git xyzzy" form (t7200 - t9001) Converts tests between t7201-t9001. Signed-off-by: Nanako Shiraishi <nanako3@lavabit.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-09-03 08:59:33 +00:00
test_description='git repack works correctly'
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 04:01:55 +00:00
. ./test-lib.sh
t7701-repack-unpack-unreachable.sh: check timestamp of unpacked objects Unpacked objects should receive the timestamp of the pack they were unpacked from. Check. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-29 00:25:05 +00:00
fsha1=
csha1=
tsha1=
repack: only unpack-unreachable if we are deleting redundant packs The -A option calls pack-objects with the --unpack-unreachable option so that the unreachable objects in local packs are left in the local object store loose. But if the -d option to repack was _not_ used, then these unpacked loose objects are redundant and unnecessary. Update tests in t7701. Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-13 20:11:46 +00:00
test_expect_success '-A with -d option leaves unreachable objects unpacked' '
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 04:01:55 +00:00
echo content > file1 &&
git add . &&
more war on "sleep" in tests Two more tests that sleep only to waste tick can be converted to use test_tick and take expiry parameters relative to $test_tick. The basic idea is to replace "sleep 1" with "test_tick" to cause the "time" to pass. These tests are interested in expiring things with "now" as the timestamp, soo use a timestamp relative to $test_tick to give them more stability and reproducibility. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-14 22:09:57 +00:00
test_tick &&
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 04:01:55 +00:00
git commit -m initial_commit &&
# create a transient branch with unique content
git checkout -b transient_branch &&
echo more content >> file1 &&
# record the objects created in the database for file, commit, tree
fsha1=$(git hash-object file1) &&
more war on "sleep" in tests Two more tests that sleep only to waste tick can be converted to use test_tick and take expiry parameters relative to $test_tick. The basic idea is to replace "sleep 1" with "test_tick" to cause the "time" to pass. These tests are interested in expiring things with "now" as the timestamp, soo use a timestamp relative to $test_tick to give them more stability and reproducibility. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-14 22:09:57 +00:00
test_tick &&
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 04:01:55 +00:00
git commit -a -m more_content &&
csha1=$(git rev-parse HEAD^{commit}) &&
tsha1=$(git rev-parse HEAD^{tree}) &&
git checkout master &&
echo even more content >> file1 &&
more war on "sleep" in tests Two more tests that sleep only to waste tick can be converted to use test_tick and take expiry parameters relative to $test_tick. The basic idea is to replace "sleep 1" with "test_tick" to cause the "time" to pass. These tests are interested in expiring things with "now" as the timestamp, soo use a timestamp relative to $test_tick to give them more stability and reproducibility. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-14 22:09:57 +00:00
test_tick &&
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 04:01:55 +00:00
git commit -a -m even_more_content &&
# delete the transient branch
git branch -D transient_branch &&
# pack the repo
git repack -A -d -l &&
# verify objects are packed in repository
test 3 = $(git verify-pack -v -- .git/objects/pack/*.idx |
tests: grep portability fixes We try to avoid using the "-q" or "-e" options, as they are largely useless, as explained in aadbe44f. There is one exception for "-e" here, which is in t7701 used to produce an "or" of patterns. This can be rewritten as an egrep pattern. This patch also removes use of "grep -F" in favor of the more widely available "fgrep". [sp: Tested on AIX 5.3 by Mike Ralphson, Tested on MinGW by Johannes Sixt] Signed-off-by: Jeff King <peff@peff.net> Tested-by: Mike Ralphson <mike@abacus.co.uk> Tested-by: Johannes Sixt <johannes.sixt@telecom.at> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2008-09-30 08:03:55 +00:00
egrep "^($fsha1|$csha1|$tsha1) " |
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 04:01:55 +00:00
sort | uniq | wc -l) &&
git show $fsha1 &&
git show $csha1 &&
git show $tsha1 &&
more war on "sleep" in tests Two more tests that sleep only to waste tick can be converted to use test_tick and take expiry parameters relative to $test_tick. The basic idea is to replace "sleep 1" with "test_tick" to cause the "time" to pass. These tests are interested in expiring things with "now" as the timestamp, soo use a timestamp relative to $test_tick to give them more stability and reproducibility. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-14 22:09:57 +00:00
# now expire the reflog, while keeping reachable ones but expiring
# unreachables immediately
test_tick &&
sometimeago=$(( $test_tick - 10000 )) &&
git reflog expire --expire=$sometimeago --expire-unreachable=$test_tick --all &&
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 04:01:55 +00:00
# and repack
git repack -A -d -l &&
# verify objects are retained unpacked
test 0 = $(git verify-pack -v -- .git/objects/pack/*.idx |
tests: grep portability fixes We try to avoid using the "-q" or "-e" options, as they are largely useless, as explained in aadbe44f. There is one exception for "-e" here, which is in t7701 used to produce an "or" of patterns. This can be rewritten as an egrep pattern. This patch also removes use of "grep -F" in favor of the more widely available "fgrep". [sp: Tested on AIX 5.3 by Mike Ralphson, Tested on MinGW by Johannes Sixt] Signed-off-by: Jeff King <peff@peff.net> Tested-by: Mike Ralphson <mike@abacus.co.uk> Tested-by: Johannes Sixt <johannes.sixt@telecom.at> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2008-09-30 08:03:55 +00:00
egrep "^($fsha1|$csha1|$tsha1) " |
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 04:01:55 +00:00
sort | uniq | wc -l) &&
git show $fsha1 &&
git show $csha1 &&
git show $tsha1
'
t7701-repack-unpack-unreachable.sh: check timestamp of unpacked objects Unpacked objects should receive the timestamp of the pack they were unpacked from. Check. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-29 00:25:05 +00:00
compare_mtimes ()
{
t/helper: 'test-chmtime (--get|-g)' to print only the mtime Compared to 'test-chmtime -v +0 file' which prints the mtime and and the file name, 'test-chmtime --get file' displays only the mtime. If it is used in combination with (+|=|=+|=-|-)seconds, it changes and prints the new value. test-chmtime -v +0 file | sed 's/[^0-9].*$//' is now equivalent to: test-chmtime --get file Signed-off-by: Paul-Sebastian Ungureanu <ungureanupaulsebastian@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-06 22:19:47 +00:00
read tref &&
while read t; do
t7701: fix ignored exit code inside loop When checking a list of file mtimes, we use a loop and break out early from the loop if any entry does not match. However, the exit code of a loop exited via break is always 0, meaning that the test will fail to notice we had a mismatch. Since the loop is inside a function, we can fix this by doing an early "return 1". Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-25 05:29:10 +00:00
test "$tref" = "$t" || return 1
Windows: Fix intermittent failures of t7701 The last test case checks whether unpacked objects receive the time stamp of the pack file. Due to different implementations of stat(2) by MSYS and our version in compat/mingw.c, the test fails in about half of the test runs. Note the following facts: - The test uses perl's -M operator to compare the time stamps. Since we depend on MSYS perl, the result of this operator is based on MSYS's implementation of the stat(2) call. - NTFS on Windows records fractional seconds. - The MSYS implementation of stat(2) *rounds* fractional seconds to full seconds instead of truncating them. This becomes obvious by comparing the modification times reported by 'ls --full-time $f' and 'stat $f' for various files $f. - Our implementation of stat(2) in compat/mingw.c *truncates* to full seconds. The consequence of this is that - add_packed_git() picks up a truncated whole second modification time from the pack file time stamp, which is then used for the loose objects, while the pack file retains its time stamp in fractional seconds; - but the test case compared the pack file's rounded modification times to the loose objects' truncated modification times. And half of the time the rounded modification time is not the same as its truncated modification time. The fix is that we replace perl by 'test-chmtime -v +0', which prints the truncated whole-second mtime without modifying it. Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-01-28 09:52:26 +00:00
done
t7701-repack-unpack-unreachable.sh: check timestamp of unpacked objects Unpacked objects should receive the timestamp of the pack they were unpacked from. Check. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-29 00:25:05 +00:00
}
repack: only unpack-unreachable if we are deleting redundant packs The -A option calls pack-objects with the --unpack-unreachable option so that the unreachable objects in local packs are left in the local object store loose. But if the -d option to repack was _not_ used, then these unpacked loose objects are redundant and unnecessary. Update tests in t7701. Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-13 20:11:46 +00:00
test_expect_success '-A without -d option leaves unreachable objects packed' '
t7701-repack-unpack-unreachable.sh: check timestamp of unpacked objects Unpacked objects should receive the timestamp of the pack they were unpacked from. Check. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-29 00:25:05 +00:00
fsha1path=$(echo "$fsha1" | sed -e "s|\(..\)|\1/|") &&
fsha1path=".git/objects/$fsha1path" &&
csha1path=$(echo "$csha1" | sed -e "s|\(..\)|\1/|") &&
csha1path=".git/objects/$csha1path" &&
tsha1path=$(echo "$tsha1" | sed -e "s|\(..\)|\1/|") &&
tsha1path=".git/objects/$tsha1path" &&
git branch transient_branch $csha1 &&
git repack -a -d -l &&
test ! -f "$fsha1path" &&
test ! -f "$csha1path" &&
test ! -f "$tsha1path" &&
test 1 = $(ls -1 .git/objects/pack/pack-*.pack | wc -l) &&
packfile=$(ls .git/objects/pack/pack-*.pack) &&
git branch -D transient_branch &&
more war on "sleep" in tests Two more tests that sleep only to waste tick can be converted to use test_tick and take expiry parameters relative to $test_tick. The basic idea is to replace "sleep 1" with "test_tick" to cause the "time" to pass. These tests are interested in expiring things with "now" as the timestamp, soo use a timestamp relative to $test_tick to give them more stability and reproducibility. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-14 22:09:57 +00:00
test_tick &&
t7701-repack-unpack-unreachable.sh: check timestamp of unpacked objects Unpacked objects should receive the timestamp of the pack they were unpacked from. Check. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-29 00:25:05 +00:00
git repack -A -l &&
repack: only unpack-unreachable if we are deleting redundant packs The -A option calls pack-objects with the --unpack-unreachable option so that the unreachable objects in local packs are left in the local object store loose. But if the -d option to repack was _not_ used, then these unpacked loose objects are redundant and unnecessary. Update tests in t7701. Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-13 20:11:46 +00:00
test ! -f "$fsha1path" &&
test ! -f "$csha1path" &&
test ! -f "$tsha1path" &&
git show $fsha1 &&
git show $csha1 &&
git show $tsha1
'
test_expect_success 'unpacked objects receive timestamp of pack file' '
tmppack=".git/objects/pack/tmp_pack" &&
ln "$packfile" "$tmppack" &&
git repack -A -l -d &&
Merge branch 'ps/test-chmtime-get' Test cleanup. * ps/test-chmtime-get: t/helper: 'test-chmtime (--get|-g)' to print only the mtime
2018-04-25 04:29:00 +00:00
test-tool chmtime --get "$tmppack" "$fsha1path" "$csha1path" "$tsha1path" \
Windows: Fix intermittent failures of t7701 The last test case checks whether unpacked objects receive the time stamp of the pack file. Due to different implementations of stat(2) by MSYS and our version in compat/mingw.c, the test fails in about half of the test runs. Note the following facts: - The test uses perl's -M operator to compare the time stamps. Since we depend on MSYS perl, the result of this operator is based on MSYS's implementation of the stat(2) call. - NTFS on Windows records fractional seconds. - The MSYS implementation of stat(2) *rounds* fractional seconds to full seconds instead of truncating them. This becomes obvious by comparing the modification times reported by 'ls --full-time $f' and 'stat $f' for various files $f. - Our implementation of stat(2) in compat/mingw.c *truncates* to full seconds. The consequence of this is that - add_packed_git() picks up a truncated whole second modification time from the pack file time stamp, which is then used for the loose objects, while the pack file retains its time stamp in fractional seconds; - but the test case compared the pack file's rounded modification times to the loose objects' truncated modification times. And half of the time the rounded modification time is not the same as its truncated modification time. The fix is that we replace perl by 'test-chmtime -v +0', which prints the truncated whole-second mtime without modifying it. Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-01-28 09:52:26 +00:00
> mtimes &&
compare_mtimes < mtimes
t7701-repack-unpack-unreachable.sh: check timestamp of unpacked objects Unpacked objects should receive the timestamp of the pack they were unpacked from. Check. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-29 00:25:05 +00:00
'
gc: do not explode objects which will be immediately pruned When we pack everything into one big pack with "git repack -Ad", any unreferenced objects in to-be-deleted packs are exploded into loose objects, with the intent that they will be examined and possibly cleaned up by the next run of "git prune". Since the exploded objects will receive the mtime of the pack from which they come, if the source pack is old, those loose objects will end up pruned immediately. In that case, it is much more efficient to skip the exploding step entirely for these objects. This patch teaches pack-objects to receive the expiration information and avoid writing these objects out. It also teaches "git gc" to pass the value of gc.pruneexpire to repack (which in turn learns to pass it along to pack-objects) so that this optimization happens automatically during "git gc" and "git gc --auto". Signed-off-by: Jeff King <peff@peff.net> Acked-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-07 10:30:09 +00:00
test_expect_success 'do not bother loosening old objects' '
obj1=$(echo one | git hash-object -w --stdin) &&
obj2=$(echo two | git hash-object -w --stdin) &&
pack1=$(echo $obj1 | git pack-objects .git/objects/pack/pack) &&
pack2=$(echo $obj2 | git pack-objects .git/objects/pack/pack) &&
git prune-packed &&
git cat-file -p $obj1 &&
git cat-file -p $obj2 &&
t/helper: merge test-chmtime into test-tool Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-24 07:44:31 +00:00
test-tool chmtime =-86400 .git/objects/pack/pack-$pack2.pack &&
gc: do not explode objects which will be immediately pruned When we pack everything into one big pack with "git repack -Ad", any unreferenced objects in to-be-deleted packs are exploded into loose objects, with the intent that they will be examined and possibly cleaned up by the next run of "git prune". Since the exploded objects will receive the mtime of the pack from which they come, if the source pack is old, those loose objects will end up pruned immediately. In that case, it is much more efficient to skip the exploding step entirely for these objects. This patch teaches pack-objects to receive the expiration information and avoid writing these objects out. It also teaches "git gc" to pass the value of gc.pruneexpire to repack (which in turn learns to pass it along to pack-objects) so that this optimization happens automatically during "git gc" and "git gc --auto". Signed-off-by: Jeff King <peff@peff.net> Acked-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-07 10:30:09 +00:00
git repack -A -d --unpack-unreachable=1.hour.ago &&
git cat-file -p $obj1 &&
test_must_fail git cat-file -p $obj2
'
repack: pack objects mentioned by the index When we pack all objects, we use only the objects reachable from references and reflogs. This misses any objects which are reachable from the index, but not yet referenced. By itself this isn't a big deal; the objects can remain loose until they are actually used in a commit. However, it does create a problem when we drop packed but unreachable objects. We try to optimize out the writing of objects that we will immediately prune, which means we must follow the same rules as prune in determining what is reachable. And prune uses the index for this purpose. This is rather uncommon in practice, as objects in the index would not usually have been packed in the first place. But it could happen in a sequence like: 1. You make a commit on a branch that references blob X. 2. You repack, moving X into the pack. 3. You delete the branch (and its reflog), so that X is unreferenced. 4. You "git add" blob X so that it is now referenced only by the index. 5. You repack again with git-gc. The pack-objects we invoke will see that X is neither referenced nor recent and not bother loosening it. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-17 00:44:49 +00:00
test_expect_success 'keep packed objects found only in index' '
echo my-unique-content >file &&
git add file &&
git commit -m "make it reachable" &&
git gc &&
git reset HEAD^ &&
git reflog expire --expire=now --all &&
git add file &&
t/helper: merge test-chmtime into test-tool Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-24 07:44:31 +00:00
test-tool chmtime =-86400 .git/objects/pack/* &&
repack: pack objects mentioned by the index When we pack all objects, we use only the objects reachable from references and reflogs. This misses any objects which are reachable from the index, but not yet referenced. By itself this isn't a big deal; the objects can remain loose until they are actually used in a commit. However, it does create a problem when we drop packed but unreachable objects. We try to optimize out the writing of objects that we will immediately prune, which means we must follow the same rules as prune in determining what is reachable. And prune uses the index for this purpose. This is rather uncommon in practice, as objects in the index would not usually have been packed in the first place. But it could happen in a sequence like: 1. You make a commit on a branch that references blob X. 2. You repack, moving X into the pack. 3. You delete the branch (and its reflog), so that X is unreferenced. 4. You "git add" blob X so that it is now referenced only by the index. 5. You repack again with git-gc. The pack-objects we invoke will see that X is neither referenced nor recent and not bother loosening it. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-17 00:44:49 +00:00
git gc --prune=1.hour.ago &&
git cat-file blob :file
'
repack: add --keep-unreachable option The usual way to do a full repack (and what is done by git-gc) is to run "repack -Ad --unpack-unreachable=<when>", which will loosen any unreachable objects newer than "<when>", and drop any older ones. This is a safer alternative to "repack -ad", because "<when>" becomes a grace period during which we will not drop any new objects that are about to be referenced. However, it isn't perfectly safe. It's always possible that a process is about to reference an old object. Even if that process were to take care to update the timestamp on the object, there is no atomicity with a simultaneously running "repack" process. So while unlikely, there is a small race wherein we may drop an object that is in the process of being referenced. If you do automated repacking on a large number of active repositories, you may hit it eventually, and the result is a corrupted repository. It would be nice to fix that race in the long run, but it's complicated. In the meantime, there is a much simpler strategy for automated repository maintenance: do not drop objects at all. We already have a "--keep-unreachable" option in pack-objects; we just need to plumb it through from git-repack. Note that this _isn't_ plumbed through from git-gc, so at this point it's strictly a tool for people doing their own advanced repository maintenance strategy. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13 04:36:28 +00:00
test_expect_success 'repack -k keeps unreachable packed objects' '
# create packed-but-unreachable object
sha1=$(echo unreachable-packed | git hash-object -w --stdin) &&
pack=$(echo $sha1 | git pack-objects .git/objects/pack/pack) &&
git prune-packed &&
# -k should keep it
git repack -adk &&
git cat-file -p $sha1 &&
# and double check that without -k it would have been removed
git repack -ad &&
test_must_fail git cat-file -p $sha1
'
repack: extend --keep-unreachable to loose objects If you use "repack -adk" currently, we will pack all objects that are already packed into the new pack, and then drop the old packs. However, loose unreachable objects will be left as-is. In theory these are meant to expire eventually with "git prune". But if you are using "repack -k", you probably want to keep things forever and therefore do not run "git prune" at all. Meaning those loose objects may build up over time and end up fooling any object-count heuristics (such as the one done by "gc --auto", though since git-gc does not support "repack -k", this really applies to whatever custom scripts people might have driving "repack -k"). With this patch, we instead stuff any loose unreachable objects into the pack along with the already-packed unreachable objects. This may seem wasteful, but it is really no more so than using "repack -k" in the first place. We are at a slight disadvantage, in that we have no useful ordering for the result, or names to hand to the delta code. However, this is again no worse than what "repack -k" is already doing for the packed objects. The packing of these objects doesn't matter much because they should not be accessed frequently (unless they actually _do_ become referenced, but then they would get moved to a different part of the packfile during the next repack). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13 04:38:04 +00:00
test_expect_success 'repack -k packs unreachable loose objects' '
# create loose unreachable object
sha1=$(echo would-be-deleted-loose | git hash-object -w --stdin) &&
objpath=.git/objects/$(echo $sha1 | sed "s,..,&/,") &&
test_path_is_file $objpath &&
# and confirm that the loose object goes away, but we can
# still access it (ergo, it is packed)
git repack -adk &&
test_path_is_missing $objpath &&
git cat-file -p $sha1
'
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 04:01:55 +00:00
test_done
Reference in a new issue Copy permalink