git/t/t1060-object-corruption.sh

141 lines
3.4 KiB
Bash
Raw Normal View History

#!/bin/sh
test_description='see how we handle various forms of corruption'
. ./test-lib.sh
# convert "1234abcd" to ".git/objects/12/34abcd"
obj_to_file() {
echo "$(git rev-parse --git-dir)/objects/$(git rev-parse "$1" | sed 's,..,&/,')"
}
# Convert byte at offset "$2" of object "$1" into '\0'
corrupt_byte() {
obj_file=$(obj_to_file "$1") &&
chmod +w "$obj_file" &&
printf '\0' | dd of="$obj_file" bs=1 seek="$2" conv=notrunc
}
test_expect_success 'setup corrupt repo' '
git init bit-error &&
(
cd bit-error &&
test_commit content &&
corrupt_byte HEAD:content.t 10
index-pack: detect local corruption in collision check When we notice that we have a local copy of an incoming object, we compare the two objects to make sure we haven't found a collision. Before we get to the actual object bytes, though, we compare the type and size from sha1_object_info(). If our local object is corrupted, then the type will be OBJ_BAD, which obviously will not match the incoming type, and we'll report "SHA1 COLLISION FOUND" (with capital letters and everything). This is confusing, as the problem is not a collision but rather local corruption. We should report that instead (just like we do if reading the rest of the object content fails a few lines later). Note that we _could_ just ignore the error and mark it as a non-collision. That would let you "git fetch" to replace a corrupted object. But it's not a very reliable method for repairing a repository. The earlier want/have negotiation tries to get the other side to omit objects we already have, and it would not realize that we are "missing" this corrupted object. So we're better off complaining loudly when we see corruption, and letting the user take more drastic measures to repair (like making a full clone elsewhere and copying the pack into place). Note that the test sets transfer.unpackLimit in the receiving repository so that we use index-pack (which is what does the collision check). Normally for such a small push we'd use unpack-objects, which would simply try to write the loose object, and discard the new one when we see that there's already an old one. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-01 08:09:32 +00:00
) &&
git init no-bit-error &&
(
# distinct commit from bit-error, but containing a
# non-corrupted version of the same blob
cd no-bit-error &&
test_tick &&
test_commit content
)
'
test_expect_success 'setup repo with missing object' '
git init missing &&
(
cd missing &&
test_commit content &&
rm -f "$(obj_to_file HEAD:content.t)"
)
'
test_expect_success 'setup repo with misnamed object' '
git init misnamed &&
(
cd misnamed &&
test_commit content &&
good=$(obj_to_file HEAD:content.t) &&
blob=$(echo corrupt | git hash-object -w --stdin) &&
bad=$(obj_to_file $blob) &&
rm -f "$good" &&
mv "$bad" "$good"
)
'
test_expect_success 'streaming a corrupt blob fails' '
(
cd bit-error &&
test_must_fail git cat-file blob HEAD:content.t
)
'
test_expect_success 'getting type of a corrupt blob fails' '
(
cd bit-error &&
test_must_fail git cat-file -s HEAD:content.t
)
'
test_expect_success 'read-tree -u detects bit-errors in blobs' '
(
cd bit-error &&
rm -f content.t &&
test_must_fail git read-tree --reset -u HEAD
)
'
test_expect_success 'read-tree -u detects missing objects' '
(
cd missing &&
rm -f content.t &&
test_must_fail git read-tree --reset -u HEAD
)
'
# We use --bare to make sure that the transport detects it, not the checkout
# phase.
test_expect_success 'clone --no-local --bare detects corruption' '
test_must_fail git clone --no-local --bare bit-error corrupt-transport
'
test_expect_success 'clone --no-local --bare detects missing object' '
test_must_fail git clone --no-local --bare missing missing-transport
'
test_expect_success 'clone --no-local --bare detects misnamed object' '
test_must_fail git clone --no-local --bare misnamed misnamed-transport
'
# We do not expect --local to detect corruption at the transport layer,
# so we are really checking the checkout() code path.
test_expect_success 'clone --local detects corruption' '
test_must_fail git clone --local bit-error corrupt-checkout
'
test_expect_success 'error detected during checkout leaves repo intact' '
test_path_is_dir corrupt-checkout/.git
'
test_expect_success 'clone --local detects missing objects' '
test_must_fail git clone --local missing missing-checkout
'
test_expect_failure 'clone --local detects misnamed objects' '
test_must_fail git clone --local misnamed misnamed-checkout
'
index-pack: detect local corruption in collision check When we notice that we have a local copy of an incoming object, we compare the two objects to make sure we haven't found a collision. Before we get to the actual object bytes, though, we compare the type and size from sha1_object_info(). If our local object is corrupted, then the type will be OBJ_BAD, which obviously will not match the incoming type, and we'll report "SHA1 COLLISION FOUND" (with capital letters and everything). This is confusing, as the problem is not a collision but rather local corruption. We should report that instead (just like we do if reading the rest of the object content fails a few lines later). Note that we _could_ just ignore the error and mark it as a non-collision. That would let you "git fetch" to replace a corrupted object. But it's not a very reliable method for repairing a repository. The earlier want/have negotiation tries to get the other side to omit objects we already have, and it would not realize that we are "missing" this corrupted object. So we're better off complaining loudly when we see corruption, and letting the user take more drastic measures to repair (like making a full clone elsewhere and copying the pack into place). Note that the test sets transfer.unpackLimit in the receiving repository so that we use index-pack (which is what does the collision check). Normally for such a small push we'd use unpack-objects, which would simply try to write the loose object, and discard the new one when we see that there's already an old one. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-01 08:09:32 +00:00
test_expect_success 'fetch into corrupted repo with index-pack' '
cp -R bit-error bit-error-cp &&
test_when_finished "rm -rf bit-error-cp" &&
index-pack: detect local corruption in collision check When we notice that we have a local copy of an incoming object, we compare the two objects to make sure we haven't found a collision. Before we get to the actual object bytes, though, we compare the type and size from sha1_object_info(). If our local object is corrupted, then the type will be OBJ_BAD, which obviously will not match the incoming type, and we'll report "SHA1 COLLISION FOUND" (with capital letters and everything). This is confusing, as the problem is not a collision but rather local corruption. We should report that instead (just like we do if reading the rest of the object content fails a few lines later). Note that we _could_ just ignore the error and mark it as a non-collision. That would let you "git fetch" to replace a corrupted object. But it's not a very reliable method for repairing a repository. The earlier want/have negotiation tries to get the other side to omit objects we already have, and it would not realize that we are "missing" this corrupted object. So we're better off complaining loudly when we see corruption, and letting the user take more drastic measures to repair (like making a full clone elsewhere and copying the pack into place). Note that the test sets transfer.unpackLimit in the receiving repository so that we use index-pack (which is what does the collision check). Normally for such a small push we'd use unpack-objects, which would simply try to write the loose object, and discard the new one when we see that there's already an old one. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-01 08:09:32 +00:00
(
cd bit-error-cp &&
index-pack: detect local corruption in collision check When we notice that we have a local copy of an incoming object, we compare the two objects to make sure we haven't found a collision. Before we get to the actual object bytes, though, we compare the type and size from sha1_object_info(). If our local object is corrupted, then the type will be OBJ_BAD, which obviously will not match the incoming type, and we'll report "SHA1 COLLISION FOUND" (with capital letters and everything). This is confusing, as the problem is not a collision but rather local corruption. We should report that instead (just like we do if reading the rest of the object content fails a few lines later). Note that we _could_ just ignore the error and mark it as a non-collision. That would let you "git fetch" to replace a corrupted object. But it's not a very reliable method for repairing a repository. The earlier want/have negotiation tries to get the other side to omit objects we already have, and it would not realize that we are "missing" this corrupted object. So we're better off complaining loudly when we see corruption, and letting the user take more drastic measures to repair (like making a full clone elsewhere and copying the pack into place). Note that the test sets transfer.unpackLimit in the receiving repository so that we use index-pack (which is what does the collision check). Normally for such a small push we'd use unpack-objects, which would simply try to write the loose object, and discard the new one when we see that there's already an old one. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-01 08:09:32 +00:00
test_must_fail git -c transfer.unpackLimit=1 \
fetch ../no-bit-error 2>stderr &&
test_i18ngrep ! -i collision stderr
)
'
rev-list: allow cached objects in existence check This fixes a regression in 7c0fe330d5 (rev-list: handle missing tree objects properly, 2018-10-05) where rev-list will now complain about the empty tree when it doesn't physically exist on disk. Before that commit, we relied on the traversal code in list-objects.c to walk through the trees. Since it uses parse_tree(), we'd do a normal object lookup that includes looking in the set of "cached" objects (which is where our magic internal empty-tree kicks in). After that commit, we instead tell list-objects.c not to die on any missing trees, and we check them ourselves using has_object_file(). But that function uses OBJECT_INFO_SKIP_CACHED, which means we won't use our internal empty tree. This normally wouldn't come up. For most operations, Git will try to write out the empty tree object as it would any other object. And pack-objects in a push or fetch will send the empty tree (even if it's virtual on the sending side). However, there are cases where this can matter. One I found in the wild: 1. The root tree of a commit became empty by deleting all files, without using an index. In this case it was done using libgit2's tree builder API, but as the included test shows, it can easily be done with regular git using hash-object. The resulting repo works OK, as we'd avoid walking over our own reachable commits for a connectivity check. 2. Cloning with --reference pointing to the repository from (1) can trigger the problem, because we tell the other side we already have that commit (and hence the empty tree), but then walk over it during the connectivity check (where we complain about it missing). Arguably the workflow in step (1) should be more careful about writing the empty tree object if we're referencing it. But this workflow did work prior to 7c0fe330d5, so let's restore it. This patch makes the minimal fix, which is to swap out a direct call to oid_object_info_extended(), minus the SKIP_CACHED flag, instead of calling has_object_file(). This is all that has_object_file() is doing under the hood. And there's little danger of unrelated fallout from other unexpected "cached" objects, since there's only one call site that ends such a cached object, and it's in git-blame. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-04 17:40:54 +00:00
test_expect_success 'internal tree objects are not "missing"' '
git init missing-empty &&
(
cd missing-empty &&
empty_tree=$(git hash-object -t tree /dev/null) &&
commit=$(echo foo | git commit-tree $empty_tree) &&
git rev-list --objects $commit
)
'
test_done