mirror of
https://github.com/git/git
synced 2024-08-28 12:09:19 +00:00
![Jeff King](/assets/img/avatar_default.png)
This fixes an occasional hang I see when running t4053 with
--verbose-log using dash.
Commit 1e3f26542a
(diff --no-index: support reading from named pipes,
2023-07-05) added a test that "diff --no-index" will complain when
comparing a named pipe and a directory. The minimum we need to test this
is to mkfifo the pipe, and then run "git diff --no-index pipe some_dir".
But the test does one thing more: it spawns a background shell process
that opens the pipe for writing, like this:
{
(>pipe) &
} &&
This extra writer _could_ be useful if Git misbehaves and tries to open
the pipe for reading. Without the writer, Git would block indefinitely
and the test would never end. But since we do not have such a bug, Git
does not open the pipe and it is the writing process which will block
indefinitely, since there are no readers. The test addresses this by
running "kill $!" in a test_when_finished block. Since the writer should
be blocking forever, this kill command will reliably find it waiting.
However, this seems to be somewhat racy, in that the writing process
sometimes hangs around even after the "kill". In a normal run of the
test script without options, this doesn't have any effect; the
main test script completes anyway. But with --verbose-log, we spawn a
"tee" process that reads the script output, and it won't end until all
descriptors pointing to its input pipe are closed. And the background
process that is hanging around still has its stderr, etc, pointed into
that pipe.
You can reproduce the situation like this:
cd t
./t4053-diff-no-index.sh --verbose-log --stress
Let that run for a few minutes, and then you'll find that some of the
runs have hung. For example, at 11:53, I ran:
$ ps xk start o pid,start,command | grep tee | head
713459 11:48:06 tee -a /home/peff/compile/git/t/test-results/t4053-diff-no-index.stress-9.out
713527 11:48:06 tee -a /home/peff/compile/git/t/test-results/t4053-diff-no-index.stress-15.out
719434 11:48:07 tee -a /home/peff/compile/git/t/test-results/t4053-diff-no-index.stress-1.out
728117 11:48:08 tee -a /home/peff/compile/git/t/test-results/t4053-diff-no-index.stress-5.out
738738 11:48:09 tee -a /home/peff/compile/git/t/test-results/t4053-diff-no-index.stress-31.out
739457 11:48:09 tee -a /home/peff/compile/git/t/test-results/t4053-diff-no-index.stress-27.out
744432 11:48:10 tee -a /home/peff/compile/git/t/test-results/t4053-diff-no-index.stress-21.out
744471 11:48:10 tee -a /home/peff/compile/git/t/test-results/t4053-diff-no-index.stress-29.out
761961 11:48:12 tee -a /home/peff/compile/git/t/test-results/t4053-diff-no-index.stress-0.out
812299 11:48:19 tee -a /home/peff/compile/git/t/test-results/t4053-diff-no-index.stress-8.out
All of these have been hung for several minutes. We can investigate one
and see that it's waiting to get EOF on its input:
$ strace -p 713459
strace: Process 713459 attached
read(0,
^C
Who else has that descriptor open?
$ lsof -a -p 713459 -d 0 +E
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
tee 713459 peff 0r FIFO 0,13 0t0 3943636 pipe 719203,sh,5w 719203,sh,7w 719203,sh,12w 719203,sh,13w
sh 719203 peff 5w FIFO 0,13 0t0 3943636 pipe 713459,tee,0r 719203,sh,7w 719203,sh,12w 719203,sh,13w
sh 719203 peff 7w FIFO 0,13 0t0 3943636 pipe 713459,tee,0r 719203,sh,5w 719203,sh,12w 719203,sh,13w
sh 719203 peff 12w FIFO 0,13 0t0 3943636 pipe 713459,tee,0r 719203,sh,5w 719203,sh,7w 719203,sh,13w
sh 719203 peff 13w FIFO 0,13 0t0 3943636 pipe 713459,tee,0r 719203,sh,5w 719203,sh,7w 719203,sh,12w
It's a shell, presumably a subshell spawned by the main script. Though
it may seem odd, having the same descriptor open several times is not
unreasonable (they're all basically the original stdout/stderr of the
script that has been copied). And they should all close when the process
exits. So what's it doing? Curiously, it will exit as soon as we strace
it:
$ strace -s 64 -p 719203
strace: Process 719203 attached
openat(AT_FDCWD, "pipe", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1 ENOENT (No such file or directory)
write(2, "./t4053-diff-no-index.sh: 7: eval: ", 35) = 35
write(2, "cannot create pipe: Directory nonexistent", 41) = 41
write(2, "\n", 1) = 1
exit_group(2) = ?
+++ exited with 2 +++
I think what happens is this:
- it is blocking in the openat() call for the pipe, as we expect (so
this is definitely the backgrounded subshell mentioned above)
- strace sends signals (probably STOP/CONT); those cause the kernel to
stop blocking, but libc will restart the system call automatically
- by this time, the "pipe" fifo is gone, so we'll actually try to
create a regular file. But of course the surrounding directory is
gone, too! So we get ENOENT, and then exit as normal.
So the blocking is something we expect to happen. But what we didn't
expect is for the process to still exist at all! It should have been
killed earlier when the parent process called "kill", but it wasn't. And
we can't catch the race at this point, because it happened much earlier.
One can guess, though, that there is some race with the shell setting up
the signal handling in the backgrounded subshell, and possibly blocking
or ignoring signals at the time that the "kill" is received. Curiously,
the race does not seem to happen if I use "bash" instead of "dash", so
presumably bash's setup here is more atomic.
One fix might be to try killing the subshell more aggressively, either
using SIGKILL, or looping on kill/wait. But that seems complex and
likely to introduce new problems/races. Instead, we can observe that the
writer is not needed at all. Git will notice the pipe via stat() before
it is ever opened. So we can simply drop the writer subshell entirely.
If we ever changed Git to open the path and fstat() it, this would
result in the test hanging. But we're not likely to do that. After all,
we have to stat() paths to see if they are openable at all (e.g., it
could be a directory), so this seems like a low risk. And anybody who
does make such a change will immediately see the issue, as Git would
hang consistently.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
269 lines
6.7 KiB
Bash
Executable file
269 lines
6.7 KiB
Bash
Executable file
#!/bin/sh
|
|
|
|
test_description='diff --no-index'
|
|
|
|
TEST_PASSES_SANITIZE_LEAK=true
|
|
. ./test-lib.sh
|
|
|
|
test_expect_success 'setup' '
|
|
mkdir a &&
|
|
mkdir b &&
|
|
echo 1 >a/1 &&
|
|
echo 2 >a/2 &&
|
|
git init repo &&
|
|
echo 1 >repo/a &&
|
|
mkdir -p non/git &&
|
|
echo 1 >non/git/a &&
|
|
echo 1 >non/git/b
|
|
'
|
|
|
|
test_expect_success 'git diff --no-index --exit-code' '
|
|
git diff --no-index --exit-code a/1 non/git/a &&
|
|
test_expect_code 1 git diff --no-index --exit-code a/1 a/2
|
|
'
|
|
|
|
test_expect_success 'git diff --no-index directories' '
|
|
test_expect_code 1 git diff --no-index a b >cnt &&
|
|
test_line_count = 14 cnt
|
|
'
|
|
|
|
test_expect_success 'git diff --no-index relative path outside repo' '
|
|
(
|
|
cd repo &&
|
|
test_expect_code 0 git diff --no-index a ../non/git/a &&
|
|
test_expect_code 0 git diff --no-index ../non/git/a ../non/git/b
|
|
)
|
|
'
|
|
|
|
test_expect_success 'git diff --no-index with broken index' '
|
|
(
|
|
cd repo &&
|
|
echo broken >.git/index &&
|
|
git diff --no-index a ../non/git/a
|
|
)
|
|
'
|
|
|
|
test_expect_success 'git diff outside repo with broken index' '
|
|
(
|
|
cd repo &&
|
|
git diff ../non/git/a ../non/git/b
|
|
)
|
|
'
|
|
|
|
test_expect_success 'git diff --no-index executed outside repo gives correct error message' '
|
|
(
|
|
GIT_CEILING_DIRECTORIES=$TRASH_DIRECTORY/non &&
|
|
export GIT_CEILING_DIRECTORIES &&
|
|
cd non/git &&
|
|
test_must_fail git diff --no-index a 2>actual.err &&
|
|
test_i18ngrep "usage: git diff --no-index" actual.err
|
|
)
|
|
'
|
|
|
|
test_expect_success 'diff D F and diff F D' '
|
|
(
|
|
cd repo &&
|
|
echo in-repo >a &&
|
|
echo non-repo >../non/git/a &&
|
|
mkdir sub &&
|
|
echo sub-repo >sub/a &&
|
|
|
|
test_must_fail git diff --no-index sub/a ../non/git/a >expect &&
|
|
test_must_fail git diff --no-index sub/a ../non/git/ >actual &&
|
|
test_cmp expect actual &&
|
|
|
|
test_must_fail git diff --no-index a ../non/git/a >expect &&
|
|
test_must_fail git diff --no-index a ../non/git/ >actual &&
|
|
test_cmp expect actual &&
|
|
|
|
test_must_fail git diff --no-index ../non/git/a a >expect &&
|
|
test_must_fail git diff --no-index ../non/git a >actual &&
|
|
test_cmp expect actual
|
|
)
|
|
'
|
|
|
|
test_expect_success 'turning a file into a directory' '
|
|
(
|
|
cd non/git &&
|
|
mkdir d e e/sub &&
|
|
echo 1 >d/sub &&
|
|
echo 2 >e/sub/file &&
|
|
printf "D\td/sub\nA\te/sub/file\n" >expect &&
|
|
test_must_fail git diff --no-index --name-status d e >actual &&
|
|
test_cmp expect actual
|
|
)
|
|
'
|
|
|
|
test_expect_success 'diff from repo subdir shows real paths (explicit)' '
|
|
echo "diff --git a/../../non/git/a b/../../non/git/b" >expect &&
|
|
test_expect_code 1 \
|
|
git -C repo/sub \
|
|
diff --no-index ../../non/git/a ../../non/git/b >actual &&
|
|
head -n 1 <actual >actual.head &&
|
|
test_cmp expect actual.head
|
|
'
|
|
|
|
test_expect_success 'diff from repo subdir shows real paths (implicit)' '
|
|
echo "diff --git a/../../non/git/a b/../../non/git/b" >expect &&
|
|
test_expect_code 1 \
|
|
git -C repo/sub \
|
|
diff ../../non/git/a ../../non/git/b >actual &&
|
|
head -n 1 <actual >actual.head &&
|
|
test_cmp expect actual.head
|
|
'
|
|
|
|
test_expect_success 'diff --no-index from repo subdir respects config (explicit)' '
|
|
echo "diff --git ../../non/git/a ../../non/git/b" >expect &&
|
|
test_config -C repo diff.noprefix true &&
|
|
test_expect_code 1 \
|
|
git -C repo/sub \
|
|
diff --no-index ../../non/git/a ../../non/git/b >actual &&
|
|
head -n 1 <actual >actual.head &&
|
|
test_cmp expect actual.head
|
|
'
|
|
|
|
test_expect_success 'diff --no-index from repo subdir respects config (implicit)' '
|
|
echo "diff --git ../../non/git/a ../../non/git/b" >expect &&
|
|
test_config -C repo diff.noprefix true &&
|
|
test_expect_code 1 \
|
|
git -C repo/sub \
|
|
diff ../../non/git/a ../../non/git/b >actual &&
|
|
head -n 1 <actual >actual.head &&
|
|
test_cmp expect actual.head
|
|
'
|
|
|
|
test_expect_success 'diff --no-index from repo subdir with absolute paths' '
|
|
cat <<-EOF >expect &&
|
|
1 1 $(pwd)/non/git/{a => b}
|
|
EOF
|
|
test_expect_code 1 \
|
|
git -C repo/sub diff --numstat \
|
|
"$(pwd)/non/git/a" "$(pwd)/non/git/b" >actual &&
|
|
test_cmp expect actual
|
|
'
|
|
|
|
test_expect_success 'diff --no-index allows external diff' '
|
|
test_expect_code 1 \
|
|
env GIT_EXTERNAL_DIFF="echo external ;:" \
|
|
git diff --no-index non/git/a non/git/b >actual &&
|
|
echo external >expect &&
|
|
test_cmp expect actual
|
|
'
|
|
|
|
test_expect_success 'diff --no-index normalizes mode: no changes' '
|
|
echo foo >x &&
|
|
cp x y &&
|
|
git diff --no-index x y >out &&
|
|
test_must_be_empty out
|
|
'
|
|
|
|
test_expect_success POSIXPERM 'diff --no-index normalizes mode: chmod +x' '
|
|
chmod +x y &&
|
|
cat >expected <<-\EOF &&
|
|
diff --git a/x b/y
|
|
old mode 100644
|
|
new mode 100755
|
|
EOF
|
|
test_expect_code 1 git diff --no-index x y >actual &&
|
|
test_cmp expected actual
|
|
'
|
|
|
|
test_expect_success POSIXPERM 'diff --no-index normalizes: mode not like git mode' '
|
|
chmod 666 x &&
|
|
chmod 777 y &&
|
|
cat >expected <<-\EOF &&
|
|
diff --git a/x b/y
|
|
old mode 100644
|
|
new mode 100755
|
|
EOF
|
|
test_expect_code 1 git diff --no-index x y >actual &&
|
|
test_cmp expected actual
|
|
'
|
|
|
|
test_expect_success POSIXPERM,SYMLINKS 'diff --no-index normalizes: mode not like git mode (symlink)' '
|
|
ln -s y z &&
|
|
X_OID=$(git hash-object --stdin <x) &&
|
|
Z_OID=$(printf y | git hash-object --stdin) &&
|
|
cat >expected <<-EOF &&
|
|
diff --git a/x b/x
|
|
deleted file mode 100644
|
|
index $X_OID..$ZERO_OID
|
|
--- a/x
|
|
+++ /dev/null
|
|
@@ -1 +0,0 @@
|
|
-foo
|
|
diff --git a/z b/z
|
|
new file mode 120000
|
|
index $ZERO_OID..$Z_OID
|
|
--- /dev/null
|
|
+++ b/z
|
|
@@ -0,0 +1 @@
|
|
+y
|
|
\ No newline at end of file
|
|
EOF
|
|
test_expect_code 1 git -c core.abbrev=no diff --no-index x z >actual &&
|
|
test_cmp expected actual
|
|
'
|
|
|
|
test_expect_success "diff --no-index treats '-' as stdin" '
|
|
cat >expect <<-EOF &&
|
|
diff --git a/- b/a/1
|
|
index $ZERO_OID..$(git hash-object --stdin <a/1) 100644
|
|
--- a/-
|
|
+++ b/a/1
|
|
@@ -1 +1 @@
|
|
-x
|
|
+1
|
|
EOF
|
|
|
|
test_write_lines x | test_expect_code 1 \
|
|
git -c core.abbrev=no diff --no-index -- - a/1 >actual &&
|
|
test_cmp expect actual &&
|
|
|
|
test_write_lines 1 | git diff --no-index -- a/1 - >actual &&
|
|
test_must_be_empty actual
|
|
'
|
|
|
|
test_expect_success 'diff --no-index refuses to diff stdin and a directory' '
|
|
test_must_fail git diff --no-index -- - a </dev/null 2>err &&
|
|
grep "fatal: cannot compare stdin to a directory" err
|
|
'
|
|
|
|
test_expect_success PIPE 'diff --no-index refuses to diff a named pipe and a directory' '
|
|
test_when_finished "rm -f pipe" &&
|
|
mkfifo pipe &&
|
|
test_must_fail git diff --no-index -- pipe a 2>err &&
|
|
grep "fatal: cannot compare a named pipe to a directory" err
|
|
'
|
|
|
|
test_expect_success PIPE,SYMLINKS 'diff --no-index reads from pipes' '
|
|
test_when_finished "rm -f old new new-link" &&
|
|
mkfifo old &&
|
|
mkfifo new &&
|
|
ln -s new new-link &&
|
|
{
|
|
(test_write_lines a b c >old) &
|
|
} &&
|
|
test_when_finished "kill $! || :" &&
|
|
{
|
|
(test_write_lines a x c >new) &
|
|
} &&
|
|
test_when_finished "kill $! || :" &&
|
|
|
|
cat >expect <<-EOF &&
|
|
diff --git a/old b/new-link
|
|
--- a/old
|
|
+++ b/new-link
|
|
@@ -1,3 +1,3 @@
|
|
a
|
|
-b
|
|
+x
|
|
c
|
|
EOF
|
|
|
|
test_expect_code 1 git diff --no-index old new-link >actual &&
|
|
test_cmp expect actual
|
|
'
|
|
|
|
test_done
|