development/git
development/git
1
0
Fork 0
mirror of https://github.com/git/git synced 2024-10-28 19:25:47 +00:00
Code Issues Projects Releases Packages Wiki Activity

t7406: avoid failures solely due to timing issues

Browse source 
Regression tests are automated tests which try to ensure a specific
behavior. The idea is: if the test case fails, the behavior indicated in
the test case's title regressed.

If a regression test that fails, even occasionally, for any reason other
than to indicate the particular regression(s) it tries to catch, it is
less useful than when it really only fails when there is a bug in the
(non-test) code that needs to be fixed.

In the instance of the test case "submodule update --init --recursive
from subdirectory" of the script t7406-submodule-update.sh, the exact
output of a recursive clone is compared with a pre-generated one. And
this is a racy test because the structure of the submodules only
guarantees a *partial* order. The 'none' and the 'rebasing' submodules
*can* be cloned in any order, which means that a mismatch with the
hard-coded order does not necessarily indicate a bug in the tested code.

See for example:
https://git-for-windows.visualstudio.com/git/_build/results?buildId=14035&view=logs

To prevent such false positives from unnecessarily costing time when
investigating test failures, let's take the exact order of the lines out
of the equation by sorting them before comparing them.

This test script seems not to have any more test cases that try to
verify any specific order in which recursive clones process the
submodules, therefore this is the only test case that is changed in this
manner.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This commit is contained in:
Johannes Schindelin 2018-07-23 06:39:42 -07:00 committed by Junio C Hamano
parent b7bd9486b0
commit 0b7d324ee5
1 changed files with 6 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
t/t7406-submodule-update.sh
View file

@ -115,17 +115,17 @@ Submodule path '../super/submodule': checked out '$submodulesha1'
EOF
cat <<EOF >expect2
Cloning into '$pwd/recursivesuper/super/merging'...
Cloning into '$pwd/recursivesuper/super/none'...
Cloning into '$pwd/recursivesuper/super/rebasing'...
Cloning into '$pwd/recursivesuper/super/submodule'...
Submodule 'merging' ($pwd/merging) registered for path '../super/merging'
Submodule 'none' ($pwd/none) registered for path '../super/none'
Submodule 'rebasing' ($pwd/rebasing) registered for path '../super/rebasing'
Submodule 'submodule' ($pwd/submodule) registered for path '../super/submodule'
Cloning into '$pwd/recursivesuper/super/merging'...
done.
Cloning into '$pwd/recursivesuper/super/none'...
done.
Cloning into '$pwd/recursivesuper/super/rebasing'...
done.
Cloning into '$pwd/recursivesuper/super/submodule'...
done.
EOF
@ -137,7 +137,8 @@ test_expect_success 'submodule update --init --recursive from subdirectory' '
git submodule update --init --recursive ../super >../../actual 2>../../actual2
) &&
test_i18ncmp expect actual &&
test_i18ncmp expect2 actual2
sort actual2 >actual2.sorted &&
test_i18ncmp expect2 actual2.sorted
'
cat <<EOF >expect2