2017-03-05 18:25:19 +00:00
|
|
|
#!/bin/sh
|
|
|
|
|
#
|
2020-04-04 01:08:48 +00:00
|
|
|
# Build and test Git inside container
|
2017-03-05 18:25:19 +00:00
|
|
|
#
|
|
|
|
|
# Usage:
|
2020-04-04 01:08:48 +00:00
|
|
|
# run-docker-build.sh <host-user-id>
|
2017-03-05 18:25:19 +00:00
|
|
|
#
|
|
|
|
|
|
2018-01-29 17:17:10 +00:00
|
|
|
set -ex
|
2017-12-27 16:36:00 +00:00
|
|
|
|
travis-ci: don't run the test suite as root in the 32 bit Linux build
Travis CI runs the 32 bit Linux build job in a Docker container, where
all commands are executed as root by default. Therefore, ever since
we added this build job in 88dedd5e7 (Travis: also test on 32-bit
Linux, 2017-03-05), we have a bit of code to create a user in the
container matching the ID of the host user and then to run the test
suite as this user. Matching the host user ID is important, because
otherwise the host user would have no access to any files written by
processes running in the container, notably the logs of failed tests
couldn't be included in the build job's trace log.
Alas, this piece of code never worked, because it sets the variable
holding the user name ($CI_USER) in a subshell, meaning it doesn't
have any effect by the time we get to the point to actually use the
variable to switch users with 'su'. So all this time we were running
the test suite as root.
Reorganize that piece of code in 'ci/run-linux32-build.sh' a bit to
avoid that problematic subshell and to ensure that we switch to the
right user. Furthermore, make the script's optional host user ID
option mandatory, so running the build accidentally as root will
become harder when debugging locally. If someone really wants to run
the test suite as root, whatever the reasons might be, it'll still be
possible to do so by explicitly passing '0' as host user ID.
Finally, one last catch: since commit 7e72cfcee (travis-ci: save prove
state for the 32 bit Linux build, 2017-12-27) the 'prove' test harness
has been writing its state to the Travis CI cache directory from
within the Docker container while running as root. After this patch
'prove' will run as a regular user, so in future build jobs it won't
be able overwrite a previously written, still root-owned state file,
resulting in build job failures. To resolve this we should manually
delete caches containing such root-owned files, but that would be a
hassle. Instead, work this around by changing the owner of the whole
contents of the cache directory to the host user ID.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-29 17:17:12 +00:00
|
|
|
if test $# -ne 1 || test -z "$1"
|
|
|
|
|
then
|
2020-04-04 01:08:48 +00:00
|
|
|
echo >&2 "usage: run-docker-build.sh <host-user-id>"
|
travis-ci: don't run the test suite as root in the 32 bit Linux build
Travis CI runs the 32 bit Linux build job in a Docker container, where
all commands are executed as root by default. Therefore, ever since
we added this build job in 88dedd5e7 (Travis: also test on 32-bit
Linux, 2017-03-05), we have a bit of code to create a user in the
container matching the ID of the host user and then to run the test
suite as this user. Matching the host user ID is important, because
otherwise the host user would have no access to any files written by
processes running in the container, notably the logs of failed tests
couldn't be included in the build job's trace log.
Alas, this piece of code never worked, because it sets the variable
holding the user name ($CI_USER) in a subshell, meaning it doesn't
have any effect by the time we get to the point to actually use the
variable to switch users with 'su'. So all this time we were running
the test suite as root.
Reorganize that piece of code in 'ci/run-linux32-build.sh' a bit to
avoid that problematic subshell and to ensure that we switch to the
right user. Furthermore, make the script's optional host user ID
option mandatory, so running the build accidentally as root will
become harder when debugging locally. If someone really wants to run
the test suite as root, whatever the reasons might be, it'll still be
possible to do so by explicitly passing '0' as host user ID.
Finally, one last catch: since commit 7e72cfcee (travis-ci: save prove
state for the 32 bit Linux build, 2017-12-27) the 'prove' test harness
has been writing its state to the Travis CI cache directory from
within the Docker container while running as root. After this patch
'prove' will run as a regular user, so in future build jobs it won't
be able overwrite a previously written, still root-owned state file,
resulting in build job failures. To resolve this we should manually
delete caches containing such root-owned files, but that would be a
hassle. Instead, work this around by changing the owner of the whole
contents of the cache directory to the host user ID.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-29 17:17:12 +00:00
|
|
|
exit 1
|
|
|
|
|
fi
|
|
|
|
|
|
2020-04-04 01:08:47 +00:00
|
|
|
case "$jobname" in
|
2021-11-23 16:29:10 +00:00
|
|
|
linux32)
|
2020-04-04 01:08:47 +00:00
|
|
|
switch_cmd="linux32 --32bit i386"
|
|
|
|
|
;;
|
2020-04-04 01:08:50 +00:00
|
|
|
linux-musl)
|
|
|
|
|
switch_cmd=
|
|
|
|
|
useradd () { adduser -D "$@"; }
|
|
|
|
|
;;
|
2020-04-04 01:08:47 +00:00
|
|
|
*)
|
|
|
|
|
exit 1
|
|
|
|
|
;;
|
|
|
|
|
esac
|
|
|
|
|
|
2020-04-04 01:08:49 +00:00
|
|
|
"${0%/*}/install-docker-dependencies.sh"
|
2017-03-05 18:25:19 +00:00
|
|
|
|
|
|
|
|
# If this script runs inside a docker container, then all commands are
|
|
|
|
|
# usually executed as root. Consequently, the host user might not be
|
|
|
|
|
# able to access the test output files.
|
travis-ci: don't run the test suite as root in the 32 bit Linux build
Travis CI runs the 32 bit Linux build job in a Docker container, where
all commands are executed as root by default. Therefore, ever since
we added this build job in 88dedd5e7 (Travis: also test on 32-bit
Linux, 2017-03-05), we have a bit of code to create a user in the
container matching the ID of the host user and then to run the test
suite as this user. Matching the host user ID is important, because
otherwise the host user would have no access to any files written by
processes running in the container, notably the logs of failed tests
couldn't be included in the build job's trace log.
Alas, this piece of code never worked, because it sets the variable
holding the user name ($CI_USER) in a subshell, meaning it doesn't
have any effect by the time we get to the point to actually use the
variable to switch users with 'su'. So all this time we were running
the test suite as root.
Reorganize that piece of code in 'ci/run-linux32-build.sh' a bit to
avoid that problematic subshell and to ensure that we switch to the
right user. Furthermore, make the script's optional host user ID
option mandatory, so running the build accidentally as root will
become harder when debugging locally. If someone really wants to run
the test suite as root, whatever the reasons might be, it'll still be
possible to do so by explicitly passing '0' as host user ID.
Finally, one last catch: since commit 7e72cfcee (travis-ci: save prove
state for the 32 bit Linux build, 2017-12-27) the 'prove' test harness
has been writing its state to the Travis CI cache directory from
within the Docker container while running as root. After this patch
'prove' will run as a regular user, so in future build jobs it won't
be able overwrite a previously written, still root-owned state file,
resulting in build job failures. To resolve this we should manually
delete caches containing such root-owned files, but that would be a
hassle. Instead, work this around by changing the owner of the whole
contents of the cache directory to the host user ID.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-29 17:17:12 +00:00
|
|
|
# If a non 0 host user id is given, then create a user "ci" with that
|
|
|
|
|
# user id to make everything accessible to the host user.
|
2018-01-29 17:17:10 +00:00
|
|
|
HOST_UID=$1
|
travis-ci: don't run the test suite as root in the 32 bit Linux build
Travis CI runs the 32 bit Linux build job in a Docker container, where
all commands are executed as root by default. Therefore, ever since
we added this build job in 88dedd5e7 (Travis: also test on 32-bit
Linux, 2017-03-05), we have a bit of code to create a user in the
container matching the ID of the host user and then to run the test
suite as this user. Matching the host user ID is important, because
otherwise the host user would have no access to any files written by
processes running in the container, notably the logs of failed tests
couldn't be included in the build job's trace log.
Alas, this piece of code never worked, because it sets the variable
holding the user name ($CI_USER) in a subshell, meaning it doesn't
have any effect by the time we get to the point to actually use the
variable to switch users with 'su'. So all this time we were running
the test suite as root.
Reorganize that piece of code in 'ci/run-linux32-build.sh' a bit to
avoid that problematic subshell and to ensure that we switch to the
right user. Furthermore, make the script's optional host user ID
option mandatory, so running the build accidentally as root will
become harder when debugging locally. If someone really wants to run
the test suite as root, whatever the reasons might be, it'll still be
possible to do so by explicitly passing '0' as host user ID.
Finally, one last catch: since commit 7e72cfcee (travis-ci: save prove
state for the 32 bit Linux build, 2017-12-27) the 'prove' test harness
has been writing its state to the Travis CI cache directory from
within the Docker container while running as root. After this patch
'prove' will run as a regular user, so in future build jobs it won't
be able overwrite a previously written, still root-owned state file,
resulting in build job failures. To resolve this we should manually
delete caches containing such root-owned files, but that would be a
hassle. Instead, work this around by changing the owner of the whole
contents of the cache directory to the host user ID.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-29 17:17:12 +00:00
|
|
|
if test $HOST_UID -eq 0
|
|
|
|
|
then
|
|
|
|
|
# Just in case someone does want to run the test suite as root.
|
|
|
|
|
CI_USER=root
|
|
|
|
|
else
|
|
|
|
|
CI_USER=ci
|
2018-01-29 17:17:13 +00:00
|
|
|
if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID
|
|
|
|
|
then
|
|
|
|
|
echo "user '$CI_USER' already exists with the requested ID $HOST_UID"
|
|
|
|
|
else
|
|
|
|
|
useradd -u $HOST_UID $CI_USER
|
|
|
|
|
fi
|
travis-ci: don't run the test suite as root in the 32 bit Linux build
Travis CI runs the 32 bit Linux build job in a Docker container, where
all commands are executed as root by default. Therefore, ever since
we added this build job in 88dedd5e7 (Travis: also test on 32-bit
Linux, 2017-03-05), we have a bit of code to create a user in the
container matching the ID of the host user and then to run the test
suite as this user. Matching the host user ID is important, because
otherwise the host user would have no access to any files written by
processes running in the container, notably the logs of failed tests
couldn't be included in the build job's trace log.
Alas, this piece of code never worked, because it sets the variable
holding the user name ($CI_USER) in a subshell, meaning it doesn't
have any effect by the time we get to the point to actually use the
variable to switch users with 'su'. So all this time we were running
the test suite as root.
Reorganize that piece of code in 'ci/run-linux32-build.sh' a bit to
avoid that problematic subshell and to ensure that we switch to the
right user. Furthermore, make the script's optional host user ID
option mandatory, so running the build accidentally as root will
become harder when debugging locally. If someone really wants to run
the test suite as root, whatever the reasons might be, it'll still be
possible to do so by explicitly passing '0' as host user ID.
Finally, one last catch: since commit 7e72cfcee (travis-ci: save prove
state for the 32 bit Linux build, 2017-12-27) the 'prove' test harness
has been writing its state to the Travis CI cache directory from
within the Docker container while running as root. After this patch
'prove' will run as a regular user, so in future build jobs it won't
be able overwrite a previously written, still root-owned state file,
resulting in build job failures. To resolve this we should manually
delete caches containing such root-owned files, but that would be a
hassle. Instead, work this around by changing the owner of the whole
contents of the cache directory to the host user ID.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-29 17:17:12 +00:00
|
|
|
fi
|
2017-03-05 18:25:19 +00:00
|
|
|
|
|
|
|
|
# Build and test
|
2020-04-04 01:08:47 +00:00
|
|
|
command $switch_cmd su -m -l $CI_USER -c "
|
2018-01-29 17:17:10 +00:00
|
|
|
set -ex
|
2020-04-04 01:08:46 +00:00
|
|
|
export DEVELOPER='$DEVELOPER'
|
|
|
|
|
export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET'
|
|
|
|
|
export GIT_PROVE_OPTS='$GIT_PROVE_OPTS'
|
|
|
|
|
export GIT_TEST_OPTS='$GIT_TEST_OPTS'
|
|
|
|
|
export GIT_TEST_CLONE_2GB='$GIT_TEST_CLONE_2GB'
|
|
|
|
|
export MAKEFLAGS='$MAKEFLAGS'
|
|
|
|
|
export cache_dir='$cache_dir'
|
2018-01-29 17:17:10 +00:00
|
|
|
cd /usr/src/git
|
2020-04-04 01:08:46 +00:00
|
|
|
test -n '$cache_dir' && ln -s '$cache_dir/.prove' t/.prove
|
2019-01-27 23:26:52 +00:00
|
|
|
make
|
travis-ci: don't be '--quiet' when running the tests
All Travis CI build jobs run the test suite with 'make --quiet test'.
On one hand, being quiet doesn't save us from much clutter in the
output:
$ make test |wc -l
861
$ make --quiet test |wc -l
848
It only spares 13 lines, mostly the output of entering the 't/'
directory and the pre- and post-cleanup commands, which is negligible
compared to the ~700 lines printed while building Git and the ~850
lines of 'prove' output.
On the other hand, it's asking for trouble. In our CI build scripts
we build Git and run the test suite in two separate 'make'
invocations. In a prelimiary version of one of the later patches in
this series, to explicitly specify which compiler to use, I changed
them to basically run:
make CC=$CC
make --quiet test
naively thinking that it should Just Work... but then that 'make
--quiet test' got all clever on me, noticed the changed build flags,
and then proceeded to rebuild everything with the default 'cc'. And
because of that '--quiet' option, it did so, well, quietly, only
saying "* new build flags", and it was by mere luck that I happened to
notice that something is amiss.
Let's just drop that '--quiet' option when running the test suite in
all build scripts.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-17 01:29:11 +00:00
|
|
|
make test
|
2020-04-04 01:08:46 +00:00
|
|
|
"
|