git/ci/run-windows-build.sh

104 lines
2.7 KiB
Bash
Raw Normal View History

travis-ci: build and test Git on Windows Most Git developers work on Linux and they have no way to know if their changes would break the Git for Windows build. Let's fix that by adding a job to TravisCI that builds and tests Git on Windows. Unfortunately, TravisCI does not support Windows. Therefore, we did the following: * Johannes Schindelin set up a Visual Studio Team Services build sponsored by Microsoft and made it accessible via an Azure Function that speaks a super-simple API. We made TravisCI use this API to trigger a build, wait until its completion, and print the build and test results. * A Windows build and test run takes up to 3h and TravisCI has a timeout after 50min for Open Source projects. Since the TravisCI job does not use heavy CPU/memory/etc. resources, the friendly TravisCI folks extended the job timeout for git/git to 3h. Things, that would need to be done: * Someone with write access to https://travis-ci.org/git/git would need to add the secret token as "GFW_CI_TOKEN" variable in the TravisCI repository setting [1]. Afterwards the build should just work. Things, that might need to be done: * The Windows box can only process a single build at a time. A second Windows build would need to wait until the first finishes. This waiting time and the build time after the wait could exceed the 3h threshold. If this is a problem, then it is likely to happen every day as usually multiple branches are pushed at the same time (pu/next/ master/maint). I cannot test this as my TravisCI account has the 50min timeout. One solution could be to limit the number of concurrent TravisCI jobs [2]. [1] https://docs.travis-ci.com/user/environment-variables#Defining-Variables-in-Repository-Settings [2] https://docs.travis-ci.com/user/customizing-the-build#Limiting-Concurrent-Builds Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-24 11:37:47 +00:00
#!/usr/bin/env bash
#
# Script to trigger the Git for Windows build and test run.
travis-ci: build and test Git on Windows Most Git developers work on Linux and they have no way to know if their changes would break the Git for Windows build. Let's fix that by adding a job to TravisCI that builds and tests Git on Windows. Unfortunately, TravisCI does not support Windows. Therefore, we did the following: * Johannes Schindelin set up a Visual Studio Team Services build sponsored by Microsoft and made it accessible via an Azure Function that speaks a super-simple API. We made TravisCI use this API to trigger a build, wait until its completion, and print the build and test results. * A Windows build and test run takes up to 3h and TravisCI has a timeout after 50min for Open Source projects. Since the TravisCI job does not use heavy CPU/memory/etc. resources, the friendly TravisCI folks extended the job timeout for git/git to 3h. Things, that would need to be done: * Someone with write access to https://travis-ci.org/git/git would need to add the secret token as "GFW_CI_TOKEN" variable in the TravisCI repository setting [1]. Afterwards the build should just work. Things, that might need to be done: * The Windows box can only process a single build at a time. A second Windows build would need to wait until the first finishes. This waiting time and the build time after the wait could exceed the 3h threshold. If this is a problem, then it is likely to happen every day as usually multiple branches are pushed at the same time (pu/next/ master/maint). I cannot test this as my TravisCI account has the 50min timeout. One solution could be to limit the number of concurrent TravisCI jobs [2]. [1] https://docs.travis-ci.com/user/environment-variables#Defining-Variables-in-Repository-Settings [2] https://docs.travis-ci.com/user/customizing-the-build#Limiting-Concurrent-Builds Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-24 11:37:47 +00:00
# Set the $GFW_CI_TOKEN as environment variable.
# Pass the branch (only branches on https://github.com/git/git are
# supported) and a commit hash.
#
. ${0%/*}/lib.sh
travis-ci: build and test Git on Windows Most Git developers work on Linux and they have no way to know if their changes would break the Git for Windows build. Let's fix that by adding a job to TravisCI that builds and tests Git on Windows. Unfortunately, TravisCI does not support Windows. Therefore, we did the following: * Johannes Schindelin set up a Visual Studio Team Services build sponsored by Microsoft and made it accessible via an Azure Function that speaks a super-simple API. We made TravisCI use this API to trigger a build, wait until its completion, and print the build and test results. * A Windows build and test run takes up to 3h and TravisCI has a timeout after 50min for Open Source projects. Since the TravisCI job does not use heavy CPU/memory/etc. resources, the friendly TravisCI folks extended the job timeout for git/git to 3h. Things, that would need to be done: * Someone with write access to https://travis-ci.org/git/git would need to add the secret token as "GFW_CI_TOKEN" variable in the TravisCI repository setting [1]. Afterwards the build should just work. Things, that might need to be done: * The Windows box can only process a single build at a time. A second Windows build would need to wait until the first finishes. This waiting time and the build time after the wait could exceed the 3h threshold. If this is a problem, then it is likely to happen every day as usually multiple branches are pushed at the same time (pu/next/ master/maint). I cannot test this as my TravisCI account has the 50min timeout. One solution could be to limit the number of concurrent TravisCI jobs [2]. [1] https://docs.travis-ci.com/user/environment-variables#Defining-Variables-in-Repository-Settings [2] https://docs.travis-ci.com/user/customizing-the-build#Limiting-Concurrent-Builds Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-24 11:37:47 +00:00
test $# -ne 2 && echo "Unexpected number of parameters" && exit 1
test -z "$GFW_CI_TOKEN" && echo "GFW_CI_TOKEN not defined" && exit
BRANCH=$1
COMMIT=$2
gfwci () {
local CURL_ERROR_CODE HTTP_CODE
CONTENT_FILE=$(mktemp -t "git-windows-ci-XXXXXX")
while test -z $HTTP_CODE
do
travis-ci: build and test Git on Windows Most Git developers work on Linux and they have no way to know if their changes would break the Git for Windows build. Let's fix that by adding a job to TravisCI that builds and tests Git on Windows. Unfortunately, TravisCI does not support Windows. Therefore, we did the following: * Johannes Schindelin set up a Visual Studio Team Services build sponsored by Microsoft and made it accessible via an Azure Function that speaks a super-simple API. We made TravisCI use this API to trigger a build, wait until its completion, and print the build and test results. * A Windows build and test run takes up to 3h and TravisCI has a timeout after 50min for Open Source projects. Since the TravisCI job does not use heavy CPU/memory/etc. resources, the friendly TravisCI folks extended the job timeout for git/git to 3h. Things, that would need to be done: * Someone with write access to https://travis-ci.org/git/git would need to add the secret token as "GFW_CI_TOKEN" variable in the TravisCI repository setting [1]. Afterwards the build should just work. Things, that might need to be done: * The Windows box can only process a single build at a time. A second Windows build would need to wait until the first finishes. This waiting time and the build time after the wait could exceed the 3h threshold. If this is a problem, then it is likely to happen every day as usually multiple branches are pushed at the same time (pu/next/ master/maint). I cannot test this as my TravisCI account has the 50min timeout. One solution could be to limit the number of concurrent TravisCI jobs [2]. [1] https://docs.travis-ci.com/user/environment-variables#Defining-Variables-in-Repository-Settings [2] https://docs.travis-ci.com/user/customizing-the-build#Limiting-Concurrent-Builds Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-24 11:37:47 +00:00
HTTP_CODE=$(curl \
-H "Authentication: Bearer $GFW_CI_TOKEN" \
--silent --retry 5 --write-out '%{HTTP_CODE}' \
--output >(sed "$(printf '1s/^\xef\xbb\xbf//')" >$CONTENT_FILE) \
travis-ci: build and test Git on Windows Most Git developers work on Linux and they have no way to know if their changes would break the Git for Windows build. Let's fix that by adding a job to TravisCI that builds and tests Git on Windows. Unfortunately, TravisCI does not support Windows. Therefore, we did the following: * Johannes Schindelin set up a Visual Studio Team Services build sponsored by Microsoft and made it accessible via an Azure Function that speaks a super-simple API. We made TravisCI use this API to trigger a build, wait until its completion, and print the build and test results. * A Windows build and test run takes up to 3h and TravisCI has a timeout after 50min for Open Source projects. Since the TravisCI job does not use heavy CPU/memory/etc. resources, the friendly TravisCI folks extended the job timeout for git/git to 3h. Things, that would need to be done: * Someone with write access to https://travis-ci.org/git/git would need to add the secret token as "GFW_CI_TOKEN" variable in the TravisCI repository setting [1]. Afterwards the build should just work. Things, that might need to be done: * The Windows box can only process a single build at a time. A second Windows build would need to wait until the first finishes. This waiting time and the build time after the wait could exceed the 3h threshold. If this is a problem, then it is likely to happen every day as usually multiple branches are pushed at the same time (pu/next/ master/maint). I cannot test this as my TravisCI account has the 50min timeout. One solution could be to limit the number of concurrent TravisCI jobs [2]. [1] https://docs.travis-ci.com/user/environment-variables#Defining-Variables-in-Repository-Settings [2] https://docs.travis-ci.com/user/customizing-the-build#Limiting-Concurrent-Builds Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-24 11:37:47 +00:00
"https://git-for-windows-ci.azurewebsites.net/api/TestNow?$1" \
)
CURL_ERROR_CODE=$?
# The GfW CI web app sometimes returns HTTP errors of
# "502 bad gateway" or "503 service unavailable".
# We also need to check the HTTP content because the GfW web
# app seems to pass through (error) results from other Azure
# calls with HTTP code 200.
# Wait a little and retry if we detect this error. More info:
# https://docs.microsoft.com/en-in/azure/app-service-web/app-service-web-troubleshoot-http-502-http-503
if test $HTTP_CODE -eq 502 ||
test $HTTP_CODE -eq 503 ||
grep "502 - Web server received an invalid response" $CONTENT_FILE >/dev/null
then
sleep 10
HTTP_CODE=
fi
done
cat $CONTENT_FILE
rm $CONTENT_FILE
travis-ci: build and test Git on Windows Most Git developers work on Linux and they have no way to know if their changes would break the Git for Windows build. Let's fix that by adding a job to TravisCI that builds and tests Git on Windows. Unfortunately, TravisCI does not support Windows. Therefore, we did the following: * Johannes Schindelin set up a Visual Studio Team Services build sponsored by Microsoft and made it accessible via an Azure Function that speaks a super-simple API. We made TravisCI use this API to trigger a build, wait until its completion, and print the build and test results. * A Windows build and test run takes up to 3h and TravisCI has a timeout after 50min for Open Source projects. Since the TravisCI job does not use heavy CPU/memory/etc. resources, the friendly TravisCI folks extended the job timeout for git/git to 3h. Things, that would need to be done: * Someone with write access to https://travis-ci.org/git/git would need to add the secret token as "GFW_CI_TOKEN" variable in the TravisCI repository setting [1]. Afterwards the build should just work. Things, that might need to be done: * The Windows box can only process a single build at a time. A second Windows build would need to wait until the first finishes. This waiting time and the build time after the wait could exceed the 3h threshold. If this is a problem, then it is likely to happen every day as usually multiple branches are pushed at the same time (pu/next/ master/maint). I cannot test this as my TravisCI account has the 50min timeout. One solution could be to limit the number of concurrent TravisCI jobs [2]. [1] https://docs.travis-ci.com/user/environment-variables#Defining-Variables-in-Repository-Settings [2] https://docs.travis-ci.com/user/customizing-the-build#Limiting-Concurrent-Builds Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-24 11:37:47 +00:00
if test $CURL_ERROR_CODE -ne 0
then
return $CURL_ERROR_CODE
fi
if test "$HTTP_CODE" -ge 400 && test "$HTTP_CODE" -lt 600
then
return 127
fi
}
# Trigger build job
BUILD_ID=$(gfwci "action=trigger&branch=$BRANCH&commit=$COMMIT&skipTests=false")
if test $? -ne 0
then
echo "Unable to trigger Visual Studio Team Services Build"
echo "$BUILD_ID"
exit 1
fi
# Check if the $BUILD_ID contains a number
case $BUILD_ID in
''|*[!0-9]*) echo "Unexpected build number: $BUILD_ID" && exit 1
esac
echo "Visual Studio Team Services Build #${BUILD_ID}"
# Tracing execued commands would produce too much noise in the waiting
# loop below.
set +x
travis-ci: build and test Git on Windows Most Git developers work on Linux and they have no way to know if their changes would break the Git for Windows build. Let's fix that by adding a job to TravisCI that builds and tests Git on Windows. Unfortunately, TravisCI does not support Windows. Therefore, we did the following: * Johannes Schindelin set up a Visual Studio Team Services build sponsored by Microsoft and made it accessible via an Azure Function that speaks a super-simple API. We made TravisCI use this API to trigger a build, wait until its completion, and print the build and test results. * A Windows build and test run takes up to 3h and TravisCI has a timeout after 50min for Open Source projects. Since the TravisCI job does not use heavy CPU/memory/etc. resources, the friendly TravisCI folks extended the job timeout for git/git to 3h. Things, that would need to be done: * Someone with write access to https://travis-ci.org/git/git would need to add the secret token as "GFW_CI_TOKEN" variable in the TravisCI repository setting [1]. Afterwards the build should just work. Things, that might need to be done: * The Windows box can only process a single build at a time. A second Windows build would need to wait until the first finishes. This waiting time and the build time after the wait could exceed the 3h threshold. If this is a problem, then it is likely to happen every day as usually multiple branches are pushed at the same time (pu/next/ master/maint). I cannot test this as my TravisCI account has the 50min timeout. One solution could be to limit the number of concurrent TravisCI jobs [2]. [1] https://docs.travis-ci.com/user/environment-variables#Defining-Variables-in-Repository-Settings [2] https://docs.travis-ci.com/user/customizing-the-build#Limiting-Concurrent-Builds Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-24 11:37:47 +00:00
# Wait until build job finished
STATUS=
RESULT=
while true
do
LAST_STATUS=$STATUS
STATUS=$(gfwci "action=status&buildId=$BUILD_ID")
test "$STATUS" = "$LAST_STATUS" || printf "\nStatus: %s " "$STATUS"
travis-ci: build and test Git on Windows Most Git developers work on Linux and they have no way to know if their changes would break the Git for Windows build. Let's fix that by adding a job to TravisCI that builds and tests Git on Windows. Unfortunately, TravisCI does not support Windows. Therefore, we did the following: * Johannes Schindelin set up a Visual Studio Team Services build sponsored by Microsoft and made it accessible via an Azure Function that speaks a super-simple API. We made TravisCI use this API to trigger a build, wait until its completion, and print the build and test results. * A Windows build and test run takes up to 3h and TravisCI has a timeout after 50min for Open Source projects. Since the TravisCI job does not use heavy CPU/memory/etc. resources, the friendly TravisCI folks extended the job timeout for git/git to 3h. Things, that would need to be done: * Someone with write access to https://travis-ci.org/git/git would need to add the secret token as "GFW_CI_TOKEN" variable in the TravisCI repository setting [1]. Afterwards the build should just work. Things, that might need to be done: * The Windows box can only process a single build at a time. A second Windows build would need to wait until the first finishes. This waiting time and the build time after the wait could exceed the 3h threshold. If this is a problem, then it is likely to happen every day as usually multiple branches are pushed at the same time (pu/next/ master/maint). I cannot test this as my TravisCI account has the 50min timeout. One solution could be to limit the number of concurrent TravisCI jobs [2]. [1] https://docs.travis-ci.com/user/environment-variables#Defining-Variables-in-Repository-Settings [2] https://docs.travis-ci.com/user/customizing-the-build#Limiting-Concurrent-Builds Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-24 11:37:47 +00:00
printf "."
case "$STATUS" in
inProgress|postponed|notStarted) sleep 10 ;; # continue
"completed: succeeded") RESULT="success"; break;; # success
"completed: failed") break;; # failure
*) echo "Unhandled status: $STATUS"; break;; # unknown
travis-ci: build and test Git on Windows Most Git developers work on Linux and they have no way to know if their changes would break the Git for Windows build. Let's fix that by adding a job to TravisCI that builds and tests Git on Windows. Unfortunately, TravisCI does not support Windows. Therefore, we did the following: * Johannes Schindelin set up a Visual Studio Team Services build sponsored by Microsoft and made it accessible via an Azure Function that speaks a super-simple API. We made TravisCI use this API to trigger a build, wait until its completion, and print the build and test results. * A Windows build and test run takes up to 3h and TravisCI has a timeout after 50min for Open Source projects. Since the TravisCI job does not use heavy CPU/memory/etc. resources, the friendly TravisCI folks extended the job timeout for git/git to 3h. Things, that would need to be done: * Someone with write access to https://travis-ci.org/git/git would need to add the secret token as "GFW_CI_TOKEN" variable in the TravisCI repository setting [1]. Afterwards the build should just work. Things, that might need to be done: * The Windows box can only process a single build at a time. A second Windows build would need to wait until the first finishes. This waiting time and the build time after the wait could exceed the 3h threshold. If this is a problem, then it is likely to happen every day as usually multiple branches are pushed at the same time (pu/next/ master/maint). I cannot test this as my TravisCI account has the 50min timeout. One solution could be to limit the number of concurrent TravisCI jobs [2]. [1] https://docs.travis-ci.com/user/environment-variables#Defining-Variables-in-Repository-Settings [2] https://docs.travis-ci.com/user/customizing-the-build#Limiting-Concurrent-Builds Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-24 11:37:47 +00:00
esac
done
# Print log
echo ""
echo ""
set -x
travis-ci: build and test Git on Windows Most Git developers work on Linux and they have no way to know if their changes would break the Git for Windows build. Let's fix that by adding a job to TravisCI that builds and tests Git on Windows. Unfortunately, TravisCI does not support Windows. Therefore, we did the following: * Johannes Schindelin set up a Visual Studio Team Services build sponsored by Microsoft and made it accessible via an Azure Function that speaks a super-simple API. We made TravisCI use this API to trigger a build, wait until its completion, and print the build and test results. * A Windows build and test run takes up to 3h and TravisCI has a timeout after 50min for Open Source projects. Since the TravisCI job does not use heavy CPU/memory/etc. resources, the friendly TravisCI folks extended the job timeout for git/git to 3h. Things, that would need to be done: * Someone with write access to https://travis-ci.org/git/git would need to add the secret token as "GFW_CI_TOKEN" variable in the TravisCI repository setting [1]. Afterwards the build should just work. Things, that might need to be done: * The Windows box can only process a single build at a time. A second Windows build would need to wait until the first finishes. This waiting time and the build time after the wait could exceed the 3h threshold. If this is a problem, then it is likely to happen every day as usually multiple branches are pushed at the same time (pu/next/ master/maint). I cannot test this as my TravisCI account has the 50min timeout. One solution could be to limit the number of concurrent TravisCI jobs [2]. [1] https://docs.travis-ci.com/user/environment-variables#Defining-Variables-in-Repository-Settings [2] https://docs.travis-ci.com/user/customizing-the-build#Limiting-Concurrent-Builds Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-24 11:37:47 +00:00
gfwci "action=log&buildId=$BUILD_ID" | cut -c 30-
# Set exit code for TravisCI
test "$RESULT" = "success"
travis-ci: record and skip successfully built trees Travis CI dutifully builds and tests each new branch tip, even if its tree has previously been successfully built and tested. This happens often enough in contributors' workflows, when a work-in-progress branch is rebased changing e.g. only commit messages or the order or number of commits while leaving the resulting code intact, and is then pushed to a Travis CI-enabled GitHub fork. This is wasting Travis CI's resources and is sometimes scary-annoying when the new tip commit with a tree identical to the previous, successfully tested one is suddenly reported in red, because one of the OSX build jobs happened to exceed the time limit yet again. So extend our Travis CI build scripts to skip building commits whose trees have previously been successfully built and tested. Use the Travis CI cache feature to keep a record of the object names of trees that tested successfully, in a plain and simple flat text file, one line per tree object name. Append the current tree's object name at the end of every successful build job to this file, along with a bit of additional info about the build job (commit object name, Travis CI job number and id). Limit the size of this file to 1000 records, to prevent it from growing too large for git/git's forever living integration branches. Check, using a simple grep invocation, in each build job whether the current commit's tree is already in there, and skip the build if it is. Include a message in the skipped build job's trace log, containing the URL to the build job successfully testing that tree for the first time and instructions on how to force a re-build. Catch the case when a build job, which successfully built and tested a particular tree for the first time, is restarted and omit the URL of the previous build job's trace log, as in this case it's the same build job and the trace log has just been overwritten. Note: this won't kick in if two identical trees are on two different branches, because Travis CI caches are not shared between build jobs of different branches. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Reviewed-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-31 10:12:05 +00:00
save_good_tree