Commit graph

95 commits

Author SHA1 Message Date
Junio C Hamano
763f20fb4a Merge branch 'tb/ci-concurrency'
Avoid unnecessary builds in CI, with settings configured in
ci-config.

* tb/ci-concurrency:
  ci: avoid unnecessary builds
2023-01-16 12:07:46 -08:00
Junio C Hamano
42f9a60013 Merge branch 'pw/ci-print-failure-name-fix'
(cosmetic) CI regression fix.

* pw/ci-print-failure-name-fix:
  ci(github): restore "print test failures" step name
2023-01-16 12:07:45 -08:00
Junio C Hamano
7ec4cccaa5 Merge branch 'cw/ci-whitespace'
CI updates.  We probably want a clean-up to move the long shell
script embedded in yaml file into a separate file, but that can
come later.

* cw/ci-whitespace:
  ci (check-whitespace): move to actions/checkout@v3
  ci (check-whitespace): add links to job output
  ci (check-whitespace): suggest fixes for errors
2023-01-08 13:25:20 +09:00
Phillip Wood
7b341645e3 ci(github): restore "print test failures" step name
As well as removing the explicit shell setting d8b21a0fe2 (CI: don't
explicitly pick "bash" shell outside of Windows, fix regression,
2022-12-07) also reverted the name of the print test failures step
introduced by 5aeb145780 (ci(github): bring back the 'print test
failures' step, 2022-06-08). This is unfortunate as 5aeb145780 added a
message to direct contributors to the "print test failures" step when a
test fails and that step is no-longer known by that name on the
non-windows ci jobs.

In principle we could update the message to print the correct name for
the step but then we'd have to deal with having two different names for
the same step on different jobs. It is simpler for the implementation
and contributors to use the same name for this step on all jobs.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-04 15:16:15 +09:00
Chris. Webster
4542582e59 ci (check-whitespace): move to actions/checkout@v3
Get rid of deprecation warnings in the CI runs.  Also gets the latest
security patches.

Signed-off-by: Chris. Webster <chris@webstech.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-20 10:48:19 +09:00
Chris. Webster
b3ecdc780d ci (check-whitespace): add links to job output
A message in the step log will refer to the Summary output.

The job summary output is using markdown to improve readability.  The
git commands and commits with errors are now in ordered lists.
Commits and files in error are links to the user's repository.

Signed-off-by: Chris. Webster <chris@webstech.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-20 10:48:18 +09:00
Chris. Webster
288e3c4e3b ci (check-whitespace): suggest fixes for errors
Make the errors more visible by adding them to the job summary and
display the git commands that will usually fix the problem.

Signed-off-by: Chris. Webster <chris@webstech.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-20 10:48:17 +09:00
Johannes Schindelin
a0da6deeec ci: only run win+VS build & tests in Git for Windows' fork
It has been a frequent matter of contention that the win+VS jobs not
only take a long time to run, but are also more easily broken than the
other jobs (because they do not use the same `Makefile`-based builds as
all other jobs), and to make matters worse, these breakages are also
much harder to diagnose and fix than other jobs', especially for
contributors who are happy to stay away from Windows.

The purpose of these win+VS jobs is to maintain the CMake-based build
of Git, with the target audience being Visual Studio users on Windows
who are typically quite unfamiliar with `make` and POSIX shell
scripting, but the benefit of whose expertise we want for the Git
project nevertheless.

The CMake support was introduced for that specific purpose, and already
early on concerns were raised that it would put an undue burden on
contributors to ensure that these jobs pass in CI, when they do not have
access to Windows machines (nor want to have that).

This developer's initial hope was that it would be enough to fix win+VS
failures and provide the changes to be squashed into contributors'
patches, and that it would be worth the benefit of attracting
Windows-based developers' contributions.

Neither of these hopes have panned out.

To lower the frustration, and incidentally benefit from using way less
build minutes, let's just not run the win+VS jobs by default, which
appears to be the consensus of the mail thread leading up to
https://lore.kernel.org/git/xmqqk0311blt.fsf@gitster.g/

Since the Git for Windows project still needs to at least try to attract
more of said Windows-based developers, let's keep the jobs, but disable
them everywhere except in Git for Windows' fork. This will help because
Git for Windows' branch thicket is "continuously rebased" via automation
to the `shears/maint`, `shears/main`, `shears/next` and `shears/seen`
branches at https://github.com/git-for-windows/git. That way, the Git
for Windows project will still be notified early on about potential
breakages, but the Git project won't be burdened with fixing them
anymore, which seems to be the best compromise we can get on this issue.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-20 10:45:37 +09:00
Junio C Hamano
481d274aae Merge branch 'js/ci-use-newer-up-down-artifact'
CI fix.

* js/ci-use-newer-up-down-artifact:
  ci: avoid using deprecated {up,down}load-artifacts Action
2022-12-10 14:01:06 +09:00
Junio C Hamano
0b32d1aea2 Merge branch 'ab/ci-use-macos-12'
CI fix.

* ab/ci-use-macos-12:
  CI: upgrade to macos-12, and pin OSX version
2022-12-10 14:01:06 +09:00
Junio C Hamano
82444ead4c Merge branch 'ab/ci-retire-set-output'
CI fix.

* ab/ci-retire-set-output:
  CI: migrate away from deprecated "set-output" syntax
2022-12-10 14:01:05 +09:00
Junio C Hamano
a64bf54bfa Merge branch 'ab/ci-musl-bash-fix'
CI fix.

* ab/ci-musl-bash-fix:
  CI: don't explicitly pick "bash" shell outside of Windows, fix regression
2022-12-10 14:01:05 +09:00
Junio C Hamano
9044a398af Merge branch 'od/ci-use-checkout-v3-when-applicable'
Update GitHub CI to use actions/checkout@v3; use of the older
checkout@v2 gets annoying deprecation notices.

* od/ci-use-checkout-v3-when-applicable:
  ci(main): upgrade actions/checkout to v3
2022-12-10 14:01:05 +09:00
Ævar Arnfjörð Bjarmason
f115c96e7a CI: migrate away from deprecated "set-output" syntax
As noted in [1] and the warnings the CI itself is spewing echoing
outputs to stdout is deprecated, and they should be written to
"$GITHUB_OUTPUT" instead.

1. https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-08 08:47:22 +09:00
Johannes Schindelin
1f398446c3 ci: avoid using deprecated {up,down}load-artifacts Action
The deprecated versions of these Actions still use node.js 12 whereas
workflows will need to use node.js 16 to avoid problems going forward.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-08 08:15:23 +09:00
Ævar Arnfjörð Bjarmason
d8b21a0fe2 CI: don't explicitly pick "bash" shell outside of Windows, fix regression
When the "js/ci-github-workflow-markup" topic was originally merged in
[1] it included a change to get rid of the "ci/print-test-failures.sh"
step[2]. This was then brought back in [3] as part of a fix-up patches
on top[4].

The problem was that [3] was not a revert of the relevant parts of
[2], but rather copy/pasted the "ci/print-test-failures.sh" step that
was present for the Windows job to all "ci/print-test-failures.sh"
steps. The Windows steps specified "shell: bash", but the non-Windows
ones did not.

This broke the "ci/print/test-failures.sh" step for the "linux-musl"
job, where we don't have a "bash" shell, just a "/bin/sh" (a
"dash"). This breakage was reported at the time[5], but hadn't been
fixed.

It would be sufficient to change this only for "linux-musl", but let's
change this for both "regular" and "dockerized" to omit the "shell"
line entirely, as we did before [2].

Let's also change undo the "name" change that [3] made while
copy/pasting the "print test failures" step for the Windows job. These
steps are now the same as they were before [2], except that the "if"
includes the "env.FAILED_TEST_ARTIFACTS" test.

1. fc5a070f59 (Merge branch 'js/ci-github-workflow-markup', 2022-06-07)
2. 08dccc8fc1 (ci: make it easier to find failed tests' logs in the
   GitHub workflow, 2022-05-21)
3. 5aeb145780 (ci(github): bring back the 'print test failures' step,
   2022-06-08)
4. d0d96b8280 (Merge branch 'js/ci-github-workflow-markup', 2022-06-17)
5. https://lore.kernel.org/git/220725.86sfmpneqp.gmgdl@evledraar.gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-08 08:06:00 +09:00
Ævar Arnfjörð Bjarmason
d11192255d CI: upgrade to macos-12, and pin OSX version
Per [1] and the warnings our CI is emitting GitHub is phasing in
"macos-12" as their "macos-latest".

As with [2], let's pin our image to a specific version so that we're
not having it swept from under us, and our upgrade cycle can be more
predictable than whenever GitHub changes their images.

1. https://github.com/actions/runner-images/issues/6384
2. 0178420b9c (github-actions: run gcc-8 on ubuntu-20.04 image,
   2022-11-25)

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-07 13:36:22 +09:00
Oscar Dominguez
6cf4d908a9 ci(main): upgrade actions/checkout to v3
To be up to date with actions/checkout opens the door to use the latest
features if necessary and get the latest security patches.

This also avoids a couple of deprecation warnings in the CI runs.

Note: The `actions/checkout` Action has been known to be broken in i686
containers as of v2, therefore we keep forcing it to v1 there. See
actions/runner#2115 for more details.

Signed-off-by: Oscar Dominguez <dominguez.celada@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-06 08:22:15 +09:00
Junio C Hamano
515ffabccf Merge branch 'jx/ci-ubuntu-fix'
Adjust the GitHub CI to newer ubuntu release.

* jx/ci-ubuntu-fix:
  ci: install python on ubuntu
  ci: use the same version of p4 on both Linux and macOS
  ci: remove the pipe after "p4 -V" to catch errors
  github-actions: run gcc-8 on ubuntu-20.04 image
2022-11-29 10:41:06 +09:00
Jiang Xin
0178420b9c github-actions: run gcc-8 on ubuntu-20.04 image
GitHub starts to upgrade its runner image "ubuntu-latest" from version
"ubuntu-20.04" to version "ubuntu-22.04". It will fail to find and
install "gcc-8" package on the new runner image.

Change some of the runner images from "ubuntu-latest" to "ubuntu-20.04"
in order to install "gcc-8" as a dependency.

The first revision of this patch tried to replace "$runs_on_pool" in
"ci/*.sh" with a new "$runs_on_os" environment variable based on the
"os" field in the matrix strategy. But these "os" fields in matrix
strategies are obsolete legacies from commit [1] and commit [2], and
are no longer useful. So remove these unused "os" fields.

[1]: c08bb26010 (CI: rename the "Linux32" job to lower-case "linux32",
                 2021-11-23)
[2]: 25715419bf (CI: don't run "make test" twice in one job, 2021-11-23)

Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-27 09:31:12 +09:00
Johannes Schindelin
63357b79c9 ci: use a newer github-script version
The old version we currently use runs in node.js v12.x, which is being
deprecated in GitHub Actions. The new version uses node.js v16.x.

Incidentally, this also avoids the warning about the deprecated
`::set-output::` workflow command because the newer version of the
`github-script` Action uses the recommended new way to specify outputs.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-08 15:35:13 -05:00
Taylor Blau
eb5b03a9c0 ci: avoid unnecessary builds
Whenever a branch is pushed to a repository which has GitHub Actions
enabled, a bunch of new workflow runs are started.

We sometimes see contributors push multiple branch updates in rapid
succession, which in conjunction with the impressive time swallowed by
even just a single CI build frequently leads to many queued-up runs.

This is particularly problematic in the case of Pull Requests where a
single contributor can easily (inadvertently) prevent timely builds for
other contributors when using a shared repository.

To help with this situation, let's use the `concurrency` feature of
GitHub workflows, essentially canceling GitHub workflow runs that are
obsoleted by more recent runs:

  https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#concurrency

For workflows that *do* want the behavior in the pre-image of this
patch, they can use the ci-config feature to disable the new behavior by
adding an executable script on the ci-config branch called
'skip-concurrent' which terminates with a non-zero exit code.

Original-patch-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-08 13:26:20 -05:00
Junio C Hamano
1c0962c0c4 ci: add address and undefined sanitizer tasks
The current code is clean with these two sanitizers, and we would
like to keep it that way by running the checks for any new code.

The signal of "passed with asan, but not ubsan" (or vice versa) is
not that useful in practice, so it is tempting to run both santizers
in a single task, but it seems to take forever, so tentatively let's
try having two separate ones.

Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-20 09:20:59 -07:00
Derrick Stolee
ef46584831 ci: update 'static-analysis' to Ubuntu 22.04
GitHub Actions scheduled a brownout of Ubuntu 18.04, which canceled all
runs of the 'static-analysis' job in our CI runs. Update to 22.04 to
avoid this as the brownout later turns into a complete deprecation.

The use of 18.04 was set in d051ed77ee (.github/workflows/main.yml: run
static-analysis on bionic, 2021-02-08) due to the lack of Coccinelle
being available on 20.04 (which continues today).

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-24 13:02:12 -07:00
Johannes Schindelin
5aeb145780 ci(github): bring back the 'print test failures' step
Git now shows better information in the GitHub workflow runs when a test
case failed. However, when a test case was implemented incorrectly and
therefore does not even run, nothing is shown.

Let's bring back the step that prints the full logs of the failed tests,
and to improve the user experience, print out an informational message
for readers so that they do not have to know/remember where to see the
full logs.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-08 16:12:37 -07:00
Johannes Schindelin
08dccc8fc1 ci: make it easier to find failed tests' logs in the GitHub workflow
When investigating a test failure, the time that matters most is the
time it takes from getting aware of the failure to displaying the output
of the failing test case.

You currently have to know a lot of implementation details when
investigating test failures in the CI runs. The first step is easy: the
failed job is marked quite clearly, but when opening it, the failed step
is expanded, which in our case is the one running
`ci/run-build-and-tests.sh`. This step, most notably, only offers a
high-level view of what went wrong: it prints the output of `prove`
which merely tells the reader which test script failed.

The actually interesting part is in the detailed log of said failed
test script. But that log is shown in the CI run's step that runs
`ci/print-test-failures.sh`. And that step is _not_ expanded in the web
UI by default. It is even marked as "successful", which makes it very
easy to miss that there is useful information hidden in there.

Let's help the reader by showing the failed tests' detailed logs in the
step that is expanded automatically, i.e. directly after the test suite
failed.

This also helps the situation where the _build_ failed and the
`print-test-failures` step was executed under the assumption that the
_test suite_ failed, and consequently failed to find any failed tests.

An alternative way to implement this patch would be to source
`ci/print-test-failures.sh` in the `handle_test_failures` function to
show these logs. However, over the course of the next few commits, we
want to introduce some grouping which would be harder to achieve that
way (for example, we do want a leaner, and colored, preamble for each
failed test script, and it would be trickier to accommodate the lack of
nested groupings in GitHub workflows' output).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-21 16:25:56 -07:00
Junio C Hamano
f9b889dd67 Merge branch 'ab/ci-updates'
Drop support for TravisCI and update test workflows at GitHub.

* ab/ci-updates:
  CI: don't run "make test" twice in one job
  CI: use "$runs_on_pool", not "$jobname" to select packages & config
  CI: rename the "Linux32" job to lower-case "linux32"
  CI: use shorter names that fit in UX tooltips
  CI: remove Travis CI support
2021-12-15 09:39:49 -08:00
Junio C Hamano
85ac30ff5c Merge branch 'hk/ci-checkwhitespace-commentfix'
Comment fix.

* hk/ci-checkwhitespace-commentfix:
  ci(check-whitespace): update stale file top comments
2021-12-10 14:35:12 -08:00
Junio C Hamano
bd16b3c39f Merge branch 'js/ci-no-directional-formatting'
CI has been taught to catch some Unicode directional formatting
sequence that can be used in certain mischief.

* js/ci-no-directional-formatting:
  ci: disallow directional formatting
2021-12-10 14:35:06 -08:00
Ævar Arnfjörð Bjarmason
25715419bf CI: don't run "make test" twice in one job
The "linux-clang" and "linux-gcc" jobs both run "make test" twice, but
with different environment variables. Running these in sequence seems
to have been done to work around some constraint on Travis, see
ae59a4e44f (travis: run tests with GIT_TEST_SPLIT_INDEX, 2018-01-07).

By having these run in parallel we'll get jobs that finish much sooner
than they otherwise would have.

We can also simplify the control flow in "ci/run-build-and-tests.sh"
as a result, since we won't run "make test" twice we don't need to run
"make" twice at all, let's default to "make all test" after setting
the variables, and then override it to just "all" for the compile-only
tests.

Add a comment to clarify that new "test" targets should adjust
$MAKE_TARGETS rather than being added after the "case/esac". This
should avoid future confusion where e.g. the compilation-only
"pedantic" target will unexpectedly start running tests. See [1] and
[2].

1. https://lore.kernel.org/git/211122.86ee78yxts.gmgdl@evledraar.gmail.com/
2. https://lore.kernel.org/git/211123.86ilwjujmd.gmgdl@evledraar.gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-23 16:51:54 -08:00
Ævar Arnfjörð Bjarmason
707d2f2fe8 CI: use "$runs_on_pool", not "$jobname" to select packages & config
Change the setup hooks for the CI to use "$runs_on_pool" for the
"$regular" job. Now we won't need as much boilerplate when adding new
jobs to the "regular" matrix, see 956d2e4639 (tests: add a test mode
for SANITIZE=leak, run it in CI, 2021-09-23) for the last such commit.

I.e. now instead of needing to enumerate each jobname when we select
packages we can install things depending on the pool we're running
in.

That we didn't do this dates back to the now gone dependency on Travis
CI, but even if we add a new CI target in the future this'll be easier
to port over, since we can probably treat "ubuntu-latest" as a
stand-in for some recent Linux that can run "apt" commands.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-23 16:51:53 -08:00
Ævar Arnfjörð Bjarmason
c08bb26010 CI: rename the "Linux32" job to lower-case "linux32"
As a follow-up to the preceding commit's shortening of CI job names,
rename the only job that starts with an upper-case letter to be
consistent with the rest. It was added in 88dedd5e72 (Travis: also
test on 32-bit Linux, 2017-03-05).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-23 16:51:53 -08:00
Ævar Arnfjörð Bjarmason
df7375d772 CI: use shorter names that fit in UX tooltips
Change the names used for the GitHub CI workflows to be short enough
to (mostly) fit in the pop-up tool-tips that GitHub shows in the
commit view. I.e. when mouse-clicking on the passing or failing
check-mark next to the commit subject.

These names are seemingly truncated to 17-20 characters followed by
three dots ("..."). Since a "CI/PR / " prefix is added to them the job
names looked like this before (windows-test and vs-test jobs omitted):

    CI/PR / ci-config (p...
    CI/PR / windows-buil...
    CI/PR / vs-build (pu...
    CI/PR / regular (lin...
    CI/PR / regular (lin...
    CI/PR / regular (os...
    CI/PR / regular (os...
    CI/PR / regular (lin...
    CI/PR / regular (lin...
    CI/PR / dockerized (...
    CI/PR / dockerized (...
    CI/PR / dockerized (...
    CI/PR / static-anal...
    CI/PR / sparse (pu...
    CI/PR / documenta...

By omitting the "/PR" from the top-level name, and pushing the
$jobname to the front we'll now instead get:

    CI / config (push)
    CI / win build (push...
    CI / win+VS build (...
    CI / linux-clang (ub...
    CI / linux-gcc (ubun...
    CI / osx-clang (osx)...
    CI / osx-gcc (osx) (...
    CI / linux-gcc-defau...
    CI / linux-leaks (ub...
    CI / linux-musl (alp...
    CI / Linux32 (daald/...
    CI / pedantic (fedor...
    CI / static-analysis...
    CI / sparse (push)...
    CI / documentation

We then have no truncation in the expanded view. See [1] for how it
looked before, [2] for a currently visible CI run using this commit,
and [3] for the GitHub workflow syntax involved being changed here.

Let's also use the existing "pool" field as before. It's occasionally
useful to know we're running on say ubuntu v.s. fedora. The "-latest"
suffix is useful to some[4], and since it's now at the end it doesn't
hurt readability in the short view compared to saying "ubuntu" or
"macos".

1. https://github.com/git/git/tree/master/
2. https://github.com/avar/git/tree/avar/ci-rm-travis-cleanup-ci-names-3
3. https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions
3. https://lore.kernel.org/git/d9b07ca5-b58d-a535-d25b-85d7f12e6295@github.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-23 16:51:53 -08:00
Hans Krentel (hakre)
bf5b83fd8a ci(check-whitespace): update stale file top comments
Earlier a066a90d (ci(check-whitespace): restrict to the intended
commits, 2021-07-14) changed the check-whitespace task to stop using a
shallow clone, and cc003621 (ci(check-whitespace): stop requiring a
read/write token, 2021-07-14) changed the way how the errors the task
discovered is signaled back to the user.

They however forgot to update the comment that outlines what is done in
the task. Correct them.

Signed-off-by: Hans Krentel (hakre) <hanskrentel@yahoo.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-19 14:45:33 -08:00
Johannes Schindelin
0e7696c64d ci: disallow directional formatting
As described in https://trojansource.codes/trojan-source.pdf, it is
possible to abuse directional formatting (a feature of Unicode) to
deceive human readers into interpreting code differently from compilers.

For example, an "if ()" expression could be enclosed in a comment, but
rendered as if it was outside of that comment. In effect, this could
fool a reviewer into misinterpreting the code flow as benign when it is
not.

It is highly unlikely that Git's source code wants to contain such
directional formatting in the first place, so let's just disallow it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-04 10:13:10 -07:00
Junio C Hamano
ada1a17261 Merge branch 'js/windows-ci-path-fix'
The PATH used in CI job may be too wide and let incompatible dlls
to be grabbed, which can cause the build&test to fail.  Tighten it.

* js/windows-ci-path-fix:
  ci(windows): ensure that we do not pick up random executables
2021-10-18 15:47:58 -07:00
Junio C Hamano
223a1bfb58 Merge branch 'js/retire-preserve-merges'
The "--preserve-merges" option of "git rebase" has been removed.

* js/retire-preserve-merges:
  sequencer: restrict scope of a formerly public function
  rebase: remove a no-longer-used function
  rebase: stop mentioning the -p option in comments
  rebase: remove obsolete code comment
  rebase: drop the internal `rebase--interactive` command
  git-svn: drop support for `--preserve-merges`
  rebase: drop support for `--preserve-merges`
  pull: remove support for `--rebase=preserve`
  tests: stop testing `git rebase --preserve-merges`
  remote: warn about unhandled branch.<name>.rebase values
  t5520: do not use `pull.rebase=preserve`
2021-10-18 15:47:56 -07:00
Johannes Schindelin
7491ef6198 ci(windows): ensure that we do not pick up random executables
On the Windows build agents, a lot of programs are installed, and added
to the PATH automatically.

One such program is Git for Windows, and due to the way it is set up,
unfortunately its copy of `gpg.exe` is also reachable via the PATH.

This usually does not pose any problems. To the contrary, it even allows
us to test the GPG parts of Git's test suite even if `gpg.exe` is not
delivered as part of `git-sdk-64-minimal`, the minimal subset of Git for
Windows' SDK that we use in the CI builds to compile Git.

However, every once in a while we build a new MSYS2 runtime, which means
that there is a mismatch between the copy in `git-sdk-64-minimal` and
the copy in C:\Program Files\Git\usr\bin. When that happens we hit the
dreaded problem where only one `msys-2.0.dll` is expected to be in the
PATH, and things start to fail.

Let's avoid all of this by restricting the PATH to the minimal set. This
is actually done by `git-sdk-64-minimal`'s `/etc/profile`, and we just
have to source this file manually (one would expect that it is sourced
automatically, but the Bash steps in Azure Pipelines/GitHub workflows
are explicitly run using `--noprofile`, hence the need for doing this
explicitly).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-13 10:36:17 -07:00
Junio C Hamano
ff09581e12 Merge branch 'cb/ci-use-upload-artifacts-v1' into maint
Use upload-artifacts v1 (instead of v2) for 32-bit linux, as the
new version has a blocker bug for that architecture.

* cb/ci-use-upload-artifacts-v1:
  ci: use upload-artifacts v1 for dockerized jobs
2021-10-12 13:51:38 -07:00
Junio C Hamano
7bdcb8ef1f Merge branch 'cb/ci-build-pedantic' into maint
CI update.

* cb/ci-build-pedantic:
  ci: run a pedantic build as part of the GitHub workflow
2021-10-12 13:51:31 -07:00
Junio C Hamano
859a585bdf Merge branch 'ab/sanitize-leak-ci'
CI learns to run the leak sanitizer builds.

* ab/sanitize-leak-ci:
  tests: add a test mode for SANITIZE=leak, run it in CI
  Makefile: add SANITIZE=leak flag to GIT-BUILD-OPTIONS
2021-10-11 10:21:47 -07:00
Junio C Hamano
df9c83bdf7 Merge branch 'jx/ci-l10n'
CI help for l10n.

* jx/ci-l10n:
  ci: new github-action for git-l10n code review
2021-10-03 21:49:17 -07:00
Ævar Arnfjörð Bjarmason
956d2e4639 tests: add a test mode for SANITIZE=leak, run it in CI
While git can be compiled with SANITIZE=leak, we have not run
regression tests under that mode. Memory leaks have only been fixed as
one-offs without structured regression testing.

This change adds CI testing for it. We'll now build and small set of
whitelisted t00*.sh tests under Linux with a new job called
"linux-leaks".

The CI target uses a new GIT_TEST_PASSING_SANITIZE_LEAK=true test
mode. When running in that mode, we'll assert that we were compiled
with SANITIZE=leak. We'll then skip all tests, except those that we've
opted-in by setting "TEST_PASSES_SANITIZE_LEAK=true".

A test setting "TEST_PASSES_SANITIZE_LEAK=true" setting can in turn
make use of the "SANITIZE_LEAK" prerequisite, should they wish to
selectively skip tests even under
"GIT_TEST_PASSING_SANITIZE_LEAK=true". In the preceding commit we
started doing this in "t0004-unwritable.sh" under SANITIZE=leak, now
it'll combine nicely with "GIT_TEST_PASSING_SANITIZE_LEAK=true".

This is how tests that don't set "TEST_PASSES_SANITIZE_LEAK=true" will
be skipped under GIT_TEST_PASSING_SANITIZE_LEAK=true:

    $ GIT_TEST_PASSING_SANITIZE_LEAK=true ./t0001-init.sh
    1..0 # SKIP skip all tests in t0001 under SANITIZE=leak, TEST_PASSES_SANITIZE_LEAK not set

The intent is to add more TEST_PASSES_SANITIZE_LEAK=true annotations
as follow-up change, but let's start small to begin with.

In ci/run-build-and-tests.sh we make use of the default "*" case to
run "make test" without any GIT_TEST_* modes. SANITIZE=leak is known
to fail in combination with GIT_TEST_SPLIT_INDEX=true in
t0016-oidmap.sh, and we're likely to have other such failures in
various GIT_TEST_* modes. Let's focus on getting the base tests
passing, we can expand coverage to GIT_TEST_* modes later.

It would also be possible to implement a more lightweight version of
this by only relying on setting "LSAN_OPTIONS". See
<YS9OT/pn5rRK9cGB@coredump.intra.peff.net>[1] and
<YS9ZIDpANfsh7N+S@coredump.intra.peff.net>[2] for a discussion of
that. I've opted for this approach of adding a GIT_TEST_* mode instead
because it's consistent with how we handle other special test modes.

Being able to add a "!SANITIZE_LEAK" prerequisite and calling
"test_done" early if it isn't satisfied also means that we can more
incrementally add regression tests without being forced to fix
widespread and hard-to-fix leaks at the same time.

We have tests that do simple checking of some tool we're interested
in, but later on in the script might be stressing trace2, or common
sources of leaks like "git log" in combination with the tool (e.g. the
commit-graph tests). To be clear having a prerequisite could also be
accomplished by using "LSAN_OPTIONS" directly.

On the topic of "LSAN_OPTIONS": It would be nice to have a mode to
aggregate all failures in our various scripts, see [2] for a start at
doing that which sets "log_path" in "LSAN_OPTIONS". I've punted on
that for now, it can be added later.

As of writing this we've got major regressions between master..seen,
i.e. the t000*.sh tests and more fixed since 31f9acf9ce (Merge branch
'ah/plugleaks', 2021-08-04) have regressed recently.

See the discussion at <87czsv2idy.fsf@evledraar.gmail.com>[3] about
the lack of this sort of test mode, and 0e5bba53af (add UNLEAK
annotation for reducing leak false positives, 2017-09-08) for the
initial addition of SANITIZE=leak.

See also 09595ab381 (Merge branch 'jk/leak-checkers', 2017-09-19),
7782066f67 (Merge branch 'jk/apache-lsan', 2019-05-19) and the recent
936e58851a (Merge branch 'ah/plugleaks', 2021-05-07) for some of the
past history of "one-off" SANITIZE=leak (and more) fixes.

As noted in [5] we can't support this on OSX yet until Clang 14 is
released, at that point we'll probably want to resurrect that
"osx-leaks" job.

1. https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer
2. https://lore.kernel.org/git/YS9OT%2Fpn5rRK9cGB@coredump.intra.peff.net/
3. https://lore.kernel.org/git/87czsv2idy.fsf@evledraar.gmail.com/
4. https://lore.kernel.org/git/YS9ZIDpANfsh7N+S@coredump.intra.peff.net/
5. https://lore.kernel.org/git/20210916035603.76369-1-carenas@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-23 11:29:45 -07:00
Junio C Hamano
613204b948 Merge branch 'cb/ci-build-pedantic'
CI update.

* cb/ci-build-pedantic:
  ci: run a pedantic build as part of the GitHub workflow
2021-09-10 11:46:32 -07:00
Jiang Xin
a788d31931 ci: new github-action for git-l10n code review
The repository of git-l10n is a fork of "git/git" on GitHub, and uses
GitHub pull request for code review. A helper program "git-po-helper"
can be used to check typos in ".po" files, validate syntax, and check
commit messages. It would be convenient to integrate this helper program
to CI and add comments in pull request.

The new github-action workflow will be enabled for l10n related
operations, such as:

 * Operations on a repository named as "git-po", such as a repository
   forked from "git-l10n/git-po".

 * Push to a branch that contains "l10n" in the name.

 * Pull request from a remote branch which has "l10n" in the name, such
   as: "l10n/fix-fuzzy-translations".

The new l10n workflow listens to two types of github events:

    on: [push, pull_request_target]

The reason we use "pull_request_target" instead of "pull_request" is
that pull requests from forks receive a read-only GITHUB_TOKEN and
workflows cannot write comments back to pull requests for security
reasons. GitHub provides a "pull_request_target" event to resolve
security risks by checking out the base commit from the target
repository, and provide write permissions for the workflow.

By default, administrators can set strict permissions for workflows. The
following code is used to modify the permissions for the GITHUB_TOKEN
and grant write permission in order to create comments in pull-requests.

    permissions:
      pull-requests: write

This workflow will scan commits one by one. If a commit does not look
like a l10n commit (no file in "po/" has been changed), the scan process
will stop immediately. For a "push" event, no error will be reported
because it is normal to push non-l10n commits merged from upstream. But
for the "pull_request_target" event, errors will be reported. For this
reason, additional option is provided for "git-po-helper".

    git-po-helper check-commits \
        --github-action-event="${{ github.event_name }}" -- \
        <base>..<head>

The output messages of "git-po-helper" contain color codes not only for
console, but also for logfile. This is because "git-po-helper" uses a
package named "logrus" for logging, and I use an additional option
"ForceColor" to initialize "logrus" to print messages in a user-friendly
format in logfile output. These color codes help produce beautiful
output for the log of workflow, but they must be stripped off when
creating comments for pull requests. E.g.:

    perl -pe 's/\e\[[0-9;]*m//g' git-po-helper.out

"git-po-helper" may generate two kinds of suggestions, errors and
warnings. All the errors and warnings will be reported in the log of the
l10n workflow. However, warnings in the log of the workflow for a
successfully running "git-po-helper" can easily be ignored by users.
For the "pull_request_target" event, this issue is resolved by creating
an additional comment in the pull request. A l10n contributor should try
to fix all the errors, and should pay attention to the warnings.

Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-09 12:55:21 -07:00
Johannes Schindelin
aa4df107e7 tests: stop testing git rebase --preserve-merges
This backend has been deprecated in favor of `git rebase
--rebase-merges`.

In preparation for dropping it, let's remove all the regression tests
that would need it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Reviewed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-07 21:45:32 -07:00
Carlo Marcelo Arenas Belón
3cf9bb36bf ci: use upload-artifacts v1 for dockerized jobs
e9f79acb28 (ci: upgrade to using actions/{up,down}load-artifacts v2,
2021-06-23) changed all calls to that action from v1 to v2, but there
is still an open bug[1] that affects all nodejs actions and prevents
its use in 32-bit linux (as used by the Linux32 container)

move all dockerized jobs to use v1 that was built in C# and therefore
doesn't have this problem, which will otherwise manifest with confusing
messages like:

  /usr/bin/docker exec  0285adacc4536b7cd962079c46f85fa05a71e66d7905b5e4b9b1a0e8b305722a sh -c "cat /etc/*release | grep ^ID"
  OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: no such file or directory: unknown

[1] https://github.com/actions/runner/issues/1011

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-15 09:45:38 -07:00
Carlo Marcelo Arenas Belón
cebead1ebf ci: run a pedantic build as part of the GitHub workflow
similar to the recently added sparse task, it is nice to know as early
as possible.

add a dockerized build using fedora (that usually has the latest gcc)
to be ahead of the curve and avoid older ISO C issues at the same time.

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-11 11:25:06 -07:00
Junio C Hamano
8a49dfacd6 Merge branch 'js/ci-check-whitespace-updates'
CI update.

* js/ci-check-whitespace-updates:
  ci(check-whitespace): restrict to the intended commits
  ci(check-whitespace): stop requiring a read/write token
2021-08-02 14:06:40 -07:00
Junio C Hamano
6d56fb28fb Merge branch 'js/ci-make-sparse'
The CI gained a new job to run "make sparse" check.

* js/ci-make-sparse:
  ci/install-dependencies: handle "sparse" job package installs
  ci: run "apt-get update" before "apt-get install"
  ci: run `make sparse` as part of the GitHub workflow
2021-07-28 13:18:01 -07:00