Commit graph

140 commits

Author SHA1 Message Date
Junio C Hamano
27d7c8599b Merge branch 'js/default-branch-name-tests-final-stretch'
Prepare tests not to be affected by the name of the default branch
"git init" creates.

* js/default-branch-name-tests-final-stretch: (28 commits)
  tests: drop prereq `PREPARE_FOR_MAIN_BRANCH` where no longer needed
  t99*: adjust the references to the default branch name "main"
  tests(git-p4): transition to the default branch name `main`
  t9[5-7]*: adjust the references to the default branch name "main"
  t9[0-4]*: adjust the references to the default branch name "main"
  t8*: adjust the references to the default branch name "main"
  t7[5-9]*: adjust the references to the default branch name "main"
  t7[0-4]*: adjust the references to the default branch name "main"
  t6[4-9]*: adjust the references to the default branch name "main"
  t64*: preemptively adjust alignment to prepare for `master` -> `main`
  t6[0-3]*: adjust the references to the default branch name "main"
  t5[6-9]*: adjust the references to the default branch name "main"
  t55[4-9]*: adjust the references to the default branch name "main"
  t55[23]*: adjust the references to the default branch name "main"
  t551*: adjust the references to the default branch name "main"
  t550*: adjust the references to the default branch name "main"
  t5503: prepare aligned comment for replacing `master` with `main`
  t5[0-4]*: adjust the references to the default branch name "main"
  t5323: prepare centered comment for `master` -> `main`
  t4*: adjust the references to the default branch name "main"
  ...
2021-01-25 14:19:18 -08:00
Junio C Hamano
3831132ace ci/install-depends: attempt to fix "brew cask" stuff
We run "git pull" against "$cask_repo"; clarify that we are
expecting not to have any of our own modifications and running "git
pull" to merely update, by passing "--ff-only" on the command line.

Also, the "brew cask install" command line triggers an error message
that says:

    Error: Calling brew cask install is disabled! Use brew install
    [--cask] instead.

In addition, "brew install caskroom/cask/perforce" step triggers an
error that says:

    Error: caskroom/cask was moved. Tap homebrew/cask instead.

Attempt to see if blindly following the suggestion in these error
messages gets us into a better shape.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-14 19:08:56 -08:00
Johannes Schindelin
334afbc76f tests: mark tests relying on the current default for init.defaultBranch
In addition to the manual adjustment to let the `linux-gcc` CI job run
the test suite with `master` and then with `main`, this patch makes sure
that GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME is set in all test scripts
that currently rely on the initial branch name being `master by default.

To determine which test scripts to mark up, the first step was to
force-set the default branch name to `master` in

- all test scripts that contain the keyword `master`,

- t4211, which expects `t/t4211/history.export` with a hard-coded ref to
  initialize the default branch,

- t5560 because it sources `t/t556x_common` which uses `master`,

- t8002 and t8012 because both source `t/annotate-tests.sh` which also
  uses `master`)

This trick was performed by this command:

	$ sed -i '/^ *\. \.\/\(test-lib\|lib-\(bash\|cvs\|git-svn\)\|gitweb-lib\)\.sh$/i\
	GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master\
	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\
	' $(git grep -l master t/t[0-9]*.sh) \
	t/t4211*.sh t/t5560*.sh t/t8002*.sh t/t8012*.sh

After that, careful, manual inspection revealed that some of the test
scripts containing the needle `master` do not actually rely on a
specific default branch name: either they mention `master` only in a
comment, or they initialize that branch specificially, or they do not
actually refer to the current default branch. Therefore, the
aforementioned modification was undone in those test scripts thusly:

	$ git checkout HEAD -- \
		t/t0027-auto-crlf.sh t/t0060-path-utils.sh \
		t/t1011-read-tree-sparse-checkout.sh \
		t/t1305-config-include.sh t/t1309-early-config.sh \
		t/t1402-check-ref-format.sh t/t1450-fsck.sh \
		t/t2024-checkout-dwim.sh \
		t/t2106-update-index-assume-unchanged.sh \
		t/t3040-subprojects-basic.sh t/t3301-notes.sh \
		t/t3308-notes-merge.sh t/t3423-rebase-reword.sh \
		t/t3436-rebase-more-options.sh \
		t/t4015-diff-whitespace.sh t/t4257-am-interactive.sh \
		t/t5323-pack-redundant.sh t/t5401-update-hooks.sh \
		t/t5511-refspec.sh t/t5526-fetch-submodules.sh \
		t/t5529-push-errors.sh t/t5530-upload-pack-error.sh \
		t/t5548-push-porcelain.sh \
		t/t5552-skipping-fetch-negotiator.sh \
		t/t5572-pull-submodule.sh t/t5608-clone-2gb.sh \
		t/t5614-clone-submodules-shallow.sh \
		t/t7508-status.sh t/t7606-merge-custom.sh \
		t/t9302-fast-import-unpack-limit.sh

We excluded one set of test scripts in these commands, though: the range
of `git p4` tests. The reason? `git p4` stores the (foreign) remote
branch in the branch called `p4/master`, which is obviously not the
default branch. Manual analysis revealed that only five of these tests
actually require a specific default branch name to pass; They were
modified thusly:

	$ sed -i '/^ *\. \.\/lib-git-p4\.sh$/i\
	GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master\
	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\
	' t/t980[0167]*.sh t/t9811*.sh

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-19 15:44:17 -08:00
Junio C Hamano
92bf1b6067 ci: avoid set-env construct in print-test-failures.sh
Imitating cac42e47 (ci: avoid using the deprecated `set-env`
construct, 2020-11-07), avoid deprecated ::set-env and use the
recommended alternative instead in print-test-failures.sh

Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-17 12:12:30 -08:00
Johannes Schindelin
4463ce75b7 ci: do not skip tagged revisions in GitHub workflows
When `master` is tagged, and then both `master` and the tag are pushed,
Travis CI will happily build both. That is a waste of energy, which is
why we skip the build for `master` in that case.

Our GitHub workflow is also triggered by tags. However, the run would
fail because the `windows-test` jobs are _not_ skipped on tags, but the
`windows-build` job _is skipped (and therefore fails to upload the
build artifacts needed by the test jobs).

In addition, we just added logic to our GitHub workflow that will skip
runs altogether if there is already a successful run for the same commit
or at least for the same tree.

Let's just change the GitHub workflow to no longer specifically skip
tagged revisions.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-08 11:58:41 -07:00
Johannes Schindelin
ef60e9f74b ci: stop linking built-ins to the dashed versions
Since e4597aae65 (run test suite without dashed git-commands in PATH,
2009-12-02), we stopped running our tests with `git-foo` binaries found
at the top-level directory of a freshly built source tree; instead we
have placed only `git` and selected `git-foo` commands that must be on
`$PATH` in `bin-wrappers/` and prepended that `bin-wrappers/` to the
`PATH` used in the test suite. We did that to catch the tests and
scripted Git commands that still try to use the dashed form.

Since CI jobs will not install the built Git to anywhere, and the
hardlinks we make at the top-level of the source tree for `git-add` and
friends are not even used during tests, they are pure waste of resources
these days.

Thanks to the newly invented `SKIP_DASHED_BUILT_INS` knob, we can now
skip creating these links in the source tree. So let's do that.

Note that this change introduces a subtle change of behavior: when Git's
`cmd_main()` calls `setup_path()`, it inserts the value of
`GIT_EXEC_PATH` (defaulting to `<prefix>/libexec/git-core`) at the
beginning of the environment variable `PATH`. This is necessary to find
e.g. scripted commands that are installed in that location. For the
purposes of Git's test suite, the `bin-wrappers/` scripts override
`GIT_EXEC_PATH` to point to the top-level directory of the source code.

In other words, if a scripted command had used a dashed invocation of a
built-in Git command, it would not have been caught previously, which is
fixed by this change.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21 15:47:54 -07:00
Junio C Hamano
e0ad9574dd Merge branch 'bc/sha-256-part-3'
The final leg of SHA-256 transition.

* bc/sha-256-part-3: (39 commits)
  t: remove test_oid_init in tests
  docs: add documentation for extensions.objectFormat
  ci: run tests with SHA-256
  t: make SHA1 prerequisite depend on default hash
  t: allow testing different hash algorithms via environment
  t: add test_oid option to select hash algorithm
  repository: enable SHA-256 support by default
  setup: add support for reading extensions.objectformat
  bundle: add new version for use with SHA-256
  builtin/verify-pack: implement an --object-format option
  http-fetch: set up git directory before parsing pack hashes
  t0410: mark test with SHA1 prerequisite
  t5308: make test work with SHA-256
  t9700: make hash size independent
  t9500: ensure that algorithm info is preserved in config
  t9350: make hash size independent
  t9301: make hash size independent
  t9300: use $ZERO_OID instead of hard-coded object ID
  t9300: abstract away SHA-1-specific constants
  t8011: make hash size independent
  ...
2020-08-11 18:04:11 -07:00
brian m. carlson
8a06d56ccb ci: run tests with SHA-256
Now that we have Git supporting SHA-256, we'd like to make sure that we
don't regress that state.  Unfortunately, it's easy to do so, so to
help, let's add code to run one of our CI jobs with SHA-256 as the
default hash.  This will help us detect any problems that may occur.

We pick the linux-clang job because it's relatively fast and the
linux-gcc job already runs the testsuite twice.  We want our tests to
run as fast as possible, so we wouldn't want to add a third run to the
linux-gcc job.  To make sure we properly exercise the code, let's run
the tests in the default mode (SHA-1) first and then run a second time
with SHA-256.  We explicitly specify SHA-1 for the first run so that if
we change the default in the future, we make sure to test both cases.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Reviewed-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-30 09:16:49 -07:00
SZEDER Gábor
60e47f6773 ci: use absolute PYTHON_PATH in the Linux jobs
In our test suite, when 'git p4' invokes a Git command as a
subprocesses, then it should run the 'git' binary we are testing.
Unfortunately, this is not the case in the 'linux-clang' and
'linux-gcc' jobs on Travis CI, where 'git p4' runs the system
'/usr/bin/git' instead.

Travis CI's default Linux image includes 'pyenv', and all Python
invocations that involve PATH lookup go through 'pyenv', e.g. our
'PYTHON_PATH=$(which python3)' sets '/opt/pyenv/shims/python3' as
PYTHON_PATH, which in turn will invoke '/usr/bin/python3'.  Alas, the
'pyenv' version included in this image is buggy, and prepends the
directory containing the Python binary to PATH even if that is a
system directory already in PATH near the end.  Consequently, 'git p4'
in those jobs ends up with its PATH starting with '/usr/bin', and then
runs '/usr/bin/git'.

So use the absolute paths '/usr/bin/python{2,3}' explicitly when
setting PYTHON_PATH in those Linux jobs to avoid the PATH lookup and
thus the bogus 'pyenv' from interfering with our 'git p4' tests.
Don't bother with special-casing Travis CI: while this issue doesn't
affect the corresponding Linux jobs on GitHub Actions, both CI systems
use Ubuntu LTS-based images, so we can safely rely on these Python
paths.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-23 15:32:06 -07:00
Junio C Hamano
24109910fe Merge branch 'jk/ci-only-on-selected-branches'
Dev support.

* jk/ci-only-on-selected-branches:
  ci/config: correct instruction for CI preferences
2020-05-29 15:12:19 -07:00
Đoàn Trần Công Danh
71800d31b5 ci/config: correct instruction for CI preferences
From e76eec3554 (ci: allow per-branch config for GitHub Actions,
2020-05-07), we started to allow contributors decide which branch
they want to build with GitHub Actions
by checking for a file named "ci/config/allow-ref".

In order to assist those contributors,
we provided a sample in "ci/config/allow-refs.sample",
and instructed them to drop the ".sample",
then commit that file to their repository.

We've misspelt the filename in that change.
Let's fix the spelling.

While we're at it, also instruct our contributors introduce that new
file to Git before commit, in case of they've never told Git before.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-18 10:18:34 -07:00
Junio C Hamano
4024295568 Revert "ci: add a problem matcher for GitHub Actions"
This reverts commit 676eb0c1ce0d380478eb16bdc5a3f2a7bc01c1d2;
as we will be reverting the change to show these extra output
tokens under bash, the pattern would not match anything.

Helped-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-15 10:25:58 -07:00
Jeff King
e76eec3554 ci: allow per-branch config for GitHub Actions
Depending on the workflows of individual developers, it can either be
convenient or annoying that our GitHub Actions CI jobs are run on every
branch. As an example of annoying: if you carry many half-finished
work-in-progress branches and rebase them frequently against master,
you'd get tons of failure reports that aren't interesting (not to
mention the wasted CPU).

This commit adds a new job which checks a special branch within the
repository for CI config, and then runs a shell script it finds there to
decide whether to skip the rest of the tests. The default will continue
to run tests for all refs if that branch or script is missing.

There have been a few alternatives discussed:

One option is to carry information in the commit itself about whether it
should be tested, either in the tree itself (changing the workflow YAML
file) or in the commit message (a "[skip ci]" flag or similar). But
these are frustrating and error-prone to use:

  - you have to manually apply them to each branch that you want to mark

  - it's easy for them to leak into other workflows, like emailing patches

We could likewise try to get some information from the branch name. But
that leads to debates about whether the default should be "off" or "on",
and overriding still ends up somewhat awkward. If we default to "on",
you have to remember to name your branches appropriately to skip CI. And
if "off", you end up having to contort your branch names or duplicate
your pushes with an extra refspec.

By comparison, this commit's solution lets you specify your config once
and forget about it, and all of the data is off in its own ref, where it
can be changed by individual forks without touching the main tree.

There were a few design decisions that came out of on-list discussion.
I'll summarize here:

 - we could use GitHub's API to retrieve the config ref, rather than a
   real checkout (and then just operate on it via some javascript). We
   still have to spin up a VM and contact GitHub over the network from
   it either way, so it ends up not being much faster. I opted to go
   with shell to keep things similar to our other tools (and really
   could implement allow-refs in any language you want). This also makes
   it easy to test your script locally, and to modify it within the
   context of a normal git.git tree.

 - we could keep the well-known refname out of refs/heads/ to avoid
   cluttering the branch namespace. But that makes it awkward to
   manipulate. By contrast, you can just "git checkout ci-config" to
   make changes.

 - we could assume the ci-config ref has nothing in it except config
   (i.e., a branch unrelated to the rest of git.git). But dealing with
   orphan branches is awkward. Instead, we'll do our best to efficiently
   check out only the ci/config directory using a shallow partial clone,
   which allows your ci-config branch to be just a normal branch, with
   your config changes on top.

 - we could provide a simpler interface, like a static list of ref
   patterns. But we can't get out of spinning up a whole VM anyway, so
   we might as well use that feature to make the config as flexible as
   possible. If we add more config, we should be able to reuse our
   partial-clone to set more outputs.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-07 12:40:36 -07:00
Junio C Hamano
9b6606f43d Merge branch 'gs/commit-graph-path-filter'
Introduce an extension to the commit-graph to make it efficient to
check for the paths that were modified at each commit using Bloom
filters.

* gs/commit-graph-path-filter:
  bloom: ignore renames when computing changed paths
  commit-graph: add GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS test flag
  t4216: add end to end tests for git log with Bloom filters
  revision.c: add trace2 stats around Bloom filter usage
  revision.c: use Bloom filters to speed up path based revision walks
  commit-graph: add --changed-paths option to write subcommand
  commit-graph: reuse existing Bloom filters during write
  commit-graph: write Bloom filters to commit graph file
  commit-graph: examine commits by generation number
  commit-graph: examine changed-path objects in pack order
  commit-graph: compute Bloom filters for changed paths
  diff: halt tree-diff early after max_changes
  bloom.c: core Bloom filter implementation for changed paths.
  bloom.c: introduce core Bloom filter constructs
  bloom.c: add the murmur3 hash implementation
  commit-graph: define and use MAX_NUM_CHUNKS
2020-05-01 13:39:53 -07:00
Johannes Schindelin
f72f328bc5 ci: let GitHub Actions upload failed tests' directories
Arguably, CI builds' most important task is to not only identify
regressions, but to make it as easy as possible to investigate what went
wrong.

In that light, we will want to provide users with a way to inspect the
tests' output as well as the corresponding directories.

This commit adds build steps that are only executed when tests failed,
uploading the relevant information as build artifacts. These artifacts
can then be downloaded by interested parties to diagnose the failures
more efficiently.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 10:30:40 -07:00
Johannes Schindelin
676eb0c1ce ci: add a problem matcher for GitHub Actions
With this patch, test failures will be annotated with a helpful,
clickable message in GitHub Actions. For details, see
https://github.com/actions/toolkit/blob/master/docs/problem-matchers.md

Note: we need to set `TEST_SHELL_PATH` to Bash so that the problem
matcher is fed a file and line number for each test failure.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 10:30:40 -07:00
Johannes Schindelin
4fef6321a5 ci: run gem with sudo to install asciidoctor
In a later patch, we will run Documentation job in GitHub Actions.
The job will run without elevated permission.

Run `gem` with `sudo` to elevate permission in order to be able to
install to system location.
This will also keep this installation in-line with other installation in
our Linux system for CI.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
[Danh: reword commit message]
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 10:30:40 -07:00
Đoàn Trần Công Danh
61432dd630 ci: explicit install all required packages
In a later patch, we will support GitHub Action.

Explicitly install all of our build dependencies on Linux.
Since GitHub Action's Linux VM hasn't installed our build dependencies.
And there're no harm to reinstall them (in Travis)

Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10 10:30:40 -07:00
Johannes Schindelin
87b68db3ac ci: fix the jobname of the GETTEXT_POISON job
In 6cdccfce1e (i18n: make GETTEXT_POISON a runtime option,
2018-11-08), the `jobname` was adjusted to have the `GIT_TEST_` prefix,
but that prefix makes no sense in this context.

Co-authored-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-07 22:17:10 -07:00
Đoàn Trần Công Danh
855c158e81 ci/lib: set TERM environment variable if not exist
GitHub Action doesn't set TERM environment variable, which is required
by "tput".

Fallback to dumb if it's not set.

Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-07 22:17:10 -07:00
Johannes Schindelin
a3f2eec862 ci/lib: allow running in GitHub Actions
For each CI system we support, we need a specific arm in that if/else
construct in ci/lib.sh. Let's add one for GitHub Actions.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-07 22:17:10 -07:00
Johannes Schindelin
5127e8cf7a ci/lib: if CI type is unknown, show the environment variables
This should help with adding new CI-specific if-else arms.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-07 22:17:10 -07:00
Junio C Hamano
5471124340 Merge branch 'dd/ci-musl-libc' into HEAD
* dd/ci-musl-libc:
  travis: build and test on Linux with musl libc and busybox
  ci/linux32: libify install-dependencies step
  ci: refactor docker runner script
  ci/linux32: parameterise command to switch arch
  ci/lib-docker: preserve required environment variables
  ci: make MAKEFLAGS available inside the Docker container in the Linux32 job
2020-04-07 22:16:30 -07:00
Đoàn Trần Công Danh
e0f8690dee travis: build and test on Linux with musl libc and busybox
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-06 13:44:42 -07:00
Đoàn Trần Công Danh
c3bc449eb1 ci/linux32: libify install-dependencies step
In a later patch, we will add new Travis Job for linux-musl.
Most of other code in this file could be reuse for that job.

Move the code to install dependencies to a common script.
Should we add new CI system that can run directly in container,
we can reuse this script for installation step.

Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-06 13:44:42 -07:00
Đoàn Trần Công Danh
5a33f541dd ci: refactor docker runner script
We will support alpine check in docker later in this series.

While we're at it, tell people to run as root in podman,
if podman is used as drop-in replacement for docker,
because podman will map host-user to container's root,
therefore, mapping their permission.

Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-06 13:44:42 -07:00
Đoàn Trần Công Danh
2bd1e2d273 ci/linux32: parameterise command to switch arch
In a later patch, the remaining of this command will be re-used for the
CI job for linux with musl libc.

Allow customisation of the emulator, now.

Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-06 13:44:42 -07:00
Đoàn Trần Công Danh
ffce2ebdd9 ci/lib-docker: preserve required environment variables
We're using "su -m" to preserve environment variables in the shell run
by "su". But, that options will be ignored while "-l" (aka "--login") is
specified in util-linux and busybox's su.

In a later patch this script will be reused for checking Git for Linux
with musl libc on Alpine Linux, Alpine Linux uses "su" from busybox.

Since we don't have interest in all environment variables,
pass only those necessary variables to the inner script.

Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-06 13:44:42 -07:00
Garima Singh
d5b873c832 commit-graph: add GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS test flag
Add GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS test flag to the test setup suite
in order to toggle writing Bloom filters when running any of the git tests.
If set to true, we will compute and write Bloom filters every time a test
calls `git commit-graph write`, as if the `--changed-paths` option was
passed in.

The test suite passes when GIT_TEST_COMMIT_GRAPH and
GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS are enabled.

Helped-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Garima Singh <garima.singh@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-06 11:08:37 -07:00
SZEDER Gábor
d2fae19e0f ci: make MAKEFLAGS available inside the Docker container in the Linux32 job
Once upon a time we ran 'make --jobs=2 ...' to build Git, its
documentation, or to apply Coccinelle semantic patches.  Then commit
eaa62291ff (ci: inherit --jobs via MAKEFLAGS in run-build-and-tests,
2019-01-27) came along, and started using the MAKEFLAGS environment
variable to centralize setting the number of parallel jobs in
'ci/libs.sh'.  Alas, it forgot to update 'ci/run-linux32-docker.sh' to
make MAKEFLAGS available inside the Docker container running the 32
bit Linux job, and, consequently, since then that job builds Git
sequentially, and it ignores any Makefile knobs that we might set in
MAKEFLAGS (though we don't set any for the 32 bit Linux job at the
moment).

So update the 'docker run' invocation in 'ci/run-linux32-docker.sh' to
make MAKEFLAGS available inside the Docker container as well.  Set
CC=gcc for the 32 bit Linux job, because that's the compiler installed
in the 32 bit Linux Docker image that we use (Travis CI nowadays sets
CC=clang by default, but clang is not installed in this image).

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-02 11:01:26 -07:00
Junio C Hamano
9a0fa1709c Merge branch 'yz/p4-py3'
Update "git p4" to work with Python 3.

* yz/p4-py3:
  ci: use python3 in linux-gcc and osx-gcc and python2 elsewhere
  git-p4: use python3's input() everywhere
  git-p4: simplify regex pattern generation for parsing diff-tree
  git-p4: use dict.items() iteration for python3 compatibility
  git-p4: use functools.reduce instead of reduce
  git-p4: fix freezing while waiting for fast-import progress
  git-p4: use marshal format version 2 when sending to p4
  git-p4: open .gitp4-usercache.txt in text mode
  git-p4: convert path to unicode before processing them
  git-p4: encode/decode communication with git for python3
  git-p4: encode/decode communication with p4 for python3
  git-p4: remove string type aliasing
  git-p4: change the expansion test from basestring to list
  git-p4: make python2.7 the oldest supported version
2020-03-25 13:57:43 -07:00
SZEDER Gábor
6bb40ed20a ci: use python3 in linux-gcc and osx-gcc and python2 elsewhere
Python2 reached end of life, and we have been preparing our Python
scripts to work with Python3.  'git p4', the main in-tree user of
Python, has just received a number of compatibility updates.  Our
other notable Python script 'contrib/svn-fe/svnrdump_sim.py' is only
used in 't9020-remote-svn.sh', and is apparently already compatible
with both Python2 and 3.

Our CI jobs currently only use Python2.  We want to make sure that
these Python scripts do indeed work with Python3, and we also want to
make sure that these scripts keep working with Python2 as well, for
the sake of some older LTS/Enterprise setups.

Therefore, pick two jobs and use Python3 there, while leaving other
jobs to still stick to Python2 for now.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-10 11:27:13 -07:00
Junio C Hamano
0108fc1b46 Merge branch 'js/ci-windows-update'
Updates to the CI settings.

* js/ci-windows-update:
  Azure Pipeline: switch to the latest agent pools
  ci: prevent `perforce` from being quarantined
  t/lib-httpd: avoid using macOS' sed
2020-03-05 10:43:04 -08:00
Johannes Schindelin
5ed9fc3fc8 ci: prevent perforce from being quarantined
The most recent Azure Pipelines macOS agents enable what Apple calls
"System Integrity Protection". This makes `p4d -V` hang: there is some
sort of GUI dialog waiting for the user to acknowledge that the copied
binaries are legit and may be executed, but on build agents, there is no
user who could acknowledge that.

Let's ask Homebrew specifically to _not_ quarantine the Perforce
binaries.

Helped-by: Aleksandr Chebotov
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-27 09:58:42 -08:00
Junio C Hamano
0de2d1409b Merge branch 'js/ci-squelch-doc-warning'
Squelch unhelpful warning message during documentation build.

* js/ci-squelch-doc-warning:
  ci: ignore rubygems warning in the "Documentation" job
2020-02-12 12:41:39 -08:00
Johannes Schindelin
30183894ea ci: ignore rubygems warning in the "Documentation" job
A recent update in the Linux VM images used by Azure Pipelines surfaced
a new problem in the "Documentation" job. Apparently, this warning
appears 396 times on `stderr` when running `make doc`:

/usr/lib/ruby/vendor_ruby/rubygems/defaults/operating_system.rb:10: warning: constant Gem::ConfigMap is deprecated

This problem was already reported to the `rubygems` project via
https://github.com/rubygems/rubygems/issues/3068.

As there is nothing Git can do about this warning, and as the
"Documentation" job reports this warning as a failure, let's just
silence it and move on.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Acked-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-10 09:52:27 -08:00
Junio C Hamano
76c57fedfa Merge branch 'js/add-p-leftover-bits'
The final leg of rewriting "add -i/-p" in C.

* js/add-p-leftover-bits:
  ci: include the built-in `git add -i` in the `linux-gcc` job
  built-in add -p: handle Escape sequences more efficiently
  built-in add -p: handle Escape sequences in interactive.singlekey mode
  built-in add -p: respect the `interactive.singlekey` config setting
  terminal: add a new function to read a single keystroke
  terminal: accommodate Git for Windows' default terminal
  terminal: make the code of disable_echo() reusable
  built-in add -p: handle diff.algorithm
  built-in add -p: support interactive.diffFilter
  t3701: adjust difffilter test
2020-02-05 14:34:58 -08:00
Johannes Schindelin
b2627cc3d4 ci: include the built-in git add -i in the linux-gcc job
This job runs the test suite twice, once in regular mode, and once with
a whole slew of `GIT_TEST_*` variables set.

Now that the built-in version of `git add --interactive` is
feature-complete, let's also throw `GIT_TEST_ADD_I_USE_BUILTIN` into
that fray.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15 12:06:17 -08:00
Junio C Hamano
ef3ce7c4b9 Merge branch 'sg/osx-force-gcc-9'
TravisCI update.

* sg/osx-force-gcc-9:
  ci: build Git with GCC 9 in the 'osx-gcc' build job
2019-12-06 15:09:21 -08:00
Junio C Hamano
e0f9ec9027 Merge branch 'sg/test-bool-env'
Recently we have declared that GIT_TEST_* variables take the
usual boolean values (it used to be that some used "non-empty
means true" and taking GIT_TEST_VAR=YesPlease as true); make
sure we notice and fail when non-bool strings are given to
these variables.

* sg/test-bool-env:
  t5608-clone-2gb.sh: turn GIT_TEST_CLONE_2GB into a bool
  tests: add 'test_bool_env' to catch non-bool GIT_TEST_* values
2019-12-05 12:52:48 -08:00
Junio C Hamano
7fd7a8ab29 Merge branch 'jc/azure-ci-osx-fix-fix'
CI jobs for macOS has been made less chatty when updating perforce
package used during testing.

* jc/azure-ci-osx-fix-fix:
  ci(osx): update homebrew-cask repository with less noise
2019-12-05 12:52:44 -08:00
SZEDER Gábor
176441bfb5 ci: build Git with GCC 9 in the 'osx-gcc' build job
Our 'osx-gcc' build job on Travis CI relied on GCC 8 being installed
(but not linked) in the image we use [1].  Alas, since the last update
of this image a few days ago this is not the case anymore, and now it
contains GCC 9 (installed and linked) instead of GCC 8.  The results
are failed 'osx-gcc' jobs, because they can't find the 'gcc-8' command
[2].

Let's move on to use GCC 9, with hopefully better error reporting and
improved -Wfoo flags and what not.  On Travis CI this has the benefit
that we can spare a few seconds while installing dependencies, because
it already comes pre-installed, at least for now.  The Azure Pipelines
OSX image doesn't include GCC, so we have to install it ourselves
anyway, and then we might as well install the newer version.

In a vain attempt I tried to future-proof this a bit:

  - Install 'gcc@9' specifically, so we'll still get what we want even
    after GCC 10 comes out, and the "plain" 'gcc' package starts to
    refer to 'gcc@10'.

  - Run both 'brew install gcc@9' and 'brew link gcc@9'.  If 'gcc@9'
    is already installed and linked, then both commands are noop and
    exit with success.  But as we saw in the past, sometimes the image
    contains the expected GCC package installed but not linked, so
    maybe it will happen again in the future as well.  In that case
    'brew install' is still a noop, and instructs the user to run
    'brew link' instead, so that's what we'll do.  And if 'gcc@9' is
    not installed, then 'brew install' will install it, and the
    subsequent 'brew link' becomes a noop.

An additional benefit of this patch is that from now on we won't
unnecessarily install GCC and its dependencies in the 'osx-clang' jobs
on Azure Pipelines.

[1] 7d4733c501 (ci: fix GCC install in the Travis CI GCC OSX job,
    2019-10-24)
[2] https://travis-ci.org/git/git/jobs/615442297#L333

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-29 13:18:48 -08:00
SZEDER Gábor
a85efb5985 t5608-clone-2gb.sh: turn GIT_TEST_CLONE_2GB into a bool
The GIT_TEST_CLONE_2GB environment variable is only ever checked with
'test -z' in 't5608-clone-2gb.sh', so any non-empty value is
interpreted as "yes, run these expensive tests", even
'GIT_TEST_CLONE_2GB=NoThanks'.

Similar GIT_TEST_* environment variables have already been turned into
bools in 3b072c577b (tests: replace test_tristate with "git
env--helper", 2019-06-21), so let's turn GIT_TEST_CLONE_2GB into a
bool as well, to follow suit.

Our CI builds set GIT_TEST_CLONE_2GB=YesPlease, so adjust them
accordingly, thus removing the last 'YesPlease' from our CI scripts.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-23 11:16:10 +09:00
Junio C Hamano
0dbc4a0edf ci(osx): update homebrew-cask repository with less noise
The OSX CI build procedure updates the homebrew-cask repository
before attempting to install perforce again, after seeing an
installation failure.  This involves a "git pull" that by default
computes and outputs diffstat, which would only grow as the time
goes by and the repository cast in stone in the CI build image
becomes more and more stale relative to the upstream repository in
the outside world.

Suppress the diffstat to both save cycles to generate it, and strain
on the eyeballs to skip it.

Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-20 11:55:46 +09:00
Junio C Hamano
04b1f4f768 Merge branch 'sg/ci-osx-gcc8-fix'
CI build fix.

* sg/ci-osx-gcc8-fix:
  ci: fix GCC install in the Travis CI GCC OSX job
2019-10-24 13:34:03 +09:00
SZEDER Gábor
7d4733c501 ci: fix GCC install in the Travis CI GCC OSX job
A few days ago Travis CI updated their existing OSX images, including
the Homebrew database in the xcode10.1 OSX image that we use.  Since
then installing dependencies in the 'osx-gcc' job fails when it tries
to link gcc@8:

  + brew link gcc@8
  Error: No such keg: /usr/local/Cellar/gcc@8

GCC8 is still installed but not linked to '/usr/local' in the updated
image, as it was before this update, but now we have to link it by
running 'brew link gcc'.  So let's do that then, and fall back to
linking gcc@8 if it doesn't, just to be sure.

Our builds on Azure Pipelines are unaffected by this issue.  The OSX
image over there doesn't contain the gcc@8 package, so we have to
'brew install' it, which already takes care of linking it to
'/usr/local'.  After that the 'brew link gcc' command added by this
patch fails, but the ||-chained fallback 'brew link gcc@8' command
succeeds with an "already linked" warning.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-24 11:31:07 +09:00
Johannes Schindelin
0eb3671ed9 ci(osx): use new location of the perforce cask
The Azure Pipelines builds are failing for macOS due to a change in the
location of the perforce cask. The command outputs the following error:

    + brew install caskroom/cask/perforce
    Error: caskroom/cask was moved. Tap homebrew/cask-cask instead.

So let's try to call `brew cask install perforce` first (which is what
that error message suggests, in a most round-about way).

Prior to 672f51cb we used to install the 'perforce' package with 'brew
install perforce' (note: no 'cask' in there). The justification for
672f51cb was that the command 'brew install perforce' simply stopped
working, after Homebrew folks decided that it's better to move the
'perforce' package to a "cask". Their justification for this move was
that 'brew install perforce' "can fail due to a checksum mismatch ...",
and casks can be installed without checksum verification. And indeed,
both 'brew cask install perforce' and 'brew install
caskroom/cask/perforce' printed something along the lines of:

  ==> No checksum defined for Cask perforce, skipping verification

It is unclear why 672f51cb used 'brew install caskroom/cask/perforce'
instead of 'brew cask install perforce'. It appears (by running both
commands on old Travis CI macOS images) that both commands worked all
the same already back then.

In any case, as the error message at the top of this commit message
shows, 'brew install caskroom/cask/perforce' has stopped working
recently, but 'brew cask install perforce' still does, so let's use
that.

CI servers are typically fresh virtual machines, but not always. To
accommodate for that, let's try harder if `brew cask install perforce`
fails, by specifically pulling the latest `master` of the
`homebrew-cask` repository.

This will still fail, of course, when `homebrew-cask` falls behind
Perforce's release schedule. But once it is updated, we can now simply
re-run the failed jobs and they will pick up that update.

As for updating `homebrew-cask`: the beginnings of automating this in
https://dev.azure.com/gitgitgadget/git/_build?definitionId=11&_a=summary
will be finished once the next Perforce upgrade comes around.

Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-23 11:46:41 +09:00
Junio C Hamano
9728ab488a Merge branch 'dl/honor-cflags-in-hdr-check'
Dev support.

* dl/honor-cflags-in-hdr-check:
  ci: run `hdr-check` as part of the `Static Analysis` job
  Makefile: emulate compile in $(HCO) target better
  pack-bitmap.h: remove magic number
  promisor-remote.h: include missing header
  apply.h: include missing header
2019-10-07 11:33:02 +09:00
Junio C Hamano
eb35c18e42 Merge branch 'sg/travis-help-debug'
Dev support update.

* sg/travis-help-debug:
  travis-ci: do not skip successfully tested trees in debug mode
2019-10-07 11:33:01 +09:00
Junio C Hamano
faf5576a8d Merge branch 'bc/doc-use-docbook-5'
Start using DocBook 5 (instead of DocBook 4.5) as Asciidoctor 2.0
no longer works with the older one.

* bc/doc-use-docbook-5:
  Documentation: fix build with Asciidoctor 2
2019-10-06 12:25:16 +09:00