Commit graph

333 commits

Author SHA1 Message Date
Peter Xu 940bf8ff1c ci: Update comment for migration-compat-aarch64
It turns out that we may not be able to enable this test even for the
upcoming v9.0.  Document what we're still missing.

Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Link: https://lore.kernel.org/r/20240207005403.242235-4-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
2024-02-07 10:51:27 +08:00
Peter Xu 7070477902 ci: Remove tag dependency for build-previous-qemu
The new build-previous-qemu job relies on QEMU release tag being present,
while that may not be always true for personal git repositories since by
default tag is not pushed.  The job can fail on those CI kicks, as reported
by Peter Maydell.

Fix it by fetching the tags remotely from the official repository, as
suggested by Dan.

[1] https://lore.kernel.org/r/ZcC9ScKJ7VvqektA@redhat.com

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Suggested-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Link: https://lore.kernel.org/r/20240207005403.242235-3-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
2024-02-07 10:51:27 +08:00
Fabiano Rosas 079687e418 ci: Disable migration compatibility tests for aarch64
Until 9.0 is out, we need to keep the aarch64 job disabled because the
tests always use the n-1 version of migration-test. That happens to be
broken for aarch64 in 8.2. Once 9.0 is out, it will become the n-1
version and it will bring the fixed tests.

We can revert this patch when 9.0 releases.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240118164951.30350-4-farosas@suse.de
[peterx: use _SKIPPED rather than _OPTIONAL]
Signed-off-by: Peter Xu <peterx@redhat.com>
2024-01-29 11:02:12 +08:00
Fabiano Rosas 94766edb35 ci: Add a migration compatibility test job
The migration tests have support for being passed two QEMU binaries to
test migration compatibility.

Add a CI job that builds the lastest release of QEMU and another job
that uses that version plus an already present build of the current
version and run the migration tests with the two, both as source and
destination. I.e.:

 old QEMU (n-1) -> current QEMU (development tree)
 current QEMU (development tree) -> old QEMU (n-1)

The purpose of this CI job is to ensure the code we're about to merge
will not cause a migration compatibility problem when migrating the
next release (which will contain that code) to/from the previous
release.

The version of migration-test used will be the one matching the older
QEMU. That way we can avoid special-casing new tests that wouldn't be
compatible with the older QEMU.

Note: for user forks, the version tags need to be pushed to gitlab
otherwise it won't be able to checkout a different version.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240118164951.30350-3-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
2024-01-29 11:02:12 +08:00
Peter Maydell 977542ded7 testing and misc updates
- add LE microblaze test to avocado
   - use modern snapshot=on to avoid trashing disk image
   - use plain bool for fe_is_open
   - various updates to qtest timeouts
   - enable meson test timeouts
   - tweak the readthedocs environment
   - partially revert un-flaking x86_64
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmWhPccACgkQ+9DbCVqe
 KkS5agf+OoW6HOitt34YeL6cGGtIKaxbta+Fs6jq+ucIbN63TmLTuKrmPiRNxjuo
 Fj2Qvh9R7Tl7Q/a7ZAym0Fze7GtsvvsidkiQS4pmi9vYuJrhS734CxXHT8JS6zJr
 ymQ0nGZODg1cVB4oAR9sXo/OwEQdDTSgKp8wdNr930fxYwokUKBUgcOqElu3SWHv
 duSYDuaflnP5B8ZGbb1ZnOlwS9lZIHTwjZyN5J1YtxF0T8Ez4A+xseEOpQ/00MoE
 Ecjdp3ELCxzOI+1U33Yni7ol//fxQpRKi+xf2fGIxhuSA3i32rmY5NWTvl7VwuS1
 gXryjX2rukSujySP3vkdtTp0dmkbpg==
 =ZuDd
 -----END PGP SIGNATURE-----

Merge tag 'pull-testing-updates-120124-2' of https://gitlab.com/stsquad/qemu into staging

testing and misc updates

  - add LE microblaze test to avocado
  - use modern snapshot=on to avoid trashing disk image
  - use plain bool for fe_is_open
  - various updates to qtest timeouts
  - enable meson test timeouts
  - tweak the readthedocs environment
  - partially revert un-flaking x86_64

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmWhPccACgkQ+9DbCVqe
# KkS5agf+OoW6HOitt34YeL6cGGtIKaxbta+Fs6jq+ucIbN63TmLTuKrmPiRNxjuo
# Fj2Qvh9R7Tl7Q/a7ZAym0Fze7GtsvvsidkiQS4pmi9vYuJrhS734CxXHT8JS6zJr
# ymQ0nGZODg1cVB4oAR9sXo/OwEQdDTSgKp8wdNr930fxYwokUKBUgcOqElu3SWHv
# duSYDuaflnP5B8ZGbb1ZnOlwS9lZIHTwjZyN5J1YtxF0T8Ez4A+xseEOpQ/00MoE
# Ecjdp3ELCxzOI+1U33Yni7ol//fxQpRKi+xf2fGIxhuSA3i32rmY5NWTvl7VwuS1
# gXryjX2rukSujySP3vkdtTp0dmkbpg==
# =ZuDd
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 12 Jan 2024 13:25:27 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* tag 'pull-testing-updates-120124-2' of https://gitlab.com/stsquad/qemu: (22 commits)
  tests/avocado: partially revert unmasking of replay_linux tests
  readthodocs: fully specify a build environment
  mtest2make: stop disabling meson test timeouts
  tests/fp: Bump fp-test-mulAdd test timeout to 3 minutes
  tests/unit: Bump test-crypto-block test timeout to 5 minutes
  tests/unit: Bump test-aio-multithread test timeout to 2 minutes
  tests/qtest: Bump the device-introspect-test timeout to 12 minutes
  qtest: bump bios-table-test timeout to 9 minutes
  qtest: bump aspeed_smc-test timeout to 6 minutes
  qtest: bump qos-test timeout to 2 minutes
  qtest: bump boot-serial-test timeout to 3 minutes
  qtest: bump prom-env-test timeout to 6 minutes
  qtest: bump pxe-test timeout to 10 minutes
  qtest: bump test-hmp timeout to 4 minutes
  qtest: bump npcm7xx_pwm-test timeout to 5 minutes
  qtest: bump qom-test timeout to 15 minutes
  qtest: bump migration-test timeout to 8 minutes
  qtest: bump min meson timeout to 60 seconds
  chardev: use bool for fe_is_open
  gitlab: include microblazeel in testing
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-01-12 14:02:53 +00:00
Alex Bennée 2d41bf0fe1 gitlab: include microblazeel in testing
This reverts aeb5f8f248 (gitlab: build the correct microblaze target)
now we actually have a little-endian test in avocado thanks to this
years advent calendar.

Message-Id: <20240103173349.398526-4-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
2024-01-12 13:23:48 +00:00
Peter Maydell 52a21689cd .gitlab-ci.d/buildtest.yml: Work around htags bug when environment is large
Sometimes the CI "pages" job fails with a message like this from
htags:

$ htags -anT --tree-view=filetree -m qemu_init -t "Welcome to the QEMU sourcecode"
htags: Negative exec line limit = -371

This is due to a bug in hflags where if the environment is too large it
falls over:
https://lists.gnu.org/archive/html/bug-global/2024-01/msg00000.html

This happens to us because GitLab CI puts the commit message of the
commit under test into the CI_COMMIT_MESSAGE and/or CI_COMMIT_TAG_MESSAGE
environment variables, so the job will fail if the commit happens to
have a verbose commit message.

Work around the htags bug by unsetting these variables while running
htags.

Cc: qemu-stable@nongnu.org
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2080
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240111125543.1573473-1-peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-01-11 17:49:21 +01:00
Nicholas Piggin cdd30f369a gitlab: fix s390x tag for avocado-system-centos
The 390x tag should be s390x.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20240107170119.82222-2-npiggin@gmail.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-01-11 11:39:26 +01:00
Alex Bennée 5d25fcb702 gitlab: add optional job to run flaky avocado tests
One problem with flaky tests is they often only fail under CI
conditions which makes it hard to debug. We add an optional allow_fail
job so developers can trigger the only the flaky tests in the CI
environment if they are debugging.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-8-alex.bennee@linaro.org>
2023-12-01 17:47:20 +00:00
Alex Bennée aeb5f8f248 gitlab: build the correct microblaze target
We inadvertently built the LE target for BE tests.

Fixes: 78ebc00b06 (gitlab: shuffle some targets and reduce avocado noise)
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-7-alex.bennee@linaro.org>
2023-12-01 17:47:20 +00:00
Philippe Mathieu-Daudé 541069e653 .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma)
Upgrade libvirt-ci so it covers macOS 14. Add a manual entry
(QEMU_JOB_OPTIONAL: 1) to test on Sonoma release. Refresh the
lci-tool generated files.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231109160504.93677-3-philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-11-24 16:21:55 +01:00
Greg Manning 4789f9d3a1 plugins: fix win plugin tests on cross compile
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1972

Cross compile gcc is more picky about argument order than msys. Changed
the meson command to take the (now renamed) libqemu_plugin_api.a as a
lib, rather than an object. This puts it in the right place on both
native and cross compile gcc commands

Reenable plugins on crossbuilds

Signed-off-by: Greg Manning <gmanning@rapitasystems.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20231109124326.21106-2-gmanning@rapitasystems.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231120150833.2552739-5-alex.bennee@linaro.org>
2023-11-23 14:10:04 +00:00
Alex Bennée 8e721c3277 tests/docker: merge debian-native with debian-amd64
debian-native isn't really needed and suffers from the problem of
tracking a distros dependencies rather than the projects. With a
little surgery we can make the debian-amd64 container architecture
neutral and allow people to use it to build a native QEMU.

Rename it so it follows the same non-arch pattern of the other distro
containers.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Anders Roxell <anders.roxell@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231120150833.2552739-4-alex.bennee@linaro.org>
2023-11-23 14:10:04 +00:00
Philippe Mathieu-Daudé 7528ef7321 .gitlab-ci.d/cirrus: Upgrade macOS to 13 (Ventura)
macOS 14 "Sonoma" was released on September 2023 [1].

According to QEMU's support policy, we stop supporting the
previous major release two years after the the new major
release has been published. Replace the macOS 12 (Monterey)
testing by macOS 13 (Ventura, released on October 2022, [2]).

Refresh the generated files by running:

  $ make lcitool-refresh

[1] https://www.apple.com/newsroom/2023/09/macos-sonoma-is-available-today/
[2] https://www.apple.com/newsroom/2022/10/macos-ventura-is-now-available/

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20231108162022.76189-1-philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231120150833.2552739-3-alex.bennee@linaro.org>
2023-11-23 14:10:02 +00:00
Daniel P. Berrangé aa5730b07e tests/docker: replace fedora-i386 with debian-i686
Fedora is gradually killing off i386 packages in its repos, via a
death-by-1000-cuts process. Thus Debian looks like a better long
term bet for i686 build testing. It has the added advantage that
we can generate it via lcitool too.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20231107164109.1449014-1-berrange@redhat.com>
[AJB: tweak commit msg, set correct prefix]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231120150833.2552739-2-alex.bennee@linaro.org>
2023-11-23 11:55:32 +00:00
Alex Bennée f8347d05df gitlab: add dlltool to Windows CI
We also --disable-plugins for the two mingw based cross builds as
although they have dlltool they seem to be unhappy linking.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2023-11-08 15:15:23 +00:00
Alex Bennée 7ccb4153fe tests/docker: use debian-all-test-cross for sparc64
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-15-alex.bennee@linaro.org>
2023-10-31 14:10:21 +00:00
Alex Bennée 26025d8e36 tests/docker: use debian-all-test-cross for riscv64
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-14-alex.bennee@linaro.org>
2023-10-31 14:10:14 +00:00
Alex Bennée b09bb6d1b8 tests/docker: use debian-all-test-cross for mips
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-13-alex.bennee@linaro.org>
2023-10-31 14:10:14 +00:00
Alex Bennée 92a3165e1a tests/docker: use debian-all-test-cross for mips64
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-12-alex.bennee@linaro.org>
2023-10-31 14:10:14 +00:00
Alex Bennée 9d9a573612 tests/docker: use debian-all-test-cross for m68k
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-11-alex.bennee@linaro.org>
2023-10-31 14:10:14 +00:00
Alex Bennée 95f5bf9521 tests/docker: use debian-all-test-cross for hppa
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-10-alex.bennee@linaro.org>
2023-10-31 14:10:14 +00:00
Alex Bennée eb4cb4ed1f tests/docker: use debian-all-test-cross for power
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-9-alex.bennee@linaro.org>
2023-10-31 14:10:06 +00:00
Alex Bennée 4e76d98ae9 tests/docker: move sh4 to use debian-legacy-test-cross
sh4 is another target which doesn't work with bookworm compilers. To
keep on buster move across to the debian-legacy-test-cross image and
update accordingly.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20231030135715.800164-1-alex.bennee@linaro.org>
2023-10-31 13:28:34 +00:00
Alex Bennée d004e27b4e tests/docker: use debian-legacy-test-cross for alpha
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-7-alex.bennee@linaro.org>
2023-10-31 13:28:30 +00:00
Alex Bennée cb8715bf0c gitlab: add build-loongarch to matrix
We have the compiler and with a few updates a container that can build
QEMU so we should at least run the check-tcg smoke tests.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-6-alex.bennee@linaro.org>
2023-10-31 13:28:30 +00:00
Alex Bennée 6f6c399976 gitlab: clean-up build-soft-softmmu job
Having dropped alpha we also now drop xtensa as we don't have the
compiler in this image. It's not all doom and gloom though as a number
of other targets have gained softmmu TCG tests so we can add them. We
will take care of the other targets with their own containers in
future commits.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-5-alex.bennee@linaro.org>
2023-10-31 13:28:30 +00:00
Alex Bennée aa39c4c098 gitlab: split alpha testing into a legacy container
The current bookworm compiler doesn't build the static binaries due to
bug #1054412 and it might be awhile before it gets fixed. The problem
of keeping older architecture compilers running isn't going to go away
so lets prepare the ground. Create a legacy container and move some
tests around so the others can get upgraded.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-4-alex.bennee@linaro.org>
2023-10-31 13:28:25 +00:00
Thomas Huth f51f90c65e gitlab-ci: Disable the riscv64-debian-cross-container by default
This job is failing since weeks. Let's mark it as manual until
it gets fixed.

Message-Id: <82aa015a-ca94-49ce-beec-679cc175b726@redhat.com>
Acked-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-10-12 14:18:03 +02:00
Alex Bennée 78ebc00b06 gitlab: shuffle some targets and reduce avocado noise
We move a couple of targets out of the avocado runs because there are
no tests to run. Tricore already has some coverage.  The cris target
only really has check-tcg tests but its getting harder to find
anything that packages the compiler.

To reduce the noise of CANCEL messages we also set AVOCADO_TAGS
appropriately so we filter down the number of tests we attempt.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231009164104.369749-5-alex.bennee@linaro.org>
2023-10-11 08:46:33 +01:00
Alex Bennée 3e3df0d84f tests/lcitool: add swtpm to the package list
We need this to test some TPM stuff.

Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231009164104.369749-4-alex.bennee@linaro.org>
2023-10-11 08:46:23 +01:00
Daniel P. Berrangé eca74afd7d gitlab: make Cirrus CI jobs gating
The Cirrus CI jobs have been non-gating for a while to let us build
confidence in their reliability. Aside from periodic dependancy
problems when FreeBSD Ports switches to be based on a new FreeBSD
image version, the jobs have been reliable. It is thus worth making
them gating to prevent build failures being missed during merges.

Signed-off-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230912184130.3056054-5-berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230914155422.426639-8-alex.bennee@linaro.org>
2023-09-20 15:06:33 +01:00
Daniel P. Berrangé c576d8bfbb gitlab: make Cirrus CI timeout explicit
On the GitLab side we're invoking the Cirrus CI job using the
cirrus-run tool which speaks to the Cirrus REST API. Cirrus
sometimes tasks 5-10 minutes to actually schedule the task,
and thus the execution time of 'cirrus-run' inside GitLab will
be slightly longer than the execution time of the Cirrus CI
task.

Setting the timeout in the GitLab CI job should thus be done
in relation to the timeout set for the Cirrus CI job. While
Cirrus CI defaults to 60 minutes, it is better to set this
explicitly, and make the relationship between the jobs
explicit

Signed-off-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230912184130.3056054-4-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230914155422.426639-7-alex.bennee@linaro.org>
2023-09-20 15:06:33 +01:00
Alex Bennée 3aa2cfa0e6 gitlab: fix typo/spelling in comments
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230914155422.426639-3-alex.bennee@linaro.org>
2023-09-20 15:06:33 +01:00
Thomas Huth e05a4beaec .gitlab-ci.d/cirrus.yml: Update FreeBSD to v13.2
The FreeBSD CI job started to fail due to linking problems ... time
to update to the latest version to get this fixed.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230823144533.230477-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230829161528.2707696-6-alex.bennee@linaro.org>
2023-08-30 14:57:50 +01:00
Daniel P. Berrangé 2f7350cd43 gitlab: enable ccache for many build jobs
The `ccache` tool can be very effective at reducing compilation times
when re-running pipelines with only minor changes each time. For example
a fresh 'build-system-fedora' job will typically take 20 minutes on the
gitlab.com shared runners. With ccache this is reduced to as little as
6 minutes.

Normally meson would auto-detect existance of ccache in $PATH and use
it automatically, but the way we wrap meson from configure breaks this,
as we're passing in an config file with explicitly set compiler paths.
Thus we need to add $CCACHE_WRAPPERSPATH to the front of $PATH. For
unknown reasons if doing this in msys though, gcc becomes unable to
invoke 'cc1' when run from meson. For msys we thus set CC='ccache gcc'
before invoking 'configure' instead.

A second problem with msys is that cache misses are incredibly
expensive, so enabling ccache massively slows down the build when
the cache isn't well populated. This is suspected to be a result of
the cost of spawning processes under the msys architecture. To deal
with this we set CCACHE_DEPEND=1 which enables ccache's 'depend_only'
strategy. This avoids extra spawning of the pre-processor during
cache misses, with the downside that is it less likely ccache will
find a cache hit after semantically benign compiler flag changes.
This is the lesser of two evils, as otherwise we can't use ccache
at all under msys and remain inside the job time limit.

If people are finding ccache to hurt their pipelines, it can be
disabled by setting the 'CCACHE_DISABLE=1' env variable against
their gitlab fork CI settings.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230804111054.281802-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230829161528.2707696-2-alex.bennee@linaro.org>
2023-08-30 14:57:33 +01:00
Paolo Bonzini 7ace219303 tests/docker: add python3-tomli dependency to containers
Instead of having CI pick tomli from the vendored wheel at configure
time, place it in the containers.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-08-28 09:55:48 +02:00
Paolo Bonzini c03f57fd5b Revert "tests: Use separate virtual environment for avocado"
This reverts commit e8e4298fea.

ensuregroup allows to specify both the acceptable versions of avocado,
and a locked version to be used when avocado is not installed as a system
pacakge.  This lets us install avocado in pyvenv/ using "mkvenv.py" and
reuse the distro package on Fedora and CentOS Stream (the only distros
where it's available).

ensuregroup's usage of "(>=..., <=...)" constraints when evaluating
the distro package, and "==" constraints when installing it from PyPI,
makes it possible to avoid conflicts between the known-good version and
a package plugins included in the distro.

This is because package plugins have "==" constraints on the version
that is included in the distro, and, using "pip install avocado==88.1"
on a venv that includes system packages will result in an error:

   avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
   avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.

But at the same time, if the venv does not include a system distribution
of avocado then we can install a known-good version and stick to LTS
releases.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1663
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-08-28 09:55:48 +02:00
Paolo Bonzini d9ab1f1f4d ci: install meson in CirrusCI KVM build environment
scripts/archive-source.sh needs meson in order to download the subprojects,
therefore meson needs to be part of the host environment in which VM-based
build jobs run.

Fixes: 2019cabfee ("meson: subprojects: replace submodules with wrap files", 2023-06-06)
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-08-04 13:56:17 +02:00
Daniel P. Berrangé f54ba56dad gitlab: disable FF_SCRIPT_SECTIONS on msys jobs
The FF_SCRIPT_SECTIONS=1 variable should ordinarily cause output from
each line of the job script to be presented in a collapsible section
with execution time listed.

While it works on Linux shared runners, when used with Windows runners
with PowerShell, this option does not create any sections, and actually
causes echo'ing of commands to be disabled, making it even worse to
debug the jobs.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-9-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:48 +02:00
Daniel P. Berrangé 63f5365cd4 gitlab: disable optimization and debug symbols in msys build
Building at -O2, adds 33% to the build time, over -O2. IOW a build that
takes 45 minutes at -O0, takes 60 minutes at -O2. Turning off debug
symbols drops it further, down to 38 minutes.

IOW, a "-O2 -g" build is 58% slower than a "-O0" build on msys in the
gitlab CI windows shared runners.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230801130403.164060-8-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:48 +02:00
Daniel P. Berrangé ef4fe31f33 gitlab: always populate cache for windows msys jobs
The cache is used to hold the msys installer. Even if the build phase
fails, we should still populate the cache as the installer will be
valid for next time.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-6-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:47 +02:00
Daniel P. Berrangé 5ef56e3b18 gitlab: drop $CI_PROJECT_DIR from cache path
The gitlab cache is limited to only handle content within the
$CI_PROJECT_DIR hierarchy, and as such relative paths are always
implicitly relative to $CI_PROJECT_DIR.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-5-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:47 +02:00
Daniel P. Berrangé 46aedd34b4 gitlab: always use updated msys installer
We current reference an msys installer binary from mid-2022, which means
after installation, it immediately has to re-download a bunch of newer
content. This wastes precious CI time.

The msys project publishes an installer binary with a fixed URL that
always references the latest content. We cache the downloads in gitlab
though and so once downloaded we would never re-fetch the installer
leading back to the same problem.

To deal with this we also fetch the pgp signature for the installer
on every run, and compare that to the previously cached signature. If
the signature changes, we re-download the full installer.

This ensures we always have the latest installer for msys, while also
maximising use of the gitlab cache.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-4-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:47 +02:00
Daniel P. Berrangé b64052cdad gitlab: print timestamps during windows msys jobs
It is hard to get visibility into where time is consumed in our Windows
msys jobs. Adding a few log console messages with the timestamp will
aid in our debugging.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:47 +02:00
Daniel P. Berrangé 11961d08fc gitlab: remove duplication between msys jobs
Although they share a common parent, the two msys jobs still have
massive duplication in their script definitions that can easily be
collapsed.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:47 +02:00
Alex Bennée dc389ce1b3 gitlab: reduce testing scope of check-gcov
This keeps timing out on gitlab due to some qtests taking a long time.
As this is just ensuring the gcov machinery is working and not
attempting to be comprehensive lets skip qtest in this run.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-4-alex.bennee@linaro.org>
2023-07-03 12:51:14 +01:00
Daniel P. Berrangé dcb242c849 gitlab: ensure coverage job also publishes meson log
The coverage job wants to publish a coverage report on success, but the
tests might fail and in that case we need the meson logs for debugging.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230503145535.91325-3-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-3-alex.bennee@linaro.org>
2023-07-03 12:51:08 +01:00
Daniel P. Berrangé cef63308e6 gitlab: explicit set artifacts publishing criteria
If not set explicitly, gitlab assumes 'when: on_success" as the
publishing criteria for artifacts. This is reasonable if the
artifact is an output deliverable of the job. This is useless
if the artifact is a log file to be used for debugging job
failures.

This change makes the desired criteria explicit for every job
that publishes artifacts.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230503145535.91325-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-2-alex.bennee@linaro.org>
2023-07-03 12:51:01 +01:00
Richard Henderson 3b746a43eb gitlab: Disable plugins for cross-i386-tci
There are timeouts in the cross-i386-tci job that are related to plugins.
Restrict this job to basic TCI testing.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230629130844.151453-1-richard.henderson@linaro.org>
2023-06-29 18:04:06 +02:00