Prior to this commit, the "Static Build" task only produced a
`bin/podman`. Update this to also include a `bin/podman-remote`
binary.
Update the pr-should-include-tests checker to ignore the `nix`
directory, which isn't applicable.
Lastly, restore the static build task to 'required' for CI success.
Leaving the comment inplace in case it needs to be bypassed in the
future on short notice.
Signed-off-by: Chris Evich <cevich@redhat.com>
It looks like we always hit the caching issue in Cirrus CI described
within #8313. A solution around that is to use cachix, which has been
pre-populated from my local machine.
To push all (runtime and build) dependencies, we can leverage a
pre-populated store by:
```
> nix-store -qR --include-outputs $(nix-instantiate nix/default.nix) | cachix push podman
```
The cache can be re-used by everybody to rapidly build static Podman
binaries: https://app.cachix.org/cache/podman
[NO TESTS NEEDED]
Signed-off-by: Sascha Grunert <mail@saschagrunert.de>
The podman documentation site uses javascript to display
API documentation at:
http://docs.podman.io/en/latest/Reference.html
As input, the javascript sources from a CORS-enabled Google Cloud
Storage object. This commit ensures the storage object is present and
updated for every Cirrus-CI execution context: Tags, Branches, and PRs.
As of this commit, the documentation site only utilizes the object
uploaded by the Cirrus-CI run on the `master` branch:
`swagger-master.yaml`. The file produced and uploaded due to a PR is
intended for testing purposes: Confirm it's generation and uploading are
both functional.
Signed-off-by: Chris Evich <cevich@redhat.com>
There is no need to re-run the same basic validation checks as were
presumably already run on a PR before it merged. There are also
possible problems properly determining `$EPOCH_TEST_COMMIT` when
there have been no successful CI-runs on the branch (i.e. it's new).
This needlessly fouls up the git-validation tool. Fix Both problems
by just skipping the 'smoke' task for branches and tags.
Signed-off-by: Chris Evich <cevich@redhat.com>
Make sure that bindings are in sync with the code. The check is similar
to what's already being done with `make vendor`, so integrate the two.
[NO TESTS NEEDED]
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Specifically, the result of 'make install.tools' is needed. Part of
that target installs tooling into `$GOPATH/bin`. A future commit
requires this tooling for the `Build Each Commit` item of the
alt_build matrix. Re-use the cache of this directory for this
task to ensure the necessary tooling/libraries are available.
Signed-off-by: Chris Evich <cevich@redhat.com>
Due to various reasons, CI results (esp. testing tasks) are completely
ignored for builds triggered by a new tag-push. Additionally, since
many of the automation scripts are in the repo., any related
failures/flakes would require code changes (therefore a new tag).
Resolve this by skipping every testing-type task for builds triggered by
tag-push. Only retain tasks which build things intended for consumption
associated with a possible official release.
Signed-off-by: Chris Evich <cevich@redhat.com>
Previously automation always dropped the minor version number for
distributions. This was intended for presentation and conditional
simplicity. Bash does not support non-integer comparison natively.
With the release of version 20.10, supporting testing with it and
the LTS release (20.04) requires scripts to consider minor version
numbers for Ubuntu VMs. This is necessary because many times in
the past, some behaviors needed to be conditional on the release
version number.
With this commit, the images and embedded scripts/tooling uses an
altered format of `$UBUNTU_NAME', `$PRIOR_UBUNTU_NAME`, and (crucially)
`$OS_RELEASE_VER` and `$OS_REL_VER`. Any `.` characters appearing
in the official version (from `/etc/os-release`) are dropped, and
the result is concatenated.
For example the current Ubuntu LTS version is `20.04`. Prior to
this commit, `$OS_RELEASE_VER` would have been `20`. With this
change, `$OS_RELEASE_VER` will now show `2004`. Similarly `20.10`
is shown as `2010`.
Signed-off-by: Chris Evich <cevich@redhat.com>
It's important to periodically update the nix cache (about 1GB in size).
If not, it can grow stale and has been observed causing task failures.
Associating the nix cache update with a VM/Container image update,
ensures it happens first in PRs where environment-related failures
are less surprising.
Signed-off-by: Chris Evich <cevich@redhat.com>
On several occasions, there have been questions about CPU/Memory/IO
trends in testing over time. Start collecting this data for all jobs,
using a common/stable format so that trending analysis can be performed
within/across multiple Cirrus-CI builds. This PR doesn't add any related
tooling, it simply arranges for the collection of the data.
Stats generation is done at the orchestration level to guarantee they
reflect everything happening inside `runner.sh`. For example, the
container-based tests re-exec `runner.sh` inside a container, but
we're only interested in the top-level stats.
Update all tasks to include collection of the stats file.
Unfortunately, due to the way the Cirrus-CI YAML parser works,
it is *not* possible to alias the artifacts collection more clearly,
for example:
```yaml
always:
<<: *runner_stats
<<: *logs_artifacts
```
Signed-off-by: Chris Evich <cevich@redhat.com>
Includes disk-space increase for all Fedora images to accommodate
the static-build job disk space requirements. This job substantially
leverages task-cache, which was previously failing to restore early on
in the Cirrus-CI task setup, due to disk-space limitations.
Also simplify .cirrus.yml slightly by removing an unncessary setup
and run directory change step.
Signed-off-by: Chris Evich <cevich@redhat.com>
On several occasions, fatal task failures were observed
during the upload of artifacts after a otherwise successful
testing. Prior to this commit, most tasks were storing both
logs and binary artifacts. Avoid possible major inconveniences
of upload failures, by only collecting binary artifacts when
necessary.
Signed-off-by: Chris Evich <cevich@redhat.com>
***Warning***: `skip` has non-obvious side-effects vs `only_if`:
https://cirrus-ci.org/guide/writing-tasks/#conditional-task-execution
The skip instruction can give a false sense of security by always
marking tasks as passed in the UI, even if they didn't run. In
contrast, the `only_if` condition will avoid creating the task
all -together; therefore, a problematic task's absense is more likely to
be noticed if it introduced a problem.
Signed-off-by: Chris Evich <cevich@redhat.com>
Initially filed as #7967 but that has run into huge complicated
snags related to Ubuntu and environment.
It is crucial to get system tests working with podman-local.
It is less important to get them on Ubuntu. Let's please
expedite this PR while we settle the Ubuntu stuff in #7967
Signed-off-by: Ed Santiago <santiago@redhat.com>
Also removed automatic exection of setup_environment.sh since most
people using this script are podman developers (not automation/CI
folks). If executing the automation scripts is necessary, manual
attendance to required variables like `$TEST_FLAVOR` is mandatory.
Signed-off-by: Chris Evich <cevich@redhat.com>
Previous to this commit, the entire suite of CI tasks run in a PR, run
again for every merge (a.k.a. branch push). This wastes time and
resources with substantively overlapping testing. The primary reason
to test on branch-push, is providing coverage for merge-semantics.
In other words, problems introduced due to the sequence of PR merging.
For this purpose, the vast majority of problems can be caught quickly by
a small subset of automated tests. If deeper debugging is necessary,
then opening a test-PR is a small price to ask for the enormous amount
of time/resource savings with more limited branch-push testing.
Signed-off-by: Chris Evich <cevich@redhat.com>
Reimplement CI-automation to remove accumulated technical-debt and
optimize workflow. The task-dependency graph designed goal was to
shorten it's depth and increase width (i.e. more parallelism). A
reduction in redundant building (and 3rd party module download) was
also realized by caching `$GOPATH` and `$GOCACHE` early on. This
cache is then reused in favor of a fresh clone of the repository
(when possible).
Note: The system tests typically execute MUCH faster than the
integration tests. However, contrary to a fail-fast/fail-early
principal, they are executed last. This was implemented due to
debug-ability related concerns/preferences of the primary
(golang-centric) project developers.
Signed-off-by: Chris Evich <cevich@redhat.com>
also add a cirrus task for building binaries with varlink.
From: Chris Evich <cevich@redhat.com>
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
All VM-building functionality has been migrated to
https://github.com/containers/automation_images
Some container-build functions are still maintained here
but are on a very-short list to also be migrated to
the repository linked above.
Signed-off-by: Chris Evich <cevich@redhat.com>
(This is an adoption of #7533 because Brent is on PTO).
Pick up new crun and crio-runc.
Also: renames from useful fedora-32 and -31 to less-useful
names; presumably this is needed by something-something in
the new VM setup.
Also: tweak two e2e tests to more properly handle a kernel
(5.8.4) with a greater set of capabilities than what we
or crun can yet handle.
Signed-off-by: Ed Santiago <santiago@redhat.com>
As of a few minutes ago (relative to this commit), Cirrus
defines the CIRRUS_CHANGE_TITLE envariable as "First line
of CIRRUS_CHANGE_MESSAGE"[1]. Replace all conditionals
accordingly.
[1] f8d2530c60
Reasoning: up until this PR, the presence of CI:IMG
or CI:DOCS *in the body* of the commit message would trigger
those magic CI code flows. This violates POLA, and actually
led to a bad PR (#7317) being merged because CI never ran.
Fixes: #7374
Signed-off-by: Ed Santiago <santiago@redhat.com>
Observed timeout problems hitting some integration-testing
tasks differently than others. Given the current `Makefile`
has a ginkgo timeout of 90-minutes, the task timeout for
integration tests should be longer. Increase the timeout
of the main integration-test running tasks to the (default)
120min global valie in `.cirrus.yml`.
Signed-off-by: Chris Evich <cevich@redhat.com>
We know these are TEST_, hoping this makes the display in
cirrus easier for users to see true|false, since this is the
valuable information is.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
