Commit graph

374 commits

Author SHA1 Message Date
jesperpedersen 0d1ba0a58f Remove changelog.txt from the repository
The changelog.txt file hasn't been kept in sync with release tags,
especially on main, so remove it.

The release notes will be featured in RELEASE_NOTES.md.

Signed-off-by: jesperpedersen <jesper.pedersen@redhat.com>

[NO TESTS NEEDED]
2021-09-07 10:53:26 -04:00
Chris Evich 9dd088e555
Cirrus: Confirm CGv1 / CGv2 VM expectations
Signed-off-by: Chris Evich <cevich@redhat.com>
2021-08-18 12:04:07 -04:00
Chris Evich dc70382886
Fix AVC denials in tests of volume mounts
This becomes a problem on hosts with upgraded policies.  Ref:
https://github.com/containers/podman/issues/10522

Also, made a small change to compose-test setup to reduce runtime.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-08-18 12:04:06 -04:00
Chris Evich 27443660ce
Enable docker-py compat. testing w/ ignored result
Significant bitrot results in almost immediate test failure.  This
commit adds only the very basic, bare-minimum needed to get them
started.

***TESTING RESULTS ARE IGNORED***

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-08-09 17:32:55 -04:00
Paul Holzinger 2c7c679584
Make rootless-cni setup more robust
The rootless cni namespace needs a valid /etc/resolv.conf file. On some
distros is a symlink to somewhere under /run. Because the kernel will
follow the symlink before mounting, it is not possible to mount a file
at exactly /etc/resolv.conf. We have to ensure that the link target will
be available in the rootless cni mount ns.

Fixes #10855

Also fixed a bug in the /var/lib/cni directory lookup logic. It used
`filepath.Base` instead of `filepath.Dir` and thus looping infinitely.

Fixes #10857

[NO TESTS NEEDED]

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-07-06 18:40:03 +02:00
Paul Holzinger 84da70a0f3
update shell completion scripts
The new cobra v1.2.0 release brings a number of bug fixes for shell
completion scripts. Regenerate the scripts with `make completions`
to sync them with the upstream version, currently we have some custom
ones to avoid some upstream bugs. Because the new cobra version has
all fixes we should use the upstream scripts.
Add a check to CI to ensure we always use the up to date scripts.

[NO TESTS NEEDED]

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-07-02 17:49:12 +02:00
Chris Evich 40ef17ac28
Cirrus: Fixes due to master->main rename
Signed-off-by: Chris Evich <cevich@redhat.com>
2021-06-30 11:39:19 -04:00
Chris Evich f8be69e1f3
Don't require tests for github-actions & metadata
Signed-off-by: Chris Evich <cevich@redhat.com>
2021-04-30 16:26:48 -04:00
Chris Evich 0a08216990
Cirrus: Update Ubuntu images to 21.04
Also simplify `lib.sh` after supporting changes incorporated
into automation library 2.x+ (present in all VM and container images).

* No need to force-load `/etc/profile` and handle it's expectation
  to **not** being in `errexit` mode.
* Slightly re-arrange loading of automation library files for
  clarity.
* Update comments.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-04-27 10:28:09 -04:00
Daniel J Walsh 9c8277247d
Fixes from make codespell
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-04-21 13:16:33 -04:00
Ed Santiago 31e31aa9dd compose test: try to get useful data from flakes
docker-compose test continues to flake even after #9961.
Let's try to get some useful data from the failures, by:

  * adding -S (--show-error) to curl. With just -s (--silent),
    curl is completely quiet. With -S, it displays errors.
    (Not in TAP form, but I'm OK with that)

  * oops, adding safety checks to the fix from #9961 (it
    was inadvertently clobbering the curl exit status)

And, as long as I'm in this code: logformatter was not
highlighting these results, because the '1..N' TAP line
needs to be spit out at the end. Have test-compose emit
a 'TAP' header <http://testanything.org/> and make
logformatter recognize it.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-04-13 15:18:34 -06:00
Chris Evich b6b0b6e8bd
Overhaul Makefile binary and release worflows
* Incorporate changes from abandoned #9918: Use dedicated `bin`
  sub-directories for `windows` and `darwin` when building
  `podman-remote`.  The linux flavor remains under `bin` as before.

* Fix MacOS Documentation-generation for release-packaging.
  The `install-podman-remote-%-docs` target requires local execution
  of `podman-remote`, but it was assuming GOOS=linux.  Fix this
  by dynamically discovering the local OS/architecture type while
  still permitting cross-building of MacOS binaries under Linux.

* Unify temporary directory/file behavior to use a common template.
  In case of left-over temporary items left in the repository,
  update the `clean` target accordingly to remove them.

* Fix broken podman-remote-static and MacOS release archive targets
  mismatching the `podman-remote-%` target.  Disambiguate this target
  for all platforms by spelling each out in full, instead of using
  a wild-card recipe.

* Fix Windows-installer target to properly recognize existing
  output files and not constantly rebuild every time.

* Include the podman version number in the Windows-installer target
  in case a user downloads multiple releases.

* Include a subdirectory containing the podman version number for
  both `tar.gz` and `zip` targets.  This prevents users clobbering
  existing directories when un-archiving from releases.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-04-12 10:21:28 -04:00
Chris Evich de8243453d
Exclude .gitignore from test req.
Also sort the explicit files by name, since the list is growing.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-04-12 09:59:30 -04:00
Chris Evich ec88445fc8
Cirrus: Use Fedora 34beta images
Also, revert 4875a8fb

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-04-07 15:20:06 -04:00
Chris Evich 82a29d24c3
Exempt Makefile changes from test requirements
Signed-off-by: Chris Evich <cevich@redhat.com>
2021-04-07 15:19:51 -04:00
Chris Evich 804da7ca07
Cirrus: Make use of shared get_ci_vm container
Depends on:
    https://github.com/containers/automation_images/pull/57
    https://github.com/containers/automation/pull/64
    https://github.com/containers/automation/pull/66
    https://github.com/containers/automation/pull/67
    https://github.com/containers/automation/pull/68

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-04-05 15:38:04 -04:00
Paul Holzinger d1e32dc6c6 Add rootless docker-compose test to the CI
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
2021-04-01 17:27:03 +02:00
Daniel J Walsh 6e250c317c
[NO TESTS NEEDED] Turn on podman-remote build --isolation
Currently podman only works with --isolation chroot.  This PR
fixes this by allowing the isolation mode to default to OCI and to
also allow users to pass the isolation mode into the containers.

The current tests for --isolation should cause this code to be tested.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-03-26 22:04:14 -04:00
Ed Santiago 3baa9da4ed WIP: run buildah bud tests using podman
Set of scripts to run buildah's bud.bats test using
podman build in podman CI.

podman build is not 100% compatible with buildah bud.
In particular:

  * podman defaults to --layers=true; buildah to false
  * podman defaults to --force-rm=true; buildah to false
  * podman error exit status is 125; buildah is 2
  * differences in error messages, command-line arguments

Some of the above can be dealt with programmatically,
by tweaking the buildah helpers.bash (BATS helpers).
Some need to be tweaked by patching bud.bats itself.
This PR includes a patch that will, I fear, need to
be periodically maintained over time.

There will likely be failures when vendoring in a
new buildah, possibly because new tests were added
for new features that don't exist in podman, possibly
(I hope unlikely) if existing tests are changed in
ways that make the patch file fail to apply. I've
tried to write good instructions and to write the run
script in such a way that it will offer helpful hints
on failure. My instructions and code will be imperfect;
I hope they will be good enough to merit continued use
of this test (possibly with improvements to the instructions
as we learn more about real-world failures).

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-03-21 06:59:01 -06:00
Ed Santiago 79eaadd3fb podman upgrade tests
Initial validation of using podman-in-podman to create an
old-podman root, then use new-podman to play with the
containers created therein.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-02-23 06:58:54 -07:00
Ed Santiago d9655b0f01 pr-should-include-tests: recognized "renamed" tests
git tries to recognize renamed files. This isn't always
as helpful as intended. Turn it off, so we'll always see
files as 'A'dded.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-02-22 07:21:04 -07:00
Chris Evich 6c75419a8f
Cirrus: Send cirrus-cron report e-mail to list.
This mailing-list was established to allow people to sub/unsub from
automated notifications.  Add it to the list of destinations picked up
by the Github Actions workflow
`.github/workflows/check_cirrus_cron.yml`.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-02-08 12:11:16 -05:00
Chris Evich 6a6e868290
Cirrus: Collect ginkgo node logs artifacts
In rare cases, it's possible for one of the ginkgo processes to "hang".
When this occurs, the main output will contain this message:

``Ginkgo timed out waiting for all parallel nodes to report``

The only way to debug this was to look through concatenated printing
of the ginkgo node logs.  This is a tedious and daunting task,
requiring special search knowledge, facing a "wall of text".

Simplify the situation by collecting the node logs separately, as
individual files in a cirrus-artifact.  In this way, it's faster to
figure out which test "hung" by examining each log individually.  The
log file which does not have a pass/fail summary at the end,
indicates the last test hung (for whatever reason), and includes it's
output (if any).

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-02-03 10:44:41 -05:00
OpenShift Merge Robot 735b16e347
Merge pull request #9063 from cevich/master_fix_validate
Cirrus: Fix running Validate task on branches
2021-01-30 06:26:58 -05:00
Chris Evich 8d979e093e
Cirrus: Build static podman-remote
Prior to this commit, the "Static Build" task only produced a
`bin/podman`.  Update this to also include a `bin/podman-remote`
binary.

Update the pr-should-include-tests checker to ignore the `nix`
directory, which isn't applicable.

Lastly, restore the static build task to 'required' for CI success.
Leaving the comment inplace in case it needs to be bypassed in the
future on short notice.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-01-28 13:58:11 -05:00
OpenShift Merge Robot 9d59daa7cc
Merge pull request #9082 from saschagrunert/static-build
Fix static build cache by using cachix
2021-01-27 15:18:22 -05:00
Chris Evich bfc1b66be1
Cirrus: Fix running Validate task on branches
Followup to dbb9943

Despite skipping the `Smoke` check, it was observed on a *new* branch,
the `validate` task (specifically `git-validation`) will fail.  This
is because:

* `$CIRRUS_LAST_GREEN_CHANGE` will be empty on a new branch.
* `$CIRRUS_BASE_SHA` is always empty for runs triggered by branch-push
* `$EPOCH_TEST_COMMIT` will be set to `YOU_FOUND_A_BUG`.

Fix this by eliminating the `Smoke` task entirely, simplifying all
the `make validate` operations into the `validate` cirrus task.  Ensure
this task does not run when a new branch or tag is pushed.

Also, eliminate the `$CIRRUS_BUILD_ID` value as it's confusing and not
actually used anywhere.  It was formerly used for building VM images,
but this has moved to another repo entirely.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-01-27 10:35:27 -05:00
Sascha Grunert f8bf509d16
Fix static build cache by using cachix
It looks like we always hit the caching issue in Cirrus CI described
within #8313. A solution around that is to use cachix, which has been
pre-populated from my local machine.

To push all (runtime and build) dependencies, we can leverage a
pre-populated store by:

```
> nix-store -qR --include-outputs $(nix-instantiate nix/default.nix) | cachix push podman
```

The cache can be re-used by everybody to rapidly build static Podman
binaries: https://app.cachix.org/cache/podman

[NO TESTS NEEDED]

Signed-off-by: Sascha Grunert <mail@saschagrunert.de>
2021-01-27 16:15:23 +01:00
Chris Evich 4e4d318b71
Cirrus: Upload swagger YAML in every context
The podman documentation site uses javascript to display
API documentation at:

http://docs.podman.io/en/latest/Reference.html

As input, the javascript sources from a CORS-enabled Google Cloud
Storage object.  This commit ensures the storage object is present and
updated for every Cirrus-CI execution context: Tags, Branches, and PRs.

As of this commit, the documentation site only utilizes the object
uploaded by the Cirrus-CI run on the `master` branch:
`swagger-master.yaml`.  The file produced and uploaded due to a PR is
intended for testing purposes: Confirm it's generation and uploading are
both functional.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-01-21 10:26:57 -05:00
Valentin Rothberg a3621a7cff Cirrus: add bindings checks
Make sure that bindings are in sync with the code.  The check is similar
to what's already being done with `make vendor`, so integrate the two.

[NO TESTS NEEDED]

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-01-20 14:38:02 +01:00
OpenShift Merge Robot 54c465bda6
Merge pull request #8911 from edsantiago/prs_must_include_tests
CI: smoke test: insist on adding tests on PRs
2021-01-20 05:57:09 -05:00
Ed Santiago 9f6bb3563d CI: smoke test: insist on adding tests on PRs
On each PR (with a few exceptions), check the list of git-touched
files, and abort if no tests are added. Include instructions
on how to bypass the check if tests really aren't needed.

Include a hardcoded exception list for PRs that only touch a
well-known subset of "safe" files: docs, .cirrus.yml, vendor,
version, hack, contrib, or *.md. This list is likely to need
tuning over time.

Add a test suite, but not one recognized by the new script
(because it's a "*.t" file), so: [NO TESTS NEEDED]

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-01-19 10:45:58 -07:00
Ed Santiago d2ee3d8151 runner.sh : deal with bash 'set -e'
Release trigger script failed[1] because the entire script
runs under 'set -e'; so a 'grep -- -dev' that finds no
results will cause a nonzero exit status and hence the
entire script to fail. Work around that.

 [1] https://cirrus-ci.com/task/4541290882793472

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-01-18 15:10:58 -07:00
Chris Evich 7490402fab
Cirrus: Upd. ext. service check host list
Since CI doesn't depend heavily on installing packages at runtime
(there is some minor use) there's no need to exhaustively check
repository mirror hosts.  Remove them from the list.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-01-14 09:40:01 -05:00
Ed Santiago 34304df281 CI: fix broken diagnostic message for -dev check
There's a CI check for the presence of "-dev" in podman-info output
(it should not appear). This test is unlikely to fail, but if it
ever does, the diagnostic output is unhelpful. This makes it helpful.

Tested via:

    $ ln -s /bin/echo ~/bin/msg
    $ ln -s /bin/echo ~/bin/die
    $ TEST_FLAVOR=release ./contrib/cirrus/runner.sh
    ...
    Releases must never contain '-dev' in output of 'podman info' ( buildahVersion: 1.19.0-dev
      Version: 3.0.0-dev)

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-01-13 10:14:10 -07:00
OpenShift Merge Robot b5c8cee932
Merge pull request #8900 from cevich/no_tag_testing
Cirrus: Skip most tests on tag-push
2021-01-12 13:38:59 -05:00
Chris Evich 1d8b9e944f
Cirrus: Add cross-compile test for alternative arches
Followup to https://github.com/containers/podman/pull/8907 that simply
ensures cross-compiling podman completes.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-01-11 13:46:37 -05:00
Chris Evich 201d853283
Cirrus: Skip most tests on tag-push
Due to various reasons, CI results (esp. testing tasks) are completely
ignored for builds triggered by a new tag-push.  Additionally, since
many of the automation scripts are in the repo., any related
failures/flakes would require code changes (therefore a new tag).

Resolve this by skipping every testing-type task for builds triggered by
tag-push.  Only retain tasks which build things intended for consumption
associated with a possible official release.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-01-07 11:26:34 -05:00
Josh Soref 4fa1fce930 Spelling
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-12-22 13:34:31 -05:00
Chris Evich e3ba7092fb
Cirrus: Add support for Ubuntu 20.x
Previously automation always dropped the minor version number for
distributions.  This was intended for presentation and conditional
simplicity. Bash does not support non-integer comparison natively.

With the release of version 20.10, supporting testing with it and
the LTS release (20.04) requires scripts to consider minor version
numbers for Ubuntu VMs.  This is necessary because many times in
the past, some behaviors needed to be conditional on the release
version number.

With this commit, the images and embedded scripts/tooling uses an
altered format of `$UBUNTU_NAME', `$PRIOR_UBUNTU_NAME`, and (crucially)
`$OS_RELEASE_VER` and `$OS_REL_VER`.  Any `.` characters appearing
in the official version (from `/etc/os-release`) are dropped, and
the result is concatenated.

For example the current Ubuntu LTS version is `20.04`.  Prior to
this commit, `$OS_RELEASE_VER` would have been `20`.  With this
change, `$OS_RELEASE_VER` will now show `2004`.  Similarly `20.10`
is shown as `2010`.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-12-15 18:49:09 -05:00
Giuseppe Scrivano 8bfba70185
contrib: drop mirror.chpc.utah.edu:443
remove unused mirror from list of required host/ports: the host is
unreachable due to DNS misconfiguration, and it doesn't look like we
need it for anything anyway.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2020-12-15 20:07:36 +01:00
baude 82d7b9f2e2 add compose regression to ci
to prevent any regressions, we should be running regression tests using
compose.

Signed-off-by: baude <bbaude@redhat.com>
2020-12-11 09:26:29 -06:00
Daniel J Walsh c20bd35589
Fix storage.conf to define driver in the VM
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-12-09 15:09:39 -05:00
Daniel J Walsh f62a356515
Remove varlink support from Podman
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-11-26 16:50:42 -05:00
Chris Evich 887f88c490
Github-Actions: Send e-mail on Cirrus cron failure
This repository has a number of automaticly triggered branch-level
testing enabled.  However, other than remembering to go look at a
specific WebUI, there is no way for anybody to notice if/when these jobs
fail.

This commit introduces a github-action workflow which runs periodically,
checking for failed cron-triggered Cirrus-CI jobs.  When it finds any, it
formats a simple report for e-mail delivery.  The list of destination
addresses is configurable at any time by merging changes to a
simple CSV file.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-11-18 15:34:01 -05:00
Ed Santiago 0da801a446 logformatter: highlight timing results
Add a :hover style to rows in the 'integration timing results'
section. Without that, it's really hard for my eye to scan
across and match a time to a test name.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-11-16 14:34:59 -07:00
OpenShift Merge Robot 2e9c94235b
Merge pull request #8107 from cevich/measure_testing_stats
Cirrus: Collect runner.sh stats
2020-11-12 20:08:47 +01:00
Ed Santiago 92e31a2085 Cirrus: update VMs
Reason: include perl-FindBin RPM in f33 VM, needed for
hack/xref-helpmsgs-manpages

Ref: https://github.com/containers/automation_images/pull/39

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-11-11 08:56:08 -07:00
Ed Santiago b0601cb34a [CI:DOCS] Restore man page cross-checker
Somewhere in the CIv2 migration we lost the man page vs --help
cross-checker. Add it back, by adding it into the man-page-check
Makefile target; this is part of 'make validate', which is run
in CI even on CI:DOCS PRs.

As happens when CI doesn't run, things broke. Man pages got out
of sync with --help. This PR:

 1) Fixes hack/xref-helpmsgs-manpages to deal with the new
    "Options" (instead of "Flags") form of podman help. #8034
    did part of that, but one of my review comments was
    accidentally left out.

 2) Fixes hack/xref-helpmsgs-manpages to deal with the new
    option syntax in man pages, post- #8292, in which each
    option is preceded by four hashes so as to make them
    HTML <h4> elements with named anchors.

 3) Fixes man pages that #8292 accidentally missed.

 4) Adds man page entries for two flags that got added
    to podman but not documented (pod create --network-alias,
    play kube --log-driver)

Fixes: #8296

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-11-11 08:31:30 -07:00
Chris Evich f44af20561
Cirrus: Detailed CPU/Memory/Time runner.sh stats
On several occasions, there have been questions about CPU/Memory/IO
trends in testing over time. Start collecting this data for all jobs,
using a common/stable format so that trending analysis can be performed
within/across multiple Cirrus-CI builds. This PR doesn't add any related
tooling, it simply arranges for the collection of the data.

Stats generation is done at the orchestration level to guarantee they
reflect everything happening inside `runner.sh`.  For example, the
container-based tests re-exec `runner.sh` inside a container, but
we're only interested in the top-level stats.

Update all tasks to include collection of the stats file.
Unfortunately, due to the way the Cirrus-CI YAML parser works,
it is *not* possible to alias the artifacts collection more clearly,
for example:

```yaml
    always:
        <<: *runner_stats
        <<: *logs_artifacts
```

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-11-09 14:32:39 -05:00
Jhon Honce 2d50ec6996 Update CI tests to run python docker library against API
* Update reference to docker-py to docker to reflect change in library
  name
* Update tests to create storage sandbox
* Enable all tests that endpoints support
* Refactor containers/{id}/rename to return 404 not 500
* Refactor tests to use quay.io vs. docker.io

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-11-09 10:45:54 -07:00
Chris Evich 46498331a3
Cirrus: Use F33beta VM image
Includes disk-space increase for all Fedora images to accommodate
the static-build job disk space requirements.  This job substantially
leverages task-cache, which was previously failing to restore early on
in the Cirrus-CI task setup, due to disk-space limitations.

Also simplify .cirrus.yml slightly by removing an unncessary setup
and run directory change step.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-11-04 09:40:38 -05:00
Chris Evich 0ebee0ce8e
Cirrus: Workaround F32 BFQ Kernel bug
Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-30 09:15:56 -04:00
OpenShift Merge Robot e439aec4fa
Merge pull request #8046 from cevich/simplify_environment
Cirrus: Simplify setting/passing env. vars.
2020-10-29 18:07:29 +01:00
Chris Evich 3ba77a5618
Cirrus: Simplify setting/passing env. vars.
Test VMs by design are to be single-purpose, single-use, and
readily disposable.  Therefore it's unnecessary to overcomplicate
storage of runtime environment variables.  This commit makes these
points clear, and reorganizes all CI-related env. vars on the system
into a single location, `/etc/ci_environment`.  This file is then
automatically loaded, and variables exported, (by `lib.sh`) from
`runner.sh` prior to executing all forms of testing.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-29 09:02:31 -04:00
Chris Evich 53fe386da0 Cirrus: Use google mirror for docker.io
Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-28 12:25:53 -06:00
Chris Evich 5d48606dae Cirrus: Always record runc/crun versions
Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-28 12:24:06 -06:00
Matthew Heon 1b4933376f Add a system test to verify --runtime is preserved
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-10-20 09:24:32 -04:00
Matthew Heon 3858fc1d01 Use runtime names instead of paths in E2E tests
My patches to fix `--runtime /usr/bin/crun` being allowed to use
a different version of the crun runtime revealed a problem: we
were actually relying on that exact behavior in our E2E tests. We
specified the runtime path as `/usr/bin/runc` for the Ubuntu
tests, but that didn't exist, so Podman was actively looking for
a different, usable runc binary and using that, instead of the
path we explicitly hardcoded. Fixing the bug broke this, and thus
broke the tests.

Instead of hard-coding OCI runtime paths, swap to just using the
runtime name, `runc` or `crun`, and letting Podman figure out
where the runtime lives - it's quite good at that. This should
un-break the tests and make them more durable.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-10-20 09:20:33 -04:00
Chris Evich 23087f77ae
Cirrus: Restore APIv2 Testing
Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-13 11:42:31 -04:00
Chris Evich 5b7c64cc97
Cirrus: Fix obtaining a CI VM
Also removed automatic exection of setup_environment.sh since most
people using this script are podman developers (not automation/CI
folks).  If executing the automation scripts is necessary, manual
attendance to required variables like `$TEST_FLAVOR` is mandatory.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-07 15:29:43 -04:00
Chris Evich ae0f655652
Cirrus: Fix running shellcheck locally
Also, check the contents of hack/get_ci_vm.sh

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-06 16:01:24 -04:00
Ed Santiago e1125558a2 Cirrus CI runner: refactor
While reviewing #6784 I found myself having a lot of trouble
with this script: it was a complicated mix of case statement
and helper functions, requiring a reader to jump back and
forth between the two.

This PR defines a convention such that a given TEST_FLAVOR=foo
must have a corresponding _run_foo() handler function. The
goal is to have all TEST_FLAVOR-related code in one place,
or at least less scattered (integration and system tests
still rely on other helper functions).

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-10-06 11:14:29 -06:00
Ed Santiago 9546638a09 logformatter: run on system tests & bindings
(that got accidentally dropped in the new Cirrus makeover).
Note that 'dotest' does not actually 'do tests', it's
only used for a small subset of tests.

Also, make logformatter work better in the new Cirrus setup.
Remove duplicate test/subtest, remove no-longer-used SPECIALMODE,
and make the Cirrus build/task display a little cleaner.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-10-05 11:31:04 -06:00
Chris Evich 0f04cf27c5
Cirrus: Fix branch-validation failure
When validating code on a branch, determining a starting commit to check
from isn't as straightforward as it would seem.  Default to using the
SHA from last time CI was green.  If for some reason that isn't
available, use an obviously wrong value to cause an intentional
failure.  Entirely skip this check on tag-push, since determining a
starting point is incredibly difficult to do automatically.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-05 10:13:24 -04:00
Chris Evich 2c9084e224
Cirrus: Implement podman automation 2.0
Reimplement CI-automation to remove accumulated technical-debt and
optimize workflow.  The task-dependency graph designed goal was to
shorten it's depth and increase width (i.e. more parallelism).  A
reduction in redundant building (and 3rd party module download) was
also realized by caching `$GOPATH` and `$GOCACHE` early on.  This
cache is then reused in favor of a fresh clone of the repository
(when possible).

Note: The system tests typically execute MUCH faster than the
integration tests.  However, contrary to a fail-fast/fail-early
principal, they are executed last.  This was implemented due to
debug-ability related concerns/preferences of the primary
(golang-centric) project developers.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-02 11:53:04 -04:00
Ed Santiago d380ba0394 logformatter: add Synopsis at top of each page
At the top of each generated page, add a Synopsis table with:

  PR number/name, and link to github
  Author name(s)
  Test name (fedora/ubuntu, rootless, etc)
  Cirrus build ID (usually uninteresting)
  Cirrus task ID (usu. important), with link to Cirrus
  The value of $SPECIALMODE

This is all we can get from the Cirrus environment in
which logformatter runs; we can't get things like
cgroup manager or username that the test runs under.

Note that the table is at the top, which is usually
unseen because we autoscroll to the bottom on
page load. I tentatively think that top is a more
natural place for this info than bottom, but am
willing to listen to arguments against.

Also, one minor tweak: highlight podman commands in
the BATS output. The idea is to make it easier for the eye
to spot those, then copy/paste them to find a reproducer.

And, sigh, disable the new 'podman network create'
system test. It is flaking much too much.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-10-01 20:33:20 -06:00
Daniel J Walsh 526f01cdf5
Fix up errors found by codespell
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-09-11 06:14:25 -04:00
Chris Evich ed1e87ecb5
Cirrus: Obsolete CI:IMG process & related files
All VM-building functionality has been migrated to
https://github.com/containers/automation_images

Some container-build functions are still maintained here
but are on a very-short list to also be migrated to
the repository linked above.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-09-09 13:46:52 -04:00
Ed Santiago 36caf4ee44 WIP: update VM images
(This is an adoption of #7533 because Brent is on PTO).

Pick up new crun and crio-runc.

Also: renames from useful fedora-32 and -31 to less-useful
names; presumably this is needed by something-something in
the new VM setup.

Also: tweak two e2e tests to more properly handle a kernel
(5.8.4) with a greater set of capabilities than what we
or crun can yet handle.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-09-03 14:08:34 -06:00
Brent Baude 053cbd5e0c use crio runc on CICID ubuntu
when running CICD on Ubuntu where no cgroups v2, we need to use a newer runc for things like seccomp and the default ubuntu runc
 is not new enough.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-08-31 16:40:22 -05:00
Sascha Grunert ba9f18e2b8
Use bash binary from env instead of /bin/bash for scripts
It's not possible to run any of the scripts on distributions which do
have `bash` not in `/bin`. This is being fixed by using `/usr/bin/env
bash` instead.

Signed-off-by: Sascha Grunert <sgrunert@suse.com>
2020-08-17 10:42:23 +02:00
Daniel J Walsh 49fa3e7399
Remove TEST_REMOTE_CLIENT from RCLI
We know these are TEST_, hoping this makes the display in
cirrus easier for users to see true|false, since this is the
valuable information is.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-08-10 12:47:43 -04:00
Chris Evich ba4a1e64db
Cirrus: Install golang 1.14 on Ubuntu
This more/less reverts efd142214 + updates to 1.13
on all Ubuntus for all `containers` projects.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-07-31 10:10:24 -04:00
Chris Evich 543606c55c
Cirrus: Add python packages to images
They are needed in support of future testing additions.

Also reduce unnecessary output by not printing the downloaded package
list.  The set can be examined using other tooling if/when necessary.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-07-31 10:10:18 -04:00
Ed Santiago 84f4b87c2e System tests: new system-df and passwd tests
- New test for #6991 - passwd file is writable even when
   run with --userns=keep-id

 - Enable another keep-id test, commented out due to #6593

 - New test for podman system df

Also, independently, removed this line:

   apt-get -y upgrade conmon

...because it's causing CI failures, probably because of the
boothole CVE, probably because the Ubuntu grub update was
rushed out. I believe it is safe to remove this, because
both Ubuntu 19 and 20 report:

   conmon is already the newest version (2.0.18~1).

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-07-30 06:16:51 -06:00
Ed Santiago 382af09784 logformatter: more libpod-podman fallout
Problem: formatted logs no longer have live links to sources
in error-report lines.

Cause: script was searching for '/libpod'.

Solution: make it more flexible.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-07-29 10:48:48 -06:00
Daniel J Walsh a5e37ad280
Switch all references to github.com/containers/libpod -> podman
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-07-28 08:23:45 -04:00
Ed Santiago 9fad55c87f CI - various fixes
Primary purpose: upgrade crun to 0.14 on f31, in hopes of
eliminating the 'cgroups.freeze' flake that is plaguing CI.

While I'm at it:
- remove a no-longer-needed dnf upgrade that was running in CI
  itself (not image building, in each actual CI run). The purpose
  was to upgrade conmon, but that was added a long time ago and
  the required conmon is now in stable. The effect of this
  dnf upgrade today was simply to cause flakes when fedora
  repos were offline.

- remove a no-longer-needed check for varlink.

- networking.sh : add a timeout! 'openssl s_client' will happily
  hang forever if a host is unreachable, which means we waste
  two hours waiting for Cirrus to time out.

- timestamp.awk : include date (not just time) in START/END msgs.
  There are times when I'm looking at a CI log and it is ultra
  important to know if it is from yesterday or today.

- add progress messages in some places where I've previously
  struggled to understand context in logs; and improve some
  unlikely error messages to include script name.

...then, after all that, wrote a new README about how to to
all this. Hope it helps someone.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-07-25 12:32:53 -06:00
Ed Santiago 81115243ef CI: fix rootless permission error
CI runs are failing in special_testing_rootless:

    mkdir /var/tmp/go/pkg: permission denied

Probable cause: #6822, which universally set GOPATH.

Solution: in rootless setup, chown -R GOPATH as well
as GOSRC (the latter was already being chowned).

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-07-23 06:17:22 -06:00
OpenShift Merge Robot d493374eb4
Merge pull request #7050 from edsantiago/logformat_trim_remote
logformatter: handle podman-remote
2020-07-22 20:53:51 +02:00
Ed Santiago 0e85768c96 logformatter: handle podman-remote
Oops! Logs of podman-remote tests are unreadable, they have
multiple (useless) --remote options plus '--url /something/long'
that makes it impossible to read the actual command being run.
This commit strips off '--remote' entirely, and hides '--url'
and its arg in the only-on-mouse-hover '[options]' text.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-07-22 10:32:06 -06:00
Chris Evich fe56be28c1
Cirrus: Add packages that provide htpasswd
Mainly needed for buildah testing: the htpasswd command was removed from
the upstream registry container image.  Making it available on the
host-side enables configuring details needed by the registry during
it's initial setup.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-07-22 11:51:32 -04:00
Chris Evich 18a1514139
Cirrus: Ensure GOPATH is properly set during image-builds
Signed-off-by: Chris Evich <cevich@redhat.com>
2020-07-22 11:51:32 -04:00
OpenShift Merge Robot 80add2902c
Merge pull request #6992 from rhatdan/apparmor
Support default profile for apparmor
2020-07-22 15:38:42 +02:00
Daniel J Walsh 4c4a00f63e
Support default profile for apparmor
Currently you can not apply an ApparmorProfile if you specify
--privileged.  This patch will allow both to be specified
simultaniosly.

By default Apparmor should be disabled if the user
specifies --privileged, but if the user specifies --security apparmor:PROFILE,
with --privileged, we should do both.

Added e2e run_apparmor_test.go

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-07-22 06:27:20 -04:00
Ed Santiago 7bdf93ee63 logformatter: update MAGIC BLOB string
Fallout from libpod->podman repo name move: the HTML logs
created by logformatter are no longer accessible. They
render as:

    https://storage.googleapis.com/SECRET-5385732420009984-fcae48/artifacts/containers/podman/6313596734930944/html/integration_test.log.html

(yes, "SECRET" instead of "cirrus-ci". Possibly because
the GCE_SSH_USERNAME key, "cirrus-ci", was overzealously
encrypted, making Cirrus censor any instances of the
string in output. Let's see if this fixes it. But anyway
this is a secondary unrelated bug).

Reason: it looks like Cirrus "generated a new magic blob"
when we renamed libpod -> podman. Chris was kind enough to
locate the new magic blob and to give me a link to where
we can discover it ourselves. I added that as a code comment.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-07-20 14:02:14 -06:00
Brent Baude 99cc076700 Fix systemd pid 1 test
fedora removed the systemd package from its standard container image causing our systemd pid1 test to fail.  Replacing usage of fedora to ubi-init.

adding ubi images to the cache for local tests.

also, remove installation of test/policy.json to the system wide /etc/containers

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-07-13 17:11:26 -05:00
Ed Santiago aa16a0aab1 system tests: invoke with abs path to podman
Reversion of one part of #6679: my handling of 'realpath'
would not work when $PODMAN is 'podman-remote --url etc'.
Trying to handle that case got unmaintainable; so instead
let's just force 'make {local,remote}system' to invoke
with a full PODMAN path. This breaks down if someone
runs the tests with a manual 'bats' invocation, but I
think I'm the only one who ever does that.

Since podman path will now be very long in the logs,
add code to logformatter to abbreviate it like we do
for the ginkgo logs.

And, one thing that has bugged me for a long time:
in the error logs, show a different prompt ('#' vs '$')
to distinguish root vs rootless. This should make it
much easier to see at-a-glance whether a log file
is root or not. Add tests for it.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-06-23 08:10:57 -06:00
Matthew Heon 0e171b7b33 Do not share container log driver for exec
When the container uses journald logging, we don't want to
automatically use the same driver for its exec sessions. If we do
we will pollute the journal (particularly in the case of
healthchecks) with large amounts of undesired logs. Instead,
force exec sessions logs to file for now; we can add a log-driver
flag later (we'll probably want to add a `podman logs` command
that reads exec session logs at the same time).

As part of this, add support for the new 'none' logs driver in
Conmon. It will be the default log driver for exec sessions, and
can be optionally selected for containers.

Great thanks to Joe Gooch (mrwizard@dok.org) for adding support
to Conmon for a null log driver, and wiring it in here.

Fixes #6555

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-06-17 11:11:46 -04:00
OpenShift Merge Robot c44992827d
Merge pull request #6530 from edsantiago/test_podman_remote
Enable, then partially disable, podman-remote testing
2020-06-10 13:19:44 -04:00
OpenShift Merge Robot 9967f28339
Merge pull request #6193 from cevich/conmon_ci_packages
Cirrus: Include packages for containers/conmon CI
2020-06-09 21:22:17 -04:00
Chris Evich 455a94d1f9
Cirrus: Include packages for containers/conmon CI
This allows the containers/conmon repository to share the same VM
images produced by containers/libpod.  Included are several packages
which are downloaded only since they might otherwise interfere with
testing for some repos.  This allows stable versions to be at the ready
at testing runtime, avoiding any version updates surprising developers.

Also, re-enable running the VM-image check test which was not working
due to a logic problem in Cirrus-CI configuration.  Update the neglected
tests so that they pass on all distros.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-06-09 14:47:23 -04:00
Ed Santiago 15f273b931 WIP: Enable (and disable) remote testing
podman-remote has not been tested. A principal part of the
problem was #5387 - the YAML I wrote did not have the
intended effect, it did not set TEST_REMOTE_CLIENT=true
and because of my multiple iterations I did not catch this
during testing.

Part 1 of this PR is to fix .cirrus.yml to enable remote tests.

Part 2 -- what I had first noticed and tried to fix -- is that
rootless_test.sh was never running remote because, of course,
envariables are not sent via ssh. I reworked integration_test.sh
and rootless_test.sh to use a command-line decision instead.

Part 3, sigh, is to disable one failing integration test
and *all* system tests, because so many of the latter are
failing. Addressing those failures needs to be done in
subsequent PRs. Issues #6538, #6539, #6540 are filed for
some of the problems I isolated. There will be more.

Also, minor, fixed some stale references to varlink.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-06-09 12:43:58 -06:00
OpenShift Merge Robot c831ae110e
Merge pull request #6521 from cevich/update_cors_docs
[CI:DOCS] Improve swagger+CORS metadata docs
2020-06-09 16:43:32 +02:00
Chris Evich 4314336ce3
Improve swagger+CORS metadata docs
Signed-off-by: Chris Evich <cevich@redhat.com>
2020-06-09 09:53:19 -04:00
Valentin Rothberg b6148b6576 force bats version to v1.1.0
We experienced regression when using the latest `v1.2.0-dev` bats in
Ubuntu 20.04 (see github.com/containers/libpod/pull/6418).  Using
bats v1.1.0 worked in the Ubuntu test VM.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-06-08 15:21:34 +02:00
Brent Baude efd1422143 Enable Ubuntu tests in CI
Add updates required for ubuntu and run integration tests

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-06-08 12:21:16 +02:00
Daniel J Walsh 77b8187ea0
Attempt to turn on special_testing_in_podman tests
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-06-04 10:58:37 -04:00
Matthew Heon e0d9404634 Enable detached exec for remote
The biggest obstacle here was cleanup - we needed a way to remove
detached exec sessions after they exited, but there's no way to
tell if an exec session will be attached or detached when it's
created, and that's when we must add the exit command that would
do the removal. The solution was adding a delay to the exit
command (5 minutes), which gives sufficient time for attached
exec sessions to retrieve the exit code of the session after it
exits, but still guarantees that they will be removed, even for
detached sessions. This requires Conmon 2.0.17, which has the new
`--exit-delay` flag.

As part of the exit command rework, we can drop the hack we were
using to clean up exec sessions (remove them as part of inspect).
This is a lot cleaner, and I'm a lot happier about it.

Otherwise, this is just plumbing - we need a bindings call for
detached exec, and that needed to be added to the tunnel mode
backend for entities.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-06-02 15:30:42 -04:00