Commit graph

2891 commits

Author SHA1 Message Date
OpenShift Merge Robot ec5c7c1f6a
Merge pull request #10895 from rhatdan/devices
Support DeviceCgroupRules to actually get added.
2021-07-24 06:09:17 -04:00
OpenShift Merge Robot d956500743
Merge pull request #10996 from cdoern/untilLog
Implemented --until flag for Libpod's Container Logs
2021-07-24 05:56:18 -04:00
OpenShift Merge Robot c44c298ae7
Merge pull request #11018 from vrothberg/fix-10994
import: write stdin to tmp file
2021-07-24 05:53:16 -04:00
Valentin Rothberg dcb5c92c0d import: write stdin to tmp file
If importing an archive via stdin write it to a temporary file such that
the temporary file can be opened multiple times later on.  Otherwise, we
may end up with an empty image.

Also fix a bug in the URL parsing code; we need to check whether there's
actually a scheme.

Add system tests for `podman import` exercising the basics.

Fixes: #10994
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-23 10:21:37 +02:00
Ed Santiago 8f9d33b7f7 Networking test: fix silent breakage
Wow did I screw up. #10982 introduced (at my suggestion) a
new wait_for_port() helper, with the goal of eliminating a
race condition. It didn't work.

First: wait_for_port() tests by connecting to the port, which
is a Bad Idea when you have a one-shot server that exits upon
the first connection closing. We should've caught that, but:

Second: I wrote wait_for_port() for a non-BATS test framework,
and used the conventional file descriptor 3. BATS uses fd3
for internal control. Overriding that made the test silently
just disappear, no "not ok" message, no warnings, nothing
except vanishing into the ether.

Third: this was caught by my log-colorizer script, which
loudly yelled "WARNING: expected 234" (tests) at the
bottom of the log. Unfortunately, since this wasn't
my PR, I didn't actually look at the test logs.

Solution: we can't use wait_for_port() in the network port
test. Use wait_for_output() instead, triggering on the
'listening' message emitted by netcat in the container.

Also: fix wait_for_port() to use fd5 instead of 3. Although
no code currently uses wait_for_port() as of this PR, it's
a useful helper that we may want to keep.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-07-22 18:27:13 -06:00
OpenShift Merge Robot 365775489f
Merge pull request #11015 from jmguzik/until-list-volume
Add until filter to volume ls filters list
2021-07-22 14:11:24 -04:00
cdoern 0f708efd8b Implemented --until flag for libpod's container logs
compat containers/logs was missing actual usage of until query param.
This led me to implement the until param for libpod's container logs as well. Added e2e tests.

Signed-off-by: cdoern <cdoern@redhat.com>
2021-07-22 10:56:56 -04:00
Jakub Guzik 2d8e837a9b Add until filter to volume ls filters list
As a conclusion of a discussion in #10861, until filter is added
by this commit to volume ls filters.

Signed-off-by: Jakub Guzik <jakubmguzik@gmail.com>
2021-07-22 00:01:07 +02:00
Daniel J Walsh 3e79296a81
Support DeviceCgroupRules to actually get added.
Fixes: https://github.com/containers/podman/issues/10302

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-07-21 16:10:09 -04:00
Matthew Heon 4376f14c30 Ensure journald events tests only run where supported
We don't support the journald events backend on RHEL8, for
example. So we can't unconditionally run these tests.

Partial fix for RHBZ1955166

Signed-off-by: Matthew Heon <mheon@redhat.com>
2021-07-21 11:23:01 -04:00
OpenShift Merge Robot 782634334e
Merge pull request #10997 from edsantiago/helpfuler_expects
e2e tests: prevent 'Expect(ExitCode())' pattern
2021-07-21 10:03:12 -04:00
OpenShift Merge Robot a7dcae5abc
Merge pull request #10970 from vikas-goel/prepare-volume
Copy the content from the underlying image into the newly created volume
2021-07-21 05:52:09 -04:00
Ed Santiago 5952270950 e2e tests: prevent 'Expect(ExitCode())' pattern
Followup to #10932: add a validation check to prevent introduction
of new 'Expect(foo.ExitCode()).To(...)' patterns. If such use is
absolutely necessary -- there is one such instance in the code
already -- require that the assertion include a description.

Also: clean up instances that were introduced since the merging
of #10932.

Also: fix one remaining instance in run_exit_test.go: it had
a FIXME comment mentioning a race condition, but unfortunately
there was no issue or bug ID, hence no way to know if the race
is fixed or not. We will assume it is.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-07-20 10:49:50 -06:00
Vikas Goel 064bd9d19f Copy the content from the underlying image into the newly created volume.
Fixes: #10262

Signed-off-by: Vikas Goel <vikas.goel@gmail.com>
2021-07-20 11:39:32 -04:00
Ed Santiago 313c7118ea system tests: cleaner, safer use of systemd
First and foremost: use ephemeral (/run, $XDG) directories
for systemd unit files, so as not to vandalize a working system.

Second, refactor common systemd-related functionality into
a new helper file, loaded by the systemd-related tests.
Shared functionality includes:

  * setting $XDG_RUNTIME_DIR if unset and rootless
  * setting $UNIT_DIR for use by tests
  * new systemctl() and journalctl() functions, which
    include "--user" when rootless (why can't systemd
    figure this out on its own?)

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-07-20 08:57:26 -06:00
Ed Santiago 2b98a226b9 system test: auto-update: multiarch fixes
auto-update test is failing on non-x86_64 arch:

   https://gitlab.com/cki-project/kernel-tests/-/issues/614#note_630010734

Reason: test was relying on alpine_nginx image; this is
a custom libpod image that only has a x86_64 version.

Solution: use busybox instead. Busybox was copied from
docker.io using skopeo copy --all, so it has all arches.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-07-20 07:10:14 -06:00
Ed Santiago caf03fd7a4 system test: auto-update: allow running as rootless
They were being skipped for no clear reason.

Also: use --files and --name option in generate-systemd. Before,
output went to stdout, resulting in completely unreadable logs
on test failure.

Also: use volatile systemd directories (/run, $XDG_RUNTIME_DIR)
instead of permanent ones. Thanks to @Luap99 for the reference.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-07-20 07:10:14 -06:00
OpenShift Merge Robot f4e81d0b88
Merge pull request #10979 from vrothberg/fix-10977
compat: image create: handle platform correctly
2021-07-20 05:26:35 -04:00
OpenShift Merge Robot 36e47768e6
Merge pull request #10982 from cevich/fix_listen_flake
Flake Fix: Wait before connecting to container port
2021-07-20 05:04:35 -04:00
Valentin Rothberg eaaca49992 compat: image create: handle platform correctly
Handle the platform parameter correctly.  The parameter was only parsed
in presence of credentials and the code was a bit complex.  Also add a
regression test.

Fixes: #10977
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-20 09:00:44 +02:00
OpenShift Merge Robot cea2f38e21
Merge pull request #10984 from edsantiago/timeout_typo
(minor) typo fix: timeout variable
2021-07-19 16:47:45 -04:00
Chris Evich 80e807a191
Flake Fix: Wait before connecting container port
It was observed during periodic testing, this test can fail due to the
container process being not fully running and listening on the expected
port:

```
[+1069s] not ok 220 podman networking: port with --userns=keep-id
[+1069s] # (in test file test/system/500-networking.bats, line 144)
[+1069s] #   `echo "$teststring" | nc 127.0.0.1 $myport' failed
[+1069s] # # /var/tmp/go/src/github.com/containers/podman/bin/podman rm
--all --force
[+1069s] # # /var/tmp/go/src/github.com/containers/podman/bin/podman ps
--all --external --format {{.ID}} {{.Names}}
[+1069s] # # /var/tmp/go/src/github.com/containers/podman/bin/podman
images --all --format {{.Repository}}:{{.Tag}} {{.ID}}
[+1069s] # quay.io/libpod/testimage:20210610 9f9ec7f2fdef
[+1069s] # # /var/tmp/go/src/github.com/containers/podman/bin/podman run
-d --userns=keep-id -p 127.0.0.1:54322:54322
quay.io/libpod/testimage:20210610 nc -l -n -v -p 54322
[+1069s] #
252c562c9a3c96892d867d1d72fb52b2efdfe62855ebedbccd2d281c472c2988
[+1069s] # Ncat: No route to host.
```

Fix this by using a new `wait_for_port()` function (thanks @edsantiago)
before attempting to communicate with the service.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-07-19 15:36:13 -04:00
Ed Santiago c622c7f2a9 (minor) typo fix: timeout variable
timeout variable was inconsistently spelled in helper function

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-07-19 13:24:25 -06:00
OpenShift Merge Robot 4e9ccb30c1
Merge pull request #10956 from flouthoc/kube-liveness-probe-systemd
Kube: Add liveness probe for containers backed by native (systemd) healthchecks instead of kubelet.
2021-07-19 14:03:57 -04:00
flouthoc 81e32b1808 Kube: Add liveness probe for containers.
Signed-off-by: flouthoc <flouthoc.git@gmail.com>
2021-07-17 18:35:22 +05:30
OpenShift Merge Robot 1961769edf
Merge pull request #10950 from edsantiago/python_flake_fix
APIv2 (python) tests: fix flake
2021-07-16 11:43:44 -04:00
OpenShift Merge Robot 3ef124b03f
Merge pull request #10962 from vrothberg/fix-10655
systemd: require network*-online*.target
2021-07-16 11:13:46 -04:00
OpenShift Merge Robot 68585b0168
Merge pull request #10848 from vrothberg/update-libimage
vendor containers/common@main
2021-07-16 11:12:44 -04:00
Valentin Rothberg 6f1c7a0b6b systemd: require network*-online*.target
Require the network to be online in all (generated) systemd units to
make sure that containers and Podman run only after the network has been
fully configured.

Fixes: #10655
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-16 14:58:08 +02:00
Valentin Rothberg e1ac0c3033 vendor containers/common@main
The `IgnorePlatform` options has been removed from the
`LookupImageOptions` in libimage to properly support multi-arch images.

Skip one buildah-bud test which requires updated CI images.  This is
currently being done in github.com/containers/podman/pull/10829 but
we need to unblock merging common and buildah into podman.

[NO TESTS NEEDED]

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-16 14:19:56 +02:00
OpenShift Merge Robot ce28dc3c4c
Merge pull request #10820 from jvanz/indfra-container-name-issue-10794
--infra-name command line argument
2021-07-16 07:53:43 -04:00
Ed Santiago be51173ed3 APIv2 (python) tests: fix flake
Python tests were flaking because they behave differently
when $DEBUG is set. It looks like something in CI sets
that envariable.

Solution: do not use $DEBUG as a debug trigger, use a
properly-named custom variable that is unlikely to be
set accidentally.

Also: get rid of AssertTrue(), which gives no visibility
into what happened. Write in proper form that can emit
useful diagnostics on failure.

Fixes: #10948

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-07-16 05:05:59 -06:00
Valentin Rothberg 9924c57d4c podman start: remove containers configured for auto removal
Make sure that containers configured for auto removal
(e.g., via `podman create --rm`) are removed in `podman start`
if starting the container failed.

Fixes: #10935
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-16 10:57:40 +02:00
OpenShift Merge Robot f0cd16cb32
Merge pull request #10947 from edsantiago/multiarch_images
System tests: fix a multiarch problem
2021-07-16 04:26:18 -04:00
José Guilherme Vanz af40dfc2bf
--infra-name command line argument
Adds the new --infra-name command line argument allowing users to define
the name of the infra container

Issue #10794

Signed-off-by: José Guilherme Vanz <jvanz@jvanz.com>
2021-07-15 21:27:51 -03:00
OpenShift Merge Robot 12b67aaf62
Merge pull request #10894 from cdoern/pidPod
podman pod create --pid flag
2021-07-15 17:30:16 -04:00
Ed Santiago e4dcb1004f System tests: fix a multiarch problem
The multi-image load test was failing on non-x86_64 arch,
because the images used by the test (:00000000, :20200902)
did not have manifests for the given arch.

Solution: all we need are two nonlocal images. Use the
predefined NONLOCAL_IMAGE for one, and a new :multiimage
tag (manually created, currently == :20210610) for the other.
Document, so if/when RHEL adds new supported arches, the
test will fail but a maintainer will have a clue what to do.

Also, as long as I'm in here: add 'image prune -f' to teardown
in build.bats, to avoid seeing lots of red "stray image"
warnings in test logs.

Also: skip a broken/flaky python test

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-07-15 11:29:58 -06:00
OpenShift Merge Robot 5d6a366017
Merge pull request #10937 from vrothberg/auto-update
auto-update: add --dry-run
2021-07-15 10:50:13 -04:00
OpenShift Merge Robot bc98c2003d
Merge pull request #10932 from edsantiago/e2e_exit_checks
e2e tests: use Should(Exit()) and ExitWithError()
2021-07-15 10:37:11 -04:00
cdoern f7321681d0 podman pod create --pid flag
added support for --pid flag. User can specify ns:file, pod, private, or host.
container returns an error since you cannot point the ns of the pods infra container
to a container outside of the pod.

Signed-off-by: cdoern <cdoern@redhat.com>
2021-07-15 10:34:09 -04:00
OpenShift Merge Robot 47f351769b
Merge pull request #10913 from rhatdan/build
podman-remote build use .containerignore over .dockerignore
2021-07-15 08:15:11 -04:00
Ed Santiago 547fff2703 e2e tests: use Should(Exit()) and ExitWithError()
e2e test failures are rife with messages like:

   Expected 1 to equal 0

These make me cry. They're anti-helpful, requiring the reader
to dive into the source code to figure out what those numbers
mean.

Solution: Go tests have a '.Should(Exit(NNN))' mechanism. I
don't know if it spits out a better diagnostic (I have no way
to run e2e tests on my laptop), but I have to fantasize that
it will, and given the state of our flakes I assume that at
least one test will fail and give me the opportunity to see
what the error message looks like.

THIS IS NOT REVIEWABLE CODE. There is no way for a human
to review it. Don't bother. Maybe look at a few random
ones for sanity. If you want to really review, here is
a reproducer of what I did:

   cd test/e2e
   ! positive assertions. The second is the same as the first,
   ! with the addition of (unnecessary) parentheses because
   ! some invocations were written that way. The third is BeZero().
   perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(Equal\((\d+)\)\)/Expect($1).Should(Exit($2))/' *_test.go
   perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(\(Equal\((\d+)\)\)\)/Expect($1).Should(Exit($2))/' *_test.go
   perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(BeZero\(\)\)/Expect($1).Should(Exit(0))/' *_test.go

   ! Same as above, but handles three non-numeric exit codes
   ! in run_exit_test.go
   perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(Equal\((\S+)\)\)/Expect($1).Should(Exit($2))/' *_test.go

   ! negative assertions. Difference is the spelling of 'To(Not)',
   ! 'ToNot', and 'NotTo'. I assume those are all the same.
   perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(Not\(Equal\((0)\)\)\)/Expect($1).To(ExitWithError())/' *_test.go
   perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.ToNot\(Equal\((0)\)\)/Expect($1).To(ExitWithError())/' *_test.go
   perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.NotTo\(Equal\((0)\)\)/Expect($1).To(ExitWithError())/' *_test.go
   ! negative, old use of BeZero()
   perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.ToNot\(BeZero\(\)\)/Expect($1).Should(ExitWithError())/' *_test.go

Run those on a clean copy of main branch (at the same branch
point as my PR, of course), then diff against a checked-out
copy of my PR. There should be no differences. Then all you
have to review is that my replacements above are sane.

UPDATE: nope, that's not enough, you also need to add gomega/gexec
to the files that don't have it:

   perl -pi -e '$_ .= "$1/gexec\"\n" if m!^(.*/onsi/gomega)"!' $(grep -L gomega/gexec $(git log -1 --stat | awk '$1 ~ /test\/e2e\// { print $1}'))

UPDATE 2: hand-edit run_volume_test.go

UPDATE 3: sigh, add WaitWithDefaultTimeout() to a couple of places

UPDATE 4: skip a test due to bug #10935 (race condition)

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-07-15 05:06:33 -06:00
Valentin Rothberg 59f31d86ac auto-update: add --dry-run
Add a --dry-run flag to `podman auto-update` which will look for new
images but won't perform any pull or restart any service or container.

The "UPDATED" column will now indicate the availability of a newer image
via "pending".

```
$ podman auto-update --dry-run
UNIT                    CONTAINER            IMAGE                   POLICY      UPDATED
container-test.service  08fd34e533fd (test)  localhost:5000/busybox  registry    false
```

Fixes: #9949
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-15 11:39:50 +02:00
OpenShift Merge Robot 1568247ec8
Merge pull request #10919 from vikas-goel/macvlan
Return macvlan object in /network REST API response
2021-07-15 04:06:11 -04:00
Daniel J Walsh c9970647ba
podman-remote build use .containerignore over .dockerignore
$ mkdir zzz;cd zzz
$ printf "FROM quay.io/libpod/testimage:20210610\nCOPY ./ ./\nCOPY subdir ./\n" >Dockerfile
$ printf "*\nsubdir\n\!*/sub1*\n" >.dockerignore
$ mkdir subdir; touch subdir/sub1.txt
$ ../bin/podman-remote build .

Should fail, but succeeds because we are not sending over the
.dockerignore file to the server side.  This PR will send the
.dockerignore so the server side and use it.

Fixes: #10907

Also if both .containerignore and .dockerignore in the context
directory, podman-remote should prefer .containerignore and not use
.dockerignore.

Fixes: #10908

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-07-14 20:22:51 -04:00
OpenShift Merge Robot ce4c23c4af
Merge pull request #10880 from vrothberg/auto-update
auto-update: make output more user friendly
2021-07-14 15:26:01 -04:00
Vikas Goel 48e6a8eed0 Return macvlan object in /network REST API response
Fixes: #10266

Signed-off-by: Vikas Goel <vikas.goel@gmail.com>
2021-07-14 12:24:03 -07:00
OpenShift Merge Robot 9570bf6b30
Merge pull request #10662 from mwhahaha/fix-apiv2-tests
Fix broken volume and container tests
2021-07-14 13:42:58 -04:00
OpenShift Merge Robot ace19c75ad
Merge pull request #10916 from mheon/fix_7360
Perform a one-sided close of HTTP attach conn on EOF
2021-07-14 13:39:58 -04:00
Alex Schultz 6ced24d0b6 Fix broken volume and container tests
There are a handful of tests that aren't actually being run because
there are missing \ which is prevented the tests from being executed.
Additionally some of the test syntax was incorrect but not showing up
because these tests didn't run.

Signed-off-by: Alex Schultz <aschultz@redhat.com>
2021-07-14 09:33:21 -06:00