Commit graph

2139 commits

Author SHA1 Message Date
Valentin Rothberg 641272d411 system tests: set PODMAN_TIMEOUT to 120
The new Ubuntu 20.04 VMs seem very slow and fail reproducibly in a build
test (i.e, "wordir, cmd, env, label").  Bumping up the time out to 120
seconds will help get the CI green.

See github.com/containers/podman/pull/8747.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-12-18 14:34:13 +01:00
Valentin Rothberg adcb3a7a60 remote copy
Implement `podman-remote cp` and break out the logic from the previously
added `pkg/copy` into it's basic building blocks and move them up into
the `ContainerEngine` interface and `cmd/podman`.

The `--pause` and `--extract` flags are now deprecated and turned into
nops.

Note that this commit is vendoring a non-release version of Buildah to
pull in updates to the copier package.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-12-18 12:08:49 +01:00
OpenShift Merge Robot 915ae6d9bf
Merge pull request #8756 from Luap99/fix-8748
Always add the default gateway to the cni config file
2020-12-16 22:30:45 +00:00
Paul Holzinger edf0e918e3 Always add the default gateway to the cni config file
`podman network create` should always add a gateway to the cni config.
If no gateway is given use the first ip in the subnet. CNI does not require
the gateway field but we need it because of network inspect.

This worked with previous version but was dropped in Commit(e7a72d72fd).

Fixes #8748

Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
2020-12-16 20:59:45 +01:00
Ed Santiago be0d56ce1c System tests: better diagnostics in completion test
Tracking down a CI failure earlier today, I was slightly
delayed by the absence of context information in a test
failure. This PR adds full command context to each
subtest, making it much easier for the developer to
narrow down the cause of a failure.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-12-16 12:20:07 -07:00
OpenShift Merge Robot bacb2fc526
Merge pull request #8312 from cevich/new_ubuntu_images
Cirrus: Update VM Images; Both Fedora and Ubuntu "prior" flavors run with CGroupsV1 & runc
2020-12-16 15:52:25 +00:00
Chris Evich f66ecc882d
Fix: unpause not supported for CGv1 rootless
Thanks Ed Santiago <santiago@redhat.com> for the fix.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-12-16 08:55:16 -05:00
OpenShift Merge Robot f1f7b8f6c8
Merge pull request #8730 from rhatdan/log
Add LogSize to container inspect
2020-12-16 11:23:23 +00:00
Chris Evich 427731ab9c
Disable incompatible rootless + CGroupsV1 tests
These tests simply will not work under these conditions.

Note: Recently updated F32 (prior-fedora) and Ubuntu 20.04
(prior-ubuntu) VMs always use CGroupsV1 with runc.  F33 and
Ubuntu 20.10 were updated to always use CGroupsV2 with crun.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-12-15 18:49:10 -05:00
Chris Evich 0bb865e6c2
Disable rootless pod stats tests w/ CgroupV1
When running as rootless, on a CgroupV1 host these tests all report:
`Error: pod stats is not supported in rootless mode without cgroups v2`

Note: Recently updated F32 (prior-fedora) and Ubuntu 20.04
(prior-ubuntu) VMs always use CGroupsV1 with runc.  F33 and
Ubuntu 20.10 were updated to always use CGroupsV2 with crun.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-12-15 18:49:09 -05:00
Chris Evich e6fbc15f26
Disable CGv1 pod stats on net=host post
This should be addressed by PR
https://github.com/containers/podman/pull/8685

Note: Recently updated F32 (prior-fedora) and Ubuntu 20.04
(prior-ubuntu) VMs always use CGroupsV1 with runc.  F33 and
Ubuntu 20.10 were updated to always use CGroupsV2 with crun.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-12-15 18:49:09 -05:00
Chris Evich 8997a2d106
Disable pod stats tests in containerized Fedora w/ CGroupsV1
Nearly/all of the 'podman stats' tests fail on Fedora when
executing testing inside a container, and CGroupsV1 is used on the
host.  The typical failure message is of the form `Error: unable to
load cgroup at /machine.slice/.../: cgroup deleted`.

Note: Recently updated F32 (prior-fedora) and Ubuntu 20.04
(prior-ubuntu) VMs always use CGroupsV1 with runc.  F33 and
Ubuntu 20.10 were updated to always use CGroupsV2 with crun.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-12-15 18:49:09 -05:00
Chris Evich e6ab566574
Disable blkio.weight test on Ubuntu
These tests fail with `Error: opening file `io.bfq.weight` for writing:
Permission denied: OCI permission denied`.  Upon examination of the
VMs, it was found the kernel and OS lacks support for the `BFQ`
scheduler (which supplies the `weight` option).  The only available
schedulers are `none` and `mq-deadline`.

Note: Recently updated F32 (prior-fedora) and Ubuntu 20.04
(prior-ubuntu) VMs always use CGroupsV1 with runc.  F33 and
Ubuntu 20.10 were updated to always use CGroupsV2 with crun.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-12-15 18:49:09 -05:00
Daniel J Walsh e42d920ebf
Add LogSize to container inspect
Other log options are available so we need to add ability to look
up LogSize.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-12-15 17:22:06 -05:00
Ed Santiago 1345d0358b system tests: the catch-up game
- run test: minor cleanup to .containerenv test. Basically,
  make it do only two podman-runs (they're expensive) and
  tighten up the results checks

- ps test: add ps -a --storage. Requires small tweak to
  run_podman helper, so we can have "timeout" be an expected
  result

- sdnotify test: workaround for #8718 (seeing MAINPID=xxx as
  last output line instead of READY=1). As found by the
  newly-added debugging echos, what we are seeing is:

      MAINPID=103530
      READY=1
      MAINPID=103530

  It's not supposed to be that way; it's supposed to be just
  the first two. But when faced with reality, we must bend
  to accommodate it, so let's accept READY=1 anywhere in
  the output stream, not just as the last line.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-12-14 15:06:43 -07:00
OpenShift Merge Robot 2854cdcd58
Merge pull request #8714 from edsantiago/more_journald_rootless_skips
RHEL gating tests: more journald exceptions
2020-12-14 10:40:34 -05:00
OpenShift Merge Robot 0fd31e2994
Merge pull request #8696 from Luap99/podman-events-until
podman events allow future time for --until
2020-12-14 09:33:42 -05:00
OpenShift Merge Robot e5741b984e
Merge pull request #8689 from bblenard/issue-8672-volume-prune
Add volume prune --filter support
2020-12-14 09:22:30 -05:00
Ed Santiago 4d7e05f9ba RHEL gating tests: more journald exceptions
Followup to #8284, due to my not having noticed #8096.

RHEL gating tests are failing again due to rhbz#1895105, the
one where we can't run journalctl rootless on RHEL. #8284 fixed
this for some RHEL builds of older podman, but I missed #8096
which added yet another logs test.

This brings us to three journalctl exceptions, which means
it gets complicated because I have to refactor it all.

**THIS IS NOT SUSTAINABLE**. We need some way to have a similar
setup in CI, with a permission-less rootless login, so we don't
add yet another logs test some day and discover, months later,
that it doesn't work on RHEL and then have to go into crisis
mode.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-12-14 06:40:46 -07:00
Baron Lenardson a0204ada09 Add volume prune --filter support
This change adds support for the `--filter` / `?filters` arguments on
the `podman volume prune` subcommand.

  * Adds ParseFilterArgumentsIntoFilters helper for consistent
    Filter string slice handling
  * Adds `--filter` support to podman volume prune cli
  * Adds `?filters...` support to podman volume prune api
  * Updates apiv2 / e2e tests

Closes #8672

Signed-off-by: Baron Lenardson <lenardson.baron@gmail.com>
2020-12-12 20:07:04 -06:00
OpenShift Merge Robot a226e6ed31
Merge pull request #8553 from baude/composeci
add compose regression to ci
2020-12-12 06:50:25 -05:00
OpenShift Merge Robot 36bec385f3
Merge pull request #7718 from QiWang19/sign-multi-arch
Sign multi-arch images
2020-12-12 03:46:31 -05:00
OpenShift Merge Robot fbcd44567c
Merge pull request #8688 from rhatdan/start
Handle --rm when starting a container
2020-12-11 21:54:55 -05:00
Paul Holzinger 74fcd9fef3 podman events allow future time for --until
The podman events aren't read until the given timestamp if the
timestamp is in the future. It just reads all events until now
and exits afterwards.
This does not make sense and does not match docker. The correct
behavior is to read all events until the given time is reached.

This fixes a bug where the wrong event log file path was used
when running first time with a new storage location.
Fixes #8694

This also fixes the events api endpoint which only exited when
an error occurred. Otherwise it just hung after reading all events.

Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
2020-12-11 23:15:09 +01:00
Qi Wang 6730556e2f Sign multi-arch images
podman image sign handles muti-arch images.
--all option to create signature for each manifest from the image manifest list.

Signed-off-by: Qi Wang <qiwan@redhat.com>
2020-12-11 14:15:56 -05:00
baude cb91bf96e0 add compose test descriptions
adding compose test descriptions and validations.

Signed-off-by: baude <bbaude@redhat.com>
2020-12-11 10:07:13 -06:00
OpenShift Merge Robot b0a287ce46
Merge pull request #8686 from Luap99/logs-stderr
podman logs honor stderr correctly
2020-12-11 10:38:17 -05:00
Ed Santiago f3e69d7300 test-compose: rewrite to new subdir form
...in which we use all-local tests

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-12-11 09:27:42 -06:00
baude 82d7b9f2e2 add compose regression to ci
to prevent any regressions, we should be running regression tests using
compose.

Signed-off-by: baude <bbaude@redhat.com>
2020-12-11 09:26:29 -06:00
Ed Santiago 47af0afbd1 WIP: test docker-compose
git-fetch a set of docker-compose examples

Add a mechanism for 'curl'ing to verify that service is up.

For each docker-compose directory:
  1) test that it comes up
  2) use 'curl' to confirm that the service is up
  3) bring it back down

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-12-11 09:23:50 -06:00
OpenShift Merge Robot 99ac30a882
Merge pull request #8690 from zhangguanzhang/apiv2-wrong-ImgName
Fix Wrong img name used when creating a container from an image which had multi names
2020-12-11 10:20:16 -05:00
Paul Holzinger 2a35387e9e Fix some network compat api problems
Network create could panic when used with a json body like this:
`{"Name":"net","IPAM":{"Config":[]}}`

The network scope for list and inspect should not be empty. It can
be swarm, global or local. We only support local networks so
hardcode this field to local.

Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
2020-12-11 14:16:05 +01:00
zhangguanzhang fb25f737e5 Fix Wrong image tag is used when creating a container from an image with multiple tags
Signed-off-by: zhangguanzhang <zhangguanzhang@qq.com>
2020-12-11 20:02:41 +08:00
Daniel J Walsh 098e0a7885
Handle --rm when starting a container
podman start should follow the same behaviour as podman run when removing a
container.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-12-11 06:07:14 -05:00
Paul Holzinger ba545c49a2 podman logs honor stderr correctly
Make the ContainerLogsOptions support two io.Writers,
one for stdout and the other for stderr. The logline already
includes the information to which Writer it has to be written.

Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
2020-12-10 22:24:43 +01:00
Paul Holzinger 3fa61f0004 Fix panic in libpod images exists endpoint
The libpod images exists endpoint panics when called with
a non existing image and therefore returns 500 as status
code instead of the expected 404.

A test is added to ensure it is working.

Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
2020-12-10 14:35:17 +01:00
OpenShift Merge Robot eb053df784
Merge pull request #8670 from rhatdan/caps
Properly handle --cap-add all when running with a --user flag
2020-12-09 16:04:45 -05:00
OpenShift Merge Robot 059c2ee739
Merge pull request #8669 from giuseppe/unmask-also-cover-ro-paths
security: honor systempaths=unconfined for ro paths
2020-12-09 15:04:41 -05:00
OpenShift Merge Robot 6db3e9bfeb
Merge pull request #8653 from rhatdan/buildah
Honor the --layers flag
2020-12-09 14:58:28 -05:00
OpenShift Merge Robot da062b5bd0
Merge pull request #8635 from Luap99/shell-completion-test
Add system test for shell completion
2020-12-09 14:47:05 -05:00
Daniel J Walsh 1ace9e3ba6
Properly handle --cap-add all when running with a --user flag
Handle the ALL Flag when running with an account as a user.

Currently we throw an error when the user specifies

podman run --user bin --cap-add all fedora echo hello

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-12-09 14:21:16 -05:00
Giuseppe Scrivano 176be90e0a
security: honor systempaths=unconfined for ro paths
we must honor systempaths=unconfined also for read-only paths, as
Docker does:

proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2020-12-09 19:26:23 +01:00
Paul Holzinger 2870a0b0a6 Add system test for shell completion
There exists a unit test to ensure that shell completion functions are
defined. However there was no check about the quality of the provided
shell completions. Lets change that.

The idea is to create a general test that makes sure we are suggesting
containers,pods,images... for the correct commands. This works by
reading the command use line and checking for each arg if we provide
the correct suggestions for this arg.

It includes the following tests:
- flag suggestions if [options] is set
- container, pod, image, network, volume, registry completion
- path completion for the appropriate arg KEYWORDS (`PATH`,`CONTEXT`,etc.)
- no completion if there are no args
- completion for more than one arg if it ends with `...]`

The test does not cover completion values for flags and not every arg KEYWORD
is supported. This is still a huge improvement and covers most use cases.

This test spotted several inconsistencies between the completion and the
command use line. All of them have been adjusted to make the test pass.

The biggest advantage is that the completions always match the latest
command changes. So if someone changes the arguments for a command this
ensures that the completions must be adjusted.

Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
2020-12-09 19:13:28 +01:00
Daniel J Walsh 0154f9e9b3
Honor the --layers flag
Currently the --layers flag set by the user is ignored, and only the BUILDAH_LAYERS
environment variable being set is observed.

Fixes: https://github.com/containers/podman/issues/8643

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-12-09 08:31:36 -05:00
Matthew Heon 6b7612062e Correct port range logic for port generation
The existing logic (Range > 0) always triggered, because range is
guaranteed to be at least 1 (a single port has a range of 1, a
two port range (e.g. 80-81) has a range of 2, and so on). As such
this could cause ports that had a host port assigned to them by
the user to randomly assign one instead.

Fixes #8650
Fixes #8651

Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-12-08 16:23:14 -05:00
OpenShift Merge Robot 7caef9c497
Merge pull request #8642 from jwhonce/issues/8444
Restore json format for fields as well as whole structs
2020-12-08 15:30:31 -05:00
OpenShift Merge Robot 47d2a4be2a
Merge pull request #8648 from mheon/fix_7883
Make `podman stats` slirp check more robust
2020-12-08 13:08:05 -05:00
OpenShift Merge Robot 3bf02fb00a
Merge pull request #8630 from umohnani8/sec-opt
Add systempaths=unconfined option
2020-12-08 12:19:26 -05:00
Matthew Heon 46337b4708 Make podman stats slirp check more robust
Just checking for `rootless.IsRootless()` does not catch all the
cases where slirp4netns is in use - we actually allow it to be
used as root as well. Fortify the conditional here so we don't
fail in the root + slirp case.

Fixes #7883

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-12-08 09:59:00 -05:00
Urvashi Mohnani 239bd57662 Add systempaths=unconfined option
Add the systempaths=unconfined option to --security-opt
to match the docker options for unmasking all the paths
that are masked by default.
Add the mask and unmask options to the podman create doc.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2020-12-08 09:42:33 -05:00