Commit graph

3029 commits

Author SHA1 Message Date
OpenShift Merge Robot 230f0b622e
Merge pull request #12130 from rhatdan/journal
Error logs --follow if events-backend != journald, event-logger=journald
2021-11-15 20:55:28 +01:00
Nalin Dahyabhai 63ef7135d9 journald logs: keep reading until the journal's end
When reading logs from the journal, keep going after the container
exits, in case it gets restarted.

Events logged to the journal via the normal paths don't include
CONTAINER_ID_FULL, so don't bother adding it to the "history" event we
use to force at least one entry for the container to show up in the log.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-11-15 13:38:36 -05:00
Aditya Rajan 014cc4b9d9
secret: honor custom target for secrets with run
Honor custom `target` if specified while running or creating containers
with secret `type=mount`.

Example:
`podman run -it --secret token,type=mount,target=TOKEN ubi8/ubi:latest
bash`

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-11-15 23:19:27 +05:30
Paul Holzinger 295d87bb0b
podman machine improve port forwarding
This commits adds port forwarding logic directly into podman. The
podman-machine cni plugin is no longer needed.

The following new features are supported:
 - works with cni, netavark and slirp4netns
 - ports can use the hostIP to bind instead of hard coding 0.0.0.0
 - gvproxy no longer listens on 0.0.0.0:7777 (requires a new gvproxy
   version)
 - support the udp protocol

With this we no longer need podman-machine-cni and should remove it from
the packaging. There is also a change to make sure we are backwards
compatible with old config which include this plugin.

Fixes #11528
Fixes #11728

[NO NEW TESTS NEEDED] We have no podman machine test at the moment.
Please test this manually on your system.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-15 15:20:47 +01:00
Adrian Reber 80e56fa12b
Added optional container restore statistics
This adds the parameter '--print-stats' to 'podman container restore'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to restore a checkpoint and print out these
information. CRIU already creates process restore statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the restored container, Podman will now print
out JSON:

 # podman container restore --latest --print-stats
 {
     "podman_restore_duration": 305871,
     "container_statistics": [
         {
             "Id": "47b02e1d474b5d5fe917825e91ac653efa757c91e5a81a368d771a78f6b5ed20",
             "runtime_restore_duration": 140614,
             "criu_statistics": {
                 "forking_time": 5,
                 "restore_time": 67672,
                 "pages_restored": 14
             }
         }
     ]
 }

The output contains 'podman_restore_duration' which contains the
number of microseconds Podman required to restore the checkpoint. The
output also includes 'runtime_restore_duration' which is the time
the runtime needed to restore that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-15 11:50:25 +00:00
Adrian Reber 6202e8102b
Added optional container checkpointing statistics
This adds the parameter '--print-stats' to 'podman container checkpoint'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to create a checkpoint and print out these
information. CRIU already creates checkpointing statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the checkpointed container, Podman will now print
out JSON:

 # podman container checkpoint --latest --print-stats
 {
     "podman_checkpoint_duration": 360749,
     "container_statistics": [
         {
             "Id": "25244244bf2efbef30fb6857ddea8cb2e5489f07eb6659e20dda117f0c466808",
             "runtime_checkpoint_duration": 177222,
             "criu_statistics": {
                 "freezing_time": 100657,
                 "frozen_time": 60700,
                 "memdump_time": 8162,
                 "memwrite_time": 4224,
                 "pages_scanned": 20561,
                 "pages_written": 2129
             }
         }
     ]
 }

The output contains 'podman_checkpoint_duration' which contains the
number of microseconds Podman required to create the checkpoint. The
output also includes 'runtime_checkpoint_duration' which is the time
the runtime needed to checkpoint that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-15 11:50:24 +00:00
Daniel J Walsh 062c887718
Error logs --follow if events-backend != journald, event-logger=journald
Fixes: https://github.com/containers/podman/issues/11255

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-11-13 07:11:09 -05:00
OpenShift Merge Robot d6d89fa79f
Merge pull request #12267 from giuseppe/safely-create-etc-mtab
libpod: create /etc/mtab safely
2021-11-11 20:47:42 +01:00
Paul Holzinger 3af19917a1
Add failing run test for netavark
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-11 17:50:10 +01:00
Paul Holzinger fe90a45e0d
Add flag to overwrite network backend from config
To make testing easier we can overwrite the network backend with the
global `--network-backend` option.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-11 17:30:27 +01:00
Giuseppe Scrivano 9f4d63f91b
libpod: create /etc/mtab safely
make sure the /etc/mtab symlink is created inside the rootfs when /etc
is a symlink.

Closes: https://github.com/containers/podman/issues/12189

[NO NEW TESTS NEEDED] there is already a test case

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-11 17:00:53 +01:00
Paul Holzinger 8041d44c93
Add network backend to podman info
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-11 16:49:46 +01:00
Paul Holzinger b2f7430b67
Add more netavark tests
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-11 16:49:28 +01:00
Paul Holzinger 1c88f741a7
select network backend based on config
You can change the network backendend in containers.conf supported
values are "cni" and "netavark".

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-11 16:26:47 +01:00
Paul Holzinger 3fe0c49174
Fix RUST_LOG envar for netavark
THe rust netlink library is very verbose. It contains way to much debug
and trave logs. We can set `RUST_LOG=netavark=<level>` to make sure this
log level only applies to netavark and not the libraries.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-11 16:25:52 +01:00
Paul Holzinger 4febe55769
netavark IPAM assignment
Add a new boltdb to handle IPAM assignment.

The db structure is the following:
Each network has their own bucket with the network name as bucket key.
Inside the network bucket there is an ID bucket which maps the container ID (key)
to a json array of ip addresses (value).
The network bucket also has a bucket for each subnet, the subnet is used as key.
Inside the subnet bucket an ip is used as key and the container ID as value.

The db should be stored on a tmpfs to ensure we always have a clean
state after a reboot.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-11 16:25:19 +01:00
Paul Holzinger eaae294628
netavark network interface
Implement a new network interface for netavark.
For now only bridge networking is supported.
The interface can create/list/inspect/remove networks. For setup and
teardown netavark will be invoked.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-11 15:54:02 +01:00
Paul Holzinger 12c62b92ff
Make networking code reusable
To prevent code duplication when creating new network backends move
reusable code into a separate internal package.

This allows all network backends to use the same code as long as they
implement the new NetUtil interface.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-11 15:54:02 +01:00
Paul Holzinger 3690532b3b
network reload return error if we cannot reload ports
As rootless we have to reload the port mappings. If it fails we should
return an error instead of the warning.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-10 21:16:30 +01:00
Paul Holzinger 27de152b5a
network reload without ports should not reload ports
When run as rootless the podman network reload command tries to reload
the rootlessport ports because the childIP could have changed.
However if the containers has no ports we should skip this instead of
printing a warning.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-10 21:16:08 +01:00
OpenShift Merge Robot 5437568fcd
Merge pull request #12227 from Luap99/net-setup
Fix rootless networking with userns and ports
2021-11-09 21:11:30 +01:00
OpenShift Merge Robot 43bd57c7fb
Merge pull request #12195 from boaz0/closes_11998
podman-generate-kube - remove empty structs from YAML
2021-11-09 19:46:31 +01:00
Paul Holzinger 216e2cb366
Fix rootless networking with userns and ports
A rootless container created with a custom userns and forwarded ports
did not work. I refactored the network setup to make the setup logic
more clear.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-09 15:58:57 +01:00
Ian Wienand 72cf389685 shm_lock: Handle ENOSPC better in AllocateSemaphore
When starting a container libpod/runtime_pod_linux.go:NewPod calls
libpod/lock/lock.go:AllocateLock ends up in here.  If you exceed
num_locks, in response to a "podman run ..." you will see:

 Error: error allocating lock for new container: no space left on device

As noted inline, this error is technically true as it is talking about
the SHM area, but for anyone who has not dug into the source (i.e. me,
before a few hours ago :) your initial thought is going to be that
your disk is full.  I spent quite a bit of time trying to diagnose
what disk, partition, overlay, etc. was filling up before I realised
this was actually due to leaking from failing containers.

This overrides this case to give a more explicit message that
hopefully puts people on the right track to fixing this faster.  You
will now see:

 $ ./bin/podman run --rm -it fedora bash
 Error: error allocating lock for new container: allocation failed; exceeded num_locks (20)

[NO NEW TESTS NEEDED] (just changes an existing error message)

Signed-off-by: Ian Wienand <iwienand@redhat.com>
2021-11-09 18:34:21 +11:00
Valentin Rothberg 6444f24028 pod/container create: resolve conflicts of generated names
Address the TOCTOU when generating random names by having at most 10
attempts to assign a random name when creating a pod or container.

[NO TESTS NEEDED] since I do not know a way to force a conflict with
randomly generated names in a reasonable time frame.

Fixes: #11735
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-08 13:33:30 +01:00
OpenShift Merge Robot 865653b661
Merge pull request #12184 from adrianreber/2021-11-05-stats-dump
Add 'stats-dump' file to exported checkpoint
2021-11-08 09:29:56 +01:00
Boaz Shuster f3fab1e17c podman-generate-kube - remove empty structs from YAML
[NO NEW TESTS NEEDED]

Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
2021-11-07 16:33:38 +02:00
OpenShift Merge Robot abbd6c167e
Merge pull request #11890 from Luap99/ports
libpod: deduplicate ports in db
2021-11-06 10:39:16 +01:00
Paul Holzinger 02f67181a2
Fix swagger definition for the new mac address type
The new mac address type broke the api docs. While we could
successfully generate the swagger file it could not be viewed in a
browser.

The problem is that the swagger generation create two type definitions
with the name `HardwareAddr` and this pointed back to itself. Thus the
render process was stucked in an endless loop. To fix this manually
rename the new type to MacAddress and overwrite the types to string
because the json unmarshaller accepts the mac as string.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-05 19:25:40 +01:00
Adrian Reber 6b8fc3bd1d
Add 'stats-dump' file to exported checkpoint
There was the question about how long it takes to create a checkpoint.
CRIU already provides some statistics about how long it takes to create
a checkpoint and similar.

With this change the file 'stats-dump' is included in the checkpoint
archive and the tool checkpointctl can be used to display these
statistics:

./checkpointctl show -t /tmp/cp.tar --print-stats

Displaying container checkpoint data from /tmp/dump.tar

[...]
CRIU dump statistics
+---------------+-------------+--------------+---------------+---------------+---------------+
| FREEZING TIME | FROZEN TIME | MEMDUMP TIME | MEMWRITE TIME | PAGES SCANNED | PAGES WRITTEN |
+---------------+-------------+--------------+---------------+---------------+---------------+
| 105405 us     | 1376964 us  | 504399 us    | 446571 us     |        492153 |         88689 |
+---------------+-------------+--------------+---------------+---------------+---------------+

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-05 16:15:00 +00:00
Paul Holzinger 7f433df7e7
rename rootless cni ns to rootless netns
Since we want to use the rootless cni ns also for netavark we should
pick a more generic name. The name is now "rootless network namespace"
or short "rootless netns".

The rename might cause some issues after the update but when the
all containers are restarted or the host is rebooted it should work
correctly.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-05 15:44:37 +01:00
Paul Holzinger 58f8c3d743
mount full XDG_RUNTIME_DIR in rootless cni ns
We should mount the full runtime directory into the namespace instead of
just the netns dir. This allows more use cases.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-05 15:41:04 +01:00
Paul Holzinger 614c6f5970
Fix rootless cni netns cleanup logic
The check if cleanup is needed reads all container and checks if there
are running containers with bridge networking. If we do not find any we
have to cleanup the ns. However there was a problem with this because
the state is empty by default so the running check never worked.
Fortunately the was a second check which relies on the CNI files so we
still did cleanup anyway.

With netavark I noticed that this check is broken because the CNI files
were not present.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-05 00:20:10 +01:00
Paul Holzinger 001d48929d
MAC address json unmarshal should allow strings
Create a new mac address type which supports json marshal/unmarshal from
and to string. This change is backwards compatible with the previous
versions as the unmarshal method still accepts the old byte array or
base64 encoded string.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-03 15:30:16 +01:00
OpenShift Merge Robot 1305902ff4
Merge pull request #12127 from vrothberg/bz-2014149
volumes: be more tolerant and fix infinite loop
2021-10-29 13:30:29 +00:00
OpenShift Merge Robot d2147bada6
Merge pull request #12117 from adrianreber/2021-10-27-set-checkpointed-false-after-restore
Set Checkpointed state to false after restore
2021-10-28 18:06:25 +00:00
Valentin Rothberg c5f0a5d788 volumes: be more tolerant and fix infinite loop
Make Podman more tolerant when parsing image volumes during container
creation and further fix an infinite loop when checking them.

Consider `VOLUME ['/etc/foo', '/etc/bar']` in a Containerfile.  While
it looks correct to the human eye, the single quotes are wrong and yield
the two volumes to be `[/etc/foo,` and `/etc/bar]` in Podman and Docker.

When running the container, it'll create a directory `bar]` in `/etc`
and a directory `[` in `/` with two subdirectories `etc/foo,`.  This
behavior is surprising to me but how Docker behaves.  We may improve on
that in the future.  Note that the correct way to syntax for volumes in
a Containerfile is `VOLUME /A /B /C` or `VOLUME ["/A", "/B", "/C"]`;
single quotes are not supported.

This change restores this behavior without breaking container creation
or ending up in an infinite loop.

BZ: https://bugzilla.redhat.com/show_bug.cgi?id=2014149
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-28 16:37:33 +02:00
OpenShift Merge Robot 3bc449371c
Merge pull request #12126 from giuseppe/fix-race-warning-message
runtime: change PID existence check
2021-10-28 12:16:24 +00:00
Giuseppe Scrivano 960831f9c8
runtime: change PID existence check
commit 6b3b0a17c6 introduced a check for
the PID file before attempting to move the PID to a new scope.

This is still vulnerable to TOCTOU race condition though, since the
PID file or the PID can be removed/killed after the check was
successful but before it was used.

Closes: https://github.com/containers/podman/issues/12065

[NO NEW TESTS NEEDED] it fixes a CI flake

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-28 11:18:48 +02:00
Giuseppe Scrivano 9e5cd32056
oci: rename sub-cgroup to runtime instead of supervisor
we are having a hard time figuring out a failure in the CI:

https://github.com/containers/podman/issues/11191

Rename the sub-cgroup created here, so we can be certain the error is
caused by this part.

[NO NEW TESTS NEEDED] we need this for the CI.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-28 09:18:08 +02:00
Paul Holzinger 0136a66a83
libpod: deduplicate ports in db
The OCICNI port format has one big problem: It does not support ranges.
So if a users forwards a range of 1k ports with podman run -p 1001-2000
we have to store each of the thousand ports individually as array element.
This bloats the db and makes the JSON encoding and decoding much slower.
In many places we already use a better port struct type which supports
ranges, e.g. `pkg/specgen` or the new network interface.

Because of this we have to do many runtime conversions between the two
port formats. If everything uses the new format we can skip the runtime
conversions.

This commit adds logic to replace all occurrences of the old format
with the new one. The database will automatically migrate the ports
to new format when the container config is read for the first time
after the update.

The `ParsePortMapping` function is `pkg/specgen/generate` has been
reworked to better work with the new format. The new logic is able
to deduplicate the given ports. This is necessary the ensure we
store them efficiently in the DB. The new code should also be more
performant than the old one.

To prove that the code is fast enough I added go benchmarks. Parsing
1 million ports took less than 0.5 seconds on my laptop.

Benchmark normalize PortMappings in specgen:
Please note that the 1 million ports are actually 20x 50k ranges
because we cannot have bigger ranges than 65535 ports.
```
$ go test -bench=. -benchmem  ./pkg/specgen/generate/
goos: linux
goarch: amd64
pkg: github.com/containers/podman/v3/pkg/specgen/generate
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
BenchmarkParsePortMappingNoPorts-12             480821532                2.230 ns/op           0 B/op          0 allocs/op
BenchmarkParsePortMapping1-12                      38972             30183 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMapping100-12                    18752             60688 ns/op          141088 B/op        315 allocs/op
BenchmarkParsePortMapping1k-12                      3104            331719 ns/op          223840 B/op       3018 allocs/op
BenchmarkParsePortMapping10k-12                      376           3122930 ns/op         1223650 B/op      30027 allocs/op
BenchmarkParsePortMapping1m-12                         3         390869926 ns/op        124593840 B/op   4000624 allocs/op
BenchmarkParsePortMappingReverse100-12             18940             63414 ns/op          141088 B/op        315 allocs/op
BenchmarkParsePortMappingReverse1k-12               3015            362500 ns/op          223841 B/op       3018 allocs/op
BenchmarkParsePortMappingReverse10k-12               343           3318135 ns/op         1223650 B/op      30027 allocs/op
BenchmarkParsePortMappingReverse1m-12                  3         403392469 ns/op        124593840 B/op   4000624 allocs/op
BenchmarkParsePortMappingRange1-12                 37635             28756 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange100-12               39604             28935 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange1k-12                38384             29921 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange10k-12               29479             40381 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange1m-12                  927           1279369 ns/op          143022 B/op        164 allocs/op
PASS
ok      github.com/containers/podman/v3/pkg/specgen/generate    25.492s
```

Benchmark convert old port format to new one:
```
go test -bench=. -benchmem  ./libpod/
goos: linux
goarch: amd64
pkg: github.com/containers/podman/v3/libpod
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
Benchmark_ocicniPortsToNetTypesPortsNoPorts-12          663526126                1.663 ns/op           0 B/op          0 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1-12                 7858082               141.9 ns/op            72 B/op          2 allocs/op
Benchmark_ocicniPortsToNetTypesPorts10-12                2065347               571.0 ns/op           536 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts100-12                138478              8641 ns/op            4216 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1k-12                   9414            120964 ns/op           41080 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts10k-12                   781           1490526 ns/op          401528 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1m-12                      4         250579010 ns/op        40001656 B/op          4 allocs/op
PASS
ok      github.com/containers/podman/v3/libpod  11.727s
```

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-27 18:59:56 +02:00
OpenShift Merge Robot 6caf5e3b7c
Merge pull request #12111 from giuseppe/fix-warning-move-pause-process
runtime: check for pause pid existence
2021-10-27 15:07:59 +00:00
Adrian Reber dcbf5cae12
Set Checkpointed state to false after restore
A restored container still had the state set to 'Checkpointed: true'
which seems wrong if it running again.

[NO NEW TESTS NEEDED]

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-10-27 13:40:54 +00:00
OpenShift Merge Robot 979b631228
Merge pull request #11956 from vrothberg/pause
remove need to download pause image
2021-10-27 10:22:56 +00:00
Giuseppe Scrivano 6b3b0a17c6
runtime: check for pause pid existence
check that the pause pid exists before trying to move it to a separate
scope.

Closes: https://github.com/containers/podman/issues/12065

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-27 11:24:50 +02:00
OpenShift Merge Robot ed3aa2acaf
Merge pull request #12098 from Luap99/slirp-dad
Slirp4netns with ipv6 set net.ipv6.conf.default.accept_dad=0
2021-10-26 20:54:27 +00:00
OpenShift Merge Robot 1243954372
Merge pull request #12067 from hshiina/logs-journal-tail
Fix a few problems in 'podman logs --tail' with journald driver
2021-10-26 20:33:26 +00:00
OpenShift Merge Robot 420ac5d13d
Merge pull request #12088 from adrianreber/2021-10-25-fix-label-ipc-host
Allow 'container restore' with '--ipc host'
2021-10-26 16:38:54 +00:00
Paul Holzinger 008075ce54
Slirp4netns with ipv6 set net.ipv6.conf.default.accept_dad=0
Duplicate Address Detection slows the ipv6 setup down for 1-2 seconds.
Since slirp4netns is run it is own namespace and not directly routed
we can skip this to make the ipv6 address immediately available.
We change the default to make sure the slirp tap interface gets the
correct value assigned so DAD is disabled for it.
Also make sure to change this value back to the original after slirp4netns
is ready in case users rely on this sysctl.

Fixes #11062

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-26 18:27:30 +02:00
Hironori Shiina c723e6b978 Fix a few problems in 'podman logs --tail' with journald driver
The following problems regarding `logs --tail` with the journald log
driver are fixed:
- One more line than a specified value is displayed.
- '--tail 0' displays all lines while the other log drivers displays
  nothing.
- Partial lines are not considered.
- If the journald events backend is used and a container has exited,
  nothing is displayed.

Integration tests that should have detected the bugs are also fixed. The
tests are executed with json-file log driver three times without this
fix.

Signed-off-by: Hironori Shiina <shiina.hironori@jp.fujitsu.com>
2021-10-26 12:18:57 -04:00
Adrian Reber bf8fd943ef
Allow 'container restore' with '--ipc host'
Trying to restore a container that was started with '--ipc host' fails
with:

Error: error creating container storage: ProcessLabel and Mountlabel must either not be specified or both specified

We already fixed this exact same error message for containers started
with '--privileged'. The previous fix was to check if the to be restored
container is a privileged container (c.config.Privileged). Unfortunately
this does not work for containers started with '--ipc host'.

This commit changes the check for a privileged container to check if
both the ProcessLabel and the MountLabel is actually set and only then
re-uses those labels.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-10-26 14:42:32 +00:00
Paul Holzinger efd1c080bf
Document to not set K8S envars for CNI
Setting these environment variables can cause issues with custom CNI
plugins, see #12083.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-26 16:11:46 +02:00
Valentin Rothberg 75f478c08b pod create: remove need for pause image
So far, the infra containers of pods required pulling down an image
rendering pods not usable in disconnected environments.  Instead, build
an image locally which uses local pause binary.

Fixes: #10354
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-26 13:51:45 +02:00
Valentin Rothberg 2e3611d61f overlay root fs: create mount on runtime dir
Make sure to create the mounts for containers with an overlay root FS in
the runtime dir (e.g., /run/user/1000/...) to guarantee that we can
actually overlay mount on the specific path which is not the case for
the graph root.

[NO NEW TESTS NEEDED] since it is not a user-facing change.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-26 13:51:45 +02:00
Daniel J Walsh a42c131c80
Update vendor github.com/opencontainers/runtime-tools
This will change mount of /dev within container to noexec, making
containers slightly more secure.

[NO NEW TESTS NEEDED]

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-25 16:50:45 -04:00
Stefan Weil d7662edf66 [NO NEW TESTS NEEDED] Fix off-by-one index comparision (reported by LGTM)
LGTM alert:

    Off-by-one index comparison against length may lead to out-of-bounds read.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2021-10-25 10:52:01 +02:00
Stefan Weil 22270fb845 Replace 'an user' => 'a user'
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2021-10-24 22:27:39 +02:00
OpenShift Merge Robot 5dd211f91b
Merge pull request #11991 from rhatdan/size
Allow API to specify size and inode quota
2021-10-22 14:18:45 +00:00
OpenShift Merge Robot 833d92d709
Merge pull request #12021 from rhatdan/kube
Generate Kube should not print default structs
2021-10-22 14:12:44 +00:00
Matthew Heon 30245f5849 Remove infra ID from DB before removing containers
If we interrupt pod removal between removing containers and
removing the whole pod, the infra ID was still in the DB, and
most pod operations would try to retrieve the infra container
(and would this fail). Clear the infra ID from the DB just before
we remove all containers to prevent this.

Fixes #12034

[NO NEW TESTS NEEDED] This is a very narrow race and I have no
idea how to repro it.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2021-10-20 14:56:22 -04:00
OpenShift Merge Robot eba281c3e0
Merge pull request #11851 from cdoern/podRm
Pod Rm Infra Handling Improvements
2021-10-20 13:20:12 +00:00
OpenShift Merge Robot 97f051f657
Merge pull request #12017 from nalind/exponential
Use exponential backoff when waiting for a journal entry
2021-10-20 12:49:10 +00:00
Daniel J Walsh 517b56b02d
Generate Kube should not print default structs
If podman uses Workdir="/" or the workdir specified in the image, it
should not add it to the yaml.
If Podman find environment variables in the image, they should not
get added to the yaml.

If the container or pod do not have changes to SELinux we should not
print seLinuxOpt{}

If the container or pod do not change any dns options the yaml should
not have a dnsOption={}

If the container is not privileged it should not have privileged=false
in the yaml.

Fixes: https://github.com/containers/podman/issues/11995

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-19 08:31:35 -04:00
Aditya Rajan 9500e11a8f
libpod: change mountpoint ownership c.Root when using overlay on top of external rootfs
Allow chainging ownership of mountpoint created on top external overlay
rootfs to support use-cases when custom --uidmap and --gidmap are
specified.

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-10-19 16:12:07 +05:30
Daniel J Walsh 087f8fc73b
Allow API to specify size and inode quota
Fixes: https://github.com/containers/podman/issues/11016

[NO NEW TESTS NEEDED] We have no easy way to tests this in
CI/CD systems.  Requires quota to be setup on directories to work.

Fixes: https://github.com/containers/podman/issues/11016

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-18 14:44:10 -04:00
Nalin Dahyabhai db7a98de40 Use exponential backoff when waiting for a journal entry
When looking for a cursor that matches the first journal entry for a
given container, wait and try to find it using exponential backoff.

[NO NEW TESTS NEEDED]

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-10-18 14:16:09 -04:00
OpenShift Merge Robot e0ffc431fe
Merge pull request #11959 from rhatdan/selinux
We should only be relabeling when on first run
2021-10-18 19:53:46 +02:00
cdoern 4f7a431daf Pod Rm Infra Improvements
Made changes so that if the pod contains all exited containers and only infra is running, remove the pod.

resolves #11713

Signed-off-by: cdoern <cdoern@redhat.com>
2021-10-18 11:39:12 -04:00
Aditya Rajan d0f7b99c6d
rootfs-overlay: fix overlaybase path for cleanups
Following commit ensures not dandling mounts are left behind when we are
creating an overlay on top of external rootfs.

Co-authored-by: Valentin Rothberg <rothberg@redhat.com>
Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-10-18 19:01:55 +05:30
Daniel J Walsh 207abc4a9a
We should only be relabeling when on first run
On the second runs, the labels should be the same so no
need to relabel.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2013548

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-15 14:27:14 -04:00
OpenShift Merge Robot 673cc5054c
Merge pull request #11982 from umohnani8/targetPort
Set targetPort to the port value in the kube yaml
2021-10-15 16:56:43 +02:00
OpenShift Merge Robot 307b1b43b2
Merge pull request #11977 from rhatdan/log
Don't include ctr.log if not using file logging
2021-10-15 15:25:46 +02:00
Urvashi Mohnani 8db62d04fd Set targetPort to the port value in the kube yaml
When the targetPort is not defined, it is supposed to
be set to the port value according to the k8s docs.
Add tests for targetPort.
Update tests to be able to check the Service yaml that
is generated.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2021-10-14 17:03:26 -04:00
Urvashi Mohnani 16e7cc8bf1 Do not add TCP to protocol in generated kube yaml
As the default protocol in k8s is TCP, don't add it
to the generate yaml when using protocol.
Add UDP to the protocol of the generated yaml when udp
is being used.
Add tests for this as well.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2021-10-14 17:03:14 -04:00
Daniel J Walsh f9ff0525c5
Don't include ctr.log if not using file logging
Checkpoint is blowing up when you use --log-driver=none

[NO NEW TESTS NEEDED] No way currently to test checkpoint restore.

Fixes: https://github.com/containers/podman/issues/11974

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-14 14:12:15 -04:00
Daniel J Walsh 50b9d82f2e
Don't use docker/pkg/archive, use containers/storage/pkg/archive
[NO NEW TESTS NEEDED]

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-14 10:28:42 -04:00
Urvashi Mohnani 5d18fb9239 Don't add image entrypoint to the generate kube yaml
If no entrypoint or command is set in the podman create
command, and the image command or entrypoint is being
used as the default, then do not add the image command or
entrypoint to the generated kube yaml.
Kubernetes knows to default to the image command and/or
entrypoint settings when not defined in the kube yaml.
Add and modify tests for this case.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2021-10-13 15:18:04 -04:00
OpenShift Merge Robot 9f1452cbb5
Merge pull request #11944 from cdoern/kubeEmit
Kube Gen run as user/group issues
2021-10-13 19:42:14 +02:00
OpenShift Merge Robot a235f43ac0
Merge pull request #11949 from Luap99/net-range
CNI: fix network create --ip-range
2021-10-13 18:42:15 +02:00
OpenShift Merge Robot 1c156f2267
Merge pull request #11948 from rhatdan/codespell
codespell code
2021-10-13 14:46:00 +02:00
OpenShift Merge Robot f12dc28bbc
Merge pull request #11565 from Luap99/rootlessport-bin
rootlessport: reduce memory usage of the process
2021-10-13 10:41:59 +02:00
OpenShift Merge Robot 0b8673d0e7
Merge pull request #11943 from baude/issue11929
No space in kube annotations for bind mounts
2021-10-13 10:01:58 +02:00
Daniel J Walsh 8600bce53a
codespell code
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-12 16:44:25 -04:00
Paul Holzinger cead185373
CNI: fix network create --ip-range
The --ip-range option did not work correctly. The endIP was accidentally
assigned to the start IP. New tests are added to make sure it works.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-12 22:36:24 +02:00
cdoern 4631f5b283 Kube Gen run as user/group issues
Removed the inclusion of RunAsUser or RunAsGroup unless a container is run with the --user flag. When building from an image
the user will be pulled from there anyway

resolves #11914

Signed-off-by: cdoern <cdoern@redhat.com>
2021-10-12 16:22:01 -04:00
Paul Holzinger 3ba69dccf7
rootlessport: reduce memory usage of the process
Don't use reexec for the rootlessport process, instead make it a
separate binary to reduce the memory usage. The problem with reexec is
that it will import all packages that podman uses and therefore loads a
lot of stuff into the heap. The rootlessport process however only needs
the rootlesskit library.
The memory usage is a concern since the rootlessport process will spawn
two process per container which has ports forwarded. The processes stay
until the container dies. On my laptop the current reexec version uses
47800 KB RSS. The new separate binary only uses 4540 KB RSS. This is
more than a 90% improvement.

The Makefile has been updated to compile the new binary and install it
to the libexec directory.

Fixes #10790

[NO TESTS NEEDED]

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-12 21:43:11 +02:00
OpenShift Merge Robot c90beedbe1
Merge pull request #11925 from rhatdan/volume
Remove a volume with --force if container is running
2021-10-12 21:27:09 +02:00
Brent Baude ab0e6630f5 No space in kube annotations for bind mounts
Kubernetes fails to deal with an annotation that has a space in it.
Trim these strings to remove spaces.

Fixes: #11929

Signed-off-by: Brent Baude <bbaude@redhat.com>

[NO TESTS NEEDED]
2021-10-12 12:38:15 -05:00
Jhon Honce 0459484bdf Fix CI flake on time of shutdown for API service
* Increase timeout for tests to 10s
* To aid in debugging add PID to shutdown package logging
* Added new message for forced service shutdown
* Always wait for HTTP server to shutdown, duration of 0 not friendly
  to clients

Note: The log event

"IdleTracker: StateClosed transition by connection marked un-managed"

denotes a TCP connection has been initiated but no HTTP request was sent.
And is expected during these tests.

Fixes #11921

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-10-12 09:53:19 -07:00
Daniel J Walsh 7580c22734
Remove a volume with --force if container is running
Currently we are not passing the force flag down to the removal of
the running container. If the container is running, and we set
--force when removing the volume, the container should be stopped.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-11 15:02:04 -04:00
OpenShift Merge Robot 60c711f789
Merge pull request #11869 from jwhonce/wip/pprof
Enable /debug/pprof API service endpoints
2021-10-10 12:54:29 +02:00
OpenShift Merge Robot c47f714fe8
Merge pull request #11880 from rhatdan/stoptimeout
Warn if podman stop timeout expires that sigkill was sent
2021-10-10 12:52:30 +02:00
Jhon Honce 8d3aec9d08 Enable /debug/pprof API service endpoints
* Refactor sidecar HTTP service for /debug/pprof endpoints to use a TCP
  address given via new podman system service --pprof-address flag

* Allow same URL parsing in "system service" as bindings/connection.go

* Refactor NewServerWithSettings() to use entities.ServiceOptions
  in place of deleted server.Options

* Updated godoc for impacted functions and types

* Fixed API service Shutdown() to do an orderly shutdown when
  terminated and running with --time=0

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-10-08 13:57:20 -07:00
Valentin Rothberg fad14dafe1 faster image inspection
Vendor the latest HEAD in c/common to pull in changes for a faster
inspection of images.  Previously, only the size computation was
optional, now the one for the parent image is as well.

In many cases, the parent image is not needed but it takes around 10ms
on my local machine.  With this change, we cut off 10ms from many code
paths, most importantly, container creation.

[NO NEW TESTS NEEDED]

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-08 14:47:33 +02:00
Daniel J Walsh 48d26a893e
Warn if podman stop timeout expires that sigkill was sent
Note: the Warning message will not come to podman-remote.
It would be difficult to plumb, and not really worth the effort.

Fixes: https://github.com/containers/podman/issues/11854

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-08 07:06:37 -04:00
OpenShift Merge Robot 470e52f272
Merge pull request #11878 from mheon/stop_stopping
Allow `podman stop` to be run on Stopping containers
2021-10-06 20:09:04 +02:00
Paul Holzinger fbce7584d7
libpod: fix race when closing STDIN
There is a race where `conn.Close()` was called before `conn.CloseWrite()`.
In this case `CloseWrite` will fail and an useless error is printed. To
fix this we move the the `CloseWrite()` call to the same goroutine to
remove the race. This ensures that `CloseWrite()` is called before
`Close()` and never afterwards.
Also fixed podman-remote run where the STDIN was never was closed.
This is causing flakes in CI testing.

[NO TESTS NEEDED]

Fixes #11856

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-06 17:51:07 +02:00
Matthew Heon 8bd9f58d1d Ensure podman ps --sync functions
The backend for `ps --sync` has been nonfunctional for a long
while now - probably since v2.0. It's questionable how useful the
flag is in modern Podman (the original case it was intended to
catch, Conmon gone via SIGKILL, should be handled now via pinging
the process with a signal to ensure it's still alive) but having
the ability to force a refresh of container state from the OCI
runtime is still useful.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2021-10-06 11:19:32 -04:00
Matthew Heon e1089e89d7 Allow podman stop to be run on Stopping containers
This allows you to stop a container after a `podman stop` process
started, but did not finish, stopping the container (probably an
ignored stop signal, with no time to SIGKILL?). This is a very
narrow case, but once you're in it the only way to recover is a
`podman rm -f` of the container or extensive manual remediation
(you'd have to kill the container yourself, manually, and then
force a `podman ps --all --sync` to update its status from the
OCI runtime).

[NO NEW TESTS NEEDED] I have no idea how to verify this one -
we need to test that it actually started *during* the other stop
command, and that's nontrivial.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2021-10-06 11:19:32 -04:00
OpenShift Merge Robot a866a2f159
Merge pull request #11763 from rhatdan/timeout
Add --time option for podman * rm -f flag
2021-10-04 15:10:36 -04:00