New functionality in hack/man-page-checker: start cross-
referencing the man page 'Synopsis' line against the
output of 'podman foo --help'. This is part 1, flag/option
consistency. Part 2 (arg consistency) is too big and will
have to wait for later.
flag/option consistency means: if 'podman foo --help'
includes the string '[flags]' in the Usage message,
make sure the man page includes '[*options*]' in its
Synopsis line, and vice-versa. This found several
inconsistencies, which I've fixed.
While doing this I realized that Cobra automatically
includes a 'Flags:' subsection in its --help output
for all subcommands that have defined flags. This
is great - it lets us cross-check against the
usage synopsis, and make sure that '[flags]' is
present or absent as needed, without fear of
human screwups. If a flag-less subcommand ever
gets extended with flags, but the developer forgets
to add '[flags]' and remove DisableFlagsInUseLine,
we now have a test that will catch that. (This,
too, caught two instances which I fixed).
I don't actually know if the new man-page-checker
functionality will work in CI: I vaguely recall that
it might run before 'make podman' does; and also
vaguely recall that some steps were taken to remedy
that.
Signed-off-by: Ed Santiago <santiago@redhat.com>
For using the `registry:2.6` image. 2.7 and beyond dropped the
`htpasswd` binary from the rootfs which parts of our CI depends
on.
While this is not a sustainable solution (assuming `htpasswd` is gone
for ever), it unblocks the CI for now.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
We experienced regression when using the latest `v1.2.0-dev` bats in
Ubuntu 20.04 (see github.com/containers/libpod/pull/6418). Using
bats v1.1.0 worked in the Ubuntu test VM.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
We need to default to building podman. If you specify no build
tags you will not build podman, not podman-remote.
Just using remote flag to indicate podman-remote and !remote for
podman.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
1) fix lost credentials.
must_pass(), added in #6375, eats the credentials
generated via 'podman run --entrypoint htpasswd'.
Run that podman instance directly, and add explicit
error check.
(The error and stdout/stderr handling here has gotten
cumbersome. There must be something I'm missing that
could make it all simpler.)
2) fix default podman path.
When setting $PODMAN, default to the locally built
one -- there may not be one in $PATH (e.g. in
Ubuntu, see #6366). This in turn requires us to:
3) run registry test in integration, not unit test
It looks like unit tests run before podman is built,
causing a chicken-egg dilemma. Try to solve that by
running the new hack/podman-registry-go test in
integration tests, not unit tests.
Signed-off-by: Ed Santiago <santiago@redhat.com>
My initial revision of the podman-registry helper script
was written in haste, with an enormous tradeoff: no
visibility into any errors. We are now paying for this
in #6366: the script is failing on Ubuntu and we
have no way of knowing why.
This PR adds a must_pass() function used for critical
steps. This runs the action silently; if the command
fails, it displays the failing command name with
full output logs, cleans up the temporary workdir,
and exits with error status.
As a reminder, the reason this is necessary is that
our script convention is to output a series of
environment variables to stdout -- we must therefore
take pains not to emit anything else to stdout.
And, unfortunately, podman and openssl tend to be
rather verbose.
Signed-off-by: Ed Santiago <santiago@redhat.com>
In response to #6207: this is a helper script intended for
use in starting and stopping a local container registry.
It takes care of port, username, password assignments;
generates a self-signed certificate; and starts the
container in an isolated podman root/runroot to avoid
conflicting with the caller's environment.
Intended usage: invoke from shell script, using 'eval'
to get results into calling process environment. See
help message (-h) for invocation details. This will
work for shell scripts but will be difficult if
called from Go or C - if that is likely to happen,
I'd love to hear suggestions for alternate ways to
get the settings back to the caller.
Signed-off-by: Ed Santiago <santiago@redhat.com>
This properly prints out image-name hints when executing the hack script
without any arguments. It is required due to changes made by Ed for
test-name beatification. An identical change was made and reviewed by
Ed in the containers/storage repo.
Signed-off-by: Chris Evich <cevich@redhat.com>
While this commit was initially meant to fix#5847, it has turned into a
bigger refactoring which I did not manage to break into smaller pieces:
* Fix#5847 by refactoring the image-removal logic.
* Make the api handler for image-removal use the ABI code. This way,
both (i.e., ABI and Tunnel) end up using the same code. Achieving
this code share required to move some code around to prevent circular
dependencies.
* Everything in pkg/api (excluding pkg/api/types) must now only be
accessed from code using `ABISupport`.
* Avoid imports from entities on handlers to prevent circular
dependencies.
* Move `podman system service` logic into `cmd` to prevent circular
dependencies - it depends on pkg/api.
* Also remove the build header from infra/abi files. It will otherwise
confuse swagger and other tools; errors we cannot fix as go doesn't
expose a build-tag env variable.
Fixes: #5847
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
New script cross-references r.Handle() and r.HandleFunc()
calls against the preceding '// swagger:operation' comments,
and exits failure (with descriptive error messages) if any
comments do not match the code.
This script should not be necessary: the swagger comments
should be autogenerated from the source code.
Signed-off-by: Ed Santiago <santiago@redhat.com>
* Added support for system service
* Enabled linting on the varlinkapi source, needed to support V2
service command
* Added support for PODMAN_SOCKET
Skip linting deprecated code
Rather than introduce bugs by correcting deprecated code, linting the
code is being skipped. Code that is being ported into V2 is being
checked.
Signed-off-by: Jhon Honce <jhonce@redhat.com>
./hack/podmanv2-retry will first invoke $PODMAN_V2 with given
arguments. If that fails with any of the following errors:
unrecognized command
unknown flag
unknown shorthand
...it will run $PODMAN_FALLBACK with the same arguments.
Output and exit code will be those of the final podman command,
although be aware that stderr and stdout are combined.
This is a quick-hack script intended for use in v2 testing, to
test implemented commands without noise from unimplemented ones.
Signed-off-by: Ed Santiago <santiago@redhat.com>
New hack/xref-helpmsgs-manpages script, added to CI 'gate'
task, runs 'podman [subcommand] --help' and cross-references
against man pages in docs/source/markdown/podman*.1.md
See #5453 and #5460 for instances of the problems the
script has found.
The careful reader will find an alarming number of special-case
bypasses. These are a tradeoff I am making: to get perfect
coverage with no handwaving, it would be necessary to make
drastic changes to some man pages, and I believe those would
be counterproductive.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add support to auto-update containers running in systemd units as
generated with `podman generate systemd --new`.
`podman auto-update` looks up containers with a specified
"io.containers.autoupdate" label (i.e., the auto-update policy).
If the label is present and set to "image", Podman reaches out to the
corresponding registry to check if the image has been updated. We
consider an image to be updated if the digest in the local storage is
different than the one of the remote image. If an image must be
updated, Podman pulls it down and restarts the container. Note that the
restarting sequence relies on systemd.
At container-creation time, Podman looks up the "PODMAN_SYSTEMD_UNIT"
environment variables and stores it verbatim in the container's label.
This variable is now set by all systemd units generated by
`podman-generate-systemd` and is set to `%n` (i.e., the name of systemd
unit starting the container). This data is then being used in the
auto-update sequence to instruct systemd (via DBUS) to restart the unit
and hence to restart the container.
Note that this implementation of auto-updates relies on systemd and
requires a fully-qualified image reference to be used to create the
container. This enforcement is necessary to know which image to
actually check and pull. If we used an image ID, we would not know
which image to check/pull anymore.
Fixes: #3575
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
A number of scripts relating to tooling used and the gate container
image were not exiting upon errors as intended. Coupled with
external service unavailability (i.e. downloading golangci-lint)
was observed to cause difficult to debug failures.
This change corrects the scripts inside/out of the gate container as
well as fixes many golang related path consistency problems vs other CI
jobs. After this change, all jobs use consistent path names reducing
the number of special-case overrides needed.
Lastly, I also made a documentation-pass, updating/correcting as needed,
including documenting a likely local validation-failure mode, related to
`$EPOCH_TEST_COMMIT`. This is dependent on the developers git
environment, so documentation is the only possible "fix".
Signed-off-by: Chris Evich <cevich@redhat.com>
in cases where the log file exceeds the available memory of a system, we had a bug that triggered an oom because the entire logfile was being read when the tail parameter was given. this reads in chunks and is more or less memory safe.
fixes: #5131
Signed-off-by: Brent Baude <bbaude@redhat.com>
We removed the Gitvalidation epoch in the Makefile. As such, we
don't need to adjust it anymore when we tag releases.
Signed-off-by: Matthew Heon <mheon@redhat.com>
Instead of only performing a presence check of the binary, also do a
version check and force installing the specified one if needed. This
will prevent users and the CI from using a wrong version in the future.
Move the logic into a dedicated shell script as I find built-in bash in
Makefiles hard to maintain.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Finding systemd devel packages using libsystemd does not work as
in RHEL based distro the package name is systemd-devel and for
deb/ubunutu it is libsystemd. It is also giving false result when
podman rpm is built with systemd but hack/systemd_tag.sh does not
return anything.
Install systemd-devel package in build_rpm.sh script
Moving to systemd/sd-daemon.h header files which comes from devel
packages fixes the issue.
Signed-off-by: Chandan Kumar (raukadah) <raukadah@gmail.com>
- Adopt bash strict mode
- Avoid cd errors as seen on CI vendor jobs:
hack/get_release_info.sh: line 9: cd: /go/src/github.com/containers/libpod: No such file or directory
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
* Refactored code and Makefile to support new docs layout
* Removed some old code packaging code
* Add Readme.md to document what we're doing
Signed-off-by: Jhon Honce <jhonce@redhat.com>
Signed-off-by: baude <bbaude@redhat.com>
Restructuring the docs dir to make integration with sphinx easier. man
pages now exist in docs/source/man and the sphinx make files exists in
docs.
Signed-off-by: baude <bbaude@redhat.com>
Instead of unconditionally pulling the x86 binary, clone the repository
and build the binary to make it independent of the architecture.
Fixes: #2699
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Make the errors more readable, with clearer instructions on
what to look for, and which filename, and what we expect to
see, and perhaps even how to approach a fix.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The initial implementation was far more complicated than necessary.
Strip out the complexities in favor of a simpler and more direct
approach.
Signed-off-by: Chris Evich <cevich@redhat.com>
* progress bar: use spinners for unknown blob sizes
* use 'containers_image_ostree' as build tag
* ostree: default is no OStree support
* Add "Env" to ImageInspectInfo
* config.go: improve debug message
* config.go: log where credentials come from
* Fix typo in docs/containers-registries.conf.5.md
* docker: delete: support all MIME types
* Try harder in storageImageDestination.TryReusingBlob
* docker: allow deleting OCI images
* ostree: improve error message
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Remove disused `build_cache_images` task, and
update relevant dockerfiles for F30.
Fix problem of cloud-init failing to expand root-device on boot
(/var/lib/cloud/instance left in improper state).
Fix problem of cloud-init racing with google-network-daemon.service on
boot (looking for cloudconfig metadata too early). Causing
root-device to _sometimes_ fail to expand.
Fix problem of hack/get_ci_vm.sh argument passing.
Signed-off-by: Chris Evich <cevich@redhat.com>
If the systemd development files are not present on the system which
builds podman, then `podman events` will error on runtime creation.
Beside this, a warning will be printed when compiling podman.
This commit mainly exists because projects which depend on libpod
would not need the podman event support and therefore do not need to
rely on the systemd headers.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
The output of this CI script leaves much to be desired: it is
output from 'diff' with little clarity on what exactly is wrong.
The proper fix is to make the output clear and readable:
podman containers --help lists a 'foo' subcommand that
is not present in docs/podman-containers.1.md
Doing this in bash would take many hours and be fragile
gibberish code. This does not seem worth the effort: the
likely case is that breakages reported by this script
will be due to a newly added subcommand, and the PR
author will find it obvious what to do. Ergo, plan B:
if the test fails, display a blurb at the end describing
how to interpret results. Three minutes' effort, plus
five for writing this commit message.
Signed-off-by: Ed Santiago <santiago@redhat.com>
