apiv2 tests emit TAP-compliant output; recognize it and
highlight it the same way we do BATS tests.
Add anchor links to TAP output, so other tools (e.g.
cirrus-flake-summarize) can link to particular lines
And, remove a "-f" from "wait" in test-apiv2; looks
like there's some version of bash used in some CI VM
that doesn't grok it.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Without the systemd build tag, podman will fail with the error "No
support for journald logging".
This commit adds the `systemd` build tag explicitly, rather than
relying on `hack/systemd_tag.sh` (because we're building an rpm and
we've explicitly included systemd-devel as a dependency).
Signed-off-by: Lars Kellogg-Stedman <lars@redhat.com>
Many of the packages required for CI in buildah overlap with libpod.
When building new VM images, attempt to source a package list
from the buildah repository. If found, also install the listed
packages on the VM.
Signed-off-by: Chris Evich <cevich@redhat.com>
this is a temporary fix for the flake that has been troubling us. once conmon is in fedora 30 and 31 stable, we can remove this fix. the images will just need to be rebuilt.
Signed-off-by: Brent Baude <bbaude@redhat.com>
Add support to auto-update containers running in systemd units as
generated with `podman generate systemd --new`.
`podman auto-update` looks up containers with a specified
"io.containers.autoupdate" label (i.e., the auto-update policy).
If the label is present and set to "image", Podman reaches out to the
corresponding registry to check if the image has been updated. We
consider an image to be updated if the digest in the local storage is
different than the one of the remote image. If an image must be
updated, Podman pulls it down and restarts the container. Note that the
restarting sequence relies on systemd.
At container-creation time, Podman looks up the "PODMAN_SYSTEMD_UNIT"
environment variables and stores it verbatim in the container's label.
This variable is now set by all systemd units generated by
`podman-generate-systemd` and is set to `%n` (i.e., the name of systemd
unit starting the container). This data is then being used in the
auto-update sequence to instruct systemd (via DBUS) to restart the unit
and hence to restart the container.
Note that this implementation of auto-updates relies on systemd and
requires a fully-qualified image reference to be used to create the
container. This enforcement is necessary to know which image to
actually check and pull. If we used an image ID, we would not know
which image to check/pull anymore.
Fixes: #3575
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
1) 'podman system info' (in logcollector): has been silently
failing in special_testing_rootless, with:
logcollector.sh: line 16: podman: command not found
Use ./bin/podman instead of just podman; this is probably
the right thing to do in the general case anyway
2) logformatter: highlight 'panic:', seen in bindings test:
https://storage.googleapis.com/cirrus-ci-5385732420009984-fcae48/artifacts/containers/libpod/6693715108429824/html/integration_test.log.html
3) logformatter: handle Unicode bullet in front of 'Running',
seen in bindings test.
4) logformatter: turn down contrast on BATS 'ok' results,
for legibility
Signed-off-by: Ed Santiago <santiago@redhat.com>
Detecting when it's time to upload a release inside Cirrus-CI is really
difficult for many automation and human reasons. Disabling it for now
until a more robust solution can be implemented
Signed-off-by: Chris Evich <cevich@redhat.com>
the binding ginkgo tests were using color mode which throws in a bunch of ansi garbage that makes it hard to read the logs
Signed-off-by: Brent Baude <bbaude@redhat.com>
during container creation, if no network is provided, we need to add a default value so the container can be later started.
use apiv2 container creation for RunTopContainer instead of an exec to the system podman. RunTopContainer now also returns the container id and an error.
added a libpod commit endpoint.
also, changed the use of the connections and bindings slightly to make it more convenient to write tests.
Fixes: 5366
Signed-off-by: Brent Baude <bbaude@redhat.com>
The podmanimage/upstream/Dockerfile had two rpms in its
build procedure that are no longer available. The atomic-registries
has been removed and the md2man has been renamed. In addtion
conmon was not being installed and I've added that.
I've been using a Containerfile to build or rebuild a
specific version of the podmanimage stored in the stable
repository with a version tag. As the other Containerfiles
have been updated by others, and in case anyone else needs
to build it, I've added it to the repo and have also updated
the readme.md.
FWIW, the builds in the quay.io/podman/upstream have been failing for a while due to missing rpms.
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Builds of this dockerfile fail on quay.io due to not being able to pull
the base image. Use a fully-qualified FROM name to work around this.
Signed-off-by: Chris Evich <cevich@redhat.com>
This introduces a new cirrus helper script, logformatter.
Usage is:
[commands...] | logformatter TEST-NAME
It reformats its input into a readable, highlighed, linkable
form. Some features:
- boring stuff (timestamps, standard podman options) is
deemphasized
- important stuff (warnings, errors) is emphasized
- in-page links to the actual failures
- active links to source files
- jumps to bottom of page on load, because that's where
the errors are. (All errors are linked)
Add it to select test commands (integration, system) and
add a new artifacts_html, run in the 'always' block, which
uploads generated *.log.html into Cirrus; from there we
generate a live URL that can be viewed in browser.
Unfortunately, due to security concerns in Cirrus, it is
not currently possible to make the link a live one.
Kludge: add a line of dashes after Restoring images; without this,
the first test ("systemd PID 1") has no dashes before it, so
logformatter doesn't see it.
Signed-off-by: Ed Santiago <santiago@redhat.com>
API v2 has been quiet for a few days, and the test script is
actually passing. Let's take advantage of this opportunity
to get them running in CI.
Requires adding a check for cgroupsv2
Signed-off-by: Ed Santiago <santiago@redhat.com>
A number of scripts relating to tooling used and the gate container
image were not exiting upon errors as intended. Coupled with
external service unavailability (i.e. downloading golangci-lint)
was observed to cause difficult to debug failures.
This change corrects the scripts inside/out of the gate container as
well as fixes many golang related path consistency problems vs other CI
jobs. After this change, all jobs use consistent path names reducing
the number of special-case overrides needed.
Lastly, I also made a documentation-pass, updating/correcting as needed,
including documenting a likely local validation-failure mode, related to
`$EPOCH_TEST_COMMIT`. This is dependent on the developers git
environment, so documentation is the only possible "fix".
Signed-off-by: Chris Evich <cevich@redhat.com>
Suspect crun might be sneaking in during VM image build via podman RPM
dependency. Add it to the removal list when building, then also force
use of runc at runtime in F30.
Also quote all true/false vars to force them as strings instead of
booleans (which will become capitalized)
Signed-off-by: Chris Evich <cevich@redhat.com>
There are a number of env. vars set during the setup script. Therefore
displaying them at end of the script is more helpful for debugging.
Signed-off-by: Chris Evich <cevich@redhat.com>
In the package_versions CI step, include Fedora/Ubuntu
version, uname -r, and cgroups version.
Cgroups version is simply the FS type of /sys/fs/cgroup,
which shows 'tmpfs' for v1 and 'cgroup2fs' for v2. I
don't think it's worth the effort to prettify those
into 'v1/v2' - I think our readers are sophisticated
enough to figure it out from context - but am willing
to add that feature if requested.
Signed-off-by: Ed Santiago <santiago@redhat.com>
In some distributions it's possible to have both runc and crun
installed and/or for podman to be confused about which to use. In these
instances, force the decision by adding `OCI_RUNTIME=/usr/bin/crun` into
`/etc/environment`. Also in-place modify libpod.conf to use 'crun'
instead of 'runc'
Signed-off-by: Chris Evich <cevich@redhat.com>
golang-github-cpuguy83-go-md2man is only available in Fedora 30 and
got renamed to golang-github-cpuguy83-md2man for Fedora 31 which breaks
the user interface for building rpm on fedora 30.
It fixes the same by installing correct md2man package on Fedora 30.
Signed-off-by: Chandan Kumar (raukadah) <raukadah@gmail.com>
The package golang-github-cpuguy83-go-md2man has been renamed into golang-github-cpuguy83-md2man
in f31 repository.
That leads to an Error: Unable to find a match: golang-github-cpuguy83-go-md2man
This patch handles the renaming of this package and fixes the command for f31 and the one that will
follows without breaking compatibility with older versions.
Signed-off-by: Allan Jacquet-Cretides <allan.jacquet@gmail.com>
As the title says. I renamed the old file from the lower case to the
upper case name. This makes it appear higher up in the listing on GitHub
and also is in line with the rest of the containers projects. Due to this
change, I also had to change a few references in a couple of build related
files.
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
the paths and instructions for running the new api via systemd needed updates due to a change in the command.
Signed-off-by: Brent Baude <bbaude@redhat.com>
This is needed to provide this image under quay.io/libpod/ namespace
to provide some resiliency to automated testing (should other
repositories be unavailable)
Signed-off-by: Chris Evich <cevich@redhat.com>
The release upload process always involves two filenames, however the
second filename might (someday) be optional. The code allowed for this,
however input validation did not. This change fixes the validation.
Signed-off-by: Chris Evich <cevich@redhat.com>
This should help use keep the codebase more consistent, and avoid sevel
whitespace related issues, or bad file permissions.
pre-commit allows us to easily introduce other linters in follow-ups,
like bashate.
Note: pre-commit tool does *not* install any git-hooks. Making commits
will will call the tool unless you deliverately tell it to install the
hooks.
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
Perftest was intended to be used for testing CPU intensive tasks of
Podman. However, it does not compile for a long while and is not
integrated in the CI which clearly indicates that it has not been
used for a considerable amount of time.
Remove contrib/perftest entirely. If the desire arises to revive it,
all code is still reachable in the git history.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
VM Base images are used as a starting point for runtime VM images.
The in-use VM base images should never be pruned, which is an
operation that potentially occurs periodically from automation
running on the master branch of the libpod repo.
However the only place which updates timestamps (blocking pruning)
of base images, occurs during runtime VM image building. Therefor,
if images are not regularly rebuilt, it's possible their base images
go stale and are pruned.
Changes:
* Add freshly-produced base images (old ones got pruned)
* Wrap the timestamp update script to include base image names
in the update list.
Notes:
* Regularly updating base image timestamps only needs to happen
on the libpod repo's meta task, since all base images live there.
* Using a wrapper is needed to maintain compatibility with multiple
versions of the imgts container image used by other repos / branchs.
Signed-off-by: Chris Evich <cevich@redhat.com>
(Stealing from: @rhatdan 's https://github.com/containers/buildah/pull/2038 )
1 We need to update all packages in the podman image to make sure they are
up2date.
2 reinstall shadow-utils. For some reason the fedora base image does not
include the file capabilities assigned to /usr/bin/newuidmap and
/usr/bin/newgidmap. Reinstalling shadow-utils, brings them back.
3 Add a default user build to the system. This will create the
/etc/subuid and /etc/subgid maps get created correctly.
Once we have this we should be able to build a container starting with a non
privileged user
podman run -ti --user build --device=/dev/fuse -v ./Dockerfile:/Dockerfile:z quay.io/podman/stable podman buildd /
Addresses: #4741
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Previously we builded RPMs that contained an outdated conmon which was
not compatible. From now on `make-install` will also call
`podman version` and `podman info` in order to perform a minimal
sanity check of the installation.
Fixes: #4665
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
Changes I am making:
1. The target `.gopathok` was listed in `.PHONY` which
looks wrong as it regenerates `.gopathok` every time we
re-run it, which was a part of the issue. I removed it
to avoid that. If `.gopathok` is present', makefile
should not need to rerun it.
2. Ensure the binaries are created only if they don't
exist by adding `bin/podman`
and `bin/podman-remote`.
3. Add a `SOURCES = $(shell find . -name "*.go")` and put
it as a dependency of the podman binaries target. It allows us
to re-generate the binaries only when there is a change in the
source files. The downside is it increases the running time of
the command that generates them (20 seconds on my virtual machine
running Centos 7). If this is a problem, we could introduce a
hidden file that would list all the files to track, that
would need to be updated only when a dev is introducing new files.
4. Fixed the make package-install as it does not work with yum.
I updated the build_rpm.sh to ensure it works on centos 7
and centos 8 with no pre-required installation.
Closes#4367
Signed-off-by: Neville Cain <neville.cain@qonto.eu>
Finding systemd devel packages using libsystemd does not work as
in RHEL based distro the package name is systemd-devel and for
deb/ubunutu it is libsystemd. It is also giving false result when
podman rpm is built with systemd but hack/systemd_tag.sh does not
return anything.
Install systemd-devel package in build_rpm.sh script
Moving to systemd/sd-daemon.h header files which comes from devel
packages fixes the issue.
Signed-off-by: Chandan Kumar (raukadah) <raukadah@gmail.com>
While building the spec file on fedora, in rpmbuild log,
.sh: No such file or directory error is shown as full path of
hack directory is not resolved leading to file not found error.
Appending the builddir and libpod path with hack will fix the issue.
Signed-off-by: Chandan Kumar (raukadah) <raukadah@gmail.com>
- make: fix python detection for multiple interpreters
- make: create generic `package` and `package-install` targets
- build_rpm.sh: move package installation into package-install
- build_rpm.sh: fix dnf/yum detection
- build_rpm.sh: install md2man rpm only on platfroms where is available
- build_rpm.sh: temporary skip packaging docs and debug on rhel-8
- docs: `make package-install`
This change is validated by new CI jobs run by rdoproject. See link
below for result.
Depends-On: https://review.rdoproject.org/r/#/c/23943/
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
- Fix bug failing to detect dnf in various locations
- Add missing sudo to yum/dnf calls
- Fix bug where pkg_manager may endup with a multi-line value
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
It changes the podman spec from using with_doc to bcond_without
for building docs so that anyone can pass --without doc as well
as other rpmbuild args to the build_rpm.sh script to skip
building docs.
Since go-md2man is not available in CentOS8 repo. without the
help fo extra_args, it makes it conditional.
It also moves the manpages to a seperate package.
Signed-off-by: Chandan Kumar (raukadah) <raukadah@gmail.com>
since go-man2md is not available in CentOS 8, making it
optional allows them to build the rpm.
Signed-off-by: Chandan Kumar (raukadah) <raukadah@gmail.com>
Since btrfs-progs-devel is not available in RHEL/CentOS 8 and
the spec fails to build it while running build_rpm.sh,
making it optional fixes the issue.
It also modifies the spec file to install btrfs-progs-devel for
fedora only.
Since golang-github-cpuguy83-go-md2man was added twice, it also
removes the repetition.
Signed-off-by: Chandan Kumar (raukadah) <raukadah@gmail.com>
Cirrus-CI only sets `$CIRRUS_BASE_SHA` when testing PRs since the
destination and it's state is easy to discover. However, when
post-merge and/or tag-push testing, the previous state is not
easily discoverable (changes have already merged). The
`upload_release_archive` script incorrectly assumed this
variable was always set, causing a constant stream of post-merge
testing failures.
Tweak the `is_release()` function to properly handle an empty
`$CIRRUS_BASE_SHA` whether or not `$CIRRUS_TAG` is also set. Also
update the unit-tests to check for this. Also account for a
corner case where hack/get_ci_vm.sh is running on a VM w/o git.
Signed-off-by: Chris Evich <cevich@redhat.com>
Move to containers/image v5 and containers/buildah to v1.11.4.
Replace an equality check with a type assertion when checking for a
docker.ErrUnauthorizedForCredentials in `podman login`.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Prior to this commit, every push to master had it's builds packaged and
uploaded to google storage. This is a waste, since potential users
are only ever concerned about tagged releases.
Unfortunately because the release process involves humans with
potentially multiple human and automation steps happening in parallel,
it's easy for automation to not detect a tagged release, or trigger on
development|pre-release tags.
Fix this in `upload_release_archive.sh` using a new unit-tested
function `is_release()`. This acts as the definitive authority
on whether or not a specific commit rage or `$CIRRUS_TAG` value
constitutes something worthy of upload.
Signed-off-by: Chris Evich <cevich@redhat.com>
When executing 'make remotesystem' testing, a varlink process is started
up but it's stdio is dumped due to the production of excessive data.
However, this also means if the process has a problem, any errors will
not be accessible.
Instead, grab only the last 100 lines and direct them into a file. Also
update automation's log collection to retrieve this file when the
`$REMOTE_CLIENT` env. var. is `true`.
Signed-off-by: Chris Evich <cevich@redhat.com>
Using `Also=` means that the target unit will also be
installed/uninstalled together with our unit. Doing
`Also=multi-user.target` essentially says: disable `multi-user.target`
if `io.podman.socket` is disabled, which sounds... not at all like
what we want.
In practice, systemd thankfully ignores this (likely because it's the
default target). I think having `Also=io.podman.socket` in the
`io.podman.service` already does what we want here: it gets installed
under `sockets.target` whenever the service is. (And the fact that
systemd ignored this means that it wasn't actually playing a role in
resolving #3998.)
This was causing `systemctl preset-all` to dump core in Fedora CoreOS:
https://github.com/coreos/fedora-coreos-tracker/issues/290
(Likely there's a systemd bug around here too.)
Signed-off-by: Jonathan Lebon <jonathan@jlebon.com>
This is needed because older versions of podman (1.5.1) do not
automatically install the new conmon package.
Also, include removal of `/usr/libexec/podman/conmon` when preparing to
install and test podman built from source.
Signed-off-by: Chris Evich <cevich@redhat.com>
This requires updating all import paths throughout, and a matching
buildah update to interoperate.
I can't figure out the reason for go.mod tracking
github.com/containers/image v3.0.2+incompatible // indirect
((go mod graph) lists it as a direct dependency of libpod, but
(go list -json -m all) lists it as an indirect dependency),
but at least looking at the vendor subdirectory, it doesn't seem
to be actually used in the built binaries.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Adds an output to all testing tasks which lists the names/versions of
critical/essential packages present on the VM.
Signed-off-by: Chris Evich <cevich@redhat.com>
A true result from reexec.Init() isn't an error, but it indicates that
main() should exit with a success exit status.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
This is fairly standard thing to have on a user's workstation, supported
by podman. When installed in a VM image, then it's useful for debugging
with `hack/get_ci_vm.sh` at the cost of a minor increase in disk-space.
Signed-off-by: Chris Evich <cevich@redhat.com>
Instead of unconditionally pulling the x86 binary, clone the repository
and build the binary to make it independent of the architecture.
Fixes: #2699
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Add the latest Ubuntu version into the testing matrix
and image-build workflow. This is also needed to support
other containers projects which share use of VM images
from this one.
Update package lists to include needs for contianers/storage
use of images.
Signed-off-by: Chris Evich <cevich@redhat.com>
* Fix one disused and two missing required env. vars.
* Slightly optomize processing of commit-author names
* Fix problem of printing duplicate author names when there are multiple
commits.
* Fix bot's IRC connection timeout too short.
* Add a single retry of IRC connection after 5-second delay.
Signed-off-by: Chris Evich <cevich@redhat.com>
Fixed a typo.
Also script was grabbing quotes and other non-email-address junk
while looping. Filter before and after to make sure we get 'em all.
Signed-off-by: Chris Evich <cevich@redhat.com>
* Update scipts to produce darwin and windows output
* Update batch file to re-direct help requests to browser
* Add pandoc filter for markdown to html links
Signed-off-by: Jhon Honce <jhonce@redhat.com>
When enabled, it's desired for the podman-varlink process to startup on
boot or upon socket-activation, whichever happens first. However,
with `KillMode=none` systemd will never kill any podman-varlink
processes. This makes it easily possible for multiple podman-varlink
processes to be running, and fight each other to service a single socket.
---
For example:
Prior to this commit, this will result in four podman-varlink processes
being run:
```
systemctl enable io.podman.socket
systemctl enable io.podman.service
systemctl start io.podman.socket
systemctl start io.podman.service
systemctl start io.podman.service
```
Fix this by setting `KillMode=process` and `TimeoutStopSec=30` (default
is 90). This results in podman-varlink exiting on its own after a minute
of being idle (--timeout=60000). Alternatively, systemd will manage the
service stop by sending a SIGTERM, then if podman-varlink has not exited
within `TimeoutStopSec`, a SIGKILL will be sent.
Signed-off-by: Chris Evich <cevich@redhat.com>
* Update Makefile to build msi
* Add podman.wxs to define podman.msi
* Version information provided by Makefile
* Add podman.bat wrapper for podman-remote-windows.exe to ensure environment
* Add wix xml schemas for reference
Signed-off-by: Jhon Honce <jhonce@redhat.com>
When constructing VM cache-images, the latest/greatest podman package is
installed to ensure all necessary dependencies are met. Prior to
testing source-built binaries, most of of the packaged files are removed.
However, if the `io.podman` service or socket is enabled/running, it
could cause the packaged podman and varlink binaries to be both resident
and cached. Since this condition would cause very difficult to diagnose
behaviors, add preventative measures to ensure these services are absent
prior to removing packaged podman files.
Signed-off-by: Chris Evich <cevich@redhat.com>
This is mostly used with Systemd, which really wants to manage
CGroups itself when managing containers via unit file.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
In the Dockerfiles that are used to build the podman images on
quay.io, we were changing the events_logger from journald to
file in libpod.conf, but we weren't enabling it as we didn't
remove the comment. This corrects that and addresses: #3464
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Also add fixes to help prevent 'fatal: Invalid revision range' error.
Should obtaining all authors from the range still fail, only grab the
HEAD commit author as a fallback.
Signed-off-by: Chris Evich <cevich@redhat.com>
Rather than spamming the podman channel with impersonal success
messages referring to PR numbers, mention the author by nick name
and include the PR title and link.
Also avoid needless logging of all bot-script interactions with
IRC when there is no error detected.
Signed-off-by: Chris Evich <cevich@redhat.com>
Rather than hard-coding all four base-image env. var name,
load the values based on the shared variable name suffix.
Thanks to Ed Santiago <santiago@redhat.com> for the suggestion.
Signed-off-by: Chris Evich <cevich@redhat.com>
The initial implementation was far more complicated than necessary.
Strip out the complexities in favor of a simpler and more direct
approach.
Signed-off-by: Chris Evich <cevich@redhat.com>
Specifically pertaining to executing tests in google cloud, there are
default, pre-allocated class-a subnetworks for each region (data-center).
Each includes a gateway using a `.1` LSB and all are routable from other
regions in google cloud via these gateways.
Because the default CNI configuration also utilizes class-a subnetworks,
this creates the possibility for IPv4 address-space clashes. Since the
default regional cloud subnets are pre-defined/known, preventing clashes
can be accomplished by seeding these subnets in a dummy CNI configuration.
The default behavior of podman is to grab the highest priority CNI
configuration. Name the dummy config. appropriate so it always loads
last. Also name the bridge itself with an obvious name `do-not-use`,
such that any related testing errors should be easier to debug.
Also:
* Minor cleanup of `install_test_configs()`
* Move install_test_configs in `setup_environment.sh` to after possible
run of `remove_packaged_podman_files()` because that also strips out
`/etc/cni/net.d/87-podman-bridge.conflist`.
Signed-off-by: Chris Evich <cevich@redhat.com>
This enables user to interact with varlink and create/manage rootless
containers through it.
Using as:
`varlink call unix:/run/user/1000/podman/io.podman/io.podman.ListContainers`
Signed-off-by: Matej Marusak <mmarusak@redhat.com>
Requirement from https://github.com/containers/libpod/issues/3575#issuecomment-512238393
Added --pull for podman create and pull to match the newly added flag in docker CLI.
`missing`: default value, podman will pull the image if it does not exist in the local.
`always`: podman will always pull the image.
`never`: podman will never pull the image.
Signed-off-by: Qi Wang <qiwan@redhat.com>
as issue #2702 describes, we want to make podman and podman-remote as
part of make install.
Fixes: #2702
Signed-off-by: baude <bbaude@redhat.com>
avoid `make` in `make install` in the rpmbuild process.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
This is mainly/initially to support use of Cirrus-CI
in https://github.com/containers/buildah since that setup
re-uses the VM images from this project. However, it also
opens doors here, if libpod ever needs/wants to do things
with a dedicated storage device and/or storage-drivers.
Signed-off-by: Chris Evich <cevich@redhat.com>
The release-task ***must*** always execute last, in order to guarantee a
consistent cache of release archives from dependent tasks. It
accomplishes this by verifying it's task-number matches one-less than
the total number of tasks. Previous to this commit, a YAML anchor/alias
was used to avoid duplication of the dependency list between 'success'
and 'release'
However, it's been observed that this opens the possibility for
'release' and 'success' tasks to race when running on a PR. Because
YAML anchor/aliases cannot be used to modify lists, duplication is
required to make 'release' actually depend upon 'success'.
This duplication will introduce an additional maintenance burden.
Though when adding a new task, it's already very easy to forget to
update the 'depends_on' list. Assist both cases by the addition
unit-tests to verify ``.cirrus.yml`` dependency contents and structure.
Signed-off-by: Chris Evich <cevich@redhat.com>
This task depends upon other tasks caching their binaries. If for
whatever reason the `release` task is re-run and/or is out-of-order
with it's dependents, the state of cache will be undefined. Previously
this would result in an error, and failing of the release task.
This commit alters this behavior to issue a warning instead.
Signed-off-by: Chris Evich <cevich@redhat.com>
It's somewhat hard to predict which units are certinly present
for any given base-image. Therefore, at image-build time, it's
distracting and unhelpful to see all the errors about units that
don't exist, on every platform. Simply ignore them and rely on
the `check_image.sh` test to confirm none are enabled.
Signed-off-by: Chris Evich <cevich@redhat.com>
For CI testing, it's important to remove as much variability from the
overall system as possible. This permits focusing just on problems
closely related to code-changes. To this end, and because VMs are very
short-lived (2 hours at most), disable all systemd services and timers
which perform periodic activities.
Signed-off-by: Chris Evich <cevich@redhat.com>
Various tasks and scripts behave differently depending on whether or not
the build is running against a PR or on a branch, post-merge. However,
a great number of them are hard-coded to the string 'master' as the
destination. Since this is not always the case (there are other
relevant branches), it makes sense to abstract the references with a
single definition.
Add a top-level `$DEST_BRANCH` variable to CI, and otherwise
default to 'master' when unset. This enables running CI builds on
additional branches without the overhead of updating all the static
references to 'master'. Simply update `$DEST_BRANCH` at the top-level
and all branch-conditional logic will function as intended.
Signed-off-by: Chris Evich <cevich@redhat.com>
Over time unless they're removed, the project could grow quite a large
collection of VM images. While generally cheap (less than a penny each,
per month), these will become a significant cost item if not kept
in-check.
Add a specialized container for handling image-pruning, but limit
it to only finding and printing (not actually deleting) images.
Also update the image-building workflow so that base-images used to
compose cache-images are also labeled with metadata.
N/B: As an additional safeguard, the service account which
executes the new container in production *DOES NOT*
have access to delete images. This can be enabled
by adding the GCE IAM role: CustomComputeImagePrune
Signed-off-by: Chris Evich <cevich@redhat.com>
It's desirable to make archives available of builds containing actual
tested content. While not official distro-releases, these will enable
third-party testing, experimentation, and development for both branches
(e.g. "master") and pull requests (e.g. "pr3106").
* Add a Makefile targets for archiving both regular podman binaries
and the remote-client. Encode release metadata within these
archives so that their exact source can be identified.
* Fix bug with cross-compiling remote clients for the Windows and Darwin
platforms.
* Add unit-testing of cross-compiles for Windows and Darwin platforms.
* A few small CI-script typo-fixes
* Add a script which operates in two modes:
1. Call Makefile targets which produce release archives.
Upload the archive to Cirrus-CI's built-in caching system
using reproducible cache keys.
2. Utilize reproduced cache keys to attempt download of cache
from each tasks. When successful, parse the file's
release metadata, using it to name the archive file. Upload
all recovered archives to a publicly accessible storage bucket
for future reference.
* Update the main testing task to call the script in mode #1 for
all primary platforms.
* Add a new `$SPECIALMODE` task to call the script in mode #1 for
Windows and Darwin targets.
* Add a new 'release' task to the CI system, dependent upon all other
tasks. This new tasks executes the script in mode #2.
* Update CI documentation
Signed-off-by: Chris Evich <cevich@redhat.com>
Building/installing dependencies from fixed source-version ensures
testing is reliable, but introduces a maintenance burden and
risks testing far outside of a real-world environment. The
sensible alternative is to install dependencies from distro-packaging
systems.
Install all development and testing dependencies at VM cache-image build
time, to help ensure testing remains stable. The existing cache-image
build workflow can be utilized at any future time to build/test
with updated packages.
***N/B***: This does not update any dockerfiles used by testing, that is
left up to future efforts.
Signed-off-by: Chris Evich <cevich@redhat.com>
I'm running the BATS tests manually once in a while, and
catching several problems each week that make it past
the rest of CI. Since the BATS tests run at RPM gating
time, we need to catch problems earlier. Try running
the tests from Cirrus.
Tests will be skipped on Ubuntu due to a too-ancient
version of coreutils (8.28; the 'timeout -v' we use
requires 8.29).
Tests are run *after* integration tests, even though
these take three minutes and would be nice to have
fail quickly, because running before causes bizarre
CI failures. Shrug.
UPDATE: also fix run test, broken by #3311.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The original solution using --wait does not function on F30, waiting
forever. Replace it with a simple 5-minute timeout loop.
Signed-off-by: Chris Evich <cevich@redhat.com>
Remove disused `build_cache_images` task, and
update relevant dockerfiles for F30.
Fix problem of cloud-init failing to expand root-device on boot
(/var/lib/cloud/instance left in improper state).
Fix problem of cloud-init racing with google-network-daemon.service on
boot (looking for cloudconfig metadata too early). Causing
root-device to _sometimes_ fail to expand.
Fix problem of hack/get_ci_vm.sh argument passing.
Signed-off-by: Chris Evich <cevich@redhat.com>
The upstream/Dockerfile had a few issues that this
cleans up. A few files were misplaced, wrong installs
and removes. This corrects those issues.
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
With multiple `containers` projects updating VM Image metadata,
it would be very difficult to discover which Cirrus-CI setup
was responsible. Add the GCE project name to the list
of metadata labels to update when this container runs. This
will give more context as to which images are currently in use.
Signed-off-by: Chris Evich <cevich@redhat.com>
...with the goal of (very soon) reusing this code, in #2947,
to run system tests in CI. This is the cleanest way I can
think of to do so without duplication or a large maintenance
burden.
Changes are:
- replace references to 'ginkgo' with 'integration'. That
target is already in Makefile, and is not only more
readable, it's also more abstract. There is no reason
for this level of code to know about ginkgo.
- allow rootless_test.sh to accept an argument,
that being the name of the test suite to run
(default: integration). #2947 will enable 'system'.
- allow integration_test.sh to serve multiple purposes,
by checking its filename. #2947 will add a symlink,
system_test.sh, which will then cascade down to
invoke system tests.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The Dockerfiles necessary to create the stable, testing and upstream container images
on quay.io/user/podman. Once this is commited, I will set up those images
such that they will be built with every git commit.
stable - Latest Fedora release image
testing - Latest release on bohdi Fedora testing
upstream - Latest version in upstream podman
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
