Commit graph

1404 commits

Author SHA1 Message Date
Miloslav Trmač 754fc8e8ec Call imageNameForSaveDestination while creating the references
Instead of creating a reference string and then checking it again
to see which kind of archive it is, just call
imageNameForSaveDestination at the place where we already know
what kind of archive it is because we are making that decision.

This also notably fixes the use of strings.CONTAINS to see
whether the just constructed strings start with one of the
transport names; that would match anywhere in the
path.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:59 +00:00
Miloslav Trmač 18c4e2c835 Exit early in the simple case in imageNameForSaveDestination
... to make it a tiny bit easier to read.

Should not change behavior (but does not add unit tests).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:59 +00:00
Miloslav Trmač b142cc6b4e Rename parameters of imageNameForSaveDestination
... to make their relationship clear, at the very least.

Should not change behavior (but does not add unit tests).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:59 +00:00
Miloslav Trmač c27c6c6707 Split imageNameForSaveDestination from saveCmd
We will need to call it from two places in the future.

Should not change behavior, the code is pretty unchanged
(down to using confusing parameter names, which we will change
immediately) (but does not add unit tests).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač a4b15548d1 Split a single if statement into two.
This should not change behavior; it will only make it
easier to show that future code move does not change it (but
does not add unit tets.)

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač 3cebdc68b8 Move source handling before destination parsing
This will allow adding the reference in the OCIArchive/DockerArchive case
in one step, instead of appending it later.

Should not change behavior, except that source-related errors
will now be reported before possible destination-related errors
(but does not add unit tests).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač 891392339f Split Image.PushImageToReference from Image.PushImage
This retains the existing string parsing heuristic for users
who must continue to use it (notably the varlink API - or is
it still subject to change?), but allows callers who can get
precise references to supply them without having to deal
with string formatting.

Should not change behavior (but does not add unit tests).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač 1153486ab0 Don't format to string and re-parse a DockerReference()
We already have a c/image/docker/reference.Named; no need to
round-trip it through a string.  This also eliminates the theoretical
parsing failure, and the unchecked .(reference.Named) cast.

Also add a check for DockerReference() == nil to be extra paranoid,
although that should never happen.

Should not change behavior (but does not add unit tests).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač 190e074459 Remove the :// end from DockerTransport
(... but keep it in DefaultTransport, which remains irregular.)

This makes DockerTransport consistent with the others, and much more importantly,
allows several instances to do
> imgRef.Transport().Name() == DockerTransport
instead of the current
> strings.HasPrefix(DockerTransport, imgRef.Transport().Name())
, which currently works but is pretty nonsensical (it does not check
the "docker://" prefix against the _full reference_, but it checks
the _transport name_ as a prefix of "docker://", i.e.  a transport named
"d" would be accepted.

Should not change behavior, because the only currently existing transport
which has a name that is a prefix of "docker://" is c/image/docker.Transport
(but does not add unit tests).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač 9770ed257e Remove the TransportNames arrays
They are not used anywhere AFAICS, and the underlying idea
that transport-specific image names are reusable across transports
is very dubious anyway.  So, drop them instead of documenting
or fixing them.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač adfd3930c1 Document the properties of DefaultTransport a bit better.
This has no ambition to change the design, just to be clear about
what the design is.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač b3e6e908ab Eliminate the "dest" variable.
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač e8f7442831 Use an early exit if a docker-archive: image has no repo tags
This avoids another "append an only item to an empty array"
pattern, and will allow us to get rid of the "dest" variable
entirely.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač d4dbe66774 Reorganize the tag loading in DockerArchive case
This should not change behavior, only to make future edits
for an early exit easier to review.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač 0ef38ba079 Return early in refNamesFromImageReference instead of appending to pullNames
Almost all paths appended to pullNames exactly once; just construct a
single-element array in place and return it.

That way we can add empty lines as separators, and still come out shorter.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač ecc1db39b5 Use srcRef.StringWithinTransport() instead of parsing imgName again
Because srcRef is created by parsing imgName, both hard-code assumptions
about transport-specific formats of the strings, so that is neither better nor worse;
but we do less explicit parsing.

Should not change behavior for dir:, nor for fully-correct docker-archive:.

docker-archive:, though, also supports docker-archive:path:reference, where
the reference is ignored (with a warning) on read; in such cases the previous
code would use the reference only (not the path), the new code uses both
as the path.  Neither works, we just change the failure mode (but
"error opening path:reference" is now more suggestive of the correct usage).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač 24da27c3e9 Use a switch instead of if/if else/.../else
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač 7c37b25b4d Remove the error return value from getPullRefName
... it is always nil.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač 6ddf81f07d Rename getPullListFromRef to refPairsFromImageReference
This is a bit more specific as to what "ref" or "list" means,
and consistent with refPairsFromPossiblyUnqualifiedName

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač d61bed2b2d Split refNamesFromImageReference from Runtime.getPullListFromRef
Again, that makes the core logic independent from Runtime == containers-storage,
and easier to test independently.

So, this also adds tests.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač 9c9401a96c Replace getPullRefPair with getPullRefName
... and use pullRefPairsFromRefNames to convert to the
desired data structure later.

This will make both getPullRefName, and later the bulk of
getPullListFromRef, independent of the storage, and thus much easier to test.

Then add tests for getPullRefName.  (Ideally they should be shorter,
e.g. hopefully the .image member can be eliminated.)

Should not change behavior, except that error messages on invalid
dstName will now include the value.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Miloslav Trmač 8e7b4944f0 Include the rejected reference when parsing it fails in pullRefPairsFromRefNames
This will make any failures easier to attribute to the cause.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1176
Approved by: rhatdan
2018-08-01 18:22:58 +00:00
Daniel J Walsh 8e1ef558eb Add --force to podman umount to force the unmounting of the rootfs
podman umount will currently only unmount file system if not other
process is using it, otherwise the umount decrements the container
storage to indicate that the caller is no longer using the mount
point, once the count gets to 0, the file system is actually unmounted.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>

Closes: #1184
Approved by: TomSweeneyRedHat
2018-08-01 17:53:30 +00:00
baude a8ae7eae9c Integration Test Improvements #3
Third round of speed improvements to the integration tests.

Signed-off-by: baude <bbaude@redhat.com>

Closes: #1193
Approved by: rhatdan
2018-08-01 13:01:44 +00:00
Matthew Heon 1a439f9fcb Ensure container and pod refresh picks up a State
refresh() is the only major command we had that did not perform a
sync before running, and thus was not guaranteed to pick up a
good copy of the state. Fix this by updating the state before a
refresh().

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1186
Approved by: rhatdan
2018-07-31 14:19:50 +00:00
Matthew Heon 21f50b9f34 Fix build on non-linux platforms
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1186
Approved by: rhatdan
2018-07-31 14:19:50 +00:00
Matthew Heon c7c56d800c Rework state testing to allow State structs to be empty
Pod and container State structs are now allowed to be empty on
first being retrieved from the database. Rework pod and container
equality functions used in testing to account for this change.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1186
Approved by: rhatdan
2018-07-31 14:19:50 +00:00
Matthew Heon f4120f9662 Add additional comments on accessing state in API
The new state changes are potentially confusing to people writing
API functions on containers or pods. Add comments to the structs
on how to safely use them.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1186
Approved by: rhatdan
2018-07-31 14:19:50 +00:00
Matthew Heon 1db70cce34 Do not fetch pod and ctr State on retrieval in Bolt
It's not necessary to fill in state immediately, as we'll be
overwriting it on any API call accessing it thanks to
syncContainer(). It is also causing races when we fetch it
without holding the container lock (which syncContainer() does).
As such, just don't retrieve the state on initial pull from the
database with Bolt.

Also, refactor some Linux-specific netns handling functions out
of container_internal_linux.go into boltdb_linux.go.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1186
Approved by: rhatdan
2018-07-31 14:19:50 +00:00
Giuseppe Scrivano cfcd928476 network: add support for rootless network with slirp4netns
slirp4netns is required to setup the network namespace:

https://github.com/rootless-containers/slirp4netns

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #1156
Approved by: rhatdan
2018-07-31 13:39:29 +00:00
baude 5b9c60cc10 varlink ImageRemove should always return image ID
When removing an image via varlink, we should always return the
ID of the image even in the case where the image has multiple
repository names and one was only untagged.

Reported by jhonce during integration testing.

Signed-off-by: baude <bbaude@redhat.com>

Closes: #1191
Approved by: jwhonce
2018-07-31 11:38:48 +00:00
Daniel J Walsh 00caebde61 Add documentations on how to setup /etc/subuid and /etc/subgid
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>

Closes: #1185
Approved by: giuseppe
2018-07-31 08:35:20 +00:00
baude 5a4e5902a0 Integration Test Improvements #2
This is the second round of performance improvements for out
integration tests.

Signed-off-by: baude <bbaude@redhat.com>

Closes: #1190
Approved by: rhatdan
2018-07-30 23:53:08 +00:00
baude 8694e5414c avoid spewing fds do to restore of cached images
due to how cstorage is designed, we were spewing thousands of fds when
we restored cached images causing unwieldy rlimits.  we now use podman
load to restore the images thereby not tripping the issue.

Signed-off-by: baude <bbaude@redhat.com>

Closes: #1188
Approved by: baude
2018-07-30 20:28:30 +00:00
umohnani8 49bdd8421b Add load test for xz compressed images
The auto decompression functionality was already vendored in
with containers/image. Adding a test for it.

Signed-off-by: umohnani8 <umohnani@redhat.com>

Closes: #1137
Approved by: rhatdan
2018-07-30 16:56:11 +00:00
baude 49b3647410 Speed up test results
Stop all containers with a zero timeout prior to trying to rm -fa.  This results
in quicker teardown times by not waiting for timeouts.

Also, with wait tests, no need to wait the full 10 second sleep.  1 will do.

Signed-off-by: baude <bbaude@redhat.com>

Closes: #1181
Approved by: rhatdan
2018-07-30 12:26:39 +00:00
baude 433cbd5254 Show duration for each ginkgo test and test speed improvements
Because our tests are getting so long, we want to be able to audit which tests are taking
the longest to complete.  This may indicate a bad test, bad CI, bad code, etc and therefore
should be auditable.

Also, make speed improvements to tests by making sure we only unpack caches images that
actually get used.

Signed-off-by: baude <bbaude@redhat.com>

Closes: #1178
Approved by: mheon
2018-07-28 22:51:08 +00:00
Giuseppe Scrivano a4a667eac9 vendor: update containers/storage
update to version 956a1971694f18fd602b1203c0a2d192e2cc88a1

inherit support for IDs shifting when fuse-overlayfs is used.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #1177
Approved by: mheon
2018-07-28 14:41:07 +00:00
Jhon Honce 5aa36c1861 Clean up pylint warnings and errors for podman
* Some of the pedantic errors were not corrected
* Clean up prep for porting to MacOS and PyPi hosting
* Fix broken unittest

Signed-off-by: Jhon Honce <jhonce@redhat.com>

Closes: #1159
Approved by: rhatdan
2018-07-28 11:49:11 +00:00
umohnani8 87d8edb4c1 podman rmi shouldn't delete named referenced images
If an image is created from another and it is deleted,
only delete the actual image and not the parent images
if the parent images have names/references.

Signed-off-by: umohnani8 <umohnani@redhat.com>

Closes: #1174
Approved by: mheon
2018-07-28 01:40:28 +00:00
Matthew Heon bd7566f170
Merge pull request #1173 from mheon/bump-0.7.4
Bump to 0.7.4
2018-07-27 12:11:50 -04:00
Matthew Heon 8a10bf5f42 Bump gitvalidation epoch
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2018-07-27 12:05:42 -04:00
Matthew Heon 07a1f33edc Bump to v0.8.1-dev
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2018-07-27 12:05:41 -04:00
Matthew Heon 80612fb046 Bump to v0.7.4
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2018-07-27 12:05:38 -04:00
haircommander f258e43c7c Add pod pause/unpause
Added Pause() and Unpause() to libpod/pod.go

Added man pages, tests and completions

Signed-off-by: haircommander <pehunt@redhat.com>

Closes: #1126
Approved by: rhatdan
2018-07-27 14:20:08 +00:00
Daniel J Walsh 50fea69fbc Fix up docker compatibility messages
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>

Closes: #1170
Approved by: mheon
2018-07-27 13:40:56 +00:00
Daniel J Walsh 02e7efc2b3 Update vendored version of runc,buildah,containers/image
There is a compiler warning that has been fixed in the
upstream, so I figured we should update to fix.

Also vendor in latest buildah to get better support for running builds in rootless
mode.

Vendor in latest containers/image to allow daemon support to be pluggable.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>

Closes: #1169
Approved by: mheon
2018-07-27 13:07:19 +00:00
haircommander 876a30590b Refactored method of getting pods
Now, for commands that have --latest and --all, the context flags are checked, and pods are grabbed in a single function

Signed-off-by: haircommander <pehunt@redhat.com>

Closes: #1161
Approved by: rhatdan
2018-07-27 12:39:12 +00:00
Matthew Heon 3dd577e93c Fix godoc comment in pkg/netns
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1165
Approved by: baude
2018-07-27 02:48:16 +00:00
Matthew Heon 8c52aa15f0 Fix handling of Linux network namespaces
The CNI plugins upstream removed their network namespace creation
code, making it a test package only. Copy it into our repository
and slightly modify it for our use (most notably, use MNT_DETACH
when unmounting namespaces).

This new CNI code splits closing and unmounting network
namespaces, which allows us to greatly reduce the number of
occasions on which we call teardownNetwork() and make more errors
in that function fatal instead of warnings. Instead, we can call
Close() and just close the open file descriptor in cases where
the namespace has already been cleaned up.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

Closes: #1165
Approved by: baude
2018-07-27 02:48:15 +00:00