2022-08-17 08:43:43 +00:00
//go:build linux || freebsd
// +build linux freebsd
2019-10-08 17:53:36 +00:00
package libpod
import (
"bufio"
"bytes"
2020-08-24 15:35:01 +00:00
"context"
2022-07-05 09:42:22 +00:00
"errors"
2019-10-08 17:53:36 +00:00
"fmt"
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
"io"
"net"
2020-08-24 15:35:01 +00:00
"net/http"
2019-10-08 17:53:36 +00:00
"os"
"os/exec"
"path/filepath"
"strconv"
"strings"
2020-08-24 15:35:01 +00:00
"sync"
2019-10-08 17:53:36 +00:00
"syscall"
2020-01-07 12:41:56 +00:00
"text/template"
2019-10-08 17:53:36 +00:00
"time"
2020-03-27 14:13:51 +00:00
"github.com/containers/common/pkg/config"
2022-07-06 12:26:11 +00:00
"github.com/containers/common/pkg/resize"
cutil "github.com/containers/common/pkg/util"
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
conmonConfig "github.com/containers/conmon/runner/config"
2022-01-18 09:14:48 +00:00
"github.com/containers/podman/v4/libpod/define"
"github.com/containers/podman/v4/libpod/logs"
"github.com/containers/podman/v4/pkg/checkpoint/crutils"
"github.com/containers/podman/v4/pkg/errorhandling"
"github.com/containers/podman/v4/pkg/rootless"
"github.com/containers/podman/v4/pkg/specgenutil"
"github.com/containers/podman/v4/pkg/util"
"github.com/containers/podman/v4/utils"
2020-10-31 11:53:58 +00:00
"github.com/containers/storage/pkg/homedir"
2019-10-08 17:53:36 +00:00
spec "github.com/opencontainers/runtime-spec/specs-go"
"github.com/sirupsen/logrus"
"golang.org/x/sys/unix"
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
)
const (
// This is Conmon's STDIO_BUF_SIZE. I don't believe we have access to it
2020-12-21 22:48:43 +00:00
// directly from the Go code, so const it here
2021-09-09 12:05:25 +00:00
// Important: The conmon attach socket uses an extra byte at the beginning of each
// message to specify the STREAM so we have to increase the buffer size by one
bufferSize = conmonConfig . BufSize + 1
2019-10-08 17:53:36 +00:00
)
// ConmonOCIRuntime is an OCI runtime managed by Conmon.
// TODO: Make all calls to OCI runtime have a timeout.
type ConmonOCIRuntime struct {
name string
path string
conmonPath string
conmonEnv [ ] string
tmpDir string
exitsDir string
logSizeMax int64
noPivot bool
reservePorts bool
2020-08-21 17:56:50 +00:00
runtimeFlags [ ] string
2019-10-08 17:53:36 +00:00
supportsJSON bool
2020-04-15 18:48:53 +00:00
supportsKVM bool
2019-10-08 17:53:36 +00:00
supportsNoCgroups bool
2020-12-22 18:06:31 +00:00
enableKeyring bool
2019-10-08 17:53:36 +00:00
}
// Make a new Conmon-based OCI runtime with the given options.
// Conmon will wrap the given OCI runtime, which can be `runc`, `crun`, or
// any runtime with a runc-compatible CLI.
// The first path that points to a valid executable will be used.
// Deliberately private. Someone should not be able to construct this outside of
// libpod.
2020-08-21 17:56:50 +00:00
func newConmonOCIRuntime ( name string , paths [ ] string , conmonPath string , runtimeFlags [ ] string , runtimeCfg * config . Config ) ( OCIRuntime , error ) {
2019-10-08 17:53:36 +00:00
if name == "" {
2022-07-05 09:42:22 +00:00
return nil , fmt . Errorf ( "the OCI runtime must be provided a non-empty name: %w" , define . ErrInvalidArg )
2019-10-08 17:53:36 +00:00
}
2020-04-15 18:48:53 +00:00
// Make lookup tables for runtime support
supportsJSON := make ( map [ string ] bool , len ( runtimeCfg . Engine . RuntimeSupportsJSON ) )
supportsNoCgroups := make ( map [ string ] bool , len ( runtimeCfg . Engine . RuntimeSupportsNoCgroups ) )
supportsKVM := make ( map [ string ] bool , len ( runtimeCfg . Engine . RuntimeSupportsKVM ) )
for _ , r := range runtimeCfg . Engine . RuntimeSupportsJSON {
supportsJSON [ r ] = true
}
for _ , r := range runtimeCfg . Engine . RuntimeSupportsNoCgroups {
supportsNoCgroups [ r ] = true
}
for _ , r := range runtimeCfg . Engine . RuntimeSupportsKVM {
supportsKVM [ r ] = true
}
2019-10-08 17:53:36 +00:00
runtime := new ( ConmonOCIRuntime )
runtime . name = name
runtime . conmonPath = conmonPath
2020-08-21 17:56:50 +00:00
runtime . runtimeFlags = runtimeFlags
2019-10-08 17:53:36 +00:00
2020-03-27 14:13:51 +00:00
runtime . conmonEnv = runtimeCfg . Engine . ConmonEnvVars
runtime . tmpDir = runtimeCfg . Engine . TmpDir
runtime . logSizeMax = runtimeCfg . Containers . LogSizeMax
runtime . noPivot = runtimeCfg . Engine . NoPivotRoot
runtime . reservePorts = runtimeCfg . Engine . EnablePortReservation
2020-12-22 18:06:31 +00:00
runtime . enableKeyring = runtimeCfg . Containers . EnableKeyring
2019-10-08 17:53:36 +00:00
// TODO: probe OCI runtime for feature and enable automatically if
// available.
2021-03-03 13:28:29 +00:00
base := filepath . Base ( name )
runtime . supportsJSON = supportsJSON [ base ]
runtime . supportsNoCgroups = supportsNoCgroups [ base ]
runtime . supportsKVM = supportsKVM [ base ]
2019-10-08 17:53:36 +00:00
foundPath := false
for _ , path := range paths {
stat , err := os . Stat ( path )
if err != nil {
if os . IsNotExist ( err ) {
continue
}
2022-07-05 09:42:22 +00:00
return nil , fmt . Errorf ( "cannot stat OCI runtime %s path: %w" , name , err )
2019-10-08 17:53:36 +00:00
}
if ! stat . Mode ( ) . IsRegular ( ) {
continue
}
foundPath = true
2022-07-12 07:37:46 +00:00
logrus . Tracef ( "found runtime %q" , path )
2019-10-08 17:53:36 +00:00
runtime . path = path
break
}
// Search the $PATH as last fallback
if ! foundPath {
if foundRuntime , err := exec . LookPath ( name ) ; err == nil {
foundPath = true
runtime . path = foundRuntime
logrus . Debugf ( "using runtime %q from $PATH: %q" , name , foundRuntime )
}
}
if ! foundPath {
2022-07-05 09:42:22 +00:00
return nil , fmt . Errorf ( "no valid executable found for OCI runtime %s: %w" , name , define . ErrInvalidArg )
2019-10-08 17:53:36 +00:00
}
runtime . exitsDir = filepath . Join ( runtime . tmpDir , "exits" )
// Create the exit files and attach sockets directories
if err := os . MkdirAll ( runtime . exitsDir , 0750 ) ; err != nil {
// The directory is allowed to exist
if ! os . IsExist ( err ) {
2022-09-10 11:40:39 +00:00
return nil , fmt . Errorf ( "creating OCI runtime exit files directory: %w" , err )
2019-10-08 17:53:36 +00:00
}
}
return runtime , nil
}
// Name returns the name of the runtime being wrapped by Conmon.
func ( r * ConmonOCIRuntime ) Name ( ) string {
return r . name
}
// Path returns the path of the OCI runtime being wrapped by Conmon.
func ( r * ConmonOCIRuntime ) Path ( ) string {
return r . path
}
2020-01-14 12:23:59 +00:00
// hasCurrentUserMapped checks whether the current user is mapped inside the container user namespace
func hasCurrentUserMapped ( ctr * Container ) bool {
if len ( ctr . config . IDMappings . UIDMap ) == 0 && len ( ctr . config . IDMappings . GIDMap ) == 0 {
return true
}
uid := os . Geteuid ( )
for _ , m := range ctr . config . IDMappings . UIDMap {
if uid >= m . HostID && uid < m . HostID + m . Size {
return true
}
}
return false
}
2019-10-08 17:53:36 +00:00
// CreateContainer creates a container.
2021-11-09 21:21:07 +00:00
func ( r * ConmonOCIRuntime ) CreateContainer ( ctr * Container , restoreOptions * ContainerCheckpointOptions ) ( int64 , error ) {
2021-01-04 10:15:28 +00:00
// always make the run dir accessible to the current user so that the PID files can be read without
// being in the rootless user namespace.
if err := makeAccessible ( ctr . state . RunDir , 0 , 0 ) ; err != nil {
2021-11-09 21:21:07 +00:00
return 0 , err
2021-01-04 10:15:28 +00:00
}
2020-01-14 12:23:59 +00:00
if ! hasCurrentUserMapped ( ctr ) {
2020-03-27 14:13:51 +00:00
for _ , i := range [ ] string { ctr . state . RunDir , ctr . runtime . config . Engine . TmpDir , ctr . config . StaticDir , ctr . state . Mountpoint , ctr . runtime . config . Engine . VolumePath } {
2019-10-08 17:53:36 +00:00
if err := makeAccessible ( i , ctr . RootUID ( ) , ctr . RootGID ( ) ) ; err != nil {
2021-11-09 21:21:07 +00:00
return 0 , err
2019-10-08 17:53:36 +00:00
}
}
// if we are running a non privileged container, be sure to umount some kernel paths so they are not
// bind mounted inside the container at all.
if ! ctr . config . Privileged && ! rootless . IsRootless ( ) {
2022-08-17 09:29:40 +00:00
return r . createRootlessContainer ( ctr , restoreOptions )
2019-10-08 17:53:36 +00:00
}
}
return r . createOCIContainer ( ctr , restoreOptions )
}
// UpdateContainerStatus retrieves the current status of the container from the
// runtime. It updates the container's state but does not save it.
// If useRuntime is false, we will not directly hit runc to see the container's
// status, but will instead only check for the existence of the conmon exit file
// and update state to stopped if it exists.
2019-10-15 19:11:26 +00:00
func ( r * ConmonOCIRuntime ) UpdateContainerStatus ( ctr * Container ) error {
2019-10-08 17:53:36 +00:00
runtimeDir , err := util . GetRuntimeDir ( )
if err != nil {
return err
}
// Store old state so we know if we were already stopped
oldState := ctr . state . State
state := new ( spec . State )
cmd := exec . Command ( r . path , "state" , ctr . ID ( ) )
cmd . Env = append ( cmd . Env , fmt . Sprintf ( "XDG_RUNTIME_DIR=%s" , runtimeDir ) )
outPipe , err := cmd . StdoutPipe ( )
if err != nil {
2022-07-05 09:42:22 +00:00
return fmt . Errorf ( "getting stdout pipe: %w" , err )
2019-10-08 17:53:36 +00:00
}
errPipe , err := cmd . StderrPipe ( )
if err != nil {
2022-07-05 09:42:22 +00:00
return fmt . Errorf ( "getting stderr pipe: %w" , err )
2019-10-08 17:53:36 +00:00
}
if err := cmd . Start ( ) ; err != nil {
2022-09-20 13:59:28 +00:00
out , err2 := io . ReadAll ( errPipe )
2019-10-08 17:53:36 +00:00
if err2 != nil {
2022-09-10 11:40:39 +00:00
return fmt . Errorf ( "getting container %s state: %w" , ctr . ID ( ) , err )
2019-10-08 17:53:36 +00:00
}
2021-11-23 11:11:23 +00:00
if strings . Contains ( string ( out ) , "does not exist" ) || strings . Contains ( string ( out ) , "No such file" ) {
2019-10-08 17:53:36 +00:00
if err := ctr . removeConmonFiles ( ) ; err != nil {
logrus . Debugf ( "unable to remove conmon files for container %s" , ctr . ID ( ) )
}
ctr . state . ExitCode = - 1
ctr . state . FinishedTime = time . Now ( )
ctr . state . State = define . ContainerStateExited
2022-07-08 13:17:43 +00:00
return ctr . runtime . state . AddContainerExitCode ( ctr . ID ( ) , ctr . state . ExitCode )
2019-10-08 17:53:36 +00:00
}
2022-09-10 11:40:39 +00:00
return fmt . Errorf ( "getting container %s state. stderr/out: %s: %w" , ctr . ID ( ) , out , err )
2019-10-08 17:53:36 +00:00
}
defer func ( ) {
_ = cmd . Wait ( )
} ( )
if err := errPipe . Close ( ) ; err != nil {
return err
}
2022-09-20 13:59:28 +00:00
out , err := io . ReadAll ( outPipe )
2019-10-08 17:53:36 +00:00
if err != nil {
2022-09-10 11:40:39 +00:00
return fmt . Errorf ( "reading stdout: %s: %w" , ctr . ID ( ) , err )
2019-10-08 17:53:36 +00:00
}
if err := json . NewDecoder ( bytes . NewBuffer ( out ) ) . Decode ( state ) ; err != nil {
2022-09-10 11:40:39 +00:00
return fmt . Errorf ( "decoding container status for container %s: %w" , ctr . ID ( ) , err )
2019-10-08 17:53:36 +00:00
}
ctr . state . PID = state . Pid
switch state . Status {
case "created" :
ctr . state . State = define . ContainerStateCreated
case "paused" :
ctr . state . State = define . ContainerStatePaused
case "running" :
ctr . state . State = define . ContainerStateRunning
case "stopped" :
ctr . state . State = define . ContainerStateStopped
default :
2022-07-05 09:42:22 +00:00
return fmt . Errorf ( "unrecognized status returned by runtime for container %s: %s: %w" ,
ctr . ID ( ) , state . Status , define . ErrInternal )
2019-10-08 17:53:36 +00:00
}
2021-10-06 15:17:43 +00:00
// Handle ContainerStateStopping - keep it unless the container
// transitioned to no longer running.
if oldState == define . ContainerStateStopping && ( ctr . state . State == define . ContainerStatePaused || ctr . state . State == define . ContainerStateRunning ) {
ctr . state . State = define . ContainerStateStopping
}
2019-10-08 17:53:36 +00:00
return nil
}
// StartContainer starts the given container.
// Sets time the container was started, but does not save it.
func ( r * ConmonOCIRuntime ) StartContainer ( ctr * Container ) error {
// TODO: streams should probably *not* be our STDIN/OUT/ERR - redirect to buffers?
runtimeDir , err := util . GetRuntimeDir ( )
if err != nil {
return err
}
env := [ ] string { fmt . Sprintf ( "XDG_RUNTIME_DIR=%s" , runtimeDir ) }
2020-04-03 15:45:51 +00:00
if path , ok := os . LookupEnv ( "PATH" ) ; ok {
env = append ( env , fmt . Sprintf ( "PATH=%s" , path ) )
}
2020-08-21 17:56:50 +00:00
if err := utils . ExecCmdWithStdStreams ( os . Stdin , os . Stdout , os . Stderr , env , r . path , append ( r . runtimeFlags , "start" , ctr . ID ( ) ) ... ) ; err != nil {
2019-10-08 17:53:36 +00:00
return err
}
ctr . state . StartedTime = time . Now ( )
return nil
}
2022-08-09 13:25:03 +00:00
// UpdateContainer updates the given container's cgroup configuration
func ( r * ConmonOCIRuntime ) UpdateContainer ( ctr * Container , resources * spec . LinuxResources ) error {
runtimeDir , err := util . GetRuntimeDir ( )
if err != nil {
return err
}
env := [ ] string { fmt . Sprintf ( "XDG_RUNTIME_DIR=%s" , runtimeDir ) }
if path , ok := os . LookupEnv ( "PATH" ) ; ok {
env = append ( env , fmt . Sprintf ( "PATH=%s" , path ) )
}
args := r . runtimeFlags
args = append ( args , "update" )
tempFile , additionalArgs , err := generateResourceFile ( resources )
if err != nil {
return err
}
defer os . Remove ( tempFile )
args = append ( args , additionalArgs ... )
return utils . ExecCmdWithStdStreams ( os . Stdin , os . Stdout , os . Stderr , env , r . path , append ( args , ctr . ID ( ) ) ... )
}
func generateResourceFile ( res * spec . LinuxResources ) ( string , [ ] string , error ) {
flags := [ ] string { }
if res == nil {
return "" , flags , nil
}
2022-09-20 13:59:28 +00:00
f , err := os . CreateTemp ( "" , "podman" )
2022-08-09 13:25:03 +00:00
if err != nil {
return "" , nil , err
}
j , err := json . Marshal ( res )
if err != nil {
return "" , nil , err
}
_ , err = f . WriteString ( string ( j ) )
if err != nil {
return "" , nil , err
}
flags = append ( flags , "--resources=" + f . Name ( ) )
return f . Name ( ) , flags , nil
}
2019-10-08 17:53:36 +00:00
// KillContainer sends the given signal to the given container.
// If all is set, send to all PIDs in the container.
// All is only supported if the container created cgroups.
func ( r * ConmonOCIRuntime ) KillContainer ( ctr * Container , signal uint , all bool ) error {
logrus . Debugf ( "Sending signal %d to container %s" , signal , ctr . ID ( ) )
runtimeDir , err := util . GetRuntimeDir ( )
if err != nil {
return err
}
env := [ ] string { fmt . Sprintf ( "XDG_RUNTIME_DIR=%s" , runtimeDir ) }
var args [ ] string
2020-08-21 17:56:50 +00:00
args = append ( args , r . runtimeFlags ... )
2019-10-08 17:53:36 +00:00
if all {
2020-08-21 17:56:50 +00:00
args = append ( args , "kill" , "--all" , ctr . ID ( ) , fmt . Sprintf ( "%d" , signal ) )
2019-10-08 17:53:36 +00:00
} else {
2020-08-21 17:56:50 +00:00
args = append ( args , "kill" , ctr . ID ( ) , fmt . Sprintf ( "%d" , signal ) )
2019-10-08 17:53:36 +00:00
}
if err := utils . ExecCmdWithStdStreams ( os . Stdin , os . Stdout , os . Stderr , env , r . path , args ... ) ; err != nil {
2021-11-23 14:21:05 +00:00
// Update container state - there's a chance we failed because
// the container exited in the meantime.
if err2 := r . UpdateContainerStatus ( ctr ) ; err2 != nil {
logrus . Infof ( "Error updating status for container %s: %v" , ctr . ID ( ) , err2 )
}
2022-06-08 13:27:19 +00:00
if ctr . ensureState ( define . ContainerStateStopped , define . ContainerStateExited ) {
2022-10-13 10:22:34 +00:00
return fmt . Errorf ( "%w: %s" , define . ErrCtrStateInvalid , ctr . state . State )
2021-11-23 11:07:31 +00:00
}
2022-09-10 11:40:39 +00:00
return fmt . Errorf ( "sending signal to container %s: %w" , ctr . ID ( ) , err )
2019-10-08 17:53:36 +00:00
}
return nil
}
// StopContainer stops a container, first using its given stop signal (or
// SIGTERM if no signal was specified), then using SIGKILL.
// Timeout is given in seconds. If timeout is 0, the container will be
// immediately kill with SIGKILL.
// Does not set finished time for container, assumes you will run updateStatus
// after to pull the exit code.
func ( r * ConmonOCIRuntime ) StopContainer ( ctr * Container , timeout uint , all bool ) error {
logrus . Debugf ( "Stopping container %s (PID %d)" , ctr . ID ( ) , ctr . state . PID )
// Ping the container to see if it's alive
// If it's not, it's already stopped, return
err := unix . Kill ( ctr . state . PID , 0 )
if err == unix . ESRCH {
return nil
}
if timeout > 0 {
2023-01-19 09:57:31 +00:00
stopSignal := ctr . config . StopSignal
if stopSignal == 0 {
stopSignal = uint ( syscall . SIGTERM )
}
2019-10-08 17:53:36 +00:00
if err := r . KillContainer ( ctr , stopSignal , all ) ; err != nil {
// Is the container gone?
// If so, it probably died between the first check and
// our sending the signal
// The container is stopped, so exit cleanly
err := unix . Kill ( ctr . state . PID , 0 )
if err == unix . ESRCH {
return nil
}
return err
}
if err := waitContainerStop ( ctr , time . Duration ( timeout ) * time . Second ) ; err != nil {
2021-09-29 14:03:46 +00:00
logrus . Debugf ( "Timed out stopping container %s with %s, resorting to SIGKILL: %v" , ctr . ID ( ) , unix . SignalName ( syscall . Signal ( stopSignal ) ) , err )
logrus . Warnf ( "StopSignal %s failed to stop container %s in %d seconds, resorting to SIGKILL" , unix . SignalName ( syscall . Signal ( stopSignal ) ) , ctr . Name ( ) , timeout )
2019-10-08 17:53:36 +00:00
} else {
// No error, the container is dead
return nil
}
}
2023-01-19 09:59:23 +00:00
// If the timeout was set to 0 or if stopping the container with the
// specified signal did not work, use the big hammer with SIGKILL.
2022-09-09 11:29:40 +00:00
if err := r . KillContainer ( ctr , uint ( unix . SIGKILL ) , all ) ; err != nil {
2023-01-19 10:03:23 +00:00
// There's an inherent race with the cleanup process (see
// #16142, #17142). If the container has already been marked as
// stopped or exited by the cleanup process, we can return
// immediately.
if errors . Is ( err , define . ErrCtrStateInvalid ) && ctr . ensureState ( define . ContainerStateStopped , define . ContainerStateExited ) {
return nil
}
// If the PID is 0, then the container is already stopped.
if ctr . state . PID == 0 {
return nil
}
// Again, check if the container is gone. If it is, exit cleanly.
if aliveErr := unix . Kill ( ctr . state . PID , 0 ) ; errors . Is ( aliveErr , unix . ESRCH ) {
return nil
2019-10-08 17:53:36 +00:00
}
2023-01-19 10:03:23 +00:00
return fmt . Errorf ( "sending SIGKILL to container %s: %w" , ctr . ID ( ) , err )
2019-10-08 17:53:36 +00:00
}
// Give runtime a few seconds to make it happen
if err := waitContainerStop ( ctr , killContainerTimeout ) ; err != nil {
return err
}
return nil
}
// DeleteContainer deletes a container from the OCI runtime.
func ( r * ConmonOCIRuntime ) DeleteContainer ( ctr * Container ) error {
runtimeDir , err := util . GetRuntimeDir ( )
if err != nil {
return err
}
env := [ ] string { fmt . Sprintf ( "XDG_RUNTIME_DIR=%s" , runtimeDir ) }
2020-08-21 17:56:50 +00:00
return utils . ExecCmdWithStdStreams ( os . Stdin , os . Stdout , os . Stderr , env , r . path , append ( r . runtimeFlags , "delete" , "--force" , ctr . ID ( ) ) ... )
2019-10-08 17:53:36 +00:00
}
// PauseContainer pauses the given container.
func ( r * ConmonOCIRuntime ) PauseContainer ( ctr * Container ) error {
runtimeDir , err := util . GetRuntimeDir ( )
if err != nil {
return err
}
env := [ ] string { fmt . Sprintf ( "XDG_RUNTIME_DIR=%s" , runtimeDir ) }
2020-08-21 17:56:50 +00:00
return utils . ExecCmdWithStdStreams ( os . Stdin , os . Stdout , os . Stderr , env , r . path , append ( r . runtimeFlags , "pause" , ctr . ID ( ) ) ... )
2019-10-08 17:53:36 +00:00
}
// UnpauseContainer unpauses the given container.
func ( r * ConmonOCIRuntime ) UnpauseContainer ( ctr * Container ) error {
runtimeDir , err := util . GetRuntimeDir ( )
if err != nil {
return err
}
env := [ ] string { fmt . Sprintf ( "XDG_RUNTIME_DIR=%s" , runtimeDir ) }
2020-08-21 17:56:50 +00:00
return utils . ExecCmdWithStdStreams ( os . Stdin , os . Stdout , os . Stderr , env , r . path , append ( r . runtimeFlags , "resume" , ctr . ID ( ) ) ... )
2019-10-08 17:53:36 +00:00
}
2022-09-06 13:58:47 +00:00
// This filters out ENOTCONN errors which can happen on FreeBSD if the
// other side of the connection is already closed.
func socketCloseWrite ( conn * net . UnixConn ) error {
err := conn . CloseWrite ( )
if err != nil && errors . Is ( err , syscall . ENOTCONN ) {
return nil
}
return err
}
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
// HTTPAttach performs an attach for the HTTP API.
2020-04-07 20:52:47 +00:00
// The caller must handle closing the HTTP connection after this returns.
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
// The cancel channel is not closed; it is up to the caller to do so after
// this function returns.
// If this is a container with a terminal, we will stream raw. If it is not, we
// will stream with an 8-byte header to multiplex STDOUT and STDERR.
2020-08-24 15:35:01 +00:00
// Returns any errors that occurred, and whether the connection was successfully
// hijacked before that error occurred.
func ( r * ConmonOCIRuntime ) HTTPAttach ( ctr * Container , req * http . Request , w http . ResponseWriter , streams * HTTPAttachStreams , detachKeys * string , cancel <- chan bool , hijackDone chan <- bool , streamAttach , streamLogs bool ) ( deferredErr error ) {
2022-10-11 14:38:42 +00:00
isTerminal := ctr . Terminal ( )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
if streams != nil {
if ! streams . Stdin && ! streams . Stdout && ! streams . Stderr {
2022-07-05 09:42:22 +00:00
return fmt . Errorf ( "must specify at least one stream to attach to: %w" , define . ErrInvalidArg )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
}
}
attachSock , err := r . AttachSocketPath ( ctr )
if err != nil {
return err
}
2020-08-24 15:35:01 +00:00
var conn * net . UnixConn
if streamAttach {
2021-01-11 10:25:43 +00:00
newConn , err := openUnixSocket ( attachSock )
2020-08-24 15:35:01 +00:00
if err != nil {
2022-07-05 09:42:22 +00:00
return fmt . Errorf ( "failed to connect to container's attach socket: %v: %w" , attachSock , err )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
}
2020-08-24 15:35:01 +00:00
conn = newConn
defer func ( ) {
if err := conn . Close ( ) ; err != nil {
2021-09-22 13:45:15 +00:00
logrus . Errorf ( "Unable to close container %s attach socket: %q" , ctr . ID ( ) , err )
2020-08-24 15:35:01 +00:00
}
} ( )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
2021-01-11 10:25:43 +00:00
logrus . Debugf ( "Successfully connected to container %s attach socket %s" , ctr . ID ( ) , attachSock )
2020-08-24 15:35:01 +00:00
}
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
2020-03-27 14:13:51 +00:00
detachString := ctr . runtime . config . Engine . DetachKeys
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
if detachKeys != nil {
detachString = * detachKeys
}
detach , err := processDetachKeys ( detachString )
if err != nil {
return err
}
attachStdout := true
attachStderr := true
attachStdin := true
if streams != nil {
attachStdout = streams . Stdout
attachStderr = streams . Stderr
attachStdin = streams . Stdin
}
2020-08-24 15:35:01 +00:00
logrus . Debugf ( "Going to hijack container %s attach connection" , ctr . ID ( ) )
// Alright, let's hijack.
hijacker , ok := w . ( http . Hijacker )
if ! ok {
2022-07-05 09:42:22 +00:00
return fmt . Errorf ( "unable to hijack connection" )
2020-08-24 15:35:01 +00:00
}
httpCon , httpBuf , err := hijacker . Hijack ( )
if err != nil {
2022-09-10 11:40:39 +00:00
return fmt . Errorf ( "hijacking connection: %w" , err )
2020-08-24 15:35:01 +00:00
}
hijackDone <- true
writeHijackHeader ( req , httpBuf )
// Force a flush after the header is written.
if err := httpBuf . Flush ( ) ; err != nil {
2022-09-10 11:40:39 +00:00
return fmt . Errorf ( "flushing HTTP hijack header: %w" , err )
2020-08-24 15:35:01 +00:00
}
defer func ( ) {
hijackWriteErrorAndClose ( deferredErr , ctr . ID ( ) , isTerminal , httpCon , httpBuf )
} ( )
logrus . Debugf ( "Hijack for container %s attach session done, ready to stream" , ctr . ID ( ) )
// TODO: This is gross. Really, really gross.
// I want to say we should read all the logs into an array before
// calling this, in container_api.go, but that could take a lot of
// memory...
// On the whole, we need to figure out a better way of doing this,
// though.
logSize := 0
if streamLogs {
logrus . Debugf ( "Will stream logs for container %s attach session" , ctr . ID ( ) )
// Get all logs for the container
logChan := make ( chan * logs . LogLine )
logOpts := new ( logs . LogOptions )
logOpts . Tail = - 1
logOpts . WaitGroup = new ( sync . WaitGroup )
errChan := make ( chan error )
go func ( ) {
var err error
// In non-terminal mode we need to prepend with the
// stream header.
logrus . Debugf ( "Writing logs for container %s to HTTP attach" , ctr . ID ( ) )
for logLine := range logChan {
if ! isTerminal {
device := logLine . Device
var header [ ] byte
headerLen := uint32 ( len ( logLine . Msg ) )
2022-12-15 17:32:55 +00:00
if ! logLine . Partial ( ) {
// we append an extra newline in this case so we need to increment the len as well
headerLen ++
}
2020-08-24 15:35:01 +00:00
logSize += len ( logLine . Msg )
switch strings . ToLower ( device ) {
case "stdin" :
header = makeHTTPAttachHeader ( 0 , headerLen )
case "stdout" :
header = makeHTTPAttachHeader ( 1 , headerLen )
case "stderr" :
header = makeHTTPAttachHeader ( 2 , headerLen )
default :
logrus . Errorf ( "Unknown device for log line: %s" , device )
header = makeHTTPAttachHeader ( 1 , headerLen )
}
_ , err = httpBuf . Write ( header )
if err != nil {
break
}
}
_ , err = httpBuf . Write ( [ ] byte ( logLine . Msg ) )
if err != nil {
break
}
2021-08-19 22:24:47 +00:00
if ! logLine . Partial ( ) {
_ , err = httpBuf . Write ( [ ] byte ( "\n" ) )
if err != nil {
break
}
2020-08-24 15:35:01 +00:00
}
err = httpBuf . Flush ( )
if err != nil {
break
}
}
errChan <- err
} ( )
2022-03-04 18:04:58 +00:00
if err := ctr . ReadLog ( context . Background ( ) , logOpts , logChan , 0 ) ; err != nil {
2022-01-18 15:27:33 +00:00
return err
}
2020-08-24 15:35:01 +00:00
go func ( ) {
logOpts . WaitGroup . Wait ( )
close ( logChan )
} ( )
logrus . Debugf ( "Done reading logs for container %s, %d bytes" , ctr . ID ( ) , logSize )
if err := <- errChan ; err != nil {
return err
}
}
if ! streamAttach {
logrus . Debugf ( "Done streaming logs for container %s attach, exiting as attach streaming not requested" , ctr . ID ( ) )
return nil
}
logrus . Debugf ( "Forwarding attach output for container %s" , ctr . ID ( ) )
2020-09-23 20:30:51 +00:00
stdoutChan := make ( chan error )
stdinChan := make ( chan error )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
// Handle STDOUT/STDERR
go func ( ) {
var err error
if isTerminal {
2020-04-07 20:52:47 +00:00
// Hack: return immediately if attachStdout not set to
// emulate Docker.
// Basically, when terminal is set, STDERR goes nowhere.
// Everything does over STDOUT.
// Therefore, if not attaching STDOUT - we'll never copy
// anything from here.
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
logrus . Debugf ( "Performing terminal HTTP attach for container %s" , ctr . ID ( ) )
2020-04-07 20:52:47 +00:00
if attachStdout {
err = httpAttachTerminalCopy ( conn , httpBuf , ctr . ID ( ) )
}
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
} else {
logrus . Debugf ( "Performing non-terminal HTTP attach for container %s" , ctr . ID ( ) )
err = httpAttachNonTerminalCopy ( conn , httpBuf , ctr . ID ( ) , attachStdin , attachStdout , attachStderr )
}
2020-09-23 20:30:51 +00:00
stdoutChan <- err
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
logrus . Debugf ( "STDOUT/ERR copy completed" )
} ( )
// Next, STDIN. Avoid entirely if attachStdin unset.
if attachStdin {
go func ( ) {
2022-07-06 12:26:11 +00:00
_ , err := cutil . CopyDetachable ( conn , httpBuf , detach )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
logrus . Debugf ( "STDIN copy completed" )
2020-09-23 20:30:51 +00:00
stdinChan <- err
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
} ( )
}
2020-09-23 20:30:51 +00:00
for {
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
select {
2020-09-23 20:30:51 +00:00
case err := <- stdoutChan :
if err != nil {
return err
}
return nil
case err := <- stdinChan :
if err != nil {
return err
}
2021-10-05 09:13:49 +00:00
// copy stdin is done, close it
2022-09-06 13:58:47 +00:00
if connErr := socketCloseWrite ( conn ) ; connErr != nil {
2021-10-05 09:13:49 +00:00
logrus . Errorf ( "Unable to close conn: %v" , connErr )
}
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
case <- cancel :
return nil
}
}
}
2020-03-31 09:15:13 +00:00
// isRetryable returns whether the error was caused by a blocked syscall or the
// specified operation on a non blocking file descriptor wasn't ready for completion.
func isRetryable ( err error ) bool {
2022-07-05 09:42:22 +00:00
var errno syscall . Errno
if errors . As ( err , & errno ) {
2020-03-31 09:15:13 +00:00
return errno == syscall . EINTR || errno == syscall . EAGAIN
}
return false
}
// openControlFile opens the terminal control file.
func openControlFile ( ctr * Container , parentDir string ) ( * os . File , error ) {
controlPath := filepath . Join ( parentDir , "ctl" )
for i := 0 ; i < 600 ; i ++ {
controlFile , err := os . OpenFile ( controlPath , unix . O_WRONLY | unix . O_NONBLOCK , 0 )
if err == nil {
2022-03-21 13:49:47 +00:00
return controlFile , nil
2020-03-31 09:15:13 +00:00
}
if ! isRetryable ( err ) {
2022-07-05 09:42:22 +00:00
return nil , fmt . Errorf ( "could not open ctl file for terminal resize for container %s: %w" , ctr . ID ( ) , err )
2020-03-31 09:15:13 +00:00
}
time . Sleep ( time . Second / 10 )
}
2022-07-05 09:42:22 +00:00
return nil , fmt . Errorf ( "timeout waiting for %q" , controlPath )
2020-03-31 09:15:13 +00:00
}
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
// AttachResize resizes the terminal used by the given container.
2022-07-06 12:26:11 +00:00
func ( r * ConmonOCIRuntime ) AttachResize ( ctr * Container , newSize resize . TerminalSize ) error {
2020-03-31 09:15:13 +00:00
controlFile , err := openControlFile ( ctr , ctr . bundlePath ( ) )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
if err != nil {
2020-03-31 09:15:13 +00:00
return err
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
}
defer controlFile . Close ( )
logrus . Debugf ( "Received a resize event for container %s: %+v" , ctr . ID ( ) , newSize )
if _ , err = fmt . Fprintf ( controlFile , "%d %d %d\n" , 1 , newSize . Height , newSize . Width ) ; err != nil {
2022-07-05 09:42:22 +00:00
return fmt . Errorf ( "failed to write to ctl file to resize terminal: %w" , err )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
}
return nil
}
2019-10-08 17:53:36 +00:00
// CheckpointContainer checkpoints the given container.
Added optional container checkpointing statistics
This adds the parameter '--print-stats' to 'podman container checkpoint'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to create a checkpoint and print out these
information. CRIU already creates checkpointing statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the checkpointed container, Podman will now print
out JSON:
# podman container checkpoint --latest --print-stats
{
"podman_checkpoint_duration": 360749,
"container_statistics": [
{
"Id": "25244244bf2efbef30fb6857ddea8cb2e5489f07eb6659e20dda117f0c466808",
"runtime_checkpoint_duration": 177222,
"criu_statistics": {
"freezing_time": 100657,
"frozen_time": 60700,
"memdump_time": 8162,
"memwrite_time": 4224,
"pages_scanned": 20561,
"pages_written": 2129
}
}
]
}
The output contains 'podman_checkpoint_duration' which contains the
number of microseconds Podman required to create the checkpoint. The
output also includes 'runtime_checkpoint_duration' which is the time
the runtime needed to checkpoint that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.
Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-09 09:15:50 +00:00
func ( r * ConmonOCIRuntime ) CheckpointContainer ( ctr * Container , options ContainerCheckpointOptions ) ( int64 , error ) {
2019-10-08 17:53:36 +00:00
// imagePath is used by CRIU to store the actual checkpoint files
imagePath := ctr . CheckpointPath ( )
2021-01-10 10:12:12 +00:00
if options . PreCheckPoint {
imagePath = ctr . PreCheckPointPath ( )
}
2019-10-08 17:53:36 +00:00
// workPath will be used to store dump.log and stats-dump
workPath := ctr . bundlePath ( )
logrus . Debugf ( "Writing checkpoint to %s" , imagePath )
logrus . Debugf ( "Writing checkpoint logs to %s" , workPath )
2021-01-10 10:12:12 +00:00
logrus . Debugf ( "Pre-dump the container %t" , options . PreCheckPoint )
2019-10-08 17:53:36 +00:00
args := [ ] string { }
2020-08-21 17:56:50 +00:00
args = append ( args , r . runtimeFlags ... )
2019-10-08 17:53:36 +00:00
args = append ( args , "checkpoint" )
args = append ( args , "--image-path" )
args = append ( args , imagePath )
args = append ( args , "--work-path" )
args = append ( args , workPath )
if options . KeepRunning {
args = append ( args , "--leave-running" )
}
if options . TCPEstablished {
args = append ( args , "--tcp-established" )
}
2021-11-17 11:53:12 +00:00
if options . FileLocks {
args = append ( args , "--file-locks" )
}
2021-01-10 10:12:12 +00:00
if ! options . PreCheckPoint && options . KeepRunning {
args = append ( args , "--leave-running" )
}
if options . PreCheckPoint {
args = append ( args , "--pre-dump" )
}
if ! options . PreCheckPoint && options . WithPrevious {
2021-06-10 12:27:09 +00:00
args = append (
args ,
"--parent-path" ,
filepath . Join ( ".." , preCheckpointDir ) ,
)
2021-01-10 10:12:12 +00:00
}
2021-11-24 12:40:26 +00:00
args = append ( args , ctr . ID ( ) )
logrus . Debugf ( "the args to checkpoint: %s %s" , r . path , strings . Join ( args , " " ) )
2020-04-03 07:59:49 +00:00
runtimeDir , err := util . GetRuntimeDir ( )
if err != nil {
Added optional container checkpointing statistics
This adds the parameter '--print-stats' to 'podman container checkpoint'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to create a checkpoint and print out these
information. CRIU already creates checkpointing statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the checkpointed container, Podman will now print
out JSON:
# podman container checkpoint --latest --print-stats
{
"podman_checkpoint_duration": 360749,
"container_statistics": [
{
"Id": "25244244bf2efbef30fb6857ddea8cb2e5489f07eb6659e20dda117f0c466808",
"runtime_checkpoint_duration": 177222,
"criu_statistics": {
"freezing_time": 100657,
"frozen_time": 60700,
"memdump_time": 8162,
"memwrite_time": 4224,
"pages_scanned": 20561,
"pages_written": 2129
}
}
]
}
The output contains 'podman_checkpoint_duration' which contains the
number of microseconds Podman required to create the checkpoint. The
output also includes 'runtime_checkpoint_duration' which is the time
the runtime needed to checkpoint that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.
Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-09 09:15:50 +00:00
return 0 , err
2020-04-03 07:59:49 +00:00
}
2021-11-24 12:40:26 +00:00
env := [ ] string { fmt . Sprintf ( "XDG_RUNTIME_DIR=%s" , runtimeDir ) }
if path , ok := os . LookupEnv ( "PATH" ) ; ok {
env = append ( env , fmt . Sprintf ( "PATH=%s" , path ) )
2020-04-03 07:59:49 +00:00
}
2021-11-24 12:40:26 +00:00
2022-08-17 09:30:30 +00:00
var runtimeCheckpointStarted time . Time
err = r . withContainerSocketLabel ( ctr , func ( ) error {
runtimeCheckpointStarted = time . Now ( )
return utils . ExecCmdWithStdStreams ( os . Stdin , os . Stdout , os . Stderr , env , r . path , args ... )
} )
Added optional container checkpointing statistics
This adds the parameter '--print-stats' to 'podman container checkpoint'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to create a checkpoint and print out these
information. CRIU already creates checkpointing statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the checkpointed container, Podman will now print
out JSON:
# podman container checkpoint --latest --print-stats
{
"podman_checkpoint_duration": 360749,
"container_statistics": [
{
"Id": "25244244bf2efbef30fb6857ddea8cb2e5489f07eb6659e20dda117f0c466808",
"runtime_checkpoint_duration": 177222,
"criu_statistics": {
"freezing_time": 100657,
"frozen_time": 60700,
"memdump_time": 8162,
"memwrite_time": 4224,
"pages_scanned": 20561,
"pages_written": 2129
}
}
]
}
The output contains 'podman_checkpoint_duration' which contains the
number of microseconds Podman required to create the checkpoint. The
output also includes 'runtime_checkpoint_duration' which is the time
the runtime needed to checkpoint that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.
Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-09 09:15:50 +00:00
runtimeCheckpointDuration := func ( ) int64 {
if options . PrintStats {
return time . Since ( runtimeCheckpointStarted ) . Microseconds ( )
}
return 0
} ( )
return runtimeCheckpointDuration , err
2019-10-08 17:53:36 +00:00
}
Ensure Conmon is alive before waiting for exit file
This came out of a conversation with Valentin about
systemd-managed Podman. He discovered that unit files did not
properly handle cases where Conmon was dead - the ExecStopPost
`podman rm --force` line was not actually removing the container,
but interestingly, adding a `podman cleanup --rm` line would
remove it. Both of these commands do the same thing (minus the
`podman cleanup --rm` command not force-removing running
containers).
Without a running Conmon instance, the container process is still
running (assuming you killed Conmon with SIGKILL and it had no
chance to kill the container it managed), but you can still kill
the container itself with `podman stop` - Conmon is not involved,
only the OCI Runtime. (`podman rm --force` and `podman stop` use
the same code to kill the container). The problem comes when we
want to get the container's exit code - we expect Conmon to make
us an exit file, which it's obviously not going to do, being
dead. The first `podman rm` would fail because of this, but
importantly, it would (after failing to retrieve the exit code
correctly) set container status to Exited, so that the second
`podman cleanup` process would succeed.
To make sure the first `podman rm --force` succeeds, we need to
catch the case where Conmon is already dead, and instead of
waiting for an exit file that will never come, immediately set
the Stopped state and remove an error that can be caught and
handled.
Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-06-08 17:34:12 +00:00
func ( r * ConmonOCIRuntime ) CheckConmonRunning ( ctr * Container ) ( bool , error ) {
if ctr . state . ConmonPID == 0 {
// If the container is running or paused, assume Conmon is
// running. We didn't record Conmon PID on some old versions, so
// that is likely what's going on...
// Unusual enough that we should print a warning message though.
if ctr . ensureState ( define . ContainerStateRunning , define . ContainerStatePaused ) {
logrus . Warnf ( "Conmon PID is not set, but container is running!" )
return true , nil
}
// Container's not running, so conmon PID being unset is
// expected. Conmon is not running.
return false , nil
}
// We have a conmon PID. Ping it with signal 0.
if err := unix . Kill ( ctr . state . ConmonPID , 0 ) ; err != nil {
if err == unix . ESRCH {
return false , nil
}
2022-09-10 11:40:39 +00:00
return false , fmt . Errorf ( "pinging container %s conmon with signal 0: %w" , ctr . ID ( ) , err )
Ensure Conmon is alive before waiting for exit file
This came out of a conversation with Valentin about
systemd-managed Podman. He discovered that unit files did not
properly handle cases where Conmon was dead - the ExecStopPost
`podman rm --force` line was not actually removing the container,
but interestingly, adding a `podman cleanup --rm` line would
remove it. Both of these commands do the same thing (minus the
`podman cleanup --rm` command not force-removing running
containers).
Without a running Conmon instance, the container process is still
running (assuming you killed Conmon with SIGKILL and it had no
chance to kill the container it managed), but you can still kill
the container itself with `podman stop` - Conmon is not involved,
only the OCI Runtime. (`podman rm --force` and `podman stop` use
the same code to kill the container). The problem comes when we
want to get the container's exit code - we expect Conmon to make
us an exit file, which it's obviously not going to do, being
dead. The first `podman rm` would fail because of this, but
importantly, it would (after failing to retrieve the exit code
correctly) set container status to Exited, so that the second
`podman cleanup` process would succeed.
To make sure the first `podman rm --force` succeeds, we need to
catch the case where Conmon is already dead, and instead of
waiting for an exit file that will never come, immediately set
the Stopped state and remove an error that can be caught and
handled.
Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-06-08 17:34:12 +00:00
}
return true , nil
}
2019-10-08 17:53:36 +00:00
// SupportsCheckpoint checks if the OCI runtime supports checkpointing
// containers.
func ( r * ConmonOCIRuntime ) SupportsCheckpoint ( ) bool {
2021-02-25 15:34:12 +00:00
return crutils . CRRuntimeSupportsCheckpointRestore ( r . path )
2019-10-08 17:53:36 +00:00
}
// SupportsJSONErrors checks if the OCI runtime supports JSON-formatted error
// messages.
func ( r * ConmonOCIRuntime ) SupportsJSONErrors ( ) bool {
return r . supportsJSON
}
// SupportsNoCgroups checks if the OCI runtime supports running containers
// without cgroups (the --cgroup-manager=disabled flag).
func ( r * ConmonOCIRuntime ) SupportsNoCgroups ( ) bool {
return r . supportsNoCgroups
}
2020-04-15 18:48:53 +00:00
// SupportsKVM checks if the OCI runtime supports running containers
// without KVM separation
func ( r * ConmonOCIRuntime ) SupportsKVM ( ) bool {
return r . supportsKVM
}
2019-10-08 17:53:36 +00:00
// AttachSocketPath is the path to a single container's attach socket.
func ( r * ConmonOCIRuntime ) AttachSocketPath ( ctr * Container ) ( string , error ) {
if ctr == nil {
2022-07-05 09:42:22 +00:00
return "" , fmt . Errorf ( "must provide a valid container to get attach socket path: %w" , define . ErrInvalidArg )
2019-10-08 17:53:36 +00:00
}
2021-01-14 18:53:28 +00:00
return filepath . Join ( ctr . bundlePath ( ) , "attach" ) , nil
2019-10-08 17:53:36 +00:00
}
// ExitFilePath is the path to a container's exit file.
func ( r * ConmonOCIRuntime ) ExitFilePath ( ctr * Container ) ( string , error ) {
if ctr == nil {
2022-07-05 09:42:22 +00:00
return "" , fmt . Errorf ( "must provide a valid container to get exit file path: %w" , define . ErrInvalidArg )
2019-10-08 17:53:36 +00:00
}
return filepath . Join ( r . exitsDir , ctr . ID ( ) ) , nil
}
// RuntimeInfo provides information on the runtime.
2020-03-15 16:53:59 +00:00
func ( r * ConmonOCIRuntime ) RuntimeInfo ( ) ( * define . ConmonInfo , * define . OCIRuntimeInfo , error ) {
2019-10-08 17:53:36 +00:00
runtimePackage := packageVersion ( r . path )
conmonPackage := packageVersion ( r . conmonPath )
runtimeVersion , err := r . getOCIRuntimeVersion ( )
if err != nil {
2022-09-10 11:40:39 +00:00
return nil , nil , fmt . Errorf ( "getting version of OCI runtime %s: %w" , r . name , err )
2019-10-08 17:53:36 +00:00
}
conmonVersion , err := r . getConmonVersion ( )
if err != nil {
2022-09-10 11:40:39 +00:00
return nil , nil , fmt . Errorf ( "getting conmon version: %w" , err )
2019-10-08 17:53:36 +00:00
}
2020-03-15 16:53:59 +00:00
conmon := define . ConmonInfo {
Package : conmonPackage ,
Path : r . conmonPath ,
Version : conmonVersion ,
2019-10-08 17:53:36 +00:00
}
2020-03-15 16:53:59 +00:00
ocirt := define . OCIRuntimeInfo {
Name : r . name ,
Path : r . path ,
Package : runtimePackage ,
Version : runtimeVersion ,
2019-10-08 17:53:36 +00:00
}
2020-03-15 16:53:59 +00:00
return & conmon , & ocirt , nil
2019-10-08 17:53:36 +00:00
}
// makeAccessible changes the path permission and each parent directory to have --x--x--x
func makeAccessible ( path string , uid , gid int ) error {
for ; path != "/" ; path = filepath . Dir ( path ) {
st , err := os . Stat ( path )
if err != nil {
if os . IsNotExist ( err ) {
return nil
}
return err
}
if int ( st . Sys ( ) . ( * syscall . Stat_t ) . Uid ) == uid && int ( st . Sys ( ) . ( * syscall . Stat_t ) . Gid ) == gid {
continue
}
if st . Mode ( ) & 0111 != 0111 {
if err := os . Chmod ( path , st . Mode ( ) | 0111 ) ; err != nil {
return err
}
}
}
return nil
}
// Wait for a container which has been sent a signal to stop
func waitContainerStop ( ctr * Container , timeout time . Duration ) error {
return waitPidStop ( ctr . state . PID , timeout )
}
// Wait for a given PID to stop
func waitPidStop ( pid int , timeout time . Duration ) error {
2023-01-18 12:36:25 +00:00
timer := time . NewTimer ( timeout )
for {
select {
case <- timer . C :
return fmt . Errorf ( "given PID did not die within timeout" )
default :
if err := unix . Kill ( pid , 0 ) ; err != nil {
if err == unix . ESRCH {
return nil
2019-10-08 17:53:36 +00:00
}
2023-01-18 12:36:25 +00:00
logrus . Errorf ( "Pinging PID %d with signal 0: %v" , pid , err )
2019-10-08 17:53:36 +00:00
}
2023-01-19 11:31:37 +00:00
time . Sleep ( 10 * time . Millisecond )
2019-10-08 17:53:36 +00:00
}
}
}
2020-01-07 12:41:56 +00:00
func ( r * ConmonOCIRuntime ) getLogTag ( ctr * Container ) ( string , error ) {
logTag := ctr . LogTag ( )
if logTag == "" {
return "" , nil
}
data , err := ctr . inspectLocked ( false )
if err != nil {
2022-03-21 13:49:47 +00:00
// FIXME: this error should probably be returned
2022-06-13 14:08:46 +00:00
return "" , nil //nolint: nilerr
2020-01-07 12:41:56 +00:00
}
tmpl , err := template . New ( "container" ) . Parse ( logTag )
if err != nil {
2022-07-05 09:42:22 +00:00
return "" , fmt . Errorf ( "template parsing error %s: %w" , logTag , err )
2020-01-07 12:41:56 +00:00
}
var b bytes . Buffer
err = tmpl . Execute ( & b , data )
if err != nil {
return "" , err
}
return b . String ( ) , nil
}
2019-10-08 17:53:36 +00:00
// createOCIContainer generates this container's main conmon instance and prepares it for starting
2021-11-09 21:21:07 +00:00
func ( r * ConmonOCIRuntime ) createOCIContainer ( ctr * Container , restoreOptions * ContainerCheckpointOptions ) ( int64 , error ) {
2019-10-08 17:53:36 +00:00
var stderrBuf bytes . Buffer
runtimeDir , err := util . GetRuntimeDir ( )
if err != nil {
2021-11-09 21:21:07 +00:00
return 0 , err
2019-10-08 17:53:36 +00:00
}
parentSyncPipe , childSyncPipe , err := newPipe ( )
if err != nil {
2022-09-10 11:40:39 +00:00
return 0 , fmt . Errorf ( "creating socket pair: %w" , err )
2019-10-08 17:53:36 +00:00
}
defer errorhandling . CloseQuiet ( parentSyncPipe )
childStartPipe , parentStartPipe , err := newPipe ( )
if err != nil {
2022-09-10 11:40:39 +00:00
return 0 , fmt . Errorf ( "creating socket pair for start pipe: %w" , err )
2019-10-08 17:53:36 +00:00
}
defer errorhandling . CloseQuiet ( parentStartPipe )
var ociLog string
if logrus . GetLevel ( ) != logrus . DebugLevel && r . supportsJSON {
ociLog = filepath . Join ( ctr . state . RunDir , "oci-log" )
}
2020-01-07 12:41:56 +00:00
logTag , err := r . getLogTag ( ctr )
if err != nil {
2021-11-09 21:21:07 +00:00
return 0 , err
2020-01-07 12:41:56 +00:00
}
2020-06-18 11:56:30 +00:00
if ctr . config . CgroupsMode == cgroupSplit {
2021-10-27 15:30:24 +00:00
if err := utils . MoveUnderCgroupSubtree ( "runtime" ) ; err != nil {
2021-11-09 21:21:07 +00:00
return 0 , err
2020-06-18 11:56:30 +00:00
}
}
2021-05-10 08:44:15 +00:00
pidfile := ctr . config . PidFile
if pidfile == "" {
pidfile = filepath . Join ( ctr . state . RunDir , "pidfile" )
}
args := r . sharedConmonArgs ( ctr , ctr . ID ( ) , ctr . bundlePath ( ) , pidfile , ctr . LogPath ( ) , r . exitsDir , ociLog , ctr . LogDriver ( ) , logTag )
2019-10-08 17:53:36 +00:00
2022-08-05 12:22:54 +00:00
if ctr . config . SdNotifyMode == define . SdNotifyModeContainer && ctr . config . SdNotifySocket != "" {
args = append ( args , fmt . Sprintf ( "--sdnotify-socket=%s" , ctr . config . SdNotifySocket ) )
2021-03-24 11:49:29 +00:00
}
2022-10-11 14:38:42 +00:00
if ctr . Terminal ( ) {
2019-10-08 17:53:36 +00:00
args = append ( args , "-t" )
} else if ctr . config . Stdin {
args = append ( args , "-i" )
}
2021-04-22 19:38:36 +00:00
if ctr . config . Timeout > 0 {
args = append ( args , fmt . Sprintf ( "--timeout=%d" , ctr . config . Timeout ) )
}
2020-12-22 18:06:31 +00:00
if ! r . enableKeyring {
args = append ( args , "--no-new-keyring" )
}
2019-10-08 17:53:36 +00:00
if ctr . config . ConmonPidFile != "" {
args = append ( args , "--conmon-pidfile" , ctr . config . ConmonPidFile )
}
if r . noPivot {
args = append ( args , "--no-pivot" )
}
2021-11-18 19:22:33 +00:00
exitCommand , err := specgenutil . CreateExitCommandArgs ( ctr . runtime . storageConfig , ctr . runtime . config , logrus . IsLevelEnabled ( logrus . DebugLevel ) , ctr . AutoRemove ( ) , false )
if err != nil {
return 0 , err
}
exitCommand = append ( exitCommand , ctr . config . ID )
args = append ( args , "--exit-command" , exitCommand [ 0 ] )
for _ , arg := range exitCommand [ 1 : ] {
args = append ( args , [ ] string { "--exit-command-arg" , arg } ... )
2019-10-08 17:53:36 +00:00
}
2021-08-24 15:34:47 +00:00
// Pass down the LISTEN_* environment (see #10443).
preserveFDs := ctr . config . PreserveFDs
if val := os . Getenv ( "LISTEN_FDS" ) ; val != "" {
if ctr . config . PreserveFDs > 0 {
logrus . Warnf ( "Ignoring LISTEN_FDS to preserve custom user-specified FDs" )
} else {
fds , err := strconv . Atoi ( val )
if err != nil {
2021-11-09 21:21:07 +00:00
return 0 , fmt . Errorf ( "converting LISTEN_FDS=%s: %w" , val , err )
2021-08-24 15:34:47 +00:00
}
preserveFDs = uint ( fds )
}
}
if preserveFDs > 0 {
args = append ( args , formatRuntimeOpts ( "--preserve-fds" , fmt . Sprintf ( "%d" , preserveFDs ) ) ... )
2020-06-16 18:22:05 +00:00
}
2019-10-08 17:53:36 +00:00
if restoreOptions != nil {
args = append ( args , "--restore" , ctr . CheckpointPath ( ) )
if restoreOptions . TCPEstablished {
args = append ( args , "--runtime-opt" , "--tcp-established" )
}
2021-11-17 11:53:12 +00:00
if restoreOptions . FileLocks {
args = append ( args , "--runtime-opt" , "--file-locks" )
}
2021-07-12 11:43:45 +00:00
if restoreOptions . Pod != "" {
mountLabel := ctr . config . MountLabel
processLabel := ctr . config . ProcessLabel
if mountLabel != "" {
args = append (
args ,
"--runtime-opt" ,
fmt . Sprintf (
"--lsm-mount-context=%s" ,
mountLabel ,
) ,
)
}
if processLabel != "" {
args = append (
args ,
"--runtime-opt" ,
fmt . Sprintf (
"--lsm-profile=selinux:%s" ,
processLabel ,
) ,
)
}
}
2019-10-08 17:53:36 +00:00
}
logrus . WithFields ( logrus . Fields {
"args" : args ,
} ) . Debugf ( "running conmon: %s" , r . conmonPath )
cmd := exec . Command ( r . conmonPath , args ... )
cmd . SysProcAttr = & syscall . SysProcAttr {
Setpgid : true ,
}
// TODO this is probably a really bad idea for some uses
// Make this configurable
cmd . Stdin = os . Stdin
cmd . Stdout = os . Stdout
cmd . Stderr = os . Stderr
2022-10-11 14:38:42 +00:00
if ctr . Terminal ( ) {
2019-10-08 17:53:36 +00:00
cmd . Stderr = & stderrBuf
}
// 0, 1 and 2 are stdin, stdout and stderr
2022-04-22 13:10:13 +00:00
conmonEnv := r . configureConmonEnv ( runtimeDir )
2019-10-08 17:53:36 +00:00
2020-07-29 14:27:12 +00:00
var filesToClose [ ] * os . File
2021-08-24 15:34:47 +00:00
if preserveFDs > 0 {
for fd := 3 ; fd < int ( 3 + preserveFDs ) ; fd ++ {
2020-07-29 14:27:12 +00:00
f := os . NewFile ( uintptr ( fd ) , fmt . Sprintf ( "fd-%d" , fd ) )
filesToClose = append ( filesToClose , f )
cmd . ExtraFiles = append ( cmd . ExtraFiles , f )
2020-06-16 18:22:05 +00:00
}
}
2020-01-13 12:01:45 +00:00
cmd . Env = r . conmonEnv
2020-06-16 18:22:05 +00:00
// we don't want to step on users fds they asked to preserve
// Since 0-2 are used for stdio, start the fds we pass in at preserveFDs+3
2021-08-24 15:34:47 +00:00
cmd . Env = append ( cmd . Env , fmt . Sprintf ( "_OCI_SYNCPIPE=%d" , preserveFDs + 3 ) , fmt . Sprintf ( "_OCI_STARTPIPE=%d" , preserveFDs + 4 ) )
2019-10-08 17:53:36 +00:00
cmd . Env = append ( cmd . Env , conmonEnv ... )
cmd . ExtraFiles = append ( cmd . ExtraFiles , childSyncPipe , childStartPipe )
2020-08-26 09:07:51 +00:00
if r . reservePorts && ! rootless . IsRootless ( ) && ! ctr . config . NetMode . IsSlirp4netns ( ) {
2022-03-17 17:54:47 +00:00
ports , err := bindPorts ( ctr . convertPortMappings ( ) )
2019-10-08 17:53:36 +00:00
if err != nil {
2021-11-09 21:21:07 +00:00
return 0 , err
2019-10-08 17:53:36 +00:00
}
2021-09-14 09:13:28 +00:00
filesToClose = append ( filesToClose , ports ... )
2019-10-08 17:53:36 +00:00
// Leak the port we bound in the conmon process. These fd's won't be used
// by the container and conmon will keep the ports busy so that another
// process cannot use them.
cmd . ExtraFiles = append ( cmd . ExtraFiles , ports ... )
}
2020-08-26 09:07:51 +00:00
if ctr . config . NetMode . IsSlirp4netns ( ) || rootless . IsRootless ( ) {
2019-10-08 17:53:36 +00:00
if ctr . config . PostConfigureNetNS {
2021-09-28 15:01:22 +00:00
havePortMapping := len ( ctr . config . PortMappings ) > 0
2020-02-18 10:46:27 +00:00
if havePortMapping {
ctr . rootlessPortSyncR , ctr . rootlessPortSyncW , err = os . Pipe ( )
if err != nil {
2022-07-05 09:42:22 +00:00
return 0 , fmt . Errorf ( "failed to create rootless port sync pipe: %w" , err )
2020-02-18 10:46:27 +00:00
}
}
2019-10-08 17:53:36 +00:00
ctr . rootlessSlirpSyncR , ctr . rootlessSlirpSyncW , err = os . Pipe ( )
if err != nil {
2022-07-05 09:42:22 +00:00
return 0 , fmt . Errorf ( "failed to create rootless network sync pipe: %w" , err )
2019-10-08 17:53:36 +00:00
}
} else {
if ctr . rootlessSlirpSyncR != nil {
defer errorhandling . CloseQuiet ( ctr . rootlessSlirpSyncR )
}
if ctr . rootlessSlirpSyncW != nil {
defer errorhandling . CloseQuiet ( ctr . rootlessSlirpSyncW )
}
}
// Leak one end in conmon, the other one will be leaked into slirp4netns
cmd . ExtraFiles = append ( cmd . ExtraFiles , ctr . rootlessSlirpSyncW )
2019-11-28 14:33:42 +00:00
if ctr . rootlessPortSyncW != nil {
defer errorhandling . CloseQuiet ( ctr . rootlessPortSyncW )
// Leak one end in conmon, the other one will be leaked into rootlessport
cmd . ExtraFiles = append ( cmd . ExtraFiles , ctr . rootlessPortSyncW )
}
2019-10-08 17:53:36 +00:00
}
2021-11-09 21:21:07 +00:00
var runtimeRestoreStarted time . Time
if restoreOptions != nil {
runtimeRestoreStarted = time . Now ( )
}
2022-01-31 17:05:43 +00:00
err = startCommand ( cmd , ctr )
2021-03-24 11:49:29 +00:00
2019-10-08 17:53:36 +00:00
// regardless of whether we errored or not, we no longer need the children pipes
childSyncPipe . Close ( )
childStartPipe . Close ( )
if err != nil {
2021-11-09 21:21:07 +00:00
return 0 , err
2019-10-08 17:53:36 +00:00
}
2019-10-21 21:04:27 +00:00
if err := r . moveConmonToCgroupAndSignal ( ctr , cmd , parentStartPipe ) ; err != nil {
2021-11-09 21:21:07 +00:00
return 0 , err
2019-10-08 17:53:36 +00:00
}
/* Wait for initial setup and fork, and reap child */
err = cmd . Wait ( )
if err != nil {
2021-11-09 21:21:07 +00:00
return 0 , err
2019-10-08 17:53:36 +00:00
}
2022-01-06 13:32:33 +00:00
pid , err := readConmonPipeData ( r . name , parentSyncPipe , ociLog )
2020-03-09 13:49:27 +00:00
if err != nil {
2019-10-08 17:53:36 +00:00
if err2 := r . DeleteContainer ( ctr ) ; err2 != nil {
2021-09-22 13:45:15 +00:00
logrus . Errorf ( "Removing container %s from runtime after creation failed" , ctr . ID ( ) )
2019-10-08 17:53:36 +00:00
}
2021-11-09 21:21:07 +00:00
return 0 , err
2019-10-08 17:53:36 +00:00
}
2020-03-09 13:49:27 +00:00
ctr . state . PID = pid
2019-10-08 17:53:36 +00:00
conmonPID , err := readConmonPidFile ( ctr . config . ConmonPidFile )
if err != nil {
2021-09-22 13:45:15 +00:00
logrus . Warnf ( "Error reading conmon pid file for container %s: %v" , ctr . ID ( ) , err )
2019-10-08 17:53:36 +00:00
} else if conmonPID > 0 {
// conmon not having a pid file is a valid state, so don't set it if we don't have it
logrus . Infof ( "Got Conmon PID as %d" , conmonPID )
ctr . state . ConmonPID = conmonPID
}
2021-11-09 21:21:07 +00:00
runtimeRestoreDuration := func ( ) int64 {
if restoreOptions != nil && restoreOptions . PrintStats {
return time . Since ( runtimeRestoreStarted ) . Microseconds ( )
}
return 0
} ( )
2020-07-29 14:27:12 +00:00
// These fds were passed down to the runtime. Close them
// and not interfere
for _ , f := range filesToClose {
errorhandling . CloseQuiet ( f )
2020-06-16 18:22:05 +00:00
}
2021-11-09 21:21:07 +00:00
return runtimeRestoreDuration , nil
2019-10-08 17:53:36 +00:00
}
// configureConmonEnv gets the environment values to add to conmon's exec struct
// TODO this may want to be less hardcoded/more configurable in the future
2022-04-22 13:10:13 +00:00
func ( r * ConmonOCIRuntime ) configureConmonEnv ( runtimeDir string ) [ ] string {
2021-01-11 12:48:02 +00:00
var env [ ] string
for _ , e := range os . Environ ( ) {
if strings . HasPrefix ( e , "LC_" ) {
env = append ( env , e )
}
}
2022-09-10 17:30:31 +00:00
if path , ok := os . LookupEnv ( "PATH" ) ; ok {
env = append ( env , fmt . Sprintf ( "PATH=%s" , path ) )
}
if conf , ok := os . LookupEnv ( "CONTAINERS_CONF" ) ; ok {
2022-01-19 02:20:53 +00:00
env = append ( env , fmt . Sprintf ( "CONTAINERS_CONF=%s" , conf ) )
}
2022-09-10 17:30:31 +00:00
if conf , ok := os . LookupEnv ( "CONTAINERS_HELPER_BINARY_DIR" ) ; ok {
env = append ( env , fmt . Sprintf ( "CONTAINERS_HELPER_BINARY_DIR=%s" , conf ) )
}
2019-10-08 17:53:36 +00:00
env = append ( env , fmt . Sprintf ( "XDG_RUNTIME_DIR=%s" , runtimeDir ) )
env = append ( env , fmt . Sprintf ( "_CONTAINERS_USERNS_CONFIGURED=%s" , os . Getenv ( "_CONTAINERS_USERNS_CONFIGURED" ) ) )
env = append ( env , fmt . Sprintf ( "_CONTAINERS_ROOTLESS_UID=%s" , os . Getenv ( "_CONTAINERS_ROOTLESS_UID" ) ) )
2020-10-31 11:53:58 +00:00
home := homedir . Get ( )
if home != "" {
env = append ( env , fmt . Sprintf ( "HOME=%s" , home ) )
2019-10-08 17:53:36 +00:00
}
2021-08-24 15:34:47 +00:00
return env
2019-10-08 17:53:36 +00:00
}
// sharedConmonArgs takes common arguments for exec and create/restore and formats them for the conmon CLI
2020-06-10 18:35:00 +00:00
func ( r * ConmonOCIRuntime ) sharedConmonArgs ( ctr * Container , cuuid , bundlePath , pidPath , logPath , exitDir , ociLogPath , logDriver , logTag string ) [ ] string {
2019-10-08 17:53:36 +00:00
// set the conmon API version to be able to use the correct sync struct keys
2020-05-20 13:48:16 +00:00
args := [ ] string {
"--api-version" , "1" ,
"-c" , ctr . ID ( ) ,
"-u" , cuuid ,
"-r" , r . path ,
"-b" , bundlePath ,
"-p" , pidPath ,
"-n" , ctr . Name ( ) ,
"--exit-dir" , exitDir ,
2021-01-14 18:53:28 +00:00
"--full-attach" ,
2020-05-20 13:48:16 +00:00
}
2020-08-21 17:56:50 +00:00
if len ( r . runtimeFlags ) > 0 {
rFlags := [ ] string { }
for _ , arg := range r . runtimeFlags {
rFlags = append ( rFlags , "--runtime-arg" , arg )
}
args = append ( args , rFlags ... )
}
2020-05-20 13:48:16 +00:00
2020-10-08 19:25:06 +00:00
if ctr . CgroupManager ( ) == config . SystemdCgroupsManager && ! ctr . config . NoCgroups && ctr . config . CgroupsMode != cgroupSplit {
2019-10-08 17:53:36 +00:00
args = append ( args , "-s" )
}
2020-06-10 18:35:00 +00:00
var logDriverArg string
switch logDriver {
2020-04-16 13:39:34 +00:00
case define . JournaldLogging :
2020-06-10 18:35:00 +00:00
logDriverArg = define . JournaldLogging
case define . NoLogging :
logDriverArg = define . NoLogging
2021-09-01 09:36:26 +00:00
case define . PassthroughLogging :
logDriverArg = define . PassthroughLogging
2021-01-12 14:16:12 +00:00
//lint:ignore ST1015 the default case has to be here
2022-09-30 12:25:31 +00:00
default : //nolint:gocritic
2019-10-08 17:53:36 +00:00
// No case here should happen except JSONLogging, but keep this here in case the options are extended
logrus . Errorf ( "%s logging specified but not supported. Choosing k8s-file logging instead" , ctr . LogDriver ( ) )
fallthrough
case "" :
// to get here, either a user would specify `--log-driver ""`, or this came from another place in libpod
// since the former case is obscure, and the latter case isn't an error, let's silently fallthrough
fallthrough
2022-06-10 10:50:27 +00:00
case define . JSONLogging :
fallthrough
2020-04-16 13:39:34 +00:00
case define . KubernetesLogging :
2020-06-10 18:35:00 +00:00
logDriverArg = fmt . Sprintf ( "%s:%s" , define . KubernetesLogging , logPath )
2019-10-08 17:53:36 +00:00
}
2020-06-10 18:35:00 +00:00
args = append ( args , "-l" , logDriverArg )
2019-10-08 17:53:36 +00:00
logLevel := logrus . GetLevel ( )
args = append ( args , "--log-level" , logLevel . String ( ) )
if logLevel == logrus . DebugLevel {
logrus . Debugf ( "%s messages will be logged to syslog" , r . conmonPath )
args = append ( args , "--syslog" )
}
2020-10-05 20:33:09 +00:00
size := r . logSizeMax
if ctr . config . LogSize > 0 {
size = ctr . config . LogSize
}
if size > 0 {
args = append ( args , "--log-size-max" , fmt . Sprintf ( "%v" , size ) )
}
2019-10-08 17:53:36 +00:00
if ociLogPath != "" {
args = append ( args , "--runtime-arg" , "--log-format=json" , "--runtime-arg" , "--log" , fmt . Sprintf ( "--runtime-arg=%s" , ociLogPath ) )
}
2020-01-07 12:41:56 +00:00
if logTag != "" {
args = append ( args , "--log-tag" , logTag )
}
2019-10-08 17:53:36 +00:00
if ctr . config . NoCgroups {
2022-01-13 19:51:06 +00:00
logrus . Debugf ( "Running with no Cgroups" )
2019-10-08 17:53:36 +00:00
args = append ( args , "--runtime-arg" , "--cgroup-manager" , "--runtime-arg" , "disabled" )
}
return args
}
2022-01-31 17:05:43 +00:00
func startCommand ( cmd * exec . Cmd , ctr * Container ) error {
2022-07-11 19:59:32 +00:00
// Make sure to unset the NOTIFY_SOCKET and reset it afterwards if needed.
2021-03-24 11:49:29 +00:00
switch ctr . config . SdNotifyMode {
case define . SdNotifyModeContainer , define . SdNotifyModeIgnore :
2022-08-05 12:22:54 +00:00
if prev := os . Getenv ( "NOTIFY_SOCKET" ) ; prev != "" {
2021-03-24 11:49:29 +00:00
if err := os . Unsetenv ( "NOTIFY_SOCKET" ) ; err != nil {
logrus . Warnf ( "Error unsetting NOTIFY_SOCKET %v" , err )
}
defer func ( ) {
2022-08-05 12:22:54 +00:00
if err := os . Setenv ( "NOTIFY_SOCKET" , prev ) ; err != nil {
logrus . Errorf ( "Resetting NOTIFY_SOCKET=%s" , prev )
2021-03-24 11:49:29 +00:00
}
} ( )
}
}
2022-01-31 17:05:43 +00:00
return cmd . Start ( )
2019-10-08 17:53:36 +00:00
}
2020-07-09 17:50:01 +00:00
// newPipe creates a unix socket pair for communication.
// Returns two files - first is parent, second is child.
func newPipe ( ) ( * os . File , * os . File , error ) {
2019-10-08 17:53:36 +00:00
fds , err := unix . Socketpair ( unix . AF_LOCAL , unix . SOCK_SEQPACKET | unix . SOCK_CLOEXEC , 0 )
if err != nil {
return nil , nil , err
}
return os . NewFile ( uintptr ( fds [ 1 ] ) , "parent" ) , os . NewFile ( uintptr ( fds [ 0 ] ) , "child" ) , nil
}
// readConmonPidFile attempts to read conmon's pid from its pid file
func readConmonPidFile ( pidFile string ) ( int , error ) {
// Let's try reading the Conmon pid at the same time.
if pidFile != "" {
2022-09-20 13:59:28 +00:00
contents , err := os . ReadFile ( pidFile )
2019-10-08 17:53:36 +00:00
if err != nil {
return - 1 , err
}
// Convert it to an int
conmonPID , err := strconv . Atoi ( string ( contents ) )
if err != nil {
return - 1 , err
}
return conmonPID , nil
}
return 0 , nil
}
// readConmonPipeData attempts to read a syncInfo struct from the pipe
2022-01-06 13:32:33 +00:00
func readConmonPipeData ( runtimeName string , pipe * os . File , ociLog string ) ( int , error ) {
2019-10-08 17:53:36 +00:00
// syncInfo is used to return data from monitor process to daemon
type syncInfo struct {
Data int ` json:"data" `
Message string ` json:"message,omitempty" `
}
// Wait to get container pid from conmon
type syncStruct struct {
si * syncInfo
err error
}
ch := make ( chan syncStruct )
go func ( ) {
var si * syncInfo
rdr := bufio . NewReader ( pipe )
b , err := rdr . ReadBytes ( '\n' )
2022-03-24 10:59:50 +00:00
// ignore EOF here, error is returned even when data was read
// if it is no valid json unmarshal will fail below
if err != nil && ! errors . Is ( err , io . EOF ) {
2019-10-08 17:53:36 +00:00
ch <- syncStruct { err : err }
}
if err := json . Unmarshal ( b , & si ) ; err != nil {
2022-03-24 10:59:50 +00:00
ch <- syncStruct { err : fmt . Errorf ( "conmon bytes %q: %w" , string ( b ) , err ) }
2019-10-08 17:53:36 +00:00
return
}
ch <- syncStruct { si : si }
} ( )
2022-11-24 15:44:15 +00:00
data := - 1
2019-10-08 17:53:36 +00:00
select {
case ss := <- ch :
if ss . err != nil {
if ociLog != "" {
2022-09-20 13:59:28 +00:00
ociLogData , err := os . ReadFile ( ociLog )
2019-10-08 17:53:36 +00:00
if err == nil {
var ociErr ociError
if err := json . Unmarshal ( ociLogData , & ociErr ) ; err == nil {
2022-01-06 13:32:33 +00:00
return - 1 , getOCIRuntimeError ( runtimeName , ociErr . Msg )
2019-10-08 17:53:36 +00:00
}
}
}
2022-07-05 09:42:22 +00:00
return - 1 , fmt . Errorf ( "container create failed (no logs from conmon): %w" , ss . err )
2019-10-08 17:53:36 +00:00
}
logrus . Debugf ( "Received: %d" , ss . si . Data )
if ss . si . Data < 0 {
if ociLog != "" {
2022-09-20 13:59:28 +00:00
ociLogData , err := os . ReadFile ( ociLog )
2019-10-08 17:53:36 +00:00
if err == nil {
var ociErr ociError
if err := json . Unmarshal ( ociLogData , & ociErr ) ; err == nil {
2022-01-06 13:32:33 +00:00
return ss . si . Data , getOCIRuntimeError ( runtimeName , ociErr . Msg )
2019-10-08 17:53:36 +00:00
}
}
}
// If we failed to parse the JSON errors, then print the output as it is
if ss . si . Message != "" {
2022-01-06 13:32:33 +00:00
return ss . si . Data , getOCIRuntimeError ( runtimeName , ss . si . Message )
2019-10-08 17:53:36 +00:00
}
2022-07-05 09:42:22 +00:00
return ss . si . Data , fmt . Errorf ( "container create failed: %w" , define . ErrInternal )
2019-10-08 17:53:36 +00:00
}
data = ss . si . Data
2019-10-21 17:48:23 +00:00
case <- time . After ( define . ContainerCreateTimeout ) :
2022-07-05 09:42:22 +00:00
return - 1 , fmt . Errorf ( "container creation timeout: %w" , define . ErrInternal )
2019-10-08 17:53:36 +00:00
}
2020-03-09 13:49:27 +00:00
return data , nil
2019-10-08 17:53:36 +00:00
}
2020-12-21 22:48:43 +00:00
// writeConmonPipeData writes nonce data to a pipe
2019-10-08 17:53:36 +00:00
func writeConmonPipeData ( pipe * os . File ) error {
someData := [ ] byte { 0 }
_ , err := pipe . Write ( someData )
return err
}
// formatRuntimeOpts prepends opts passed to it with --runtime-opt for passing to conmon
func formatRuntimeOpts ( opts ... string ) [ ] string {
args := make ( [ ] string , 0 , len ( opts ) * 2 )
for _ , o := range opts {
args = append ( args , "--runtime-opt" , o )
}
return args
}
// getConmonVersion returns a string representation of the conmon version.
func ( r * ConmonOCIRuntime ) getConmonVersion ( ) ( string , error ) {
output , err := utils . ExecCmd ( r . conmonPath , "--version" )
if err != nil {
return "" , err
}
return strings . TrimSuffix ( strings . Replace ( output , "\n" , ", " , 1 ) , "\n" ) , nil
}
// getOCIRuntimeVersion returns a string representation of the OCI runtime's
// version.
func ( r * ConmonOCIRuntime ) getOCIRuntimeVersion ( ) ( string , error ) {
output , err := utils . ExecCmd ( r . path , "--version" )
if err != nil {
return "" , err
}
return strings . TrimSuffix ( output , "\n" ) , nil
}
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
// Copy data from container to HTTP connection, for terminal attach.
// Container is the container's attach socket connection, http is a buffer for
// the HTTP connection. cid is the ID of the container the attach session is
// running for (used solely for error messages).
func httpAttachTerminalCopy ( container * net . UnixConn , http * bufio . ReadWriter , cid string ) error {
buf := make ( [ ] byte , bufferSize )
for {
numR , err := container . Read ( buf )
2020-05-08 15:33:44 +00:00
logrus . Debugf ( "Read fd(%d) %d/%d bytes for container %s" , int ( buf [ 0 ] ) , numR , len ( buf ) , cid )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
if numR > 0 {
switch buf [ 0 ] {
case AttachPipeStdout :
// Do nothing
default :
logrus . Errorf ( "Received unexpected attach type %+d, discarding %d bytes" , buf [ 0 ] , numR )
continue
}
numW , err2 := http . Write ( buf [ 1 : numR ] )
if err2 != nil {
if err != nil {
2021-09-22 13:45:15 +00:00
logrus . Errorf ( "Reading container %s STDOUT: %v" , cid , err )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
}
return err2
} else if numW + 1 != numR {
return io . ErrShortWrite
}
// We need to force the buffer to write immediately, so
// there isn't a delay on the terminal side.
if err2 := http . Flush ( ) ; err2 != nil {
if err != nil {
2021-09-22 13:45:15 +00:00
logrus . Errorf ( "Reading container %s STDOUT: %v" , cid , err )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
}
return err2
}
}
if err != nil {
if err == io . EOF {
return nil
}
return err
}
}
}
// Copy data from a container to an HTTP connection, for non-terminal attach.
// Appends a header to multiplex input.
func httpAttachNonTerminalCopy ( container * net . UnixConn , http * bufio . ReadWriter , cid string , stdin , stdout , stderr bool ) error {
buf := make ( [ ] byte , bufferSize )
for {
numR , err := container . Read ( buf )
if numR > 0 {
2020-04-07 20:52:47 +00:00
var headerBuf [ ] byte
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
2020-04-07 20:52:47 +00:00
// Subtract 1 because we strip the first byte (used for
// multiplexing by Conmon).
headerLen := uint32 ( numR - 1 )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
// Practically speaking, we could make this buf[0] - 1,
2022-07-11 19:59:32 +00:00
// but we need to validate it anyway.
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
switch buf [ 0 ] {
case AttachPipeStdin :
2020-04-07 20:52:47 +00:00
headerBuf = makeHTTPAttachHeader ( 0 , headerLen )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
if ! stdin {
continue
}
case AttachPipeStdout :
if ! stdout {
continue
}
2020-04-07 20:52:47 +00:00
headerBuf = makeHTTPAttachHeader ( 1 , headerLen )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
case AttachPipeStderr :
if ! stderr {
continue
}
2020-04-07 20:52:47 +00:00
headerBuf = makeHTTPAttachHeader ( 2 , headerLen )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
default :
logrus . Errorf ( "Received unexpected attach type %+d, discarding %d bytes" , buf [ 0 ] , numR )
continue
}
numH , err2 := http . Write ( headerBuf )
if err2 != nil {
if err != nil {
2021-09-22 13:45:15 +00:00
logrus . Errorf ( "Reading container %s standard streams: %v" , cid , err )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
}
return err2
}
// Hardcoding header length is pretty gross, but
// fast. Should be safe, as this is a fixed part
// of the protocol.
if numH != 8 {
if err != nil {
2021-09-22 13:45:15 +00:00
logrus . Errorf ( "Reading container %s standard streams: %v" , cid , err )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
}
return io . ErrShortWrite
}
numW , err2 := http . Write ( buf [ 1 : numR ] )
if err2 != nil {
if err != nil {
2021-09-22 13:45:15 +00:00
logrus . Errorf ( "Reading container %s standard streams: %v" , cid , err )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
}
return err2
} else if numW + 1 != numR {
if err != nil {
2021-09-22 13:45:15 +00:00
logrus . Errorf ( "Reading container %s standard streams: %v" , cid , err )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
}
return io . ErrShortWrite
}
// We need to force the buffer to write immediately, so
// there isn't a delay on the terminal side.
if err2 := http . Flush ( ) ; err2 != nil {
if err != nil {
2021-09-22 13:45:15 +00:00
logrus . Errorf ( "Reading container %s STDOUT: %v" , cid , err )
Add an API for Attach over HTTP API
The new APIv2 branch provides an HTTP-based remote API to Podman.
The requirements of this are, unfortunately, incompatible with
the existing Attach API. For non-terminal attach, we need append
a header to what was copied from the container, to multiplex
STDOUT and STDERR; to do this with the old API, we'd need to copy
into an intermediate buffer first, to handle the headers.
To avoid this, provide a new API to handle all aspects of
terminal and non-terminal attach, including closing the hijacked
HTTP connection. This might be a bit too specific, but for now,
it seems to be the simplest approach.
At the same time, add a Resize endpoint. This needs to be a
separate endpoint, so our existing channel approach does not work
here.
I wanted to rework the rest of attach at the same time (some
parts of it, particularly how we start the Attach session and how
we do resizing, are (in my opinion) handled much better here.
That may still be on the table, but I wanted to avoid breaking
existing APIs in this already massive change.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-10 18:37:10 +00:00
}
return err2
}
}
if err != nil {
if err == io . EOF {
return nil
}
return err
}
}
}