self-hosted/teleport
self-hosted/teleport
1
0
Fork 0
mirror of https://github.com/gravitational/teleport synced 2024-10-22 10:13:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add Kubernetes 5.0 migration guide (#4933)

Browse source 
* Add Kubernetes 5.0 migration guide

This guide lists 3 common pre-5.0 setups and how to migrate them to 5.0.
This commit is contained in:
Andrew Lytvynov 2020-11-24 00:15:42 +00:00 committed by GitHub
parent 9ccfab0e96
commit 835bfaa5a1
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 395 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

395
docs/5.0/kubernetes-5.0-migration.md Normal file
View file

@ -0,0 +1,395 @@
---
title: Kubernetes 5.0 Migration Guide
description: How to migrate pre-5.0 Teleport clusters to new configuration
---
# Migrating to Teleport Kubernetes Access 5.0
In release 5.0, Teleport has changed the [Kubernetes
integration](kubernetes-ssh.md) to improve configuration and user experience.
These changes are backwards compatible and there is no _required_ migration.
However, to get the most out of Teleport, you should consider migrating as
described below.
The main changes in 5.0 are:
- new `kubernetes_service` configuration section in `teleport.yaml`, decoupled
from `proxy_service`
- support for multiple Kubernetes clusters per Teleport cluster (replaces the
need to use [Trusted Clusters](admin-guide.md#trusted-clusters) to achieve this)
- RBAC support for Kubernetes clusters
Important note: `proxy_service` with an open port for Kubernetes requests is
still required. A proxy is always the public-facing gateway into the Teleport
cluster, acting as an authentication point and a connection router.
# Example scenarios
We'll describe the migration for several common Teleport configurations.
## Standalone proxy for a single cluster
In this scenario, we're running a Teleport proxy outside of the Kubernetes
cluster and connecting to it using a `kubeconfig_file`.
Before 5.0:
```yaml
# teleport.yaml
proxy_service:
kubernetes:
enabled: yes
listen_addr: 0.0.0.0:3026
kubeconfig_file: /path/to/kubeconfig
```
After 5.0:
```yaml
# teleport.yaml
proxy_service:
kube_listen_addr: 0.0.0.0:3026
kubernetes_service:
enabled: yes
listen_addr: 0.0.0.0:3027
kubeconfig_file: /path/to/kubeconfig
```
The `listen_port` for `kubernetes_service` only needs to be exposed to the
Proxies within your cluster. This port will not be directly used by the
clients.
## Proxy in a Kubernetes pod of a single cluster
In this scenario, we're running a Teleport proxy inside of the Kubernetes
cluster and connecting to it using the pod service account credentials.
Before 5.0:
```yaml
# teleport.yaml
proxy_service:
kubernetes:
enabled: yes
listen_addr: 0.0.0.0:3026
```
After 5.0:
```yaml
# teleport.yaml
proxy_service:
kube_listen_addr: 0.0.0.0:3026
kubernetes_service:
enabled: yes
listen_addr: 0.0.0.0:3027
```
## Multiple Kubernetes clusters
In this scenario, we're running multiple Trusted Teleport clusters, one for
each Kubernetes cluster.
### Recommend Option 1: Teleport Kubernetes agents in each Kubernetes cluster
This is the recommended approach for handling multiple Kubernetes clusters.
Instead of running multiple Teleport clusters (auth and proxy for each
Kubernetes cluster), we will connect multiple `kubernetes_service` agents to
the same root Teleport cluster.
Each `kubernetes_service` will create its own reverse tunnel back to a Teleport
proxy to join the cluster.
Before 5.0:
```yaml
# teleport-root.yaml
auth_service:
enabled: yes
proxy_service:
kubernetes:
enabled: yes
public_addr: [root.example.com:3026]
listen_addr: 0.0.0.0:3026
---
# teleport-leaf-1.yaml
auth_service:
enabled: yes
proxy_service:
kubernetes:
enabled: yes
public_addr: [leaf-1.example.com:3026]
listen_addr: 0.0.0.0:3026
# Not specifying kubeconfig_file, we'll assume that this process runs in a
# Kubernetes pod
---
# teleport-leaf-2.yaml
auth_service:
enabled: yes
proxy_service:
kubernetes:
enabled: yes
public_addr: [leaf-2.example.com:3026]
listen_addr: 0.0.0.0:3026
# Not specifying kubeconfig_file, we'll assume that this process runs in a
# Kubernetes pod
```
After 5.0:
```yaml
# teleport-root.yaml
auth_service:
enabled: yes
tokens:
# This is a static token, you can also use dynamic join tokens
- kube:secret_static_join_token
proxy_service:
public_addr: [root.example.com]
kube_listen_addr: 0.0.0.0:3026
---
# teleport-kube-1.yaml
teleport:
# Note: auth_servers points at the default proxy listening port.
auth_servers: ["https://root.example.com:3080"]
auth_token: secret_static_join_token
auth_service:
enabled: no
proxy_service:
enabled: no
kubernetes_service:
enabled: yes
kube_cluster_name: kube-1
# Not specifying kubeconfig_file, we'll assume that this process runs in a
# Kubernetes pod
---
# teleport-kube-2.yaml
teleport:
# Note: auth_servers points at the default proxy listening port.
auth_servers: ["https://root.example.com:3080"]
auth_token: secret_static_join_token
auth_service:
enabled: no
proxy_service:
enabled: no
kubernetes_service:
enabled: yes
kube_cluster_name: kube-2
# Not specifying kubeconfig_file, we'll assume that this process runs in a
# Kubernetes pod
```
After all the Teleport agents are running, you can view the list of Kubernetes
clusters using `tsh kube ls` and switch between them using `tsh kube login
<kube_cluster_name>`:
```sh
$ tsh kube ls
Kube Cluster Name Selected
----------------- --------
kube-1 *
kube-2
$ tsh kube login kube-2
Logged into kubernetes cluster "kube-2"
```
### Option 2: One Teleport agent connected to multiple clusters
This approach requires you to manually craft a `kubeconfig` file which contains
the exact set of Kubernetes cluster entries you wish to connect. Updates to the
set of clusters require a restart.
Instead of running multiple Teleport clusters (auth and proxy for each
Kubernetes cluster), we will connect multiple Kubernetes clusters to a single
`kubernetes_service` using `kubeconfig_file`.
Before 5.0:
```yaml
# teleport-root.yaml
auth_service:
enabled: yes
proxy_service:
kubernetes:
enabled: yes
public_addr: [root.example.com:3026]
listen_addr: 0.0.0.0:3026
---
# teleport-leaf-1.yaml
auth_service:
enabled: yes
proxy_service:
kubernetes:
enabled: yes
public_addr: [leaf-1.example.com:3026]
listen_addr: 0.0.0.0:3026
kubeconfig_file: kubeconfig-leaf-1.yaml
---
# teleport-leaf-2.yaml
auth_service:
enabled: yes
proxy_service:
kubernetes:
enabled: yes
public_addr: [leaf-2.example.com:3026]
listen_addr: 0.0.0.0:3026
kubeconfig_file: kubeconfig-leaf-2.yaml
```
After 5.0:
```yaml
# teleport.yaml
proxy_service:
public_addr: [root.example.com]
kube_listen_addr: 0.0.0.0:3026
kubernetes_service:
enabled: yes
listen_addr: 0.0.0.0:3027
kubeconfig_file: kubeconfig-all.yaml
---
# kubeconfig-all.yaml
apiVersion: v1
kind: Config
clusters:
- cluster:
certificate-authority-data: ...
server: https://10.0.0.1
name: kube-1-server
- cluster:
certificate-authority-data: ...
server: https://10.0.0.2
name: kube-2-server
contexts:
- context:
cluster: kube-1-server
user: kube-1-creds
name: kube-1
- context:
cluster: kube-2-server
user: kube-2-creds
name: kube-2
current-context: kube-1
users:
- name: kube-1-creds
user:
...cluster-specific credentials...
- name: kube-2-creds
user:
...cluster-specific credentials...
```
You can view the list of Kubernetes clusters using `tsh kube ls` and switch
between them using `tsh kube login <kube_cluster_name>`:
```sh
$ tsh kube ls
Kube Cluster Name Selected
----------------- --------
kube-1 *
kube-2
$ tsh kube login kube-2
Logged into kubernetes cluster "kube-2"
```
### Option 3: Keep using Trusted Clusters
You may keep using the Trusted Clusters as before and migrate them internally
to `kubernetes_service`, as described in the first two scenarios.
Note that `tsh kube ls` will **not** show all Kubernetes clusters across all
Trusted Clusters. It only shows the Kubernetes clusters from the Teleport
cluster you are currently logged into.
# RBAC
!!! note
Kubernetes RBAC integration only works with `kubernetes_service`. Make sure
you've migrated all the clusters as described above before proceeding.
By default, all users have access to all Kubernetes clusters (as a limited set
of Kubernetes user/group identities). In 5.0, you can restrict user access to a
subset of Kubernetes clusters using Kubernetes cluster labels and Teleport
roles.
This RBAC integration works exactly like the [integration for SSH
nodes](enterprise/ssh-rbac.md).
First, set the set of Kubernetes labels a user can access on their roles:
```yaml
# role-admin.yaml
kind: role
version: v3
metadata:
name: admin
spec:
allow:
logins: ['{% raw %}{{internal.logins}}{% endraw %}']
# Define the Kubernetes users and groups mapped to users with this role.
kubernetes_users: ["{% raw %}{{internal.kubernetes_users}}{% endraw %}"]]
kubernetes_groups: ["some-group", "{% raw %}{{internal.kubernetes_groups}}{% endraw %}"]]
# List of kubernetes labels a user will be allowed to connect to:
kubernetes_labels:
# a user can only connect to clusters marked with 'test' label:
'environment': 'test'
# the wildcard ('*') means "any cluster"
'*': '*'
# labels can be specified as a list:
'environment': ['test', 'staging']
# regular expressions are also supported, for example the equivalent
# of the list example above can be expressed as:
'environment': '^test|staging$'
# You can also strictly restrict access using deny kubernetes_labels.
deny: {}
```
Next, define Kubernetes labels on the cluster side:
```yaml
# teleport.yaml
kubernetes_service:
enabled: yes
listen_addr: 0.0.0.0:3027
kube_cluster_name: staging
# Note: if you use kubeconfig_file, the below labels apply to all Kubernetes
# clusters in that kubeconfig. To get different sets of labels, you need to
# run separate kubernetes_service instances.
labels:
environment: staging
commands:
- name: project
command: [fetch-project-name.sh]
period: 1h
```
Users with a role that matches a specific Kubernetes cluster labels will be
able to see and connect to that cluster. Other users will neither see nor
be able to connect to it.