This commit expands the usage of the caching layer
for auth server API:
* Introduces in-memory cache that is used to serve all
Auth server API requests. This is done to achieve scalability
on 10K+ node clusters, where each node fetches certificate authorities,
roles, users and join tokens. It is not possible to scale
DynamoDB backend or other backends on 10K reads per seconds
on a single shard or partition. The solution is to introduce
an in-memory cache of the backend state that is always used
for reads.
* In-memory cache has been expanded to support all resources
required by the auth server.
* Experimental `tctl top` command has been introduced to display
common single node metrics.
Replace SQLite Memory Backend with BTree
SQLite in memory backend was suffering from
high tail latencies under load (up to 8 seconds
in 99.9%-ile on load configurations).
This commit replaces the SQLite memory caching
backend with in-memory BTree backend that
brought down tail latencies to 2 seconds (99.9%-ile)
and brought overall performance improvement.
This commit implements disconnects of kubernetes
clients with expired certificates and terminates
idle sessions, according to 'disconnect_expired_cert'
and 'client_idle_timeout' existing settings making
the behavior consistent with SSH sessions.
This commit switches Teleport proxy to use impersonation
API instead of the CSR API.
This allows Teleport to work on EKS clusters, GKE and all
other CNCF compabitble clusters.
This commit updates helm chart RBAC as well.
It introduces extra configuration flag to proxy_service
configuration parameter:
```yaml
proxy_service:
# kubeconfig_file is used for scenarios
# when Teleport Proxy is deployed outside
# of the kubernetes cluster
kubeconfig_file: /path/to/kube/config
```
It deprecates similar flag in auth_service:
```yaml
auth_service:
# DEPRECATED. THIS FLAG IS IGNORED
kubeconfig_file: /path/to/kube/config
```
* Fetch groups for GSuite SSO. (#2456)
Fixes#2455
This commit adds support for fetching
groups for GSuite SSO logins via
OIDC connector interface.
If OIDC connector has a special scope:
`https://www.googleapis.com/auth/admin.directory.group.readonly`
teleport will fetch user's group membership and populate
groups claim.
* Pass kubernetes groups to the remote cluster. (#2484)
This commit allows remote cluster to reference
the kubernetes groups coming from the roles
of the main cluster in the trusted clusters
configuration.
For example, main cluster can have a user
with a role 'main' and kubernetes groups:
kube_groups: ['system:masters']
and SSH logins:
logins: ['root']
Remote cluster can choose to map
this 'main' cluster to it's own:
'remote-admin' cluster in the trusted cluster config:
role_map:
- remote: 'main'
local: 'remote-admin'
The role 'remote-admin' of the remote cluster
can now be templated to use the main cluster role main
logins and kubernetes_groups using variables:
logins: ['{{internal.logins}}']
kubernetes_groups: ['{{internal.kubernetes_groups}}']
This is possible because teleport now encodes
both values in X509 certificate metadata
and remote cluster passes these values as 'internal' traits
to the template engine.
This commit introduces several key changes to
Teleport backend and API infrastructure
in order to achieve scalability improvements
on 10K+ node deployments.
Events and plain keyspace
--------------------------
New backend interface supports events,
pagination and range queries
and moves away from buckets to
plain keyspace, what better aligns
with DynamoDB and Etcd featuring similar
interfaces.
All backend implementations are
exposing Events API, allowing
multiple subscribers to consume the same
event stream and avoid polling database.
Replacing BoltDB, Dir with SQLite
-------------------------------
BoltDB backend does not support
having two processes access the database at the
same time. This prevented Teleport
using BoltDB backend to be live reloaded.
SQLite supports reads/writes by multiple
processes and makes Dir backend obsolete
as SQLite is more efficient on larger collections,
supports transactions and can detect data
corruption.
Teleport automatically migrates data from
Bolt and Dir backends into SQLite.
GRPC API and protobuf resources
-------------------------------
GRPC API has been introduced for
the auth server. The auth server now serves both GRPC
and JSON-HTTP API on the same TLS socket and uses
the same client certificate authentication.
All future API methods should use GRPC and HTTP-JSON
API is considered obsolete.
In addition to that some resources like
Server and CertificateAuthority are now
generated from protobuf service specifications in
a way that is fully backward compatible with
original JSON spec and schema, so the same resource
can be encoded and decoded from JSON, YAML
and protobuf.
All models should be refactored
into new proto specification over time.
Streaming presence service
--------------------------
In order to cut bandwidth, nodes
are sending full updates only when changes
to labels or spec have occured, otherwise
new light-weight GRPC keep alive updates are sent
over to the presence service, reducing
bandwidth usage on multi-node deployments.
In addition to that nodes are no longer polling
auth server for certificate authority rotation
updates, instead they subscribe to event updates
to detect updates as soon as they happen.
This is a new API, so the errors are inevitable,
that's why polling is still done, but
on a way slower rate.
Fixes#1986
When deployed outside of the kubernetes cluster,
teleport now reads all configuration from kubernetes
config file, supplied via parameter.
Auth server then passes information about
target api server back to the proxy.
This commit moves proxy kubernetes configuration
to a separate nested block to provide more fine
grained settings:
```yaml
auth:
kubernetes_ca_cert_path: /tmp/custom-ca
proxy:
enabled: yes
kubernetes:
enabled: yes
public_addr: [custom.example.com:port]
api_addr: kuberentes.example.com:443
listen_addr: localhost:3026
```
1. Kubernetes config section is explicitly enabled
and disabled. It is disabled by default.
2. Public address in kubernetes section
is propagated to tsh profile
The other part of the commit updates Ping
endpoint to send proxy configuration back to
the client, including kubernetes public address
and ssh listen address.
Clients updates profile accordingly to configuration
received from the proxy.
