teleport/assets/loadtest

loadtest

Automation for the loadtest kuberentes cluster and performing Teleport load tests.

About

This automation sets up a kubernetes cluster named loadtest in the configured GCP project.
This cluster is used for, among other things, the 1k/10k scaling tests that are performed as part of Teleports manual release test plan.

Setup

Prerequisites

• Make sure you have the following tools installed:
  • terraform
  • gcloud
  • kubectl
• Make sure that you have a GCP service account key with Compute Admin, Compute Network Admin, Kubernetes Engine Admin, Kubernetes Engine Cluster Admin, and Service Account User
  • To authenticate as the service account follow these instructions
• Make sure you have reserved static ip addresses for the proxy
  • This only needs to be done once per GCP project, see the network docs for details

Creating the Cluster

First create a cluster, if you are running this automation for the first time, you may be asked to run terraform init from the cluster directory before continuing. To resize the cluster, edit terraform.tfvars as needed.

$ make create-cluster

DNS Entries

Before deploying anything to the cluster you first need to set PROXY_HOST. These variables should be the DNS names to be used for the proxy. When everything is successfully deployed you should be able to navigate to https://PROXY_HOST:3080 in your browser.

$ export PROXY_HOST=proxy.loadtest.com 

TLS Certificates

Certificates can be provisioned automatically via cert-manager or by hand.

cert-manager

If you would like to use cert-manager to automatically retrieve TLS certificates for you, create cetificate.yaml with your cert-manager.io/v1/ClusterIssuer, cert-manager.io/v1/Certificate and any secrets required for your solver.

Kubernetes secret

To manual supply TLS certificates create a tls secret, run the following:

$ kubectl create secret tls teleport-tls -n loadtest \
    --cert=path/to/cert/file \
    --key=path/to/key/file

You must also provide USE_CERT_MANAGER=no to all make commands below.

Teleport Configuration

You must supply an OIDC Connector that will be used for authentication. Create oidc.yaml before attempting to deploy Teleport to the cluster.

Deploy Teleport

etcd Backend

$ make deploy-etcd-cluster

Firestore Backend

To use the firestore backend you must have a GCP service account key with Cloud Datastore User, Cloud Datastore Index Admin Storage Object Admin permissions. Set GCP_CREDS_LOCATION to the location that you saved the service account key.

$ export GCP_CREDS_LOCATION=/path/to/service/account/key
$ make deploy-firestore-cluster

Running Tests

To run soak tests:

$ make run-soak-tests

Note: You must have enough nodes in the cluster to run the following tests. Ensure your node_count in terraform.tfvars is correctly set.

To run the 10k node scaling tests:

$ make run-scaling-test

To run the trusted cluster scaling test:

$ make run-tc-scaling-test

Cleanup

To delete the loadtest deployment:

$ make delete-deploy

To delete the entire cluster:

$ make delete-cluster