Now that we have automation in place for updating the webassets
repo, this script no longer needs to build webassets. Instead,
it just updates the webassets submodule to point at the tip of
whatever branch is specified and opens the Teleport PR.
* aliases are read from global and user configs.
* we prevent Kingpin from terminating `tsh`; we handle parsing errors better.
* added support for `TELEPORT_DEBUG` env variable, changed how logging is initialized.
* debugging aliases is possible via `TELEPORT_DEBUG=1` env variable; `--debug` is ineffective as it comes into play too late.
* if alias definition calls `tsh`, we call the `Run()` function directly instead of spawning fresh `tsh`; this improves the UX.
* alias loops are detected and a proper error is shown.
* all flags are made repeatable; if only one value for a given flag is possible, the last instance of the flag will be effective.
Co-authored-by: Marek Smoliński <marek@goteleport.com>
This commit adds the Teleport operator. The operator reconciles
TeleportUsers and TeleportRoles Kubernetes resources with Users and
Roles Teleport resources.
Allow dynamic libfido2 builds via `make full FIDO2=dynamic`.
The behavior of `FIDO2=yes` remains unchanged (it does static builds). For
completeness, I've added `FIDO2=static` as an alias for "yes".
This allows dynamic builds in environments where that makes sense. For example,
it is trivial to change the [Homebrew formula][1] to depend on `libfido2` and do
`make full FIDO2=dynamic` after this change.
[1]: 4379fb3a04/Formula/teleport.rb (L43)
* Added Machine ID and Web Apps Guide.
Co-authored-by: Tim Buckley <tim@goteleport.com>
* Update reference pages with latest v9.3 content
Also, update the mongodb example
* Address review feedback
* Remove unfinished though and fix a few typos
* Tweak wording for the webapps guide
Also, fix broken lints in the go examples.
* Fix Postgres example
* Rename "webapps" guide to "database" guide (with custom app)
* Apply suggestions from code review
Co-authored-by: Noah Stride <noah.stride@goteleport.com>
* Address review feedback
* Fix go.mod issue breaking linter, remove duped postgres library
* Fix broken links
* Rephrase the final sentence a bit.
* Update docs/pages/machine-id/guides.mdx
Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com>
* First batch of review feedback
* Apply more suggestions from review
* Restore original go.sum / go.mod
* Show different version requirement text for cloud vs OSS/enterprise
* Add small note about `tbot db` limitations
* Add new Machine ID / Database Access diagram
* Fix lint errors
* Fix broken link
* Apply suggestions from code review
Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com>
* Address review feedback
* Fix whitespace
* Address review feedback
* Update docs/pages/machine-id/guides/databases.mdx
Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com>
Co-authored-by: Russell Jones <rjones@gravitational.com>
Co-authored-by: Noah Stride <noah.stride@goteleport.com>
Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com>
- Enables the docker BuildKit in an attempt to speed up builds
- Trims slightly under 2GB off image size
- Break more dependencies out into separate build stages
- Adds some simple supply-chain protections for dependencies sourced
via git. The Docker build now checks that the commit SHAs are what
we expect, and not just assume that the tags haven't changed.
- Moves the `cbindgen` build to a stage to avoid pulling in extra
dependencies not needed for the Teleport build
- Combines the `gcloud` and firestore emulator install into one step to
reduce the layer count.
- Ports some of the above the Centos7 Dockerfile.
This code was unmaintained, created issues with our build system,
and didn't actually match the behavior of Teleport's RBAC engine.
We will revisit this functionality in the future when we investigate
"acess policies as code."
Recent Rust dependency upgrades include a newer version of prost.
This new version no longer ships embedded protoc binaries, and
instead tries to build protoc from source. This would require us
to install cmake on our buildboxes. We want to avoid this and
instead leverage the version of protoc already installed.
This change was made to the standard buildbox, but the CentOS 7
buildbox was missed.
Additionally, I noticed that Rust was installed in
Dockerfile-centos7-fips, but not in Dockerfile-fips, which means
the FIPS binaries have different functionality depending on which
version you use. To correct this, I removed Rust from the CentOS 7
FIPS builds (since the Rust features are not FIPS compliant anyway).
This adds proxy peering support. A configurable setting that allows for agents
to connect to a subset of proxies and be reachable through any proxy in the
cluster. This is achieved by creating grpc connections between each proxy
server. Client connections can then be passed between proxies to the desired
agent.
Add the TOUCHID=yes Makefile toggle and enable it on Drone.
Complements #12751.
#9160
* Enable touchid builds on Drone
* Update Drone URL in error message
* Run `make dronegen`
Changes how `make pkg-tsh` works so instead of building an installer for the
`tsh` binary, placed under `/usr/local/bin`, we install an app to
`/Applications/tsh-vXXX.app` and link its `tsh` binary to `/usr/local/bin`.
The app shell is necessary to distribute a provisioning profile along with the
signed/entitled/notarized binary. All of that is required for Touch ID to work.
Naked `tsh` binaries are unable to use Touch ID, even if built with the correct
build tags.
I've elected to split the logic from `build-package.sh` into a separate script -
it already does too much as-is. `build-pkg-tsh.sh` is more idiomatic, clears
additional `shellcheck` rules and is easier to dry-run.
#9160
* Build macOS installer for tsh.app
* Add resources to build the tshdev app
Moved from e/
* Add resources to build the tsh app (prod)
* Use production values
* Remove 'tsh' mode from build-package.tsh
* Appease buildbox linter
* Clarify one-time setup
Reinstates Linux/amd64 and Centos7/amd64 builds using libfido2, now hidden
behind an explicit FIDO2 flag (similarly to FIPS).
This PR pulls in gravitational/go-libfido2#4 and adds the required pkg-config
setup so we can perform both dynamic (mostly testing) and static (tsh) builds.
Additionally, pkg-config is now the gateway for whether we run libfido2-related
tests (which should always happen in CI).
#9160
* Re-enable libfido2 builds for amd64 and Centos7
* Use pkg-config to build tsh with libfido2
* Install Centos7 libudev-zero to /usr/local/lib64
* Update gravitational/go-libfido2
* Remove /usr/local/lib from Centos PKG_CONFIG_PATH
Build `tsh` with static `libfido2`, `libcbor`,`libcrypto` and `libudev-zero`.
Dockerfiles for buildbox and Centos7 changed. FIPS and macOS to be addressed at
a later date.
Add the `tsh fido2 diag` hidden command for ease of testing.
#9160
* Update go-libfido2 and tidy modules
* Add a fido2 diagnostic command to tsh
* Add a few build artifacts to .gitignore
* Build tsh with static libfido2 in buildbox
* Build tsh with static libfido2 in centos7
* Add a few relevant cmake flags
* Use illiliti/libudev-zero
* Do multi-stage build on centos7, image tweaks
* Add `make enter/centos7`
* s/OFf/OFF/g
The PAM tag is not needed when building tsh. Moreover, it was causing
the push-build-windows-amd64 pipeline to fail since lib/teleterm imports
lib/srv/alpnproxy which in turn indirectly depends on lib/pam.
The grpc-tools package is needed to generate gRPC files for JavaScript.
However, at the moment it can't be installed on M1 MacBooks because of
missing prebuilt binaries for arm64. [1]
One of them, protoc, is already installed in our buildbox. We still need
to compile grpc_node_plugin from source though. This adds significant
overhead as we need to pull in cmake, build-essential and then about
300 MB of git repos from protocolbuffers/protobuf.
Initially, those Teleterm gRPC were generated within `make grpc` with other
files. M1 users who don't work on Teleterm would not be happy about incurring
that additional overhead, hence I extracted everything into separate target
and Dockerfile.
Teleterm proto files don't depend on any other proto files. Once grpc-tools
adds support for arm64, we'll be able to essentially almost revert this
commit and generate Teleterm gRPC files within `make grpc`.
[1] https://github.com/grpc/grpc-node/issues/1405
- Lint libfido2 (and other) Go build tags
- `make test-go` exercises the libfido2 build tag, as long as `libfido2` is present in the system
- Install `libfido2` (and dependencies) in the teleport-buildbox image
Libraries are installed from source, instead of apt or ppas, so we can guarantee deterministic (and current!) versions.
(Binary releases are not available.)
At the present moment, `librdp_client` and `libfido2` can't be used together. This is because `librdp_client` embeds
openssl/`libcrypto`, which is also a dependency for `libfido2`, causing duplicate symbol errors. In practice both
libraries never coexist in the same binary, so it's easy to sidestep the issue (`librdp_client` links to `teleport`,
while FIDO2 code is only used by `tsh`). I may be able to make them coexist, but not without changes to how go-libfido2
builds.
This change is only for linting/testing libfido2 code, I'll address `tsh` releases in a future PR.
#9160
* Install libfido2 in buildbox
libfido2 and libcbor are installed from source to make sure we get
deterministic versions (apt is outdated and ppas are likely to move
forward with time).
* Run libfido2 tests on test-go
* Lint libfido2 Go build tag
* Lint other Go build tags
* Comment build tags that break the linter
* Tidy modules
* Re-enable roletester linter
* Pass tags conditionally to golangci-lint
* Clarify and improve libfido2 wildcard
* Drop `:$LD_LIBRARY_PATH` from variable
* Replace LD_LIBRARY_PATH with `ldconfig`
* Test for ARM homebrew location too
* POC for Helm unit tests
This uses https://github.com/vbehar/helm3-unittest to define
expectations of our helm templates
* Test that enterprise is configured correctly
* Added tests for teleport-cluster
* Added tests for teleport-kube-agent
* Removed tests for teleport chart
* Add tests for teleport-cluster Deployment
* Run shorter tests first
* Fix Docker plugin installation and add update-helm-snapshots target
* Add README
* Fix lint syntax error and add some missing linters
* Add missing ImagePullPolicy to Deployment and StatefulSet
* Add Deployment tests for teleport-kube-agent
* Fix replicaCount logic
* Add clarification to values
* Add StatefulSet suite for teleport-kube-agent
* Update snapshots after merge with master
* Helm tests are quicker than bash tests
* Add tests for extraEnv
* Random space
* Tidy up formatting of multiple tests
* [debug] List helm plugins and directories
* Special case Helm linting when running in CI
* Make trailing line breaks consistent
* Special case Helm linting when running in CI
* Add contribution guidelines for Helm charts
* Add contribution guidelines to READMEs
* Deprecate old charts
* Typo
* Spacing
* Clarification
* Update examples/chart/CONTRIBUTING.md
* Don't erroneously set extraEnv for initContainers
* Rename update-helm-snapshots -> test-helm-update-snapshots for clarity
Co-authored-by: Gus Luxton <gus@goteleport.com>
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
API_IMPORT_PATH is consistently being resolved as an empty string, breaking
proto generation.
Since the path is fixed, it seems simpler to read api/go.mod and do away with
the Go program.
* Explicitly set API_IMPORT_PATH
* Delete the print-import-path program
* Read api module from api/go.mod, push variables to target
* Add certificate renewal bot
This adds a new `tbot` tool to continuously renew a set of
certificates after registering with a Teleport cluster using a
similar process to standard node joining.
This makes some modifications to user certificate generation to allow
for certificates that can be renewed beyond their original TTL, and
exposes new gRPC endpoints:
* `CreateBotJoinToken` creates a join token for a bot user
* `GenerateInitialRenewableUserCerts` exchanges a token for a set of
certificates with a new `renewable` flag set
A new `tctl` command, `tctl bots add`, creates a bot user and calls
`CreateBotJoinToken` to issue a token. A bot instance can then be
started using a provided command.
* Cert bot refactoring pass
* Use role requests to split renewable certs from end-user certs
* Add bot configuration file
* Use `teleport.dev/bot` label
* Remove `impersonator` flag on initial bot certs
* Remove unnecessary `renew` package
* Misc other cleanup
* Do not pass through `renewable` flag when role requests are set
This adds additional restrictions on when a certificate's `renewable`
flag is carried over to a new certificate. In particular, it now also
denies the flag when either role requests are present, or the
`disallowReissue` flag has been previously set.
In practice `disallow-reissue` would have prevented any undesired
behavior but this improves consistency and resolves a TODO.
* Various tbot UX improvements; render SSH config
* Fully flesh out config template rendering
* Fix rendering for SSH configuration templates
* Added `String()` impls for destination types
* Improve certificate renewal logging; show more detail
* Properly fall back to default (all) roles
* Add mode hints for files
* Add/update copyright headers
* Add stubs for tbot init and watch commands
* Add gRPC endpoints for managing bots
* Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints
* Replace `tctl bot (add|rm|ls)` implementations with gRPC calls
* Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`,
`BotGenerationLabel`
* Fix outdated destination flag in example tbot command
* Bugfix pass for demo
* Fixed a few nil pointer derefs when using config from CLI args
* Properly create destination if `--destination-dir` flag is used
* Remove improper default on CLI flag
* `DestinationConfig` is now a list of pointers
* Address first wave of review feedback
Fixes the majority of smaller issues caught by reviewers, thanks all!
* Add doc comments for bot.go functions
* Return the token TTL from CreateBot
* Split initial user cert issuance from `generateUserCerts()`
Issuing initial renewable certificate ended up requiring a lot of
hacks to skip checks that prevented anonymous bots from getting
certs even though we'd verified their identity elsewhere (via token).
This reverts all those hacks and splits initial bot cert logic into a
dedicated `generateInitialRenewableUserCerts()` function which should
make the whole process much easier to follow.
* Set bot traits to silence log messages
* tbot log message consistency pass
* Resolve lints
* Add config tests
* Remove CreateBotJoinToken endpoint
Users should instead use the CreateBot/DeleteBot endpoints.
* Create a fresh private key for every impersonated identity renewal
* Hide `config` subcommand
* Rename bot label prefix to `teleport.internal/`
* Use types.NewRole() to create bot roles
* Clean up error handling in custom YAML unmarshallers
Also, add notes about the supported YAML shapes.
* Fetch proxy host via gRPC Ping() instead of GetProxies()
* Update lib/auth/bot.go
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
* Fix some review comments
* Add renewable certificate generation checks (#10098)
* Add renewable certificate generation checks
This adds a new validation check for renewable certificates that
maintains a renewal counter as both a certificate extension and a
user label. This counter is used to ensure only a single certificate
lineage can exist: for example, if a renewable certificate is stolen,
only one copy of the certificate can be renewed as the generation
counter will not match
When renewing a certificate, first the generation counter presented
by the user (via their TLS identity) is compared to a value stored
with the associated user (in a new `teleport.dev/bot-generation`
label field). If they aren't equal, the renewal attempt fails.
Otherwise, the generation counter is incremented by 1, stored to the
database using a `CompareAndSwap()` to ensure atomicity, and set on
the generated certificate for use in future renewals.
* Add unit tests for the generation counter
This adds new unit tests to exercise the generation counter checks.
Additionally, it fixes two other renewable cert tests that were
failing.
* Remove certRequestGeneration() function
* Emit audit event when cert generations don't match
* Fully implement `tctl bots lock`
* Show bot name in `tctl bots ls`
* Lock bots when a cert generation mismatch is found
* Make CompareFailed respones from validateGenerationLabel() more actionable
* Update lib/services/local/users.go
Co-authored-by: Nic Klaassen <nic@goteleport.com>
* Backend changes for tbot IoT and AWS joining (#10360)
* backend changes
* add token permission check
* pass ctx from caller
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
* fix comment typo
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
* use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event
* Client changes for tbot IoT joining (#10397)
* client changes
* delete replaced APIs
* delete unused tbot/auth.go
* add license header
* don't unecessarily fetch host CA
* log fixes
* s/tunnelling/tunneling/
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
* auth server addresses may be proxies
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
* comment typo fix
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
* move *Server methods out of auth_with_roles.go (#10416)
Co-authored-by: Tim Buckley <tim@goteleport.com>
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
Co-authored-by: Tim Buckley <tim@goteleport.com>
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
Co-authored-by: Tim Buckley <tim@goteleport.com>
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
Co-authored-by: Nic Klaassen <nic@goteleport.com>
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
* Address another batch of review feedback
* Addres another batch of review feedback
Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()`
calls, clear some TODOs and lints, and address other misc feedback
items.
* Fix lint
* Add missing doc comments to SaveIdentity / LoadIdentity
* Remove pam tag from tbot build
* Update note about bot lock deletion
* Another pass of review feedback
Ensure all requestable roles exist when creating a bot, adjust the
default renewable cert TTL down to 1 hour, and check types during
`CompareAndSwapUser()`
Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com>
Co-authored-by: Nic Klaassen <nic@goteleport.com>
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
In order to make it easier to do post-analysis in test logs, this patch makes the CI build save the JSON-formatted logs into a Google Cloud Storage bucket.
GCB has a built-in artifact storage system, but unfortunately it only stores artifacts on successful builds. Given that we're interested on doing post-analysis on failed builds, this is unhelpful. The build scripts instead use the GCP API to explicitly upload the test logs to a bucket.
This patch
* Adds an artifact uploader using the GCP Storage API
* Updates the build yaml files to point to the appropriate artifacts & bucket
* Updates the makefile to save JSON logs to $TELEPORT/test-logs when running tests
* Adds entries to .gitignore to not automatically commit test reports
The existing test rendering filter formats the JSON outut for human consumption, meaning that we can both save the JSON logs and that humans manually running tests with via make will still get an intelligible report.
In addition to the above, this patch also:
* standardizes the build scripts on `logrus` for logging, as this was used by some dependencies.
* adds a self-test to the CI code, run as part of `make test`
* Begin Rust implementation of MS_RDPECLIP
Includes encoding/decoding support for the Clipboard Capabilities
PDU, as specified in section 2.2.2.1.
* Add support for the format list PDU (and response)
This message is the notification from one endpoint to another that
the data on the local clipboard has been updated.
* Start integrating cliprdr client
We're receiving the start of the initialization flow from the server,
but it doesn't like our response just yet. Probably missing a header
of some kind.
* Allow providing flags for the channel PDU header
This didn't appear to fix anything, but we will want to be able
to adjust those flags based on the message we're sending.
* Continue to flesh out cliprdr client
- Add docs
- Build out handlers for remaining messages
- Set the SHOW_PROTOCOL channel header flag
(this was key to getting the RDP server to respond to us)
* Respond to format data request
* Add lint and test targets for Rust code
And fix lint warnings
* Suppress collapsible_if warning for role_tester
This is a giant crepe macro, so the lint warning isn't even accurate.
* Hook up Rust/Go callbacks for passing clipboard data
* Address first round of review feedback
* Address review feedback
* Mark RFD 49 as implemented
* Add more lint coverage
golanglint-ci doesn't pick up subdirectories with their own go.mod
which left certain directories unlinted. To get around this we can
run golanglint-ci directly against those submodules.
In Rust 1.58, deriving Debug no longer counts as using a struct's
fields, so we need to allow dead_code for our structs that implement
RDP protocols. (Just because we don't use the fields doesn't mean
we shoudln't decode them)
The workspace includes both parts of the project that use Rust.
(The roletester and RDP client).
This has several advantages:
- Rust Analyzer will work on the codebase as a whole, so we get
nice development features without needing to open the Rust
projects separately
- Dependencies are resolved at the project level, ensuring that
role tester and RDP client use the same version of common
dependencies.
Starting with the Teleport 9 release, we will be versioning the
API module. This change ensures that the generated protobuf code
imports the correct version of the API by:
- introducing a small new command to print the correct version
- adding import rewrite rules to the protoc invocation
