teleport/tool/tbot/init.go

544 lines
16 KiB
Go
Raw Normal View History

Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
/*
Copyright 2022 Gravitational, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"fmt"
"os"
"os/user"
"path/filepath"
"runtime"
"strconv"
"strings"
"github.com/google/uuid"
2022-10-28 20:20:28 +00:00
"github.com/gravitational/trace"
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
"github.com/gravitational/teleport/api/constants"
"github.com/gravitational/teleport/lib/tbot/botfs"
"github.com/gravitational/teleport/lib/tbot/config"
"github.com/gravitational/teleport/lib/tbot/identity"
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
)
// RootUID is the UID of the root user
const RootUID = "0"
// aclTestFailedMessage is printed when an ACL test fails.
const aclTestFailedMessage = "ACLs are not usable for destination %s; " +
"Change the destination's ACL mode to `off` to silence this warning."
// getInitArtifacts returns a map of all desired artifacts for the destination
Machine ID: New configuration format (#27152) * Introduce new tbot output configuration * Remove code that will be included in future test refactor PR * Add config version header * Fix TestInitSymlink * Add support for `standard` database type * Set CA type * `make fix-imports` * More closely mimic original database output behaviour * Make output of additional TLS files for application output optional * Spell compatability properly * Add test for config marshalling * Fix cluster field yaml name * Fix YAML marshalling/unmarshalling * Fix sidecar invocation of tbot * Add WrapDestination helper function to protect wrappedDestination * Apply changes to sidecar * Spell Marshalling the way the linter insists * Fix some logging * Tidy mutex usage on outputRenewalCache * Fix ssh_host generation * Get rid of `destinationWrapper` * Single l in Unmarshaling * Fix operator sidecar tbot * Add UnmarshalYAML for SSHHostOutput * Use `KubernetesCluster` instead of `ClusterName` for clarity * Rename `Subtype` -> `Format` for `DatabaseOutput` * Remove a very british "u" from Behaviour * Add godoc for interfacemethods * Fix double dot in comment Co-authored-by: Michael Wilson <mike@mdwn.dev> * Use const for database formats * Reuse constant type string in Stringer * Add godoc comment explaining behaviour if no destination found * Inject executablePathGetter rather than using package level variable * Use correct case in error * Add warning for destination reuse * Try to improve confusing log message * Remove 'u' from behaviour --------- Co-authored-by: Michael Wilson <mike@mdwn.dev>
2023-06-27 08:03:38 +00:00
func getInitArtifacts(output config.Output) map[string]bool {
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
// true = directory, false = regular file
toCreate := map[string]bool{}
// Collect all base artifacts and filter for the destination.
for _, artifact := range identity.GetArtifacts() {
Add new config templates to `tbot` for databases and identity files (#11596) * Add new `identityfile` config template to `tbot` This adds a new `identityfile` config template to tbot which generates an identity file from any of the formats supported by `tctl auth sign`. It can be used by specifying one or more formats in the configuration like so: ```yaml destinations: - directory: /foo kinds: [ssh, tls] configs: - identityfile: formats: [file] ``` It requires both SSH and TLS certificates to work properly. App, Kubernetes, and Database certs are unlikely to work as they have additional cert requirements that will be added in separate PRs. Multiple formats can be specified, and each will be written to its own subdirectory within the destination using the name of the format. The particular files written inside this directory depend on the particular format selected, but n the above example, this means a file named `/foo/file/identity` is written. The files all have an `identity` prefix at the moment. This could be made configurable if desired. The `file` format can be used in conjunction with `tsh -i` and `tctl -i` to use those tools with a tbot-generated identity. Fixes #10812 * Make identityfile formats first-class config templates This promotes most of the important identityfile formats to proper config templates. User-facing `kinds` are removed to reduce confusion and several config templates are now required. * The `ssh_client` template is now required and will be added automatically in all cases if not specified. * A new required `tls_cas` template is added to always export the current Teleport server CAs in a usable format. * A new required `identity` template is added to always export an identity file usable with tsh/tctl. * New optional `cockroach`, `mongo`, and `tls` templates can export specifically-formatted TLS certs for various databases and apps. Additionally some other changes were caught during testing: * `botfs` now allows users to specify if files should be opened for reading or for writing; previously, written files were never truncated when opened for writing leading to garbage at the end of files if the length changed. Truncation isn't sane for reading so the two use-cases are now split. * Update lib/client/identityfile/identity.go Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com> * Address first batch of review comments Tweaked the `botfs.openStandard` and `botfs.openSecure` functions to accept a plain file mode, and removed a ton of boilerplate in `configtemplate.go`. * Fix problematic nil interface check in configtemplate * Clarify comment about `client.Key` DB certs * Address review feedback - Use `DatabaseCA` for database specific templates; make the `tls` template's CA configurable; write the database CA alongside the others. - Simplify nil interface check * Fix outdated var names Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
2022-05-06 01:03:05 +00:00
if artifact.Matches(identity.DestinationKinds()...) {
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
toCreate[artifact.Key] = false
}
}
// Collect all config template artifacts.
Machine ID: New configuration format (#27152) * Introduce new tbot output configuration * Remove code that will be included in future test refactor PR * Add config version header * Fix TestInitSymlink * Add support for `standard` database type * Set CA type * `make fix-imports` * More closely mimic original database output behaviour * Make output of additional TLS files for application output optional * Spell compatability properly * Add test for config marshalling * Fix cluster field yaml name * Fix YAML marshalling/unmarshalling * Fix sidecar invocation of tbot * Add WrapDestination helper function to protect wrappedDestination * Apply changes to sidecar * Spell Marshalling the way the linter insists * Fix some logging * Tidy mutex usage on outputRenewalCache * Fix ssh_host generation * Get rid of `destinationWrapper` * Single l in Unmarshaling * Fix operator sidecar tbot * Add UnmarshalYAML for SSHHostOutput * Use `KubernetesCluster` instead of `ClusterName` for clarity * Rename `Subtype` -> `Format` for `DatabaseOutput` * Remove a very british "u" from Behaviour * Add godoc for interfacemethods * Fix double dot in comment Co-authored-by: Michael Wilson <mike@mdwn.dev> * Use const for database formats * Reuse constant type string in Stringer * Add godoc comment explaining behaviour if no destination found * Inject executablePathGetter rather than using package level variable * Use correct case in error * Add warning for destination reuse * Try to improve confusing log message * Remove 'u' from behaviour --------- Co-authored-by: Michael Wilson <mike@mdwn.dev>
2023-06-27 08:03:38 +00:00
for _, fd := range output.Describe() {
toCreate[fd.Name] = fd.IsDir
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
}
Machine ID: New configuration format (#27152) * Introduce new tbot output configuration * Remove code that will be included in future test refactor PR * Add config version header * Fix TestInitSymlink * Add support for `standard` database type * Set CA type * `make fix-imports` * More closely mimic original database output behaviour * Make output of additional TLS files for application output optional * Spell compatability properly * Add test for config marshalling * Fix cluster field yaml name * Fix YAML marshalling/unmarshalling * Fix sidecar invocation of tbot * Add WrapDestination helper function to protect wrappedDestination * Apply changes to sidecar * Spell Marshalling the way the linter insists * Fix some logging * Tidy mutex usage on outputRenewalCache * Fix ssh_host generation * Get rid of `destinationWrapper` * Single l in Unmarshaling * Fix operator sidecar tbot * Add UnmarshalYAML for SSHHostOutput * Use `KubernetesCluster` instead of `ClusterName` for clarity * Rename `Subtype` -> `Format` for `DatabaseOutput` * Remove a very british "u" from Behaviour * Add godoc for interfacemethods * Fix double dot in comment Co-authored-by: Michael Wilson <mike@mdwn.dev> * Use const for database formats * Reuse constant type string in Stringer * Add godoc comment explaining behaviour if no destination found * Inject executablePathGetter rather than using package level variable * Use correct case in error * Add warning for destination reuse * Try to improve confusing log message * Remove 'u' from behaviour --------- Co-authored-by: Michael Wilson <mike@mdwn.dev>
2023-06-27 08:03:38 +00:00
return toCreate
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
}
// getExistingArtifacts fetches all entries in a destination directory
func getExistingArtifacts(dir string) (map[string]bool, error) {
existing := map[string]bool{}
entries, err := os.ReadDir(dir)
if err != nil {
return nil, trace.Wrap(err)
}
for _, entry := range entries {
existing[entry.Name()] = entry.IsDir()
}
return existing, nil
}
// diffArtifacts computes the difference of two artifact sets
func diffArtifacts(a, b map[string]bool) map[string]bool {
diff := map[string]bool{}
for k, v := range a {
if _, ok := b[k]; !ok {
diff[k] = v
}
}
return diff
}
// testACL creates a temporary file and attempts to apply an ACL to it. If the
// ACL is successfully applied, returns nil; otherwise, returns the error.
func testACL(directory string, ownerUser *user.User, opts *botfs.ACLOptions) error {
// Note: we need to create the test file in the dest dir to ensure we
// actually test the target filesystem.
id, err := uuid.NewRandom()
if err != nil {
return trace.Wrap(err)
}
testFile := filepath.Join(directory, id.String())
if err := botfs.Create(testFile, false, botfs.SymlinksInsecure); err != nil {
return trace.Wrap(err)
}
defer func() {
err := os.Remove(testFile)
if err != nil {
log.Debugf("Failed to delete ACL test file %q", testFile)
}
}()
//nolint:staticcheck // staticcheck doesn't like nop implementations in fs_other.go
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
if err := botfs.ConfigureACL(testFile, ownerUser, opts); err != nil {
return trace.Wrap(err)
}
return nil
}
// ensurePermissionsParams sets permissions options
type ensurePermissionsParams struct {
// dirPath is the directory containing the file
dirPath string
// symlinksMode configures symlink attack mitigation behavior
symlinksMode botfs.SymlinksMode
// ownerUser is the user that should own the file
ownerUser *user.User
// ownerGroup is the group that should own the file
ownerGroup *user.Group
// aclOptions contains configuration for ACLs, if they are in use. nil if
// ACLs are disabled
aclOptions *botfs.ACLOptions
// toCreate is a set of files that will be newly created, used to reduce
// log spam when configuring new files
toCreate map[string]bool
}
// ensurePermissions verifies permissions on the given path and, when
// possible, attempts to fix permissions / ACLs on any misconfigured paths.
func ensurePermissions(params *ensurePermissionsParams, key string, isDir bool) error {
path := filepath.Join(params.dirPath, key)
//nolint:staticcheck // this entirely innocuous line generates "related
// information" lints for a false positive staticcheck lint relating to
// nop function implementations in fs_other.go.
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
stat, err := os.Stat(path)
if err != nil {
return trace.Wrap(err)
}
cleanPath := filepath.Clean(path)
resolved, err := filepath.EvalSymlinks(path)
if err != nil {
return trace.Wrap(err)
}
// Precomputed flag to determine if certain log messages should be hidden.
// We expect ownership, ACLs, etc to be wrong on initial create so warnings
// are not desirable.
_, newlyCreated := params.toCreate[key]
verboseLogging := key != "" && !newlyCreated
// This is unlikely as CreateSecure() refuses to follow symlinks, but users
// could move things around after the fact.
if cleanPath != resolved {
switch params.symlinksMode {
case botfs.SymlinksSecure:
return trace.BadParameter("Path %q contains symlinks which is not "+
"allowed for security reasons.", path)
case botfs.SymlinksInsecure:
// do nothing
default:
log.Warnf("Path %q contains symlinks and may be subject to symlink "+
"attacks. If this is intentional, consider setting `symlinks: "+
"insecure` in destination config.", path)
}
}
// A few conditions we won't try to handle...
if isDir && !stat.IsDir() {
return trace.BadParameter("File %s is expected to be a directory but is a file", path)
} else if !isDir && stat.IsDir() {
return trace.BadParameter("File %s is expected to be a file but is a directory", path)
}
currentUser, err := user.Current()
if err != nil {
return trace.Wrap(err)
}
// Correct ownership.
ownedByDesiredOwner, err := botfs.IsOwnedBy(stat, params.ownerUser)
if err != nil {
log.WithError(err).Debugf("Could not determine file ownership of %q", path)
// Can't read file ownership on this platform (e.g. Windows), so always
// attempt to chown (which does work on Windows)
ownedByDesiredOwner = false
}
if !ownedByDesiredOwner {
// If we're not running as root, this will probably fail.
if currentUser.Uid != RootUID && runtime.GOOS != constants.WindowsOS {
log.Warnf("Not running as root, ownership change is likely to fail.")
}
uid, err := strconv.Atoi(params.ownerUser.Uid)
if err != nil {
return trace.Wrap(err)
}
gid, err := strconv.Atoi(params.ownerGroup.Gid)
if err != nil {
return trace.Wrap(err)
}
if verboseLogging {
log.Warnf("Ownership of %q is incorrect and will be corrected to %s", path, params.ownerUser.Username)
}
err = os.Chown(path, uid, gid)
if err != nil {
return trace.Wrap(err)
}
}
if params.aclOptions != nil {
// We can verify ACLs as any user with read, but can only correct ACLs
// as root or the owner.
// Note that we rely on VerifyACL to return some error if permissions
// are incorrect.
//nolint:staticcheck // staticcheck doesn't like nop implementations in fs_other.go
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
err = botfs.VerifyACL(path, params.aclOptions)
//nolint:staticcheck // staticcheck doesn't like nop implementations in fs_other.go
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
if err != nil && (currentUser.Uid == RootUID || currentUser.Uid == params.ownerUser.Uid) {
if verboseLogging {
log.Warnf("ACL for %q is not correct and will be corrected: %v", path, err)
}
return trace.Wrap(botfs.ConfigureACL(path, params.ownerUser, params.aclOptions))
} else if err != nil {
log.Errorf("ACL for %q is incorrect but `tbot init` must be run "+
"as root or the owner (%s) to correct it: %v",
path, params.ownerUser.Username, err)
return trace.AccessDenied("Elevated permissions required")
}
// ACL is valid.
return nil
}
desiredMode := botfs.DefaultMode
if stat.IsDir() {
desiredMode = botfs.DefaultDirMode
}
// Using regular permissions, so attempt to correct the file mode.
if stat.Mode().Perm() != desiredMode {
if err := os.Chmod(path, desiredMode); err != nil {
return trace.Wrap(err, "Could not fix permissions on file %q, expected %#o", path, desiredMode)
}
log.Infof("Corrected permissions on %q from %#o to %#o", path, stat.Mode().Perm(), botfs.DefaultMode)
}
return nil
}
// parseOwnerString parses and looks up an user and group from a user:group
// owner string.
func parseOwnerString(owner string) (*user.User, *user.Group, error) {
ownerParts := strings.Split(owner, ":")
if len(ownerParts) != 2 {
return nil, nil, trace.BadParameter("invalid owner string: %q", owner)
}
ownerUser, err := user.Lookup(ownerParts[0])
if err != nil {
return nil, nil, trace.Wrap(err)
}
ownerGroup, err := user.LookupGroup(ownerParts[1])
if err != nil {
return nil, nil, trace.Wrap(err)
}
return ownerUser, ownerGroup, nil
}
// getOwner determines the user/group owner given a CLI parameter (--owner)
// and desired default value. An empty defaultOwner defaults to the current
// user and their primary group.
func getOwner(cliOwner, defaultOwner string) (*user.User, *user.Group, error) {
if cliOwner != "" {
// If --owner is set, always use it.
log.Debugf("Attempting to use explicitly requested owner: %s", cliOwner)
return parseOwnerString(cliOwner)
}
if defaultOwner != "" {
log.Debugf("Attempting to use default owner: %s", defaultOwner)
// If a default owner is specified, try it instead.
return parseOwnerString(defaultOwner)
}
log.Debugf("Will use current user as owner.")
// Otherwise, return the current user and group
currentUser, err := user.Current()
if err != nil {
return nil, nil, trace.Wrap(err)
}
currentGroup, err := user.LookupGroupId(currentUser.Gid)
if err != nil {
return nil, nil, trace.Wrap(err)
}
return currentUser, currentGroup, nil
}
// getAndTestACLOptions gets options needed to configure an ACL from CLI
// options and attempts to configure a test ACL to validate them. Ownership is
// not validated here.
func getAndTestACLOptions(cf *config.CLIConf, destDir string) (*user.User, *user.Group, *botfs.ACLOptions, error) {
if cf.BotUser == "" {
return nil, nil, nil, trace.BadParameter("--bot-user must be set")
}
if cf.ReaderUser == "" {
return nil, nil, nil, trace.BadParameter("--reader-user must be set")
}
botUser, err := user.Lookup(cf.BotUser)
if err != nil {
return nil, nil, nil, trace.Wrap(err)
}
botGroup, err := user.LookupGroupId(botUser.Gid)
if err != nil {
return nil, nil, nil, trace.Wrap(err)
}
readerUser, err := user.Lookup(cf.ReaderUser)
if err != nil {
return nil, nil, nil, trace.Wrap(err)
}
opts := botfs.ACLOptions{
BotUser: botUser,
ReaderUser: readerUser,
}
// Default to letting the bot own the destination, since by this point we
// know the bot user definitely exists and is a reasonable owner choice.
defaultOwner := fmt.Sprintf("%s:%s", botUser.Username, botGroup.Name)
ownerUser, ownerGroup, err := getOwner(cf.Owner, defaultOwner)
if err != nil {
return nil, nil, nil, trace.Wrap(err)
}
err = testACL(destDir, ownerUser, &opts)
if err != nil && trace.IsAccessDenied(err) {
return nil, nil, nil, trace.Wrap(err, "The destination %q does not appear to be writable", destDir)
} else if err != nil {
return nil, nil, nil, trace.Wrap(err, "ACL support may need to be enabled for the filesystem.")
}
return ownerUser, ownerGroup, &opts, nil
}
func onInit(botConfig *config.BotConfig, cf *config.CLIConf) error {
Machine ID: New configuration format (#27152) * Introduce new tbot output configuration * Remove code that will be included in future test refactor PR * Add config version header * Fix TestInitSymlink * Add support for `standard` database type * Set CA type * `make fix-imports` * More closely mimic original database output behaviour * Make output of additional TLS files for application output optional * Spell compatability properly * Add test for config marshalling * Fix cluster field yaml name * Fix YAML marshalling/unmarshalling * Fix sidecar invocation of tbot * Add WrapDestination helper function to protect wrappedDestination * Apply changes to sidecar * Spell Marshalling the way the linter insists * Fix some logging * Tidy mutex usage on outputRenewalCache * Fix ssh_host generation * Get rid of `destinationWrapper` * Single l in Unmarshaling * Fix operator sidecar tbot * Add UnmarshalYAML for SSHHostOutput * Use `KubernetesCluster` instead of `ClusterName` for clarity * Rename `Subtype` -> `Format` for `DatabaseOutput` * Remove a very british "u" from Behaviour * Add godoc for interfacemethods * Fix double dot in comment Co-authored-by: Michael Wilson <mike@mdwn.dev> * Use const for database formats * Reuse constant type string in Stringer * Add godoc comment explaining behaviour if no destination found * Inject executablePathGetter rather than using package level variable * Use correct case in error * Add warning for destination reuse * Try to improve confusing log message * Remove 'u' from behaviour --------- Co-authored-by: Michael Wilson <mike@mdwn.dev>
2023-06-27 08:03:38 +00:00
var output config.Output
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
var err error
Machine ID: New configuration format (#27152) * Introduce new tbot output configuration * Remove code that will be included in future test refactor PR * Add config version header * Fix TestInitSymlink * Add support for `standard` database type * Set CA type * `make fix-imports` * More closely mimic original database output behaviour * Make output of additional TLS files for application output optional * Spell compatability properly * Add test for config marshalling * Fix cluster field yaml name * Fix YAML marshalling/unmarshalling * Fix sidecar invocation of tbot * Add WrapDestination helper function to protect wrappedDestination * Apply changes to sidecar * Spell Marshalling the way the linter insists * Fix some logging * Tidy mutex usage on outputRenewalCache * Fix ssh_host generation * Get rid of `destinationWrapper` * Single l in Unmarshaling * Fix operator sidecar tbot * Add UnmarshalYAML for SSHHostOutput * Use `KubernetesCluster` instead of `ClusterName` for clarity * Rename `Subtype` -> `Format` for `DatabaseOutput` * Remove a very british "u" from Behaviour * Add godoc for interfacemethods * Fix double dot in comment Co-authored-by: Michael Wilson <mike@mdwn.dev> * Use const for database formats * Reuse constant type string in Stringer * Add godoc comment explaining behaviour if no destination found * Inject executablePathGetter rather than using package level variable * Use correct case in error * Add warning for destination reuse * Try to improve confusing log message * Remove 'u' from behaviour --------- Co-authored-by: Michael Wilson <mike@mdwn.dev>
2023-06-27 08:03:38 +00:00
// First, resolve the correct output. If using a config file with only
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
// 1 destination we can assume we want to init that one; otherwise,
// --init-dir is required.
if cf.InitDir == "" {
Machine ID: New configuration format (#27152) * Introduce new tbot output configuration * Remove code that will be included in future test refactor PR * Add config version header * Fix TestInitSymlink * Add support for `standard` database type * Set CA type * `make fix-imports` * More closely mimic original database output behaviour * Make output of additional TLS files for application output optional * Spell compatability properly * Add test for config marshalling * Fix cluster field yaml name * Fix YAML marshalling/unmarshalling * Fix sidecar invocation of tbot * Add WrapDestination helper function to protect wrappedDestination * Apply changes to sidecar * Spell Marshalling the way the linter insists * Fix some logging * Tidy mutex usage on outputRenewalCache * Fix ssh_host generation * Get rid of `destinationWrapper` * Single l in Unmarshaling * Fix operator sidecar tbot * Add UnmarshalYAML for SSHHostOutput * Use `KubernetesCluster` instead of `ClusterName` for clarity * Rename `Subtype` -> `Format` for `DatabaseOutput` * Remove a very british "u" from Behaviour * Add godoc for interfacemethods * Fix double dot in comment Co-authored-by: Michael Wilson <mike@mdwn.dev> * Use const for database formats * Reuse constant type string in Stringer * Add godoc comment explaining behaviour if no destination found * Inject executablePathGetter rather than using package level variable * Use correct case in error * Add warning for destination reuse * Try to improve confusing log message * Remove 'u' from behaviour --------- Co-authored-by: Michael Wilson <mike@mdwn.dev>
2023-06-27 08:03:38 +00:00
if len(botConfig.Outputs) == 1 {
output = botConfig.Outputs[0]
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
} else {
Machine ID: New configuration format (#27152) * Introduce new tbot output configuration * Remove code that will be included in future test refactor PR * Add config version header * Fix TestInitSymlink * Add support for `standard` database type * Set CA type * `make fix-imports` * More closely mimic original database output behaviour * Make output of additional TLS files for application output optional * Spell compatability properly * Add test for config marshalling * Fix cluster field yaml name * Fix YAML marshalling/unmarshalling * Fix sidecar invocation of tbot * Add WrapDestination helper function to protect wrappedDestination * Apply changes to sidecar * Spell Marshalling the way the linter insists * Fix some logging * Tidy mutex usage on outputRenewalCache * Fix ssh_host generation * Get rid of `destinationWrapper` * Single l in Unmarshaling * Fix operator sidecar tbot * Add UnmarshalYAML for SSHHostOutput * Use `KubernetesCluster` instead of `ClusterName` for clarity * Rename `Subtype` -> `Format` for `DatabaseOutput` * Remove a very british "u" from Behaviour * Add godoc for interfacemethods * Fix double dot in comment Co-authored-by: Michael Wilson <mike@mdwn.dev> * Use const for database formats * Reuse constant type string in Stringer * Add godoc comment explaining behaviour if no destination found * Inject executablePathGetter rather than using package level variable * Use correct case in error * Add warning for destination reuse * Try to improve confusing log message * Remove 'u' from behaviour --------- Co-authored-by: Michael Wilson <mike@mdwn.dev>
2023-06-27 08:03:38 +00:00
return trace.BadParameter("An output to initialize must be specified with --init-dir")
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
}
} else {
Machine ID: New configuration format (#27152) * Introduce new tbot output configuration * Remove code that will be included in future test refactor PR * Add config version header * Fix TestInitSymlink * Add support for `standard` database type * Set CA type * `make fix-imports` * More closely mimic original database output behaviour * Make output of additional TLS files for application output optional * Spell compatability properly * Add test for config marshalling * Fix cluster field yaml name * Fix YAML marshalling/unmarshalling * Fix sidecar invocation of tbot * Add WrapDestination helper function to protect wrappedDestination * Apply changes to sidecar * Spell Marshalling the way the linter insists * Fix some logging * Tidy mutex usage on outputRenewalCache * Fix ssh_host generation * Get rid of `destinationWrapper` * Single l in Unmarshaling * Fix operator sidecar tbot * Add UnmarshalYAML for SSHHostOutput * Use `KubernetesCluster` instead of `ClusterName` for clarity * Rename `Subtype` -> `Format` for `DatabaseOutput` * Remove a very british "u" from Behaviour * Add godoc for interfacemethods * Fix double dot in comment Co-authored-by: Michael Wilson <mike@mdwn.dev> * Use const for database formats * Reuse constant type string in Stringer * Add godoc comment explaining behaviour if no destination found * Inject executablePathGetter rather than using package level variable * Use correct case in error * Add warning for destination reuse * Try to improve confusing log message * Remove 'u' from behaviour --------- Co-authored-by: Michael Wilson <mike@mdwn.dev>
2023-06-27 08:03:38 +00:00
output, err = botConfig.GetOutputByPath(cf.InitDir)
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
if err != nil {
return trace.WrapWithMessage(err, "Could not find specified destination %q", cf.InitDir)
}
Machine ID: New configuration format (#27152) * Introduce new tbot output configuration * Remove code that will be included in future test refactor PR * Add config version header * Fix TestInitSymlink * Add support for `standard` database type * Set CA type * `make fix-imports` * More closely mimic original database output behaviour * Make output of additional TLS files for application output optional * Spell compatability properly * Add test for config marshalling * Fix cluster field yaml name * Fix YAML marshalling/unmarshalling * Fix sidecar invocation of tbot * Add WrapDestination helper function to protect wrappedDestination * Apply changes to sidecar * Spell Marshalling the way the linter insists * Fix some logging * Tidy mutex usage on outputRenewalCache * Fix ssh_host generation * Get rid of `destinationWrapper` * Single l in Unmarshaling * Fix operator sidecar tbot * Add UnmarshalYAML for SSHHostOutput * Use `KubernetesCluster` instead of `ClusterName` for clarity * Rename `Subtype` -> `Format` for `DatabaseOutput` * Remove a very british "u" from Behaviour * Add godoc for interfacemethods * Fix double dot in comment Co-authored-by: Michael Wilson <mike@mdwn.dev> * Use const for database formats * Reuse constant type string in Stringer * Add godoc comment explaining behaviour if no destination found * Inject executablePathGetter rather than using package level variable * Use correct case in error * Add warning for destination reuse * Try to improve confusing log message * Remove 'u' from behaviour --------- Co-authored-by: Michael Wilson <mike@mdwn.dev>
2023-06-27 08:03:38 +00:00
if output == nil {
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
// TODO: in the future if/when other backends are supported,
// destination might be nil because the user tried to enter a non
// filesystem path, so this error message could be misleading.
return trace.NotFound("Cannot initialize destination %q because "+
"it has not been configured.", cf.InitDir)
}
}
Machine ID: New configuration format (#27152) * Introduce new tbot output configuration * Remove code that will be included in future test refactor PR * Add config version header * Fix TestInitSymlink * Add support for `standard` database type * Set CA type * `make fix-imports` * More closely mimic original database output behaviour * Make output of additional TLS files for application output optional * Spell compatability properly * Add test for config marshalling * Fix cluster field yaml name * Fix YAML marshalling/unmarshalling * Fix sidecar invocation of tbot * Add WrapDestination helper function to protect wrappedDestination * Apply changes to sidecar * Spell Marshalling the way the linter insists * Fix some logging * Tidy mutex usage on outputRenewalCache * Fix ssh_host generation * Get rid of `destinationWrapper` * Single l in Unmarshaling * Fix operator sidecar tbot * Add UnmarshalYAML for SSHHostOutput * Use `KubernetesCluster` instead of `ClusterName` for clarity * Rename `Subtype` -> `Format` for `DatabaseOutput` * Remove a very british "u" from Behaviour * Add godoc for interfacemethods * Fix double dot in comment Co-authored-by: Michael Wilson <mike@mdwn.dev> * Use const for database formats * Reuse constant type string in Stringer * Add godoc comment explaining behaviour if no destination found * Inject executablePathGetter rather than using package level variable * Use correct case in error * Add warning for destination reuse * Try to improve confusing log message * Remove 'u' from behaviour --------- Co-authored-by: Michael Wilson <mike@mdwn.dev>
2023-06-27 08:03:38 +00:00
destImpl := output.GetDestination()
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
destDir, ok := destImpl.(*config.DestinationDirectory)
if !ok {
return trace.BadParameter("`tbot init` only supports directory destinations")
}
log.Infof("Initializing destination: %s", destImpl)
// Create the directory if needed. We haven't checked directory ownership,
// but it will fail when the ACLs are created if anything is misconfigured.
Machine ID: New configuration format (#27152) * Introduce new tbot output configuration * Remove code that will be included in future test refactor PR * Add config version header * Fix TestInitSymlink * Add support for `standard` database type * Set CA type * `make fix-imports` * More closely mimic original database output behaviour * Make output of additional TLS files for application output optional * Spell compatability properly * Add test for config marshalling * Fix cluster field yaml name * Fix YAML marshalling/unmarshalling * Fix sidecar invocation of tbot * Add WrapDestination helper function to protect wrappedDestination * Apply changes to sidecar * Spell Marshalling the way the linter insists * Fix some logging * Tidy mutex usage on outputRenewalCache * Fix ssh_host generation * Get rid of `destinationWrapper` * Single l in Unmarshaling * Fix operator sidecar tbot * Add UnmarshalYAML for SSHHostOutput * Use `KubernetesCluster` instead of `ClusterName` for clarity * Rename `Subtype` -> `Format` for `DatabaseOutput` * Remove a very british "u" from Behaviour * Add godoc for interfacemethods * Fix double dot in comment Co-authored-by: Michael Wilson <mike@mdwn.dev> * Use const for database formats * Reuse constant type string in Stringer * Add godoc comment explaining behaviour if no destination found * Inject executablePathGetter rather than using package level variable * Use correct case in error * Add warning for destination reuse * Try to improve confusing log message * Remove 'u' from behaviour --------- Co-authored-by: Michael Wilson <mike@mdwn.dev>
2023-06-27 08:03:38 +00:00
if err := output.Init(); err != nil {
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
return trace.Wrap(err)
}
// Next, test if we want + have ACL support, and set aclOpts if we do.
// Desired owner is derived from the ACL mode.
var aclOpts *botfs.ACLOptions
var ownerUser *user.User
var ownerGroup *user.Group
switch destDir.ACLs {
case botfs.ACLRequired, botfs.ACLTry:
log.Debug("Testing for ACL support...")
// Awkward control flow here, but we want these to fail together.
ownerUser, ownerGroup, aclOpts, err = getAndTestACLOptions(cf, destDir.Path)
if err != nil {
if destDir.ACLs == botfs.ACLRequired {
// ACLs were specifically requested (vs "try" mode), so fail.
return trace.Wrap(err, aclTestFailedMessage, destImpl)
}
// Otherwise, fall back to no ACL with a warning.
log.WithError(err).Warnf(aclTestFailedMessage, destImpl)
aclOpts = nil
// We'll also need to re-fetch the owner as the defaults are
// different in the fallback case.
ownerUser, ownerGroup, err = getOwner(cf.Owner, "")
if err != nil {
return trace.Wrap(err)
}
} else if aclOpts.ReaderUser.Uid == ownerUser.Uid {
log.Warnf("The destination owner (%s) and reader (%s) are the "+
"same. This will break OpenSSH.", aclOpts.ReaderUser.Username,
ownerUser.Username)
}
default:
log.Info("ACLs disabled for this destination.")
ownerUser, ownerGroup, err = getOwner(cf.Owner, "")
if err != nil {
return trace.Wrap(err)
}
}
// Next, resolve what we want and what we already have.
Machine ID: New configuration format (#27152) * Introduce new tbot output configuration * Remove code that will be included in future test refactor PR * Add config version header * Fix TestInitSymlink * Add support for `standard` database type * Set CA type * `make fix-imports` * More closely mimic original database output behaviour * Make output of additional TLS files for application output optional * Spell compatability properly * Add test for config marshalling * Fix cluster field yaml name * Fix YAML marshalling/unmarshalling * Fix sidecar invocation of tbot * Add WrapDestination helper function to protect wrappedDestination * Apply changes to sidecar * Spell Marshalling the way the linter insists * Fix some logging * Tidy mutex usage on outputRenewalCache * Fix ssh_host generation * Get rid of `destinationWrapper` * Single l in Unmarshaling * Fix operator sidecar tbot * Add UnmarshalYAML for SSHHostOutput * Use `KubernetesCluster` instead of `ClusterName` for clarity * Rename `Subtype` -> `Format` for `DatabaseOutput` * Remove a very british "u" from Behaviour * Add godoc for interfacemethods * Fix double dot in comment Co-authored-by: Michael Wilson <mike@mdwn.dev> * Use const for database formats * Reuse constant type string in Stringer * Add godoc comment explaining behaviour if no destination found * Inject executablePathGetter rather than using package level variable * Use correct case in error * Add warning for destination reuse * Try to improve confusing log message * Remove 'u' from behaviour --------- Co-authored-by: Michael Wilson <mike@mdwn.dev>
2023-06-27 08:03:38 +00:00
desired := getInitArtifacts(output)
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
existing, err := getExistingArtifacts(destDir.Path)
if err != nil {
return trace.Wrap(err)
}
toCreate := diffArtifacts(desired, existing)
toRemove := diffArtifacts(existing, desired)
// Based on this, create any new files.
if len(toCreate) > 0 {
log.Infof("Attempting to create: %v", toCreate)
for key, isDir := range toCreate {
path := filepath.Join(destDir.Path, key)
if err := botfs.Create(path, isDir, destDir.Symlinks); err != nil {
return trace.Wrap(err)
}
log.Infof("Created: %s", path)
}
} else {
log.Info("Nothing to create.")
}
// ... and warn about / remove any unneeded files.
if len(toRemove) > 0 && cf.Clean {
log.Infof("Attempting to remove: %v", toRemove)
var errors []error
for key := range toRemove {
path := filepath.Join(destDir.Path, key)
if err := os.RemoveAll(path); err != nil {
errors = append(errors, err)
} else {
log.Infof("Removed: %s", path)
}
}
if err := trace.NewAggregate(errors...); err != nil {
return trace.Wrap(err)
}
} else if len(toRemove) > 0 {
log.Warnf("Unexpected files found in destination directory, consider " +
"removing it manually or rerunning `tbot init` with the `--clean` " +
"flag.")
} else {
log.Info("Nothing to remove.")
}
params := ensurePermissionsParams{
dirPath: destDir.Path,
aclOptions: aclOpts,
ownerUser: ownerUser,
ownerGroup: ownerGroup,
symlinksMode: destDir.Symlinks,
toCreate: toCreate,
}
// Check and set permissions on the directory itself.
if err := ensurePermissions(&params, "", true); err != nil {
return trace.Wrap(err)
}
// Lastly, set and check permissions on all the desired files.
for key, isDir := range desired {
if err := ensurePermissions(&params, key, isDir); err != nil {
return trace.Wrap(err)
}
}
Machine ID: New configuration format (#27152) * Introduce new tbot output configuration * Remove code that will be included in future test refactor PR * Add config version header * Fix TestInitSymlink * Add support for `standard` database type * Set CA type * `make fix-imports` * More closely mimic original database output behaviour * Make output of additional TLS files for application output optional * Spell compatability properly * Add test for config marshalling * Fix cluster field yaml name * Fix YAML marshalling/unmarshalling * Fix sidecar invocation of tbot * Add WrapDestination helper function to protect wrappedDestination * Apply changes to sidecar * Spell Marshalling the way the linter insists * Fix some logging * Tidy mutex usage on outputRenewalCache * Fix ssh_host generation * Get rid of `destinationWrapper` * Single l in Unmarshaling * Fix operator sidecar tbot * Add UnmarshalYAML for SSHHostOutput * Use `KubernetesCluster` instead of `ClusterName` for clarity * Rename `Subtype` -> `Format` for `DatabaseOutput` * Remove a very british "u" from Behaviour * Add godoc for interfacemethods * Fix double dot in comment Co-authored-by: Michael Wilson <mike@mdwn.dev> * Use const for database formats * Reuse constant type string in Stringer * Add godoc comment explaining behaviour if no destination found * Inject executablePathGetter rather than using package level variable * Use correct case in error * Add warning for destination reuse * Try to improve confusing log message * Remove 'u' from behaviour --------- Co-authored-by: Michael Wilson <mike@mdwn.dev>
2023-06-27 08:03:38 +00:00
log.Infof("destination %s has been initialized. Note that these files "+
Implement `tbot init` subcommand and ACL management (#10289) * Add certificate renewal bot This adds a new `tbot` tool to continuously renew a set of certificates after registering with a Teleport cluster using a similar process to standard node joining. This makes some modifications to user certificate generation to allow for certificates that can be renewed beyond their original TTL, and exposes new gRPC endpoints: * `CreateBotJoinToken` creates a join token for a bot user * `GenerateInitialRenewableUserCerts` exchanges a token for a set of certificates with a new `renewable` flag set A new `tctl` command, `tctl bots add`, creates a bot user and calls `CreateBotJoinToken` to issue a token. A bot instance can then be started using a provided command. * Cert bot refactoring pass * Use role requests to split renewable certs from end-user certs * Add bot configuration file * Use `teleport.dev/bot` label * Remove `impersonator` flag on initial bot certs * Remove unnecessary `renew` package * Misc other cleanup * Do not pass through `renewable` flag when role requests are set This adds additional restrictions on when a certificate's `renewable` flag is carried over to a new certificate. In particular, it now also denies the flag when either role requests are present, or the `disallowReissue` flag has been previously set. In practice `disallow-reissue` would have prevented any undesired behavior but this improves consistency and resolves a TODO. * Various tbot UX improvements; render SSH config * Fully flesh out config template rendering * Fix rendering for SSH configuration templates * Added `String()` impls for destination types * Improve certificate renewal logging; show more detail * Properly fall back to default (all) roles * Add mode hints for files * Add/update copyright headers * Add stubs for tbot init and watch commands * Add gRPC endpoints for managing bots * Add `CreateBot`, `DeleteBot`, and `GetBotUsers` gRPC endpoints * Replace `tctl bot (add|rm|ls)` implementations with gRPC calls * Define a few new constants, `DefaultBotJoinTTL`, `BotLabel`, `BotGenerationLabel` * Fix outdated destination flag in example tbot command * Bugfix pass for demo * Fixed a few nil pointer derefs when using config from CLI args * Properly create destination if `--destination-dir` flag is used * Remove improper default on CLI flag * `DestinationConfig` is now a list of pointers * Address first wave of review feedback Fixes the majority of smaller issues caught by reviewers, thanks all! * Add doc comments for bot.go functions * Return the token TTL from CreateBot * Split initial user cert issuance from `generateUserCerts()` Issuing initial renewable certificate ended up requiring a lot of hacks to skip checks that prevented anonymous bots from getting certs even though we'd verified their identity elsewhere (via token). This reverts all those hacks and splits initial bot cert logic into a dedicated `generateInitialRenewableUserCerts()` function which should make the whole process much easier to follow. * Set bot traits to silence log messages * tbot log message consistency pass * Implement `tbot init` subcommand This adds a new CLI subcommand to initialize a tbot destination directory by creating required files ahead of time and assigning proper permissions (and ACLs, where possible). * Resolve lints * Add config tests * Remove CreateBotJoinToken endpoint Users should instead use the CreateBot/DeleteBot endpoints. * Create a fresh private key for every impersonated identity renewal * Hide `config` subcommand * Rename bot label prefix to `teleport.internal/` * Use types.NewRole() to create bot roles * Clean up error handling in custom YAML unmarshallers Also, add notes about the supported YAML shapes. * Fetch proxy host via gRPC Ping() instead of GetProxies() * Update lib/auth/bot.go Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Fix some review comments * Add renewable certificate generation checks (#10098) * Add renewable certificate generation checks This adds a new validation check for renewable certificates that maintains a renewal counter as both a certificate extension and a user label. This counter is used to ensure only a single certificate lineage can exist: for example, if a renewable certificate is stolen, only one copy of the certificate can be renewed as the generation counter will not match When renewing a certificate, first the generation counter presented by the user (via their TLS identity) is compared to a value stored with the associated user (in a new `teleport.dev/bot-generation` label field). If they aren't equal, the renewal attempt fails. Otherwise, the generation counter is incremented by 1, stored to the database using a `CompareAndSwap()` to ensure atomicity, and set on the generated certificate for use in future renewals. * Add unit tests for the generation counter This adds new unit tests to exercise the generation counter checks. Additionally, it fixes two other renewable cert tests that were failing. * Remove certRequestGeneration() function * Emit audit event when cert generations don't match * Fully implement `tctl bots lock` * Show bot name in `tctl bots ls` * Lock bots when a cert generation mismatch is found * Make CompareFailed respones from validateGenerationLabel() more actionable * Update lib/services/local/users.go Co-authored-by: Nic Klaassen <nic@goteleport.com> * Backend changes for tbot IoT and AWS joining (#10360) * backend changes * add token permission check * pass ctx from caller Co-authored-by: Roman Tkachenko <roman@goteleport.com> * fix comment typo Co-authored-by: Roman Tkachenko <roman@goteleport.com> * use UserMetadata instead of Identity in RenewableCertificateGenerationMismatch event * Client changes for tbot IoT joining (#10397) * client changes * delete replaced APIs * delete unused tbot/auth.go * add license header * don't unecessarily fetch host CA * log fixes * s/tunnelling/tunneling/ Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * auth server addresses may be proxies Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * comment typo fix Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * move *Server methods out of auth_with_roles.go (#10416) Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Tim Buckley <tim@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> * Address another batch of review feedback * Addres another batch of review feedback Add `Role.SetMetadata()`, simplify more `trace.WrapWithMessage()` calls, clear some TODOs and lints, and address other misc feedback items. * Fix lint * Add missing doc comments to SaveIdentity / LoadIdentity * Remove pam tag from tbot build * Update note about bot lock deletion * Another pass of review feedback Ensure all requestable roles exist when creating a bot, adjust the default renewable cert TTL down to 1 hour, and check types during `CompareAndSwapUser()` * Remove ModeHint * Rename Identity.Cert and Identity.XCert * Add `symlinks` flag to tbot config The optional symlinks flag for directory destinations allows users to opt in / out of whichever symlink attack hardening mode is selected by default. * Add mostly-working secure implementation of botfs.Create/Write This adds symlink mode selection (secure, try-secure, insecure) and Linux `Create()`/`Write()` implementations to open files safely. * Add configurable ACL modes and verify ACL support in tbot init * Initialize destinations at startup and test before renewal This initializes destinations at startup (to create directories if not using `tbot init`) and tests them to ensure the bot can write _before_ attempting to renew certificates; this should prevent most accidental generation counter locks. * Hide watch for now * Issue a new identity if a token change is detected * Warn if identity appears to be expired on startup * Fully implement ACL Verify and Configure - Fully implements ACL support for Linux - Adds bot-side verification support to ensure ACLs are configured properly at runtime. - Gracefully falls back to no ACLs if the platform / filesystem doesn't support them - Clear up outstanding lints * Make `tbot init` work without a config file * Show init instructions in tctl bots add Also: - Make --bot-user a flag in init (the tctl instructions were confusing otherwise) - Handle IsOwnedBy sanely on unsupported platforms - Add Bold colorizing support * Clear some TODOs and rephrase tctl help * Fix typo * Fix token hash detection bug * Actually read and write certs with symlink enforcement Also, fix a config loading bug where CheckAndSetDefaults() wasn't being called in all cases with CLI destinations. * Add workaround for OpenSSH permissions check with ACLs OpenSSH has an overly-paranoid permissions check that forces key files to be exclusively owner-readable. Unfortunately, for POSIX compatibility purposes, when ACLs are set, the ACL mask is set as the group permissions. This effectively makes any ACL incompatible with OpenSSH. However, OpenSSH's check does have an escape hatch: it only applies if the current user is the owner of the file. Therefore, this change tweaks the `tbot init` flow to create files as root, owned by a separate user (either `nobody` or even the bot user), with ACL permissions granting both the bot and reader user access to the certificates. This effectively bypasses OpenSSH's permissions check and should preserve our security boundaries. * Fix lints * Fix an improper directory chmod to 0600 if ACL test fails * First pass of tbot init unit tests * Add symlink tests and fix bug with resolving the default owner * Fix err misuse * Fix an ACL error if the bot or reader user is the owner. * Fix typo * Fix missing error case in VerifyACL causing unreadable directories * Address review feedback - Rename ACLOn -> ACLRequired - Simplify fs_linux.Read() - Add missing fs_other.Read() - Hoist renewal loop logic into its own function - A few misc bugfixes * Apply suggestions from code review Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com> * Address review feedback - Only log syscall warning once - Formatting and wording changes - Improve error handling for `--clean` * Fix lint error * Fix imports in fs_other * Fix possible nil pointer deref if storage is unset * Use the bot user as default owner This is more likely to be a safe owner choice than `nobody:nobody`. * Apply suggestions from code review Co-authored-by: Roman Tkachenko <roman@goteleport.com> * Code review fixes Co-authored-by: Zac Bergquist <zmb3@users.noreply.github.com> Co-authored-by: Nic Klaassen <nic@goteleport.com> Co-authored-by: Roman Tkachenko <roman@goteleport.com> Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
2022-03-10 06:09:01 +00:00
"will be empty and invalid until the bot issues certificates.",
destImpl)
return nil
}