The Cf-Access-Token header seems to be a infrequently used header that can
easily increase the size of the header by `len(roles) + len(traits)`, which
can cause problems. Users are able to add this in on their own if they need
it using header rewriting, so we'll remove this.
* Move gRPC error intercetpors to api/utils/grpc/interceptors.
* Use error interceptors in api client and mock server.
* Apply suggestions from CR.
* Unwrap FromGRPC errors in middleware.
* Use gRPC auth service in tests instead of external example service.
* It's gRPC!!!
* Fix unit test.
* Add error interceptor to proxy client.
* Fix merge conflict.
* test RDS database discovery
* test RDS postgres instance connection
* organize some common test helpers for eks/rds e2e tests
* exclude e2e tests from flaky test base step
* exclude e2e tests in other test flows
* skip e2e db tests by default via env var check
* add postgres web conn test
* Adds a UnifiedResource watcher to the auth server. This watcher
will watch all the types that are displayed in the web UI and store
them in-memory to allow us to search/filter/query and get multiple
kinds returned at the same time.
[RFD](https://github.com/gravitational/teleport/pull/28162)
* Add ListUnifiedResources gRPC and web endpoints (#29661)
Adds the `ListUnifiedResources` grpc endpoint that returns paginated unified resources and exposes it in the web apiserver
* Fix test
* Fix Kubernetes Legacy Proxy heartbeats
When a Teleport Kubernetes Proxy runs in legacy mode - i.e. - spec
enables `proxy_service.kubernetes.enabled`, the proxy is capable of
proxy requests to a target Kubernetes cluster instead of forwarding them
to a Teleport Kubernetes Service.
The following config enables the legacy proxy mode in proxy:
```yaml
proxy_service:
enabled: yes
web_listen_addr: 0.0.0.0:3080
public_addr: tele.local:3080
kubernetes:
enabled: yes
listen_addr: 0.0.0.0:3026
kubeconfig_file: path # optional
```
Because the proxy forwards and heartbeat the configured Kube clusters,
it needs permissions to upsert the kube cluster in auth's backend and
to extend its expiration date.
This PR fixes a problem that exists since Teleport 8 where the proxy
is allowed to upsert the kube server but it's not allowed to extend the
expiration time.
When the request fails, the proxy becomes in degraded state and the
performance is affected.
Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>
* add tests
---------
Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>
* Ignore specific teleterm proto file rather than whole package
* clusters.Storage: Avoid unnecessary casting of URI back to string
* Storage.fromProfile: Move loading profile status to separate function
* Return client.TeleportClient together with clusters.Cluster
This is a stopgap to make clusters.Cluster a regular struct with no extra
behavior and a much smaller interface.
At the moment, almost all RPCs go through layers like these:
gRPC handler → daemon.Service → clusters.Storage → clusters.Cluster →
→ TeleportClient
As a result, clusters.Cluster has a gigantic interface that's hard to test.
Instead, we want to make it hold basic information about the cluster.
The real work will be done by collaborator structs which take clusters.Cluster
and TeleportClient as args. This way we can mock them out more easily in tests.
See the issue linked in the comment of clusters.Storage.GetByResourceURI
for more details.
* Make AddMetadataToRetryableError public
Since we're moving away from doing everything in clusters.Cluster, we'll
need to use this function from within the daemon package.
* Add IsRoot and IsLeaf methods to uri.ResourceURI
These methods will be used to return early if someone tries to set up
roles for a leaf cluster. Connect My Computer works with root clusters only.
* Add handler for creating Connect My Computer role
* Add test for calling GenerateUserCerts with bogus request ID
* Fix checking logins of existing role
* Remove commented out argument to ReissueUserCerts
* Add a stopgap comment to Storage.GetByURI
* Remove invalid doc links
* Ensure owner node label has expected value if role already exists
* Add unit test which checks RoleSetup.Run idempotency
* Add godoc for Resolver
* Simplify watcher equality check, add a comment
* Expand the comment for DropAccessRequests
* Fix position of Connect My Computer popover
* Integrate role setup with setup document
* Add temporary action to additional actions
This will be removed once the parent PR is approved. I just wanted to give
reviewers a shortcut to triggering the endpoint from the actual app.
* Add return type to createRole
* Add useRetryWithRelogin
* Wrap role setup in retryWithRelogin
* Remove useRetryWithRelogin in favor of retryWithRelogin & useCallback
* Remove CMC role setup from AdditionalActions
* Prettier fix
* Fix eslint
* Adds automatic approver user for use with access plugins
Adds an internal user `@teleport-access-approval-bot` with the associated
role `@teleport-access-approver`. This new role has grants the right to
approve any and all role access requests, and both the user and role
resources are automatically created on startup if not already present.
While the `@teleport-access-approval-bot` should under no circumstances
be allowed to log into the cluster or appear in UI user listings of any
kind (except as the approver in an access request), this PR does not
implement any or the requisite filtering or enforcement. This is coming
in a future PR.
* Don't crash when building OSS
* fix test fixup
* Adds labels to the Preset approval bot user
Also ads basic update functionality for preset users, and tests
for same.
* typo fix
* Commentary
* linter appeasement
* Update tests
* Update docs
* Post-merge cleanup
* Update lib/auth/init_test.go
Co-authored-by: Marco André Dinis <marco.dinis@goteleport.com>
* Address review feedback
* Filter out user in WebUI and plumb Teleport user through
---------
Co-authored-by: Marco André Dinis <marco.dinis@goteleport.com>
* ai: add embeddings basic support
- add Embeddings service and its local implementation
- add Embedding type and proto message
- add nodeEmbeddingCollector tracking nodes
- add NodeEmbeddingWatcher watching for events adn sending them to the
collector
- add the Embedder interface and its openai implementation
* ai: adapt embeddings to the vector index
* fixup! ai: adapt embeddings to the vector index
* fixup! fixup! ai: adapt embeddings to the vector index
* Update lib/service/service.go
Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
* address feedback pt.1
* address feedback pt.2: store protobuf message in backend
* address feedback pt.3: have GetEmbeddings return a stream
* Update lib/services/embeddings.go
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* address feedback pt.4: extract embedding logic out of Embeddings service
* fixup! address feedback pt.4: extract embedding logic out of Embeddings service
* address feedback pt.5: simpler error handling when embedding fails
* fix tests pt.1
* fix tests pt.2
* fix tests pt.3
* [Assist] Replace embedding watcher (#27953)
Change the way how the embeddings are calculated. Instead of creating a watcher in Auth, we will process all nodes every hour and process embeddings if any embeddings are missing or any node has been updated.
---------
Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Add reviewer and requester roles.
Reviewer and requester roles have been added to allow for easy defaults for
reviewing and requesting applications and user groups for the Okta service.
* Update comments, add in requester/reviewer references all over docs.
* Update constants.go
Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com>
* Better names in resource-requests.mdx, no options for reviewer/requester presets.
* Add dbreviewer to cspell.json.
* Update init_test.go to handle the new number of presets.
* Remove access review/request role creation from resource-requests.mdx.
* Improve wording.
* Only add reviewer/requester/group access if in enterprise.
* Docs updates.
* Remove errant scoped block.
* Tune access controls getting started.
* Prevent customer roles from being overwritten by new defaults.
* Apply suggestions from code review
Co-authored-by: Nic Klaassen <nic@goteleport.com>
* Doc updates, function name updates, bootstrap label behavior changes.
* Adjust TeleportManagedLabel comment to sound less dire.
* Remove unnecessary references to reviewer/requester in the docs.
* Apply suggestions from code review
Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com>
* Docs tweaks, removal of unnecessary login setting.
* Use internal resource type as discussed offline.
* Update api/types/constants.go
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
---------
Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com>
Co-authored-by: Nic Klaassen <nic@goteleport.com>
Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com>
Co-authored-by: Roman Tkachenko <roman@goteleport.com>
Fixes a discrepancy in overwriting the environment value with the
address observed for the Web UI for sessions not originating from
the Web UI. All sessions will now use `tc.WebProxyAddr` as the
default value and *only* update if an override is provided.
`TestIntegrations/EnvironmentVars` was updated to ensure that the
expected environment variables are present in both interactive and
non-interactive sessions.
* Add new role defaults to allow requesting and reviewing user groups.
User groups are now default requestable by the access role and default
reviewable by the editor role. This will make the onboarding experience
for the Okta service much easier, as there will be no need to set up
new roles to support it.
A new role called group_access has also been added to support it, which
allows read access to all groups.
* Tweak group access name, add in IsEmpty functions, add in tests for AddRoleDefaults.
* Only inject access request/access review conditions if the build type is enterprise.
* Add in tests for enterprise role logic.
* GCI.
* Assist - Execution web endpoint
* Add test
Clean up code a bit
* Add missing username
* Address review comments
* Make more implementation shared between Terminal and Command Web Handlers
* Address review comments
* Address review comments
* Fixes after rebase
Add comments
* Add comments
Fix linter
* Add TELEPORT to Teleport related environment variable.
* Add full IP pinning enforcement
We're adding IP pinning check to `authorizer.Authorize` which is used for every call,
so now all communications with teleport should enforce IP pinning.
Also making sure we always provide login IP for user certificate creation
and correct client IP propagation everywhere.
* Add integration test for App IP pinning.
* Fix wording
Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>
* Wrap error
Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>
* Add godocs
* Clone TLS config
* Improve proxyHeaderSigner usage
* Wider use proxyHeaderDialer and remove adhoc writing of singed header
* Add helper function TLSDial
* Use proxyHeader dialer in authConnect
* Simplify tlsConfig manipulation
Co-authored-by: Przemko Robakowski <przemko.robakowski@goteleport.com>
* Remove redundant channels processing in TLSDial
* Reduce nesting
* Update generated protobufs
* Remove ignoring of bad IP on signed PROXY header generation
* Provide logger to CheckIPPinning function
---------
Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>
Co-authored-by: Przemko Robakowski <przemko.robakowski@goteleport.com>
* Export desktop recordings to video
Add a new tsh command that will write Windows desktop recordings
to an AVI file for offline playback. Encoding is done client side
to avoid consuming server resources.
This uses the Motion JPEG codec (https://en.wikipedia.org/wiki/Motion_JPEG)
for its simplicity and ease of use. Something like ffmpeg would perform
better in nearly every aspect (run time, compression / file size, video
quality, etc), but that would complicate our build process and add extra
native dependencies. This implementation uses pure Go and works on any
platform where tsh runs today.
Also make sure `tsh recordings ls` shows Windows and SSH recordings.
* Untangle test imports
lib/events/eventstest is allowed to import lib/events
(it needs to in order to implement interfaces and use types)
This means lib/events can not import lib/events/eventstest,
which requires that we move some tests from package events
to package events_test
* tdp: break dependency on lib/srv
The lib/srv package is large and contains Unix-specific code.
Now that tsh needs to understand the TDP protocol, we need to
avoid importing lib/srv so that tsh can still build on Windows.
* Fixes unsused flags from `tsh kube exec` command
Since its introduction, `tsh kube exec` didn't have any use for the `--reason` and `--invite` flags and completely ignored what users did with it.
This PR transports those flags as query parameters to Teleport Kubernetes Service which includes them when creating the session resource in Teleport.
* add tests
* fix flaky test
* Remove support for DEBUG_ASSETS_PATH now that webapps is in the same repository.
* Remove unused DebugEnvVar variable.
* Don't return an error when explicitly build without webassets.
* Create a simple util to return a http file system for apiserver tests.
* revert
* Add missing license.
* Add comment and remove lint exception.
* add in lint exception.
Part of https://github.com/gravitational/teleport/pull/18274
This commit introduces a new hidden `wait` CLI subcommand:
- `teleport wait no-resolve <domain-name>` resolves a domain name and exits only when no IPs are resolved. This CLI command should be used in the Helm chart, as an init-container, to block proxies from rolling out until all auth pods have been successfully rolled-out.
- `teleport wait duration 30s` has the same behaviour as `sleep 30`. Due to image hardening we won't have `sleep` available, but waiting 30 seconds in a preStop hook is required to ensure a 100% seamless pod rollout on kube-proxy-based clusters.
Add support for device extensions for TLS and SSH issued certificates.
This is a first step in issuing certificates augmented with device extensions.
gravitational/teleport.e#514
* [draft] Add a new usage reporter
This adds a new usage reporter service to the auth server. It's
disabled by default in OSS and can only be turned on via startup hook
in Cloud / Enterprise. In OSS, the audit log wrapper is never
configured and any usage events are sent to a no-op discard reporter.
Usage events are defined in prehog and can be sent to the new
UsageReporter Service on the auth server. An audit event wrapper is
used to capture certain events that are otherwise difficult to hook.
Events are anonymized before submission, then held in a non-blocking
queue for batching and submission purposes.
* Remove dead code
* Add SubmitUsageEvent RPC to Auth.
This adds a new SubmitUsageEvent RPC to the Auth API that external
clients (e.g. the UI) can use to submit usage events externally.
* Slight refactor for unit testing
* Add Prometheus metrics and add initial working prehog submitter
* Add more metrics, tweak prehog client, and add unit tests
* Further tweak http transport settings based on Teleport defaults
* Add missing metrics
* Fix goimports
* Add new UI usage events
* Update e ref
* Add prehog directly for now. Improve logging.
* update prehog
* Add new prehog events; use username from request identity
* add HTTP server for user events
* Add username back to pre-onboard events
* unauthenticated user events
* Fix userevent build error
* Use event-provided username where appropriate
* Move barebones prehog reqs to lib/prehog and generate here.
Also, use prod tunable values.
* Fix license lints
* De-flake tests by adding unfortunate amounts of synchronization.
* Add missing license header
* Misc PR cleanup for review
* Update lib/events/usageevents/usageevents.go
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Address a batch of review comments
Adds `anonymizer.AnonymizeString` and parent loggers
* Update e ref
* Clean up comments
* Remove onboard prefix from recovery code event
* Address another batch of feedback
* Use defaults.HTTPClient()
* Remove a noisy log message
* Demote noisy log message to debug
* Temporarily revert e ref for merge
Co-authored-by: Michelle Bergquist <michelle.bergquist@goteleport.com>
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
- Fix an incorrect link
- Be more specific about what roles/permissions are required
- Remove some text and an image that didn't meaningfully contribute
* switch underlying protocol used for 'tsh scp' to SFTP
* address TODO
* appease linter
* add method to make it easier for other callers to transfer files
* add tests
* print transfer progress with progress bar by default
Also allow a SIGINT to gracefully stop the SFTP connection. This is
necessary because the progress bar will ignore signals and prevent the
process from exiting.
* address SFTP fork issues
* make tests less flakey
* fix specifying dir for dst not copying files to correct paths
* make tests less flakey (again)
* don't check file access times, often differs when run in CI
* few small fixes from review, simplify Create method now that HTTP FS isn't needed
* create dst files and dirs with src mode
* improved error messages when doing file operations
* expand home dirs in remote paths
* addressed more feedback
* add license to get_home_dir.go
* address minor feedback of tests, add home dir expansion test
* update sftp fork to point to latest commit on master branch
* addressed feedback
* don't cache home dir lookups, only one remote path can ever be used
* Allow tsh to retrieve cluster details in one request
Prior to connecting to a node via `tsh ssh` we need two bits of
information about the cluster:
1) The session recording mode
2) Whether FIPS is enabled
In order to retrieve the information `tsh` first would send the
global ssh request `RecordingProxyReqType` to determine the
session recording mode. Later on `tsh` would Ping the auth
server to determine if the cluster was running in FIPS mode.
In an effort to reduce the number of round trips to retrieve
this data, a new global ssh request `ClusterDetailsReqType` is
introduced that returns both the session recording mode and
whether FIPS is enabled. This allows `tsh` to leverage the new
request to get all the information it needs, and is extensible
in case more information is needed` in one request which helps
reduce latency for `tsh ssh`.
While looking up github.com/gokyle/hotp I found some old deprecation warnings
and decided to address them.
* Remove HOTP support
* Update comment on checkOTP
* Remove OTPType
* Remove a few more HOTP references
* Add initial version of installer
* Resolve comments
- Use aws waiters when checking commands
- Use SSMRunRequest rather than passing instances
- General comments
* Resolve comments, (rebase) pass scriptname parameter
This resolves comments regarding running on multiple ec2 instances at
once by adding state to the instances cache to check if the instance
is known about and how far into installation it is
* Revert cache
* Dont cache on non discovery nodes
* Resolve some comments
* Move discovery out to its own serviec
* Add a `discovery_service` section
* Fix messed up conflict merge
* Make starting a standalone discovery agent work
* Resolve comments
* Resolve comments
- use a regular events.Emitter
- resolve a thousand typos :)
* Resolve comments
* resolve comments, fix a bad merge
* Fail when a non ec2 matcher type is configured
* fix lint-go
* Resolve comments
* Resolve comments, add initial test (currently broken)
* Fix log string so only 1 pair of [] are used
* Chunk instances for sending commands
* add 'isInitialized' to watchers
* Add test for chunked discovery, log output
* lints
* explicetly set matcher.Tags to "*":"*" if its unset
This PR implements the SSH Tester for ConnectionDiagnostic feature.
This feature is also known as Test Connection, part of Teleport
Discover.
The goal here is to provide immediante feedback about a newly added
resource. Can the user connect to it?
We are targetting SSH Nodes as a first ResourceKind.
To test the access to an SSH Node we require the ResourceName and a
login username (ssh principal).
Then a series of checks will occur in two places:
- SSH client in the Web server
- SSH server in the SSH agent
The ssh client creates a new Connection Diagnostic with some initial
state.
Then it tries to build up the necessary SSH config
This already gives us a couple of things to check for:
- does the node exist and does the current user (inherited from
websession) can access it?
- is the node accepting TCP connections (in the specific port)?
- is the node accepting SSH protocol on top of the TCP connection?
Then, the ConnectionDiagnosticID is inject into the certificate and the
SSH Server receives it and will also Append traces into it:
- is the requested principal allowed for the current user?
- does the requested principal exist in the target node?
This is not an exhaustive list of checks.
For a complete list of which checks are verified please see the
TestDiagnoseSSHConnection test.
After all those checks, it returns if the Connection was successful and
what all of the traces generated along the way.
Demo:
* WebAPI: update user traits
Web API only supports updating the roles property for a given User.
This PR adds the possibility of updating User's traits
- Logins
- DB Users
- DB Names
- Kube Users
- Kube Groups
- Windows Logins
- AWS Role ARNs
It only updates if the requests contains a non-nil value for the trait's
list.
It deduplicates the trait's list before applying it.
* Add support for automatic user provisioning
* Add UID parker to reexec
* Add a `teleport park` subcommand that does nothing
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
This adds proxy peering support. A configurable setting that allows for agents
to connect to a subset of proxies and be reachable through any proxy in the
cluster. This is achieved by creating grpc connections between each proxy
server. Client connections can then be passed between proxies to the desired
agent.
