This change drains unused SSH channels and requests to prevent a
situation where an attacker could repeatedly open channels and
send data that won't be read, causing Teleport to eventually run
out of memory.
The ssh session was not being closed for web sessions which resulted
in zombie sessions being left around until the ssh service was
restarted. TestTerminal was updated to assert that the session
tracker eventually transitions to the terminated state when the
client terminates the web socket.
Fixes#32120
* Make SSH node listener to use multiplexer to leverage parsing of signed PROXY headers
Before for backward compatibility reason we used regular listener and later
we wrapped connection into handler that could parse signed PROXY header or
proxyHelloSignature for getting correct user IP. But
after removal of proxyHelloSignature we don't need the wrapper anymore.
* Improve error message
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
---------
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
* Added Fuzz Seeds for services/fuzz_test.go
* Fuzzing improvements for `lib/srv/db` fuzzing
This ensures that all fuzzing under `lib/srv/db` has the following:
* Good seeds that provide basic levels of coverage
* Enabled in OSS-Fuzz
* Fuzzing improvements for `lib/sshutils`
This improves the fuzzing in `lib/sshutils/sftp/parse_test.go` and `lib/sshutils/x11/display_test.go`.
Seeds were added and additional local fuzzing was done. In addition these functions are now enabled in `oss-fuzz`.
This commit updates all `err == io.EOF` comparisons to use
`errors.Is(err, io.EOF)`. This is necessary when the error may have been
wrapped and fixes at least one current breakage (in `tsh request ls`).
`golang.org/x/tools/refactor/eg` was very handy for this, I used the
following template:
```go
package teleport
import (
"errors"
"io"
)
func before(err error) bool { return err == io.EOF }
func after(err error) bool { return errors.Is(err, io.EOF) }
```
A complex regex was previously used to parse 'tsh scp'
destinations, which was hard to understand and even harder
to maintain for those not intimately familiar with regex,
despite it being heavily commented. Instead parse destinations
directly and add more test cases and a fuzz test to ensure
parsing won't panic and the new parse function doesn't
have any regressions.
Previously a Teleport client using SFTP would resolve remote host user
home directories by making a subsystem request to a Teleport server
which would return the home directory. The problem was the subsystem
request counted as an open session, which could make the SFTP file
transfer fail. This was frustrating and didn't make much sense, but
after reading the SFTP specification again I realized that SFTP servers
are to handle relative paths by assuming they start at the user's home
directory. So let the server figure out the correct path and remove any
tilde prefixes from remote paths.
* refactor SFTP backend to use upstream dep, not our fork
This change also greatly reduces the number of SFTP audit logs.
Now SFTP events are only sent when files are opened or modified
in any way, instead of for *every* SFTP request.
* added to SFTP integration test
* fix error when handling setstat on dirs
* fix linter warning
* move file/dir permission constants to lib/defaults package
Part of https://github.com/gravitational/teleport/pull/23546
This will add a fileTransferRequest to a session and allow environment variables to be passed from the webUI in order to validate a request that happens "outside" the moderated session (via HTTP request).
* Fix panic when incoming request is nil
This PR prevents a panic when the SSH request channel is closed by [`mux.loop`](c3f983bc73/ssh/mux.go (L197-L199)) if it encounters a connection error.
Fixes#24186
* force connection close
* lazily forward SSH agents when connecting to agentless nodes
When connecting to unregistered OpenSSH nodes, the SSH agent is always
forwarded. When connecting to registered OpenSSH (agentless) nodes
however, the SSH agent doesn't *need* to be forwarded, so only do so
if the user explicitly asks to.
* test SSH agent forwarding in agentless integration test
To prevent users from being able to directly connect to registered
OpenSSH (Agentless) nodes, the OpenSSH CA was introduced. When
connecting to Agentless nodes, the Proxy will forward the connection
so the session can be recorded. The forwarding SSH server is configured
to authenticate to the target node with a certificate signed by OpenSSH
CA, which users do not have access to.
* Allow node to handle old and new way of client IP propagation on same listener
With addition of signed PROXY headers, node was listening on multiplexer, but because
of that it couldn't processing incoming connection from older proxies
when ProxyHelloSignature was used, because
both ends were waiting for the other side to send data first.
Here we integrate ability to handle PROXY headers into connection itself,
so we can start ssh server without waiting for multiplexer to detect connection
* Remove unneeded code
* Improve comment's wording
Co-authored-by: Michael Wilson <mike@mdwn.dev>
* Improve comment's wording
Co-authored-by: Michael Wilson <mike@mdwn.dev>
* Fix imports
* Unexport function
* Add godoc
* Remove unneeded comment
* Move ProxyHelloSignature to constants
* Check that proxyline is verified
* Use Warn() instead of Warnf()
* Move ProxyHelloSignature to api/constants
* Add timeout for getting host CA during proxyline verification.
* Rearrange conditions
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* Clarify comment.
---------
Co-authored-by: Michael Wilson <mike@mdwn.dev>
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
* SCP -Change file attrs only when requested
OpenSSH sets 0644 by default and allows copy the source permissions by passing `-p` flag.
Our current implementation sets the file permissions even when client didn't use `-p` flag which can fail if the user doesn't have permissions to set it. This patch matches the OpenSSH behavior.
* Fix formatting
* sshserver: Correctly handle PuTTY winadj channel requests
Fixes#19365
* Actually send a failure
* dispatch changes
* Reimplement wantReply handling
* Also handle windadj in forwarding server
* lint: Fix typo
* lint: Invert return order to put err last
* Use nil return payload instead
* Use trace.ReplyAlreadyHandled instead
* Handle reply internally and set req.WantReply to false
* Use t.Cleanup()
* behaviour is not a misspelling but whatever makes you happy
* Suggestions from code review
* use 'path.Join' instead of 'filepath.Join' to build remote file paths
tsh on Windows when using 'filepath.Join' would create paths with forward
slashes in them, which non-Windows Nodes couldn't handle. Consistently use
'path.Join' to avoid issues.
Fixes#18919
* add comment explaining why 'path.Join' is used
* move comment to make linter happy
* Our current SFTP implementation uses a single channel to transfer data. Connections with a high latency cause very slow transfer. github.com/pkg/sftp allows transferring data using concurrent streams, which seems to solve the problem. In order to utilize the feature, the remote file must implement the WriteTo() or ReadFrom() and Stat() methods. I've moved the progress bar to relay the data from a local file, as we can still call Read() or Write() on it to update the progress.
This PR doesn't address the high CPU usage by github.com/pkg/sftp on low latency links.
Closes#20579
* Cleanup
* Fix fileTransferProgress
* Simplify interfaces
* Move helper to utils package
Add comments
* Added comments
* Change canceler implementation to return correct values from Read/Write
* Fix infinite recursive call in cancelWriter.
Introduces `limiter.Listener` to provide a consistent and reusable
mechanism for limiting incoming connections per client. The new
listener is used by `sshutils/server.go` instead of manually applying
limits in `HandleConnection`.
This is particularly important now that the Proxy SSH port multiplexes
both SSH and gRPC. Each listener is now wrapped in a `limiter.Listener`
that uses the same `limiter.ConnectionsListener` to ensure that the
connection limits for the Proxy are enforced for all traffic on the
port.
- Add a generalized client store made up of a key, profile, and trusted certs store. Each sub store can support different backends (~/.tsh, identity_file, in-memory).
- Replace custom identity file handling with in-memory client store.
- Fix issues with trusted certs handling.
* Enable nolintlint linter
* Fix nolint comments in the api package
* Fix RDP client comment
* Address review comment
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Allow unused for nolintlint linter
* Remove redundant casting
* Add comment on why allowed unused is enabled
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
Our need for customizing our security response headers primarily exists with the Content-Security-Policy. Other headers can be applied more broadly to provide safe defaults across our service.
This commit changes httpheaders.go to allow for the CSP to be applied separate from the other headers. Then the changes to httplib.go's handlers help make sure these headers are set generally (which we were already partially doing for caching directives).
Other changes were primarily making sure these tools are configured or used. Or a removal of previous settings as this function is now being handled more centrally.
Many of our tests (db package, I'm looking at you) generate many RSA keys. This has two main side effects; makes our tests slow and flaky as CPU usage spikes in random moments when the tests are run in parallel.
This change pre-generates RSA keys at the beginning of each test module and reuse them in randomized order to reduce the situation that one key has been used multiple times in one test.
I had to move a few files to avoid circular dependencies.
* Ensure ssh connection rejection errors are returned
#18302 altered the connection flow which changed the communication
via ssh. Prior to the change an ssh channel was opened first, then
global requests were sent. After the change a global request is sent
prior to opening a channel. This caused users to no longer receive a
human readable rejection message a la: `ssh: rejected: administratively
prohibited (too many concurrent ssh connections for user`, instead only
an `EOF` was returned.
To ensure that the correct message is provided to users two changes are
made, although only the client side change is strictly required.
1) the tracing ssh client now returns any rejected errors cached from
opening the tracing channel
2) the ssh server now processes global requests during it's wait period
waiting for a channel to reject and send the replies to any requests
with said rejection
`TestLockInForce` was updated to include attempting to send a global
request so both scenarios are covered as order should not matter.
This change introduces a workaround for OpenSSH 7.x bug related to SHA-2 signed certificates. Teleport signs all certificates with SHA-2-512, which prevents us from connecting to older sshd. As a workaround, we will send the same certificate signed with SHA-2-512 (as we do right now) and SHA-1 if the first authentication fails. We always send the SHA-2 certificates first, so newer OpenSSH versions should always use SHA-2 certs as they do right now.
* switch underlying protocol used for 'tsh scp' to SFTP
* address TODO
* appease linter
* add method to make it easier for other callers to transfer files
* add tests
* print transfer progress with progress bar by default
Also allow a SIGINT to gracefully stop the SFTP connection. This is
necessary because the progress bar will ignore signals and prevent the
process from exiting.
* address SFTP fork issues
* make tests less flakey
* fix specifying dir for dst not copying files to correct paths
* make tests less flakey (again)
* don't check file access times, often differs when run in CI
* few small fixes from review, simplify Create method now that HTTP FS isn't needed
* create dst files and dirs with src mode
* improved error messages when doing file operations
* expand home dirs in remote paths
* addressed more feedback
* add license to get_home_dir.go
* address minor feedback of tests, add home dir expansion test
* update sftp fork to point to latest commit on master branch
* addressed feedback
* don't cache home dir lookups, only one remote path can ever be used
* Allow tsh to retrieve cluster details in one request
Prior to connecting to a node via `tsh ssh` we need two bits of
information about the cluster:
1) The session recording mode
2) Whether FIPS is enabled
In order to retrieve the information `tsh` first would send the
global ssh request `RecordingProxyReqType` to determine the
session recording mode. Later on `tsh` would Ping the auth
server to determine if the cluster was running in FIPS mode.
In an effort to reduce the number of round trips to retrieve
this data, a new global ssh request `ClusterDetailsReqType` is
introduced that returns both the session recording mode and
whether FIPS is enabled. This allows `tsh` to leverage the new
request to get all the information it needs, and is extensible
in case more information is needed` in one request which helps
reduce latency for `tsh ssh`.
Update metalinter, fix a few lint warnings and replace deprecated linters.
`deadcode`, `structcheck` and `varcheck` are abandoned and now replaced by [`unused`][1].
Since 1.19, `go fmt` reformats godocs according to https://go.dev/doc/comment. I've done a bulk-reformatting of the codebase to keep the linter happy. Backporting is mostly harmless (the exception being `lib/services/role_test.go`, that for some reason breaks the _old_ linter using the new format).
[1]: https://golangci-lint.run/usage/linters/
* Bump golangci-lint version
* Replace abandoned linters
* Fix bodyclose on lib/auth/github.com
* Fix bodyclose on lib/kube/proxy/streamproto/proto_test.go
* Fix bodyclose on lib/srv/alpnproxy/proxy_test.go
* Fix bodyclose on lib/web/conn_upgrade_test.go
* Silence staticcheck on lib/kube/proxy/forwarder_test.go
* Silence staticcheck on lib/utils/certs_test.go
* Address BuildNameToCertificate deprecation warnings
* Run `go fmt ./...`
* Run `go fmt ./...` on api/
* Ignore formatting in role_test.go
* Remove redundant initializers in lib/srv/uacc/
* Update e/
* Prevent ssh.Session SendRequest from wrapping payload twice
Payloads were inadvertently being wrapped twice by sessions making
the ssh server unable to parse the request payload. This is largely
only an issue for window-change requests, as they are the only request
being sent directly via a session that contains a payload. All window-change
requests now use ssh.Session WindowChange instead or sending the request
manually.
