FIPS is only built on amd64, and only on our centos:7 buildboxes. These
other dockerfiles and targets are vestigial. Furthermore, the buildbox
is a security risk, as ubuntu:18.04 is not supported after 2023-05-31.
If/when we want to support FIPS on ARM, we can build off the centos:7
infrastructure, or its successor.
* Make devbox sandbox friendly, add devbox CI.
The devbox is now sandbox friendly, and devbox CI has been added to ensure
that any changes to Devbox are properly validated.
* Install correct binary name for node protoc plugin.
* Add in zlib to support LIBFIDO compilation.
* Speed up yarn install, set GOROOT to nix go.
Yarn is now being installed via yarn's generic install script, which seems
to be much faster than the corresponding nix package. Additionally, the GOROOT
environment variable is being properly set to the nix's go installation.
* Unset goroot instead of trying to set it.
* Initial devbox.
As devbox has added in version pinning, it seems like a viable way for
maintaining consistent tooling across devboxes. This is an initial pass
at using devbox in Teleport.
* Use latest git/bash, use clang instead of gcc.
* Add in TELEPORT_DEVBOX env variable for future use.
* Alphabetize and add in gotestsum.
* Remove gotestsum.
* Bump golangci-lint to 1.53.2
* Make sure libbpf works in linux.
* Add in notes to update devbox.json, add in a few more dependencies to devbox.json.
* Remove extraneous comments in Dockerfile, add in README.md blurb.
* tooling: Refactor render-tests
Refactor render-tests to simplify and group the logic prior to adding a
mode to report flaky tests. No additional functionality has been added,
but the coverage detection has been fixed as the regexp was incorrect
(presumably the output format changed).
* tooling: Rework render-tests counting
Rework the counting of pass/fail/skip events in render-test to record
these counts at the test and package level instead of a single status of
pass/fail/skip. This will allow us to accumulate results from more than
one test run to be able to test for flaky tests.
Tally and output package and test counts separately as previously it was
technically incorrect when it said "n tests passed", as that also
included package results. This may make it a little clearer just how
much is failing when a failure occurs.
* tooling: Add flakiness mode to render-tests
Add a `-report-by flakiness` mode to `render-tests` that can accumulate
multiple test runs and report the top N flaky tests. This mode is
intended to be used in a daily run of the tests for a couple of hours,
or perhaps 200 times, and to report on the tests that have failed the
most.
The output of multiple test runs should be fed into the single run of
`render-tests`. A `rerun` utility is forthcoming with which you could
do:
rerun -n 200 -t 2h go test -shuffle on -cover -json . | \
render-tests -report-by flakiness -top 10
* tooling: Have render-tests write summary to file
Add the `-summary-file` flag to have render-tests write a summary of the
test run to the specified file. This is to be used to get a flaky test
summary that can be sent on slack via CI (GitHub Actions).
* tooling: Add rerun command for multiple test runs
Add a `rerun` command that is intended to be used to run tests multiple
times for a duration. It allows `go test -json` to be run many times
with the output piped to `render-tests -report-by flakiness` to generate
a summary of flaky tests over a large number of runs.
* Revert the removal of quintush/helm-unittest
Commit 5d53c91c7a removed
quintush/helm-unittest from the buildbox. It seems we still need that
version so revert those hunks that removed it.
* tests: Fix invalid sudoers file test
Fix the invalid sudoers file test to not look for an exact string but
just a substring. The error message has changed from Ubuntu 20.04 to
22.04 and it has removed some extra wording.
After moving the Connect to a separate Docker image https://github.com/gravitational/teleport/pull/27175 we're able to use the latest ubuntu LTS on our build image. We're not using this image to produce any releases (only CI runs), so updating the image will have no effect on our releases.
* Move Connect build to a new Docker container
* Update comments
* Update comments
Remove unused packages and unused arguments
* Always use UID=1000 for building teleterm.
* * Refactor tool/tsh to enable tsh e2e tests outside of the tsh package.
* Add tool/teleport/testenv to enable easier e2e tests from outside
packages.
* Skip all flaky test checks when * is provided.
* Set the correct file permission on make grpc
https://github.com/gravitational/teleport/pull/26640 introduced the new GRPC buildbox. The new Docker image uses the default user (root) which changes the generated files owner on all generates files.
This PR sets XDG_CACHE_HOME to allow buf to run as a provided user.
Note: This is mainly Linux issue, as MacOS does not change the owner of modified files in mounted volumes.
* Use podman for GRPC generation
* Remove docker override on Linux
* Restore example ARG values
* Update build.assets/Dockerfile-grpcbox
Co-authored-by: Marco André Dinis <marco.dinis@goteleport.com>
---------
Co-authored-by: Marco André Dinis <marco.dinis@goteleport.com>
* Bump golang-ci to `v1.53.0` and upgrade `depguard` config to `v2`
* pin golangci-lint version
* Keep golangci version only in the Dockerfile
* Bump golangci-lint to v1.53.1
---------
Co-authored-by: Alan Parra <alan.parra@goteleport.com>
* Speedup OpenSSL build
Our Docker images build OpenSSL using one job. Running the build using multiple jobs speeds up the process significantly.
* Simplify make command
* Add the babybox Dockerfile and Makefile
* Change Makefiles to use the babybox
* Change buildbox to use version args
* Keep the old "if BUILDARCH" on protoc install
* Drop --platform directive on babybox (its platform doesn't matter)
* Use mktemp to download protoc
* Remove defaults from ARGs
* Copy ARG comments to buildbox Dockerfile
* Rename babybox to grpcbox
* Bump go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
Bumps [go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp](https://github.com/open-telemetry/opentelemetry-go-contrib) from 0.40.0 to 0.41.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go-contrib/compare/zpages/v0.40.0...zpages/v0.41.0)
---
updated-dependencies:
- dependency-name: go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
Bumps [go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc](https://github.com/open-telemetry/opentelemetry-go) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.14.0...v1.15.0)
---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* merge outstanding otel updates
* serialize TestTraceProvider subtests
This test modifies the global trace provider and a race can reliably
detected with the latest version of the oteltrace library.
This doesn't have much impact on the test runtime, which is dominated by
a single subtest.
```
--- PASS: TestTraceProvider (1.07s)
--- PASS: TestTraceProvider/not_sampling_prevents_exporting (0.00s)
--- PASS: TestTraceProvider/spans_exported_with_gRPC+TLS (0.00s)
--- PASS: TestTraceProvider/spans_exported_with_gRPC (0.00s)
--- PASS: TestTraceProvider/spans_exported_with_HTTP (0.00s)
--- PASS: TestTraceProvider/spans_exported_with_HTTPS (1.06s)
```
* fix races in tsh tests
* Add TestForwardingTraces to flaky test detector skip list
* fix race in TestExportingTraces
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tim Ross <tim.ross@goteleport.com>
Update the version in tsh.app (tsh and tshdev) `Info.plist` files when
`make version` (`make update-version`) is run. If the version number has
any semver prerelease tags, then the tsh version number will be set to
"1.0", as those tags are not valid in the CFBundleShortVersionString or
CFBundleVersion fields. For a release without a semver tag, the value
will be set.
This will update the CFBundleVersion and CFBundleShortVersionString
fields of the `Info.plist` file.
This commit contains re-written plist files as they get reindented by
the tool, the CFBundleVersion becomes "1.0" (was "1") and one empty
element is shortened to the empty-element form.
Link: https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundleversion
Link: https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundleshortversionstring
* Restore Kubernetes Integration tests
This PR re-enables the Kubernetes integrations tests using a KinD
(Kubernetes in Docker) cluster.
New steps have been introduced to GitHub's Integrations (Non-Root)
Action that configure the KinD cluster using
[`helm/kind-action`](https://github.com/helm/kind-action) and do some
network configurations allowing the container where tests run to connect
to the KinD control plane.
This PR also fixes some of the tests and fixes a bug that affected
joining operations when the target service was a legacy kubernetes
proxy. Some improvements will be introduced in future patches to improve
the logic and reduce the time required for the tests to run.
Fixes#25539
* fix data race in spdystream dep
* address feedback
* remove docker installation
* fix test
As part of restoring Kubernetes integration tests, we need to have
docker cli on our build box image in order to spawn a KinD (Kubernetes
in Docker) cluster.
Part of #25539
Retry GitHub API requests on transient server errors, using
`github.com/hashicorp/retryablehttp-go`. We get the occasional 502 error
which breaks the whole drone pipeline run:
Failed to start workflow run Failed polling workflow jobs
Failed to fetch workflow jobs
GET https://api.github.com/repos/gravitational/teleport.e/actions/runs/4858067495/jobs: 502 Server Error []
These targets were originally set up to allow parallel arm64 builds
using GHA. These targets were obsoleted when the ARM64 builds were
expanded to be full-fledged teleport releases, but were not removed
at that time.
Leaving these targets is messy and confusing, so this patch removes
them.
* build: Support ARM64 (cross)builds of fido2 et al
Add support for building/cross-building the fido2 libraries (cbor,
openssl and fido2), supporting ARM64 builds. This is done by adding the
appropriate flags to the library builds in `build-fido2-macos.sh` based
on the `C_ARCH` environment variable. If unset then the host
architecture is used. The `Makefile` defined `C_ARCH` based on the
`ARCH` variable, mapping it to an appropriate value for the C compiler.
Building the libraries should now be done through the new `build-fido2`
target, and getting the pkg-config path should be done with the
`print-fido2-pkg-path`. This is instead of calling the
`build-fido2-macos.sh` script directly as the `Makefile` takes care of
setting the `C_ARCH` environment variable appropriately.
* build: Add make target to install rust cross toolchain
Add the `rustup-set-target-toolchain` target to the Makefile to ensure
the right rust toolchain is installed for the version of Rust we use as
well as the target architecture we wish to generate code for, based on
the `ARCH` variable. This is intended to be used by CI jobs to ensure
they build with the correct toolchain.
* build: Support building MacOS packages for ARM64
Remove the restriction that allows only AMD64 packages to be built on
MacOS for the teleport and tsh packages. This is via the existing `-a`
flag to `build-package.sh` and a newly added `-a` flag to
`build-pkg-tsh.sh`.
This adds the architecture to the filename of the package to distinguish
the packages for different architectures.
Update the comments in the Makefile mentioning that `arch` is ignored.
build: add architecture to package names
* build: Build Teleport Connect with target architecture
When packaging Teleport Connect with electron-builder, pass an
architecture flag so that we can cross-build Teleport Connect. This will
allow us to build MacOS ARM64 binaries on the AMD64 runners.
Add the architecture to the `dmg` filename via the electron-builder
config, so that the filenames for different architectures don't clash.
* build: Copy Mac release artifacts to release directory
Copy the Mac release artifacts to a release artifact directory so that
the CI scripts do not have to. This makes it clearer what is and is not
a release artifact and puts the logic in the Makefile instead of the CI
yaml, so it can more easily be tested locally and to make it easier to
migrate to the next CI system.
This will also be useful for building universal binaries for Mac as the
CI system can put the architecture-specific binaries from a previous
workflow job into a common location.
We should look at copying all release artifacts for the other builds
(Linux tarballs and packages, etc) into this directory too. It may help
with unifying the GitHub Actions release workflows.
* build: Add MacOS universal builds
Add support for ARCH=universal on Darwin to produce universal (fat)
binaries from pre-built arm64 and amd64 binaries.
Packages (pkg) and disk images (dmg) for containing universal binaries
are named without an architecture in the filename, as that is the
current naming for the current AMD64-only releases. These universal ones
will replace those AMD64-only ones providing a single release artifact
working across architectures.
Co-authored-by: Grzegorz Zdunek <grzegorz.zdunek@goteleport.com>
* build: Do not clean before release-darwin
Remove the `clean` prerequisite from the `release-darwin-unsigned`
target as it is not needed when building on GitHub Actions, as it starts
with a fresh slate each run. We do not make releases manually so we
don't need to ensure a clean working directory there either.
Not doing a clean makes it easier to build a MacOS universal release as
it depends on the architecture-specific tarballs from a previous release
build. We would need to manually save the tarballs from the first
architecture release build as they would get deleted by the `clean` from
the second. So just stop cleaning as it is not needed.
---------
Co-authored-by: Grzegorz Zdunek <grzegorz.zdunek@goteleport.com>
* Use web address when appropriate for a jump hosts
Determines whether the jump host provided via `tsh ssh -J` is belongs
to the Proxy SSH or Web server to ensure when using jump hosts that
connections are established directly on the target cluster.
Closes#25178
* Modify tsh tests to capture issues with jump hosts
Alters the root and leaf cluster and node names used by tsh tests
so that the root cluster is named `root` instead of `localhost` and
sets a unique `NodeName` for each cluster instead of reusing
`localnode` for both. This was masking problems in jump hosts tests
by connecting to the node in the root cluster instead of the leaf
cluster.
Some additional changes to tsh tests were made as a result of
changing the cluster and node names.
* fix proxy client tests
* update TestList to login once
* ignore TestList in flaky test detector
