mirror of
https://github.com/gravitational/teleport
synced 2024-10-20 01:03:40 +00:00
8f43fdbcf5
* render-tests: Remove test output from flaky test reporter Remove the output from the summary output by render-tests in the flakiness reporting mode. Sometimes this output is very large depending on the test that failed, and we fail to post it to Slack due to the length. This only changes what is written to the summary file. Stdout still contains the failed test output. When posted to Slack, the message will have a link to the run logs which will contain the failed test output. This output will be sorted alphabetically by package/test name and not by failure rate, as it was before. * render-tests: Print number of failure, not percentage Print the number of failures from the total number of tests in the flakiness report instead of percentages. Since we run less than 100 test runs, the percentage is not that useful. It is more useful to see that a test failed only once, for example, not 14.1% of the time. * render-tests: Fix expected/actual ordering in tests Fix the tests for render-tests where expected/actual params were backwards. * render-tests: Dont collect test output against package Keep individual test output separate from package-level output. Previously, all individual test output was recorded against a package too and only the package output was printed if no tests in the package failed. This meant we lost output for package-level failures such as data races and crashes (segfaults) when there was an individual test failure too. There was no reason I could tell why it was done this way, so remove it. This should now print all test failures. * render-tests: Include packages in top flakiness summary Include packages as well as individual tests in the flakiness summary, as packages can have failures where no individual tests failed, such as data races or crashes - both of these are recorded as a package-level failure and not against individual tests. Without these in the summary, we miss some failures. * Change (M from N) to (M/N) in flaky summary |
||
|---|---|---|
| .. | ||
| _sample | ||
| testdata | ||
| args.go | ||
| main.go | ||
| main_test.go | ||
| result.go | ||
| result_test.go | ||