system/serenity
system/serenity
1
0
Fork 0
mirror of https://github.com/SerenityOS/serenity synced 2024-10-07 00:19:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Documentation: Add Sanitizer section to RunningTests

Browse source 
Also, add link to RunningTests BuildInstructions, and clean up stale
commands in RunningTests to align to current build strategies.
This commit is contained in:
Andrew Kaster 2021-05-15 20:13:31 -06:00 committed by Linus Groh
parent ac1d87b990
commit cfdd231a4d
2 changed files with 62 additions and 20 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
Documentation/BuildInstructions.md
View file

@ -267,6 +267,11 @@ For the changes to take effect, SerenityOS needs to be recompiled and the disk i
To add a package from the ports collection to Serenity, for example curl, go into `Ports/curl/` and run `./package.sh`. The sourcecode for the package will be downloaded and the package will be built. After that, rebuild the disk image. The next time you start Serenity, `curl` will be available.
## Tests
For information on running host and target tests, see [Running Tests](RunningTests.md). The documentation there explains the difference between host tests run with Lagom and
target tests run on SerenityOS. It also contains useful information for debugging CI test failures.
## Customize disk image
To add, modify or remove files of the disk image's file system, e.g. to change the default keyboard layout, you can create a shell script with the name `sync-local.sh` in the project root, with content like this:

77
Documentation/RunningTests.md
View file

@ -1,10 +1,10 @@
## Running SerenityOS Tests
# Running SerenityOS Tests
There are two classes of tests built during a Serenity build: host tests and target tests. Host tests run on the build
machine, and use [Lagom](../Meta/Lagom/ReadMe.md) to build Serenity userspace libraries for the host platform. Target
tests run on the Serenity machine, either emulated or bare metal.
There are two classes of tests built during a SerenityOS build: host tests and target tests. Host tests run on the build
machine, and use [Lagom](../Meta/Lagom/ReadMe.md) to build SerenityOS userspace libraries for the host platform. Target
tests run on the SerenityOS machine, either emulated or bare metal.
### Running Host Tests
## Running Host Tests
There are two ways to build host tests: from a full build, or from a Lagom-only build. The only difference is the CMake
command used to initialize the build directory.
@ -12,41 +12,78 @@ command used to initialize the build directory.
For a full build, pass `-DBUILD_LAGOM=ON` to the CMake command.
```sh
mkdir Build
cd Build
cmake .. -GNinja -DBUILD_LAGOM=ON
mkdir -p Build/lagom
cd Build/lagom
cmake ../.. -GNinja -DBUILD_LAGOM=ON
```
For a Lagom-only build, pass the Lagom directory to CMake.
For a Lagom-only build, pass the Lagom directory to CMake. The `BUILD_LAGOM` CMake option is still required.
```sh
mkdir BuildLagom
cd BuildLagom
cmake ../Meta/Lagom -GNinja
cmake ../Meta/Lagom -GNinja -DBUILD_LAGOM=ON
```
In both cases, the tests can be run via ninja after doing a build:
In both cases, the tests can be run via ninja after doing a build. Note that `test-js` requires the `SERENITY_SOURCE_DIR` environment variable to be set
to the root of the serenity source tree when running on a non-SerenityOS host.
```sh
# /path/to/serenity repository
export SERENITY_SOURCE_DIR=${PWD}/..
ninja && ninja test
```
### Running Target Tests
To see the stdout/stderr output of failing tests, the reccommended way is to set the environment variable [`CTEST_OUTPUT_ON_FAILURE`](https://cmake.org/cmake/help/latest/manual/ctest.1.html#options) to 1.
Tests built for the Serenity target get installed either into `/usr/Tests` or `/bin`. `/usr/Tests` is preferred, but
```sh
CTEST_OUTPUT_ON_FAILURE=1 ninja test
# or, using ctest directly...
ctest --output-on-failure
```
### Running with Sanitizers
CI runs host tests with Address Sanitizer and Undefined Sanitizer instrumentation enabled. These tools catch many
classes of common C++ errors, including memory leaks, out of bounds access to stack and heap allocations, and
signed integer overflow. For more info on the sanitizers, check out the Address Sanitizer [wiki page](https://github.com/google/sanitizers/wiki),
or the Undefined Sanitizer [documentation](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html) from clang.
Note that a sanitizer build will take significantly longer than a non-santizer build, and will mess with caches in tools such as `ccache`.
The sanitizers can be enabled with the `-DENABLE_FOO_SANITIZER` set of flags. Sanitizers are only supported for Lagom tests, as SerenityOS support
for gcc's `libsanitizer` is not yet implemented.
```sh
mkdir BuildLagom
cd BuildLagom
cmake ../Meta/Lagom -GNinja -DBUILD_LAGOM=ON -DENABLE_ADDRESS_SANITIZER=ON -DENABLE_UNDEFINED_SANITIZER=ON
ninja
CTEST_OUTPUT_ON_FAILURE=1 SERENITY_SOURCE_DIR=${PWD}/.. ninja test
```
To ensure that Undefined Sanitizer errors fail the test, the `halt_on_error` flag should be set to 1 in the environment variable `UBSAN_OPTIONS`.
```sh
UBSAN_OPTIONS=halt_on_error=1 CTEST_OUTPUT_ON_FAILURE=1 SERENITY_SOURCE_DIR=${PWD}/.. ninja test
```
## Running Target Tests
Tests built for the SerenityOS target get installed either into `/usr/Tests` or `/bin`. `/usr/Tests` is preferred, but
some system tests are installed into `/bin` for historical reasons.
The easiest way to run all of the known tests in the system is to use the `run-tests-and-shutdown.sh` script that gets
installed into `/home/anon/tests`. When running in CI, the environment variable `$DO_SHUTDOWN_AFTER_TESTS` is set, which
will run `shutdown -n` after running all the tests.
For completeness, a basic on-target test run will need the Serenity image built and run via QEMU.
For completeness, a basic on-target test run will need the SerenityOS image built and run via QEMU.
```sh
mkdir Build
cd Build
cmake .. -GNinja
ninja && ninja install && ninja image && ninja run
mkdir Build/i686
cd Build/i686
cmake ../.. -GNinja
ninja install && ninja image && ninja run
```
In the initial terminal, one can easily run the test runner script:
@ -77,8 +114,8 @@ BootModes=self-test
the serial debug output to `./debug.log` so that both stdout of the tests and the dbgln from the kernel/tests can be
captured.
To run with CI's TestRunner system server entry, Serenity needs booted in self-test mode. Running the following shell
lines will boot Serenity in self-test mode, run tests, and exit.
To run with CI's TestRunner system server entry, SerenityOS needs booted in self-test mode. Running the following shell
lines will boot SerenityOS in self-test mode, run tests, and exit.
```sh
export SERENITY_RUN=ci