We often see PR's opened and then immediately closed because folks think
they did something bad, or don't know how to fix the situation. So lets
try to give them a few pointers.
We didn't initially upgrade because it started to (incorrectly) see
files as strict mode and chokes on things that then would be syntax
errors - but we're starting to fall behind a bit, so I'd rather put
these files on the ignore list instead.
Previously if a commit message contained any carriage returns it would
correctly fail the 'contains CRLF line breaks' test, but it would also
report 'Commit message lines are too long' and 'Commit title ends in a
period', even if neither is true.
This rule appears to produce a lot of noise, most of them look like
false positives (400+). Lets suppress for now to try to move the signal
to noise ratio higher for PVS-Studio.
Reference: https://pvs-studio.com/en/docs/warnings/v1047/
tim-actions/commit-message-checker-with-regex@v0.3.1 only uses the
keys 'sha' and 'commit.message'. Passing more information than that
is unnecessary and can lead to CI failures like this one:
https://github.com/SerenityOS/serenity/runs/4029269017?check_suite_focus=true#step:4:7
Instead of trying to pass data between workflow steps, we can instead
just do it all at once (plus this gives us more control over
formatting, which has also been improved).
'bootmode' now only controls which set of services are started by
SystemServer, so it is more appropriate to rename it to system_mode, and
no longer validate it in the Kernel.
Bootmode used to control framebuffers, panic behavior, and SystemServer.
This patch factors framebuffer control into a separate flag.
Note that the combination 'bootmode=self-test fbdev=on' leads to
unexpected behavior, which can only be fixed in a later commit.
Much like the sonar cloud workflow, this workflow runs pvs-studio
static analysis, and uploads the SARIF results to github. This
is the most "convenient" way to publish results, but unfortunately
users need write access to the repository to reach static analysis
results rendered in github.
As a work around folks can just look at the logs where issues are
printed during analysis, this works reasonably well.
In the future it might make sense to also render the results as HTML
and publish them using github page, much like we do with man pages.
I believe the pvs-studio plog-converter tool supports that as well.
https://pvs-studio.com/en/docs/manual/0036/
We need to exclude this file from analysis for now, as there is a bug in
the sonar-runner tool where it crashes when trying to understand the use
of AK::Variant in LibWasm/Parser/Parser.cpp
See #10122 for details + link to the bug report to Sonar Cloud.
I was experimenting with using caching while doing the initial prototype
of the Sonar Cloud workflow. However the cache size for the static
analysis data ended up being large enough that it would put us over the
git hub actions limit. Given that we currently only run this pipeline
once a day, it seems reasonable to just remove caching.
If in the future we decide to run the pipeline on every PR, caching
would become crucial as the current un-cached analysis time is around
1 hour and 50 minutes. If we did this we would need to move the pipeline
to Azure DevOps where we have effectively infinite cache available.
This requires exposing the `configure` step on the `serenity`
ExternalProject in the SuperBuild CMakeLists so that we can continue to
only build the generated sources and not the entire OS.
Replace the old logic where we would start with a host build, and swap
all the CMake compiler and target variables underneath it to trick
CMake into building for Serenity after we configured and built the Lagom
code generators.
The SuperBuild creates two ExternalProjects, one for Lagom and one for
Serenity. The Serenity project depends on the install stage for the
Lagom build. The SuperBuild also generates a CMakeToolchain file for the
Serenity build to use that replaces the old toolchain file that was only
used for Ports.
To ensure that code generators are rebuilt when core libraries such as
AK and LibCore are modified, developers will need to direct their manual
`ninja` invocations to the SuperBuild's binary directory instead of the
Serenity binary directory.
This commit includes warning coalescing and option style cleanup for the
affected CMakeLists in the Kernel, top level, and runtime support
libraries. A large part of the cleanup is replacing USE_CLANG_TOOLCHAIN
with the proper CMAKE_CXX_COMPILER_ID variable, which will no longer be
confused by a host clang compiler.
This commit snuck into the tree via a PR for some sonar cloud fixes.
Some how I cross contaminated my branches.
Unfortunately the coverity workflow isn't ready for prime time yet,
so lets remove it until we have all the issues ironed out.
The matrix variables were left over from copy/pasting the contents
of the normal CI workflow. We also should always skip saving the
cache, as the normal CI pipeliens will refresh the toolchain and
we should just be reading the cache.
The cache is saving, but by the time we run again, it looks like the
cache has been purged from other jobs consuming the cache.
This causes the cache to fail restore. Given we run nightly and there
is no time bound, we can just run without cache.
Test files were getting analyzed twice, which the tool does
not like, and causes it to exit with a fatal error.
Also make the workflow run in PRs anytime the file is edited,
so that we can get immediate feedback without waiting till the
next day.
This action executes once a day, the sonar cloud runner analyzes the
code and then uploads the results.
The current code base takes almost 3 hours of computer time to analyze.
The runner supports multi threaded executing and caching of results, so
we save that cache as part of the github action work flow to allow for
the analysis to skip unchanged files.
Moving this helper CMake file to the centralized Meta/CMake folder helps
to get a better grasp on what extra files are required for the build,
and what files are generated.
While we're at it, don't use add_compile_definitions for
ENABLE_UNICODE_DATA, which only needs to be seen by LibUnicode sources.
We were over-hashing for the GNU build on GitHub Actions by including
the LLVM patch as well. The GNU Toolchain doesn't care about our LLVM
patches.
For Azure, fix the inversion of the condition for which jobs check which
Build*.sh script, and add the Toolchain patch files to the cache
hash calculation.
Clang builds will no longer be apart of the automated CI for every Push/
Pull Request, and will instead be ran at 00:00 UTC every day, with the
results posted to the discord #clang-toolchain channel.
We need test-js for the parser tests for test262, but we don't need to
rebuild all of Lagom twice. This was missed when we did the initial
change to shared libraries. Before #9017, the Lagom build for test-js
is what built libLagom.a for the libjs-test262-runner to link against.
Now that we are building libjs.so and its dependencies in the runner's
build directory, we should build test-js there as well.
Requires linusg/libjs-test262#32 in order to properly find the built
test-js.
We already cache these files to prevent re-downloading them in the other
CI workflows, so this just brings the test262 runner up to speed with
the rest of them.
After linusg/libjs-test262/pull/30 goes into libjs-test262, we'll need
to pass SERENITY_SOURCE_DIR manually to the job to prevent it from
trying to do its own shallow clone. Also, remove the now defunct static
library build from the test262 workflow.
The WASM spec tests caused a stack overflow when generated with wat2wasm
version 1.0.23, which ships with homebrew. To give feature parity,
manually download the same version from GitHub packages for Ubuntu.
Document the dependencies of the WASM spec tests option, as well.
This should speed it up quite a bit and give us more consistent
performance. (So this workflow could eventually be used for perf
regression testing as well)
It turns out that ccache caches are highly compressible (total size is
reduced by about 65%), so we should be able to increase the cache limit
for some free speedups. :^)
`wasm-as` will do some semantic analysis on the modules, which is not
something we're looking for here.
Instead, use `wat2wasm` to generate the exact module.
While these look nice, they require the discord workflow to sit around
and wait for the build-and-lint workflow to finish in order to get its
status which means we waste an extra workflow runner that does nothing
for each build-and-lint run.
With the increased volume of PRs being opened and merged lately,
multiple people have complained that the IRC is absolutely flooded with
SerenityBot posts. Remove the IRC notifications from the CI scripts, and
the Meta script that handles parsing the github actions context into
an IRC message.
Github Actions added clang-12 to their ubuntu 20.04 images, so let's
take full advantage of that. Stop relying on the llvm upstream
repository for clang-12, since it often causes jobs to fail when
their mirrors are syncing.
clang-tidy-11, libstdc++-10-dev and npm 6.14.x are also all already
pre-installed, so we don't need to waste time fetching them in the
dependency fetch step.
Note that until UBSAN is made deadly by default in LibSanitizer, UBSAN
warnings will not fail the build.
Also remove BUILD_LAGOM=ON from the NORMAL_DEBUG build as it's
unnecessary and extends the build time for no benefit when building with
sanitizers
This option replaces the use of ENABLE_ALL_THE_DEBUG_MACROS in CI runs,
and enables all debug options that might be broken by developers
unintentionally that are only used in specific debugging situations.
Options shamelessly stolen from this article on systemd's website:
https://systemd.io/TESTING_WITH_SANITIZERS/
We make ASAN more strict and tell UBSAN to print more verbose output on
failure. One of the more interesting ASAN options,
detect_stack_use_after_return, sadly causes both UBSAN and ASAN failures
in test-js.
Change run-tests-and-shutdown.sh to output a dead simple results file
that just records how many tests failed.
In the CI script, mount the _disk_image after running tests and verify
that the number of failed tests is 0. Otherwise, fail the build :^)
While we're here, bump the timeout for the tests up to 30 minutes, to
make sure that less powerful runners don't fail the job unecessarily.
Ubuntu 20.04 only ships QEMU 4.2.1, which is quite an older release.
The BuildQemu.sh script uses version 6.0.0, while the server-backports
PPA is currently shipping 5.2.1. If it turns out the server-backports
PPA is not right, then we can switch to manually building and caching
the source build.
Uncomment the tests that were disabled due to frequent freezes when
running without KVM. This also adds a new github actions group for
every single test, which makes it easier to browse test boundaries
during test runs.
Move catting the serial output log back to its own step, so that it
has higher visibility. The previous solution was also shown to not
actually cat the log in the case of a failed boot and timeout :^(.
We don't need it. A recent change to the ubuntu-20.04 image has made it
the default, causing builds to fail - we're installing and want to use
clang-12 anyway, so let's just get rid of the other installed versions.
Index page:
- Change links from "Man 1" to "Section 1"
Section index pages:
- Change title from "1" to "Section 1 - SerenityOS man pages"
- Change links from "foo" to "foo(1)"
Man pages:
- Change title from "foo" to "foo(1) - SerenityOS man pages"
This new check will ensure that all commit message lines are at most 72
characters long, as well as ensure the commit title conforms to the
"Category: Brief description of what's being changed" format.
Initially this was configured to 2 an hour so that we could easily
disable stale-bot if something went haywire. Now that it's successfully
proved it's doing what it's supposed to, let it run the default number
of actions per hour.
This change adds the required configuration to setup the
https://github.com/probot/stale github bot to close stale pull requests.
We currently have a large amount of PRs just sitting around with no
activity, clogging up the PR queue. This will hopefully make that
situation better.
Since some builds can take even longer than 1 hour
(for example: f033416893)
this commit just increases the timeout to be github's
own workflow timeout (effectively disabling it) and just
lets github handle it instead.
The previous filter would filter out queued checks as well, which would
result in erroneous build success notifications going out if github
started the discord notifications workflow before all other workflows.
Since our tests usually take at least 10 minutes theres no point in
checking every 10 seconds, and github was starting to complain about
the very high API usage.
It looks like some particularly long builds (After a toolchain cache
reset and on a slow worker) can take much longer than the current set
timeout of 20 minutes.
While we did allow the notify_discord job to run inside the context,
we didnt ask github to run in that context. This commit also uses the
"action-wait-for-check" sub-action to ensure the posted build results
are accurate for pull requests (since the build workflow is done in a
separate context for PRs)
If the job fails early (e.g. during linting), the 'cat debug.log' step would *also* fail.
This would confuse GA into thinking that this is the crucial thing and highlights it.
This misleads the user into looking at the wrong thing.
As documented in #5541, there are some Kernel issues that can
sporadically cause the test run to fail. Add continue on error with a
loud comment to let readers know what the issue(s) might be.
Build a new version of Serenity in CI that doesn't have all the debug
symbols on, or we'd be waiting a very long time to boot.
Insert a TestRunner entry into SystemServer.ini that will run a shell
script that runs tests in /bin and /usr/Tests and shuts down the system
in the new self-test boot mode. Also make sure enough basic services are
started in self-test such that the tests will actually run properly.
This will make it easier to keep macos tests and non-mac tests in
lockstep. Also, make sure flake8 and python are installed. This also
makes it easier to add other OS targets if we want.
* Add SERENITY_ARCH option to CMake for selecting the target toolchain
* Port all build scripts but continue to use i686
* Update GitHub Actions cache to include BuildIt.sh
A good number of contributors use macOS. However, we have a bit of
a tendency of breaking the macOS build without realising it.
Luckily, GitHub Actions does actually supply macOS environments,
so let's use it.
CMake tests usually takes ~40 seconds. However, sometimes it deadlocks
and is only timed out after the 6 hour time limit.
Let's set a 2 minute timeout to make it fail sooner. 2 minutes instead
of 1 for good measure.
There are cases where Lagom will build with GCC but not Clang.
This often goes unnoticed for a while as we don't often build with
Clang.
However, this is now important to test in CI because of the
OSS-Fuzz integration.
Note that this only tests the build, it does not run any tests.
Note that it also only builds LagomCore, Lagom and the fuzzers.
It does not build the other programs that use Lagom.
CodeQL is a static analysis technology that was purchased by GitHub
and has been tightly integrated into the platform. It's different
from most other static analysis solutions because it's based on a
database built from your codebase, and then language specific rules
can be executed against that database. The rules are fully user
extensible, and are written in a datalog/query language.
The default cpp language rules coming from CodeQL will probably find
some issues, the ability to easily write custom rules/queries will
lend it self nicely to allowing us to validate Serenity specific
semantics are followed throughout the code.
References:
- https://www.youtube.com/watch?v=AMzGorD28Ks
- https://securitylab.github.com/tools/codeql
