The current config on GitHub Actions does not use ccache, so it takes
quite a while to build. Instead, let's just run these tests on Azure
where we already build Ladybird and have ccache enabled. This also lets
us sanitize LibWeb on both Linux and macOS.
The script changes here are to A) handle differences between Azure and
GitHub Actions and B) to support running on macOS.
This was useful when building both i686 and x86_64 SerenityOS targets as
we could use a single toolchain build for both targets. But now all this
extra job does is create the opportunity for the toolchain to need to be
built twice (i.e. if the pipelines are backed up and the toolchain cache
is busted between these jobs while the x86_64 step is waiting for a VM).
They currently reside under Build/<arch>, meaning that they would be
redownloaded for each architecture/toolchain build combo. Move them to a
location that can be re-used for all builds.
The two major changes noticeable on the SerenityOS codebase are:
- Much improved support for const placement, clang-format-14 ignored
our east-const configuration in various places
- Different formatting for requires clauses, now breaking them onto
their own line, which helps with readability a bit
Current versions of CLion also ship LLVM 15, so the built-in formatting
now matches CI formatting again :^)
This *also* got missed in the gcc-12 update, because we weren't
installing an explicit gcc version prior. Hopefully that's the last of
the long tail of issues from that migration!
So far we've gotten away with using GCC 11 for Lagom and to compile the
toolchain, but via #15795 we discovered a compiler bug that has been
fixed in the latest version but would error the build with CI's GCC 11.
Time for an upgrade :^)
We already use ubuntu-22.04 images in most places, so this is pretty
straightforward. The only exception is Idan's self-hosted runner, which
uses Ubuntu Focal. LibJS should build fine with GCC 11, still.
To prepare for placing all CLDR generated data in a new library,
LibLocale, this moves the code generators for the CLDR data to the
LibLocale subfolder.
It currently takes upwards of 40 minutes to download the ccache on macOS
and often errors-out near the end. Change the cache version to bust it
so we can start anew. Reduce its max size to 2 GB (a clean build is ~0.9
GB, so this allows just over 2 clean builds to be cached).
The recently added generate-emoji-txt.sh script uses a Bash 4
substitution feature, causing CI to fail as macOS ships an ancient Bash
3.x. We'll want to use a more recent version anyway, so let's do that
instead of updading the script to older syntax.
This cache was disabled in 3127454 because it wasn't needed and there
was a race between the builders for this cache. Then commit 0c95d99
started fuzzing the generated Unicode / TZDB data. Since then, we've
been pulling this data from the live servers instead of Azure's cache.
We do a similar trick for the compiler cache. This allows each builder
to separately push their local data cache (if it changed) while pulling
a shared cache, without the race outlined in commit 3127454. This is
needed for a subsequent commit which will enable this cache for Fuzzer
builds.
This is a start to properly letting us cross-compile Lagom where both
the Tools and the BUILD_LAGOM=ON build are using Lagom CMakeLists.
The initial cut allows an Android build to succeed, more or less.
But there are issues with namespace clashes when using FetchContent with
this approach.
This commit upgrades Github Actions workers to ubuntu-22.04
As part of that change, we (currently) no longer need the backports
nor toolchain-r/test PPAs, because ubuntu-22.04 include
recent-enough version of QEMU and gcc
This shouldn't cause any breaking changes, so a toolchain rebuild is not
required.
As per Hendiadyoin's request, math errno is disabled by default, which
should enable some extra compiler optimizations in LibGL and LibSoftGPU
code that uses math functions heavily.
Co-Authored-By: Ali Mohammad Pur <mpfard@serenityos.org>
Add a job to the Azure pipelines to run tests with coverage enabled, and
aggregate the test results in a folder of html pages showing the
coverage results overall, and per-file.
Future work is needed to take the published pipeline artifact for the
coverage results and display them somewhere interesting.
The LLVM patch has been broken up into smaller commits and moved to a
separate directory. CI should look at this new location to determine if
the toolchain needs to be rebuilt.
Now that clang-format-14 ubuntu packages are available, it's time to
finally upgrade our clang-format version. This version brings with it
a bunch of useful features with const-placement being the most notable.
These will be enabled in the following commits.
This allows us to fuzz the generated unicode and timezone database
helpers, and to fuzz things like LibJS using Fuzzilli to get proper
coverage of our unicode handling code.
Update the Azure CI to use the new two-stage build as well, and cleanup
some unused CMake options there.
In the last few commits, a second patch was added to the LLVM toolchain,
and it no longer uses our binutils patch. This commit changes the CI
cache keys accordingly, in order to prevent unnecessary rebuilds of both
toolchains when only one is changed.
The Clang toolchain's cache now only takes into account patches that
begin with `llvm`, and the GNU toolchain excludes those from the hash
calculation. We now also hash the two CMake cache files that we use for
building LLVM and its runtime libraries.
The toolchain is built in a previous stage, but once the Serenity stage
has begun, we have to re-pull the toolchain from the Azure cache. There
is a timing window where a cache-busting change can be commited between
these steps; to alleviate the affect this has, pull the toolchain ccache
so that the build only takes a few minutes instead of a couple hours.
'bootmode' now only controls which set of services are started by
SystemServer, so it is more appropriate to rename it to system_mode, and
no longer validate it in the Kernel.
Bootmode used to control framebuffers, panic behavior, and SystemServer.
This patch factors framebuffer control into a separate flag.
Note that the combination 'bootmode=self-test fbdev=on' leads to
unexpected behavior, which can only be fixed in a later commit.
As of the Clang 13 upgrade, we only need to build the toolchain once and
can use that toolchain for both x86_64 and i686. To do this, this breaks
the main Azure configuration into 3 "stages" (Lagom, Toolchain, and
Serenity), where the Serenity stage depends on the Toolchain stage.
This has the added benefit of uploading a new prebuilt toolchain cache
sooner than before, which should help alleviate pressure from PRs.
