Commit graph

169 commits

Author SHA1 Message Date
Nico Weber 17ab44e7d7 Meta: Make BuildClang.sh produce less output when running in a TTY
Ninja disables its fancy output mode when it's not writing to a TTY.
So don't pipe its output into something else, so that it writes to
a TTY if the invoking terminal is a TTY.
2021-09-07 20:29:22 +01:00
Nico Weber eb7b8a7bbe Meta: Don't pass LLVM_LLVM_BUILD_LLVM_DYLIB to cmake
`LLVM_LLVM_BUILD_LLVM_DYLIB` does not exist, so passing this does
nothing but make CMake warn.

However, since we pass `LLVM_LINK_LLVM_DYLIB`, `LLVM_BUILD_LLVM_DYLIB`
(the correct spelling) defaults to true anyways.  So let's pass fewer
flags.

No behavior change, but fixes a CMake warning.
2021-09-07 20:29:22 +01:00
Nico Weber 1ed1a57707 Meta: Make serenity.sh rebuild-toolchain aarch64 clang work 2021-09-07 20:29:22 +01:00
Idan Horowitz 783a58dbc7 Toolchain: Build aarch64-gdb for cross-debugging on x86 2021-09-07 12:58:46 +02:00
Tom 77953a937d Meta: Add the ability to specify clang with serenity.sh
This enables maintaining gcc and clang builds side-by-side.
2021-09-03 23:12:17 +02:00
Nico Weber a900792921 Toolchain: Also build aarch64-softmmu in BuildQemu.sh 2021-08-28 21:51:30 +01:00
Nico Weber 7052f403c8 Toolchain: Enough to make rebuild-toolchain aarch64 work
The gcc patch might not be completely correct, but at least the
toolchain completes building.
2021-08-28 14:43:07 +01:00
Nico Weber 61b6f69947 Toolchain: Regenerate binutils.patch and gcc.patch
I locally modified Meta/serenity.sh to pass `--dev` to BuildIt.sh
in build_toolchain(). Then I ran `Meta/serenity.sh rebuild-toolchain`,
cd'd into Toolchain/Tarballs/binutils-2.37, `git add`ed unadded files in
`git status`, and then ran `git diff > ../../Patches/binutils.patch`.
Then I did the same for Toolchain/Tarballs/gcc-11.2.0 (and was careful
not to `git add` serenity-kernel.h, since that's created by
Toolchain/BuildIt.sh).

No behavior change. This just rewrites the patch like git writes it.
2021-08-28 14:43:07 +01:00
Brian Gianforcaro aceea42ba4 Toolchain: Update BuildQemu.sh to latest 6.1.0
Change Log: https://wiki.qemu.org/ChangeLog/6.1
2021-08-25 12:13:13 +02:00
Timothy Flynn 7ea55c883b Toolchain: Make Kernel/API headers available to Clang toolchain build
Copied from 9b79867909.
2021-08-17 13:31:43 +01:00
Andreas Kling 9b79867909 Toolchain: Make Kernel/API/ headers available during toolchain build 2021-08-14 20:25:56 +02:00
Daniel Bertalan 657fbc1e6c Toolchain: Remove static LLVM libraries
We link against these dynamically anyways, so having them around is not
useful. Removing them frees precious storage space on CI.
2021-08-08 23:54:00 +02:00
Daniel Bertalan d3595477d8 Toolchain: Build libLLVM as a dynamic library
This library is used by virtually all executables in the Clang
toolchain. By default, it is linked statically, which leads to huge
file sizes and us running out of artifact storage disk space on CI.
2021-08-08 23:54:00 +02:00
Gunnar Beutner a29c18f969 Toolchain+Ports: Let -nostdlib make us not link against libgcc_s
For now this is not a mandatory toolchain rebuild.
2021-08-08 16:41:51 +02:00
Daniel Bertalan 9809e685c1 Toolchain: Add caching to BuildClang.sh for CI 2021-08-08 10:55:36 +02:00
Daniel Bertalan 15e217ea68 Toolchain: Add LLVM patch and script for building it
This contains all the bits and pieces necessary to build a Clang binary
that will correctly compile SerenityOS.

I had some trouble with getting LLVM building with a single command, so
for now, I decided to build each LLVM component in a separate command
invocation. In the future, we can also make the main llvm build step
architecture-independent, but that would come with extra work to make
library and include paths work.

The binutils build invocation and related boilerplate is duplicated
because we only use `objdump` from GNU binutils in the Clang toolchain,
so most features can be disabled.
2021-08-08 10:55:36 +02:00
Linus Groh a613e00caa Ports: Change Python's auth_type to sha256 2021-08-03 21:01:07 +01:00
Gunnar Beutner 5df8c274bb Toolchain+Ports: Fix building ports on x86_64
Fixes #8377.
2021-07-28 21:28:31 +02:00
Gunnar Beutner 9c431ae7ef Toolchain: Update GCC to version 11.2.0 2021-07-28 21:28:31 +02:00
Gunnar Beutner cfeffbe524 Toolchain: Update binutils to version 2.37 2021-07-28 21:28:31 +02:00
Gunnar Beutner 12bfc7b3fc Toolchain+Ports: Ignore -arch on macOS M1 hosts
CMake specifies -arch arm64 for our toolchain. Unfortunately that's an
option GCC only understands when built for macOS. This causes the build
to fail.

I haven't been able to get CMake to not specify that option so this adds
a dummy option to GCC.
2021-07-24 14:04:13 +02:00
Gunnar Beutner 9bc3ad75b4 Toolchain: Use correct variable when deleting the QEMU tarball 2021-07-10 11:13:56 +01:00
modmuss50 036f2a14ed Toolchain: Fix building toolchain on arm64 macs
This patch is based off:
https://github.com/osx-cross/homebrew-avr/pull/248 and
https://github.com/riscv/riscv-gnu-toolchain/issues/800
2021-07-08 16:07:14 +02:00
Gunnar Beutner d86275d271 Kernel+Toolchain: Remove the kernel-specific toolchain
This is no longer necessary now that the kernel doesn't use libsupc++
anymore.
2021-07-06 19:08:22 +02:00
Gunnar Beutner 128e504de6 Meta: Change the QEMU binary directory to Toolchain/Local/qemu
Previously we'd place the QEMU binaries into the architecture-specific
toolchain directory. This is a problem because the BuildIt.sh script
clears those directories which also removes the QEMU binaries users
may have built earlier. Also, the QEMU binaries are not specific to
the target architecture.
2021-07-03 12:06:20 +02:00
kleines Filmröllchen 7ff363127b Toolchain: Allow containerized work with Serenity via Docker
Docker is a nice way of doing build automation, or just
containerizing builds for increased safety and isolating unstable
packages. The old Dockerfile in the toolchain did not satisfy these
needs. The new Dockerfile is known to run successfully on Docker
version 20.10.7. It clones the SerenityOS repo and builds the
toolchain. In this way, it is intended to be a starting point for other
Docker images that can e.g. run builds. For example, one can simply run
this docker image as-is, exec a shell in it and run a build there.
2021-06-29 15:56:02 +02:00
Idan Horowitz 5b909a94e7 Toolchain: Build the x86_64 target in addition to i386 in BuildQemu.sh 2021-06-26 17:59:54 +02:00
Jelle Raaijmakers 3067ee9c4c Toolchain: Add gettext as a dependency to Dockerfile
We need `msgfmt` inside of the `gettext` package in order to build the
git port.
2021-06-07 22:46:53 +02:00
Jelle Raaijmakers 6af9d87258 Toolchain: Add ccache to Dockerfile
Following up on 2d38d56e, we were missing this in our Dockerfile.
2021-06-07 19:22:03 +02:00
Gunnar Beutner 988dfa7f33 Toolchain+Ports: Fix building binutils on FreeBSD
This imports the upstream patch from
https://sourceware.org/bugzilla/show_bug.cgi?id=27382

Fixes #7407.
2021-06-07 09:51:12 +02:00
Jelle Raaijmakers 68f0170bc6 Toolchain: Add ImageMagick to Dockerfile 2021-06-04 12:54:27 +02:00
Hediadyoin1 6fe7d4d7b8 Toolchain: Use gcc's ar
the vanilla versions might not handle all things, that gcc can do;
For example is lto not really supported by the vanilla versions
source:
https://gcc.gnu.org/wiki/LinkTimeOptimizationFAQ
2021-06-03 00:24:06 +01:00
Oleg Kosenkov 839aad6e5b
Toolchain: Set CMAKE_AR to the right executable 2021-06-02 23:25:44 +01:00
Gunnar Beutner 7dedf09f3d Toolchain: Remove dependency check for texinfo
Turns out we don't need that after all.
2021-05-30 23:36:10 +01:00
Gunnar Beutner 259822493f Toolchain: Check whether required tools and libraries are available
Rather than having the toolchain build fail half-way through we should
check whether the user has installed all the required tools and
libraries early on.
2021-05-30 13:06:28 +01:00
Gunnar Beutner afbab621aa Toolchain: Use set -o pipefail to the toolchain build script
Previously the buildstep function would obscure error codes because
the return value of the function was the exit code for the sed command
which caused us to continue execution even though one of the build
steps had failed.

With set -o pipefail the return value of the buildstep function is
the real command's exit code.
2021-05-27 08:58:37 +01:00
Idan Horowitz bcfde43849 Toolchain: Replace the -march=native flag with -mtune=native
This ensures inter-machine compatibility by not emitting any processor
specific instructions. This fixes the issue raised by the non AVX-512
supporting GitHub actions runners.
2021-05-21 15:23:07 +02:00
Brian Gianforcaro 17ff895e1c Toolchain: Enable native host optimizations when building toolchain
-march=native specializes the binaries for the CPU features available on
the CPU the binary is being compiled on. This matches the needs of the
Toolchain, as it's always built and used on that machine only.

This should be safe for the github actions VMs as well, as they all run
on a standard VM SKU in "the cloud".

I saw small but notable improvements in end-2-end build times in my
local testing. Each compilation unit is on average around a second
faster on my Intel(R) Core(TM) i7-8705G CPU @ 3.10GHz.
2021-05-21 10:04:49 +01:00
Gunnar Beutner b223233b15 Toolchain+Ports: Skip link tests for libstdc++v3
This makes stdlib.h and stdio.h functions available in the std
namespace for C++.

libstdc++v3's link tests can fail if you don't have an up-to-date
build directory, for example:

1. Have libc with missing _Exit symbol because you haven't done
   a build since that was added.

2. Run toolchain rebuild. libstdc++v3's configure script will
   realize that it can do link tests in general but will fail
   later on when it tries to link a program that tests for _Exit.

Even though this is a toolchain patch this does not necessarily
require rebuilding the toolchain right away. This is only required
once we start using any of these new members in the std namespace,
e.g. for ports.
2021-05-09 15:35:01 +02:00
Gunnar Beutner 1889006b90 Toolchain: Make -nodefaultlibs not link against libgcc_s
This fixes the -nodefaultlibs flag for gcc which previously
linked against libgcc_s anyway. Even though this is a toolchain
patch we don't need to rebuild the toolchain right away.
2021-05-07 15:35:50 +02:00
Brian Gianforcaro 8b856bd5a1 Toolchain: Fix expansion bugs and make BuildIt.sh shellcheck compliant
BuildIt.sh had a bunch of SC2086 errors, where we were not quoting
variables in variable expansions. The logic being:

    Quoting variables prevents word splitting and glob expansion,
    and prevents the script from breaking when input contains spaces,
    line feeds, glob characters and such.

    Reference: https://github.com/koalaman/shellcheck/wiki/SC2086

As bcoles noticed in #6772, shellcheck actually found a real bug here,
where the user's build directory included spaces.

Close: #6772
2021-05-05 21:26:37 +02:00
Brian Gianforcaro f558a44610 Toolchain: Make BuildFuseExt2.sh shellcheck compliant
BuildFuseExt2.sh was saying it should be run under /bin/sh but it is
using bash extensions like pushd/popd, ${BASH_SOURCE[0]}, etc. So just
run it under bash to avoid any potential issues.
2021-05-05 21:26:37 +02:00
Brian Gianforcaro b4927a753d Toolchain: Make BuildPython.sh shellcheck compliant
Shellcheck is unable to source non-literal includes,
so inform shellcheck to just ignore this include.
2021-05-05 21:26:37 +02:00
Gunnar Beutner d685db6eb6 Toolchain: Enable building all code with -fPIC
Ordinarily this would force the compiler to not inline certain
symbols and call them via the PLT instead. To counteract this
I've also added -fno-semantic-interposition which disables
ELF symbol interposition. Our dynamic loader doesn't support
this anyway and we might even consider not implementing this
at all.

Even though this is a toolchain change this doesn't require
rebuilding the toolchain unless you're planning to build
for the x86_64 arch.
2021-05-03 08:42:39 +02:00
Brian Gianforcaro 5acac54085 Toolchain: Upgrade to QEMU 6.0 release 2021-04-30 08:36:50 +02:00
Gunnar Beutner 19064eaaca Toolchain: Update the binutils patch file to match the port
Previously the toolchain's binutils would not have been able to
build binaries on 32-bit host systems (not that this would be
much of an issue nowadays) because one of the #ifdefs was in
the wrong place.

I moved the #ifdef in the port's patch and this now updates
the toolchain's patch file to match the port's patch.
2021-04-29 10:33:44 +02:00
Gunnar Beutner 8cd62b5780 Toolchain+Ports: Update GCC to version 11.1.0 2021-04-29 10:33:44 +02:00
Brian Gianforcaro d19180433f Toolchain: Update QEMU to 6.0.0-rc5
Changes since rc4:

0cef06d187: Update version for v6.0.0-rc5 release
5351fb7cb2: hw/block/nvme: fix invalid msix exclusive uninit
ffa090bc56: target/s390x: fix s390_probe_access to check PAGE_WRITE_ORG
bc38e31b4e: net: check the existence of peer before trying to pad
2021-04-28 09:43:42 +02:00
Brian Gianforcaro 2ef93a3c07 Build: Use variables when concatenating Toolchain paths.
Make this stuff a bit easier to maintain by using the
root level variables to build up the Toolchain paths.

Also leave a note for future editors of BuildIt.sh to
give them warning about the other changes they'll need
to make.
2021-04-27 13:07:04 +02:00
Jelle Raaijmakers 03569f9686 Toolchain/Dockerfile: Add rsync and unzip
Both utilies are used in the .port_include.sh file.
2021-04-27 08:58:29 +02:00