rustbuild: allow dynamically linking LLVM
The makefiles and `mklldeps.py` called `llvm-config --shared-mode` to
find out if LLVM defaulted to shared or static libraries, and just went
with that. But under rustbuild, `librustc_llvm/build.rs` was assuming
that LLVM should be static, and even forcing `--link-static` for 3.9+.
Now that build script also uses `--shared-mode` to learn the default,
which should work better for pre-3.9 configured for dynamic linking, as
it wasn't possible back then to choose differently via `llvm-config`.
Further, the configure script now has a new `--enable-llvm-link-shared`
option, which allows one to manually override `--link-shared` on 3.9+
instead of forcing static.
Update: There are now four static/shared scenarios that can happen
for the supported LLVM versions:
- 3.9+: By default use `llvm-config --link-static`
- 3.9+ and `--enable-llvm-link-shared`: Use `--link-shared` instead.
- 3.8: Use `llvm-config --shared-mode` and go with its answer.
- 3.7: Just assume static, maintaining the status quo.
The makefiles and `mklldeps.py` called `llvm-config --shared-mode` to
find out if LLVM defaulted to shared or static libraries, and just went
with that. But under rustbuild, `librustc_llvm/build.rs` was assuming
that LLVM should be static, and even forcing `--link-static` for 3.9+.
Now that build script also uses `--shared-mode` to learn the default,
which should work better for pre-3.9 configured for dynamic linking, as
it wasn't possible back then to choose differently via `llvm-config`.
Further, the configure script now has a new `--enable-llvm-link-shared`
option, which allows one to manually override `--link-shared` on 3.9+
instead of forcing static.
Fix grammar verification
* Use make check-lexer to verify the grammar.
* Extend grammar/README
* Add make clean-grammar rule
* Add target check-build-lexer-verifier to make tidy, so it will build the verifier with every build and catch future errors
This is the continuation of #34994
r? @steveklabnik @jonathandturner @alexcrichton
* Use `make check-lexer` to verify the grammar.
* Extend grammar/README
* Add make clean-grammar rule
* Add target `check-build-lexer-verifier` to `make tidy`, so it will build the verifier with every build and catch future errors
* Search for antlr4 with configure and find
Add llvm debuginfo configure option
CC @nnethercote @Mark-Simulacrum
We add a new configure option, `--enable-llvm-debuginfo`, to do exactly what you'd think.
Re: #31033Fixes#37738
A few changes are included here:
* The `winapi` and `url` dependencies were dropped. The source code for these
projects is pretty weighty, and we're about to vendor them, so let's not
commit to that intake just yet. If necessary we can vendor them later but for
now it shouldn't be necessary.
* The `--frozen` flag is now always passed to Cargo, obviating the need for
tidy's `cargo_lock` check.
* Tidy was updated to not check the vendor directory
Closes#34687
armv7l is armv7 architecture and CFG_CPUTYPE should be armv7 in order to end up
with armv7-unknown-linux-gnueabihf.mk rather than
arm-unknown-linux-gnueabihf.mk
This commit enables by default passing the `-C debuginfo=1` argument to the
compiler for the stable, beta, and nightly release channels. A new configure
option was also added, `--enable-debuginfo-lines`, to enable this behavior in
developer builds as well.
Closes#36452
configure: Add options for separate musl roots
This allows using the `./configure` script to enable rustbuild to compile
multiple musl targets at once. We'll hopefully use this soon on our bots to
produce a bunch of targets.
This allows using the `./configure` script to enable rustbuild to compile
multiple musl targets at once. We'll hopefully use this soon on our bots to
produce a bunch of targets.
Allow setting --docdir
This will allow setting `--docdir` during configure, this is useful because not all linux distributions install documentation to `/usr/share/doc`. For example in Slackware documentation is installed to `/usr/doc/$PRGNAM-$VERSION` and `/usr/share/doc` is a symlink to `/usr/doc`.
To use this `./configure --docdir=/usr/doc/$PRGNAM-$VERSION` can be used.
Add s390x support
This adds support for building the Rust compiler and standard
library for s390x-linux, allowing a full cross-bootstrap sequence
to complete. This includes:
- Makefile/configure changes to allow native s390x builds
- Full Rust compiler support for the s390x C ABI
(only the non-vector ABI is supported at this point)
- Port of the standard library to s390x
- Update the liblibc submodule to a version including s390x support
- Testsuite fixes to allow clean "make check" on s390x
Caveats:
- Resets base cpu to "z10" to bring support in sync with the default
behaviour of other compilers on the platforms. (Usually, upstream
supports all older processors; a distribution build may then chose
to require a more recent base version.) (Also, using zEC12 causes
failures in the valgrind tests since valgrind doesn't fully support
this CPU yet.)
- z13 vector ABI is not yet supported. To ensure compatible code
generation, the -vector feature is passed to LLVM. Note that this
means that even when compiling for z13, no vector instructions
will be used. In the future, support for the vector ABI should be
added (this will require common code support for different ABIs
that need different data_layout strings on the same platform).
- Two test cases are (temporarily) ignored on s390x to allow passing
the test suite. The underlying issues still need to be fixed:
* debuginfo/simd.rs fails because of incorrect debug information.
This seems to be a LLVM bug (also seen with C code).
* run-pass/union/union-basic.rs simply seems to be incorrect for
all big-endian platforms.
Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
This adds support for building the Rust compiler and standard
library for s390x-linux, allowing a full cross-bootstrap sequence
to complete. This includes:
- Makefile/configure changes to allow native s390x builds
- Full Rust compiler support for the s390x C ABI
(only the non-vector ABI is supported at this point)
- Port of the standard library to s390x
- Update the liblibc submodule to a version including s390x support
- Testsuite fixes to allow clean "make check" on s390x
Caveats:
- Resets base cpu to "z10" to bring support in sync with the default
behaviour of other compilers on the platforms. (Usually, upstream
supports all older processors; a distribution build may then chose
to require a more recent base version.) (Also, using zEC12 causes
failures in the valgrind tests since valgrind doesn't fully support
this CPU yet.)
- z13 vector ABI is not yet supported. To ensure compatible code
generation, the -vector feature is passed to LLVM. Note that this
means that even when compiling for z13, no vector instructions
will be used. In the future, support for the vector ABI should be
added (this will require common code support for different ABIs
that need different data_layout strings on the same platform).
- Two test cases are (temporarily) ignored on s390x to allow passing
the test suite. The underlying issues still need to be fixed:
* debuginfo/simd.rs fails because of incorrect debug information.
This seems to be a LLVM bug (also seen with C code).
* run-pass/union/union-basic.rs simply seems to be incorrect for
all big-endian platforms.
Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
Use libraries from local-rust-root directory in configure when using …
…--enable-local-rebuild
When using --enable-local-rebuild configure options, the configure
script will test rustc version. But when running it, it will not use the
libraries in the local-rust-root directory.
So use `LD_LIBRARY_PATH` environment variable to correct it.
Under OpenBSD, we use `--enable-local-rebuild` for rebuilding rustc-1.11.0 using rustc-1.11.0. But as it is in use in -current (the developpment branch of OpenBSD), system libraries could easily have ABI/API changes. The (unofficial) bootstrap (rustc-1.11.0 used for building rustc-1.11.0) include several system libraries to permit it to run. The build system use libraries in stage0/lib directory. But the configure script doesn't.
Due to my special use of `--enable-local-rebuild`, I dunno if this PR is suitable for inclusion. But it corrects a difference between build and configure use of the binary.
r? @alexcrichton
Remove --{enable|disable}-orbit from configure.
Fixes#35956.
r? @eddyb
There are only two buildbots left, though they are both failing. Is there something to be done there other than wait?
When using --enable-local-rebuild configure options, the configure
script will test rustc version. But when running it, it will not use the
libraries in the local-rust-root directory.
So use `LD_LIBRARY_PATH` environment variable to correct it.
LLVM upgrade
As discussed in https://internals.rust-lang.org/t/need-help-with-emscripten-port/3154/46 I'm trying to update the used LLVM checkout in Rust.
I basically took @shepmaster's code and applied it on top (though I did the commits manually, the [original commits have better descriptions](https://github.com/rust-lang/rust/compare/master...avr-rust:avr-support).
With these changes I was able to build rustc. `make check` throws one last error on `run-pass/issue-28950.rs`. Output: https://gist.github.com/badboy/bcdd3bbde260860b6159aa49070a9052
I took the metadata changes as is and they seem to work, though it now uses the module in another step. I'm not sure if this is the best and correct way.
Things to do:
* [x] ~~Make `run-pass/issue-28950.rs` pass~~ unrelated
* [x] Find out how the `PositionIndependentExecutable` setting is now used
* [x] Is the `llvm::legacy` still the right way to do these things?
cc @brson @alexcrichton
The targets are:
- `arm-unknown-linux-musleabi`
- `arm-unknown-linux-musleabihf`
- `armv7-unknown-linux-musleabihf`
These mirror the existing `gnueabi` targets.
All of these targets produce fully static binaries, similar to the
x86 MUSL targets.
For now these targets can only be used with `--rustbuild` builds, as
https://github.com/rust-lang/compiler-rt/pull/22 only made the
necessary compiler-rt changes in the CMake configs, not the plain
GNU Make configs.
I've tested these targets GCC 5.3.0 compiled again musl-1.1.12
(downloaded from http://musl.codu.org/). An example `./configure`
invocation is:
```
./configure \
--enable-rustbuild
--target=arm-unknown-linux-musleabi \
--musl-root="$MUSL_ROOT"
```
where `MUSL_ROOT` points to the `arm-linux-musleabi` prefix.
Usually that path will be of the form
`/foobar/arm-linux-musleabi/arm-linux-musleabi`.
Usually the cross-compile toolchain will live under
`/foobar/arm-linux-musleabi/bin`. That path should either by added
to your `PATH` variable, or you should add a section to your
`config.toml` as follows:
```
[target.arm-unknown-linux-musleabi]
cc = "/foobar/arm-linux-musleabi/bin/arm-linux-musleabi-gcc"
cxx = "/foobar/arm-linux-musleabi/bin/arm-linux-musleabi-g++"
```
As a prerequisite you'll also have to put a cross-compiled static
`libunwind.a` library in `$MUSL_ROOT/lib`. This is similar to [how
the x86_64 MUSL targets are built]
(https://doc.rust-lang.org/book/advanced-linking.html).
llvm, rt: build using the Ninja generator if available
The Ninja generator generally builds much faster than make. It may also
be used on Windows to have a vast speed improvement over the Visual
Studio generators.
Currently hidden behind an `--enable-ninja` flag because it does not
obey the top-level `-j` or `-l` flags given to `make`.
Use --release-channel=stable by default on releases
> Release tarballs should be compilable with just basic ./configure ;
> make ; sudo make install without having to pass special flags to
> configure. This is the case of the --release-channel option, that must
> be changed in the releases.
This commit detects the presence of .git, as it happens on other parts
of `configure` to assume it is a tarball. Then it changes the default
value stored, before parsing the arguments, while still allowing it to
be overriden before any action verifying the flag is done.
Closes#28322
The Ninja generator generally builds much faster than make. It may also
be used on Windows to have a vast speed improvement over the Visual
Studio generators.
Currently hidden behind an `--enable-ninja` flag because it does not
obey the top-level `-j` or `-l` flags given to `make`.
configure: Check for valid Python on MinGW as well
The LLVM build system is somewhat picky about which Python is used to build it
as it's known to be incompatible with the default `python2` package that ships
with MinGW. This was previously detected for MSVC builds but the logic was left
out for MinGW by accident (now that we've switched to cmake builds for LLVM
everywhere).
This corrects the `./configure` check and also updates the `README.md`
accordingly. Additionally, a number of instructions were updated to work with
the most recent copy of MSYS2.
Closes https://github.com/rust-lang/rust/issues/28260Closes#34489
Actually got it working this time, and it was again just a problem specifying
the llvm-tblgen binary. We need to point it at the $CFG_BUILD target's tblgen
and then we also needed to correct the path a bit.
The LLVM build system is somewhat picky about which Python is used to build it
as it's known to be incompatible with the default `python2` package that ships
with MinGW. This was previously detected for MSVC builds but the logic was left
out for MinGW by accident (now that we've switched to cmake builds for LLVM
everywhere).
This corrects the `./configure` check and also updates the `README.md`
accordingly. Additionally, a number of instructions were updated to work with
the most recent copy of MSYS2.
Closes#34489
Convert makefiles to build LLVM/compiler-rt with CMake
This is certainly buggy, but I have successfully built on x86_64-unknown-linux-gnu and x86_64-pc-windows-gnu. I haven't built successfully on mac yet, and I've seen mysterious test failures on Linux, but I'm interested in throwing this at the bots to see what they think.
We no C++ and an incredibly small amount of C code as part of the build, so
there's not really much need for us to strictly check the version of compilers
as we're not really stressing anything. LLVM is a pretty huge chunk of C++ but
it should be the responsibility of LLVM to ensure that it can build with a
particular clang/gcc version, not ours (as this logic changes over time).
These version checks seem to basically just by us a regular stream of PRs every
six weeks or so when a new version is releases, so they're not really buying us
much. As a result, remove them and we can add then back piecemeal perhaps as a
blacklist if we really need to.
> Release tarballs should be compilable with just basic ./configure ;
> make ; sudo make install without having to pass special flags to
> configure. This is the case of the --release-channel option, that must
> be changed in the releases.
This commit detects the presence of .git, as it happens on other parts
of `configure` to assume it is a tarball. Then it changes the default
value stored, before parsing the arguments, while still allowing it to
be overriden before any action verifying the flag is done.
Closes#28322
In Linux distributions, it is often necessary to rebuild packages for
cases like applying new patches or linking against new system libraries.
In this scenario, the rustc in the distro build environment may already
match the current release that we're trying to rebuild. Thus we don't
want to use the prior release's bootstrap key, nor `--cfg stage0` for
the prior unstable features.
The new `configure --enable-local-rebuild` option specifies that we are
rebuilding from the current release. The current bootstrap key is used
for the local rustc, and current stage1 features are also assumed.
Sanity check Python on OSX for LLDB tests
Two primary changes:
* Don't get past the configure stage if `python` isn't coming from `/usr/bin`
* Call `debugger.Terminate()` to prevent segfaults on newer versions of LLDB.
Closes#32994
mk: Hardcode the bootstrap key for each release
Starting with the 1.10.0 release we would like to bootstrap all compilers from
the previous stable release. For example the 1.10.0 compiler should bootstrap
from the literal 1.9.0 release artifacts. To do this, however, we need a way to
enable unstable features temporarily in a stable compiler (as the released
compiler is stable), but it turns out we already have a way to do that!
At compile time the configure script selects a `CFG_BOOTSTRAP_KEY` variable
value and then exports it into the makefiles. If the `RUSTC_BOOTSTRAP_KEY`
environment variable is set to this value, then the compiler is allowed to
"cheat" and use unstable features.
This method of choosing the bootstrap key, however, is problematic for the
intention of bootstrapping from the previous release. Each time a 1.9.0 compiler
is created, a new bootstrap key will be selected. That means that the 1.10.0
compiler will only compile from *our* literal release artifacts. Instead
distributions would like to bootstrap from their own compilers, so instead we
simply hardcode the bootstrap key for each release.
This patch uses the same `CFG_FILENAME_EXTRA` value (a hash of the release
string) as the bootstrap key. Consequently all 1.9.0 compilers, no matter where
they are compiled, will have the same bootstrap key. Additionally we won't need
to keep updating this as it'll be based on the release number anyway.
Once the 1.9.0 beta has been created, we can update the 1.10.0 nightly sources
(the `master` branch at that time) to bootstrap from that release using this
hard-coded bootstrap key. We will likely just hardcode into the makefiles what
the previous bootstrap key was and we'll change that whenever the stage0
compiler is updated.
Starting with the 1.10.0 release we would like to bootstrap all compilers from
the previous stable release. For example the 1.10.0 compiler should bootstrap
from the literal 1.9.0 release artifacts. To do this, however, we need a way to
enable unstable features temporarily in a stable compiler (as the released
compiler is stable), but it turns out we already have a way to do that!
At compile time the configure script selects a `CFG_BOOTSTRAP_KEY` variable
value and then exports it into the makefiles. If the `RUSTC_BOOTSTRAP_KEY`
environment variable is set to this value, then the compiler is allowed to
"cheat" and use unstable features.
This method of choosing the bootstrap key, however, is problematic for the
intention of bootstrapping from the previous release. Each time a 1.9.0 compiler
is created, a new bootstrap key will be selected. That means that the 1.10.0
compiler will only compile from *our* literal release artifacts. Instead
distributions would like to bootstrap from their own compilers, so instead we
simply hardcode the bootstrap key for each release.
This patch uses the same `CFG_FILENAME_EXTRA` value (a hash of the release
string) as the bootstrap key. Consequently all 1.9.0 compilers, no matter where
they are compiled, will have the same bootstrap key. Additionally we won't need
to keep updating this as it'll be based on the release number anyway.
Once the 1.9.0 beta has been created, we can update the 1.10.0 nightly sources
(the `master` branch at that time) to bootstrap from that release using this
hard-coded bootstrap key. We will likely just hardcode into the makefiles what
the previous bootstrap key was and we'll change that whenever the stage0
compiler is updated.
Our `codegen` test suite requires the LLVM `FileCheck` utility but unfortunately
this isn't always available in all custom LLVM roots (e.g. those specified via
`--llvm-root`). This commit adds a `./configure` option called
`--disable-codegen-tests` which will manually disable running these tests. In
the case that this option is passed we can forgo the need for the `FileCheck`
executable. Note that we still require `FileCheck` by default as we will attempt
to run these tests.
Closes#28667
Detect the triple in the configure script for probing MSVC shenanigans and also
be sure to use `llvm-config` from the build host and not the target when
configuring compiler-rt.
Rust 1.7.0 and newer appears to require LLVM 3.6.0 or newer when
building against a version that's out of the tree with the --llvm-root
flag.
Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
The `--disable-jemalloc` configure option has a failure mode where it will
create a distribution that is not compatible with other compilers. For example
the nightly for Linux will assume that it will link to jemalloc by default as
an allocator for executable crates. If, however, a standard library is used
which was built via `./configure --disable-jemalloc` then this will fail
because the jemalloc crate wasn't built.
While this seems somewhat reasonable as a niche situation, the same mechanism is
used for disabling jemalloc for platforms that just don't support it. For
example if the rumprun target is compiled then the sibiling Linux target *also*
doesn't have jemalloc. This is currently a problem for our cross-build nightlies
which build many targets. If rumprun is also built, it will disable jemalloc for
all targets, which isn't desired.
This commit moves the platform-specific disabling of jemalloc as hardcoded logic
into the makefiles that is scoped per-platform. This way when configuring
multiple targets **without the `--disable-jemalloc` option specified** all
targets will get jemalloc as they should.
In other words, enforce what was documented in #30626 (and also stop blaming it on LLVM, we have at least one Python script of our own).
Also, there is no Python later than 2.7 and there never will be.
This commit adds a `--enable-rustbuild` option to the configure script which
will copy a different `Makefile.in` into place to intercept all `make`
invocations.
Currently this makefile only has one target, but it's expected to be filled out
quite a bit over time!
Backtraces, and the compilation of libbacktrace for asmjs, are disabled.
This port doesn't use jemalloc so, like pnacl, it disables jemalloc *for all targets*
in the configure file.
It disables stack protection.
This target covers MIPS devices that run the trunk version of OpenWRT.
The x86_64-unknown-linux-musl target always links statically to C libraries. For
the mips(el)-unknown-linux-musl target, we opt for dynamic linking (like most of
other targets do) to keep binary size down.
As for the C compiler flags used in the build system, we use the same flags used
for the mips(el)-unknown-linux-gnu target.
r? @alexcrichton
This target covers MIPS devices that run the trunk version of OpenWRT.
The x86_64-unknown-linux-musl target always links statically to C libraries. For
the mips(el)-unknown-linux-musl target, we opt for dynamic linking (like most of
other targets do) to keep binary size down.
As for the C compiler flags used in the build system, we use the same flags used
for the mips(el)-unknown-linux-gnu target.
The purpose of the translation item collector is to find all monomorphic instances of functions, methods and statics that need to be translated into LLVM IR in order to compile the current crate.
So far these instances have been discovered lazily during the trans path. For incremental compilation we want to know the set of these instances in advance, and that is what the trans::collect module provides.
In the future, incremental and regular translation will be driven by the collector implemented here.
We've been seeing a lot of timeouts in tests on the bots and investigation ended
pointing to jemalloc/jemalloc#315 as the culprit. Unfortunately it looks like
that doesn't seem to have a fix on the way soon, so let's temporarily downgrade
back to the previous version of jemalloc we were using (where #30434 was the
most recent upgrade)
We've been seeing a lot of timeouts in tests on the bots and investigation ended
pointing to jemalloc/jemalloc#315 as the culprit. Unfortunately it looks like
that doesn't seem to have a fix on the way soon, so let's temporarily downgrade
back to the previous version of jemalloc we were using (where #30434 was the
most recent upgrade)
