languages/rust
languages/rust
1
0
Fork 0
mirror of https://github.com/rust-lang/rust synced 2024-09-15 22:50:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

Auto merge of #46941 - ScottAbbey:freebsd-build-update, r=alexcrichton

Browse source 
Re-do the FreeBSD cross-builds to use Clang and libc++. Fixes #44433

Reviving #45077, from @jld:

> The main goal here is to use FreeBSD's normal libc++, instead of
> statically linking the libstdc++ packaged with GCC, because that
> libstdc++ has bugs that cause rustc to deadlock inside LLVM.
>
> But the easiest way to use libc++ is to switch the build from GCC to
> Clang, and the Clang package in the Ubuntu image already knows how to
> cross-compile (given a sysroot and preferably cross-binutils), so the
> toolchain script now uses that instead of building a custom compiler.
>
> This also de-duplicates the build-toolchain.sh script.

#45077 was close but didn't quite make it.  I rebased @jld's work off the current `master` and started with that.

I was able to determine that this Travis error (https://github.com/rust-lang/rust/pull/45077#issuecomment-336029862) was ultimately caused by `src/librustc_llvm/build.rs` attempting to follow a wrong value in `LLVM_STATIC_STDCPP` (https://github.com/rust-lang/rust/pull/45077#issuecomment-352639456).

I looked at the downstream port for FreeBSD (https://svnweb.freebsd.org/ports/head/lang/rust/) and it seems like they do not use `--enable-llvm-static-stdcpp`.

Since `libc++` is included in the FreeBSD 10+ base system, we don't need to statically link it either?

So in b989428f7d I have set the FreeBSD build to not actually use `LLVM_STATIC_STDCPP`.

I was able to run `./src/ci/docker/run.sh` with both `dist-i686-freebsd` and `dist-x86_64-freebsd` successfully and in about 1 minute of testing it seemed like the dist-x86_64-freebsd results worked on a FreeBSD 11 system.

It should fix #44433, which seems to be affecting many potential users.  Also FreeBSD users should be able to `./x.py build` which should help anyone who wants to upstream fixes for FreeBSD.

Questions:

Does this approach seem to be the right way to go? Do we actually really want to statically link `libc++`? (I tried that here, but it ultimately ran into a roadblock on x86_64: https://github.com/rust-lang/rust/pull/45077#issuecomment-353293414)

Can we rewrite the comment here to be more clear about why some systems aren't going to actually use this option:
b989428f7d/src/bootstrap/compile.rs (L550-L553)

How does this affect users of older FreeBSD systems? It seemed like no one was complaining about using a 10.3 base version in the thread for #45077.  FreeBSD seems to only officially support 10.3, 10.4, and 11.x right now, do we have to consider older users? The `libc++` stuff came in for FreeBSD 10, older FreeBSD used `libstdc++`.

Looks like @alexcrichton was leading the discussion on the previous issue:

r? @alexcrichton

Let me know what I can do to help get this through.
This commit is contained in:
bors 2017-12-26 11:16:12 +00:00
parent 8cdde6db71 b989428f7d
commit 0efdfa1d62
7 changed files with 116 additions and 234 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
src/bootstrap/compile.rs
View file

@ -550,6 +550,7 @@ pub fn rustc_cargo(build: &Build,
// Building with a static libstdc++ is only supported on linux right now,
// not for MSVC or macOS
if build.config.llvm_static_stdcpp &&
!target.contains("freebsd") &&
!target.contains("windows") &&
!target.contains("apple") {
cargo.env("LLVM_STATIC_STDCPP",

10
src/ci/docker/dist-i686-freebsd/Dockerfile
View file

@ -1,7 +1,7 @@
FROM ubuntu:16.04
RUN apt-get update && apt-get install -y --no-install-recommends \
g++ \
clang \
make \
file \
curl \
@ -16,16 +16,16 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
libssl-dev \
pkg-config
COPY dist-i686-freebsd/build-toolchain.sh /tmp/
RUN /tmp/build-toolchain.sh i686
COPY scripts/freebsd-toolchain.sh /tmp/
RUN /tmp/freebsd-toolchain.sh i686
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENV \
AR_i686_unknown_freebsd=i686-unknown-freebsd10-ar \
CC_i686_unknown_freebsd=i686-unknown-freebsd10-gcc \
CXX_i686_unknown_freebsd=i686-unknown-freebsd10-g++
CC_i686_unknown_freebsd=i686-unknown-freebsd10-clang \
CXX_i686_unknown_freebsd=i686-unknown-freebsd10-clang++
ENV HOSTS=i686-unknown-freebsd

112
src/ci/docker/dist-i686-freebsd/build-toolchain.sh
View file

@ -1,112 +0,0 @@
#!/usr/bin/env bash
# Copyright 2016 The Rust Project Developers. See the COPYRIGHT
# file at the top-level directory of this distribution and at
# http://rust-lang.org/COPYRIGHT.
#
# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
# option. This file may not be copied, modified, or distributed
# except according to those terms.
set -ex
ARCH=$1
BINUTILS=2.25.1
GCC=6.4.0
hide_output() {
set +x
on_err="
echo ERROR: An error was encountered with the build.
cat /tmp/build.log
exit 1
"
trap "$on_err" ERR
bash -c "while true; do sleep 30; echo \$(date) - building ...; done" &
PING_LOOP_PID=$!
$@ &> /tmp/build.log
trap - ERR
kill $PING_LOOP_PID
set -x
}
mkdir binutils
cd binutils
# First up, build binutils
curl https://ftp.gnu.org/gnu/binutils/binutils-$BINUTILS.tar.bz2 | tar xjf -
mkdir binutils-build
cd binutils-build
hide_output ../binutils-$BINUTILS/configure \
--target=$ARCH-unknown-freebsd10
hide_output make -j10
hide_output make install
cd ../..
rm -rf binutils
# Next, download the FreeBSD libc and relevant header files
mkdir freebsd
case "$ARCH" in
x86_64)
URL=ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/10.2-RELEASE/base.txz
;;
i686)
URL=ftp://ftp.freebsd.org/pub/FreeBSD/releases/i386/10.2-RELEASE/base.txz
;;
esac
curl $URL | tar xJf - -C freebsd ./usr/include ./usr/lib ./lib
dst=/usr/local/$ARCH-unknown-freebsd10
cp -r freebsd/usr/include $dst/
cp freebsd/usr/lib/crt1.o $dst/lib
cp freebsd/usr/lib/Scrt1.o $dst/lib
cp freebsd/usr/lib/crti.o $dst/lib
cp freebsd/usr/lib/crtn.o $dst/lib
cp freebsd/usr/lib/libc.a $dst/lib
cp freebsd/usr/lib/libutil.a $dst/lib
cp freebsd/usr/lib/libutil_p.a $dst/lib
cp freebsd/usr/lib/libm.a $dst/lib
cp freebsd/usr/lib/librt.so.1 $dst/lib
cp freebsd/usr/lib/libexecinfo.so.1 $dst/lib
cp freebsd/lib/libc.so.7 $dst/lib
cp freebsd/lib/libm.so.5 $dst/lib
cp freebsd/lib/libutil.so.9 $dst/lib
cp freebsd/lib/libthr.so.3 $dst/lib/libpthread.so
ln -s libc.so.7 $dst/lib/libc.so
ln -s libm.so.5 $dst/lib/libm.so
ln -s librt.so.1 $dst/lib/librt.so
ln -s libutil.so.9 $dst/lib/libutil.so
ln -s libexecinfo.so.1 $dst/lib/libexecinfo.so
rm -rf freebsd
# Finally, download and build gcc to target FreeBSD
mkdir gcc
cd gcc
curl https://ftp.gnu.org/gnu/gcc/gcc-$GCC/gcc-$GCC.tar.gz | tar xzf -
cd gcc-$GCC
./contrib/download_prerequisites
mkdir ../gcc-build
cd ../gcc-build
hide_output ../gcc-$GCC/configure \
--enable-languages=c,c++ \
--target=$ARCH-unknown-freebsd10 \
--disable-multilib \
--disable-nls \
--disable-libgomp \
--disable-libquadmath \
--disable-libssp \
--disable-libvtv \
--disable-libcilkrts \
--disable-libada \
--disable-libsanitizer \
--disable-libquadmath-support \
--disable-lto
hide_output make -j10
hide_output make install
cd ../..
rm -rf gcc

10
src/ci/docker/dist-x86_64-freebsd/Dockerfile
View file

@ -1,7 +1,7 @@
FROM ubuntu:16.04
RUN apt-get update && apt-get install -y --no-install-recommends \
g++ \
clang \
make \
file \
curl \
@ -16,16 +16,16 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
libssl-dev \
pkg-config
COPY dist-x86_64-freebsd/build-toolchain.sh /tmp/
RUN /tmp/build-toolchain.sh x86_64
COPY scripts/freebsd-toolchain.sh /tmp/
RUN /tmp/freebsd-toolchain.sh x86_64
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENV \
AR_x86_64_unknown_freebsd=x86_64-unknown-freebsd10-ar \
CC_x86_64_unknown_freebsd=x86_64-unknown-freebsd10-gcc \
CXX_x86_64_unknown_freebsd=x86_64-unknown-freebsd10-g++
CC_x86_64_unknown_freebsd=x86_64-unknown-freebsd10-clang \
CXX_x86_64_unknown_freebsd=x86_64-unknown-freebsd10-clang++
ENV HOSTS=x86_64-unknown-freebsd

112
src/ci/docker/dist-x86_64-freebsd/build-toolchain.sh
View file

@ -1,112 +0,0 @@
#!/usr/bin/env bash
# Copyright 2016 The Rust Project Developers. See the COPYRIGHT
# file at the top-level directory of this distribution and at
# http://rust-lang.org/COPYRIGHT.
#
# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
# option. This file may not be copied, modified, or distributed
# except according to those terms.
set -ex
ARCH=$1
BINUTILS=2.25.1
GCC=6.4.0
hide_output() {
set +x
on_err="
echo ERROR: An error was encountered with the build.
cat /tmp/build.log
exit 1
"
trap "$on_err" ERR
bash -c "while true; do sleep 30; echo \$(date) - building ...; done" &
PING_LOOP_PID=$!
$@ &> /tmp/build.log
trap - ERR
kill $PING_LOOP_PID
set -x
}
mkdir binutils
cd binutils
# First up, build binutils
curl https://ftp.gnu.org/gnu/binutils/binutils-$BINUTILS.tar.bz2 | tar xjf -
mkdir binutils-build
cd binutils-build
hide_output ../binutils-$BINUTILS/configure \
--target=$ARCH-unknown-freebsd10
hide_output make -j10
hide_output make install
cd ../..
rm -rf binutils
# Next, download the FreeBSD libc and relevant header files
mkdir freebsd
case "$ARCH" in
x86_64)
URL=ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/10.2-RELEASE/base.txz
;;
i686)
URL=ftp://ftp.freebsd.org/pub/FreeBSD/releases/i386/10.2-RELEASE/base.txz
;;
esac
curl $URL | tar xJf - -C freebsd ./usr/include ./usr/lib ./lib
dst=/usr/local/$ARCH-unknown-freebsd10
cp -r freebsd/usr/include $dst/
cp freebsd/usr/lib/crt1.o $dst/lib
cp freebsd/usr/lib/Scrt1.o $dst/lib
cp freebsd/usr/lib/crti.o $dst/lib
cp freebsd/usr/lib/crtn.o $dst/lib
cp freebsd/usr/lib/libc.a $dst/lib
cp freebsd/usr/lib/libutil.a $dst/lib
cp freebsd/usr/lib/libutil_p.a $dst/lib
cp freebsd/usr/lib/libm.a $dst/lib
cp freebsd/usr/lib/librt.so.1 $dst/lib
cp freebsd/usr/lib/libexecinfo.so.1 $dst/lib
cp freebsd/lib/libc.so.7 $dst/lib
cp freebsd/lib/libm.so.5 $dst/lib
cp freebsd/lib/libutil.so.9 $dst/lib
cp freebsd/lib/libthr.so.3 $dst/lib/libpthread.so
ln -s libc.so.7 $dst/lib/libc.so
ln -s libm.so.5 $dst/lib/libm.so
ln -s librt.so.1 $dst/lib/librt.so
ln -s libutil.so.9 $dst/lib/libutil.so
ln -s libexecinfo.so.1 $dst/lib/libexecinfo.so
rm -rf freebsd
# Finally, download and build gcc to target FreeBSD
mkdir gcc
cd gcc
curl https://ftp.gnu.org/gnu/gcc/gcc-$GCC/gcc-$GCC.tar.gz | tar xzf -
cd gcc-$GCC
./contrib/download_prerequisites
mkdir ../gcc-build
cd ../gcc-build
hide_output ../gcc-$GCC/configure \
--enable-languages=c,c++ \
--target=$ARCH-unknown-freebsd10 \
--disable-multilib \
--disable-nls \
--disable-libgomp \
--disable-libquadmath \
--disable-libssp \
--disable-libvtv \
--disable-libcilkrts \
--disable-libada \
--disable-libsanitizer \
--disable-libquadmath-support \
--disable-lto
hide_output make -j10
hide_output make install
cd ../..
rm -rf gcc

103
src/ci/docker/scripts/freebsd-toolchain.sh Executable file
View file

@ -0,0 +1,103 @@
#!/bin/bash
# Copyright 2016-2017 The Rust Project Developers. See the COPYRIGHT
# file at the top-level directory of this distribution and at
# http://rust-lang.org/COPYRIGHT.
#
# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
# option. This file may not be copied, modified, or distributed
# except according to those terms.
set -eux
arch=$1
binutils_version=2.25.1
freebsd_version=10.3
triple=$arch-unknown-freebsd10
sysroot=/usr/local/$triple
hide_output() {
set +x
local on_err="
echo ERROR: An error was encountered with the build.
cat /tmp/build.log
exit 1
"
trap "$on_err" ERR
bash -c "while true; do sleep 30; echo \$(date) - building ...; done" &
local ping_loop_pid=$!
$@ &> /tmp/build.log
trap - ERR
kill $ping_loop_pid
set -x
}
# First up, build binutils
mkdir binutils
cd binutils
curl https://ftp.gnu.org/gnu/binutils/binutils-${binutils_version}.tar.bz2 | tar xjf -
mkdir binutils-build
cd binutils-build
hide_output ../binutils-${binutils_version}/configure \
--target="$triple" --with-sysroot="$sysroot"
hide_output make -j"$(getconf _NPROCESSORS_ONLN)"
hide_output make install
cd ../..
rm -rf binutils
# Next, download the FreeBSD libraries and header files
mkdir -p "$sysroot"
case $arch in
(x86_64) freebsd_arch=amd64 ;;
(i686) freebsd_arch=i386 ;;
esac
files_to_extract=(
"./usr/include"
"./usr/lib/*crt*.o"
)
# Try to unpack only the libraries the build needs, to save space.
for lib in c cxxrt gcc_s m thr util; do
files_to_extract=("${files_to_extract[@]}" "./lib/lib${lib}.*" "./usr/lib/lib${lib}.*")
done
for lib in c++ c_nonshared compiler_rt execinfo gcc pthread rt ssp_nonshared; do
files_to_extract=("${files_to_extract[@]}" "./usr/lib/lib${lib}.*")
done
URL=https://download.freebsd.org/ftp/releases/${freebsd_arch}/${freebsd_version}-RELEASE/base.txz
curl "$URL" | tar xJf - -C "$sysroot" --wildcards "${files_to_extract[@]}"
# Fix up absolute symlinks from the system image. This can be removed
# for FreeBSD 11. (If there's an easy way to make them relative
# symlinks instead, feel free to change this.)
set +x
find "$sysroot" -type l | while read symlink_path; do
symlink_target=$(readlink "$symlink_path")
case $symlink_target in
(/*)
echo "Fixing symlink ${symlink_path} -> ${sysroot}${symlink_target}" >&2
ln -nfs "${sysroot}${symlink_target}" "${symlink_path}" ;;
esac
done
set -x
# Clang can do cross-builds out of the box, if we give it the right
# flags. (The local binutils seem to work, but they set the ELF
# header "OS/ABI" (EI_OSABI) field to SysV rather than FreeBSD, so
# there might be other problems.)
#
# The --target option is last because the cross-build of LLVM uses
# --target without an OS version ("-freebsd" vs. "-freebsd10"). This
# makes Clang default to libstdc++ (which no longer exists), and also
# controls other features, like GNU-style symbol table hashing and
# anything predicated on the version number in the __FreeBSD__
# preprocessor macro.
for tool in clang clang++; do
tool_path=/usr/local/bin/${triple}-${tool}
cat > "$tool_path" <<EOF
#!/bin/sh
exec $tool --sysroot=$sysroot --prefix=${sysroot}/bin "\$@" --target=$triple
EOF
chmod +x "$tool_path"
done

2
src/librustc_llvm/build.rs
View file

@ -234,6 +234,8 @@ fn main() {
let stdcppname = if target.contains("openbsd") {
// llvm-config on OpenBSD doesn't mention stdlib=libc++
"c++"
} else if target.contains("freebsd") {
"c++"
} else if target.contains("netbsd") && llvm_static_stdcpp.is_some() {
// NetBSD uses a separate library when relocation is required
"stdc++_pic"