crunchgen generates a foo.lo for each binary it will end up crunching
into the final product. While they have a dependency on the libs that
are used to link them, nothing will force relinking if the set of libs
needed to link them is changed. Because of this, incremental builds may
not be possible if one builds a version of, e.g., rescue/ with a broken
set of libs specified for a project -- a subsequent fix won't be rolled
in cleanly, it will require purging the rescue/ objdir.
This is a bit crude, but the foo.mk we generate doesn't actually get
regenerated all that often in practice, so a spurious relink for the
vast majority of crunched objects won't actually happen all that often.
Reviewed by: bapt, emaste, imp
(cherry picked from commit 6e2cfb24ee)
"keep" currently adds a leading underscore, which hasn't been useful or
accurate since a.out days. Preserve the symbol name as it's given
rather than mangle it to match ELF-style symbol names.
This was partially fixed back in
6cd35234a0 ("Assume ELF-style symbol names now.") for crunchgen, but
the keeplist wasn't changed to match it.
While we're here, break it out to bsd.crunchgen.mk for later use in
bsdbox.
Reviewed by: adrian, imp
(cherry picked from commit 8f2848eafa)
In GNU ld and ld.lld, -dc is used with -r to allocate space to COMMON
symbols. It is presumably to work around legacy code which cannot
handle COMMON symbols in relocatable output. ld.lld may remove -dc or
make it a no-op for the 15.0.0 release.
As of 7420b323a0 crunch/crunchide does not require -dc, as the symbol
hiding technique no longer relied on making symbols local.
In addition -fno-common is now the default in Clang and GCC, so -dc
serves no purpose as the compiler does not generate COMMON symbols
anyway.
See https://maskray.me/blog/2022-02-06-all-about-common-symbols for more
detail on common symbols.
Reviewed by: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D34215
(cherry picked from commit ec81497cc7)
- One (1) constify
- One (1) argument is unused
- One (1) local shadows a global
- Various globals that should be static
(cherry picked from commit dd7c7ff192)
It can be assumed that the contents of the buffer was still allocated and
valid at the point of the out-of-scope access, so there was no security
issue in practice.
Reported by: Coverity Scan CID 1437697
MFC after: 3 days
While porting over the local changes from CheriBSD for upstreaming, I
accidentally committed a broken version of find_entry_point(): we have to
return NULL if the value is not found instead of a value with
ep->name == NULL, since the checks in main were changed to check ep instead
of ep->name for NULL.
This only matters if the crunched tool cannot be found using normal lookup
and one of the fallback paths is used, so it's unlikely to be triggered
in rescue. However, I noticed that one of our CheriBSD test scripts was
failing to run commands under `su` on minimal disk images where all
binaries are hardlinks to a `cheribsdbox` tool generated with crunchgen.
This also updates the bootstrapping check in Makefile.inc1 to bootstrap
crunchgen up to the next version bump.
Reviewed By: kevans
Differential Revision: https://reviews.freebsd.org/D27474
r364166 converted echo -n `/bin/pwd` to a raw pwd invocation, leaving a
trailing newline at the end of path. This caused a later stat() of it to
erroneously fail and the fallback to MK_AUTO_OBJ=no logic proceeded as
unexpected.
Harry Schmalzbauer bissected the resulting build failure he experienced
(stable/12 host, -HEAD build) down to r365887. This change is mostly
unrelated, except it switches the build to bootstrapped crunchgen - clue!
I then bissected recent crunchgen changes going back a bit since we wouldn't
observe the failure immediately with -CURRENT in most configurations, which
landed me on r364166. After many intense head-scratching minutes and printf
debugging, I realized that the newline was the difference. This is where our
tale ends.
Reported by: Harry Schmalzbauer, O. Hartmann, Mike Tancsa, kevans
MFC after: 3 days
This fixes a compile issue under gcc6 which complains about
legacy style C function declarations.
Differential Revision: https://reviews.freebsd.org/D26504
This was revealed by the rescue build with a patch I'm working on to default
WARNS=6 everywhere. The issues resolved were:
- Missing prototype for _crunched_${ident}_stub in the *_stub.c generated
bits
- Missing prototype for crunched_main
- Incomplete prototype for _crunched_${ident}_stub in the generated parts of
crunched_main
- Literal strings in the stub table must drop const qualifier, unless we
const'ify name
- f field in struct stub didn't have a proper prototype
Most of these issues are minor formalities and easily addressed.
I note that if my patch to eventually raise WARNS for the rescue build
lands, we'll need to bump the __FreeBSD_version requirement for
bootstrapping crunchgen and wipe out the rescue .OBJDIR if it's stale, which
we should be able to detect pretty easily from a couple of the issues that
have been fixed here.
Reviewed by: arichardson
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D26363
This should fix cases like su setting argv[0] to _su for /bin/sh.
Previously cheribsdbox (a crunched tool we use in CheriBSD to reduce the
size of our minimal disk images to allow loading them onto FPGAs without
waiting forever for the transfer) would complain about _su not being
compiled in, but now that we also look at AT_EXECPATH it correctly
invokes the sh tool.
Note: we use use AT_EXECPATH instead of the KERN_PROC_PATHNAME sysctl to get
the crunchgen binary name since it seems like KERN_PROC_PATHNAME just
returns the last cached path for a given hardlink.
When using `su`, instead of invoking /bin/csh this would invoke the last
used hardlink to cheribsdbox. This caused weird test failures when running
tests due to `id` being executed instead of `echo`:
$ id # id is a hardlink to /bin/cheribsdbox
$ su postgres -c 'echo 1' # su is also a hardlink
uid=1001(postgres) gid=1001(postgres) groups=1001(postgres)
Obtained from: CheriBSD
Reviewed By: emaste, brooks
Differential Revision: https://reviews.freebsd.org/D25998
The -P flag is required by POSIX so we don't have to care whether pwd is
a shell builtin or not. This also allows removing pwd from the list of
bootstrap tools since all shells we care about for building have a
builtin pwd command. This effectively reverts r364190.
Suggested By: rgrimes, jrtc27
In r364166 I changed /bin/pwd to pwd, but pwd can be shell builtin that
may not correctly return a real path. To ensure that all symlinks are
resolved use `env pwd -P` instead (the -P flag is part of POSIX so
should be supported everywhere).
Reported By: rgrimes
Suggested By: jrtc27
On Glibc systems mkstemp can only be used once with the same template
string since it will be modified in-place and no longer contain any 'X' chars.
It is fine to reuse the same file here but we need to be explicit and use
open() instead of mkstemp() on the second use.
While touching this file also avoid a hardcoded /bin/pwd since that may not
work when building on non-FreeBSD systems.
Reviewed By: brooks
Differential Revision: https://reviews.freebsd.org/D25990
rcmds removed in r324351.
Historical references in the README are maintained. There's a paragraph
describing a "980K crunched 'fixit'" that references rsh and rlogin.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
No functional change intended.
Failure modes of the modern (that is, produced in the last 25 years)
hard drives and SSDs made the utility outdated. Since the kernel
interface to support it was removed in r324853, cut the userspace
remnants as well.
Discussed with: bde (who does not like the removal)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
in favor of just rendering the manpage instead of relying on pre-formatted
catpages. Note, this does not impede the ability to use existing catpages,
it just removes the utility to generate them.
Reviewed by: imp, allanjude
Approved by: emaste (mentor)
Differential Revision: https://reviews.freebsd.org/D12317
Some tools produce objects with a combined strtab and shstrtab.
These objects are not supported by crunchide since it rewrites the
symtab and strtab to "hide" symbols. This invalidates section header
offsets into a combined strtab/shstrtab.
In the future we could support these objects (by ensuring that we retain
unmodified section name strings in the output .strtab, and then rewriting
each section header's sh_name).
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
- DIRDEPS_BUILD: Fix crunchgen builds losing their library dependencies
on a nop-rebuild.
- META_MODE: Fix not rebuilding various crunch.mk targets if their .meta
files warrant a rebuild. They were lacking .meta files previously.
This adds .NOMETA to the crunch objects being used since they are
already built. Bmake was forcing a rebuild on them since their
.meta files were not in the expected place; there is no reason to
rebuild them.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Passing MAKEOBJDIRPREFIX to the main prog build (rescue) would confuse
WITH_AUTO_OBJ and cause it to create a recursed object directory that
then broke the actual prog build. This is normally not a problem since
we do not call 'make -f prog.mk obj' before building anything in it.
Crunchgen(1) also assumes that if -o is not passed then if an object
directory does not already exist then it should build in the source
directories. The normal buildworld process will have already ran 'make
obj' in each of the component directories so this is not a problem.
With WITH_AUTO_OBJ though this is not the case. So we must tell
crunchgen(1) that MK_AUTO_OBJ=yes will create the directory and to not
require it be present before generating its Makefile.
Sponsored by: EMC / Isilon Storage Division
These are no longer needed after the recent 'beforebuild: depend' changes
and hooking DIRDEPS_BUILD into a subset of FAST_DEPEND which supports
skipping 'make depend'.
Sponsored by: EMC / Isilon Storage Division
This was causing the following error:
rescue
sh: rescue: not found
*** [rescue] Error code 127
make[1]: stopped in /usr/obj/usr/src/rescue/rescue
Sponsored by: EMC / Isilon Storage Division
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.
Still need to add real targets under targets/ to build packages.
Differential Revision: D2796
Reviewed by: brooks imp
