cross-build: Avoid adding <sys/cdefs.h> to SYSINCS

Summary:
The sys/cdefs.h in src is incompatible with glibc's sys/cdefs.h so
cross-building broke when the former was added to SYSINCS in 1c9ff80f06

cross-build: Avoid adding <sys/cdefs.h> to SYSINCS

Summary:
The sys/cdefs.h in src is incompatible with glibc's sys/cdefs.h so
cross-building broke when the former was added to SYSINCS in 1c9ff80f06. This
commit adds a guard around that to only do that when building on FreeBSD. This
should fix github CI.

Test Plan:
Ran buildkernel using tools/build/make.py on linux in github CI and
locally on FreeBSD to double check nothing broke.

Reviewed by: dim
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D54385

show more ...

tools.build: also add sys/_visible.h to SYSINCS

This is needed since sys/cdefs.h includes sys/_visible.h.

Reported by: kib
Fixes: 1c9ff80f0635
MFC after: 3 days

tools.build: add sys/cdefs.h to SYSINCS, since lots of other headers use it

This is needed to let the legacy stage compile against newer versions of
sys/font.h, which transitively includes sys/cdefs

tools.build: add sys/cdefs.h to SYSINCS, since lots of other headers use it

This is needed to let the legacy stage compile against newer versions of
sys/font.h, which transitively includes sys/cdefs.h, and requires the
new __nonstring macro from it.

Fixes: e2c93ed09f25
MFC after: 3 days

show more ...

Use install instead of cp to copy bootstrap tools

We need to preserve modification times on bootstrap tools, but `cp -p`
also tries to preserve flags, which fails if OBJROOT is on NFS. A -N
option

Use install instead of cp to copy bootstrap tools

We need to preserve modification times on bootstrap tools, but `cp -p`
also tries to preserve flags, which fails if OBJROOT is on NFS. A -N
option was added to cp for this purpose, but trying to use that would
break cross-building on hosts that don't have that option. The best
remaining option is `install -p`, which we already assume is present.

PR: 275030
Reviewed by: imp, emaste
Differential Revision: https://reviews.freebsd.org/D53751

show more ...

tools/build: Bring in stdckdint.h if needed

This is needed when bootstrapping libc, reallocarray.c and
recallocarray.c include stdckdint.h now.

Reviewed by: emaste
Fixes: 7233893e9496 ("lib{c,openb

tools/build: Bring in stdckdint.h if needed

This is needed when bootstrapping libc, reallocarray.c and
recallocarray.c include stdckdint.h now.

Reviewed by: emaste
Fixes: 7233893e9496 ("lib{c,openbsd}: use ckd_mul() for overflow checking in re(c)allocarray")
Differential Revision: https://reviews.freebsd.org/D52932

show more ...

flua: support our flua modules in the bootstrap flua

This version builds every module into the flua binary itself, since all
of the bootstrap tools are built -DNO_SHARED. As a result, we also
canno

flua: support our flua modules in the bootstrap flua

This version builds every module into the flua binary itself, since all
of the bootstrap tools are built -DNO_SHARED. As a result, we also
cannot dlsym(), so we can't really discover the names of our newly
builtin modules. Instead, just build out a linker set with all of our
luaopen_*() functions to register everything up-front.

Building in all of the modules isn't strictly necessary, but it means
that we have an example of how to add a bootstrap module everywhere you
go and one doesn't need to consider whether bootstrap flua can use a
module when writing scripts. On my build machine, the consequence on
our binary size is an increase from around 1.6M -> 1.9M, which isn't
really that bad.

.lua modules can install into their usual path below $WORLDTMP/legacy
and we'll pick them up automagically by way of the ctor that sets up
LUA_PATH early on.

This re-lands bootstrap module support with a more sensible subset, and
after having verified that it cross-builds fine on macOS and Linux -- we
cannot do libfreebsd on !FreeBSD because it's more system header
dependant. We also need to bootstrap libmd to bring in libhash, and
libucl + libyaml.

Reviewed by: bapt, emaste (both previous version)
Differential Revision: https://reviews.freebsd.org/D51890

show more ...

Revert "flua: support our flua modules in the bootstrap flua"

This reverts commit 1953a12ee2cde1afacb3e3f7612d89695c96e04f, because it
cannot work at all on macOS without more work, at a minimum. W

Revert "flua: support our flua modules in the bootstrap flua"

This reverts commit 1953a12ee2cde1afacb3e3f7612d89695c96e04f, because it
cannot work at all on macOS without more work, at a minimum. We use
linker sets for module discovery, but we don't have a version of this
that works for mach-o at the moment.

show more ...

flua: support our flua modules in the bootstrap flua

This version builds every module into the flua binary itself, since all
of the bootstrap tools are built -DNO_SHARED. As a result, we also
canno

flua: support our flua modules in the bootstrap flua

This version builds every module into the flua binary itself, since all
of the bootstrap tools are built -DNO_SHARED. As a result, we also
cannot dlsym(), so we can't really discover the names of our newly
builtin modules. Instead, just build out a linker set with all of our
luaopen_*() functions to register everything up-front.

Building in all of the modules isn't strictly necessary, but it means
that we have an example of how to add a bootstrap module everywhere you
go and one doesn't need to consider whether bootstrap flua can use a
module when writing scripts. On my build machine, the consequence on
our binary size is an increase from around 1.6M -> 1.9M, which isn't
really that bad.

.lua modules can install into their usual path below $WORLDTMP/legacy
and we'll pick them up automagically by way of the ctor that sets up
LUA_PATH early on.

Reviewed by: bapt, emaste
Differential Revision: https://reviews.freebsd.org/D51890

show more ...

build: remove certctl requirement for host OpenSSL libs on macOS

Some platforms, like macOS, do not expose headers for the system's
libcrypto for public consumption. libcrypto is relatively heavy a

build: remove certctl requirement for host OpenSSL libs on macOS

Some platforms, like macOS, do not expose headers for the system's
libcrypto for public consumption. libcrypto is relatively heavy and
needs to know, e.g., the host system's endianness, so we scope the build
down to macOS where OpenSSL headers are known to not be present and we
can be reasonably certain that most of the systems today that would be
cross-building are little endian.

We still don't bother if building WITHOUT_OPENSSL since the end result
is expected to be used by OpenSSL, but perhaps we could revisit that
independently in case one, e.g., brings their own implementation.

Reported by: jrtc27
Reviewed by: jrtc27, ngie
Fixes: c340ef28fd38 ("certctl: Reimplement in C")
Differential Revision: https://reviews.freebsd.org/D51935

show more ...

tools/build: Provide extra headers on non-FreeBSD

These will be needed by future changes to continue to allow building
makefs as a bootstrap tool on Linux and macOS. This also requires
defining __sb

tools/build: Provide extra headers on non-FreeBSD

These will be needed by future changes to continue to allow building
makefs as a bootstrap tool on Linux and macOS. This also requires
defining __sbintime_t in our cross-build sys/_types.

show more ...

Makefile: Hook MIT KRB5 into the build

Add tests for MK_MITKRB5. If "yes" build MIT KRB5. If "no" build Heimdal.
The default is MK_MITKRB5 = no, added by "krb5: Add build plumbing".

At some point w

Makefile: Hook MIT KRB5 into the build

Add tests for MK_MITKRB5. If "yes" build MIT KRB5. If "no" build Heimdal.
The default is MK_MITKRB5 = no, added by "krb5: Add build plumbing".

At some point we will change the default to MK_MITKRB5 = yes. A ports
exp-run will need to be successfully run first.

Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D50815

show more ...

cross-build: Provide real fflagstostr/strtofflags on Linux

These are used by mtree and makefs, so in order to be able to set schg
on /var/empty in METALOG and set schg on various files and directori

cross-build: Provide real fflagstostr/strtofflags on Linux

These are used by mtree and makefs, so in order to be able to set schg
on /var/empty in METALOG and set schg on various files and directories
in the resulting disk images we need to have a real implementation
rather than always giving no flags.

Ideally mtree wouldn't rely on round-tripping the textual flags field
via the "native" flags encoding using these functions, and ideally
makefs wouldn't rely on the "native" flags encoding matching FreeBSD's,
but in practice macOS's schg is the same and we can pretend Linux has
the same.

This fixes Linux-produced disk images lacking schg on any files or
directories, and Linux-produced distribution tarballs lacking schg on
/var/empty (note though that they do set schg on files, as install
already preserves file flags on Linux).

Reviewed by: emaste, markj
Differential Revision: https://reviews.freebsd.org/D50080

show more ...

nv: Add <sys/_nv.h> header for nvlist_t declaration

This can be useful for headers that wish to use nvlist_t pointers in a
structure or function argument without pulling in all of the headers
from <

nv: Add <sys/_nv.h> header for nvlist_t declaration

This can be useful for headers that wish to use nvlist_t pointers in a
structure or function argument without pulling in all of the headers
from <sys/nv.h>.

Reviewed by: imp
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D48229

show more ...

cross-build: fix missing <sys/md4.h> the proper way

Add it to tools/build/Makefile with a short note of where it's needed,
rather than hamfistedly copying it into the tools/build hierarchy.

Reporte

cross-build: fix missing <sys/md4.h> the proper way

Add it to tools/build/Makefile with a short note of where it's needed,
rather than hamfistedly copying it into the tools/build hierarchy.

Reported by: jrtc27
Reviewed by: jrtc27
Fixes: aad507854efd13c43 ("Fix the cross-build after recent commits")
Differential Revision: https://reviews.freebsd.org/D46854

show more ...

Remove residual blank line at start of Makefile

This is a residual of the $FreeBSD$ removal.

MFC After: 3 days (though I'll just run the command on the branches)
Sponsored by: Netflix

build: only inspect the first word of toolchain tools

CC/CXX/CPP/LD may all have arguments supplied in various circumstances,
which break the logic here. We only need to determine which of these
to

build: only inspect the first word of toolchain tools

CC/CXX/CPP/LD may all have arguments supplied in various circumstances,
which break the logic here. We only need to determine which of these
tools we're expecting to invoke from PATH, which just requires
examination of the first word. Limit our scope to exactly that.

Patch suggested by: jrtc27
Reviewed by: imp, jrtc27
Differential Revision: https://reviews.freebsd.org/D43372

show more ...

Make kldxref a bootstrap tool and use unconditionally

Now that kldxref is a generic cross tool and can be built on non-FreeBSD
we can bootstrap it during the build and thus remove the condition for

Make kldxref a bootstrap tool and use unconditionally

Now that kldxref is a generic cross tool and can be built on non-FreeBSD
we can bootstrap it during the build and thus remove the condition for
whether it exists. We also need to make sure to add it to the METALOG
for -DNO_ROOT builds.

Reviewed by: brooks, imp
Differential Revision: https://reviews.freebsd.org/D43051

show more ...

tools/build: Provide sys/linker_set.h when cross-building

This is needed for kldxref, which will shortly become a bootstrap tool.
Linux can use the same one as FreeBSD (provided the cross-building
s

tools/build: Provide sys/linker_set.h when cross-building

This is needed for kldxref, which will shortly become a bootstrap tool.
Linux can use the same one as FreeBSD (provided the cross-building
sys/cdefs.h is augmented appropriately), whilst macOS needs its own
Mach-O-specific implementation.

Reviewed by: brooks
Differential Revision: https://reviews.freebsd.org/D43049

show more ...

tools/build: Support building with glibc 2.38

Ubuntu 23.10 uses glibc 2.38. This adds strlcpy and strlcmp so we need
to remove them from the cross build environment.

Reviewed by: jrtc27 (earlier ve

tools/build: Support building with glibc 2.38

Ubuntu 23.10 uses glibc 2.38. This adds strlcpy and strlcmp so we need
to remove them from the cross build environment.

Reviewed by: jrtc27 (earlier version), arichardson
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D42303

show more ...

Add support for host32 for DIRDEPS_BUILD

Allow building 32bit libs for host.

Move CFLAGS additions from local.sys.dirdeps.mk (which is too early
and impacts CFLAGS defaults) to local.sys.mk

Review

Add support for host32 for DIRDEPS_BUILD

Allow building 32bit libs for host.

Move CFLAGS additions from local.sys.dirdeps.mk (which is too early
and impacts CFLAGS defaults) to local.sys.mk

Reviewed by: stevek
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D41946

show more ...

kbdcontrol: Support building as a bootstrap tool on old and non-FreeBSD

Systems that predate 971bac5ace7a ("kbd: consolidate kb interfaces
(phase one)") cannot build kbdcontrol since kbdelays and kb

kbdcontrol: Support building as a bootstrap tool on old and non-FreeBSD

Systems that predate 971bac5ace7a ("kbd: consolidate kb interfaces
(phase one)") cannot build kbdcontrol since kbdelays and kbrates moved
to sys/kbio.h. Moreover, on non-FreeBSD, it requires all kinds of ioctls
and sysctls that are highly FreeBSD-specific to build, but we use it as
a bootstrap tool to generate the keymaps used by some kernels (LINT ones
in particular). Thus, when bootstrapping kbdcontrol, disable everything
that's not needed for that singular use, and use the in-tree kbio.h to
get the definitions of the necessary structures.

This allows KBDMUX_DFLT_KEYMAP, UKBD_DFLT_KEYMAP and ATKBD_DFLT_KEYMAP
to be enabled when building on non-FreeBSD, and thus LINT kernels.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D41541

show more ...

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

tools/build: Work around broken Clang FreeBSD resource dir logic pre-13

Prior to Clang 13 (e.g. in the Clang 11 present in 13.0-RELEASE), the
resource directory logic for FreeBSD was broken and woul

tools/build: Work around broken Clang FreeBSD resource dir logic pre-13

Prior to Clang 13 (e.g. in the Clang 11 present in 13.0-RELEASE), the
resource directory logic for FreeBSD was broken and would not resolve
symlinks, meaning symlinks would only work if in a directory next to the
containing lib directory. Therefore we cannot even use a symlink for
worldtmp, we have to make a wrapper script that execs the real binary
via an absolute path.

Reported by: markj
Reviewed by: markj
Fixes: 65f28f63a73d ("tools/build: Create toolchain symlinks for non-absolute compiler/linker")
Differential Revision: https://reviews.freebsd.org/D41238

show more ...

tools/build: Create toolchain symlinks for non-absolute compiler/linker

If any of the toolchain variables are not absolute then we need to
create a symlink in WORLDTMP/legacy/bin in order to make th

tools/build: Create toolchain symlinks for non-absolute compiler/linker

If any of the toolchain variables are not absolute then we need to
create a symlink in WORLDTMP/legacy/bin in order to make them available
during a BUILD_WITH_STRICT_TMPPATH build.

Reviewed by: brooks, jhb
Differential Revision: https://reviews.freebsd.org/D41188

show more ...

tools/build: Tidy up whitespace and comments, and delete duplicate code

We already handle the make and bmake links unconditionally above.