release: Sort etcupdate entries for base metalog

Supports reproducible builds.

PR: 283214
Reviewed-by: emaste
Signed-off-by: Pat Maddox <pat@patmaddox.com>

release: Always use NO_ROOT for distribute* and package*

We used to have two different code paths for generating release media
dist sets, one which expected to run as root, and one which did not.
It

release: Always use NO_ROOT for distribute* and package*

We used to have two different code paths for generating release media
dist sets, one which expected to run as root, and one which did not.
It was possible that the two different paths could generate different
results.

We can use the path that does not *require* root also when running as
root, so use it always. We'll want to do eventually do the same for all
release targets, but these ones are an easy place to start.

Reviewed by: brooks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D48041

show more ...

Clean pkg cache in release media

Recently wifi-firmware-kmod-release pkgs were added in the release
installation media, but unfortunately the pkgs were not cleaned up after
the installation. These k

Clean pkg cache in release media

Recently wifi-firmware-kmod-release pkgs were added in the release
installation media, but unfortunately the pkgs were not cleaned up after
the installation. These kept the stale pkgs in the /var/cache. The pkgs
should be removed with pkg clean.

PR: 283112
Reported by: yanhui.he@broadcom.com
Reviewed by: cperciva, emaste
Approved by: cperciva, emaste, re (implicit)
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D47942

show more ...

release: Don't break if firmware install fails

On some platforms (e.g. powerpc) we don't have packages, so we can't
install them onto the ISOs. Proceed with building the images anyway.

Reported by

release: Don't break if firmware install fails

On some platforms (e.g. powerpc) we don't have packages, so we can't
install them onto the ISOs. Proceed with building the images anyway.

Reported by: Weekly snapshot builds
Fixes: 7e2996c1f5b4 ("release: install wireless firmware onto disc1 and dvd")
MFC after: 1 minute

show more ...

release: install wireless firmware onto disc1 and dvd

Wireless driver firmware is no longer added to the src tree.
In order to have wireless support in the installer for the new drivers
we install t

release: install wireless firmware onto disc1 and dvd

Wireless driver firmware is no longer added to the src tree.
In order to have wireless support in the installer for the new drivers
we install the firmware packages onto disc1 (and memstick) and dvd
if built on FreeBSD and NOPKG is not defined (to not break cross-builds
from Linux or OSX and to allow people to opt-out).

Sponsored by: The FreeBSD Foundation
Submitted by: cperciva (the orig. commands and where to place them)
Reviewed by: jrtc27
MFC after: 6 hours
Differential Revision: https://reviews.freebsd.org/D47407

show more ...

release: Don't copy container images into /ftp/

These are installed into /ociimages/ and stage onto the download site
at /OCI-IMAGES/; they don't belong in the same directory as the
"distribution" .

release: Don't copy container images into /ftp/

These are installed into /ociimages/ and stage onto the download site
at /OCI-IMAGES/; they don't belong in the same directory as the
"distribution" .txz files.

Fixes: d03c82c28da8 ("release: add optional OCI images")
MFC after: 1 minute
Sponsored by: Amazon

show more ...

release: Exclude container images from globs

We use a *.txz glob to get all of the "distributions" which comprise
the FreeBSD release, but we now (optionally) also build container
images which are .

release: Exclude container images from globs

We use a *.txz glob to get all of the "distributions" which comprise
the FreeBSD release, but we now (optionally) also build container
images which are .txz files. Grep those out from the distribution
lists.

A better long-term fix would probably be to generate an explicit list
of the .txz files we want rather than using an overbroad glob and
filtering out the files we *don't* want.

Fixes: d03c82c28da8 ("release: add optional OCI images")
MFC after: 1 minute
Sponsored by: Amazon

show more ...

Retire MK_PROFILE infrastructure

It was disabled by default in fe52b7f60ef4. We planned to (but did not)
remove the option before FreeBSD 14. Remove it now, for FreeBSD 15.

Relnotes: Yes
Reviewed

Retire MK_PROFILE infrastructure

It was disabled by default in fe52b7f60ef4. We planned to (but did not)
remove the option before FreeBSD 14. Remove it now, for FreeBSD 15.

Relnotes: Yes
Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31558

show more ...

release: add optional OCI images

This adds three OCI archive format files to the release containing
FreeBSD base images suitable for static linked, dynamic linked and shell
workloads. The shell imag

release: add optional OCI images

This adds three OCI archive format files to the release containing
FreeBSD base images suitable for static linked, dynamic linked and shell
workloads. The shell image also contains pkg-bootstrap and can be easily
extended by installing packages (including pkgbase packages).

Reviewed by: dch, cpersiva, jlduran, zlei
Differential Revision: https://reviews.freebsd.org/D46759
MFC after: 2 days

show more ...

release: Remove useless ls

Replace "$((ls *.txz))" with "*.txz"

Reported by: jrtc27

release: Include -dbg files in dvd image

In 2016, commit 8834318685bc reworked the code for excluding -dbg
files from install media, and in the process accidentally broke it
for dvd images ('grep -v

release: Include -dbg files in dvd image

In 2016, commit 8834318685bc reworked the code for excluding -dbg
files from install media, and in the process accidentally broke it
for dvd images ('grep -v' should have been 'grep -vE').

FreeBSD Update builds later began to depend on this, and in any case
since DVD images are intended as "include everything" images there's
no point excluding those; so remove the (broken) filtering from DVD
image builds.

Sponsored by: Amazon
MFC after: 2 days

show more ...

release: Redirect etcupdate logfile to /dev/null.

Stop shipping a log file for etcupdate. This is a source of
non-reproducability as it uses mktemp thereby guaranteeing the output is
different each

release: Redirect etcupdate logfile to /dev/null.

Stop shipping a log file for etcupdate. This is a source of
non-reproducability as it uses mktemp thereby guaranteeing the output is
different each run.

MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D46317

show more ...

release: Publish non-uploaded cloudware bits

While most "cloudware" (AWS, Azure, GCP, Vagrant) images get uploaded
to their respective clouds, we have two images -- BASIC-CI and
BASIC-CLOUDINIT -- w

release: Publish non-uploaded cloudware bits

While most "cloudware" (AWS, Azure, GCP, Vagrant) images get uploaded
to their respective clouds, we have two images -- BASIC-CI and
BASIC-CLOUDINIT -- which are published via the FTP mirrors. We have
been handling these using a manual and error-prone process of copying
them from the internal build directories into the FTP staging tree,
but they should be handled using the "make install ftp-stage"
mechanism which is used for all the other published images.

This commit 1. Adds a new 'make cloud-install' target which is run as
part of 'make install' and installs BASIC-CI and BASIC-CLOUDINIT images,
and 2. Extends 'make ftp-stage' in Makefile.mirrors to copy those bits
into the FTP staging area.

As a side benefit, this will also make BASIC-CI and BASIC-CLOUDINIT
images available for arm64; until now they have only been available
for amd64.

MFC after: 1 week

show more ...

release: Build dvd image before staging pkgs

This missing dependency is only visible with -jN (which was otherwise
broken until recently) and only when building DVD images (which aren't
included in

release: Build dvd image before staging pkgs

This missing dependency is only visible with -jN (which was otherwise
broken until recently) and only when building DVD images (which aren't
included in weekly snapshots and thereby slipped past testing).

MFC after: 3 days

show more ...

release: Don't reuse disc1/bootonly directories

The disc1 and bootonly directories have files distributed into them
for use in "full" and "mini" images; the former are disc1.iso and
memstick.img, an

release: Don't reuse disc1/bootonly directories

The disc1 and bootonly directories have files distributed into them
for use in "full" and "mini" images; the former are disc1.iso and
memstick.img, and the latter is bootonly.iso and mini-memstick.img.

Unfortunately the scripts which package a directory tree into an ISO
or memory stick image also modify the directory, for example to
create an appropriate /etc/fstab file; so creating two images at the
same time breaks.

Resolve this by copying disc1 to disc1-disc1 and disc1-memstick,
and copying bootonly to bootonly-bootonly and bootonly-memstick,
before using those directories for constructing the ISO+memstick
images.

MFC after: 1 week

show more ...

release: distributekernel before packagekernel

With these as a single make command, `make -j` breaks when it tries to
package up a kernel which hasn't been distributed yet.

MFC after: 1 week

release: make -j compat: cd inside subshell

Place instances of "cd foo && bar" inside subshells for compatibility
with modern make(8) which uses a single shell for the duration of a
makefile target.

release: make -j compat: cd inside subshell

Place instances of "cd foo && bar" inside subshells for compatibility
with modern make(8) which uses a single shell for the duration of a
makefile target.

MFC after: 1 week

show more ...

release: rework distributions list

Components like base.txz and ports.txz are called distributions in the
installer, and with the introduction of pkgbase we will start dealing
with normal pkg packag

release: rework distributions list

Components like base.txz and ports.txz are called distributions in the
installer, and with the introduction of pkgbase we will start dealing
with normal pkg packages in the installer. Rename EXTRA_PACKAGES to
DISTRIBUTIONS, and move base.txz and kernel.txz to that list.

This introduces no functional change but is a small cleanup in advance
of some pkgbase experimentation.

Reviewed by: cperciva
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43544

show more ...

release: stop generating mergemaster databases

mergemaster was deprecated some time ago and will be removed from
FreeBSD 15.

Reviewed by: imp
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Diff

release: stop generating mergemaster databases

mergemaster was deprecated some time ago and will be removed from
FreeBSD 15.

Reviewed by: imp
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41797

show more ...

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

release: Fix make :@ modifier syntax

This spits out errors but seemingly isn't actually fatal, so was missed.

Fixes: 6853d893c7fe ("release: Automatically generate MK_LIB${LIBCOMPAT} and lib${libco

release: Fix make :@ modifier syntax

This spits out errors but seemingly isn't actually fatal, so was missed.

Fixes: 6853d893c7fe ("release: Automatically generate MK_LIB${LIBCOMPAT} and lib${libcompat}-dbg lists")

show more ...

release: Automatically generate MK_LIB${LIBCOMPAT} and lib${libcompat}-dbg lists

Reviewed by: brooks, jhb
Differential Revision: https://reviews.freebsd.org/D41184

release: Set loader_menu_multi_user_prompt to "Installer" for installer

This makes it more obvious that the media being booted is an installer
rather than an installed system, which is otherwise har

release: Set loader_menu_multi_user_prompt to "Installer" for installer

This makes it more obvious that the media being booted is an installer
rather than an installed system, which is otherwise hard to distinguish.
It also provides a more user-friendly, and more accurate, prompt.

Reviewed by: gjb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D36419

show more ...

release: Include an etcupdate tree in -DNO_ROOT release image builds

This makes use of the new -N etcupdate flag and merges the resulting
METALOG into base.meta re-rooted to /var/db/etcupdate/curren

release: Include an etcupdate tree in -DNO_ROOT release image builds

This makes use of the new -N etcupdate flag and merges the resulting
METALOG into base.meta re-rooted to /var/db/etcupdate/current.

Reviewed by: gjb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35858

show more ...

release: Forward ${MAKE} to etcupdate via the new -m flag

This is in preparation for non-FreeBSD builds where make is GNU make and
so etcupdate needs to know the name of or path to the bmake binary

release: Forward ${MAKE} to etcupdate via the new -m flag

This is in preparation for non-FreeBSD builds where make is GNU make and
so etcupdate needs to know the name of or path to the bmake binary to
use for its own builds.

Reviewed by: gjb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35855

show more ...