1.\" Copyright (c) 2002 Murray Stokely <murray@FreeBSD.org> 2.\" All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. 9.\" 2. Redistributions in binary form must reproduce the above copyright 10.\" notice, this list of conditions and the following disclaimer in the 11.\" documentation and/or other materials provided with the distribution. 12.\" 13.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND 14.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 15.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 16.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE 17.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 19.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 20.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 21.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 22.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 23.\" SUCH DAMAGE. 24.\" 25.\" $FreeBSD$ 26.\" 27.Dd February 23, 2021 28.Dt RELEASE 7 29.Os 30.Sh NAME 31.Nm release 32.Nd "release building infrastructure" 33.Sh DESCRIPTION 34.Fx 35provides a complete build environment suitable for users to make 36full releases of the 37.Fx 38operating system. 39All of the tools necessary to build a release are available from the 40.Fx 41source code repository in 42.Pa src/release . 43A complete release can be built with only a single command, 44including the creation of ISO images suitable for burning to CD-ROM, 45memory stick images, and a network install directory. 46This command is aptly named 47.Dq Li "make release" . 48.Pp 49For some users, it may be desirable to provide an absolutely clean 50build environment, with no local modifications to the source tree or to 51.Xr make.conf 5 , 52and with clean checkouts of specific versions of the doc, src, and ports 53trees. 54For this purpose, a script 55.Pq Pa src/release/release.sh 56is provided to automate these checkouts and then execute 57.Dq Li "make release" 58in a clean 59.Xr chroot 8 . 60.Pp 61Before attempting to build a release, the user is expected to be 62familiar with the contents of 63.Xr build 7 , 64and should have experience upgrading systems from source. 65.Pp 66The release build process requires that 67.Pa /usr/obj 68be populated with the output of 69.Dq Li "make buildworld" 70and 71.Dq Li "make buildkernel" . 72This is necessary to provide the object files for the release or, when 73using 74.Pa release.sh , 75so that the object files for a complete system can be installed into a clean 76.Xr chroot 8 77environment. 78.Pp 79If the target release build is for a different architecture or machine type, 80the 81.Va TARGET 82and 83.Va TARGET_ARCH 84variables must be used. 85See the supported 86.Fa release.conf 87variables for more information. 88.Pp 89The release procedure on some architectures may also require that the 90.Xr md 4 91(memory disk) device driver be present in the kernel 92.Pq either by being compiled in or available as a module . 93.Pp 94This document does not cover source code management, quality 95assurance, or other aspects of the release engineering process. 96.Sh CLEAN RELEASE GENERATION 97Official releases of 98.Fx 99are produced in a clean environment to 100ensure consistency between the versions of the src, ports, and doc trees 101and to avoid contamination from the host system 102.Po such as local patches, changes 103to 104.Xr make.conf 5 , 105etc. 106.Pc . 107This is accomplished using the wrapper script 108.Pa src/release/release.sh . 109.Pp 110.Ic release.sh 111.Op Fl c Ar release.conf 112.Pp 113.Ic release.sh 114checks out the 115.Li src/ , 116.Li ports/ , 117and 118.Li doc/ 119trees to 120.Va CHROOTDIR , 121then calls 122.Dq Li "make buildworld" 123and 124.Dq Li "make installworld" 125to generate a 126.Xr chroot 8 127environment. 128Next, 129.Dq Li "make release" 130is run within the 131.Xr chroot 8 132environment and places the result in 133.Pa $CHROOTDIR/R . 134.Pp 135The optional 136.Fa release.conf 137configuration file supports the following variables: 138.Bl -tag -width Ev 139.It Va CHROOTDIR 140The directory within which the release will be built. 141.It Va CHROOT_MAKEENV 142Additional 143.Xr make 1 144arguments to pass through, which directly affect the 145tuning of the build chroot. 146.It Va GITROOT 147The 148.Xr git 1 149host used to check out the various trees. 150Defaults to 151.Pa https://git.FreeeBSD.org . 152.It Va SRCBRANCH 153The 154.Li src/ 155branch to use. 156Defaults to 157.Fl b Va main . 158.It Va PORTBRANCH 159The 160.Li ports/ 161branch to use. 162Defaults to 163.Va head/@rHEAD . 164.It Va TARGET 165The target machine type for cross-building a release. 166.It Va TARGET_ARCH 167The target machine architecture for cross-building a release. 168.Pp 169For the supported list of 170.Va TARGET 171and 172.Va TARGET_ARCH 173combinations, consult the output of 174.Dq make targets 175as documented in 176.Xr build 7 . 177.It Va KERNEL 178The target kernel configuration to use. 179Defaults to 180.Va GENERIC . 181Multiple 182.Va KERNEL 183entries may be specified. 184.It Va MAKE_CONF 185The 186.Xr make.conf 5 187to use for the release build. 188Defaults to 189.Fa /dev/null 190to prevent polluting the release with local system changes. 191.It Va SRC_CONF 192The 193.Xr src.conf 5 194to use for the release build. 195Defaults to 196.Fa /dev/null 197to prevent polluting the release with local system changes. 198.It Va MAKE_FLAGS 199Additional flags to pass to 200.Xr make 1 . 201.It Va WORLD_FLAGS 202Additional flags to pass to 203.Xr make 1 204during the 205.Dq buildworld 206phase. 207Defaults to setting the number of 208.Xr make 1 209jobs 210.Pq Ar -j 211to the number of CPUs available on a SMP-capable system. 212.It Va KERNEL_FLAGS 213Additional flags to pass to 214.Xr make 1 215during the 216.Dq buildkernel 217phase. 218Defaults to setting the number of 219.Xr make 1 220jobs 221.Pq Ar -j 222to half the number of CPUs available on a SMP-capable system. 223.It Va NOPORTS 224Set to a non-empty value to skip the 225.Li ports/ 226tree checkout. 227When set, 228.Va NOPORTS 229will prevent the 230.Fa ports.txz 231distribution package from being created. 232.It Va WITH_DVD 233Set to a non-empty value to include the 234.Cm dvdrom 235target. 236.It Va WITH_COMPRESSED_IMAGES 237Set to a non-empty value to compress the release images with 238.Xr xz 1 . 239The original 240.Pq uncompressed 241images are not removed. 242.It Va XZ_THREADS Pq Vt int 243Set to the number of threads 244.Xr xz 1 245should use when compressing images. 246By default, 247.Va XZ_THREADS 248is set to 249.Va 0 , 250which uses all available cores on the system. 251.It Va VCSCMD 252The command run to obtain the source trees. 253Defaults to 254.Qq Cm git clone Fl q . 255.It Va CHROOTBUILD_SKIP 256If defined, the 257.Li buildworld , 258.Li installworld , 259and 260.Li distribution 261stages of the 262.Xr chroot 8 263build environment setup are skipped. 264This is intended solely for cases where the 265.Xr chroot 8 266userland are provided by alternate means. 267.It Va SRC_UPDATE_SKIP 268Set to a non-empty value to prevent checkout or update of 269.Fa /usr/src 270within the 271.Xr chroot 8 . 272This is intended for use only when 273.Fa /usr/src 274is expected to exist by alternative means. 275.It Va PORTS_UPDATE_SKIP 276Set to a non-empty value to prevent checkout or update of 277.Fa /usr/ports 278within the 279.Xr chroot 8 . 280This is intended for use only when 281.Fa /usr/ports 282is expected to exist by alternative means. 283.El 284.Sh EMBEDDED BUILDS 285The following 286.Fa release.conf 287variables are relevant only to release builds for embedded systems: 288.Bl -tag -width Ev 289.It Va EMBEDDEDBUILD 290Set to a non-null value to enable functionality for embedded device 291release builds. 292.Pp 293When set, 294.Va WITH_DVD 295is unset. 296Additionally, 297.Va EMBEDDED_TARGET 298and 299.Va EMBEDDED_TARGET_ARCH 300must also be defined. 301When the build environment is created, 302.Fa release.sh 303runs a separate build script located in an architecture-specific 304directory in 305.Pa src/release/${EMBEDDED_TARGET}/ . 306.It Va EMBEDDEDPORTS 307Set to the list of any ports that are required for the target device 308in the format of 309.Fa category/port . 310The 311.Fa devel/subversion 312port is built by default. 313.It Va EMBEDDED_TARGET 314When set, its value is passed to 315.Xr make 1 316to set the 317.Va TARGET 318.Pq value of Cm uname Fl m 319to cross build the target userland. 320.It Va EMBEDDED_TARGET_ARCH 321When set, its value is passed to 322.Xr make 1 323to set the 324.Va TARGET_ARCH 325.Pq value of Cm uname Fl p 326to cross build the target userland. 327.El 328.Sh VIRTUAL MACHINE DISK IMAGES 329The following 330.Fa release.conf 331variables are relevant only to virtual machine disk image builds: 332.Bl -tag -width Ev 333.It Va WITH_VMIMAGES 334Set to a non-null value to build virtual machine disk images as part 335of the release build. 336.Va WITH_VMIMAGES 337may also be specified as an environment variable passed to 338.Xr make 1 . 339.Pp 340The option requires 341.Xr mkimg 1 342version 20140927 or later. 343.It Va WITH_COMPRESSED_VMIMAGES 344Set to a non-null value to compress the virtual machine disk images with 345.Xr xz 1 346as part of the 347.Cm install 348.Xr make 1 349target. 350Note that compressing virtual machine disk images may take a very long 351time on some systems. 352.It Va VMBASE 353Set to change the name of the resulting virtual machine disk image file. 354The default value is 355.Va vm . 356.It Va VMSIZE 357Set to change the size of the virtual machine disk capacity. 358The default value is 359.Va 20G . 360See 361.Xr truncate 1 362for valid values. 363.Pp 364Virtual machine disk images are, by default, created as sparse images. 365When 366.Va WITH_COMPRESSED_VMIMAGES 367is used, the resulting files compressed with 368.Xr xz 1 369compress to roughly the same size, regardless of the specified disk image 370size. 371.It Va VMFORMATS 372Set to the target virtual disk image format(s) to create. 373By default, the 374.Va vhdf , Va vmdk , Va qcow2 , 375and 376.Va raw 377formats are created. 378See 379.Xr mkimg 1 380for valid format values 381.Pq requires version 20140927 or later . 382.El 383.Pp 384For a list of supported 385.Va VMFORMATS 386values 387.Pq including cloud hosting provider formats 388along with a brief description, run: 389.Bd -literal -offset indent 390cd /usr/src 391make -C release list-vmtargets 392.Ed 393.Sh CLOUD HOSTING MACHINE IMAGES 394The 395.Fx 396release build tools support building virtual machine images for various 397cloud hosting providers, each with their own specific configuration to 398include support for each hosting provider by default. 399.Pp 400The following 401.Xr make 1 402environment variables are supported: 403.Bl -tag -width Ev 404.It Va CLOUDWARE 405Set to a list of one or more cloud hosting providers, enclosed in quotes. 406Requires 407.Va WITH_CLOUDWARE 408to also be set. 409.It Va WITH_CLOUDWARE 410Set to a non-empty value to enable building virtual machine images 411for various cloud hosting providers. 412Requires 413.Va CLOUDWARE 414to also be set. 415.El 416.Pp 417Additionally, the 418.Va CLOUDWARE 419and 420.Va WITH_CLOUDWARE 421variables can be added to 422.Pa release.conf , 423and used in conjunction with 424.Pa release.sh . 425.Pp 426For a list of supported 427.Va CLOUDWARE 428values, run: 429.Bd -literal -offset indent 430cd /usr/src 431make -C release list-cloudware 432.Ed 433.Sh MAKEFILE TARGETS 434The release makefile 435.Pq Pa src/release/Makefile 436is fairly abstruse. 437Most developers will only be concerned with the 438.Cm release 439and 440.Cm install 441targets. 442.\" XXX: Some sort of introduction to this list? All the others have one. 443.Bl -tag -width ".Cm packagesystem" 444.It Cm release 445Meta-target to build all release media and distributions applicable to this 446platform. 447.It Cm install 448Copy all produced release media to 449.Pa ${DESTDIR} . 450.It Cm cdrom 451Builds installation CD-ROM images. 452This may require the 453.Xr md 4 454(memory disk) device driver be present in the kernel 455(either by being compiled in or available as a module). 456This target produces files called 457.Pa disc1.iso 458and 459.Pa bootonly.iso 460as its output. 461.It Cm dvdrom 462Builds installation DVD-ROM images. 463This may require the 464.Xr md 4 465(memory disk) device driver be present in the kernel 466(either by being compiled in or available as a module). 467This target produces the 468.Pa dvd1.iso 469file as its output. 470.It Cm memstick 471Builds an installation memory stick image named 472.Pa memstick.img . 473Not applicable on all platforms. 474Requires that the 475.Xr md 4 476.Pq memory disk 477device driver be present in the kernel 478.Pq either by being compiled in or available as a module . 479.It Cm mini-memstick 480Similar to 481.Cm memstick , 482with the exception that the installation distribution sets 483are not included. 484.It Cm ftp 485Creates a directory named 486.Pa ftp 487containing the distribution files used in network installations 488and suitable for upload to an FTP mirror. 489.It Cm vm-image 490Creates virtual machine disk images in various formats. 491The 492.Cm vm-image 493target requires the 494.Va WITH_VMIMAGES 495.Xr make 1 496environment variable to be set to a non-null value. 497.It Cm vm-cloudware 498Builds 499.Fx 500virtual machine images for various cloud hosting providers. 501See 502.Qq CLOUD HOSTING MACHINE IMAGES 503for implementation details. 504.It Cm list-cloudware 505Displays the list of valid 506.Va CLOUDWARE 507values. 508.It Cm list-vmtargets 509Displays the list of valid 510.Va VMFORMAT 511and 512.Va CLOUDWARE 513values. 514.El 515.Pp 516Major subtargets called by targets above: 517.Bl -tag -width ".Cm packagesystem" 518.It Cm packagesystem 519Generates all the distribution archives 520.Pq base, kernel, ports, doc 521applicable on this platform. 522.It Cm disc1 523Builds a bootable installation system containing all the distribution files 524packaged by the 525.Cm packagesystem 526target, and suitable for imaging by the 527.Cm cdrom , 528.Cm dvdrom 529and 530.Cm memstick 531targets. 532.It Cm reldoc 533Builds the release documentation. 534This includes the release notes, 535hardware guide, and installation instructions. 536Other documentation, such as the Handbook, 537is built during the 538.Cm base.txz 539target invoked by 540.Cm packagesystem . 541.El 542.Sh ENVIRONMENT 543Optional variables: 544.Bl -tag -width ".Ev TARGET_ARCH" 545.It Ev OSRELEASE 546Optional base name for generated media images when invoking the 547.Cm install 548target 549.Pq e.g., FreeBSD-12.1-RELEASE-amd64 . 550Defaults to the output of 551.Ic `uname -s`-`uname -r`-`uname -p` 552within the chroot. 553.It Ev WORLDDIR 554Location of a directory containing the src tree. 555By default, the directory 556above the one containing the makefile 557.Pq Pa src . 558.It Ev PORTSDIR 559Location of a directory containing the ports tree. 560By default, 561.Pa /usr/ports . 562If it is unset or cannot be found, ports will not be included in the release. 563.It Ev NOPORTS 564If defined, the Ports Collection will be omitted from the release. 565.It Ev NOSRC 566If set, do not include system source code in the release. 567.It Ev TARGET 568The target hardware platform. 569This is analogous to the 570.Dq Nm uname Fl m 571output. 572This is necessary to cross-build some target architectures. 573For example, cross-building for ARM64 machines requires 574.Ev TARGET_ARCH Ns = Ns Li aarch64 575and 576.Ev TARGET Ns = Ns Li arm64 . 577If not set, 578.Ev TARGET 579defaults to the current hardware platform. 580.It Ev TARGET_ARCH 581The target machine processor architecture. 582This is analogous to the 583.Dq Nm uname Fl p 584output. 585Set this to cross-build for a different architecture. 586If not set, 587.Ev TARGET_ARCH 588defaults to the current machine architecture, unless 589.Ev TARGET 590is also set, in which case it defaults to the appropriate 591value for that platform. 592Typically, one only needs to set 593.Ev TARGET . 594.El 595.Sh FILES 596.Bl -tag -compact -width Pa 597.It Pa /usr/doc/Makefile 598.It Pa /usr/doc/share/mk/doc.project.mk 599.It Pa /usr/ports/Mk/bsd.port.mk 600.It Pa /usr/ports/Mk/bsd.sites.mk 601.It Pa /usr/share/examples/etc/make.conf 602.It Pa /usr/src/Makefile 603.It Pa /usr/src/Makefile.inc1 604.It Pa /usr/src/release/Makefile 605.It Pa /usr/src/release/Makefile.vm 606.It Pa /usr/src/release/release.sh 607.It Pa /usr/src/release/release.conf.sample 608.It Pa /usr/src/release/tools/*.conf 609.It Pa /usr/src/release/tools/vmimage.subr 610.El 611.Sh EXAMPLES 612The following sequence of commands can be used to build a 613.Dq "-CURRENT snapshot": 614.Bd -literal -offset indent 615cd /usr 616git clone -b main https://git.freebsd.org/src.git src 617cd src 618make buildworld buildkernel 619cd release 620make obj 621make release 622make install DESTDIR=/var/freebsd-snapshot 623.Ed 624.Pp 625After running these commands, all produced distribution files (tarballs 626for FTP, CD-ROM images, etc.) are available in the 627.Pa /var/freebsd-snapshot 628directory. 629.Pp 630The following sequence of commands can be used to build a 631.Dq "-CURRENT snapshot" 632in a clean environment, including ports and documentation: 633.Bd -literal -offset indent 634cd /usr/src/release 635sh release.sh 636.Ed 637.Pp 638Optionally, a configuration file can be used customize the release build, 639such as the subversion revision to use, the branch of the subversion tree for 640.Li src/ , 641.Li ports/ , 642and 643.Li doc/ . 644.Bd -literal -offset indent 645cd /usr/src/release 646sh release.sh -c $HOME/release.conf 647.Ed 648.Pp 649Configuration files specific to various supported embedded systems, such as 650the Raspberry Pi, exist in the directory corresponding to the 651.Va TARGET 652.Xr make 1 653variable. 654For example, to build an image for the Raspberry Pi: 655.Bd -literal -offset indent 656cd /usr/src/release 657sh release.sh -c arm/RPI-B.conf 658.Ed 659.Pp 660To build an image for the Raspberry Pi 3: 661.Bd -literal -offset indent 662cd /usr/src/release 663sh release.sh -c arm64/RPI3.conf 664.Ed 665.Pp 666After running these commands, all prepared release files are available in the 667.Pa /scratch 668directory. 669The target directory can be changed by specifying the 670.Va CHROOTDIR 671variable in 672.Li release.conf . 673.Sh COMPATIBILITY 674The reldoc target was removed in commit f61e92ca5a23, and 675.Ev DOCDIR , 676.Ev DOCBRANCH , 677.Ev DOC_UPDATE_SKIP , 678and 679.Ev NODOC 680are therefore no longer supported. 681.Sh SEE ALSO 682.Xr cc 1 , 683.Xr git 1 Pq Pa ports/devel/git , 684.Xr install 1 , 685.Xr make 1 , 686.Xr uname 1 , 687.Xr md 4 , 688.Xr make.conf 5 , 689.Xr build 7 , 690.Xr ports 7 , 691.Xr chroot 8 , 692.Xr mtree 8 , 693.Xr sysctl 8 694.Rs 695.%T "FreeBSD Release Engineering" 696.%U https://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/freebsd-releng/ 697.Re 698.Rs 699.%T "FreeBSD Developers' Handbook" 700.%U https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/developers-handbook/ 701.Re 702.Sh HISTORY 703.Fx 7041.x 705used a manual checklist, compiled by 706.An Rod Grimes , 707to produce a release. 708Apart from being incomplete, the list put a lot of specific demands on 709available file systems and was quite torturous to execute. 710.Pp 711As part of the 712.Fx 2.0 713release engineering effort, significant 714effort was spent getting 715.Pa src/release/Makefile 716into a shape where it could at least automate most of the tediousness 717of building a release in a sterile environment. 718.Pp 719For the 720.Fx 9.0 721release, 722.Pa src/release/Makefile 723was overhauled and the wrapper script 724.Pa src/release/generate-release.sh 725introduced to support the introduction of a new installer. 726.Pp 727For the 728.Fx 9.2 729release, 730.Pa src/release/release.sh 731was introduced to support per-build configuration files. 732.Pa src/release/release.sh 733is heavily based on the 734.Pa src/release/generate-release.sh 735script. 736.Pp 737At near 1000 revisions spread over multiple branches, the 738.Xr git 1 739log of 740.Pa src/release/Makefile 741contains a vivid historical record of some 742of the hardships release engineers go through. 743.Sh AUTHORS 744.Pa src/release/Makefile 745was originally written by 746.An -nosplit 747.An Rod Grimes , 748.An Jordan Hubbard , 749and 750.An Poul-Henning Kamp . 751.Pp 752This manual page was originally written by 753.An Murray Stokely Aq Mt murray@FreeBSD.org . 754.Pp 755It was updated by 756.An Nathan Whitehorn Aq Mt nwhitehorn@FreeBSD.org 757to include the 758.Fa generate-release.sh 759script used for the 760.Fx 9.0 761release cycle. 762.Pp 763It was later updated by 764.An Glen Barber Aq Mt gjb@FreeBSD.org 765to include the 766.Fa release.sh 767script used for the 768.Fx 9.2 769release cycle. 770