1mk-files 2******** 3 4The term ``mk-files`` refers to a collection of ``*.mk`` files. 5 6You need bmake_ or a *recent* NetBSD_ make. 7If in doubt use bmake_. 8 9Introduction 10============ 11 12Many years ago, when building large software projects, I used GNU make 13(or my own patched version of it), and had developed a set of macros 14to simplify developing complex build trees. 15 16Since the early 90's my main development machines, run BSD 17(NetBSD_ to be precise, and more recently FreeBSD), and the BSD source 18tree is good example of a large software project. 19It quickly became clear that ``/usr/share/mk/*.mk`` were a great 20model, but at the time were quite tightly linked to building the BSD tree. 21 22Much as I liked using NetBSD, my customers were more likely to be 23using SunOS, HP-UX etc, so I started on bmake_ and a portable collection 24of mk-files (mk.tar.gz_). NetBSD provided much of the original structure. 25 26Since then I've added a lot of features to NetBSD's make and hence to 27bmake which is kept closely in sync. The mk-files however have 28diverged quite a bit, though ideas are still picked up from NetBSD 29and FreeBSD. 30 31Basics 32------ 33 34The BSD build model is very simple. A directory produces one 35component, which is generally either a library or a program. 36Library makefiles include ``lib.mk`` and programs include ``prog.mk`` 37and they *do the right thing*. 38 39A simple library makefile might look like:: 40 41 LIB = sig 42 43 SRCS = \ 44 sigaction.c \ 45 sigcompat.c \ 46 sighdl.c 47 48 .include <lib.mk> 49 50a simple program makefile:: 51 52 PROG = cat 53 54 SRCS = cat.c 55 56 .include <prog.mk> 57 58in such cases even the ``SRCS`` line is unnecessary as ``prog.mk`` 59will default it to ``${PROG}.c``. 60 61It is the sensible use of defaults and the plethora of macro modifiers 62provided by bmake_ that allow simple makefiles such as the above to 63*just work* on many different systems. 64 65 66mk-files 67======== 68 69This section provides a brief description of some of the ``*.mk`` 70files. 71 72sys.mk 73------ 74 75When bmake starts, it looks for ``sys.mk`` and reads it before doing 76anything else. Thus, this is the place to setup the environment for 77everyone else. 78 79In this distribution, ``sys.mk`` avoids doing anything platform or 80site dependent. 81It is quite short, and includes a number of other files (which may or 82may not exists) 83 84sys.env.mk 85 If it exists, is expected to do things like conditioning the 86 environment. Since it will only be included by the initial 87 instance of bmake, it should ``.export`` anything that 88 sub-makes might need. 89 90examples/sys.clean-env.mk 91 An example of how to clean the environment. 92 See the file for all the details:: 93 94 .if ${MAKE_VERSION} >= 20100606 && ${.MAKE.LEVEL} == 0 95 # we save any env var that starts with these 96 MAKE_SAVE_ENV_PREFIX += SB MK MAKE MACHINE NEED_ CCACHE DISTCC USE_ SSH 97 MAKE_SAVE_ENV_VARS += \ 98 PATH HOME USER LOGNAME \ 99 SRCTOP OBJTOP OBJROOT \ 100 ${_env_vars} 101 102 _env_vars != env | egrep '^(${MAKE_SAVE_ENV_PREFIX:ts|})' | sed 's,=.*,,'; echo 103 _export_list = 104 .for v in ${MAKE_SAVE_ENV_VARS:O:u} 105 .if !empty($v) 106 _export_list += $v 107 $v := ${$v} 108 .endif 109 .endfor 110 # now clobber the environment 111 .unexport-env 112 113 # list of vars that we handle specially below 114 _tricky_env_vars = MAKEOBJDIR 115 # export our selection - sans tricky ones 116 .export ${_export_list:${_tricky_env_vars:${M_ListToSkip}}} 117 118 # this next bit may need tweaking 119 .if defined(MAKEOBJDIR) 120 srctop := ${SRCTOP:U${SB_SRC:U${SB}/src}} 121 objroot := ${OBJROOT:U${SB_OBJROOT:U${SB}/${SB_OBJPREFIX}}} 122 # we'll take care of MACHINE below 123 objtop := ${OBJTOP:U${objroot}${MACHINE}} 124 .if !empty(objtop) 125 # we would normally want something like (/bin/sh): 126 # MAKEOBJDIR="\${.CURDIR:S,${SRCTOP},${OBJROOT}\${MACHINE},}" 127 # the $$ below is how we achieve the same result here. 128 # since everything saved from the environment above 129 # has run through := we need to compensate for ${MACHINE} 130 MAKEOBJDIR = $${.CURDIR:S,${srctop},${objtop:S,${MACHINE},\${MACHINE},},} 131 132 # export these as-is, and do not track... 133 .export-env ${_tricky_env_vars} 134 # now evaluate for ourselves 135 .for v in ${_tricky_env_vars} 136 $v := ${$v} 137 .endfor 138 139 .endif 140 .endif 141 .endif 142 143 144host-target.mk 145 Is used to set macros like ``HOST_TARGET``, ``HOST_OS`` and 146 ``host_os`` which are used to find the next step. 147 Note: since 20130303 bmake provides ``.MAKE.OS`` set to 148 the equivalent of ``HOST_OS``. 149 150sys/\*.mk 151 Platform specific additions, such as ``Darwin.mk`` or ``SunOS.mk`` 152 set things like ``HOST_LIBEXT = .dylib`` for Darwin or 153 ``SHLIB_FULLVERSION = ${SHLIB_MAJOR}`` for SunOS 5. 154 If there is no OS specific file, ``sys/Generic.mk`` is used. 155 156local.sys.mk 157 Any ``local.*.mk`` file is not part of the distribution. 158 This provides a hook for sites to do extra setup without 159 having to edit the distributed files. 160 161 162The above arrangement makes it easy for the mk files to be part of a 163src tree on an NFS volume and to allow building on multiple platforms. 164 165options.mk 166---------- 167 168Inspired by FreeBSD's ``bsd.own.mk`` but more flexible. 169FreeBSD now have similar functionality in ``bsd.mkopt.mk``. 170 171It allows users to express their intent with respect to options 172``MK_*`` by setting ``WITH_*`` or ``WITHOUT_*``. 173 174Note: ``WITHOUT_*`` wins if both are set, and makefiles can set 175``NO_*`` to say they cannot handle that option, or even ``MK_*`` if 176they really need to. 177 178lib.mk 179------ 180 181This file is used to build a number of different libraries from the 182same SRCS. 183 184``lib${LIB}.a`` 185 An archive lib of ``.o`` files, this is the default 186 187``lib${LIB}_p.a`` 188 A profiled lib of ``.po`` files. 189 Still an archive lib, but all the objects are built with 190 profiling in mind - hence the different extension. 191 It is skipped if ``MK_PROFILE`` is "no". 192 193``lib${LIB}_pic.a`` 194 An archive of ``.so`` objects compiled for relocation. 195 On NetBSD this is the input to ``lib${LIB}.${LD_so}``, it is 196 skipped if ``MK_PIC`` or ``MK_PICLIB`` are "no". 197 198``lib${LIB}.${LD_so}`` 199 A shared library. The value of ``LD_so`` is very platform 200 specific. For example:: 201 202 # SunOS 5 and most other ELF systems 203 libsslfd.so.1 204 205 # Darwin 206 libsslfd.1.dylib 207 208 This library will only be built if ``SHLIB_MAJOR`` has 209 a value, and ``MK_PIC`` is not set to "no". 210 211There is a lot of platform specific tweaking in ``lib.mk``, largely the 212result of the original distributions trying to avoid interfering with 213the system's ``sys.mk``. 214 215libnames.mk 216----------- 217 218This is included by both ``prog.mk`` and ``lib.mk`` and tries to 219include ``*.libnames.mk`` of which: 220 221``local.libnames.mk`` 222 does not exist unless you create it. It is a handy way for you 223 to customize without touching the distributed files. 224 For example, on a test machine I needed to build openssl but 225 not install it, so put the following in ``local.libnames.mk``:: 226 227 .if ${host_os} == "sunos" 228 LIBCRYPTO = ${OBJTOP}/openssl/lib/crypto/libcrypto${DLIBEXT} 229 LIBSSL = ${OBJTOP}/openssl/lib/ssl/libssl${DLIBEXT} 230 INCLUDES_libcrypto = -I${OBJ_libcrypto} 231 .endif 232 233 The makefile created an openssl dir in ``${OBJ_libcrypto}`` to 234 gather all the headers. dpadd.mk_ did the rest. 235 236``host.libnames.mk`` 237 contains logic to find any libs named in ``HOST_LIBS`` in 238 ``HOST_LIBDIRS``. 239 240Each file above gets an opportunity to define things like:: 241 242 LIBSSLFD ?= ${OBJTOP}/ssl/lib/sslfd/libsslfd${DLIBEXT} 243 INCLUDES_libsslfd = -I${SRC_libsslfd}/h -I${OBJ_libslfd} 244 245these are used by dpadd.mk_ and will be explained below. 246 247dpadd.mk 248-------- 249 250This file looks like line noise, and is best considered read-only. 251However it provides some very useful functionality, which simplifies the build. 252 253Makefiles can use the LIB* macros defined via libnames.mk_ or anywhere 254else in various ways:: 255 256 # indicate that we need to include headers from LIBCRYPTO 257 # this would result in ${INCLUDES_libcrypto} being added to CFLAGS. 258 SRC_LIBS += ${LIBCRYPTO} 259 260 # indicate that libsslfd must be built already. 261 # it also has the same effect as SRC_LIBS 262 DPADD += ${LIBSSLFD} 263 264 # indicate that not only must libsslfd be built, 265 # but that we need to link with it. 266 # this is almost exactly equivalent to 267 # DPADD += ${LIBSSLFD} 268 # LDADD += -L${LIBSSLFD:H} -lsslfd 269 # and mostly serves to ensure that DPADD and LDADD are in sync. 270 DPLIBS += ${LIBSSLFD} 271 272Any library (referenced by its full path) in any of the above, is 273added to ``DPMAGIC_LIBS`` with the following results, for each lib *foo*. 274 275``SRC_libfoo`` 276 Is set to indicate where the src for libfoo is. 277 By default it is derived from ``LIBFOO`` by replacing 278 ``${OBJTOP}`` with ``${SRCTOP}``. 279 280``OBJ_libfoo`` 281 Not very exciting, is just the dir where libfoo lives. 282 283``INCLUDES_libfoo`` 284 What to add to ``CFLAGS`` to find the public headers. 285 The default varies. If ``${SRC_libfoo}/h`` exists, it is assumed 286 to be the home of all public headers and thus the default is 287 ``-I${SRC_libfoo}/h`` 288 289 Otherwise we make no assumptions and the default is 290 ``-I${SRC_libfoo} -I${OBJ_libfoo}`` 291 292``LDADD_libfoo`` 293 This only applies to libs reference via ``DPLIBS``. 294 The default is ``-lfoo``, ``LDADD_*`` provides a hook to 295 instantiate other linker flags at the appropriate point 296 without losing the benfits of ``DPLIBS``. 297 298prog.mk 299------- 300 301Compiles the specified SRCS and links them and the nominated libraries 302into a program. Prog makefiles usually need to list the libraries 303that need to be linked. We prefer use of ``DPLIBS`` but the more 304traditional ``DPADD`` and ``LDADD`` work just as well. 305That is:: 306 307 DPLIBS += ${LIBCRYPTO} 308 309is equivalent to:: 310 311 DPADD += ${LIBCRYPTO} 312 LDADD += -lcrypto 313 314obj.mk 315------ 316 317One of the cool aspects of BSD make, is its support for separating 318object files from the src tree. This is also the source of much 319confusion for people unfamiliar with it. 320 321Traditionally one had to do a separate ``make obj`` pass through the 322tree. If ``MK_AUTO_OBJ`` is set we include auto.obj.mk_. 323 324In fact if ``MKOBJDIRS`` is set to "auto", `sys.mk`_ will set 325``MK_AUTO_OBJ=yes`` and include auto.obj.mk_ since it is best done early. 326 327auto.obj.mk 328----------- 329 330Creates object dirs and leverages the ``.OBJDIR`` target introduced 331some years ago to NetBSD make, to use them. 332 333 334subdir.mk 335--------- 336 337This is the traditional means of walking the tree. A makefile sets 338``SUBDIR`` to the list of sub-dirs to visit. 339 340If ``SUBDIR_MUST_EXIST`` is set, missing directories cause an error, 341otherwise a warning is issued. If you don't even want the warning, 342set ``MISSING_DIR=continue``. 343 344Traditionally, ``subdir.mk`` prints clues as it visits each subdir:: 345 346 ===> ssl 347 ===> ssl/lib 348 ===> ssl/lib/sslfd 349 350you can suppress that - or enhance it by setting ``ECHO_DIR``:: 351 352 # suppress subdir noise 353 ECHO_DIR=: 354 # print time stamps 355 ECHO_DIR=echo @ `date "+%s [%Y-%m-%d %T] "` 356 357I prefer to use `dirdeps.mk`_ which makes ``subdir.mk`` irrelevant. 358 359links.mk 360-------- 361 362Provides rules for processing lists of ``LINKS`` and ``SYMLINKS``. 363Each is expected to be a list of ``link`` and ``target`` pairs 364(``link`` -> ``target``). 365 366The logic is generally in a ``_*_SCRIPT`` which is referenced in a 367``_*_USE`` (``.USE``) target. 368 369The ``_BUILD_*`` forms are identical, but do not use ``${DESTDIR}`` 370and so are useful for creating symlinks during the build phase. 371For example:: 372 373 SYMLINKS += ${.CURDIR}/${MACHINE_ARCH}/include machine 374 header_links: _BUILD_SYMLINKS_USE 375 376 md.o: header_links 377 378would create a symlink called ``machine`` in ``${.OBJDIR}`` pointing to 379``${.CURDIR}/${MACHINE_ARCH}/include`` before compiling ``md.o`` 380 381 382autoconf.mk 383----------- 384 385Deals with running (or generating) GNU autoconf ``configure`` scripts. 386 387dep.mk 388------ 389 390Deals with collecting dependencies. Another useful feature of BSD 391make is the separation of this sort of information into a ``.depend`` 392file. ``MKDEP_CMD`` needs to point to a suitable tool (like mkdeps.sh_) 393 394If ``MK_AUTODEP`` is "yes" it sets ``MKDEP_MK`` to autodep.mk_ by default. 395 396``MKDEP_MK`` can also be set to `auto.dep.mk`_ which is more efficient 397but does not support an explicit ``depend`` target. 398 399autodep.mk 400---------- 401 402Leverages the ``-MD`` feature of recent GCC to collect dependency 403information as a side effect of compilation. With this GCC puts 404dependency info into a ``.d`` file. 405 406Unfortunately GCC bases the name of the ``.d`` file on the name of the 407input rather than the output file, which causes problems when the same 408source is compiled different ways. The latest GCC supports ``-MF`` to 409name the ``.d`` file and ``-MT`` to control the name to put as the 410dependent. 411 412Recent bmake allows dependencies for the ``.END`` target (run at the 413end if everything was successful), and ``autodep.mk`` uses this to 414post process the ``.d`` files into ``.depend``. 415 416auto.dep.mk 417----------- 418 419A much simpler implementation than autodep.mk_ it uses 420``-MF ${.TARGET:T}.d`` 421to avoid possible conflicts during parallel builds. 422This precludes the use of suffix rules to drive ``make depend``, so 423dep.mk_ handles that if specifically requested. 424 425If ``bmake`` is 20160218 or newer, ``auto.dep.mk`` uses ``.dinclude`` 426to includes the ``*.d`` files directly thus avoiding the need to 427create a ``.depend`` file from them. 428 429own.mk 430------ 431 432Normally included by ``init.mk`` (included by ``lib.mk`` and 433``prog.mk`` etc), sets macros for default ownership etc. 434 435It includes ``${MAKECONF}`` if it is defined and exists. 436 437ldorder.mk 438---------- 439 440Leverages ``bmake`` to compute optimal link order for libraries. 441This works nicely and makes refactoring a breeze - so long as you 442have no (or few) cicular dependencies between libraries. 443 444man.mk 445------ 446 447Deals with man pages. 448 449warnings.mk 450----------- 451 452This provides a means of fine grained control over warnings on a per 453``${MACHINE}`` or even file basis. 454 455A makefile sets ``WARNINGS_SET`` to name a list of warnings 456and individual ``W_*`` macros can be used to tweak them. 457For example:: 458 459 WARNINGS_SET = HIGH 460 W_unused_sparc = -Wno-unused 461 462would add all the warnings in ``${HIGH_WARNINGS}`` to CFLAGS, but 463on sparc, ``-Wno-unused`` would replace ``-Wunused``. 464 465You should never need to edit ``warnings.mk``, it will include 466``warnings-sets.mk`` and/or ``local.warnings.mk`` to pick up 467customizations. 468 469rst2htm.mk 470---------- 471 472Logic to simplify generating HTML (and PDF) documents from ReStructuredText. 473 474cython.mk 475--------- 476 477Logic to build Python C interface modules using Cython_ 478 479.. _Cython: http://www.cython.org/ 480 481cc-wrap.mk 482---------- 483 484This makefile leverages two new features in bmake 20220126 and later. 485 486First is the ablity to set target local variables (GNU make has done 487this for ages). 488 489The second (only intersting if using `meta mode`_) 490allows filtering commands before comparison with previous run to 491decide if a target is out-of-date. 492 493In the past, making use of compiler wrappers like ``ccache``, 494``distcc`` or the newer ``icecc`` could get quite ugly. 495Using ``cc-wrap.mk`` it could not be simpler. 496 497jobs.mk 498------- 499 500This should be included by the top-level makefile. 501If you do:: 502 503 make something-jobs 504 505then ``jobs.mk`` will run:: 506 507 make -j${JOB_MAX} someting > ${JOB_LOGDIR}/something.log 2>&1 508 509this ensures you get a build log and JOB_MAX is assumed to be set 510optimally for the host. 511 512Meta mode 513========= 514 515The 20110505 and later versions of ``mk-files`` include a number of 516makefiles contributed by Juniper Networks, Inc. 517These allow the latest version of bmake_ to run in `meta mode`_ 518see `dirdeps.mk`_ 519 520.. _`dirdeps.mk`: /help/sjg/dirdeps.htm 521.. _`meta mode`: bmake-meta-mode.htm 522 523Install 524======= 525 526You can use the content of mk.tar.gz_ without installing at all. 527 528The script ``install-mk`` takes care of copying ``*.mk`` into a 529destination directory, and unless told not to, create ``bsd.*.mk`` links 530for ``lib.mk`` etc. 531 532If you just want to create the ``bsd.*.mk`` links in the directory 533where you unpacked the tar file, you can:: 534 535 ./mk/install-mk ./mk 536 537------ 538 539.. _bmake: bmake.htm 540.. _NetBSD: http://www.netbsd.org/ 541.. _mkdeps.sh: http://www.crufty.net/ftp/pub/sjg/mkdeps.sh 542.. _mk.tar.gz: http://www.crufty.net/ftp/pub/sjg/mk.tar.gz 543 544:Author: sjg@crufty.net 545:Revision: $Id: mk-files.txt,v 1.22 2023/04/16 23:43:33 sjg Exp $ 546:Copyright: Crufty.NET 547