1mk-files 2******** 3 4The term ``mk-files`` refers to a collection of ``*.mk`` files. 5 6You need bmake_ or a *recent* NetBSD_ make. 7If in doubt use bmake_. 8 9Introduction 10============ 11 12Many years ago, when building large software projects, I used GNU make 13(or my own patched version of it), and had developed a set of macros 14to simplify developing complex build trees. 15 16Since the early 90's my main development machines, run BSD 17(NetBSD_ to be precise, and more recently FreeBSD), and the BSD source 18tree is good example of a large software project. 19It quickly became clear that ``/usr/share/mk/*.mk`` were a great 20model, but at the time were quite tightly linked to building the BSD tree. 21 22Much as I liked using NetBSD, my customers were more likely to be 23using SunOS, HP-UX etc, so I started on bmake_ and a portable collection 24of mk-files (mk.tar.gz_). NetBSD provided much of the original structure. 25 26Since then I've added a lot of features to NetBSD's make and hence to 27bmake which is kept closely in sync. The mk-files however have 28diverged quite a bit, though ideas are still picked up from NetBSD 29and FreeBSD. 30 31Basics 32------ 33 34The BSD build model is very simple. A directory produces one 35component, which is generally either a library or a program. 36Library makefiles include ``lib.mk`` and programs include ``prog.mk`` 37and they *do the right thing*. 38 39A simple library makefile might look like:: 40 41 LIB = sig 42 43 SRCS = \ 44 sigaction.c \ 45 sigcompat.c \ 46 sighdl.c 47 48 .include <lib.mk> 49 50a simple program makefile:: 51 52 PROG = cat 53 54 SRCS = cat.c 55 56 .include <prog.mk> 57 58in such cases even the ``SRCS`` line is unnecessary as ``prog.mk`` 59will default it to ``${PROG}.c``. 60 61It is the sensible use of defaults and the plethora of macro modifiers 62provided by bmake_ that allow simple makefiles such as the above to 63*just work* on many different systems. 64 65 66mk-files 67======== 68 69This section provides a brief description of some of the ``*.mk`` 70files. 71 72sys.mk 73------ 74 75When bmake starts, it looks for ``sys.mk`` and reads it before doing 76anything else. Thus, this is the place to setup the environment for 77everyone else. 78 79In this distribution, ``sys.mk`` avoids doing anything platform dependent. 80It is quite short, and includes a number of other files (which may or 81may not exists) 82 83sys.env.mk 84 If it exists, is expected to do things like conditioning the 85 environment. Since it will only be included by the initial 86 instance of bmake, it should ``.export`` anything that 87 sub-makes might need. 88 89examples/sys.clean-env.mk 90 An example of how to clean the environment. 91 See the file for all the details:: 92 93 .if ${MAKE_VERSION} >= 20100606 && ${.MAKE.LEVEL} == 0 94 # we save any env var that starts with these 95 MAKE_SAVE_ENV_PREFIX += SB MK MAKE MACHINE NEED_ CCACHE DISTCC USE_ SSH 96 MAKE_SAVE_ENV_VARS += \ 97 PATH HOME USER LOGNAME \ 98 SRCTOP OBJTOP OBJROOT \ 99 ${_env_vars} 100 101 _env_vars != env | egrep '^(${MAKE_SAVE_ENV_PREFIX:ts|})' | sed 's,=.*,,'; echo 102 _export_list = 103 .for v in ${MAKE_SAVE_ENV_VARS:O:u} 104 .if !empty($v) 105 _export_list += $v 106 $v := ${$v} 107 .endif 108 .endfor 109 # now clobber the environment 110 .unexport-env 111 112 # list of vars that we handle specially below 113 _tricky_env_vars = MAKEOBJDIR 114 # export our selection - sans tricky ones 115 .export ${_export_list:${_tricky_env_vars:${M_ListToSkip}}} 116 117 # this next bit may need tweaking 118 .if defined(MAKEOBJDIR) 119 srctop := ${SRCTOP:U${SB_SRC:U${SB}/src}} 120 objroot := ${OBJROOT:U${SB_OBJROOT:U${SB}/${SB_OBJPREFIX}}} 121 # we'll take care of MACHINE below 122 objtop := ${OBJTOP:U${objroot}${MACHINE}} 123 .if !empty(objtop) 124 # we would normally want something like (/bin/sh): 125 # MAKEOBJDIR="\${.CURDIR:S,${SRCTOP},${OBJROOT}\${MACHINE},}" 126 # the $$ below is how we achieve the same result here. 127 # since everything saved from the environment above 128 # has run through := we need to compensate for ${MACHINE} 129 MAKEOBJDIR = $${.CURDIR:S,${srctop},${objtop:S,${MACHINE},\${MACHINE},},} 130 131 # export these as-is, and do not track... 132 .export-env ${_tricky_env_vars} 133 # now evaluate for ourselves 134 .for v in ${_tricky_env_vars} 135 $v := ${$v} 136 .endfor 137 138 .endif 139 .endif 140 .endif 141 142 143host-target.mk 144 Is used to set macros like ``HOST_TARGET``, ``HOST_OS`` and 145 ``host_os`` which are used to find the next step. 146 Note: since 20130303 bmake provides ``.MAKE.OS`` set to 147 the equivalent of ``HOST_OS``. 148 149sys/\*.mk 150 Platform specific additions, such as ``Darwin.mk`` or ``SunOS.mk`` 151 set things like ``HOST_LIBEXT = .dylib`` for Darwin or 152 ``SHLIB_FULLVERSION = ${SHLIB_MAJOR}`` for SunOS 5. 153 If there is no OS specific file, ``sys/Generic.mk`` is used. 154 155local.sys.mk 156 Any ``local.*.mk`` file is not part of the distribution. 157 This provides a hook for sites to do extra setup without 158 having to edit the distributed files. 159 160 161The above arrangement makes it easy for the mk files to be part of a 162src tree on an NFS volume and to allow building on multiple platforms. 163 164options.mk 165---------- 166 167Inspired by FreeBSD's ``bsd.own.mk`` but more flexible. 168FreeBSD now have similar functionality in ``bsd.mkopt.mk``. 169 170It allows users to express their intent with respect to options 171``MK_*`` by setting ``WITH_*`` or ``WITHOUT_*``. 172 173Note: ``WITHOUT_*`` wins if both are set, and makefiles can set 174``NO_*`` to say they cannot handle that option, or even ``MK_*`` if 175they really need to. 176 177lib.mk 178------ 179 180This file is used to build a number of different libraries from the 181same SRCS. 182 183``lib${LIB}.a`` 184 An archive lib of ``.o`` files, this is the default 185 186``lib${LIB}_p.a`` 187 A profiled lib of ``.po`` files. 188 Still an archive lib, but all the objects are built with 189 profiling in mind - hence the different extension. 190 It is skipped if ``MK_PROFILE`` is "no". 191 192``lib${LIB}_pic.a`` 193 An archive of ``.so`` objects compiled for relocation. 194 On NetBSD this is the input to ``lib${LIB}.${LD_so}``, it is 195 skipped if ``MK_PIC`` or ``MK_PICLIB`` are "no". 196 197``lib${LIB}.${LD_so}`` 198 A shared library. The value of ``LD_so`` is very platform 199 specific. For example:: 200 201 # SunOS 5 and most other ELF systems 202 libsslfd.so.1 203 204 # Darwin 205 libsslfd.1.dylib 206 207 This library will only be built if ``SHLIB_MAJOR`` has 208 a value, and ``MK_PIC`` is not set to "no". 209 210There is a lot of platform specific tweaking in ``lib.mk``, largely the 211result of the original distributions trying to avoid interfering with 212the system's ``sys.mk``. 213 214libnames.mk 215----------- 216 217This is included by both ``prog.mk`` and ``lib.mk`` and tries to 218include ``*.libnames.mk`` of which: 219 220``local.libnames.mk`` 221 does not exist unless you create it. It is a handy way for you 222 to customize without touching the distributed files. 223 For example, on a test machine I needed to build openssl but 224 not install it, so put the following in ``local.libnames.mk``:: 225 226 .if ${host_os} == "sunos" 227 LIBCRYPTO = ${OBJTOP}/openssl/lib/crypto/libcrypto${DLIBEXT} 228 LIBSSL = ${OBJTOP}/openssl/lib/ssl/libssl${DLIBEXT} 229 INCLUDES_libcrypto = -I${OBJ_libcrypto} 230 .endif 231 232 The makefile created an openssl dir in ``${OBJ_libcrypto}`` to 233 gather all the headers. dpadd.mk_ did the rest. 234 235``host.libnames.mk`` 236 contains logic to find any libs named in ``HOST_LIBS`` in 237 ``HOST_LIBDIRS``. 238 239Each file above gets an opportunity to define things like:: 240 241 LIBSSLFD ?= ${OBJTOP}/ssl/lib/sslfd/libsslfd${DLIBEXT} 242 INCLUDES_libsslfd = -I${SRC_libsslfd}/h -I${OBJ_libslfd} 243 244these are used by dpadd.mk_ and will be explained below. 245 246dpadd.mk 247-------- 248 249This file looks like line noise, and is best considered read-only. 250However it provides some very useful functionality, which simplifies the build. 251 252Makefiles can use the LIB* macros defined via libnames.mk_ or anywhere 253else in various ways:: 254 255 # indicate that we need to include headers from LIBCRYPTO 256 # this would result in ${INCLUDES_libcrypto} being added to CFLAGS. 257 SRC_LIBS += ${LIBCRYPTO} 258 259 # indicate that libsslfd must be built already. 260 # it also has the same effect as SRC_LIBS 261 DPADD += ${LIBSSLFD} 262 263 # indicate that not only must libsslfd be built, 264 # but that we need to link with it. 265 # this is almost exactly equivalent to 266 # DPADD += ${LIBSSLFD} 267 # LDADD += -L${LIBSSLFD:H} -lsslfd 268 # and mostly serves to ensure that DPADD and LDADD are in sync. 269 DPLIBS += ${LIBSSLFD} 270 271Any library (referenced by its full path) in any of the above, is 272added to ``DPMAGIC_LIBS`` with the following results, for each lib *foo*. 273 274``SRC_libfoo`` 275 Is set to indicate where the src for libfoo is. 276 By default it is derived from ``LIBFOO`` by replacing 277 ``${OBJTOP}`` with ``${SRCTOP}``. 278 279``OBJ_libfoo`` 280 Not very exciting, is just the dir where libfoo lives. 281 282``INCLUDES_libfoo`` 283 What to add to ``CFLAGS`` to find the public headers. 284 The default varies. If ``${SRC_libfoo}/h`` exists, it is assumed 285 to be the home of all public headers and thus the default is 286 ``-I${SRC_libfoo}/h`` 287 288 Otherwise we make no assumptions and the default is 289 ``-I${SRC_libfoo} -I${OBJ_libfoo}`` 290 291``LDADD_libfoo`` 292 This only applies to libs reference via ``DPLIBS``. 293 The default is ``-lfoo``, ``LDADD_*`` provides a hook to 294 instantiate other linker flags at the appropriate point 295 without losing the benfits of ``DPLIBS``. 296 297prog.mk 298------- 299 300Compiles the specified SRCS and links them and the nominated libraries 301into a program. Prog makefiles usually need to list the libraries 302that need to be linked. We prefer use of ``DPLIBS`` but the more 303traditional ``DPADD`` and ``LDADD`` work just as well. 304That is:: 305 306 DPLIBS += ${LIBCRYPTO} 307 308is equivalent to:: 309 310 DPADD += ${LIBCRYPTO} 311 LDADD += -lcrypto 312 313obj.mk 314------ 315 316One of the cool aspects of BSD make, is its support for separating 317object files from the src tree. This is also the source of much 318confusion to some. 319 320Traditionally one had to do a separate ``make obj`` pass through the 321tree. If ``MK_AUTO_OBJ`` is set we include auto.obj.mk_. 322 323In fact if ``MKOBJDIRS`` is set to "auto", `sys.mk`_ will set 324``MK_AUTO_OBJ=yes`` and include auto.obj.mk_ since it is best done early. 325 326auto.obj.mk 327----------- 328 329This leverages the ``.OBJDIR`` target introduced some years ago to 330NetBSD make, to automatically create and use the desired object dir. 331 332subdir.mk 333--------- 334 335This is the traditional means of walking the tree. A makefile sets 336``SUBDIR`` to the list of sub-dirs to visit. 337 338If ``SUBDIR_MUST_EXIST`` is set, missing directories cause an error, 339otherwise a warning is issued. If you don't even want the warning, 340set ``MISSING_DIR=continue``. 341 342Traditionally, ``subdir.mk`` prints clues as it visits each subdir:: 343 344 ===> ssl 345 ===> ssl/lib 346 ===> ssl/lib/sslfd 347 348you can suppress that - or enhance it by setting ``ECHO_DIR``:: 349 350 # suppress subdir noise 351 ECHO_DIR=: 352 # print time stamps 353 ECHO_DIR=echo @ `date "+%s [%Y-%m-%d %T] "` 354 355I prefer to use `dirdeps.mk`_ which makes ``subdir.mk`` irrelevant. 356 357links.mk 358-------- 359 360Provides rules for processing lists of ``LINKS`` and ``SYMLINKS``. 361Each is expected to be a list of ``link`` and ``target`` pairs 362(``link`` -> ``target``). 363 364The logic is generally in a ``_*_SCRIPT`` which is referenced in a 365``_*_USE`` (``.USE``) target. 366 367The ``_BUILD_*`` forms are identical, but do not use ``${DESTDIR}`` 368and so are useful for creating symlinks during the build phase. 369For example:: 370 371 SYMLINKS += ${.CURDIR}/${MACHINE_ARCH}/include machine 372 header_links: _BUILD_SYMLINKS_USE 373 374 md.o: header_links 375 376would create a symlink called ``machine`` in ``${.OBJDIR}`` pointing to 377``${.CURDIR}/${MACHINE_ARCH}/include`` before compiling ``md.o`` 378 379 380autoconf.mk 381----------- 382 383Deals with running (or generating) GNU autoconf ``configure`` scripts. 384 385dep.mk 386------ 387 388Deals with collecting dependencies. Another useful feature of BSD 389make is the separation of this sort of information into a ``.depend`` 390file. ``MKDEP_CMD`` needs to point to a suitable tool (like mkdeps.sh_) 391 392If ``MK_AUTODEP`` is "yes" it sets ``MKDEP_MK`` to autodep.mk_ by default. 393 394``MKDEP_MK`` can also be set to `auto.dep.mk`_ which is more efficient 395but does not support an explicit ``depend`` target. 396 397autodep.mk 398---------- 399 400Leverages the ``-MD`` feature of recent GCC to collect dependency 401information as a side effect of compilation. With this GCC puts 402dependency info into a ``.d`` file. 403 404Unfortunately GCC bases the name of the ``.d`` file on the name of the 405input rather than the output file, which causes problems when the same 406source is compiled different ways. The latest GCC supports ``-MF`` to 407name the ``.d`` file and ``-MT`` to control the name to put as the 408dependent. 409 410Recent bmake allows dependencies for the ``.END`` target (run at the 411end if everything was successful), and ``autodep.mk`` uses this to 412post process the ``.d`` files into ``.depend``. 413 414auto.dep.mk 415----------- 416 417A much simpler implementation than autodep.mk_ it uses 418``-MF ${.TARGET:T}.d`` 419to avoid possible conflicts during parallel builds. 420This precludes the use of suffix rules to drive ``make depend``, so 421dep.mk_ handles that if specifically requested. 422 423If ``bmake`` is 20160218 or newer, ``auto.dep.mk`` uses ``.dinclude`` 424to includes the ``*.d`` files directly thus avoiding the need to 425create a ``.depend`` file from them. 426 427own.mk 428------ 429 430Normally included by ``init.mk`` (included by ``lib.mk`` and 431``prog.mk`` etc), sets macros for default ownership etc. 432 433It includes ``${MAKECONF}`` if it is defined and exists. 434 435ldorder.mk 436---------- 437 438Leverages ``bmake`` to compute optimal link order for libraries. 439This works nicely and makes refactoring a breeze - so long as you 440have no (or few) cicular dependencies between libraries. 441 442man.mk 443------ 444 445Deals with man pages. 446 447warnings.mk 448----------- 449 450This provides a means of fine grained control over warnings on a per 451``${MACHINE}`` or even file basis. 452 453A makefile sets ``WARNINGS_SET`` to name a list of warnings 454and individual ``W_*`` macros can be used to tweak them. 455For example:: 456 457 WARNINGS_SET = HIGH 458 W_unused_sparc = -Wno-unused 459 460would add all the warnings in ``${HIGH_WARNINGS}`` to CFLAGS, but 461on sparc, ``-Wno-unused`` would replace ``-Wunused``. 462 463You should never need to edit ``warnings.mk``, it will include 464``warnings-sets.mk`` if it exists and you use that to make any local 465customizations. 466 467rst2htm.mk 468---------- 469 470Logic to simplify generating HTML (and PDF) documents from ReStructuredText. 471 472cython.mk 473--------- 474 475Logic to build Python C interface modules using Cython_ 476 477.. _Cython: http://www.cython.org/ 478 479cc-wrap.mk 480---------- 481 482This makefile leverages two new features in bmake 20220126 and later. 483 484First is the ablity to set target local variables (GNU make has done 485this for ages). 486 487The second (only intersting if using `meta mode`_) 488allows filtering commands before comparison with previous run to 489decide if a target is out-of-date. 490 491In the past, making use of compiler wrappers like ``ccache``, 492``distcc`` or the newer ``icecc`` could get quite ugly. 493Using ``cc-wrap.mk`` it could not be simpler. 494 495Meta mode 496========= 497 498The 20110505 and later versions of ``mk-files`` include a number of 499makefiles contributed by Juniper Networks, Inc. 500These allow the latest version of bmake_ to run in `meta mode`_ 501see `dirdeps.mk`_ 502 503.. _`dirdeps.mk`: /help/sjg/dirdeps.htm 504.. _`meta mode`: bmake-meta-mode.htm 505 506Install 507======= 508 509You can use the content of mk.tar.gz_ without installing at all. 510 511The script ``install-mk`` takes care of copying ``*.mk`` into a 512destination directory, and unless told not to, create ``bsd.*.mk`` links 513for ``lib.mk`` etc. 514 515If you just want to create the ``bsd.*.mk`` links in the directory 516where you unpacked the tar file, you can:: 517 518 ./mk/install-mk ./mk 519 520------ 521 522.. _bmake: bmake.htm 523.. _NetBSD: http://www.netbsd.org/ 524.. _mkdeps.sh: http://www.crufty.net/ftp/pub/sjg/mkdeps.sh 525.. _mk.tar.gz: http://www.crufty.net/ftp/pub/sjg/mk.tar.gz 526 527:Author: sjg@crufty.net 528:Revision: $Id: mk-files.txt,v 1.21 2022/02/04 19:01:05 sjg Exp $ 529:Copyright: Crufty.NET 530