1mk-files 2******** 3 4The term ``mk-files`` refers to a collection of ``*.mk`` files. 5 6You need bmake_ or a *recent* NetBSD_ make. 7If in doubt use bmake_. 8 9Introduction 10============ 11 12Many years ago, when building large software projects, I used GNU make 13(or my own patched version of it), and had developed a set of macros 14to simplify developing complex build trees. 15 16Since the early 90's my main development machines, run BSD 17(NetBSD_ to be precise), and the BSD source tree is good example of a 18large software project. It quickly became clear that 19``/usr/share/mk/*.mk`` were a great model, but were quite tightly 20linked to building the BSD tree. 21 22Much as I liked using NetBSD, my customers were more likely to be 23using SunOS, HP-UX etc, so I started on bmake_ and a portable collection 24of mk-files (mk.tar.gz_). NetBSD provided much of the original structure. 25 26Since then I've added a lot of features to NetBSD's make and hence to 27bmake which is kept closely in sync. The mk-files however have 28diverged quite a bit, though ideas are still picked up from NetBSD. 29 30Basics 31------ 32 33The BSD build model is very simple. A directory produces one 34component, which is generally either a library or a program. 35Library makefiles include ``lib.mk`` and programs include ``prog.mk`` 36and they *do the right thing*. 37 38A simple library makefile might look like:: 39 40 LIB = sig 41 42 SRCS = \ 43 sigaction.c \ 44 sigcompat.c \ 45 sighdl.c 46 47 .include <lib.mk> 48 49a simple program makefile:: 50 51 PROG = cat 52 53 SRCS = cat.c 54 55 .include <prog.mk> 56 57in such cases even the ``SRCS`` line is unnecessary as ``prog.mk`` 58will default it to ``${PROG}.c``. 59 60It is the sensible use of defaults and the plethora of macro modifiers 61provided by bmake_ that allow simple makefiles such as the above 62*just work* on many different systems. 63 64 65mk-files 66======== 67 68This section provides a brief description of some of the ``*.mk`` 69files. 70 71sys.mk 72------ 73 74When bmake starts, it looks for ``sys.mk`` and reads it before doing 75anything else. Thus, this is the place to setup the environment for 76everyone else. 77 78In this distribution, sys.mk avoids doing anything platform dependent. 79It is quite short, and includes a number of other files (which may or 80may not exists) 81 82sys.env.mk 83 If it exists, is expected to do things like conditioning the 84 environment. Since it will only be included by the initial 85 instance of bmake, it should ``.export`` anything that 86 sub-makes might need. 87 88examples/sys.clean-env.mk 89 An example of how to clean the environment. 90 See the file for all the details:: 91 92 .if ${MAKE_VERSION} >= 20100606 && ${.MAKE.LEVEL} == 0 93 # we save any env var that starts with these 94 MAKE_SAVE_ENV_PREFIX += SB MK MAKE MACHINE NEED_ CCACHE DISTCC USE_ SSH 95 MAKE_SAVE_ENV_VARS += \ 96 PATH HOME USER LOGNAME \ 97 SRCTOP OBJTOP OBJROOT \ 98 ${_env_vars} 99 100 _env_vars != env | egrep '^(${MAKE_SAVE_ENV_PREFIX:ts|})' | sed 's,=.*,,'; echo 101 _export_list = 102 .for v in ${MAKE_SAVE_ENV_VARS:O:u} 103 .if !empty($v) 104 _export_list += $v 105 $v := ${$v} 106 .endif 107 .endfor 108 # now clobber the environment 109 .unexport-env 110 111 # list of vars that we handle specially below 112 _tricky_env_vars = MAKEOBJDIR 113 # export our selection - sans tricky ones 114 .export ${_export_list:${_tricky_env_vars:${M_ListToSkip}}} 115 116 # this next bit may need tweaking 117 .if defined(MAKEOBJDIR) 118 srctop := ${SRCTOP:U${SB_SRC:U${SB}/src}} 119 objroot := ${OBJROOT:U${SB_OBJROOT:U${SB}/${SB_OBJPREFIX}}} 120 # we'll take care of MACHINE below 121 objtop := ${OBJTOP:U${objroot}${MACHINE}} 122 .if !empty(objtop) 123 # we would normally want something like (/bin/sh): 124 # MAKEOBJDIR="\${.CURDIR:S,${SRCTOP},${OBJROOT}\${MACHINE},}" 125 # the $$ below is how we achieve the same result here. 126 # since everything saved from the environment above 127 # has run through := we need to compensate for ${MACHINE} 128 MAKEOBJDIR = $${.CURDIR:S,${srctop},${objtop:S,${MACHINE},\${MACHINE},},} 129 130 # export these as-is, and do not track... 131 .export-env ${_tricky_env_vars} 132 # now evaluate for ourselves 133 .for v in ${_tricky_env_vars} 134 $v := ${$v} 135 .endfor 136 137 .endif 138 .endif 139 .endif 140 141 142host-target.mk 143 Is used to set macros like ``HOST_TARGET``, ``HOST_OS`` and 144 ``host_os`` which are used to find the next step. 145 146sys/\*.mk 147 Platform specific additions, such as ``Darwin.mk`` or ``SunOS.mk`` 148 set things like ``HOST_LIBEXT = .dylib`` for Darwin or 149 ``SHLIB_FULLVERSION = ${SHLIB_MAJOR}`` for SunOS 5. 150 If there is no OS specific file, ``sys/Generic.mk`` is used. 151 152local.sys.mk 153 Any ``local.*.mk`` file is not part of the distribution. 154 This provides a hook for sites to do extra setup without 155 having to edit the distributed files. 156 157 158The above arrangement makes it easy for the mk files to be part of a 159src tree on an NFS volume and to allow building on multiple platforms. 160 161lib.mk 162------ 163 164This file is used to build a number of different libraries from the 165same SRCS. 166 167lib${LIB}.a 168 An archive lib of ``.o`` files, this is the default 169 170lib${LIB}_p.a 171 A profiled lib of ``.po`` files. 172 Still an archive lib, but all the objects are built with 173 profiling in mind - hence the different extension. 174 It is skipped if ``MKPROFILE`` is "no". 175 176lib${LIB}_pic.a 177 An archive of ``.so`` objects compiled for relocation. 178 On NetBSD this is the input to ``lib${LIB}.${LD_so}``, it is 179 skipped if ``MKPICLIB`` is "no". 180 181lib${LIB}.${LD_so} 182 A shared library. The value of ``LD_so`` is very platform 183 specific. For example:: 184 185 # SunOS 5 and most other ELF systems 186 libsslfd.so.1 187 188 # Darwin 189 libsslfd.1.dylib 190 191 This library will only be built if ``SHLIB_MAJOR`` has 192 a value, and ``MKPIC`` is not set to "no". 193 194There is a lot of platform specific tweaking in ``lib.mk``, largely the 195result of the original distributions trying to avoid interfering with 196the system's ``sys.mk``. 197 198libnames.mk 199----------- 200 201This is included by both ``prog.mk`` and ``lib.mk`` and tries to 202include ``*.libnames.mk`` of which: 203 204local.libnames.mk 205 does not exist unless you create it. It is a handy way for you 206 to customize without touching the distributed files. 207 For example, on a test machine I needed to build openssl but 208 not install it, so put the following in ``local.libnames.mk``:: 209 210 .if ${host_os} == "sunos" 211 LIBCRYPTO = ${OBJTOP}/openssl/lib/crypto/libcrypto${DLIBEXT} 212 LIBSSL = ${OBJTOP}/openssl/lib/ssl/libssl${DLIBEXT} 213 INCLUDES_libcrypto = -I${OBJ_libcrypto} 214 .endif 215 216 The makefile created an openssl dir in ``${OBJ_libcrypto}`` to 217 gather all the headers. dpadd.mk_ did the rest. 218 219sjg.libnames.mk 220 not part of the mk-files distribution. 221 222host.libnames.mk 223 contains logic to find any libs named in ``HOST_LIBS`` in 224 ``HOST_LIBDIRS``. 225 226Each file above gets an opportunity to define things like:: 227 228 LIBSSLFD ?= ${OBJTOP}/ssl/lib/sslfd/libsslfd${DLIBEXT} 229 INCLUDES_libsslfd = -I${SRC_libsslfd}/h -I${OBJ_libslfd} 230 231these are used by dpadd.mk_ and will be explained below. 232 233dpadd.mk 234-------- 235 236This file looks like line noise, and is best considered read-only. 237However it provides some very useful functionality, which simplifies the build. 238 239Makefiles can use the LIB* macros defined via libnames.mk_ or anywhere 240else in various ways:: 241 242 # indicate that we need to include headers from LIBCRYPTO 243 # this would result in ${INCLUDES_libcrypto} being added to CFLAGS. 244 SRC_LIBS += ${LIBCRYPTO} 245 246 # indicate that libsslfd must be built already. 247 # it also has the same effect as SRC_LIBS 248 DPADD += ${LIBSSLFD} 249 250 # indicate that not only must libsslfd be built, 251 # but that we need to link with it. 252 # this is almost exactly equivalent to 253 # DPADD += ${LIBSSLFD} 254 # LDADD += -L${LIBSSLFD:H} -lsslfd 255 # and mostly serves to ensure that DPADD and LDADD are in sync. 256 DPLIBS += ${LIBSSLFD} 257 258Any library (referenced by its full path) in any of the above, is 259added to ``DPMAGIC_LIBS`` with the following results, for each lib *foo*. 260 261SRC_libfoo 262 Is set to indicate where the src for libfoo is. 263 By default it is derived from ``LIBFOO`` by replacing 264 ``${OBJTOP}`` with ``${SRCTOP}``. 265 266OBJ_libfoo 267 Not very exciting, is just the dir where libfoo lives. 268 269INCLUDES_libfoo 270 What to add to ``CFLAGS`` to find the public headers. 271 The default varies. If ``${SRC_libfoo}/h`` exists, it is assumed 272 to be the home of all public headers and thus the default is 273 ``-I${SRC_libfoo}/h`` 274 275 Otherwise we make no assumptions and the default is 276 ``-I${SRC_libfoo} -I${OBJ_libfoo}`` 277 278LDADD_libfoo 279 This only applies to libs reference via ``DPLIBS``. 280 The default is ``-lfoo``, ``LDADD_*`` provides a hook to 281 instantiate other linker flags at the appropriate point 282 without losing the benfits of ``DPLIBS``. 283 284prog.mk 285------- 286 287Compiles the specified SRCS and links them and the nominated libraries 288into a program. Prog makefiles usually need to list the libraries 289that need to be linked. We prefer use of ``DPLIBS`` but the more 290traditional ``DPADD`` and ``LDADD`` work just as well. 291That is:: 292 293 DPLIBS += ${LIBCRYPTO} 294 295is equivalent to:: 296 297 DPADD += ${LIBCRYPTO} 298 LDADD += -lcrypto 299 300obj.mk 301------ 302 303One of the cool aspects of BSD make, is its support for separating 304object files from the src tree. This is also the source of much 305confusion to some. 306 307Traditionally one had to do a separate ``make obj`` pass through the 308tree. If ``MKOBJDIRS`` is "auto", we include auto.obj.mk_. 309 310auto.obj.mk 311----------- 312 313This leverages the ``.OBJDIR`` target introduced some years ago to 314NetBSD make, to automatically create the desired object dir. 315 316subdir.mk 317--------- 318 319This is the traditional means of walking the tree. A makefile sets 320``SUBDIR`` to the list of sub-dirs to visit. 321 322If ``SUBDIR_MUST_EXIST`` is set, missing directories cause an error, 323otherwise a warning is issued. If you don't even want the warning, 324set ``MISSING_DIR=continue``. 325 326Traditionally, ``subdir.mk`` prints clue as it visits each subdir:: 327 328 ===> ssl 329 ===> ssl/lib 330 ===> ssl/lib/sslfd 331 332you can suppress that - or enhance it by setting ``ECHO_DIR``:: 333 334 # suppress subdir noise 335 ECHO_DIR=: 336 # print time stamps 337 ECHO_DIR=echo @ `date "+%s [%Y-%m-%d %T] "` 338 339links.mk 340-------- 341 342Provides rules for processing lists of ``LINKS`` and ``SYMLINKS``. 343Each is expected to be a list of ``link`` and ``target`` pairs 344(``link`` -> ``target``). 345 346The logic is generally in a ``_*_SCRIPT`` which is referenced in a 347``_*_USE`` (``.USE``) target. 348 349The ``_BUILD_*`` forms are identical, but do not use ``${DESTDIR}`` 350and so are useful for creating symlinks during the build phase. 351For example:: 352 353 SYMLINKS += ${.CURDIR}/${MACHINE_ARCH}/include machine 354 header_links: _BUILD_SYMLINKS_USE 355 356 md.o: header_links 357 358would create a symlink called ``machine`` in ``${.OBJDIR}`` pointing to 359``${.CURDIR}/${MACHINE_ARCH}/include`` before compiling ``md.o`` 360 361 362autoconf.mk 363----------- 364 365Deals with running (or generating) GNU autoconf ``configure`` scripts. 366 367dep.mk 368------ 369 370Deals with collecting dependencies. Another useful feature of BSD 371make is the separation of this sort of information into a ``.depend`` 372file. ``MKDEP`` needs to point to a suitable tool (like mkdeps.sh_) 373 374If ``USE_AUTODEP_MK`` is "yes" includes autodep.mk_ 375 376autodep.mk 377---------- 378 379Leverages the ``-MD`` feature of recent GCC to collect dependency 380information as a side effect of compilation. With this GCC puts 381dependency info into a ``.d`` file. 382 383Unfortunately GCC bases the name of the ``.d`` file on the name of the 384input rather than the output file, which causes problems when the same 385source is compiled different ways. The latest GCC supports ``-MF`` to 386name the ``.d`` file and ``-MT`` to control the name to put as the 387dependent. 388 389Recent bmake allows dependencies for the ``.END`` target (run at the 390end if everything was successful), and ``autodep.mk`` uses this to 391post process the ``.d`` files into ``.depend``. 392 393auto.dep.mk 394----------- 395 396A much simpler implementation than autodep.mk_ it uses 397``-MF ${.TARGET:T}.d`` 398to avoid possible conflicts during parallel builds. 399This precludes the use of suffix rules to drive ``make depend``, so 400dep.mk_ handles that if specifically requested. 401 402own.mk 403------ 404 405Normally included by ``init.mk`` (included by ``lib.mk`` and 406``prog.mk`` etc), sets macros for default ownership etc. 407 408It includes ``${MAKECONF}`` if it is defined and exists. 409 410man.mk 411------ 412 413Deals with man pages. 414 415warnings.mk 416----------- 417 418This provides a means of fine grained control over warnings on a per 419``${MACHINE}`` or even file basis. 420 421A makefile sets ``WARNINGS_SET`` to name a list of warnings 422and individual ``W_*`` macros can be used to tweak them. 423For example:: 424 425 WARNINGS_SET = HIGH 426 W_unused_sparc = -Wno-unused 427 428would add all the warnings in ``${HIGH_WARNINGS}`` to CFLAGS, but 429on sparc, ``-Wno-unused`` would replace ``-Wunused``. 430 431You should never need to edit ``warnings.mk``, it will include 432``warnings-sets.mk`` if it exists and you use that to make any local 433customizations. 434 435rst2htm.mk 436---------- 437 438Logic to simplify generating HTML (and PDF) documents from ReStructuredText. 439 440cython.mk 441--------- 442 443Logic to build Python C interface modules using Cython_ 444 445.. _Cython: http://www.cython.org/ 446 447Meta mode 448========= 449 450The 20110505 and later versions of ``mk-files`` include a number of 451makefiles contributed by Juniper Networks, Inc. 452These allow the latest version of bmake_ to run in `meta mode`_ 453see `dirdeps.mk`_ 454 455.. _`dirdeps.mk`: /help/sjg/dirdeps.htm 456.. _`meta mode`: bmake-meta-mode.htm 457 458Install 459======= 460 461You can use the content of mk.tar.gz_ without installing at all. 462 463The script ``install-mk`` takes care of copying ``*.mk`` into a 464destination directory, and unless told not to, create ``bsd.*.mk`` links 465for ``lib.mk`` etc. 466 467If you just want to create the ``bsd.*.mk`` links in the directory 468where you unpacked the tar file, you can:: 469 470 ./mk/install-mk ./mk 471 472------ 473 474.. _bmake: bmake.htm 475.. _NetBSD: http://www.netbsd.org/ 476.. _mkdeps.sh: http://www.crufty.net/ftp/pub/sjg/mkdeps.sh 477.. _mk.tar.gz: http://www.crufty.net/ftp/pub/sjg/mk.tar.gz 478 479:Author: sjg@crufty.net 480:Revision: $Id: mk-files.txt,v 1.16 2014/09/05 04:41:16 sjg Exp $ 481:Copyright: Crufty.NET 482