1# 2# CDDL HEADER START 3# 4# The contents of this file are subject to the terms of the 5# Common Development and Distribution License (the "License"). 6# You may not use this file except in compliance with the License. 7# 8# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE 9# or http://www.opensolaris.org/os/licensing. 10# See the License for the specific language governing permissions 11# and limitations under the License. 12# 13# When distributing Covered Code, include this CDDL HEADER in each 14# file and include the License file at usr/src/OPENSOLARIS.LICENSE. 15# If applicable, add the following below this CDDL HEADER, with the 16# fields enclosed by brackets "[]" replaced with your own identifying 17# information: Portions Copyright [yyyy] [name of copyright owner] 18# 19# CDDL HEADER END 20# 21 22# 23# Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 24# 25 26Introduction 27------------ 28 29This README describes some basics about creating, modifying, and 30building packages in ON. All package creation in ON is done using 31tools available to our customers. If terminology in this document 32is confusing, you may wish to review the pkg(5) manpage. 33 34Packaging Overview 35------------------ 36 37usr/src/pkg/ contains the definitions and rules needed to build 38a set of IPS repositories which contain the deliverables from an 39ON build. 40 41The manifests directory contains all manifests, and has one file 42per package. For most packaging changes you only need to add or 43change the packaging manifests themselves. 44 45The build rules create two repositories. These are both built 46for DEBUG and non-DEBUG, and are thus available at 47 $CODEMGR_WS/packages/$MACH/nightly[-nd]/repo.(extra|redist) 48 49 repo.redist 50 Contains the bulk of ON, and is what is delivered to the 51 main OpenSolaris 'dev' and 'release' repositories. All 52 components delivered there must be redistributable. 53 54 repo.extra 55 Is only built for Closed builds, and contains 56 non-redistributable (without a proper legal agreement) pieces 57 which may or may not be suitable for inclusion in the 58 pkg.sun.com/opensolaris/extra repository, including the 59 internal crypto bits necessary for working crypto in a Closed 60 build. Do not distribute any of the bits in repo.extra 61 without prior agreement from the appropriate lawyers. 62 63Building Packages 64----------------- 65 66The -p option to nightly will build the IPS repositories. 67 68Alternatively, in usr/src/pkg/Makefile there are make targets for: 69 70 all 71 Process manifests into their final form with unresolved 72 dependencies before publication. 73 74 install 75 Publish packages. 76 77 gendeps 78 Run `pkgdepend resolve`. See Dependencies section. 79 80 protocmp 81 Compare the proto area against package definitions for 82 missing or incorrect files. 83 84 pmodes 85 Check file and directory modes for best practices. 86 87 check 88 Run protocmp and pmodes. 89 90The build behavior may modified by the following variables: 91 92 CLOSED_IS_PRESENT 93 If CLOSED_IS_PRESENT is set to "yes," repo.extra will be built. 94 95 ON_CRYPTO_BINS or CODESIGN_USER 96 If ON_CRYPTO_BINS or CODESIGN_USER is set in your build env, 97 no packages will depend on the internal crypto certificates. 98 99 If neither is set, your bits depend on the internal crypto 100 certificates being available and packages will depend on 101 pkg:/driver/crypto/dprov, which is only available in the 102 on-extra repository. 103 104 *** Important *** This means that, if you build using 105 internal crypto, but you try to do an image-update with 106 only repo.redist, you will be told that there are no 107 updates available. 108 109 SUPPRESSPKGDEP 110 If SUPPRESSPKGDEP is set to "true" in your environment, 111 package dependencies will not be generated. This variable 112 should not be set in normal builds as it will mask product 113 bugs. 114 115 PKGDEBUG 116 If PKGDEBUG is set in your environment, $MAKE will print 117 detailed information about the commands it executes to 118 process and publish packages. 119 120 ONNV_BUILDNUM 121 If ONNV_BUILDNUM is set to an older ON build number, 122 your packages will be published at that version instead 123 of the one defined in usr/src/Makefile.buildnum, which 124 is managed by the gatekeepers. 125 126A set of intermediate build products are placed in 127usr/src/pkg/packages.$MACH/. These can be useful during development. 128 129 .mog 130 The resulting package manifest after running pkgmogrify(1) 131 on the supplied manifest. See below for details on 132 pkgmogrify(1) use in ON. 133 134 .dep 135 The resulting manifest after running `pkgdepend generate` 136 on the .mog manifest. 137 138 .res 139 The resulting manifest after running `pkgdepend resolve` 140 on the .dep manifest. 141 142Incremental Builds 143------------------ 144 145 If you want to process a subset of manifests, simply set PKGS on the 146 make command line and specify the "all" target. If you want to process 147 them all, simply specify the "all" target. 148 149 % dmake -e PKGS="BRCMbnx BRCMbnxe" all 150 % dmake -e all 151 152 If you want to publish a subset of packages, simply set PKGS on the 153 make command line and specify the "install" target. Overriding PKGS 154 will cause dependency resolution to be limited to PKGS from the 155 current build, with a fallback to packages installed on the build 156 system. If you want to publish them all, simply specify the 157 "install" target. 158 159 % dmake -e PKGS="BRCMbnx BRCMbnxe" install 160 % dmake -e install 161 162 You can also use package names, or package names with ".pub" 163 extensions, as build targets. This will cause processing or 164 publication of the specified package(s). 165 166 The on-disk repository will be initialized when it does not exist, 167 or when you run nightly -p. If you build incrementally, 168 packages will simply be added to the existing repository. 169 170 To do cross-platform packaging, prefix your target with (for 171 example) "sparc/", as in "dmake sparc/install". Note that we 172 currently pull some license files directly from a built source 173 tree, so if you do this in a workspace that had proto area copied 174 in via nightly -U, then you'll need to set $SRC to point to the 175 workspace that was actually built. 176 177Testing Packages 178---------------- 179 180To test the generated repositories, you should use the "onu" tool 181available from /opt/onbld/bin or usr/src/tools/scripts/ to setup and 182upgrade your system. A manpage is available in /opt/onbld/man 183or usr/src/tools/scripts/onu.1. 184 185Alternatively, you can manually start a pkg.depot(1M) server to 186serve the generated repositories to multiple test machines. 187 188 Start up a depot on your build machine. 189 cd $CODEMGR_WS/packages/$MACH/nightly[-nd] 190 /usr/lib/pkg.depotd -d repo.redist -p <port> & 191 192 Make SURE you choose an unused port and the depot 193 starts successfully. 194 195 The depot can be started across NFS as well if you 196 have a fast pipe. 197 198 If you used internal crypto in your builds, then you must 199 do this step for both repo.redist and repo.extra, picking 200 different ports for each. Otherwise, you will be unable to 201 image-update. 202 203 Configure your test system. 204 pkg set-publisher -P -g http://<your server host>:<port> on-nightly 205 pkg set-publisher --non-sticky opensolaris.org 206 pkg uninstall entire 207 208 If you used internal crypto in your builds, then you must also: 209 pkg set-publisher -P \ 210 -g http://<your server host>:<extra-port> on-extra 211 212 pkg image-update your test system. 213 pkg image-update will upgrade all packages on your test system 214 215Always make sure your bits are installed with image-update. 216 Check your versions. 217 pkg info osnet-incorporation 218 219 Multiple packages should be updated. 220 If you did a full build, all ON packages will be updated. 221 When image-update tells you that only one or two packages has 222 been updated, you likely did not get the updates you expected. 223 224There are various tactics for troubleshooting a failed upgrade. 225 Make sure entire is uninstalled. 226 227 pkg install -nv osnet-incorporation@<your version> 228 Ask IPS to explicitly check if ON can be installed, and 229 if it can't, tell you why not. 230 231 Obsolete and renamed packages can cause trouble. 232 pkg search -l ::pkg.renamed:true 233 pkg search -l ::pkg.obsolete:true 234 235 Check to see if you used internal crypto, but failed to provide 236 a server for repo.extra. 237 Use a web browser to view the system/kernel manifest from 238 your on-nightly publisher and look for a dependency on the 239 driver/crypto/dprov package. 240 241 242Making Packaging Changes 243------------------------ 244 245Package definitions are in usr/src/pkg/manifests/, and have one 246file per package, including for multi-architecture packages. For 247most packaging changes you only need to add or change the packaging 248manifests themselves. No packaging metadata may be kept outside of the 249manifests. If you find yourself needing to modify usr/src/pkg/Makefile, 250you're almost certainly doing something wrong. 251 252Remember that the "check" target is available to check many of 253your packaging changes. 254 255We run pkgmogrify(1) over the manifests before publication. This 256allows us to apply a set of macros and package transformations in 257order to make the manifests themselves easier to maintain. 258 259We supply a set of commonly-used macros for use in package manifests. 260These are the PKGMOG_DEFINES in usr/src/pkg/Makefile. 261 262 $(i386_ONLY) 263 $(sparc_ONLY) 264 $(ARCH) 265 $(ARCH32) 266 $(ARCH64) 267 $(PKGVERS), which is set to 268 $(PKGVERS_COMPONENT),$(PKGVERS_BUILTON)-0.$(PKGVERS_BRANCH) 269 270pkgmogrify(1) also allows us to include a set of default transformations. 271The definitions for these transforms are in usr/src/pkg/transforms/. An 272overview of their use is supplied here, but for a full accounting, please 273read pkmogrify(1) and the files themselves. 274 275 defaults 276 This transform is applied to all manifests. It specifies 277 a set of sensible default permissions, a set of 278 directory locations for which the reboot-needed actuator 279 is always applied to files, and some other basic defaults. 280 281 publish 282 This transform is applied to all manifests. It ensures 283 that manifest lines which don't apply to the current 284 architecture are stripped. It also ensures non-redistributable 285 packages aren't included in an open-only build. 286 287 restart_fmri 288 This transform is applied to all manifests. It modifies 289 all package manifest lines for SMF manifests in standard 290 locations to include an actuator which runs manifest-import 291 on installation/update/removal, as well as some others. If 292 you're adding a new class of file that would benefit from 293 a restart or refresh of a specific SMF service, please add 294 it here. 295 296 extract_metadata 297 This transform is applied to all manifests. It deals with 298 manipulations required for packaging metadata like 299 org.opensolaris.redist, pkg.renamed, and pkg.obsolete. 300 301 hollow_zone_pkg 302 This transform is available for explicit inclusion in 303 some manifests. It ensures that all contents of the 304 package are not installed within a non-global zone, but the 305 package and its metadata are available in order to satisfy 306 packaging dependencies. 307 308pkgmogrify(1) also allows us to use comments and continuation lines 309in our manifests. 310 311Zones and Packages 312------------------ 313 314pkg(5) uses variants to implement zones. If a package is marked 315with both global and non-global zone variants, the package contents will 316be installed in both global and non-global by default. 317 set name=variant.opensolaris.zone value=global value=nonglobal 318 319Specific actions within a package can be tagged as applying to only 320the global zone or only the non-global zones. 321 322The hollow_zone_pkg transform described above is also available to 323simplify a common packaging scenario. 324 325Dependencies 326------------ 327 328Package dependencies are automatically calculated during build time 329using pkgdepend(1). After you've done a build, you can verify your 330dependencies in the <package>.res file described above. If the 331file is missing a dependency that you believe could be auto-detected, 332please file a bug against pkgdepend(1). 333 334Dependencies can be added manually using the "depend" action. Please 335add a comment describing why the dependency is required. 336 337Moving Content Between Packages and Removing Content 338---------------------------------------------------- 339 340pkg(5) tracks when content is removed from packages, and automatically 341removes it. 342 343If you need to move content between packages, there are two primary 344things to do. 345 346 "preserve" files must be marked with original_name. 347 The first time a "preserve" file moves between packages, 348 you must set original_name=<original package>:<file> 349 in that file's action. Subsequent moves do not require 350 modification. 351 352 Consider adding a dependency on the new package. 353 The only way a system will end up with a new package 354 after upgrade is if an existing package depends on it. 355 356Renaming a Package 357------------------ 358 359To rename a package, leave the old package manifest in place, but 360empty it of all delivered content. The old package should include: 361 362 set name=pkg.fmri with the version set explicitly to the 363 build you're integrating into. For example, if you wanted 364 to rename SUNWrmodu in build 133 you'd change the pkg.fmri 365 line to read 366 set name=pkg.fmri value=pkg:/SUNWrmodu@0.5.11,5.11-0.133 367 368 set name=pkg.renamed value=true 369 370 The architectures and variants you're renaming. These 371 should just be copied from your old package, as you 372 must rename a package on all architectures and 373 variants simultaneously. 374 375 A dependency on the new package. 376 377If there were "preserve" files in the package you're renaming, make 378sure to create original_name settings in the new package. 379 380If there was a org.opensolaris.noincorp property in the package being 381renamed, make sure you keep it in both the original and the renamed 382packages. 383 384EOFs and Removals 385----------------- 386 387To remove files, directories, drivers, or anything else within a package, 388simply stop delivering them in the package. IPS will manage the removal 389of no longer delivered content. 390 391Package removals have impact on the rest of the system. Before marking 392a package as obsolete, search in the OpenSolaris development 393repository (http://pkg.opensolaris.org/dev or http://ipkg.sfbay/dev) 394to find out if any other packages depend on the software you intend 395to EOF. If any packages do, you need to coordinate with those 396consolidations. 397 398The "slim_install" package may depend on ON packages. If it does, 399you must send a FLAG DAY message for ON users and PIT who test 400install. You must also file an installation bug to get that package 401updated in the same build or earlier than you intend to integrate. 402You should also test install yourself. You can do this by replacing 403the "slim_install" in your Distro Constructor manifest with the 404explicit list of packages to install. 405 406To remove a package, you must mark it as obsolete. You must *also* mark 407as obsolete any packages which are renamed ancestors of this package, and 408remove their rename dependencies. Here is what you must do. 409 410To find rename ancestors, select all of the manifests which are renames, 411then look for the one which was renamed to the package you care about. 412For example, to find rename ancestors of 'system/zones', do the following: 413 414 $ cd usr/src/pkg/manifests 415 $ mypkgname=system/zones 416 $ grep -l "fmri=pkg:/$mypkgname@" `grep -l pkg.renamed *.mf` /dev/null 417 SUNWzone.mf 418 419Make sure to check that the package has not undergone multiple renames! 420 421 $ mypkgname=SUNWzone 422 $ grep -l "fmri=pkg:/$mypkgname@" `grep -l pkg.renamed *.mf` /dev/null 423 $ 424 425Once you have the renamed ancestor list, for *each* of the packages (the 426newly obsolete package, and its renamed ancestors), edit the package as 427follows: 428 429 Update 'set name=pkg.fmri' with the version set explicitly to the 430 build you're integrating into. For example, if you wanted 431 to remove SUNWwbsd in build 133 you'd change the pkg.fmri 432 line to read: 433 'set name=pkg.fmri value=pkg:/SUNWwbsd@0.5.11,5.11-0.133' 434 435 Add 'set name=pkg.obsolete value=true'. 436 437 Maintain the architecture and variant declarations in the 438 package you are obsoleting. Note that you must obsolete a 439 package on all architectures and variants simultaneously. 440 441 Maintain 'set name=org.opensolaris.redist' actions if present. 442 443 Delete everything else. 444 445 If the package is a renamed ancestor, leave a comment behind as 446 follows: 447 448 # Was renamed to <other-pkg-name>, both now obsolete. 449 450Here is a complete example. SUNWfoobar was a package which was renamed 451to system/foobar in build 140, and then later obsoleted in build 150. 452Note that in all cases the package FMRI is updated to the obsoletion 453build, 150. 454 455 SUNWfoobar.mf: 456 # Was renamed to system/foobar, both now obsolete. 457 set name=pkg.fmri value=pkg:/SUNWfoobar@0.5.11,5.11-0.150 458 set name=pkg.obsolete value=true 459 set name=variant.arch value=$(ARCH) 460 461 system-foobar.mf: 462 set name=pkg.fmri value=pkg:/system/foobar@0.5.11,5.11-0.150 463 set name=pkg.obsolete value=true 464 set name=variant.arch value=$(ARCH) 465 466Creating a Package 467------------------ 468 469The easiest thing is to copy a package similar to the one you're 470trying to create. Note that packages are no longer split on the 471/usr and / boundary. 472 473The following actions are required for all packages in ON. 474 set name=pkg.fmri 475 Every package must have an FMRI. That is the package's 476 name. 477 478 set name=pkg.summary 479 Every package must have a short summary of its purpose. 480 481 set name=pkg.description 482 Every package must have a one-sentence description of 483 its purpose. 484 485 set name=variant.arch 486 Every package must specify which architectures it delivers. 487 488 set name=info.classification 489 Every package must specify a category for the packaging GUI. 490 You must use an existing category, and may not invent new ones. 491 Existing categories and their subcategories are listed 492 in /usr/share/package-manager/data/opensolaris.org.sections. 493 494 license 495 All packages must specify a set of license actions. If 496 you're adding items here that are not already included in 497 usr/src/pkg/license_files, then you will also need to modify 498 usr/src/tools/opensolaris/license-list. 499 500The following actions are uncommon but specific to ON. 501 502 set name=org.opensolaris.redist 503 This may be set to nonredist or internal. If a package 504 is redistributable, do not create this action. "internal" 505 packages which are legally not allowed to be distributed 506 at all are strongly discouraged. If you're adding 507 content to a package with this action, you should have 508 modified bindrop.sh, and test open-only builds. 509 510You don't need to set the following. They're taken care of for all OS/Net 511packages in the transforms/common_actions file. 512 513 set name=variant.opensolaris.zone 514 Every package must specify whether it can be installed in 515 global zones, non-global zones, or both. All ON packages are 516 delivered in both global and non-global zones. 517 518 set name=org.opensolaris.consolidation value=osnet 519 All packages from OS/Net come from OS/Net... 520 521Drivers and Packages 522-------------------- 523 524Scripting is no longer required to deal with addition or removal of 525drivers in ON. A "driver" action should be specified for each driver, 526and IPS will handle updates to all the driver files. 527