1# 2# CDDL HEADER START 3# 4# The contents of this file are subject to the terms of the 5# Common Development and Distribution License (the "License"). 6# You may not use this file except in compliance with the License. 7# 8# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE 9# or http://www.opensolaris.org/os/licensing. 10# See the License for the specific language governing permissions 11# and limitations under the License. 12# 13# When distributing Covered Code, include this CDDL HEADER in each 14# file and include the License file at usr/src/OPENSOLARIS.LICENSE. 15# If applicable, add the following below this CDDL HEADER, with the 16# fields enclosed by brackets "[]" replaced with your own identifying 17# information: Portions Copyright [yyyy] [name of copyright owner] 18# 19# CDDL HEADER END 20# 21 22# 23# Copyright 2010 Sun Microsystems, Inc. All rights reserved. 24# Use is subject to license terms. 25# 26 27Introduction 28------------ 29 30This README describes some basics about creating, modifying, and 31building packages in ON. All package creation in ON is done using 32tools available to our customers. If terminology in this document 33is confusing, you may wish to review the pkg(5) manpage. 34 35Packaging Overview 36------------------ 37 38usr/src/pkg/ contains the definitions and rules needed to build 39a set of IPS repositories which contain the deliverables from an 40ON build. 41 42The manifests directory contains all manifests, and has one file 43per package. For most packaging changes you only need to add or 44change the packaging manifests themselves. 45 46The build rules create two repositories. These are both built 47for DEBUG and non-DEBUG, and are thus available at 48 $CODEMGR_WS/packages/$MACH/nightly[-nd]/repo.(extra|redist) 49 50 repo.redist 51 Contains the bulk of ON, and is what is delivered to the 52 main OpenSolaris 'dev' and 'release' repositories. All 53 components delivered there must be redistributable. 54 55 repo.extra 56 Is only built for Closed builds, and contains 57 non-redistributable (without a proper legal agreement) pieces 58 which may or may not be suitable for inclusion in the 59 pkg.sun.com/opensolaris/extra repository, including the 60 SUNWcryptoint bits necessary for working crypto in a Closed 61 build. Do not distribute any of the bits in repo.extra 62 without prior agreement from the appropriate lawyers. 63 64Building Packages 65----------------- 66 67The -p option to nightly will build the IPS repositories. 68 69Alternatively, in usr/src/pkg/Makefile there are make targets for: 70 71 all 72 Process manifests into their final form with unresolved 73 dependencies before publication. 74 75 install 76 Publish packages. 77 78 gendeps 79 Run `pkgdepend resolve`. See Dependencies section. 80 81 protocmp 82 Compare the proto area against package definitions for 83 missing or incorrect files. 84 85 pmodes 86 Check file and directory modes for best practices. 87 88 check 89 Run protocmp and pmodes. 90 91The build behavior may modified by the following variables: 92 93 CLOSED_IS_PRESENT 94 If CLOSED_IS_PRESENT is set to "yes," repo.extra will be built. 95 96 ON_CRYPTO_BINS or CODESIGN_USER 97 If ON_CRYPTO_BINS or CODESIGN_USER is set in your build env, 98 no packages will depend on the internal crypto certificates. 99 If neither is set, your bits depend on the internal crypto 100 certificates being available and packages will depend on 101 pkg:/driver/crypto/dprov, which is only available in the 102 on-extra repository. 103 104 SUPPRESSPKGDEP 105 If SUPPRESSPKGDEP is set to "true" in your environment, 106 package dependencies will not be generated. This variable 107 should not be set in normal builds as it will mask product 108 bugs. 109 110 PKGDEBUG 111 If PKGDEBUG is set in your environment, $MAKE will print 112 detailed information about the commands it executes to 113 process and publish packages. 114 115 ONNV_BUILDNUM 116 If ONNV_BUILDNUM is set to an older ON build number, 117 your packages will be published at that version instead 118 of the one defined in usr/src/Makefile.buildnum, which 119 is managed by the gatekeepers. 120 121A set of intermediate build products are placed in 122usr/src/pkg/packages.$MACH/. These can be useful during development. 123 124 .mog 125 The resulting package manifest after running pkgmogrify(1) 126 on the supplied manifest. See below for details on 127 pkgmogrify(1) use in ON. 128 129 .dep 130 The resulting manifest after running `pkgdepend generate` 131 on the .mog manifest. 132 133 .res 134 The resulting manifest after running `pkgdepend resolve` 135 on the .dep manifest. 136 137Incremental Builds 138------------------ 139 140 If you want to process a subset of manifests, simply set PKGS on the 141 make command line and specify the "all" target. If you want to process 142 them all, simply specify the "all" target. 143 144 % dmake -e PKGS="BRCMbnx BRCMbnxe" all 145 % dmake -e all 146 147 If you want to publish a subset of packages, simply set PKGS on the 148 make command line and specify the "install" target. Overriding PKGS 149 will cause dependency resolution to be limited to PKGS from the 150 current build, with a fallback to packages installed on the build 151 system. If you want to publish them all, simply specify the 152 "install" target. 153 154 % dmake -e PKGS="BRCMbnx BRCMbnxe" install 155 % dmake -e install 156 157 You can also use package names, or package names with ".pub" 158 extensions, as build targets. This will cause processing or 159 publication of the specified package(s). 160 161 The on-disk repository will be initialized when it does not exist, 162 or when you run nightly -p. If you build incrementally, 163 packages will simply be added to the existing repository. 164 165 To do cross-platform packaging, prefix your target with (for 166 example) "sparc/", as in "dmake sparc/install". Note that we 167 currently pull some license files directly from a built source 168 tree, so if you do this in a workspace that had proto area copied 169 in via nightly -U, then you'll need to set $SRC to point to the 170 workspace that was actually built. 171 172Testing Packages 173---------------- 174 175To test the generated repositories, you should use the "onu" tool 176available from /opt/onbld/bin or usr/src/tools/scripts/ to setup and 177upgrade your system. A manpage is available in /opt/onbld/man 178or usr/src/tools/scripts/onu.1. 179 180Alternatively, you can manually start a pkg.depot(1M) server to 181serve the generated repositories to multiple test machines. 182 183 Start up a depot on your build machine. 184 cd $CODEMGR_WS/packages/$MACH/nightly[-nd] 185 /usr/lib/pkg.depotd -d repo.redist -p <port> & 186 187 Make SURE you choose an unused port and the depot 188 starts successfully. 189 190 The depot can be started across NFS as well if you 191 have a fast pipe. 192 193 Configure your test system. 194 pkg set-publisher -P -g http://<your server host>:<port> on-nightly 195 pkg set-publisher --non-sticky opensolaris.org 196 pkg uninstall entire 197 198 pkg image-update your test system. 199 pkg image-update will upgrade all packages on your test system 200 201Always make sure your bits are installed with image-update. 202 Check your versions. 203 pkg info osnet-incorporation 204 205 Multiple packages should be updated. 206 If you did a full build, all ON packages will be updated. 207 When image-update tells you that only one or two packages has 208 been updated, you likely did not get the updates you expected. 209 210There are various tactics for troubleshooting a failed upgrade. 211 Make sure entire is uninstalled. 212 213 pkg install -nv osnet-incorporation@<your version> 214 Ask IPS to explicitly check if ON can be installed, and 215 if it can't, tell you why not. 216 217 Obsolete and renamed packages can cause trouble. 218 pkg search -l ::pkg.renamed:true 219 pkg search -l ::pkg.obsolete:true 220 221Making Packaging Changes 222------------------------ 223 224Package definitions are in usr/src/pkg/manifests/, and have one 225file per package, including for multi-architecture packages. For 226most packaging changes you only need to add or change the packaging 227manifests themselves. No packaging metadata may be kept outside of the 228manifests. If you find yourself needing to modify usr/src/pkg/Makefile, 229you're almost certainly doing something wrong. 230 231Remember that the "check" target is available to check many of 232your packaging changes. 233 234We run pkgmogrify(1) over the manifests before publication. This 235allows us to apply a set of macros and package transformations in 236order to make the manifests themselves easier to maintain. 237 238We supply a set of commonly-used macros for use in package manifests. 239These are the PKGMOG_DEFINES in usr/src/pkg/Makefile. 240 241 $(i386_ONLY) 242 $(sparc_ONLY) 243 $(ARCH) 244 $(ARCH32) 245 $(ARCH64) 246 $(PKGVERS), which is set to 247 $(PKGVERS_COMPONENT),$(PKGVERS_BUILTON)-0.$(PKGVERS_BRANCH) 248 249pkgmogrify(1) also allows us to include a set of default transformations. 250The definitions for these transforms are in usr/src/pkg/transforms/. An 251overview of their use is supplied here, but for a full accounting, please 252read pkmogrify(1) and the files themselves. 253 254 defaults 255 This transform is applied to all manifests. It specifies 256 a set of sensible default permissions, a set of 257 directory locations for which the reboot-needed actuator 258 is always applied to files, and some other basic defaults. 259 260 publish 261 This transform is applied to all manifests. It ensures 262 that manifest lines which don't apply to the current 263 architecture are stripped. It also ensures non-redistributable 264 packages aren't included in an open-only build. 265 266 restart_fmri 267 This transform is applied to all manifests. It modifies 268 all package manifest lines for SMF manifests in standard 269 locations to include an actuator which runs manifest-import 270 on installation/update/removal, as well as some others. If 271 you're adding a new class of file that would benefit from 272 a restart or refresh of a specific SMF service, please add 273 it here. 274 275 extract_metadata 276 This transform is applied to all manifests. It deals with 277 manipulations required for packaging metadata like 278 org.opensolaris.redist, pkg.renamed, and pkg.obsolete. 279 280 hollow_zone_pkg 281 This transform is available for explicit inclusion in 282 some manifests. It ensures that all contents of the 283 package are not installed within a non-global zone, but the 284 package and its metadata are available in order to satisfy 285 packaging dependencies. 286 287pkgmogrify(1) also allows us to use comments and continuation lines 288in our manifests. 289 290Zones and Packages 291------------------ 292 293pkg(5) uses variants to implement zones. If a package is marked 294with both global and non-global zone variants, the package contents will 295be installed in both global and non-global by default. 296 set name=variant.opensolaris.zone value=global value=nonglobal 297 298Specific actions within a package can be tagged as applying to only 299the global zone or only the non-global zones. 300 301The hollow_zone_pkg transform described above is also available to 302simplify a common packaging scenario. 303 304Dependencies 305------------ 306 307Package dependencies are automatically calculated during build time 308using pkgdepend(1). After you've done a build, you can verify your 309dependencies in the <package>.res file described above. If the 310file is missing a dependency that you believe could be auto-detected, 311please file a bug against pkgdepend(1). 312 313Dependencies can be added manually using the "depend" action. Please 314add a comment describing why the dependency is required. 315 316Moving Content Between Packages and Removing Content 317---------------------------------------------------- 318 319pkg(5) tracks when content is removed from packages, and automatically 320removes it. 321 322If you need to move content between packages, there are two primary 323things to do. 324 325 "preserve" files must be marked with original_name. 326 The first time a "preserve" file moves between packages, 327 you must set original_name=<original package>:<file> 328 in that file's action. Subsequent moves do not require 329 modification. 330 331 Consider adding a dependency on the new package. 332 The only way a system will end up with a new package 333 after upgrade is if an existing package depends on it. 334 335Renaming a Package 336------------------ 337 338To rename a package, leave the old package manifest in place, but 339empty it of all delivered content. The old package should include: 340 341 set name=pkg.fmri with the version set explicitly to the 342 build you're integrating into. For example, if you wanted 343 to rename SUNWrmodu in build 133 you'd change the pkg.fmri 344 line to read 345 set name=pkg.fmri value=pkg:/SUNWrmodu@0.5.11,5.11-0.133 346 347 set name=pkg.renamed value=true 348 349 The architectures and variants you're renaming. These 350 should just be copied from your old package, as you 351 must rename a package on all architectures and 352 variants simultaneously. 353 354 A dependency on the new package. 355 356If there were "preserve" files in the package you're renaming, make 357sure to create original_name settings in the new package. 358 359If there was a org.opensolaris.noincorp property in the package being 360renamed, make sure you keep it in both the original and the renamed 361packages. 362 363EOFs and Removals 364----------------- 365 366To remove files, directories, drivers, or anything else within a package, 367simply stop delivering them in the package. IPS will manage the removal 368of no longer delivered content. 369 370Package removals have impact on the rest of the system. Before marking 371a package as obsolete, search in the OpenSolaris development 372repository (http://pkg.opensolaris.org/dev or http://ipkg.sfbay/dev) 373to find out if any other packages depend on the software you intend 374to EOF. If any packages do, you need to coordinate with those 375consolidations. 376 377The "slim_install" package may depend on ON packages. If it does, 378you must send a FLAG DAY message for ON users and PIT who test 379install. You must also file an installation bug to get that package 380updated in the same build or earlier than you intend to integrate. 381You should also test install yourself. You can do this by replacing 382the "slim_install" in your Distro Constructor manifest with the 383explicit list of packages to install. 384 385To remove a package, you must mark it as obsolete. The obsoleted 386package manifest should include 387 388 set name=pkg.fmri with the version set explicitly to the 389 build you're integrating into. For example, if you wanted 390 to remove SUNWwbsd in build 133 you'd change the pkg.fmri 391 line to read 392 set name=pkg.fmri value=pkg:/SUNWwbsd@0.5.11,5.11-0.133 393 394 set name=pkg.obsolete value=true 395 396 The architectures and variants you're obsoleting. These 397 should just be copied from your old package, as you 398 must obsolete a package on all architectures and 399 variants simultaneously. 400 401Creating a Package 402------------------ 403 404The easiest thing is to copy a package similar to the one you're 405trying to create. Note that packages are no longer split on the 406/usr and / boundary. 407 408The following actions are required for all packages in ON. 409 set name=pkg.fmri 410 Every package must have an FMRI. That is the package's 411 name. 412 413 set name=pkg.summary 414 Every package must have a short summary of its purpose. 415 416 set name=pkg.description 417 Every package must have a one-sentence description of 418 its purpose. 419 420 set name=variant.arch 421 Every package must specify which architectures it delivers. 422 423 set name=variant.opensolaris.zone 424 Every package must specify whether it can be installed in 425 global zones, non-global zones, or both. 426 427 set name=info.classification 428 Every package must specify a category for the packaging GUI. 429 You must use an existing category, and may not invent new ones. 430 Existing categories and their subcategories are listed 431 in /usr/share/package-manager/data/opensolaris.org.sections. 432 433 license 434 All packages must specify a set of license actions. If 435 you're adding items here that are not already included in 436 usr/src/pkg/license_files, then you will also need to modify 437 usr/src/tools/opensolaris/license-list. 438 439The following actions are uncommon but specific to ON. 440 441 set name=org.opensolaris.redist 442 This may be set to nonredist or internal. If a package 443 is redistributable, do not create this action. "internal" 444 packages which are legally not allowed to be distributed 445 at all are strongly discouraged. If you're adding 446 content to a package with this action, you should have 447 modified bindrop.sh, and test open-only builds. 448 449Drivers and Packages 450-------------------- 451 452Scripting is no longer required to deal with addition or removal of 453drivers in ON. A "driver" action should be specified for each driver, 454and IPS will handle updates to all the driver files. 455