1# 2# CDDL HEADER START 3# 4# The contents of this file are subject to the terms of the 5# Common Development and Distribution License (the "License"). 6# You may not use this file except in compliance with the License. 7# 8# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE 9# or http://www.opensolaris.org/os/licensing. 10# See the License for the specific language governing permissions 11# and limitations under the License. 12# 13# When distributing Covered Code, include this CDDL HEADER in each 14# file and include the License file at usr/src/OPENSOLARIS.LICENSE. 15# If applicable, add the following below this CDDL HEADER, with the 16# fields enclosed by brackets "[]" replaced with your own identifying 17# information: Portions Copyright [yyyy] [name of copyright owner] 18# 19# CDDL HEADER END 20# 21 22# 23# Copyright 2010 Sun Microsystems, Inc. All rights reserved. 24# Use is subject to license terms. 25# 26 27Introduction 28------------ 29 30This README describes some basics about creating, modifying, and 31building packages in ON. All package creation in ON is done using 32tools available to our customers. If terminology in this document 33is confusing, you may wish to review the pkg(5) manpage. 34 35Packaging Overview 36------------------ 37 38usr/src/pkg/ contains the definitions and rules needed to build 39a set of IPS repositories which contain the deliverables from an 40ON build. 41 42The manifests directory contains all manifests, and has one file 43per package. For most packaging changes you only need to add or 44change the packaging manifests themselves. 45 46The build rules create two repositories. These are both built 47for DEBUG and non-DEBUG, and are thus available at 48 $CODEMGR_WS/packages/$MACH/nightly[-nd]/repo.(extra|redist) 49 50 repo.redist 51 Contains the bulk of ON, and is what is delivered to the 52 main OpenSolaris 'dev' and 'release' repositories. All 53 components delivered there must be redistributable. 54 55 repo.extra 56 Is only built for Closed builds, and contains 57 non-redistributable (without a proper legal agreement) pieces 58 which may or may not be suitable for inclusion in the 59 pkg.sun.com/opensolaris/extra repository, including the 60 SUNWcryptoint bits necessary for working crypto in a Closed 61 build. Do not distribute any of the bits in repo.extra 62 without prior agreement from the appropriate lawyers. 63 64Building Packages 65----------------- 66 67The -p option to nightly will build the IPS repositories. 68 69Alternatively, in usr/src/pkg/Makefile there are make targets for: 70 71 all 72 Process manifests into their final form with unresolved 73 dependencies before publication. 74 75 install 76 Publish packages. 77 78 gendeps 79 Run `pkgdepend resolve`. See Dependencies section. 80 81 protocmp 82 Compare the proto area against package definitions for 83 missing or incorrect files. 84 85 pmodes 86 Check file and directory modes for best practices. 87 88 check 89 Run protocmp and pmodes. 90 91The build behavior may modified by the following variables: 92 93 CLOSED_IS_PRESENT 94 If CLOSED_IS_PRESENT is set to "yes," repo.extra will be built. 95 96 ON_CRYPTO_BINS or CODESIGN_USER 97 If ON_CRYPTO_BINS or CODESIGN_USER is set in your build env, 98 no packages will depend on the internal crypto certificates. 99 If neither is set, your bits depend on the internal crypto 100 certificates being available and packages will depend on 101 pkg:/driver/crypto/dprov, which is only available in the 102 on-extra repository. 103 104 SUPPRESSPKGDEP 105 If SUPPRESSPKGDEP is set to "true" in your environment, 106 package dependencies will not be generated. This variable 107 should not be set in normal builds as it will mask product 108 bugs. 109 110 PKGDEBUG 111 If PKGDEBUG is set in your environment, $MAKE will print 112 detailed information about the commands it executes to 113 process and publish packages. 114 115 ONNV_BUILDNUM 116 If ONNV_BUILDNUM is set to an older ON build number, 117 your packages will be published at that version instead 118 of the one defined in usr/src/Makefile.buildnum, which 119 is managed by the gatekeepers. 120 121A set of intermediate build products are placed in 122usr/src/pkg/packages.$MACH/. These can be useful during development. 123 124 .mog 125 The resulting package manifest after running pkgmogrify(1) 126 on the supplied manifest. See below for details on 127 pkgmogrify(1) use in ON. 128 129 .dep 130 The resulting manifest after running `pkgdepend generate` 131 on the .mog manifest. 132 133 .res 134 The resulting manifest after running `pkgdepend resolve` 135 on the .dep manifest. 136 137Incremental Builds 138------------------ 139 140 If you want to process a subset of manifests, simply set PKGS on the 141 make command line and specify the "all" target. If you want to process 142 them all, simply specify the "all" target. 143 144 % dmake -e PKGS="BRCMbnx BRCMbnxe" all 145 % dmake -e all 146 147 If you want to publish a subset of packages, simply set PKGS on the 148 make command line and specify the "install" target. Overriding PKGS 149 will cause dependency resolution to be limited to PKGS from the 150 current build, with a fallback to packages installed on the build 151 system. If you want to publish them all, simply specify the 152 "install" target. 153 154 % dmake -e PKGS="BRCMbnx BRCMbnxe" install 155 % dmake -e install 156 157 You can also use package names, or package names with ".pub" 158 extensions, as build targets. This will cause processing or 159 publication of the specified package(s). 160 161 The on-disk repository will be initialized when it does not exist, 162 or when you run nightly -p. If you build incrementally, 163 packages will simply be added to the existing repository. 164 165 To do cross-platform packaging, prefix your target with (for 166 example) "sparc/", as in "dmake sparc/install". Note that we 167 currently pull some license files directly from a built source 168 tree, so if you do this in a workspace that had proto area copied 169 in via nightly -U, then you'll need to set $SRC to point to the 170 workspace that was actually built. 171 172Testing Packages 173---------------- 174 175To test the generated repositories, you should use the "onu" tool 176available from /opt/onbld/bin or usr/src/tools/scripts/ to setup and 177upgrade your system. A manpage is available in /opt/onbld/man 178or usr/src/tools/scripts/onu.1. 179 180Alternatively, you can manually start a pkg.depot(1M) server to 181serve the generated repositories to multiple test machines. 182 183 Start up a depot on your build machine. 184 cd $CODEMGR_WS/packages/$MACH/nightly[-nd] 185 /usr/lib/pkg.depotd -d repo.redist -p <port> & 186 187 Make SURE you choose an unused port and the depot 188 starts successfully. 189 190 The depot can be started across NFS as well if you 191 have a fast pipe. 192 193 Configure your test system. 194 pkg set-publisher -P -g http://<your server host>:<port> on-nightly 195 pkg set-publisher --non-sticky opensolaris.org 196 pkg uninstall entire 197 198 pkg image-update your test system. 199 pkg image-update will upgrade all packages on your test system 200 201Always make sure your bits are installed with image-update. 202 Check your versions. 203 pkg info osnet-incorporation 204 205 Multiple packages should be updated. 206 If you did a full build, all ON packages will be updated. 207 When image-update tells you that only one or two packages has 208 been updated, you likely did not get the updates you expected. 209 210There are various tactics for troubleshooting a failed upgrade. 211 Make sure entire is uninstalled. 212 213 pkg install -nv osnet-incorporation@<your version> 214 Ask IPS to explicitly check if ON can be installed, and 215 if it can't, tell you why not. 216 217 Obsolete and renamed packages can cause trouble. 218 pkg search -l ::pkg.renamed:true 219 pkg search -l ::pkg.obsolete:true 220 221Making Packaging Changes 222------------------------ 223 224Package definitions are in usr/src/pkg/manifests/, and have one 225file per package, including for multi-architecture packages. For 226most packaging changes you only need to add or change the packaging 227manifests themselves. No packaging metadata may be kept outside of the 228manifests. If you find yourself needing to modify usr/src/pkg/Makefile, 229you're almost certainly doing something wrong. 230 231Remember that the "check" target is available to check many of 232your packaging changes. 233 234We run pkgmogrify(1) over the manifests before publication. This 235allows us to apply a set of macros and package transformations in 236order to make the manifests themselves easier to maintain. 237 238We supply a set of commonly-used macros for use in package manifests. 239These are the PKGMOG_DEFINES in usr/src/pkg/Makefile. 240 241 $(i386_ONLY) 242 $(sparc_ONLY) 243 $(ARCH) 244 $(ARCH32) 245 $(ARCH64) 246 $(PKGVERS), which is set to 247 $(PKGVERS_COMPONENT),$(PKGVERS_BUILTON)-0.$(PKGVERS_BRANCH) 248 249pkgmogrify(1) also allows us to include a set of default transformations. 250The definitions for these transforms are in usr/src/pkg/transforms/. An 251overview of their use is supplied here, but for a full accounting, please 252read pkmogrify(1) and the files themselves. 253 254 defaults 255 This transform is applied to all manifests. It specifies 256 a set of sensible default permissions, a set of 257 directory locations for which the reboot-needed actuator 258 is always applied to files, and some other basic defaults. 259 260 publish 261 This transform is applied to all manifests. It ensures 262 that manifest lines which don't apply to the current 263 architecture are stripped. It also ensures non-redistributable 264 packages aren't included in an open-only build. 265 266 restart_fmri 267 This transform is applied to all manifests. It modifies 268 all package manifest lines for SMF manifests in standard 269 locations to include an actuator which runs manifest-import 270 on installation/update/removal, as well as some others. If 271 you're adding a new class of file that would benefit from 272 a restart or refresh of a specific SMF service, please add 273 it here. 274 275 extract_metadata 276 This transform is applied to all manifests. It deals with 277 manipulations required for packaging metadata like 278 org.opensolaris.redist, pkg.renamed, and pkg.obsolete. 279 280 hollow_zone_pkg 281 This transform is available for explicit inclusion in 282 some manifests. It ensures that all contents of the 283 package are not installed within a non-global zone, but the 284 package and its metadata are available in order to satisfy 285 packaging dependencies. 286 287pkgmogrify(1) also allows us to use comments and continuation lines 288in our manifests. 289 290Zones and Packages 291------------------ 292 293pkg(5) uses variants to implement zones. If a package is marked 294with both global and non-global zone variants, the package contents will 295be installed in both global and non-global by default. 296 set name=variant.opensolaris.zone value=global value=nonglobal 297 298Specific actions within a package can be tagged as applying to only 299the global zone or only the non-global zones. 300 301The hollow_zone_pkg transform described above is also available to 302simplify a common packaging scenario. 303 304Dependencies 305------------ 306 307Package dependencies are automatically calculated during build time 308using pkgdepend(1). After you've done a build, you can verify your 309dependencies in the <package>.res file described above. If the 310file is missing a dependency that you believe could be auto-detected, 311please file a bug against pkgdepend(1). 312 313Dependencies can be added manually using the "depend" action. Please 314add a comment describing why the dependency is required. 315 316Moving Content Between Packages and Removing Content 317---------------------------------------------------- 318 319pkg(5) tracks when content is removed from packages, and automatically 320removes it. 321 322If you need to move content between packages, there are two primary 323things to do. 324 325 "preserve" files must be marked with original_name. 326 The first time a "preserve" file moves between packages, 327 you must set original_name=<original package>:<file> 328 in that file's action. Subsequent moves do not require 329 modification. 330 331 Consider adding a dependency on the new package. 332 The only way a system will end up with a new package 333 after upgrade is if an existing package depends on it. 334 335Renaming a Package 336------------------ 337 338To rename a package, leave the old package manifest in place, but 339empty it of all delivered content. The old package should include: 340 341 set name=pkg.fmri with the version set explicitly to the 342 build you're integrating into. For example, if you wanted 343 to rename SUNWrmodu in build 133 you'd change the pkg.fmri 344 line to read 345 set name=pkg.fmri value=pkg:/SUNWrmodu@0.5.11,5.11-0.133 346 347 set name=pkg.renamed value=true 348 349 The architectures and variants you're renaming. These 350 should just be copied from your old package, as you 351 must rename a package on all architectures and 352 variants simultaneously. 353 354 A dependency on the new package. 355 356If there were "preserve" files in the package you're renaming, make 357sure to create original_name settings in the new package. 358 359EOFs and Removals 360----------------- 361 362To remove files, directories, drivers, or anything else within a package, 363simply stop delivering them in the package. IPS will manage the removal 364of no longer delivered content. 365 366Package removals have impact on the rest of the system. Before marking 367a package as obsolete, search in the OpenSolaris development 368repository (http://pkg.opensolaris.org/dev or http://ipkg.sfbay/dev) 369to find out if any other packages depend on the software you intend 370to EOF. If any packages do, you need to coordinate with those 371consolidations. 372 373The "slim_install" package may depend on ON packages. If it does, 374you must send a FLAG DAY message for ON users and PIT who test 375install. You must also file an installation bug to get that package 376updated in the same build or earlier than you intend to integrate. 377You should also test install yourself. You can do this by replacing 378the "slim_install" in your Distro Constructor manifest with the 379explicit list of packages to install. 380 381To remove a package, you must mark it as obsolete. The obsoleted 382package manifest should include 383 384 set name=pkg.fmri with the version set explicitly to the 385 build you're integrating into. For example, if you wanted 386 to remove SUNWwbsd in build 133 you'd change the pkg.fmri 387 line to read 388 set name=pkg.fmri value=pkg:/SUNWwbsd@0.5.11,5.11-0.133 389 390 set name=pkg.obsolete value=true 391 392 The architectures and variants you're obsoleting. These 393 should just be copied from your old package, as you 394 must obsolete a package on all architectures and 395 variants simultaneously. 396 397Creating a Package 398------------------ 399 400The easiest thing is to copy a package similar to the one you're 401trying to create. Note that packages are no longer split on the 402/usr and / boundary. 403 404The following actions are required for all packages in ON. 405 set name=pkg.fmri 406 Every package must have an FMRI. That is the package's 407 name. 408 409 set name=pkg.summary 410 Every package must have a short summary of its purpose. 411 412 set name=pkg.description 413 Every package must have a one-sentence description of 414 its purpose. 415 416 set name=variant.arch 417 Every package must specify which architectures it delivers. 418 419 set name=variant.opensolaris.zone 420 Every package must specify whether it can be installed in 421 global zones, non-global zones, or both. 422 423 set name=info.classification 424 Every package must specify a category for the packaging GUI. 425 You must use an existing category, and may not invent new ones. 426 Existing categories and their subcategories are listed 427 in /usr/share/package-manager/data/opensolaris.org.sections. 428 429 license 430 All packages must specify a set of license actions. If 431 you're adding items here that are not already included in 432 usr/src/pkg/license_files, then you will also need to modify 433 usr/src/tools/opensolaris/license-list. 434 435The following actions are uncommon but specific to ON. 436 437 set name=org.opensolaris.redist 438 This may be set to nonredist or internal. If a package 439 is redistributable, do not create this action. "internal" 440 packages which are legally not allowed to be distributed 441 at all are strongly discouraged. If you're adding 442 content to a package with this action, you should have 443 modified bindrop.sh, and test open-only builds. 444 445Drivers and Packages 446-------------------- 447 448Scripting is no longer required to deal with addition or removal of 449drivers in ON. A "driver" action should be specified for each driver, 450and IPS will handle updates to all the driver files. 451