1# 2# CDDL HEADER START 3# 4# The contents of this file are subject to the terms of the 5# Common Development and Distribution License (the "License"). 6# You may not use this file except in compliance with the License. 7# 8# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE 9# or http://www.opensolaris.org/os/licensing. 10# See the License for the specific language governing permissions 11# and limitations under the License. 12# 13# When distributing Covered Code, include this CDDL HEADER in each 14# file and include the License file at usr/src/OPENSOLARIS.LICENSE. 15# If applicable, add the following below this CDDL HEADER, with the 16# fields enclosed by brackets "[]" replaced with your own identifying 17# information: Portions Copyright [yyyy] [name of copyright owner] 18# 19# CDDL HEADER END 20# 21 22# 23# Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 24# 25 26Introduction 27------------ 28 29This README describes some basics about creating, modifying, and 30building packages in ON. All package creation in ON is done using 31tools available to our customers. If terminology in this document 32is confusing, you may wish to review the pkg(7) manpage. 33 34Packaging Overview 35------------------ 36 37usr/src/pkg/ contains the definitions and rules needed to build an IPS 38repository which contain the deliverables from an ON build. 39 40The manifests directory contains all manifests, and has one file 41per package. For most packaging changes you only need to add or 42change the packaging manifests themselves. 43 44The build rules create a package repository. Separate DEBUG and 45non-DEBUG versions are built, and are available at: 46 $CODEMGR_WS/packages/$MACH/nightly[-nd]/repo.redist 47 48Building Packages 49----------------- 50 51The -p option to nightly will build the IPS repository. 52 53Alternatively, in usr/src/pkg/Makefile there are make targets for: 54 55 all 56 Process manifests into their final form with unresolved 57 dependencies before publication. 58 59 install 60 Publish packages. 61 62 gendeps 63 Run `pkgdepend resolve`. See Dependencies section. 64 65 protocmp 66 Compare the proto area against package definitions for 67 missing or incorrect files. 68 69 pmodes 70 Check file and directory modes for best practices. 71 72 check 73 Run protocmp and pmodes. 74 75The build behavior may modified by the following variables: 76 77 SUPPRESSPKGDEP 78 If SUPPRESSPKGDEP is set to "true" in your environment, 79 package dependencies will not be generated. This variable 80 should not be set in normal builds as it will mask product 81 bugs. 82 83 PKGDEBUG 84 If PKGDEBUG is set in your environment, $MAKE will print 85 detailed information about the commands it executes to 86 process and publish packages. 87 88 ONNV_BUILDNUM 89 If ONNV_BUILDNUM is set to an older ON build number, 90 your packages will be published at that version instead 91 of the one defined in usr/src/Makefile.buildnum, which 92 is managed by the gatekeepers. 93 94A set of intermediate build products are placed in 95usr/src/pkg/packages.$MACH/. These can be useful during development. 96 97 .mog 98 The resulting package manifest after running pkgmogrify(1) 99 on the supplied manifest. See below for details on 100 pkgmogrify(1) use in ON. 101 102 .dep 103 The resulting manifest after running `pkgdepend generate` 104 on the .mog manifest. 105 106 .res 107 The resulting manifest after running `pkgdepend resolve` 108 on the .dep manifest. 109 110Incremental Builds 111------------------ 112 113 If you want to process a subset of manifests, simply set PKGS on the 114 make command line and specify the "all" target. If you want to process 115 them all, simply specify the "all" target. 116 117 % dmake -e PKGS="BRCMbnx BRCMbnxe" all 118 % dmake -e all 119 120 If you want to publish a subset of packages, simply set PKGS on the 121 make command line and specify the "install" target. Overriding PKGS 122 will cause dependency resolution to be limited to PKGS from the 123 current build, with a fallback to packages installed on the build 124 system. If you want to publish them all, simply specify the 125 "install" target. 126 127 % dmake -e PKGS="BRCMbnx BRCMbnxe" install 128 % dmake -e install 129 130 You can also use package names, or package names with ".pub" 131 extensions, as build targets. This will cause processing or 132 publication of the specified package(s). 133 134 The on-disk repository will be initialized when it does not exist, 135 or when you run nightly -p. If you build incrementally, 136 packages will simply be added to the existing repository. 137 138 To do cross-platform packaging, prefix your target with (for 139 example) "i386/", as in "dmake i386/install". Note that we 140 currently pull some license files directly from a built source 141 tree, so if you do this in a workspace that had proto area copied 142 in via nightly -U, then you'll need to set $SRC to point to the 143 workspace that was actually built. 144 145Testing Packages 146---------------- 147 148To test the generated repository, you should use the "onu" tool 149available from /opt/onbld/bin or usr/src/tools/scripts/ to setup and 150upgrade your system. A manpage is available in /opt/onbld/man 151or usr/src/tools/scripts/onu.1. 152 153Alternatively, you can manually start a pkg.depot(8) server to 154serve the generated repository to multiple test machines. 155 156 Start up a depot on your build machine. 157 cd $CODEMGR_WS/packages/$MACH/nightly[-nd] 158 /usr/lib/pkg.depotd -d repo.redist -p <port> & 159 160 Make SURE you choose an unused port and the depot 161 starts successfully. 162 163 The depot can be started across NFS as well if you 164 have a fast pipe. 165 166 Configure your test system. 167 pkg set-publisher -P -g http://<your server host>:<port> on-nightly 168 pkg set-publisher --non-sticky opensolaris.org 169 pkg uninstall entire 170 171 pkg image-update your test system. 172 pkg image-update will upgrade all packages on your test system 173 174Always make sure your bits are installed with image-update. 175 Check your versions. 176 pkg info osnet-incorporation 177 178 Multiple packages should be updated. 179 If you did a full build, all ON packages will be updated. 180 When image-update tells you that only one or two packages has 181 been updated, you likely did not get the updates you expected. 182 183There are various tactics for troubleshooting a failed upgrade. 184 Make sure entire is uninstalled. 185 186 pkg install -nv osnet-incorporation@<your version> 187 Ask IPS to explicitly check if ON can be installed, and 188 if it can't, tell you why not. 189 190 Obsolete and renamed packages can cause trouble. 191 pkg search -l ::pkg.renamed:true 192 pkg search -l ::pkg.obsolete:true 193 194 195Making Packaging Changes 196------------------------ 197 198Package definitions are in usr/src/pkg/manifests/, and have one 199file per package, including for multi-architecture packages. For 200most packaging changes you only need to add or change the packaging 201manifests themselves. No packaging metadata may be kept outside of the 202manifests. If you find yourself needing to modify usr/src/pkg/Makefile, 203you're almost certainly doing something wrong. 204 205Remember that the "check" target is available to check many of 206your packaging changes. 207 208We run pkgmogrify(1) over the manifests before publication. This 209allows us to apply a set of macros and package transformations in 210order to make the manifests themselves easier to maintain. 211 212We supply a set of commonly-used macros for use in package manifests. 213These are the PKGMOG_DEFINES in usr/src/pkg/Makefile. 214 215 $(i386_ONLY) 216 $(ARCH) 217 $(ARCH32) 218 $(ARCH64) 219 $(PKGVERS), which is set to 220 $(PKGVERS_COMPONENT),$(PKGVERS_BUILTON)-$(PKGVERS_BRANCH) 221 222pkgmogrify(1) also allows us to include a set of default transformations. 223The definitions for these transforms are in usr/src/pkg/transforms/. An 224overview of their use is supplied here, but for a full accounting, please 225read pkmogrify(1) and the files themselves. 226 227 defaults 228 This transform is applied to all manifests. It specifies 229 a set of sensible default permissions, a set of 230 directory locations for which the reboot-needed actuator 231 is always applied to files, and some other basic defaults. 232 233 publish 234 This transform is applied to all manifests. It ensures 235 that manifest lines which don't apply to the current 236 architecture are stripped. 237 238 restart_fmri 239 This transform is applied to all manifests. It modifies 240 all package manifest lines for SMF manifests in standard 241 locations to include an actuator which runs manifest-import 242 on installation/update/removal, as well as some others. If 243 you're adding a new class of file that would benefit from 244 a restart or refresh of a specific SMF service, please add 245 it here. 246 247 extract_metadata 248 This transform is applied to all manifests. It deals with 249 manipulations required for packaging metadata like 250 pkg.renamed, and pkg.obsolete. 251 252 hollow_zone_pkg 253 This transform is available for explicit inclusion in 254 some manifests. It ensures that all contents of the 255 package are not installed within a non-global zone, but the 256 package and its metadata are available in order to satisfy 257 packaging dependencies. 258 259pkgmogrify(1) also allows us to use comments and continuation lines 260in our manifests. 261 262Zones and Packages 263------------------ 264 265pkg(7) uses variants to implement zones. If a package is marked 266with both global and non-global zone variants, the package contents will 267be installed in both global and non-global by default. 268 set name=variant.opensolaris.zone value=global value=nonglobal 269 270Specific actions within a package can be tagged as applying to only 271the global zone or only the non-global zones. 272 273The hollow_zone_pkg transform described above is also available to 274simplify a common packaging scenario. 275 276Dependencies 277------------ 278 279Package dependencies are automatically calculated during build time 280using pkgdepend(1). After you've done a build, you can verify your 281dependencies in the <package>.res file described above. If the 282file is missing a dependency that you believe could be auto-detected, 283please file a bug against pkgdepend(1). 284 285Dependencies can be added manually using the "depend" action. Please 286add a comment describing why the dependency is required. 287 288Moving Content Between Packages and Removing Content 289---------------------------------------------------- 290 291pkg(7) tracks when content is removed from packages, and automatically 292removes it. 293 294If you need to move content between packages, there are two primary 295things to do. 296 297 "preserve" files must be marked with original_name. 298 The first time a "preserve" file moves between packages, 299 you must set original_name=<original package>:<file> 300 in that file's action. Subsequent moves do not require 301 modification. 302 303 Consider adding a dependency on the new package. 304 The only way a system will end up with a new package 305 after upgrade is if an existing package depends on it. 306 307Renaming a Package 308------------------ 309 310To rename a package, leave the old package manifest in place, but 311empty it of all delivered content. The old package should include: 312 313 set name=pkg.fmri with the version set explicitly to the 314 build you're integrating into. For example, if you wanted 315 to rename SUNWrmodu in build 133 you'd change the pkg.fmri 316 line to read 317 set name=pkg.fmri value=pkg:/SUNWrmodu@0.5.11,5.11-0.133 318 319 set name=pkg.renamed value=true 320 321 The architectures and variants you're renaming. These 322 should just be copied from your old package, as you 323 must rename a package on all architectures and 324 variants simultaneously. 325 326 A dependency on the new package. 327 328If there were "preserve" files in the package you're renaming, make 329sure to create original_name settings in the new package. 330 331If there was a org.opensolaris.noincorp property in the package being 332renamed, make sure you keep it in both the original and the renamed 333packages. 334 335EOFs and Removals 336----------------- 337 338To remove files, directories, drivers, or anything else within a package, 339simply stop delivering them in the package. IPS will manage the removal 340of no longer delivered content. 341 342Package removals have impact on the rest of the system. Before marking 343a package as obsolete, search in the OpenSolaris development 344repository (http://pkg.opensolaris.org/dev or http://ipkg.sfbay/dev) 345to find out if any other packages depend on the software you intend 346to EOF. If any packages do, you need to coordinate with those 347consolidations. 348 349The "slim_install" package may depend on ON packages. If it does, 350you must send a FLAG DAY message for ON users and PIT who test 351install. You must also file an installation bug to get that package 352updated in the same build or earlier than you intend to integrate. 353You should also test install yourself. You can do this by replacing 354the "slim_install" in your Distro Constructor manifest with the 355explicit list of packages to install. 356 357To remove a package, you must mark it as obsolete. You must *also* mark 358as obsolete any packages which are renamed ancestors of this package, and 359remove their rename dependencies. Here is what you must do. 360 361To find rename ancestors, select all of the manifests which are renames, 362then look for the one which was renamed to the package you care about. 363For example, to find rename ancestors of 'system/zones', do the following: 364 365 $ cd usr/src/pkg/manifests 366 $ mypkgname=system/zones 367 $ grep -l "fmri=pkg:/$mypkgname@" `grep -l pkg.renamed *.p5m` /dev/null 368 SUNWzone.p5m 369 370Make sure to check that the package has not undergone multiple renames! 371 372 $ mypkgname=SUNWzone 373 $ grep -l "fmri=pkg:/$mypkgname@" `grep -l pkg.renamed *.p5m` /dev/null 374 $ 375 376Once you have the renamed ancestor list, for *each* of the packages (the 377newly obsolete package, and its renamed ancestors), edit the package as 378follows: 379 380 Update 'set name=pkg.fmri' with the version set explicitly to the 381 build you're integrating into. For example, if you wanted 382 to remove SUNWwbsd in build 133 you'd change the pkg.fmri 383 line to read: 384 'set name=pkg.fmri value=pkg:/SUNWwbsd@0.5.11,5.11-0.133' 385 386 Add 'set name=pkg.obsolete value=true'. 387 388 Maintain the architecture and variant declarations in the 389 package you are obsoleting. Note that you must obsolete a 390 package on all architectures and variants simultaneously. 391 392 Delete everything else. 393 394 If the package is a renamed ancestor, leave a comment behind as 395 follows: 396 397 # Was renamed to <other-pkg-name>, both now obsolete. 398 399Here is a complete example. SUNWfoobar was a package which was renamed 400to system/foobar in build 140, and then later obsoleted in build 150. 401Note that in all cases the package FMRI is updated to the obsoletion 402build, 150. 403 404 SUNWfoobar.p5m: 405 # Was renamed to system/foobar, both now obsolete. 406 set name=pkg.fmri value=pkg:/SUNWfoobar@0.5.11,5.11-0.150 407 set name=pkg.obsolete value=true 408 set name=variant.arch value=$(ARCH) 409 410 system-foobar.p5m: 411 set name=pkg.fmri value=pkg:/system/foobar@0.5.11,5.11-0.150 412 set name=pkg.obsolete value=true 413 set name=variant.arch value=$(ARCH) 414 415Creating a Package 416------------------ 417 418The easiest thing is to copy a package similar to the one you're 419trying to create. Note that packages are no longer split on the 420/usr and / boundary. 421 422The following actions are required for all packages in ON. 423 set name=pkg.fmri 424 Every package must have an FMRI. That is the package's 425 name. 426 427 set name=pkg.summary 428 Every package must have a short summary of its purpose. 429 430 set name=pkg.description 431 Every package must have a one-sentence description of 432 its purpose. 433 434 set name=variant.arch 435 Every package must specify which architectures it delivers. 436 437 set name=info.classification 438 Every package must specify a category for the packaging GUI. 439 You must use an existing category, and may not invent new ones. 440 Existing categories and their subcategories are listed 441 in /usr/share/package-manager/data/opensolaris.org.sections. 442 443 license 444 All packages must specify a set of license actions. 445 446You don't need to set the following. They're taken care of for all OS/Net 447packages in the transforms/common_actions file. 448 449 set name=variant.opensolaris.zone 450 Every package must specify whether it can be installed in 451 global zones, non-global zones, or both. All ON packages are 452 delivered in both global and non-global zones. 453 454 set name=org.opensolaris.consolidation value=osnet 455 All packages from OS/Net come from OS/Net... 456 457Drivers and Packages 458-------------------- 459 460Scripting is no longer required to deal with addition or removal of 461drivers in ON. A "driver" action should be specified for each driver, 462and IPS will handle updates to all the driver files. 463