1# 2# CDDL HEADER START 3# 4# The contents of this file are subject to the terms of the 5# Common Development and Distribution License (the "License"). 6# You may not use this file except in compliance with the License. 7# 8# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE 9# or http://www.opensolaris.org/os/licensing. 10# See the License for the specific language governing permissions 11# and limitations under the License. 12# 13# When distributing Covered Code, include this CDDL HEADER in each 14# file and include the License file at usr/src/OPENSOLARIS.LICENSE. 15# If applicable, add the following below this CDDL HEADER, with the 16# fields enclosed by brackets "[]" replaced with your own identifying 17# information: Portions Copyright [yyyy] [name of copyright owner] 18# 19# CDDL HEADER END 20# 21 22# 23# Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 24# 25 26Introduction 27------------ 28 29This README describes some basics about creating, modifying, and 30building packages in ON. All package creation in ON is done using 31tools available to our customers. If terminology in this document 32is confusing, you may wish to review the pkg(7) manpage. 33 34Packaging Overview 35------------------ 36 37usr/src/pkg/ contains the definitions and rules needed to build an IPS 38repository which contain the deliverables from an ON build. 39 40The manifests directory contains all manifests, and has one file 41per package. For most packaging changes you only need to add or 42change the packaging manifests themselves. 43 44The build rules create a package repository. Separate DEBUG and 45non-DEBUG versions are built, and are available at: 46 $CODEMGR_WS/packages/$MACH/nightly[-nd]/repo.redist 47 48Building Packages 49----------------- 50 51The -p option to nightly will build the IPS repository. 52 53Alternatively, in usr/src/pkg/Makefile there are make targets for: 54 55 all 56 Process manifests into their final form with unresolved 57 dependencies before publication. 58 59 install 60 Publish packages. 61 62 gendeps 63 Run `pkgdepend resolve`. See Dependencies section. 64 65 protocmp 66 Compare the proto area against package definitions for 67 missing or incorrect files. 68 69 pmodes 70 Check file and directory modes for best practices. 71 72 check 73 Run protocmp and pmodes. 74 75The build behavior may modified by the following variables: 76 77 SUPPRESSPKGDEP 78 If SUPPRESSPKGDEP is set to "true" in your environment, 79 package dependencies will not be generated. This variable 80 should not be set in normal builds as it will mask product 81 bugs. 82 83 PKGDEBUG 84 If PKGDEBUG is set in your environment, $MAKE will print 85 detailed information about the commands it executes to 86 process and publish packages. 87 88A set of intermediate build products are placed in 89usr/src/pkg/packages.$MACH/. These can be useful during development. 90 91 .mog 92 The resulting package manifest after running pkgmogrify(1) 93 on the supplied manifest. See below for details on 94 pkgmogrify(1) use in ON. 95 96 .dep 97 The resulting manifest after running `pkgdepend generate` 98 on the .mog manifest. 99 100 .res 101 The resulting manifest after running `pkgdepend resolve` 102 on the .dep manifest. 103 104Incremental Builds 105------------------ 106 107 If you want to process a subset of manifests, simply set PKGS on the 108 make command line and specify the "all" target. If you want to process 109 them all, simply specify the "all" target. 110 111 % dmake -e PKGS="BRCMbnx BRCMbnxe" all 112 % dmake -e all 113 114 If you want to publish a subset of packages, simply set PKGS on the 115 make command line and specify the "install" target. Overriding PKGS 116 will cause dependency resolution to be limited to PKGS from the 117 current build, with a fallback to packages installed on the build 118 system. If you want to publish them all, simply specify the 119 "install" target. 120 121 % dmake -e PKGS="BRCMbnx BRCMbnxe" install 122 % dmake -e install 123 124 You can also use package names, or package names with ".pub" 125 extensions, as build targets. This will cause processing or 126 publication of the specified package(s). 127 128 The on-disk repository will be initialized when it does not exist, 129 or when you run nightly -p. If you build incrementally, 130 packages will simply be added to the existing repository. 131 132 To do cross-platform packaging, prefix your target with (for 133 example) "i386/", as in "dmake i386/install". Note that we 134 currently pull some license files directly from a built source 135 tree, so if you do this in a workspace that had proto area copied 136 in via nightly -U, then you'll need to set $SRC to point to the 137 workspace that was actually built. 138 139Testing Packages 140---------------- 141 142To test the generated repository, you should use the "onu" tool 143available from /opt/onbld/bin or usr/src/tools/scripts/ to setup and 144upgrade your system. A manpage is available in /opt/onbld/man 145or usr/src/tools/scripts/onu.1. 146 147Alternatively, you can manually start a pkg.depot(8) server to 148serve the generated repository to multiple test machines. 149 150 Start up a depot on your build machine. 151 cd $CODEMGR_WS/packages/$MACH/nightly[-nd] 152 /usr/lib/pkg.depotd -d repo.redist -p <port> & 153 154 Make SURE you choose an unused port and the depot 155 starts successfully. 156 157 The depot can be started across NFS as well if you 158 have a fast pipe. 159 160 Configure your test system. 161 pkg set-publisher -P -g http://<your server host>:<port> on-nightly 162 pkg set-publisher --non-sticky opensolaris.org 163 pkg uninstall entire 164 165 pkg image-update your test system. 166 pkg image-update will upgrade all packages on your test system 167 168Always make sure your bits are installed with image-update. 169 Check your versions. 170 pkg info osnet-incorporation 171 172 Multiple packages should be updated. 173 If you did a full build, all ON packages will be updated. 174 When image-update tells you that only one or two packages has 175 been updated, you likely did not get the updates you expected. 176 177There are various tactics for troubleshooting a failed upgrade. 178 Make sure entire is uninstalled. 179 180 pkg install -nv osnet-incorporation@<your version> 181 Ask IPS to explicitly check if ON can be installed, and 182 if it can't, tell you why not. 183 184 Obsolete and renamed packages can cause trouble. 185 pkg search -l ::pkg.renamed:true 186 pkg search -l ::pkg.obsolete:true 187 188 189Making Packaging Changes 190------------------------ 191 192Package definitions are in usr/src/pkg/manifests/, and have one 193file per package, including for multi-architecture packages. For 194most packaging changes you only need to add or change the packaging 195manifests themselves. No packaging metadata may be kept outside of the 196manifests. If you find yourself needing to modify usr/src/pkg/Makefile, 197you're almost certainly doing something wrong. 198 199Remember that the "check" target is available to check many of 200your packaging changes. 201 202We run pkgmogrify(1) over the manifests before publication. This 203allows us to apply a set of macros and package transformations in 204order to make the manifests themselves easier to maintain. 205 206We supply a set of commonly-used macros for use in package manifests. 207These are the PKGMOG_DEFINES in usr/src/pkg/Makefile. 208 209 $(i386_ONLY) 210 $(ARCH) 211 $(ARCH32) 212 $(ARCH64) 213 $(PKGVERS), which is set to 214 $(PKGVERS_COMPONENT),$(PKGVERS_BUILTON)-$(PKGVERS_BRANCH) 215 216pkgmogrify(1) also allows us to include a set of default transformations. 217The definitions for these transforms are in usr/src/pkg/transforms/. An 218overview of their use is supplied here, but for a full accounting, please 219read pkmogrify(1) and the files themselves. 220 221 defaults 222 This transform is applied to all manifests. It specifies 223 a set of sensible default permissions, a set of 224 directory locations for which the reboot-needed actuator 225 is always applied to files, and some other basic defaults. 226 227 publish 228 This transform is applied to all manifests. It ensures 229 that manifest lines which don't apply to the current 230 architecture are stripped. 231 232 restart_fmri 233 This transform is applied to all manifests. It modifies 234 all package manifest lines for SMF manifests in standard 235 locations to include an actuator which runs manifest-import 236 on installation/update/removal, as well as some others. If 237 you're adding a new class of file that would benefit from 238 a restart or refresh of a specific SMF service, please add 239 it here. 240 241 extract_metadata 242 This transform is applied to all manifests. It deals with 243 manipulations required for packaging metadata like 244 pkg.renamed, and pkg.obsolete. 245 246 hollow_zone_pkg 247 This transform is available for explicit inclusion in 248 some manifests. It ensures that all contents of the 249 package are not installed within a non-global zone, but the 250 package and its metadata are available in order to satisfy 251 packaging dependencies. 252 253pkgmogrify(1) also allows us to use comments and continuation lines 254in our manifests. 255 256Zones and Packages 257------------------ 258 259pkg(7) uses variants to implement zones. If a package is marked 260with both global and non-global zone variants, the package contents will 261be installed in both global and non-global by default. 262 set name=variant.opensolaris.zone value=global value=nonglobal 263 264Specific actions within a package can be tagged as applying to only 265the global zone or only the non-global zones. 266 267The hollow_zone_pkg transform described above is also available to 268simplify a common packaging scenario. 269 270Dependencies 271------------ 272 273Package dependencies are automatically calculated during build time 274using pkgdepend(1). After you've done a build, you can verify your 275dependencies in the <package>.res file described above. If the 276file is missing a dependency that you believe could be auto-detected, 277please file a bug against pkgdepend(1). 278 279Dependencies can be added manually using the "depend" action. Please 280add a comment describing why the dependency is required. 281 282Moving Content Between Packages and Removing Content 283---------------------------------------------------- 284 285pkg(7) tracks when content is removed from packages, and automatically 286removes it. 287 288If you need to move content between packages, there are two primary 289things to do. 290 291 "preserve" files must be marked with original_name. 292 The first time a "preserve" file moves between packages, 293 you must set original_name=<original package>:<file> 294 in that file's action. Subsequent moves do not require 295 modification. 296 297 Consider adding a dependency on the new package. 298 The only way a system will end up with a new package 299 after upgrade is if an existing package depends on it. 300 301Renaming a Package 302------------------ 303 304To rename a package, leave the old package manifest in place, but 305empty it of all delivered content. The old package should include: 306 307 set name=pkg.fmri with the version set explicitly to the 308 build you're integrating into. For example, if you wanted 309 to rename SUNWrmodu in build 133 you'd change the pkg.fmri 310 line to read 311 set name=pkg.fmri value=pkg:/SUNWrmodu@0.5.11,5.11-0.133 312 313 set name=pkg.renamed value=true 314 315 The architectures and variants you're renaming. These 316 should just be copied from your old package, as you 317 must rename a package on all architectures and 318 variants simultaneously. 319 320 A dependency on the new package. 321 322If there were "preserve" files in the package you're renaming, make 323sure to create original_name settings in the new package. 324 325If there was a org.opensolaris.noincorp property in the package being 326renamed, make sure you keep it in both the original and the renamed 327packages. 328 329EOFs and Removals 330----------------- 331 332To remove files, directories, drivers, or anything else within a package, 333simply stop delivering them in the package. IPS will manage the removal 334of no longer delivered content. 335 336Package removals have impact on the rest of the system. Before marking 337a package as obsolete, search in the OpenSolaris development 338repository (http://pkg.opensolaris.org/dev or http://ipkg.sfbay/dev) 339to find out if any other packages depend on the software you intend 340to EOF. If any packages do, you need to coordinate with those 341consolidations. 342 343The "slim_install" package may depend on ON packages. If it does, 344you must send a FLAG DAY message for ON users and PIT who test 345install. You must also file an installation bug to get that package 346updated in the same build or earlier than you intend to integrate. 347You should also test install yourself. You can do this by replacing 348the "slim_install" in your Distro Constructor manifest with the 349explicit list of packages to install. 350 351To remove a package, you must mark it as obsolete. You must *also* mark 352as obsolete any packages which are renamed ancestors of this package, and 353remove their rename dependencies. Here is what you must do. 354 355To find rename ancestors, select all of the manifests which are renames, 356then look for the one which was renamed to the package you care about. 357For example, to find rename ancestors of 'system/zones', do the following: 358 359 $ cd usr/src/pkg/manifests 360 $ mypkgname=system/zones 361 $ grep -l "fmri=pkg:/$mypkgname@" `grep -l pkg.renamed *.p5m` /dev/null 362 SUNWzone.p5m 363 364Make sure to check that the package has not undergone multiple renames! 365 366 $ mypkgname=SUNWzone 367 $ grep -l "fmri=pkg:/$mypkgname@" `grep -l pkg.renamed *.p5m` /dev/null 368 $ 369 370Once you have the renamed ancestor list, for *each* of the packages (the 371newly obsolete package, and its renamed ancestors), edit the package as 372follows: 373 374 Update 'set name=pkg.fmri' with the version set explicitly to the 375 build you're integrating into. For example, if you wanted 376 to remove SUNWwbsd in build 133 you'd change the pkg.fmri 377 line to read: 378 'set name=pkg.fmri value=pkg:/SUNWwbsd@0.5.11,5.11-0.133' 379 380 Add 'set name=pkg.obsolete value=true'. 381 382 Maintain the architecture and variant declarations in the 383 package you are obsoleting. Note that you must obsolete a 384 package on all architectures and variants simultaneously. 385 386 Delete everything else. 387 388 If the package is a renamed ancestor, leave a comment behind as 389 follows: 390 391 # Was renamed to <other-pkg-name>, both now obsolete. 392 393Here is a complete example. SUNWfoobar was a package which was renamed 394to system/foobar in build 140, and then later obsoleted in build 150. 395Note that in all cases the package FMRI is updated to the obsoletion 396build, 150. 397 398 SUNWfoobar.p5m: 399 # Was renamed to system/foobar, both now obsolete. 400 set name=pkg.fmri value=pkg:/SUNWfoobar@0.5.11,5.11-0.150 401 set name=pkg.obsolete value=true 402 set name=variant.arch value=$(ARCH) 403 404 system-foobar.p5m: 405 set name=pkg.fmri value=pkg:/system/foobar@0.5.11,5.11-0.150 406 set name=pkg.obsolete value=true 407 set name=variant.arch value=$(ARCH) 408 409Creating a Package 410------------------ 411 412The easiest thing is to copy a package similar to the one you're 413trying to create. Note that packages are no longer split on the 414/usr and / boundary. 415 416The following actions are required for all packages in ON. 417 set name=pkg.fmri 418 Every package must have an FMRI. That is the package's 419 name. 420 421 set name=pkg.summary 422 Every package must have a short summary of its purpose. 423 424 set name=pkg.description 425 Every package must have a one-sentence description of 426 its purpose. 427 428 set name=variant.arch 429 Every package must specify which architectures it delivers. 430 431 set name=info.classification 432 Every package must specify a category for the packaging GUI. 433 You must use an existing category, and may not invent new ones. 434 Existing categories and their subcategories are listed 435 in /usr/share/package-manager/data/opensolaris.org.sections. 436 437 license 438 All packages must specify a set of license actions. 439 440You don't need to set the following. They're taken care of for all OS/Net 441packages in the transforms/common_actions file. 442 443 set name=variant.opensolaris.zone 444 Every package must specify whether it can be installed in 445 global zones, non-global zones, or both. All ON packages are 446 delivered in both global and non-global zones. 447 448 set name=org.opensolaris.consolidation value=osnet 449 All packages from OS/Net come from OS/Net... 450 451Drivers and Packages 452-------------------- 453 454Scripting is no longer required to deal with addition or removal of 455drivers in ON. A "driver" action should be specified for each driver, 456and IPS will handle updates to all the driver files. 457