# # CDDL HEADER START # # The contents of this file are subject to the terms of the # Common Development and Distribution License (the "License"). # You may not use this file except in compliance with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE # or http://www.opensolaris.org/os/licensing. # See the License for the specific language governing permissions # and limitations under the License. # # When distributing Covered Code, include this CDDL HEADER in each # file and include the License file at usr/src/OPENSOLARIS.LICENSE. # If applicable, add the following below this CDDL HEADER, with the # fields enclosed by brackets "[]" replaced with your own identifying # information: Portions Copyright [yyyy] [name of copyright owner] # # CDDL HEADER END # # # Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. # Introduction ------------ This README describes some basics about creating, modifying, and building packages in ON. All package creation in ON is done using tools available to our customers. If terminology in this document is confusing, you may wish to review the pkg(7) manpage. Packaging Overview ------------------ usr/src/pkg/ contains the definitions and rules needed to build an IPS repository which contain the deliverables from an ON build. The manifests directory contains all manifests, and has one file per package. For most packaging changes you only need to add or change the packaging manifests themselves. The build rules create a package repository. Separate DEBUG and non-DEBUG versions are built, and are available at: $CODEMGR_WS/packages/$MACH/nightly[-nd]/repo.redist Building Packages ----------------- The -p option to nightly will build the IPS repository. Alternatively, in usr/src/pkg/Makefile there are make targets for: all Process manifests into their final form with unresolved dependencies before publication. install Publish packages. gendeps Run `pkgdepend resolve`. See Dependencies section. protocmp Compare the proto area against package definitions for missing or incorrect files. pmodes Check file and directory modes for best practices. check Run protocmp and pmodes. The build behavior may modified by the following variables: SUPPRESSPKGDEP If SUPPRESSPKGDEP is set to "true" in your environment, package dependencies will not be generated. This variable should not be set in normal builds as it will mask product bugs. PKGDEBUG If PKGDEBUG is set in your environment, $MAKE will print detailed information about the commands it executes to process and publish packages. ONNV_BUILDNUM If ONNV_BUILDNUM is set to an older ON build number, your packages will be published at that version instead of the one defined in usr/src/Makefile.buildnum, which is managed by the gatekeepers. A set of intermediate build products are placed in usr/src/pkg/packages.$MACH/. These can be useful during development. .mog The resulting package manifest after running pkgmogrify(1) on the supplied manifest. See below for details on pkgmogrify(1) use in ON. .dep The resulting manifest after running `pkgdepend generate` on the .mog manifest. .res The resulting manifest after running `pkgdepend resolve` on the .dep manifest. Incremental Builds ------------------ If you want to process a subset of manifests, simply set PKGS on the make command line and specify the "all" target. If you want to process them all, simply specify the "all" target. % dmake -e PKGS="BRCMbnx BRCMbnxe" all % dmake -e all If you want to publish a subset of packages, simply set PKGS on the make command line and specify the "install" target. Overriding PKGS will cause dependency resolution to be limited to PKGS from the current build, with a fallback to packages installed on the build system. If you want to publish them all, simply specify the "install" target. % dmake -e PKGS="BRCMbnx BRCMbnxe" install % dmake -e install You can also use package names, or package names with ".pub" extensions, as build targets. This will cause processing or publication of the specified package(s). The on-disk repository will be initialized when it does not exist, or when you run nightly -p. If you build incrementally, packages will simply be added to the existing repository. To do cross-platform packaging, prefix your target with (for example) "i386/", as in "dmake i386/install". Note that we currently pull some license files directly from a built source tree, so if you do this in a workspace that had proto area copied in via nightly -U, then you'll need to set $SRC to point to the workspace that was actually built. Testing Packages ---------------- To test the generated repository, you should use the "onu" tool available from /opt/onbld/bin or usr/src/tools/scripts/ to setup and upgrade your system. A manpage is available in /opt/onbld/man or usr/src/tools/scripts/onu.1. Alternatively, you can manually start a pkg.depot(8) server to serve the generated repository to multiple test machines. Start up a depot on your build machine. cd $CODEMGR_WS/packages/$MACH/nightly[-nd] /usr/lib/pkg.depotd -d repo.redist -p & Make SURE you choose an unused port and the depot starts successfully. The depot can be started across NFS as well if you have a fast pipe. Configure your test system. pkg set-publisher -P -g http://: on-nightly pkg set-publisher --non-sticky opensolaris.org pkg uninstall entire pkg image-update your test system. pkg image-update will upgrade all packages on your test system Always make sure your bits are installed with image-update. Check your versions. pkg info osnet-incorporation Multiple packages should be updated. If you did a full build, all ON packages will be updated. When image-update tells you that only one or two packages has been updated, you likely did not get the updates you expected. There are various tactics for troubleshooting a failed upgrade. Make sure entire is uninstalled. pkg install -nv osnet-incorporation@ Ask IPS to explicitly check if ON can be installed, and if it can't, tell you why not. Obsolete and renamed packages can cause trouble. pkg search -l ::pkg.renamed:true pkg search -l ::pkg.obsolete:true Making Packaging Changes ------------------------ Package definitions are in usr/src/pkg/manifests/, and have one file per package, including for multi-architecture packages. For most packaging changes you only need to add or change the packaging manifests themselves. No packaging metadata may be kept outside of the manifests. If you find yourself needing to modify usr/src/pkg/Makefile, you're almost certainly doing something wrong. Remember that the "check" target is available to check many of your packaging changes. We run pkgmogrify(1) over the manifests before publication. This allows us to apply a set of macros and package transformations in order to make the manifests themselves easier to maintain. We supply a set of commonly-used macros for use in package manifests. These are the PKGMOG_DEFINES in usr/src/pkg/Makefile. $(i386_ONLY) $(ARCH) $(ARCH32) $(ARCH64) $(PKGVERS), which is set to $(PKGVERS_COMPONENT),$(PKGVERS_BUILTON)-$(PKGVERS_BRANCH) pkgmogrify(1) also allows us to include a set of default transformations. The definitions for these transforms are in usr/src/pkg/transforms/. An overview of their use is supplied here, but for a full accounting, please read pkmogrify(1) and the files themselves. defaults This transform is applied to all manifests. It specifies a set of sensible default permissions, a set of directory locations for which the reboot-needed actuator is always applied to files, and some other basic defaults. publish This transform is applied to all manifests. It ensures that manifest lines which don't apply to the current architecture are stripped. restart_fmri This transform is applied to all manifests. It modifies all package manifest lines for SMF manifests in standard locations to include an actuator which runs manifest-import on installation/update/removal, as well as some others. If you're adding a new class of file that would benefit from a restart or refresh of a specific SMF service, please add it here. extract_metadata This transform is applied to all manifests. It deals with manipulations required for packaging metadata like pkg.renamed, and pkg.obsolete. hollow_zone_pkg This transform is available for explicit inclusion in some manifests. It ensures that all contents of the package are not installed within a non-global zone, but the package and its metadata are available in order to satisfy packaging dependencies. pkgmogrify(1) also allows us to use comments and continuation lines in our manifests. Zones and Packages ------------------ pkg(7) uses variants to implement zones. If a package is marked with both global and non-global zone variants, the package contents will be installed in both global and non-global by default. set name=variant.opensolaris.zone value=global value=nonglobal Specific actions within a package can be tagged as applying to only the global zone or only the non-global zones. The hollow_zone_pkg transform described above is also available to simplify a common packaging scenario. Dependencies ------------ Package dependencies are automatically calculated during build time using pkgdepend(1). After you've done a build, you can verify your dependencies in the .res file described above. If the file is missing a dependency that you believe could be auto-detected, please file a bug against pkgdepend(1). Dependencies can be added manually using the "depend" action. Please add a comment describing why the dependency is required. Moving Content Between Packages and Removing Content ---------------------------------------------------- pkg(7) tracks when content is removed from packages, and automatically removes it. If you need to move content between packages, there are two primary things to do. "preserve" files must be marked with original_name. The first time a "preserve" file moves between packages, you must set original_name=: in that file's action. Subsequent moves do not require modification. Consider adding a dependency on the new package. The only way a system will end up with a new package after upgrade is if an existing package depends on it. Renaming a Package ------------------ To rename a package, leave the old package manifest in place, but empty it of all delivered content. The old package should include: set name=pkg.fmri with the version set explicitly to the build you're integrating into. For example, if you wanted to rename SUNWrmodu in build 133 you'd change the pkg.fmri line to read set name=pkg.fmri value=pkg:/SUNWrmodu@0.5.11,5.11-0.133 set name=pkg.renamed value=true The architectures and variants you're renaming. These should just be copied from your old package, as you must rename a package on all architectures and variants simultaneously. A dependency on the new package. If there were "preserve" files in the package you're renaming, make sure to create original_name settings in the new package. If there was a org.opensolaris.noincorp property in the package being renamed, make sure you keep it in both the original and the renamed packages. EOFs and Removals ----------------- To remove files, directories, drivers, or anything else within a package, simply stop delivering them in the package. IPS will manage the removal of no longer delivered content. Package removals have impact on the rest of the system. Before marking a package as obsolete, search in the OpenSolaris development repository (http://pkg.opensolaris.org/dev or http://ipkg.sfbay/dev) to find out if any other packages depend on the software you intend to EOF. If any packages do, you need to coordinate with those consolidations. The "slim_install" package may depend on ON packages. If it does, you must send a FLAG DAY message for ON users and PIT who test install. You must also file an installation bug to get that package updated in the same build or earlier than you intend to integrate. You should also test install yourself. You can do this by replacing the "slim_install" in your Distro Constructor manifest with the explicit list of packages to install. To remove a package, you must mark it as obsolete. You must *also* mark as obsolete any packages which are renamed ancestors of this package, and remove their rename dependencies. Here is what you must do. To find rename ancestors, select all of the manifests which are renames, then look for the one which was renamed to the package you care about. For example, to find rename ancestors of 'system/zones', do the following: $ cd usr/src/pkg/manifests $ mypkgname=system/zones $ grep -l "fmri=pkg:/$mypkgname@" `grep -l pkg.renamed *.p5m` /dev/null SUNWzone.p5m Make sure to check that the package has not undergone multiple renames! $ mypkgname=SUNWzone $ grep -l "fmri=pkg:/$mypkgname@" `grep -l pkg.renamed *.p5m` /dev/null $ Once you have the renamed ancestor list, for *each* of the packages (the newly obsolete package, and its renamed ancestors), edit the package as follows: Update 'set name=pkg.fmri' with the version set explicitly to the build you're integrating into. For example, if you wanted to remove SUNWwbsd in build 133 you'd change the pkg.fmri line to read: 'set name=pkg.fmri value=pkg:/SUNWwbsd@0.5.11,5.11-0.133' Add 'set name=pkg.obsolete value=true'. Maintain the architecture and variant declarations in the package you are obsoleting. Note that you must obsolete a package on all architectures and variants simultaneously. Delete everything else. If the package is a renamed ancestor, leave a comment behind as follows: # Was renamed to , both now obsolete. Here is a complete example. SUNWfoobar was a package which was renamed to system/foobar in build 140, and then later obsoleted in build 150. Note that in all cases the package FMRI is updated to the obsoletion build, 150. SUNWfoobar.p5m: # Was renamed to system/foobar, both now obsolete. set name=pkg.fmri value=pkg:/SUNWfoobar@0.5.11,5.11-0.150 set name=pkg.obsolete value=true set name=variant.arch value=$(ARCH) system-foobar.p5m: set name=pkg.fmri value=pkg:/system/foobar@0.5.11,5.11-0.150 set name=pkg.obsolete value=true set name=variant.arch value=$(ARCH) Creating a Package ------------------ The easiest thing is to copy a package similar to the one you're trying to create. Note that packages are no longer split on the /usr and / boundary. The following actions are required for all packages in ON. set name=pkg.fmri Every package must have an FMRI. That is the package's name. set name=pkg.summary Every package must have a short summary of its purpose. set name=pkg.description Every package must have a one-sentence description of its purpose. set name=variant.arch Every package must specify which architectures it delivers. set name=info.classification Every package must specify a category for the packaging GUI. You must use an existing category, and may not invent new ones. Existing categories and their subcategories are listed in /usr/share/package-manager/data/opensolaris.org.sections. license All packages must specify a set of license actions. You don't need to set the following. They're taken care of for all OS/Net packages in the transforms/common_actions file. set name=variant.opensolaris.zone Every package must specify whether it can be installed in global zones, non-global zones, or both. All ON packages are delivered in both global and non-global zones. set name=org.opensolaris.consolidation value=osnet All packages from OS/Net come from OS/Net... Drivers and Packages -------------------- Scripting is no longer required to deal with addition or removal of drivers in ON. A "driver" action should be specified for each driver, and IPS will handle updates to all the driver files.