xref: /titanic_53/usr/src/tools/README.tools (revision 75ce41a57ff334bd8fe2cb9ed51eea835892f944)

#
# CDDL HEADER START
#
# The contents of this file are subject to the terms of the
# Common Development and Distribution License (the "License").
# You may not use this file except in compliance with the License.
#
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
# or http://www.opensolaris.org/os/licensing.
# See the License for the specific language governing permissions
# and limitations under the License.
#
# When distributing Covered Code, include this CDDL HEADER in each
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
# If applicable, add the following below this CDDL HEADER, with the
# fields enclosed by brackets "[]" replaced with your own identifying
# information: Portions Copyright [yyyy] [name of copyright owner]
#
# CDDL HEADER END
#
#
# Copyright 2009 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.


This directory contains the tools used to do a full build of the
OS/Net workspace.  They usually live in the /opt/onbld directory on build
machines. From here, 'make install' will build and install the tools
in $ROOT/opt/onbld. If you like, 'make pkg' will build the SUNWonbld
package in $(PKGARCHIVE). Installing that package will populate the
/opt/onbld directory, and create a root account for building called 'gk',
which uses csh and has a home directory of /opt/onbld/gk. You can
use this account to do full builds with 'nightly'. You don't have to,
but the 'gk' account has the path setup properly, has a .make.machines
file for dmake, and has a .login that sets up for dmake.

Layout of /opt/onbld
--------------------

/opt/onbld/etc/abi
	contains Solaris ABI database (ABI_*.db) and exceptions
	for ABI Auditing tool (interface_check, interface_cmp).

/opt/onbld/gk
	gk account's home directory.

/opt/onbld/bin
	basic bin directory - contains scripts.

/opt/onbld/bin/${MACH}
	architecture-specific bin directory for binaries.

/opt/onbld/env
	build environment files.

/opt/onbld/lib
	libraries used by the build tools.

/opt/onbld/lib/python
	python modules used by the build tools.

/opt/onbld/lib/python/onbld/hgext
	Mercurial extensions.

/opt/onbld/man
	rudimentary man pages for some of the tools.


Tool Summary
------------

bfu
	bonwick/faulkner upgrade. Loads a set of cpio archives created
	by 'mkbfu' onto a machine, either live or on alternate root
	and /usr filesystems. Attempts to preserve important files,
	but may require manual intervention before reboot to resolve
	changes to preserved files.

bfuld
	Used by bfu to survive getting a new runtime linker when extracting
	new cpio archives onto a live system. Patches binaries to use
	a saved runtime linker in /tmp during the bfu process.
	Not run by anything but bfu.

bldenv
	companion to 'nightly.' Takes the same environment file you
	used with 'nightly,' and starts a shell with the environment
	set up the same way as 'nightly' set it up. This is useful
	if you're trying to quickly rebuild portions of a workspace
	built by 'nightly'. 'ws' should not be used for this since it
	sets the environment up differently and may cause everything
	to rebuild (because of different -I or -L paths).

build_cscope
	builds cscope databases in the uts, the platform subdirectories
	of uts, and in usr/src. Uses cscope-fast.

cdm
	A Mercurial extension providing various commands useful for ON
	development

check_rtime
	checks ELF attributes used by ELF dynamic objects in the proto area.
	Used by 'nightly's -r option, to check a number of ELF runtime
	attributes for consistency with common build rules.  nightly uses
	the -o option to simplify the output for diffing with previous
	build results.  It also uses the -i option to obtain NEEDED and RUNPATH
	entries, which help detect changes in software dependencies and makes
	sure objects don't have any strange runpaths like /opt/SUNWspro/lib.

checkproto
	Runs protocmp and protolist on a workspace (or uses the environment
	variable CODEMGR_WS to determine the workspace). Checks the proto area
	against the packages.

codereview
	Given two filenames, creates a postscript file with the file
	differences highlighted.

codesign
	Tools for signing cryptographic modules using the official
	Sun release keys stored on a remote signing server. This
	directory contains signit, a client program for signing
	files with the signing server; signproto, a shell script
	that finds crypto modules in $ROOT and signs them using
	signit; and codesign_server.pl, the code that runs on the
	server. The codesign_server code is not used on an ON
	build machine but is kept here for source control purposes.

copyrightchk
	Checks that files have appropriate SMI copyright notices.
	Primarily used by wx

cscope-fast
	The fast version of cscope that we use internally. Seems to work,
	but may need more testing before it's placed in the gate. The source
	just really needs to be here.

cstyle
	checks C source for compliance with OS/Net guidelines.

ctfconvert
	Convert symbolic debugging information in an object file to the Compact
	ANSI-C Type Format (CTF).

ctfdump
	Decode and display CTF data stored in a raw file or in an ELF file.

ctfmerge
	Merge the CTF data from one or more object files.

depcheck
	A tool to try an assess the dependencies of executables.  This tool
	is not a definitive dependency check, but it does use "strings" and
	"ldd" to gather as much information as it can.  The dependency check
	tool can handle filenames and pkgnames.  Before using the dependency
	checker you must build a database which reflects the properties and
	files in your system.

elfcmp
	Compares two ELF modules (e.g. .o files, executables) section by
	section.  Useful for determining whether "trivial" changes -
	cstyle, lint, etc - actually changed the code.  The -S option
	is used to test whether two binaries are the same except for
	the elfsign signature.

elfsign
	Built from the same sources as the shipped elfsign(1), this
	version is used in nightly -t builds to assure that the signing
	process and format is the same as will be used on the target
	system.

elfsigncmp
	This script can be used in lieu of elfsign during a build.
	It uses elfsign to sign a copy of the object and elfcmp -S to
	verify that the signing caused no damage before updating
	the object to be signed.

find_elf
	Search a directory tree for ELF objects, and produce one line of
	output per object. Used by check_rtime and interface_check to locate
	the objects to examine.

findunref
	Finds all files in a source tree that have access times older than a
	certain time and are not in a specified list of exceptions.  Since
	'nightly' timestamps the start of the build, and findunref uses its
	timestamp (by default), this can be used to find all files that were
	unreferenced during a nightly build).  Since some files are only used
	during a SPARC or Intel build, 'findunref' needs to be run on
	workspaces from both architectures and the results need to be merged.
	For instance, if $INTELSRC and $SPARCSRC are set to the usr/src
	directories of your Intel and SPARC nightly workspaces, then you
	can merge the results like so:

	$ findunref $INTELSRC $INTELSRC/tools/findunref/exception_list | \
	  sort > ~/unref-i386.out
	$ findunref $SPARCSRC $SPARCSRC/tools/findunref/exception_list | \
	  sort > ~/unref-sparc.out
	$ comm -12 ~/unref-i386.out ~/unref-sparc.out > ~/unref.out

hdrchk
	checks headers for compliance with OS/Net standards (form, includes,
	C++ guards).

hgsetup
	creates a basic Mercurial configuration for the user.

hg-active
	helper used by webrev to generate file lists for Mercurial
	workspaces.

install.bin
	binary version of /usr/sbin/install. Used to be vastly faster
	(since /usr/sbin/install is a shell script), but may only be a bit
	faster now. One speedup includes avoiding the name service for the
	well-known, never-changing password entries like 'root' and 'sys.'

interface_check
	detects and reports invalid versioning in ELF objects.
	Optionally generates an interface description file for
	the workspace.

interface_cmp
	Compares two interface description files, as produced by
	interface_check, and flags invalid deviations in ELF object
	versioning between them. interface_cmp can be used between Solaris
	gates to ensure that older releases remain compatible with the
	development gate. It can also be used to validate new changes to
	the development gate before they are integrated.

lintdump
	dumps the contents of one or more lint libraries; see lintdump(1)

keywords
	checks files for proper SCCS keywords.

makebfu
	simple wrapper around 'mkbfu' for use outside nightly (when in a build
	shell from 'ws' or 'bldenv').

mkbfu
	makes cpio archives out of the proto area suitable for bfu'ing.
	Used by 'nightly' and 'makebfu'.

ndrgen
	Network Data Language (NDL) RPC protocol compiler to support DCE
	RPC/MSRPC and SMB/CIFS.  ndrgen takes an input protocol definition
	file (say, proto.ndl) and generates an output C source file
	(proto_ndr.c) containing the Network Data Representation (NDR)
	marshalling routines to implement the RPC protocol.

nightly
	nightly build script. Takes an environment (or 'env') file describing
	such things as the workspace, the parent, and what to build. See
	env/developer and env/gatekeeper for sample, hopefully well-commented
	env files.

pmodes
	enforces proper file ownership and permissions in pkgmap and package
	prototype* files.  converts files if necessary

protocmp
	compares proto lists and the package definitions. Used by nightly
	to determine if the proto area matches the packages, and to detect
	differences between a childs proto area and a parents.

protocmp.terse
	transforms the output of protocmp into something a bit more friendly

protolist
	create a list of what's in the proto area, to feed to protocmp.

rtichk
	checks that a set of CRs have approved RTIs.  Primarily used
	by wx

sccscp
	copy a file under SCCS control to another location in a workspace.
	also updates teamware's nametable.

sccshist
	Display the history, comments and diffs, of a file under SCCS
	control.

sccsmv
	rename a file under SCCS control to another location in a workspace.
	also updates teamware's nametable.

sccsrm
	delete a file under SCCS control workspace. also updates teamware's
	nametable. Actually renames it to .del-<file>-`date` so that others
	will see it move when it is brought over (in case they were working
	on it).

ws
	creates a shell with the environment set up to build in the given
	workspace. Used mostly for non-full-build workspaces, so it sets up
	to pull headers and libraries from the proto area of the parent if
	they aren't in the childs proto area.

wx
	A great workspace tool by bonwick. See wx.README for information
	and warnings.

wx2hg
	Converts a TeamWare workspace under the control of wx to a
	Mercurial workspace, discarding intermediate deltas.

tokenize
	Used to build the sun4u boot block.

webrev
	Generates a set of HTML pages that show side-by-side diffs of
	changes in your workspace, for easy communication of code
	review materials.  Can automagically find edited files or use a
	manually-generated list; knows how to use wx's active file for
	lists of checked-out files and proposed SCCS comments.

which_scm
	Reports the current Source Code Management (SCM) system in use
	and the top-level directory of the workspace.

wsdiff
	Detect object differences between two ON proto areas. Used by
	nightly(1) to determine what changed between two builds. Handy
	for identifying the set of built objects impacted by a given
	source change. This information is needed for patch construction.


How to do a full build
----------------------

1. Find an environment file that might do what you want to do. If you're just
   a developer wanting to do a full build in a child of the gate, copy the
   'developer' environment file to a new name (private to you and/or the
   work being done in this workspace, to avoid collisions with others). Then
   edit the file and tailor it to your workspace. Remember that this file
   is a shell script, so it can do more than set environment variables.

2. Login as 'gk' (or root, but your PATH and .make.machines for dmake will
   not be right). Run 'nightly' and give it your environment file as an
   option. 'nightly' will first look for your environment file in
   /opt/onbld/env, and if it's not there then it will look for it as an
   absolute or relative path. Some people put their environment files in
   their workspace to keep them close.

3. When 'nightly' is complete, it will send a summary of what happened to
   $MAILTO. Usually, the less info in the mail the better. If you have failures,
   you can go look at the full log of what happened, generally in
   $CODEMGR_WS/log/log.<date>/nightly.log (the mail_msg it sent and the proto
   list are there too). You can also find the individual build logs, like
   'make clobber' and 'make install' output in $SRC, under names like
   clobber-${MACH}.out and install-${MACH}.out (for a DEBUG build). These
   will be smaller than nightly.log, and maybe more searchable.

Files you have to update to add a tool
--------------------------------------

1.  Add the tool in its appropriate place.
2.  Update the Makefile as required.
3.  Update usr/src/tools/SUNWonbld/prototype_*.
4.  Update usr/src/tools/README.tools (this file).
5.  Repeat 1-4 for any man pages.