xref: /titanic_52/usr/src/tools/README.tools (revision c84b7bbef5ecc2a27799422588073deefd9db715)
1#
2# CDDL HEADER START
3#
4# The contents of this file are subject to the terms of the
5# Common Development and Distribution License (the "License").
6# You may not use this file except in compliance with the License.
7#
8# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
9# or http://www.opensolaris.org/os/licensing.
10# See the License for the specific language governing permissions
11# and limitations under the License.
12#
13# When distributing Covered Code, include this CDDL HEADER in each
14# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
15# If applicable, add the following below this CDDL HEADER, with the
16# fields enclosed by brackets "[]" replaced with your own identifying
17# information: Portions Copyright [yyyy] [name of copyright owner]
18#
19# CDDL HEADER END
20#
21#
22# Copyright (c) 1999, 2010, Oracle and/or its affiliates. All rights reserved.
23
24
25This directory contains the tools used to do a full build of the
26OS/Net workspace.  They usually live in the /opt/onbld directory on build
27machines. From here, 'make install' will build and install the tools
28in $ROOT/opt/onbld. If you like, 'make pkg' will build the SUNWonbld
29package in $(PKGARCHIVE). Installing that package will populate the
30/opt/onbld directory, and create a root account for building called 'gk',
31which uses csh and has a home directory of /opt/onbld/gk. You can
32use this account to do full builds with 'nightly'. You don't have to,
33but the 'gk' account has the path setup properly, has a .make.machines
34file for dmake, and has a .login that sets up for dmake.
35
36Layout of /opt/onbld
37--------------------
38
39/opt/onbld/etc/abi
40	contains Solaris ABI database (ABI_*.db) and exceptions
41	for ABI Auditing tool (interface_check, interface_cmp).
42
43/opt/onbld/gk
44	gk account's home directory.
45
46/opt/onbld/bin
47	basic bin directory - contains scripts.
48
49/opt/onbld/bin/${MACH}
50	architecture-specific bin directory for binaries.
51
52/opt/onbld/env
53	build environment files.
54
55/opt/onbld/lib
56	libraries used by the build tools.
57
58/opt/onbld/lib/python<version>/
59	python modules used by the build tools.
60
61/opt/onbld/lib/python<version>/onbld/hgext
62	Mercurial extensions.
63
64/opt/onbld/lib/python/
65        symlink to the modules directory of the currently preferred
66        python version.  This exists to retain compatibility both for
67        tools expecting only one supported version of python, and for
68        user .hgrc files that expect to find cdm.py in
69        /opt/onbld/lib/python/onbld/hgext.
70
71/opt/onbld/man
72	rudimentary man pages for some of the tools.
73
74
75Tool Summary
76------------
77
78bfu
79	bonwick/faulkner upgrade. Loads a set of cpio archives created
80	by 'mkbfu' onto a machine, either live or on alternate root
81	and /usr filesystems. Attempts to preserve important files,
82	but may require manual intervention before reboot to resolve
83	changes to preserved files.
84
85bfuld
86	Used by bfu to survive getting a new runtime linker when extracting
87	new cpio archives onto a live system. Patches binaries to use
88	a saved runtime linker in /tmp during the bfu process.
89	Not run by anything but bfu.
90
91bldenv
92	companion to 'nightly.' Takes the same environment file you
93	used with 'nightly,' and starts a shell with the environment
94	set up the same way as 'nightly' set it up. This is useful
95	if you're trying to quickly rebuild portions of a workspace
96	built by 'nightly'. 'ws' should not be used for this since it
97	sets the environment up differently and may cause everything
98	to rebuild (because of different -I or -L paths).
99
100build_cscope
101	builds cscope databases in the uts, the platform subdirectories
102	of uts, and in usr/src. Uses cscope-fast.
103
104cdm
105	A Mercurial extension providing various commands useful for ON
106	development
107
108check_rtime
109	checks ELF attributes used by ELF dynamic objects in the proto area.
110	Used by 'nightly's -r option, to check a number of ELF runtime
111	attributes for consistency with common build rules.  nightly uses
112	the -o option to simplify the output for diffing with previous
113	build results.  It also uses the -i option to obtain NEEDED and RUNPATH
114	entries, which help detect changes in software dependencies and makes
115	sure objects don't have any strange runpaths like /opt/SUNWspro/lib.
116
117checkproto
118	Runs protocmp and protolist on a workspace (or uses the environment
119	variable CODEMGR_WS to determine the workspace). Checks the proto area
120	against the packages.
121
122codereview
123	Given two filenames, creates a postscript file with the file
124	differences highlighted.
125
126codesign
127	Tools for signing cryptographic modules using the official
128	Sun release keys stored on a remote signing server. This
129	directory contains signit, a client program for signing
130	files with the signing server; signproto, a shell script
131	that finds crypto modules in $ROOT and signs them using
132	signit; and codesign_server.pl, the code that runs on the
133	server. The codesign_server code is not used on an ON
134	build machine but is kept here for source control purposes.
135
136copyrightchk
137	Checks that files have appropriate SMI copyright notices.
138	Primarily used by wx
139
140cscope-fast
141	The fast version of cscope that we use internally. Seems to work,
142	but may need more testing before it's placed in the gate. The source
143	just really needs to be here.
144
145cstyle
146	checks C source for compliance with OS/Net guidelines.
147
148ctfconvert
149	Convert symbolic debugging information in an object file to the Compact
150	ANSI-C Type Format (CTF).
151
152ctfdump
153	Decode and display CTF data stored in a raw file or in an ELF file.
154
155ctfmerge
156	Merge the CTF data from one or more object files.
157
158depcheck
159	A tool to try an assess the dependencies of executables.  This tool
160	is not a definitive dependency check, but it does use "strings" and
161	"ldd" to gather as much information as it can.  The dependency check
162	tool can handle filenames and pkgnames.  Before using the dependency
163	checker you must build a database which reflects the properties and
164	files in your system.
165
166elfcmp
167	Compares two ELF modules (e.g. .o files, executables) section by
168	section.  Useful for determining whether "trivial" changes -
169	cstyle, lint, etc - actually changed the code.  The -S option
170	is used to test whether two binaries are the same except for
171	the elfsign signature.
172
173elfsign
174	Built from the same sources as the shipped elfsign(1), this
175	version is used in nightly -t builds to assure that the signing
176	process and format is the same as will be used on the target
177	system.
178
179elfsigncmp
180	This script can be used in lieu of elfsign during a build.
181	It uses elfsign to sign a copy of the object and elfcmp -S to
182	verify that the signing caused no damage before updating
183	the object to be signed.
184
185find_elf
186	Search a directory tree for ELF objects, and produce one line of
187	output per object. Used by check_rtime and interface_check to locate
188	the objects to examine.
189
190findunref
191	Finds all files in a source tree that have access times older than a
192	certain time and are not in a specified list of exceptions.  Since
193	'nightly' timestamps the start of the build, and findunref uses its
194	timestamp (by default), this can be used to find all files that were
195	unreferenced during a nightly build).  Since some files are only used
196	during a SPARC or Intel build, 'findunref' needs to be run on
197	workspaces from both architectures and the results need to be merged.
198	For instance, if $INTELSRC and $SPARCSRC are set to the usr/src
199	directories of your Intel and SPARC nightly workspaces, then you
200	can merge the results like so:
201
202	$ findunref $INTELSRC $INTELSRC/tools/findunref/exception_list | \
203	  sort > ~/unref-i386.out
204	$ findunref $SPARCSRC $SPARCSRC/tools/findunref/exception_list | \
205	  sort > ~/unref-sparc.out
206	$ comm -12 ~/unref-i386.out ~/unref-sparc.out > ~/unref.out
207
208hdrchk
209	checks headers for compliance with OS/Net standards (form, includes,
210	C++ guards).
211
212hgsetup
213	creates a basic Mercurial configuration for the user.
214
215hg-active
216	helper used by webrev to generate file lists for Mercurial
217	workspaces.
218
219install.bin
220	binary version of /usr/sbin/install. Used to be vastly faster
221	(since /usr/sbin/install is a shell script), but may only be a bit
222	faster now. One speedup includes avoiding the name service for the
223	well-known, never-changing password entries like 'root' and 'sys.'
224
225interface_check
226	detects and reports invalid versioning in ELF objects.
227	Optionally generates an interface description file for
228	the workspace.
229
230interface_cmp
231	Compares two interface description files, as produced by
232	interface_check, and flags invalid deviations in ELF object
233	versioning between them. interface_cmp can be used between Solaris
234	gates to ensure that older releases remain compatible with the
235	development gate. It can also be used to validate new changes to
236	the development gate before they are integrated.
237
238lintdump
239	dumps the contents of one or more lint libraries; see lintdump(1)
240
241keywords
242	checks files for proper SCCS keywords.
243
244makebfu
245	simple wrapper around 'mkbfu' for use outside nightly (when in a build
246	shell from 'ws' or 'bldenv').
247
248mkbfu
249	makes cpio archives out of the proto area suitable for bfu'ing.
250	Used by 'nightly' and 'makebfu'.
251
252ndrgen
253	Network Data Language (NDL) RPC protocol compiler to support DCE
254	RPC/MSRPC and SMB/CIFS.  ndrgen takes an input protocol definition
255	file (say, proto.ndl) and generates an output C source file
256	(proto_ndr.c) containing the Network Data Representation (NDR)
257	marshalling routines to implement the RPC protocol.
258
259nightly
260	nightly build script. Takes an environment (or 'env') file describing
261	such things as the workspace, the parent, and what to build. See
262	env/developer and env/gatekeeper for sample, hopefully well-commented
263	env files.
264
265pmodes
266	enforces proper file ownership and permissions in pkgmap and package
267	prototype* files.  converts files if necessary
268
269protocmp
270	compares proto lists and the package definitions. Used by nightly
271	to determine if the proto area matches the packages, and to detect
272	differences between a childs proto area and a parents.
273
274protocmp.terse
275	transforms the output of protocmp into something a bit more friendly
276
277protolist
278	create a list of what's in the proto area, to feed to protocmp.
279
280rtichk
281	checks that a set of CRs have approved RTIs.  Primarily used
282	by wx
283
284sccscp
285	copy a file under SCCS control to another location in a workspace.
286	also updates teamware's nametable.
287
288sccshist
289	Display the history, comments and diffs, of a file under SCCS
290	control.
291
292sccsmv
293	rename a file under SCCS control to another location in a workspace.
294	also updates teamware's nametable.
295
296sccsrm
297	delete a file under SCCS control workspace. also updates teamware's
298	nametable. Actually renames it to .del-<file>-`date` so that others
299	will see it move when it is brought over (in case they were working
300	on it).
301
302ws
303	creates a shell with the environment set up to build in the given
304	workspace. Used mostly for non-full-build workspaces, so it sets up
305	to pull headers and libraries from the proto area of the parent if
306	they aren't in the childs proto area.
307
308wx
309	A great workspace tool by bonwick. See wx.README for information
310	and warnings.
311
312wx2hg
313	Converts a TeamWare workspace under the control of wx to a
314	Mercurial workspace, discarding intermediate deltas.
315
316tokenize
317	Used to build the sun4u boot block.
318
319webrev
320	Generates a set of HTML pages that show side-by-side diffs of
321	changes in your workspace, for easy communication of code
322	review materials.  Can automagically find edited files or use a
323	manually-generated list; knows how to use wx's active file for
324	lists of checked-out files and proposed SCCS comments.
325
326which_scm
327	Reports the current Source Code Management (SCM) system in use
328	and the top-level directory of the workspace.
329
330wsdiff
331	Detect object differences between two ON proto areas. Used by
332	nightly(1) to determine what changed between two builds. Handy
333	for identifying the set of built objects impacted by a given
334	source change. This information is needed for patch construction.
335
336
337How to do a full build
338----------------------
339
3401. Find an environment file that might do what you want to do. If you're just
341   a developer wanting to do a full build in a child of the gate, copy the
342   'developer' environment file to a new name (private to you and/or the
343   work being done in this workspace, to avoid collisions with others). Then
344   edit the file and tailor it to your workspace. Remember that this file
345   is a shell script, so it can do more than set environment variables.
346
3472. Login as 'gk' (or root, but your PATH and .make.machines for dmake will
348   not be right). Run 'nightly' and give it your environment file as an
349   option. 'nightly' will first look for your environment file in
350   /opt/onbld/env, and if it's not there then it will look for it as an
351   absolute or relative path. Some people put their environment files in
352   their workspace to keep them close.
353
3543. When 'nightly' is complete, it will send a summary of what happened to
355   $MAILTO. Usually, the less info in the mail the better. If you have failures,
356   you can go look at the full log of what happened, generally in
357   $CODEMGR_WS/log/log.<date>/nightly.log (the mail_msg it sent and the proto
358   list are there too). You can also find the individual build logs, like
359   'make clobber' and 'make install' output in $SRC, under names like
360   clobber-${MACH}.out and install-${MACH}.out (for a DEBUG build). These
361   will be smaller than nightly.log, and maybe more searchable.
362
363Files you have to update to add a tool
364--------------------------------------
365
3661.  Add the tool in its appropriate place.
3672.  Update the Makefile as required.
3683.  Update usr/src/tools/SUNWonbld/prototype_*.
3694.  Update usr/src/tools/README.tools (this file).
3705.  Repeat 1-4 for any man pages.
371