1.\" Copyright (c) 2010-2012 Advanced Computing Technologies LLC 2.\" Written by: John H. Baldwin <jhb@FreeBSD.org> 3.\" All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.\" $FreeBSD$ 27.\" 28.Dd March 16, 2012 29.Dt ETCUPDATE 8 30.Os 31.Sh NAME 32.Nm etcupdate 33.Nd "manage updates to system files not updated by installworld" 34.Sh SYNOPSIS 35.Nm 36.Op Fl nBF 37.Op Fl d Ar workdir 38.Op Fl r | Fl s Ar source | Fl t Ar tarball 39.Op Fl A Ar patterns 40.Op Fl D Ar destdir 41.Op Fl I Ar patterns 42.Op Fl L Ar logfile 43.Op Fl M Ar options 44.Nm 45.Cm build 46.Op Fl B 47.Op Fl d Ar workdir 48.Op Fl s Ar source 49.Op Fl L Ar logfile 50.Op Fl M Ar options 51.Ar tarball 52.Nm 53.Cm diff 54.Op Fl d Ar workdir 55.Op Fl D Ar destdir 56.Op Fl I Ar patterns 57.Op Fl L Ar logfile 58.Nm 59.Cm extract 60.Op Fl B 61.Op Fl d Ar workdir 62.Op Fl s Ar source | Fl t Ar tarball 63.Op Fl L Ar logfile 64.Op Fl M Ar options 65.Nm 66.Cm resolve 67.Op Fl d Ar workdir 68.Op Fl D Ar destdir 69.Op Fl L Ar logfile 70.Nm 71.Cm status 72.Op Fl d Ar workdir 73.Op Fl D Ar destdir 74.Sh DESCRIPTION 75The 76.Nm 77utility is a tool for managing updates to files that are not updated as 78part of 79.Sq make installworld 80such as files in 81.Pa /etc . 82It manages updates by doing a three-way merge of changes made to these 83files against the local versions. 84It is also designed to minimize the amount of user intervention with 85the goal of simplifying upgrades for clusters of machines. 86.Pp 87To perform a three-way merge, 88.Nm 89keeps copies of the current and previous versions of files that it manages. 90These copies are stored in two trees known as the 91.Dq current 92and 93.Dq previous 94trees. 95During a merge, 96.Nm 97compares the 98.Dq current 99and 100.Dq previous 101copies of each file to determine which changes need to be merged into the 102local version of each file. 103If a file can be updated without generating a conflict, 104.Nm 105will update the file automatically. 106If the local changes to a file conflict with the changes made to a file in 107the source tree, 108then a merge conflict is generated. 109The conflict must be resolved after the merge has finished. 110The 111.Nm 112utility will not perform a new merge until all conflicts from an earlier 113merge are resolved. 114.Sh MODES 115The 116.Nm 117utility supports several modes of operation. 118The mode is specified via an optional command argument. 119If present, the command must be the first argument on the command line. 120If a command is not specified, the default mode is used. 121.Ss Default Mode 122The default mode merges changes from the source tree to the destination 123directory. 124First, 125it updates the 126.Dq current 127and 128.Dq previous 129trees. 130Next, 131it compares the two trees merging changes into the destination directory. 132Finally, 133it displays warnings for any conditions it could not handle automatically. 134.Pp 135If the 136.Fl r 137option is not specified, 138then the first step taken is to update the 139.Dq current 140and 141.Dq previous 142trees. 143If a 144.Dq current 145tree already exists, 146then that tree is saved as the 147.Dq previous 148tree. 149An older 150.Dq previous 151tree is removed if it exists. 152By default the new 153.Dq current 154tree is built from a source tree. 155However, 156if a tarball is specified via the 157.Fl t 158option, 159then the tree is extracted from that tarball instead. 160.Pp 161Next, 162.Nm 163compares the files in the 164.Dq current 165and 166.Dq previous 167trees. 168If a file was removed from the 169.Dq current 170tree, 171then it will be removed from the destination directory only if it 172does not have any local modifications. 173If a file was added to the 174.Dq current 175tree, 176then it will be copied to the destination directory only if it 177would not clobber an existing file. 178If a file is changed in the 179.Dq current 180tree, 181then 182.Nm 183will attempt to merge the changes into the version of the file in the 184destination directory. 185If the merge encounters conflicts, 186then a version of the file with conflict markers will be saved for 187future resolution. 188If the merge does not encounter conflicts, 189then the merged version of the file will be saved in the destination 190directory. 191If 192.Nm 193is not able to safely merge in changes to a file other than a merge conflict, 194it will generate a warning. 195.Pp 196For each file that is updated a line will be output with a leading character 197to indicate the action taken. 198The possible actions follow: 199.Pp 200.Bl -tag -width "A" -compact -offset indent 201.It A 202Added 203.It C 204Conflict 205.It D 206Deleted 207.It M 208Merged 209.It U 210Updated 211.El 212.Pp 213Finally, 214if any warnings were encountered they are displayed after the merge has 215completed. 216.Pp 217Note that for certain files 218.Nm 219will perform post-install actions any time that the file is updated. 220Specifically, 221.Xr pwd_mkdb 8 222is invoked if 223.Pa /etc/master.passwd 224is changed, 225.Xr cap_mkdb 1 226is invoked to update 227.Pa /etc/login.conf.db 228if 229.Pa /etc/login.conf 230is changed, 231.Xr newaliases 1 232is invoked if 233.Pa /etc/mail/aliases 234is changed, 235and 236.Pa /etc/rc.d/motd 237is invoked if 238.Pa /etc/motd 239is changed. 240One exception is that if 241.Pa /etc/mail/aliases 242is changed and the destination directory is not the default, 243then a warning will be issued instead. 244This is due to a limitation of the 245.Xr newaliases 1 246command. 247Similarly, 248if 249.Pa /etc/motd 250is changed and the destination directory is not the default, 251then 252.Pa /etc/rc.d/motd 253will not be executed due to a limitation of that script. 254In this case no warning is issued as the result of 255.Pa /etc/rc.d/motd 256is merely cosmetic and will be corrected on the next reboot. 257.Ss Build Mode 258The 259.Cm build 260mode is used to build a tarball that contains a snapshot of a 261.Dq current 262tree. 263This tarball can be used by the default and extract modes. 264Using a tarball can allow 265.Nm 266to perform a merge without requiring a source tree that matches the 267currently installed world. 268The 269.Fa tarball 270argument specifies the name of the file to create. 271The file will be a 272.Xr tar 5 273file compressed with 274.Xr bzip2 1 . 275.Ss Diff Mode 276The 277.Cm diff 278mode compares the versions of files in the destination directory to the 279.Dq current 280tree and generates a unified format diff of the changes. 281This can be used to determine which files have been locally modified and how. 282Note that 283.Nm 284does not manage files that are not maintained in the source tree such as 285.Pa /etc/fstab 286and 287.Pa /etc/rc.conf . 288.Ss Extract Mode 289The 290.Cm extract 291mode generates a new 292.Dq current 293tree. 294Unlike the default mode, 295it does not save any existing 296.Dq current 297tree and does not modify any existing 298.Dq previous 299tree. 300The new 301.Dq current 302tree can either be built from a source tree or extracted from a tarball. 303.Ss Resolve Mode 304The 305.Cm resolve 306mode is used to resolve any conflicts encountered during a merge. 307In this mode, 308.Nm 309iterates over any existing conflicts prompting the user for actions to take 310on each conflicted file. 311For each file, the following actions are available: 312.Pp 313.Bl -tag -width "(tf) theirs-full" -compact 314.It (p) postpone 315Ignore this conflict for now. 316.It (df) diff-full 317Show all changes made to the merged file as a unified diff. 318.It (e) edit 319Change the merged file in an editor. 320.It (r) resolved 321Install the merged version of the file into the destination directory. 322.It (mf) mine-full 323Use the version of the file in the destination directory and ignore any 324changes made to the file in the 325.Dq current 326tree. 327.It (tf) theirs-full 328Use the version of the file from the 329.Dq current 330tree and discard any local changes made to the file. 331.It (h) help 332Display the list of commands. 333.El 334.Ss Status Mode 335The 336.Cm status 337mode shows a summary of the results of the most recent merge. 338First it lists any files for which there are unresolved conflicts. 339Next it lists any warnings generated during the last merge. 340If the last merge did not generate any conflicts or warnings, 341then nothing will be output. 342.Sh OPTIONS 343The following options are available. 344Note that most options do not apply to all modes. 345.Bl -tag -width ".Fl d Ar workdir" 346.It Fl B 347Do not build generated files in a private object tree. 348Instead, 349reuse the generated files from a previously built object tree that matches 350the source tree. 351This can be useful to avoid gratuitous conflicts in 352.Xr sendmail 8 353configuration 354files when bootstrapping. 355It can also be useful for building a tarball that matches a specific 356world build. 357.It Fl d Ar workdir 358Specify an alternate directory to use as the work directory. 359The work directory is used to store the 360.Dq current 361and 362.Dq previous 363trees as well as unresolved conflicts. 364The default work directory is 365.Pa <destdir>/var/db/etcupdate . 366.It Fl A Ar patterns 367Always install the new version of any files that match any of the patterns 368listed in 369.Ar patterns . 370Each pattern is evaluated as an 371.Xr sh 1 372shell pattern. 373This option may be specified multiple times to specify multiple patterns. 374Multiple space-separated patterns may also be specified in a single 375option. 376Note that ignored files specified via the 377.Ev IGNORE_FILES 378variable or the 379.Fl I 380option will not be installed. 381.It Fl D Ar destdir 382Specify an alternate destination directory as the target of a merge. 383This is analogous to the 384.Dv DESTDIR 385variable used with 386.Sq make installworld . 387The default destination directory is an empty string which results in 388merges updating 389.Pa /etc 390on the local machine. 391.It Fl F 392Ignore changes in the FreeBSD ID string when comparing files in the 393destination directory to files in either of the 394.Dq current 395or 396.Dq previous 397trees. 398In 399.Cm diff 400mode, 401this reduces noise due to FreeBSD ID string changes in the output. 402During an update this can simplify handling for harmless conflicts caused 403by FreeBSD ID string changes. 404.Pp 405Specifically, 406if a file in the destination directory is identical to the same file in the 407.Dq previous 408tree modulo the FreeBSD ID string, 409then the file is treated as if it was unmodified and the 410.Dq current 411version of the file will be installed. 412Similarly, 413if a file in the destination directory is identical to the same file in the 414.Dq current 415tree modulo the FreeBSD ID string, 416then the 417.Dq current 418version of the file will be installed to update the ID string. 419If the 420.Dq previous 421and 422.Dq current 423versions of the file are identical, 424then 425.Nm 426will not change the file in the destination directory. 427.Pp 428Due to limitations in the 429.Xr diff 1 430command, 431this option may not have an effect if there are other changes in a file that 432are close to the FreeBSD ID string. 433.It Fl I Ar patterns 434Ignore any files that match any of the patterns listed in 435.Ar patterns . 436No warnings or other messages will be generated for those files during a 437merge. 438Each pattern is evaluated as an 439.Xr sh 1 440shell pattern. 441This option may be specified multiple times to specify multiple patterns. 442Multiple space-separated patterns may also be specified in a single 443option. 444.It Fl L Ar logfile 445Specify an alternate path for the log file. 446The 447.Nm 448utility logs each command that it invokes along with the standard output 449and standard error to this file. 450By default the log file is stored in a file named 451.Pa log 452in the work directory. 453.It Fl M Ar options 454Pass 455.Ar options 456as additional parameters to 457.Xr make 1 458when building a 459.Dq current 460tree. 461This can be used for to set the 462.Dv TARGET 463or 464.Dv TARGET_ARCH 465variables for a cross-build. 466.It Fl n 467Enable 468.Dq dry-run 469mode. 470Do not merge any changes to the destination directory. 471Instead, 472report what actions would be taken during a merge. 473Note that the existing 474.Dq current 475and 476.Dq previous 477trees will not be changed. 478If the 479.Fl r 480option is not specified, 481then a temporary 482.Dq current 483tree will be extracted to perform the comparison. 484.It Fl r 485Do not update the 486.Dq current 487and 488.Dq previous 489trees during a merge. 490This can be used to 491.Dq re-run 492a previous merge operation. 493.It Fl s Ar source 494Specify an alternate source tree to use when building or extracting a 495.Dq current 496tree. 497The default source tree is 498.Pa /usr/src . 499.It Fl t Ar tarball 500Extract a new 501.Dq current 502tree from a tarball previously generated by the 503.Cm build 504command rather than building the tree from a source tree. 505.El 506.Sh CONFIG FILE 507The 508.Nm 509utility can also be configured by setting variables in an optional 510configuration file named 511.Pa /etc/etcupdate.conf . 512Note that command line options override settings in the configuration file. 513The configuration file is executed by 514.Xr sh 1 , 515so it uses that syntax to set configuration variables. 516The following variables can be set: 517.Bl -tag -width ".Ev ALWAYS_INSTALL" 518.It Ev ALWAYS_INSTALL 519Always install files that match any of the patterns listed in this variable 520similar to the 521.Fl A 522option. 523.It Ev DESTDIR 524Specify an alternate destination directory similar to the 525.Fl D 526option. 527.It Ev EDITOR 528Specify a program to edit merge conflicts. 529.It Ev FREEBSD_ID 530Ignore changes in the FreeBSD ID string similar to the 531.Fl F 532option. 533This is enabled by setting the variable to a non-empty value. 534.It Ev IGNORE_FILES 535Ignore files that match any of the patterns listed in this variable 536similar to the 537.Fl I 538option. 539.It Ev LOGFILE 540Specify an alternate path for the log file similar to the 541.Fl L 542option. 543.It Ev MAKE_OPTIONS 544Pass additional options to 545.Xr make 1 546when building a 547.Dq current 548tree similar to the 549.Fl M 550option. 551.It Ev SRCDIR 552Specify an alternate source tree similar to the 553.Fl s 554option. 555.It Ev WORKDIR 556Specify an alternate work directory similar to the 557.Fl d 558option. 559.El 560.Sh ENVIRONMENT 561The 562.Nm 563utility uses the program identified in the 564.Ev EDITOR 565environment variable to edit merge conflicts. 566If 567.Ev EDITOR 568is not set, 569.Xr vi 1 570is used as the default editor. 571.Sh FILES 572.Bl -tag -width ".Pa /var/db/etcupdate/log" -compact 573.It Pa /etc/etcupdate.conf 574Optional config file. 575.It Pa /var/db/etcupdate 576Default work directory used to store trees and other data. 577.It Pa /var/db/etcupdate/log 578Default log file. 579.El 580.Sh EXIT STATUS 581.Ex -std 582.Sh EXAMPLES 583If the source tree matches the currently installed world, 584then the following can be used to bootstrap 585.Nm 586so that it can be used for future upgrades: 587.Pp 588.Dl "etcupdate extract" 589.Pp 590To merge changes after an upgrade via the buildworld and installworld process: 591.Pp 592.Dl "etcupdate" 593.Pp 594To resolve any conflicts generated during a merge: 595.Pp 596.Dl "etcupdate resolve" 597.Sh DIAGNOSTICS 598The following warning messages may be generated during a merge. 599Note that several of these warnings cover obscure cases that should occur 600rarely if at all in practice. 601For example, 602if a file changes from a file to a directory in the 603.Dq current 604tree 605and the file was modified in the destination directory, 606then a warning will be triggered. 607In general, 608when a warning references a pathname, 609the corresponding file in the destination directory is not changed by a 610merge operation. 611.Bl -diag 612.It "Directory mismatch: <path> (<type>)" 613An attempt was made to create a directory at 614.Pa path 615but an existing file of type 616.Dq type 617already exists for that path name. 618.It "Modified link changed: <file> (<old> became <new>)" 619The target of a symbolic link named 620.Pa file 621was changed from 622.Dq old 623to 624.Dq new 625in the 626.Dq current 627tree. 628The symbolic link has been modified to point to a target that is neither 629.Dq old 630nor 631.Dq new 632in the destination directory. 633.It "Modified mismatch: <file> (<new> vs <dest>)" 634A file named 635.Pa file 636of type 637.Dq new 638was modified in the 639.Dq current 640tree, 641but the file exists as a different type 642.Dq dest 643in the destination directory. 644.It "Modified <type> changed: <file> (<old> became <new>)" 645A file named 646.Pa file 647changed type from 648.Dq old 649in the 650.Dq previous 651tree to type 652.Dq new 653in the 654.Dq current 655tree. 656The file in the destination directory of type 657.Dq type 658has been modified, 659so it could not be merged automatically. 660.It "Modified <type> remains: <file>" 661The file of type 662.Dq type 663named 664.Pa file 665has been removed from the 666.Dq current 667tree, 668but it has been locally modified. 669The modified version of the file remains in the destination directory. 670.It "Needs update: /etc/mail/aliases.db (required manual update via newaliases(1))" 671The file 672.Pa /etc/mail/aliases 673was updated during a merge with a non-empty destination directory. 674Due to a limitation of the 675.Xr newaliases 1 676command, 677.Nm 678was not able to automatically update the corresponding aliases database. 679.It "New file mismatch: <file> (<new> vs <dest>)" 680A new file named 681.Pa file 682of type 683.Dq new 684has been added to the 685.Dq current 686tree. 687A file of that name already exists in the destination directory, 688but it is of a different type 689.Dq dest . 690.It "New link conflict: <file> (<new> vs <dest>)" 691A symbolic link named 692.Pa file 693has been added to the 694.Dq current 695tree that links to 696.Dq new . 697A symbolic link of the same name already exists in the destination 698directory, 699but it links to a different target 700.Dq dest . 701.It "Non-empty directory remains: <file>" 702The directory 703.Pa file 704was removed from the 705.Dq current 706tree, 707but it contains additional files in the destination directory. 708These additional files as well as the directory remain. 709.It "Remove mismatch: <file> (<old> became <new>)" 710A file named 711.Pa file 712changed from type 713.Dq old 714in the 715.Dq previous 716tree to type 717.Dq new 718in the 719.Dq current 720tree, 721but it has been removed in the destination directory. 722.It "Removed file changed: <file>" 723A file named 724.Pa file 725was modified in the 726.Dq current 727tree, 728but it has been removed in the destination directory. 729.It "Removed link changed: <file> (<old> became <new>)" 730The target of a symbolic link named 731.Pa file 732was changed from 733.Dq old 734to 735.Dq new 736in the 737.Dq current 738tree, 739but it has been removed in the destination directory. 740.El 741.Sh SEE ALSO 742.Xr cap_mkdb 1 , 743.Xr diff 1 , 744.Xr make 1 , 745.Xr newaliases 1 , 746.Xr sh 1 , 747.Xr pwd_mkdb 8 748.Sh HISTORY 749The 750.Nm 751utility first appeared in 752.Fx 10.0 . 753.Sh AUTHORS 754The 755.Nm 756utility was written by 757.An John Baldwin Aq jhb@FreeBSD.org . 758.Sh BUGS 759Rerunning a merge does not automatically delete conflicts left over from a 760previous merge. 761Any conflicts must be resolved before the merge can be rerun. 762It it is not clear if this is a feature or a bug. 763.Pp 764There is no way to easily automate conflict resolution for specific files. 765For example, one can imagine a syntax along the lines of 766.Pp 767.Dl "etcupdate resolve tf /some/file" 768.Pp 769to resolve a specific conflict in an automated fashion. 770.Pp 771It might be nice to have something like a 772.Sq revert 773command to replace a locally modified version of a file with the stock 774version of the file. 775For example: 776.Pp 777.Dl "etcupdate revert /etc/mail/freebsd.cf" 778.Pp 779Bootstrapping 780.Nm 781often results in gratuitous diffs in 782.Pa /etc/mail/*.cf 783that cause conflicts in the first merge. 784If an object tree that matches the source tree is present when bootstrapping, 785then passing the 786.Fl B 787flag to the 788.Cm extract 789command can work around this. 790