1.\" Copyright (c) 2010-2013 Advanced Computing Technologies LLC 2.\" Written by: John H. Baldwin <jhb@FreeBSD.org> 3.\" All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.\" $FreeBSD$ 27.\" 28.Dd October 29, 2014 29.Dt ETCUPDATE 8 30.Os 31.Sh NAME 32.Nm etcupdate 33.Nd "manage updates to system files not updated by installworld" 34.Sh SYNOPSIS 35.Nm 36.Op Fl npBF 37.Op Fl d Ar workdir 38.Op Fl r | Fl s Ar source | Fl t Ar tarball 39.Op Fl A Ar patterns 40.Op Fl D Ar destdir 41.Op Fl I Ar patterns 42.Op Fl L Ar logfile 43.Op Fl M Ar options 44.Nm 45.Cm build 46.Op Fl B 47.Op Fl d Ar workdir 48.Op Fl s Ar source 49.Op Fl L Ar logfile 50.Op Fl M Ar options 51.Ar tarball 52.Nm 53.Cm diff 54.Op Fl d Ar workdir 55.Op Fl D Ar destdir 56.Op Fl I Ar patterns 57.Op Fl L Ar logfile 58.Nm 59.Cm extract 60.Op Fl B 61.Op Fl d Ar workdir 62.Op Fl s Ar source | Fl t Ar tarball 63.Op Fl L Ar logfile 64.Op Fl M Ar options 65.Nm 66.Cm resolve 67.Op Fl p 68.Op Fl d Ar workdir 69.Op Fl D Ar destdir 70.Op Fl L Ar logfile 71.Nm 72.Cm status 73.Op Fl d Ar workdir 74.Op Fl D Ar destdir 75.Sh DESCRIPTION 76The 77.Nm 78utility is a tool for managing updates to files that are not updated as 79part of 80.Sq make installworld 81such as files in 82.Pa /etc . 83It manages updates by doing a three-way merge of changes made to these 84files against the local versions. 85It is also designed to minimize the amount of user intervention with 86the goal of simplifying upgrades for clusters of machines. 87.Pp 88To perform a three-way merge, 89.Nm 90keeps copies of the current and previous versions of files that it manages. 91These copies are stored in two trees known as the 92.Dq current 93and 94.Dq previous 95trees. 96During a merge, 97.Nm 98compares the 99.Dq current 100and 101.Dq previous 102copies of each file to determine which changes need to be merged into the 103local version of each file. 104If a file can be updated without generating a conflict, 105.Nm 106will update the file automatically. 107If the local changes to a file conflict with the changes made to a file in 108the source tree, 109then a merge conflict is generated. 110The conflict must be resolved after the merge has finished. 111The 112.Nm 113utility will not perform a new merge until all conflicts from an earlier 114merge are resolved. 115.Sh MODES 116The 117.Nm 118utility supports several modes of operation. 119The mode is specified via an optional command argument. 120If present, the command must be the first argument on the command line. 121If a command is not specified, the default mode is used. 122.Ss Default Mode 123The default mode merges changes from the source tree to the destination 124directory. 125First, 126it updates the 127.Dq current 128and 129.Dq previous 130trees. 131Next, 132it compares the two trees merging changes into the destination directory. 133Finally, 134it displays warnings for any conditions it could not handle automatically. 135.Pp 136If the 137.Fl r 138option is not specified, 139then the first step taken is to update the 140.Dq current 141and 142.Dq previous 143trees. 144If a 145.Dq current 146tree already exists, 147then that tree is saved as the 148.Dq previous 149tree. 150An older 151.Dq previous 152tree is removed if it exists. 153By default the new 154.Dq current 155tree is built from a source tree. 156However, 157if a tarball is specified via the 158.Fl t 159option, 160then the tree is extracted from that tarball instead. 161.Pp 162Next, 163.Nm 164compares the files in the 165.Dq current 166and 167.Dq previous 168trees. 169If a file was removed from the 170.Dq current 171tree, 172then it will be removed from the destination directory only if it 173does not have any local modifications. 174If a file was added to the 175.Dq current 176tree, 177then it will be copied to the destination directory only if it 178would not clobber an existing file. 179If a file is changed in the 180.Dq current 181tree, 182then 183.Nm 184will attempt to merge the changes into the version of the file in the 185destination directory. 186If the merge encounters conflicts, 187then a version of the file with conflict markers will be saved for 188future resolution. 189If the merge does not encounter conflicts, 190then the merged version of the file will be saved in the destination 191directory. 192If 193.Nm 194is not able to safely merge in changes to a file other than a merge conflict, 195it will generate a warning. 196.Pp 197For each file that is updated a line will be output with a leading character 198to indicate the action taken. 199The possible actions follow: 200.Pp 201.Bl -tag -width "A" -compact -offset indent 202.It A 203Added 204.It C 205Conflict 206.It D 207Deleted 208.It M 209Merged 210.It U 211Updated 212.El 213.Pp 214Finally, 215if any warnings were encountered they are displayed after the merge has 216completed. 217.Pp 218Note that for certain files 219.Nm 220will perform post-install actions any time that the file is updated. 221Specifically, 222.Xr pwd_mkdb 8 223is invoked if 224.Pa /etc/master.passwd 225is changed, 226.Xr cap_mkdb 1 227is invoked to update 228.Pa /etc/login.conf.db 229if 230.Pa /etc/login.conf 231is changed, 232.Xr newaliases 1 233is invoked if 234.Pa /etc/mail/aliases 235is changed, 236and 237.Pa /etc/rc.d/motd 238is invoked if 239.Pa /etc/motd 240is changed. 241One exception is that if 242.Pa /etc/mail/aliases 243is changed and the destination directory is not the default, 244then a warning will be issued instead. 245This is due to a limitation of the 246.Xr newaliases 1 247command. 248Similarly, 249if 250.Pa /etc/motd 251is changed and the destination directory is not the default, 252then 253.Pa /etc/rc.d/motd 254will not be executed due to a limitation of that script. 255In this case no warning is issued as the result of 256.Pa /etc/rc.d/motd 257is merely cosmetic and will be corrected on the next reboot. 258.Ss Build Mode 259The 260.Cm build 261mode is used to build a tarball that contains a snapshot of a 262.Dq current 263tree. 264This tarball can be used by the default and extract modes. 265Using a tarball can allow 266.Nm 267to perform a merge without requiring a source tree that matches the 268currently installed world. 269The 270.Fa tarball 271argument specifies the name of the file to create. 272The file will be a 273.Xr tar 5 274file compressed with 275.Xr bzip2 1 . 276.Ss Diff Mode 277The 278.Cm diff 279mode compares the versions of files in the destination directory to the 280.Dq current 281tree and generates a unified format diff of the changes. 282This can be used to determine which files have been locally modified and how. 283Note that 284.Nm 285does not manage files that are not maintained in the source tree such as 286.Pa /etc/fstab 287and 288.Pa /etc/rc.conf . 289.Ss Extract Mode 290The 291.Cm extract 292mode generates a new 293.Dq current 294tree. 295Unlike the default mode, 296it does not save any existing 297.Dq current 298tree and does not modify any existing 299.Dq previous 300tree. 301The new 302.Dq current 303tree can either be built from a source tree or extracted from a tarball. 304.Ss Resolve Mode 305The 306.Cm resolve 307mode is used to resolve any conflicts encountered during a merge. 308In this mode, 309.Nm 310iterates over any existing conflicts prompting the user for actions to take 311on each conflicted file. 312For each file, the following actions are available: 313.Pp 314.Bl -tag -width "(tf) theirs-full" -compact 315.It (p) postpone 316Ignore this conflict for now. 317.It (df) diff-full 318Show all changes made to the merged file as a unified diff. 319.It (e) edit 320Change the merged file in an editor. 321.It (r) resolved 322Install the merged version of the file into the destination directory. 323.It (mf) mine-full 324Use the version of the file in the destination directory and ignore any 325changes made to the file in the 326.Dq current 327tree. 328.It (tf) theirs-full 329Use the version of the file from the 330.Dq current 331tree and discard any local changes made to the file. 332.It (h) help 333Display the list of commands. 334.El 335.Ss Status Mode 336The 337.Cm status 338mode shows a summary of the results of the most recent merge. 339First it lists any files for which there are unresolved conflicts. 340Next it lists any warnings generated during the last merge. 341If the last merge did not generate any conflicts or warnings, 342then nothing will be output. 343.Sh OPTIONS 344The following options are available. 345Note that most options do not apply to all modes. 346.Bl -tag -width ".Fl A Ar patterns" 347.It Fl A Ar patterns 348Always install the new version of any files that match any of the patterns 349listed in 350.Ar patterns . 351Each pattern is evaluated as an 352.Xr sh 1 353shell pattern. 354This option may be specified multiple times to specify multiple patterns. 355Multiple space-separated patterns may also be specified in a single 356option. 357Note that ignored files specified via the 358.Ev IGNORE_FILES 359variable or the 360.Fl I 361option will not be installed. 362.It Fl B 363Do not build generated files in a private object tree. 364Instead, 365reuse the generated files from a previously built object tree that matches 366the source tree. 367This can be useful to avoid gratuitous conflicts in 368.Xr sendmail 8 369configuration 370files when bootstrapping. 371It can also be useful for building a tarball that matches a specific 372world build. 373.It Fl D Ar destdir 374Specify an alternate destination directory as the target of a merge. 375This is analogous to the 376.Dv DESTDIR 377variable used with 378.Sq make installworld . 379The default destination directory is an empty string which results in 380merges updating 381.Pa /etc 382on the local machine. 383.It Fl d Ar workdir 384Specify an alternate directory to use as the work directory. 385The work directory is used to store the 386.Dq current 387and 388.Dq previous 389trees as well as unresolved conflicts. 390The default work directory is 391.Pa <destdir>/var/db/etcupdate . 392.It Fl F 393Ignore changes in the FreeBSD ID string when comparing files in the 394destination directory to files in either of the 395.Dq current 396or 397.Dq previous 398trees. 399In 400.Cm diff 401mode, 402this reduces noise due to FreeBSD ID string changes in the output. 403During an update this can simplify handling for harmless conflicts caused 404by FreeBSD ID string changes. 405.Pp 406Specifically, 407if a file in the destination directory is identical to the same file in the 408.Dq previous 409tree modulo the FreeBSD ID string, 410then the file is treated as if it was unmodified and the 411.Dq current 412version of the file will be installed. 413Similarly, 414if a file in the destination directory is identical to the same file in the 415.Dq current 416tree modulo the FreeBSD ID string, 417then the 418.Dq current 419version of the file will be installed to update the ID string. 420If the 421.Dq previous 422and 423.Dq current 424versions of the file are identical, 425then 426.Nm 427will not change the file in the destination directory. 428.Pp 429Due to limitations in the 430.Xr diff 1 431command, 432this option may not have an effect if there are other changes in a file that 433are close to the FreeBSD ID string. 434.It Fl I Ar patterns 435Ignore any files that match any of the patterns listed in 436.Ar patterns . 437No warnings or other messages will be generated for those files during a 438merge. 439Each pattern is evaluated as an 440.Xr sh 1 441shell pattern. 442This option may be specified multiple times to specify multiple patterns. 443Multiple space-separated patterns may also be specified in a single 444option. 445.It Fl L Ar logfile 446Specify an alternate path for the log file. 447The 448.Nm 449utility logs each command that it invokes along with the standard output 450and standard error to this file. 451By default the log file is stored in a file named 452.Pa log 453in the work directory. 454.It Fl M Ar options 455Pass 456.Ar options 457as additional parameters to 458.Xr make 1 459when building a 460.Dq current 461tree. 462This can be used for to set the 463.Dv TARGET 464or 465.Dv TARGET_ARCH 466variables for a cross-build. 467.It Fl n 468Enable 469.Dq dry-run 470mode. 471Do not merge any changes to the destination directory. 472Instead, 473report what actions would be taken during a merge. 474Note that the existing 475.Dq current 476and 477.Dq previous 478trees will not be changed. 479If the 480.Fl r 481option is not specified, 482then a temporary 483.Dq current 484tree will be extracted to perform the comparison. 485.It Fl p 486Enable 487.Dq pre-world 488mode. 489Only merge changes to files that are necessary to successfully run 490.Sq make installworld 491or 492.Sq make installkernel . 493When this flag is enabled, 494the existing 495.Dq current 496and 497.Dq previous 498trees are left alone. 499Instead, 500a temporary tree is populated with the necessary files. 501This temporary tree is compared against the 502.Dq current 503tree. 504This allows a normal update to be run after 505.Sq make installworld 506has completed. 507Any conflicts generated during a 508.Dq pre-world 509update should be resolved by a 510.Dq pre-world 511.Cm resolve . 512.It Fl r 513Do not update the 514.Dq current 515and 516.Dq previous 517trees during a merge. 518This can be used to 519.Dq re-run 520a previous merge operation. 521.It Fl s Ar source 522Specify an alternate source tree to use when building or extracting a 523.Dq current 524tree. 525The default source tree is 526.Pa /usr/src . 527.It Fl t Ar tarball 528Extract a new 529.Dq current 530tree from a tarball previously generated by the 531.Cm build 532command rather than building the tree from a source tree. 533.El 534.Sh CONFIG FILE 535The 536.Nm 537utility can also be configured by setting variables in an optional 538configuration file named 539.Pa /etc/etcupdate.conf . 540Note that command line options override settings in the configuration file. 541The configuration file is executed by 542.Xr sh 1 , 543so it uses that syntax to set configuration variables. 544The following variables can be set: 545.Bl -tag -width ".Ev ALWAYS_INSTALL" 546.It Ev ALWAYS_INSTALL 547Always install files that match any of the patterns listed in this variable 548similar to the 549.Fl A 550option. 551.It Ev DESTDIR 552Specify an alternate destination directory similar to the 553.Fl D 554option. 555.It Ev EDITOR 556Specify a program to edit merge conflicts. 557.It Ev FREEBSD_ID 558Ignore changes in the FreeBSD ID string similar to the 559.Fl F 560option. 561This is enabled by setting the variable to a non-empty value. 562.It Ev IGNORE_FILES 563Ignore files that match any of the patterns listed in this variable 564similar to the 565.Fl I 566option. 567.It Ev LOGFILE 568Specify an alternate path for the log file similar to the 569.Fl L 570option. 571.It Ev MAKE_OPTIONS 572Pass additional options to 573.Xr make 1 574when building a 575.Dq current 576tree similar to the 577.Fl M 578option. 579.It Ev SRCDIR 580Specify an alternate source tree similar to the 581.Fl s 582option. 583.It Ev WORKDIR 584Specify an alternate work directory similar to the 585.Fl d 586option. 587.El 588.Sh ENVIRONMENT 589The 590.Nm 591utility uses the program identified in the 592.Ev EDITOR 593environment variable to edit merge conflicts. 594If 595.Ev EDITOR 596is not set, 597.Xr vi 1 598is used as the default editor. 599.Sh FILES 600.Bl -tag -width ".Pa /var/db/etcupdate/log" -compact 601.It Pa /etc/etcupdate.conf 602Optional config file. 603.It Pa /var/db/etcupdate 604Default work directory used to store trees and other data. 605.It Pa /var/db/etcupdate/log 606Default log file. 607.El 608.Sh EXIT STATUS 609.Ex -std 610.Sh EXAMPLES 611To compare the files in 612.Pa /etc 613with the stock versions: 614.Pp 615.Dl "etcupdate diff" 616.Pp 617To merge changes after an upgrade via the buildworld and installworld process: 618.Pp 619.Dl "etcupdate" 620.Pp 621To resolve any conflicts generated during a merge: 622.Pp 623.Dl "etcupdate resolve" 624.Ss Bootstrapping 625The 626.Nm 627utility may need to be bootstrapped before it can be used. 628The 629.Cm diff 630command will fail with an error about a missing reference tree if 631bootstrapping is needed. 632.Pp 633Bootstrapping 634.Nm 635requires a source tree that matches the currently installed world. 636The easiest way to ensure this is to bootstrap 637.Nm 638before updating the source tree to start the next world upgrade cycle. 639First, 640generate a reference tree: 641.Pp 642.Dl "etcupdate extract" 643.Pp 644Second, 645use the 646.Cm diff 647command to compare the reference tree to your current files in 648.Pa /etc . 649Undesired differences should be removed using an editor, 650.Xr patch 1 , 651or by copying files from the reference tree 652.Po 653located at 654.Pa /var/db/etcupdate/current 655by default 656.Pc 657. 658.Pp 659If the tree at 660.Pa /usr/src 661is already newer than the currently installed world, 662a new tree matching the currently installed world can be checked out to 663a temporary location. 664The reference tree for 665.Nm 666can then be generated via: 667.Pp 668.Dl "etcupdate extract -s /path/to/tree" 669.Pp 670The 671.Cm diff 672command can be used as above to remove undesired differences. 673Afterwards, 674the changes in the tree at 675.Pa /usr/src 676can be merged via a regular merge. 677.Sh DIAGNOSTICS 678The following warning messages may be generated during a merge. 679Note that several of these warnings cover obscure cases that should occur 680rarely if at all in practice. 681For example, 682if a file changes from a file to a directory in the 683.Dq current 684tree 685and the file was modified in the destination directory, 686then a warning will be triggered. 687In general, 688when a warning references a pathname, 689the corresponding file in the destination directory is not changed by a 690merge operation. 691.Bl -diag 692.It "Directory mismatch: <path> (<type>)" 693An attempt was made to create a directory at 694.Pa path 695but an existing file of type 696.Dq type 697already exists for that path name. 698.It "Modified link changed: <file> (<old> became <new>)" 699The target of a symbolic link named 700.Pa file 701was changed from 702.Dq old 703to 704.Dq new 705in the 706.Dq current 707tree. 708The symbolic link has been modified to point to a target that is neither 709.Dq old 710nor 711.Dq new 712in the destination directory. 713.It "Modified mismatch: <file> (<new> vs <dest>)" 714A file named 715.Pa file 716of type 717.Dq new 718was modified in the 719.Dq current 720tree, 721but the file exists as a different type 722.Dq dest 723in the destination directory. 724.It "Modified <type> changed: <file> (<old> became <new>)" 725A file named 726.Pa file 727changed type from 728.Dq old 729in the 730.Dq previous 731tree to type 732.Dq new 733in the 734.Dq current 735tree. 736The file in the destination directory of type 737.Dq type 738has been modified, 739so it could not be merged automatically. 740.It "Modified <type> remains: <file>" 741The file of type 742.Dq type 743named 744.Pa file 745has been removed from the 746.Dq current 747tree, 748but it has been locally modified. 749The modified version of the file remains in the destination directory. 750.It "Needs update: /etc/localtime (required manual update via tzsetup(1))" 751The 752.Fa /var/db/zoneinfo 753file does not exist, 754so 755.Nm 756was not able to refresh 757.Fa /etc/localtime 758from its source file in 759.Fa /usr/share/zoneinfo . 760Running 761.Xr tzsetup 1 762will both refresh 763.Fa /etc/localtime 764and generate 765.Fa /var/db/zoneinfo 766permitting future updates to refresh 767.Fa /etc/localtime 768automatically. 769.It "Needs update: /etc/mail/aliases.db (required manual update via newaliases(1))" 770The file 771.Pa /etc/mail/aliases 772was updated during a merge with a non-empty destination directory. 773Due to a limitation of the 774.Xr newaliases 1 775command, 776.Nm 777was not able to automatically update the corresponding aliases database. 778.It "New file mismatch: <file> (<new> vs <dest>)" 779A new file named 780.Pa file 781of type 782.Dq new 783has been added to the 784.Dq current 785tree. 786A file of that name already exists in the destination directory, 787but it is of a different type 788.Dq dest . 789.It "New link conflict: <file> (<new> vs <dest>)" 790A symbolic link named 791.Pa file 792has been added to the 793.Dq current 794tree that links to 795.Dq new . 796A symbolic link of the same name already exists in the destination 797directory, 798but it links to a different target 799.Dq dest . 800.It "Non-empty directory remains: <file>" 801The directory 802.Pa file 803was removed from the 804.Dq current 805tree, 806but it contains additional files in the destination directory. 807These additional files as well as the directory remain. 808.It "Remove mismatch: <file> (<old> became <new>)" 809A file named 810.Pa file 811changed from type 812.Dq old 813in the 814.Dq previous 815tree to type 816.Dq new 817in the 818.Dq current 819tree, 820but it has been removed in the destination directory. 821.It "Removed file changed: <file>" 822A file named 823.Pa file 824was modified in the 825.Dq current 826tree, 827but it has been removed in the destination directory. 828.It "Removed link changed: <file> (<old> became <new>)" 829The target of a symbolic link named 830.Pa file 831was changed from 832.Dq old 833to 834.Dq new 835in the 836.Dq current 837tree, 838but it has been removed in the destination directory. 839.El 840.Sh SEE ALSO 841.Xr cap_mkdb 1 , 842.Xr diff 1 , 843.Xr make 1 , 844.Xr newaliases 1 , 845.Xr sh 1 , 846.Xr pwd_mkdb 8 847.Sh HISTORY 848The 849.Nm 850utility first appeared in 851.Fx 10.0 . 852.Sh AUTHORS 853The 854.Nm 855utility was written by 856.An John Baldwin Aq Mt jhb@FreeBSD.org . 857.Sh BUGS 858Rerunning a merge does not automatically delete conflicts left over from a 859previous merge. 860Any conflicts must be resolved before the merge can be rerun. 861It it is not clear if this is a feature or a bug. 862.Pp 863There is no way to easily automate conflict resolution for specific files. 864For example, one can imagine a syntax along the lines of 865.Pp 866.Dl "etcupdate resolve tf /some/file" 867.Pp 868to resolve a specific conflict in an automated fashion. 869.Pp 870It might be nice to have something like a 871.Sq revert 872command to replace a locally modified version of a file with the stock 873version of the file. 874For example: 875.Pp 876.Dl "etcupdate revert /etc/mail/freebsd.cf" 877.Pp 878Bootstrapping 879.Nm 880often results in gratuitous diffs in 881.Pa /etc/mail/*.cf 882that cause conflicts in the first merge. 883If an object tree that matches the source tree is present when bootstrapping, 884then passing the 885.Fl B 886flag to the 887.Cm extract 888command can work around this. 889