1.\" Copyright (c) 2010-2013 Hudson River Trading LLC 2.\" Written by: John H. Baldwin <jhb@FreeBSD.org> 3.\" All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.\" $FreeBSD$ 27.\" 28.Dd September 29, 2015 29.Dt ETCUPDATE 8 30.Os 31.Sh NAME 32.Nm etcupdate 33.Nd "manage updates to system files not updated by installworld" 34.Sh SYNOPSIS 35.Nm 36.Op Fl npBF 37.Op Fl d Ar workdir 38.Op Fl r | Fl s Ar source | Fl t Ar tarball 39.Op Fl A Ar patterns 40.Op Fl D Ar destdir 41.Op Fl I Ar patterns 42.Op Fl L Ar logfile 43.Op Fl M Ar options 44.Nm 45.Cm build 46.Op Fl B 47.Op Fl d Ar workdir 48.Op Fl s Ar source 49.Op Fl L Ar logfile 50.Op Fl M Ar options 51.Ar tarball 52.Nm 53.Cm diff 54.Op Fl d Ar workdir 55.Op Fl D Ar destdir 56.Op Fl I Ar patterns 57.Op Fl L Ar logfile 58.Nm 59.Cm extract 60.Op Fl B 61.Op Fl d Ar workdir 62.Op Fl s Ar source | Fl t Ar tarball 63.Op Fl L Ar logfile 64.Op Fl M Ar options 65.Nm 66.Cm resolve 67.Op Fl p 68.Op Fl d Ar workdir 69.Op Fl D Ar destdir 70.Op Fl L Ar logfile 71.Nm 72.Cm status 73.Op Fl d Ar workdir 74.Op Fl D Ar destdir 75.Sh DESCRIPTION 76The 77.Nm 78utility is a tool for managing updates to files that are not updated as 79part of 80.Sq make installworld 81such as files in 82.Pa /etc . 83It manages updates by doing a three-way merge of changes made to these 84files against the local versions. 85It is also designed to minimize the amount of user intervention with 86the goal of simplifying upgrades for clusters of machines. 87.Pp 88To perform a three-way merge, 89.Nm 90keeps copies of the current and previous versions of files that it manages. 91These copies are stored in two trees known as the 92.Dq current 93and 94.Dq previous 95trees. 96During a merge, 97.Nm 98compares the 99.Dq current 100and 101.Dq previous 102copies of each file to determine which changes need to be merged into the 103local version of each file. 104If a file can be updated without generating a conflict, 105.Nm 106will update the file automatically. 107If the local changes to a file conflict with the changes made to a file in 108the source tree, 109then a merge conflict is generated. 110The conflict must be resolved after the merge has finished. 111The 112.Nm 113utility will not perform a new merge until all conflicts from an earlier 114merge are resolved. 115.Sh MODES 116The 117.Nm 118utility supports several modes of operation. 119The mode is specified via an optional command argument. 120If present, the command must be the first argument on the command line. 121If a command is not specified, the default mode is used. 122.Ss Default Mode 123The default mode merges changes from the source tree to the destination 124directory. 125First, 126it updates the 127.Dq current 128and 129.Dq previous 130trees. 131Next, 132it compares the two trees merging changes into the destination directory. 133Finally, 134it displays warnings for any conditions it could not handle automatically. 135.Pp 136If the 137.Fl r 138option is not specified, 139then the first step taken is to update the 140.Dq current 141and 142.Dq previous 143trees. 144If a 145.Dq current 146tree already exists, 147then that tree is saved as the 148.Dq previous 149tree. 150An older 151.Dq previous 152tree is removed if it exists. 153By default the new 154.Dq current 155tree is built from a source tree. 156However, 157if a tarball is specified via the 158.Fl t 159option, 160then the tree is extracted from that tarball instead. 161.Pp 162Next, 163.Nm 164compares the files in the 165.Dq current 166and 167.Dq previous 168trees. 169If a file was removed from the 170.Dq current 171tree, 172then it will be removed from the destination directory only if it 173does not have any local modifications. 174If a file was added to the 175.Dq current 176tree, 177then it will be copied to the destination directory only if it 178would not clobber an existing file. 179If a file is changed in the 180.Dq current 181tree, 182then 183.Nm 184will attempt to merge the changes into the version of the file in the 185destination directory. 186If the merge encounters conflicts, 187then a version of the file with conflict markers will be saved for 188future resolution. 189If the merge does not encounter conflicts, 190then the merged version of the file will be saved in the destination 191directory. 192If 193.Nm 194is not able to safely merge in changes to a file other than a merge conflict, 195it will generate a warning. 196.Pp 197For each file that is updated a line will be output with a leading character 198to indicate the action taken. 199The possible actions follow: 200.Pp 201.Bl -tag -width "A" -compact -offset indent 202.It A 203Added 204.It C 205Conflict 206.It D 207Deleted 208.It M 209Merged 210.It U 211Updated 212.El 213.Pp 214Finally, 215if any warnings were encountered they are displayed after the merge has 216completed. 217.Pp 218Note that for certain files 219.Nm 220will perform post-install actions any time that the file is updated. 221Specifically, 222.Xr pwd_mkdb 8 223is invoked if 224.Pa /etc/master.passwd 225is changed, 226.Xr cap_mkdb 1 227is invoked to update 228.Pa /etc/login.conf.db 229if 230.Pa /etc/login.conf 231is changed, 232.Xr newaliases 1 233is invoked if 234.Pa /etc/mail/aliases 235is changed, 236.Xr services_mkdb 8 237is invoked if 238.Pa /etc/services 239is changed, 240.Xr tzsetup 8 241is invoked if 242.Pa /etc/localtime 243is changed and if 244.Fa /var/db/zoneinfo 245exists, 246and 247.Pa /etc/rc.d/motd 248is invoked if 249.Pa /etc/motd 250is changed. 251One exception is that if 252.Pa /etc/mail/aliases 253is changed and the destination directory is not the default, 254then a warning will be issued instead. 255This is due to a limitation of the 256.Xr newaliases 1 257command. 258Similarly, 259if 260.Pa /etc/motd 261is changed and the destination directory is not the default, 262then 263.Pa /etc/rc.d/motd 264will not be executed due to a limitation of that script. 265In this case no warning is issued as the result of 266.Pa /etc/rc.d/motd 267is merely cosmetic and will be corrected on the next reboot. 268.Ss Build Mode 269The 270.Cm build 271mode is used to build a tarball that contains a snapshot of a 272.Dq current 273tree. 274This tarball can be used by the default and extract modes. 275Using a tarball can allow 276.Nm 277to perform a merge without requiring a source tree that matches the 278currently installed world. 279The 280.Fa tarball 281argument specifies the name of the file to create. 282The file will be a 283.Xr tar 5 284file compressed with 285.Xr bzip2 1 . 286.Ss Diff Mode 287The 288.Cm diff 289mode compares the versions of files in the destination directory to the 290.Dq current 291tree and generates a unified format diff of the changes. 292This can be used to determine which files have been locally modified and how. 293Note that 294.Nm 295does not manage files that are not maintained in the source tree such as 296.Pa /etc/fstab 297and 298.Pa /etc/rc.conf . 299.Ss Extract Mode 300The 301.Cm extract 302mode generates a new 303.Dq current 304tree. 305Unlike the default mode, 306it does not save any existing 307.Dq current 308tree and does not modify any existing 309.Dq previous 310tree. 311The new 312.Dq current 313tree can either be built from a source tree or extracted from a tarball. 314.Ss Resolve Mode 315The 316.Cm resolve 317mode is used to resolve any conflicts encountered during a merge. 318In this mode, 319.Nm 320iterates over any existing conflicts prompting the user for actions to take 321on each conflicted file. 322For each file, the following actions are available: 323.Pp 324.Bl -tag -width "(tf) theirs-full" -compact 325.It (p) postpone 326Ignore this conflict for now. 327.It (df) diff-full 328Show all changes made to the merged file as a unified diff. 329.It (e) edit 330Change the merged file in an editor. 331.It (r) resolved 332Install the merged version of the file into the destination directory. 333.It (mf) mine-full 334Use the version of the file in the destination directory and ignore any 335changes made to the file in the 336.Dq current 337tree. 338.It (tf) theirs-full 339Use the version of the file from the 340.Dq current 341tree and discard any local changes made to the file. 342.It (h) help 343Display the list of commands. 344.El 345.Ss Status Mode 346The 347.Cm status 348mode shows a summary of the results of the most recent merge. 349First it lists any files for which there are unresolved conflicts. 350Next it lists any warnings generated during the last merge. 351If the last merge did not generate any conflicts or warnings, 352then nothing will be output. 353.Sh OPTIONS 354The following options are available. 355Note that most options do not apply to all modes. 356.Bl -tag -width ".Fl A Ar patterns" 357.It Fl A Ar patterns 358Always install the new version of any files that match any of the patterns 359listed in 360.Ar patterns . 361Each pattern is evaluated as an 362.Xr sh 1 363shell pattern. 364This option may be specified multiple times to specify multiple patterns. 365Multiple space-separated patterns may also be specified in a single 366option. 367Note that ignored files specified via the 368.Ev IGNORE_FILES 369variable or the 370.Fl I 371option will not be installed. 372.It Fl B 373Do not build generated files in a private object tree. 374Instead, 375reuse the generated files from a previously built object tree that matches 376the source tree. 377This can be useful to avoid gratuitous conflicts in 378.Xr sendmail 8 379configuration 380files when bootstrapping. 381It can also be useful for building a tarball that matches a specific 382world build. 383.It Fl D Ar destdir 384Specify an alternate destination directory as the target of a merge. 385This is analogous to the 386.Dv DESTDIR 387variable used with 388.Sq make installworld . 389The default destination directory is an empty string which results in 390merges updating 391.Pa /etc 392on the local machine. 393.It Fl d Ar workdir 394Specify an alternate directory to use as the work directory. 395The work directory is used to store the 396.Dq current 397and 398.Dq previous 399trees as well as unresolved conflicts. 400The default work directory is 401.Pa <destdir>/var/db/etcupdate . 402.It Fl F 403Ignore changes in the FreeBSD ID string when comparing files in the 404destination directory to files in either of the 405.Dq current 406or 407.Dq previous 408trees. 409In 410.Cm diff 411mode, 412this reduces noise due to FreeBSD ID string changes in the output. 413During an update this can simplify handling for harmless conflicts caused 414by FreeBSD ID string changes. 415.Pp 416Specifically, 417if a file in the destination directory is identical to the same file in the 418.Dq previous 419tree modulo the FreeBSD ID string, 420then the file is treated as if it was unmodified and the 421.Dq current 422version of the file will be installed. 423Similarly, 424if a file in the destination directory is identical to the same file in the 425.Dq current 426tree modulo the FreeBSD ID string, 427then the 428.Dq current 429version of the file will be installed to update the ID string. 430If the 431.Dq previous 432and 433.Dq current 434versions of the file are identical, 435then 436.Nm 437will not change the file in the destination directory. 438.Pp 439Due to limitations in the 440.Xr diff 1 441command, 442this option may not have an effect if there are other changes in a file that 443are close to the FreeBSD ID string. 444.It Fl I Ar patterns 445Ignore any files that match any of the patterns listed in 446.Ar patterns . 447No warnings or other messages will be generated for those files during a 448merge. 449Each pattern is evaluated as an 450.Xr sh 1 451shell pattern. 452This option may be specified multiple times to specify multiple patterns. 453Multiple space-separated patterns may also be specified in a single 454option. 455.It Fl L Ar logfile 456Specify an alternate path for the log file. 457The 458.Nm 459utility logs each command that it invokes along with the standard output 460and standard error to this file. 461By default the log file is stored in a file named 462.Pa log 463in the work directory. 464.It Fl M Ar options 465Pass 466.Ar options 467as additional parameters to 468.Xr make 1 469when building a 470.Dq current 471tree. 472This can be used for to set the 473.Dv TARGET 474or 475.Dv TARGET_ARCH 476variables for a cross-build. 477.It Fl n 478Enable 479.Dq dry-run 480mode. 481Do not merge any changes to the destination directory. 482Instead, 483report what actions would be taken during a merge. 484Note that the existing 485.Dq current 486and 487.Dq previous 488trees will not be changed. 489If the 490.Fl r 491option is not specified, 492then a temporary 493.Dq current 494tree will be extracted to perform the comparison. 495.It Fl p 496Enable 497.Dq pre-world 498mode. 499Only merge changes to files that are necessary to successfully run 500.Sq make installworld 501or 502.Sq make installkernel . 503When this flag is enabled, 504the existing 505.Dq current 506and 507.Dq previous 508trees are left alone. 509Instead, 510a temporary tree is populated with the necessary files. 511This temporary tree is compared against the 512.Dq current 513tree. 514This allows a normal update to be run after 515.Sq make installworld 516has completed. 517Any conflicts generated during a 518.Dq pre-world 519update should be resolved by a 520.Dq pre-world 521.Cm resolve . 522.It Fl r 523Do not update the 524.Dq current 525and 526.Dq previous 527trees during a merge. 528This can be used to 529.Dq re-run 530a previous merge operation. 531.It Fl s Ar source 532Specify an alternate source tree to use when building or extracting a 533.Dq current 534tree. 535The default source tree is 536.Pa /usr/src . 537.It Fl t Ar tarball 538Extract a new 539.Dq current 540tree from a tarball previously generated by the 541.Cm build 542command rather than building the tree from a source tree. 543.El 544.Sh CONFIG FILE 545The 546.Nm 547utility can also be configured by setting variables in an optional 548configuration file named 549.Pa /etc/etcupdate.conf . 550Note that command line options override settings in the configuration file. 551The configuration file is executed by 552.Xr sh 1 , 553so it uses that syntax to set configuration variables. 554The following variables can be set: 555.Bl -tag -width ".Ev ALWAYS_INSTALL" 556.It Ev ALWAYS_INSTALL 557Always install files that match any of the patterns listed in this variable 558similar to the 559.Fl A 560option. 561.It Ev DESTDIR 562Specify an alternate destination directory similar to the 563.Fl D 564option. 565.It Ev EDITOR 566Specify a program to edit merge conflicts. 567.It Ev FREEBSD_ID 568Ignore changes in the FreeBSD ID string similar to the 569.Fl F 570option. 571This is enabled by setting the variable to a non-empty value. 572.It Ev IGNORE_FILES 573Ignore files that match any of the patterns listed in this variable 574similar to the 575.Fl I 576option. 577.It Ev LOGFILE 578Specify an alternate path for the log file similar to the 579.Fl L 580option. 581.It Ev MAKE_OPTIONS 582Pass additional options to 583.Xr make 1 584when building a 585.Dq current 586tree similar to the 587.Fl M 588option. 589.It Ev SRCDIR 590Specify an alternate source tree similar to the 591.Fl s 592option. 593.It Ev WORKDIR 594Specify an alternate work directory similar to the 595.Fl d 596option. 597.El 598.Sh ENVIRONMENT 599The 600.Nm 601utility uses the program identified in the 602.Ev EDITOR 603environment variable to edit merge conflicts. 604If 605.Ev EDITOR 606is not set, 607.Xr vi 1 608is used as the default editor. 609.Sh FILES 610.Bl -tag -width ".Pa /var/db/etcupdate/log" -compact 611.It Pa /etc/etcupdate.conf 612Optional config file. 613.It Pa /var/db/etcupdate 614Default work directory used to store trees and other data. 615.It Pa /var/db/etcupdate/log 616Default log file. 617.El 618.Sh EXIT STATUS 619.Ex -std 620.Sh EXAMPLES 621To compare the files in 622.Pa /etc 623with the stock versions: 624.Pp 625.Dl "etcupdate diff" 626.Pp 627To merge changes after an upgrade via the buildworld and installworld process: 628.Pp 629.Dl "etcupdate" 630.Pp 631To resolve any conflicts generated during a merge: 632.Pp 633.Dl "etcupdate resolve" 634.Ss Bootstrapping 635The 636.Nm 637utility may need to be bootstrapped before it can be used. 638The 639.Cm diff 640command will fail with an error about a missing reference tree if 641bootstrapping is needed. 642.Pp 643Bootstrapping 644.Nm 645requires a source tree that matches the currently installed world. 646The easiest way to ensure this is to bootstrap 647.Nm 648before updating the source tree to start the next world upgrade cycle. 649First, 650generate a reference tree: 651.Pp 652.Dl "etcupdate extract" 653.Pp 654Second, 655use the 656.Cm diff 657command to compare the reference tree to your current files in 658.Pa /etc . 659Undesired differences should be removed using an editor, 660.Xr patch 1 , 661or by copying files from the reference tree 662.Po 663located at 664.Pa /var/db/etcupdate/current 665by default 666.Pc 667. 668.Pp 669If the tree at 670.Pa /usr/src 671is already newer than the currently installed world, 672a new tree matching the currently installed world can be checked out to 673a temporary location. 674The reference tree for 675.Nm 676can then be generated via: 677.Pp 678.Dl "etcupdate extract -s /path/to/tree" 679.Pp 680The 681.Cm diff 682command can be used as above to remove undesired differences. 683Afterwards, 684the changes in the tree at 685.Pa /usr/src 686can be merged via a regular merge. 687.Sh DIAGNOSTICS 688The following warning messages may be generated during a merge. 689Note that several of these warnings cover obscure cases that should occur 690rarely if at all in practice. 691For example, 692if a file changes from a file to a directory in the 693.Dq current 694tree 695and the file was modified in the destination directory, 696then a warning will be triggered. 697In general, 698when a warning references a pathname, 699the corresponding file in the destination directory is not changed by a 700merge operation. 701.Bl -diag 702.It "Directory mismatch: <path> (<type>)" 703An attempt was made to create a directory at 704.Pa path 705but an existing file of type 706.Dq type 707already exists for that path name. 708.It "Modified link changed: <file> (<old> became <new>)" 709The target of a symbolic link named 710.Pa file 711was changed from 712.Dq old 713to 714.Dq new 715in the 716.Dq current 717tree. 718The symbolic link has been modified to point to a target that is neither 719.Dq old 720nor 721.Dq new 722in the destination directory. 723.It "Modified mismatch: <file> (<new> vs <dest>)" 724A file named 725.Pa file 726of type 727.Dq new 728was modified in the 729.Dq current 730tree, 731but the file exists as a different type 732.Dq dest 733in the destination directory. 734.It "Modified <type> changed: <file> (<old> became <new>)" 735A file named 736.Pa file 737changed type from 738.Dq old 739in the 740.Dq previous 741tree to type 742.Dq new 743in the 744.Dq current 745tree. 746The file in the destination directory of type 747.Dq type 748has been modified, 749so it could not be merged automatically. 750.It "Modified <type> remains: <file>" 751The file of type 752.Dq type 753named 754.Pa file 755has been removed from the 756.Dq current 757tree, 758but it has been locally modified. 759The modified version of the file remains in the destination directory. 760.It "Needs update: /etc/localtime (required manual update via tzsetup(1))" 761The 762.Fa /var/db/zoneinfo 763file does not exist, 764so 765.Nm 766was not able to refresh 767.Fa /etc/localtime 768from its source file in 769.Fa /usr/share/zoneinfo . 770Running 771.Xr tzsetup 1 772will both refresh 773.Fa /etc/localtime 774and generate 775.Fa /var/db/zoneinfo 776permitting future updates to refresh 777.Fa /etc/localtime 778automatically. 779.It "Needs update: /etc/mail/aliases.db (required manual update via newaliases(1))" 780The file 781.Pa /etc/mail/aliases 782was updated during a merge with a non-empty destination directory. 783Due to a limitation of the 784.Xr newaliases 1 785command, 786.Nm 787was not able to automatically update the corresponding aliases database. 788.It "New file mismatch: <file> (<new> vs <dest>)" 789A new file named 790.Pa file 791of type 792.Dq new 793has been added to the 794.Dq current 795tree. 796A file of that name already exists in the destination directory, 797but it is of a different type 798.Dq dest . 799.It "New link conflict: <file> (<new> vs <dest>)" 800A symbolic link named 801.Pa file 802has been added to the 803.Dq current 804tree that links to 805.Dq new . 806A symbolic link of the same name already exists in the destination 807directory, 808but it links to a different target 809.Dq dest . 810.It "Non-empty directory remains: <file>" 811The directory 812.Pa file 813was removed from the 814.Dq current 815tree, 816but it contains additional files in the destination directory. 817These additional files as well as the directory remain. 818.It "Remove mismatch: <file> (<old> became <new>)" 819A file named 820.Pa file 821changed from type 822.Dq old 823in the 824.Dq previous 825tree to type 826.Dq new 827in the 828.Dq current 829tree, 830but it has been removed in the destination directory. 831.It "Removed file changed: <file>" 832A file named 833.Pa file 834was modified in the 835.Dq current 836tree, 837but it has been removed in the destination directory. 838.It "Removed link changed: <file> (<old> became <new>)" 839The target of a symbolic link named 840.Pa file 841was changed from 842.Dq old 843to 844.Dq new 845in the 846.Dq current 847tree, 848but it has been removed in the destination directory. 849.El 850.Sh SEE ALSO 851.Xr cap_mkdb 1 , 852.Xr diff 1 , 853.Xr make 1 , 854.Xr newaliases 1 , 855.Xr sh 1 , 856.Xr pwd_mkdb 8 , 857.Xr services_mkdb 8 , 858.Xr tzsetup 8 859.Sh HISTORY 860The 861.Nm 862utility first appeared in 863.Fx 10.0 . 864.Sh AUTHORS 865The 866.Nm 867utility was written by 868.An John Baldwin Aq Mt jhb@FreeBSD.org . 869.Sh BUGS 870Rerunning a merge does not automatically delete conflicts left over from a 871previous merge. 872Any conflicts must be resolved before the merge can be rerun. 873It it is not clear if this is a feature or a bug. 874.Pp 875There is no way to easily automate conflict resolution for specific files. 876For example, one can imagine a syntax along the lines of 877.Pp 878.Dl "etcupdate resolve tf /some/file" 879.Pp 880to resolve a specific conflict in an automated fashion. 881.Pp 882It might be nice to have something like a 883.Sq revert 884command to replace a locally modified version of a file with the stock 885version of the file. 886For example: 887.Pp 888.Dl "etcupdate revert /etc/mail/freebsd.cf" 889.Pp 890Bootstrapping 891.Nm 892often results in gratuitous diffs in 893.Pa /etc/mail/*.cf 894that cause conflicts in the first merge. 895If an object tree that matches the source tree is present when bootstrapping, 896then passing the 897.Fl B 898flag to the 899.Cm extract 900command can work around this. 901