1.\" Copyright (c) 2010-2013 Hudson River Trading LLC 2.\" Written by: John H. Baldwin <jhb@FreeBSD.org> 3.\" All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.\" $FreeBSD$ 27.\" 28.Dd April 20, 2021 29.Dt ETCUPDATE 8 30.Os 31.Sh NAME 32.Nm etcupdate 33.Nd "manage updates to system files not updated by installworld" 34.Sh SYNOPSIS 35.Nm 36.Op Fl npBF 37.Op Fl d Ar workdir 38.Op Fl r | Fl s Ar source | Fl t Ar tarball 39.Op Fl A Ar patterns 40.Op Fl D Ar destdir 41.Op Fl I Ar patterns 42.Op Fl L Ar logfile 43.Op Fl M Ar options 44.Nm 45.Cm build 46.Op Fl B 47.Op Fl d Ar workdir 48.Op Fl s Ar source 49.Op Fl L Ar logfile 50.Op Fl M Ar options 51.Ar tarball 52.Nm 53.Cm diff 54.Op Fl d Ar workdir 55.Op Fl D Ar destdir 56.Op Fl I Ar patterns 57.Op Fl L Ar logfile 58.Nm 59.Cm extract 60.Op Fl B 61.Op Fl d Ar workdir 62.Op Fl s Ar source | Fl t Ar tarball 63.Op Fl D Ar destdir 64.Op Fl L Ar logfile 65.Op Fl M Ar options 66.Nm 67.Cm resolve 68.Op Fl p 69.Op Fl d Ar workdir 70.Op Fl D Ar destdir 71.Op Fl L Ar logfile 72.Nm 73.Cm revert 74.Op Fl d Ar workdir 75.Op Fl D Ar destdir 76.Op Fl L Ar logfile 77.Ar 78.Nm 79.Cm status 80.Op Fl d Ar workdir 81.Op Fl D Ar destdir 82.Sh DESCRIPTION 83The 84.Nm 85utility is a tool for managing updates to files that are not updated as 86part of 87.Sq make installworld 88such as files in 89.Pa /etc . 90It manages updates by doing a three-way merge of changes made to these 91files against the local versions. 92It is also designed to minimize the amount of user intervention with 93the goal of simplifying upgrades for clusters of machines. 94.Pp 95To perform a three-way merge, 96.Nm 97keeps copies of the current and previous versions of files that it manages. 98These copies are stored in two trees known as the 99.Dq current 100and 101.Dq previous 102trees. 103During a merge, 104.Nm 105compares the 106.Dq current 107and 108.Dq previous 109copies of each file to determine which changes need to be merged into the 110local version of each file. 111If a file can be updated without generating a conflict, 112.Nm 113will update the file automatically. 114If the local changes to a file conflict with the changes made to a file in 115the source tree, 116then a merge conflict is generated. 117The conflict must be resolved after the merge has finished. 118The 119.Nm 120utility will not perform a new merge until all conflicts from an earlier 121merge are resolved. 122.Sh MODES 123The 124.Nm 125utility supports several modes of operation. 126The mode is specified via an optional command argument. 127If present, the command must be the first argument on the command line. 128If a command is not specified, the default mode is used. 129.Ss Default Mode 130The default mode merges changes from the source tree to the destination 131directory. 132First, 133it updates the 134.Dq current 135and 136.Dq previous 137trees. 138Next, 139it compares the two trees merging changes into the destination directory. 140Finally, 141it displays warnings for any conditions it could not handle automatically. 142.Pp 143If the 144.Fl r 145option is not specified, 146then the first step taken is to update the 147.Dq current 148and 149.Dq previous 150trees. 151If a 152.Dq current 153tree already exists, 154then that tree is saved as the 155.Dq previous 156tree. 157An older 158.Dq previous 159tree is removed if it exists. 160By default the new 161.Dq current 162tree is built from a source tree. 163However, 164if a tarball is specified via the 165.Fl t 166option, 167then the tree is extracted from that tarball instead. 168.Pp 169Next, 170.Nm 171compares the files in the 172.Dq current 173and 174.Dq previous 175trees. 176If a file was removed from the 177.Dq current 178tree, 179then it will be removed from the destination directory only if it 180does not have any local modifications. 181If a file was added to the 182.Dq current 183tree, 184then it will be copied to the destination directory only if it 185would not clobber an existing file. 186If a file is changed in the 187.Dq current 188tree, 189then 190.Nm 191will attempt to merge the changes into the version of the file in the 192destination directory. 193If the merge encounters conflicts, 194then a version of the file with conflict markers will be saved for 195future resolution. 196If the merge does not encounter conflicts, 197then the merged version of the file will be saved in the destination 198directory. 199If 200.Nm 201is not able to safely merge in changes to a file other than a merge conflict, 202it will generate a warning. 203.Pp 204For each file that is updated a line will be output with a leading character 205to indicate the action taken. 206The possible actions follow: 207.Pp 208.Bl -tag -width "A" -compact -offset indent 209.It A 210Added 211.It C 212Conflict 213.It D 214Deleted 215.It M 216Merged 217.It U 218Updated 219.El 220.Pp 221Finally, 222if any warnings were encountered they are displayed after the merge has 223completed. 224.Pp 225Note that for certain files 226.Nm 227will perform post-install actions any time that the file is updated. 228Specifically, 229.Xr pwd_mkdb 8 230is invoked if 231.Pa /etc/master.passwd 232is changed, 233.Xr cap_mkdb 1 234is invoked to update 235.Pa /etc/login.conf.db 236if 237.Pa /etc/login.conf 238is changed, 239.Xr newaliases 1 240is invoked if 241.Pa /etc/mail/aliases 242is changed, 243.Xr services_mkdb 8 244is invoked if 245.Pa /etc/services 246is changed, 247.Xr tzsetup 8 248is invoked if 249.Pa /etc/localtime 250is changed and if 251.Fa /var/db/zoneinfo 252exists, 253and 254.Pa /etc/rc.d/motd 255is invoked if 256.Pa /etc/motd 257is changed. 258One exception is that if 259.Pa /etc/mail/aliases 260is changed and the destination directory is not the default, 261then a warning will be issued instead. 262This is due to a limitation of the 263.Xr newaliases 1 264command. 265Similarly, 266if 267.Pa /etc/motd 268is changed and the destination directory is not the default, 269then 270.Pa /etc/rc.d/motd 271will not be executed due to a limitation of that script. 272In this case no warning is issued as the result of 273.Pa /etc/rc.d/motd 274is merely cosmetic and will be corrected on the next reboot. 275.Ss Build Mode 276The 277.Cm build 278mode is used to build a tarball that contains a snapshot of a 279.Dq current 280tree. 281This tarball can be used by the default and extract modes. 282Using a tarball can allow 283.Nm 284to perform a merge without requiring a source tree that matches the 285currently installed world. 286The 287.Fa tarball 288argument specifies the name of the file to create. 289The file will be a 290.Xr tar 5 291file compressed with 292.Xr bzip2 1 . 293.Ss Diff Mode 294The 295.Cm diff 296mode compares the versions of files in the destination directory to the 297.Dq current 298tree and generates a unified format diff of the changes. 299This can be used to determine which files have been locally modified and how. 300Note that 301.Nm 302does not manage files that are not maintained in the source tree such as 303.Pa /etc/fstab 304and 305.Pa /etc/rc.conf . 306.Ss Extract Mode 307The 308.Cm extract 309mode generates a new 310.Dq current 311tree. 312Unlike the default mode, 313it does not save any existing 314.Dq current 315tree and does not modify any existing 316.Dq previous 317tree. 318The new 319.Dq current 320tree can either be built from a source tree or extracted from a tarball. 321.Ss Resolve Mode 322The 323.Cm resolve 324mode is used to resolve any conflicts encountered during a merge. 325In this mode, 326.Nm 327iterates over any existing conflicts prompting the user for actions to take 328on each conflicted file. 329For each file, the following actions are available: 330.Pp 331.Bl -tag -width "(tf) theirs-full" -compact 332.It (p) postpone 333Ignore this conflict for now. 334.It (df) diff-full 335Show all changes made to the merged file as a unified diff. 336.It (e) edit 337Change the merged file in an editor. 338.It (r) resolved 339Install the merged version of the file into the destination directory. 340.It (mf) mine-full 341Use the version of the file in the destination directory and ignore any 342changes made to the file in the 343.Dq current 344tree. 345.It (tf) theirs-full 346Use the version of the file from the 347.Dq current 348tree and discard any local changes made to the file. 349.It (h) help 350Display the list of commands. 351.El 352.Ss Revert Mode 353The 354.Cm revert 355mode is used to restore the stock versions of files. 356In this mode, 357.Nm 358installs the stock version of requested files. 359This mode cannot be used to restore directories, only individual files. 360.Ss Status Mode 361The 362.Cm status 363mode shows a summary of the results of the most recent merge. 364First it lists any files for which there are unresolved conflicts. 365Next it lists any warnings generated during the last merge. 366If the last merge did not generate any conflicts or warnings, 367then nothing will be output. 368.Sh OPTIONS 369The following options are available. 370Note that most options do not apply to all modes. 371.Bl -tag -width ".Fl A Ar patterns" 372.It Fl A Ar patterns 373Always install the new version of any files that match any of the patterns 374listed in 375.Ar patterns . 376Each pattern is evaluated as an 377.Xr sh 1 378shell pattern. 379This option may be specified multiple times to specify multiple patterns. 380Multiple space-separated patterns may also be specified in a single 381option. 382Note that ignored files specified via the 383.Ev IGNORE_FILES 384variable or the 385.Fl I 386option will not be installed. 387.It Fl B 388Do not build generated files in a private object tree. 389Instead, 390reuse the generated files from a previously built object tree that matches 391the source tree. 392This can be useful to avoid gratuitous conflicts in 393.Xr sendmail 8 394configuration 395files when bootstrapping. 396It can also be useful for building a tarball that matches a specific 397world build. 398.It Fl D Ar destdir 399Specify an alternate destination directory as the target of a merge. 400This is analogous to the 401.Dv DESTDIR 402variable used with 403.Sq make installworld . 404The default destination directory is an empty string which results in 405merges updating 406.Pa /etc 407on the local machine. 408.It Fl d Ar workdir 409Specify an alternate directory to use as the work directory. 410The work directory is used to store the 411.Dq current 412and 413.Dq previous 414trees as well as unresolved conflicts. 415The default work directory is 416.Pa <destdir>/var/db/etcupdate . 417.It Fl F 418Ignore changes in the FreeBSD ID string when comparing files in the 419destination directory to files in either of the 420.Dq current 421or 422.Dq previous 423trees. 424In 425.Cm diff 426mode, 427this reduces noise due to FreeBSD ID string changes in the output. 428During an update this can simplify handling for harmless conflicts caused 429by FreeBSD ID string changes. 430.Pp 431Specifically, 432if a file in the destination directory is identical to the same file in the 433.Dq previous 434tree modulo the FreeBSD ID string, 435then the file is treated as if it was unmodified and the 436.Dq current 437version of the file will be installed. 438Similarly, 439if a file in the destination directory is identical to the same file in the 440.Dq current 441tree modulo the FreeBSD ID string, 442then the 443.Dq current 444version of the file will be installed to update the ID string. 445If the 446.Dq previous 447and 448.Dq current 449versions of the file are identical, 450then 451.Nm 452will not change the file in the destination directory. 453.Pp 454Due to limitations in the 455.Xr diff 1 456command, 457this option may not have an effect if there are other changes in a file that 458are close to the FreeBSD ID string. 459.It Fl I Ar patterns 460Ignore any files that match any of the patterns listed in 461.Ar patterns . 462No warnings or other messages will be generated for those files during a 463merge. 464Each pattern is evaluated as an 465.Xr sh 1 466shell pattern. 467This option may be specified multiple times to specify multiple patterns. 468Multiple space-separated patterns may also be specified in a single 469option. 470.It Fl L Ar logfile 471Specify an alternate path for the log file. 472The 473.Nm 474utility logs each command that it invokes along with the standard output 475and standard error to this file. 476By default the log file is stored in a file named 477.Pa log 478in the work directory. 479.It Fl M Ar options 480Pass 481.Ar options 482as additional parameters to 483.Xr make 1 484when building a 485.Dq current 486tree. 487This can be used for to set the 488.Dv TARGET 489or 490.Dv TARGET_ARCH 491variables for a cross-build. 492.It Fl n 493Enable 494.Dq dry-run 495mode. 496Do not merge any changes to the destination directory. 497Instead, 498report what actions would be taken during a merge. 499Note that the existing 500.Dq current 501and 502.Dq previous 503trees will not be changed. 504If the 505.Fl r 506option is not specified, 507then a temporary 508.Dq current 509tree will be extracted to perform the comparison. 510.It Fl p 511Enable 512.Dq pre-world 513mode. 514Only merge changes to files that are necessary to successfully run 515.Sq make installworld 516or 517.Sq make installkernel . 518When this flag is enabled, 519the existing 520.Dq current 521and 522.Dq previous 523trees are left alone. 524Instead, 525a temporary tree is populated with the necessary files. 526This temporary tree is compared against the 527.Dq current 528tree. 529This allows a normal update to be run after 530.Sq make installworld 531has completed. 532Any conflicts generated during a 533.Dq pre-world 534update should be resolved by a 535.Dq pre-world 536.Cm resolve . 537.It Fl r 538Do not update the 539.Dq current 540and 541.Dq previous 542trees during a merge. 543This can be used to 544.Dq re-run 545a previous merge operation. 546.It Fl s Ar source 547Specify an alternate source tree to use when building or extracting a 548.Dq current 549tree. 550The default source tree is 551.Pa /usr/src . 552.It Fl t Ar tarball 553Extract a new 554.Dq current 555tree from a tarball previously generated by the 556.Cm build 557command rather than building the tree from a source tree. 558.El 559.Sh CONFIG FILE 560The 561.Nm 562utility can also be configured by setting variables in an optional 563configuration file named 564.Pa /etc/etcupdate.conf . 565Note that command line options override settings in the configuration file. 566The configuration file is executed by 567.Xr sh 1 , 568so it uses that syntax to set configuration variables. 569The following variables can be set: 570.Bl -tag -width ".Ev ALWAYS_INSTALL" 571.It Ev ALWAYS_INSTALL 572Always install files that match any of the patterns listed in this variable 573similar to the 574.Fl A 575option. 576.It Ev DESTDIR 577Specify an alternate destination directory similar to the 578.Fl D 579option. 580.It Ev EDITOR 581Specify a program to edit merge conflicts. 582.It Ev FREEBSD_ID 583Ignore changes in the FreeBSD ID string similar to the 584.Fl F 585option. 586This is enabled by setting the variable to a non-empty value. 587.It Ev IGNORE_FILES 588Ignore files that match any of the patterns listed in this variable 589similar to the 590.Fl I 591option. 592.It Ev LOGFILE 593Specify an alternate path for the log file similar to the 594.Fl L 595option. 596.It Ev MAKE_OPTIONS 597Pass additional options to 598.Xr make 1 599when building a 600.Dq current 601tree similar to the 602.Fl M 603option. 604.It Ev SRCDIR 605Specify an alternate source tree similar to the 606.Fl s 607option. 608.It Ev WORKDIR 609Specify an alternate work directory similar to the 610.Fl d 611option. 612.El 613.Sh ENVIRONMENT 614The 615.Nm 616utility uses the program identified in the 617.Ev EDITOR 618environment variable to edit merge conflicts. 619If 620.Ev EDITOR 621is not set, 622.Xr vi 1 623is used as the default editor. 624.Sh FILES 625.Bl -tag -width ".Pa /var/db/etcupdate/log" -compact 626.It Pa /etc/etcupdate.conf 627Optional config file. 628.It Pa /var/db/etcupdate 629Default work directory used to store trees and other data. 630.It Pa /var/db/etcupdate/log 631Default log file. 632.El 633.Sh EXIT STATUS 634.Ex -std 635.Sh EXAMPLES 636To compare the files in 637.Pa /etc 638with the stock versions: 639.Pp 640.Dl "etcupdate diff" 641.Pp 642To merge changes after an upgrade via the buildworld and installworld process: 643.Pp 644.Dl "etcupdate" 645.Pp 646To resolve any conflicts generated during a merge: 647.Pp 648.Dl "etcupdate resolve" 649.Ss Bootstrapping 650The 651.Nm 652utility may need to be bootstrapped before it can be used. 653The 654.Cm diff 655command will fail with an error about a missing reference tree if 656bootstrapping is needed. 657.Pp 658Bootstrapping 659.Nm 660requires a source tree that matches the currently installed world. 661The easiest way to ensure this is to bootstrap 662.Nm 663before updating the source tree to start the next world upgrade cycle. 664First, 665generate a reference tree: 666.Pp 667.Dl "etcupdate extract" 668.Pp 669Second, 670use the 671.Cm diff 672command to compare the reference tree to your current files in 673.Pa /etc . 674Undesired differences should be removed using an editor, 675.Xr patch 1 , 676or by copying files from the reference tree 677.Po 678located at 679.Pa /var/db/etcupdate/current 680by default 681.Pc 682. 683.Pp 684If the tree at 685.Pa /usr/src 686is already newer than the currently installed world, 687a new tree matching the currently installed world can be checked out to 688a temporary location. 689The reference tree for 690.Nm 691can then be generated via: 692.Pp 693.Dl "etcupdate extract -s /path/to/tree" 694.Pp 695The 696.Cm diff 697command can be used as above to remove undesired differences. 698Afterwards, 699the changes in the tree at 700.Pa /usr/src 701can be merged via a regular merge. 702.Sh DIAGNOSTICS 703The following warning messages may be generated during a merge. 704Note that several of these warnings cover obscure cases that should occur 705rarely if at all in practice. 706For example, 707if a file changes from a file to a directory in the 708.Dq current 709tree 710and the file was modified in the destination directory, 711then a warning will be triggered. 712In general, 713when a warning references a pathname, 714the corresponding file in the destination directory is not changed by a 715merge operation. 716.Bl -diag 717.It "Directory mismatch: <path> (<type>)" 718An attempt was made to create a directory at 719.Pa path 720but an existing file of type 721.Dq type 722already exists for that path name. 723.It "Modified link changed: <file> (<old> became <new>)" 724The target of a symbolic link named 725.Pa file 726was changed from 727.Dq old 728to 729.Dq new 730in the 731.Dq current 732tree. 733The symbolic link has been modified to point to a target that is neither 734.Dq old 735nor 736.Dq new 737in the destination directory. 738.It "Modified mismatch: <file> (<new> vs <dest>)" 739A file named 740.Pa file 741of type 742.Dq new 743was modified in the 744.Dq current 745tree, 746but the file exists as a different type 747.Dq dest 748in the destination directory. 749.It "Modified <type> changed: <file> (<old> became <new>)" 750A file named 751.Pa file 752changed type from 753.Dq old 754in the 755.Dq previous 756tree to type 757.Dq new 758in the 759.Dq current 760tree. 761The file in the destination directory of type 762.Dq type 763has been modified, 764so it could not be merged automatically. 765.It "Modified <type> remains: <file>" 766The file of type 767.Dq type 768named 769.Pa file 770has been removed from the 771.Dq current 772tree, 773but it has been locally modified. 774The modified version of the file remains in the destination directory. 775.It "Needs update: /etc/localtime (required manual update via tzsetup(8))" 776The 777.Fa /var/db/zoneinfo 778file does not exist, 779so 780.Nm 781was not able to refresh 782.Fa /etc/localtime 783from its source file in 784.Fa /usr/share/zoneinfo . 785Running 786.Xr tzsetup 8 787will both refresh 788.Fa /etc/localtime 789and generate 790.Fa /var/db/zoneinfo 791permitting future updates to refresh 792.Fa /etc/localtime 793automatically. 794.It "Needs update: /etc/mail/aliases.db (required manual update via newaliases(1))" 795The file 796.Pa /etc/mail/aliases 797was updated during a merge with a non-empty destination directory. 798Due to a limitation of the 799.Xr newaliases 1 800command, 801.Nm 802was not able to automatically update the corresponding aliases database. 803.It "New file mismatch: <file> (<new> vs <dest>)" 804A new file named 805.Pa file 806of type 807.Dq new 808has been added to the 809.Dq current 810tree. 811A file of that name already exists in the destination directory, 812but it is of a different type 813.Dq dest . 814.It "New link conflict: <file> (<new> vs <dest>)" 815A symbolic link named 816.Pa file 817has been added to the 818.Dq current 819tree that links to 820.Dq new . 821A symbolic link of the same name already exists in the destination 822directory, 823but it links to a different target 824.Dq dest . 825.It "Non-empty directory remains: <file>" 826The directory 827.Pa file 828was removed from the 829.Dq current 830tree, 831but it contains additional files in the destination directory. 832These additional files as well as the directory remain. 833.It "Remove mismatch: <file> (<old> became <new>)" 834A file named 835.Pa file 836changed from type 837.Dq old 838in the 839.Dq previous 840tree to type 841.Dq new 842in the 843.Dq current 844tree, 845but it has been removed in the destination directory. 846.It "Removed file changed: <file>" 847A file named 848.Pa file 849was modified in the 850.Dq current 851tree, 852but it has been removed in the destination directory. 853.It "Removed link changed: <file> (<old> became <new>)" 854The target of a symbolic link named 855.Pa file 856was changed from 857.Dq old 858to 859.Dq new 860in the 861.Dq current 862tree, 863but it has been removed in the destination directory. 864.El 865.Sh SEE ALSO 866.Xr cap_mkdb 1 , 867.Xr diff 1 , 868.Xr make 1 , 869.Xr newaliases 1 , 870.Xr sh 1 , 871.Xr mergemaster 8 , 872.Xr pwd_mkdb 8 , 873.Xr services_mkdb 8 , 874.Xr tzsetup 8 875.Sh HISTORY 876The 877.Nm 878utility first appeared in 879.Fx 10.0 . 880.Sh AUTHORS 881The 882.Nm 883utility was written by 884.An John Baldwin Aq Mt jhb@FreeBSD.org . 885.Sh BUGS 886Rerunning a merge does not automatically delete conflicts left over from a 887previous merge. 888Any conflicts must be resolved before the merge can be rerun. 889It is not clear if this is a feature or a bug. 890.Pp 891There is no way to easily automate conflict resolution for specific files. 892For example, one can imagine a syntax along the lines of 893.Pp 894.Dl "etcupdate resolve tf /some/file" 895.Pp 896to resolve a specific conflict in an automated fashion. 897.Pp 898Bootstrapping 899.Nm 900often results in gratuitous diffs in 901.Pa /etc/mail/*.cf 902that cause conflicts in the first merge. 903If an object tree that matches the source tree is present when bootstrapping, 904then passing the 905.Fl B 906flag to the 907.Cm extract 908command can work around this. 909