bsdconfig: rework packages selection TUI

Rework the packages TUI, do that the index caching is now done with
dialog --gauge (tested with cdialog and bsddialog).
With pkg we can know in avance the nu

bsdconfig: rework packages selection TUI

Rework the packages TUI, do that the index caching is now done with
dialog --gauge (tested with cdialog and bsddialog).
With pkg we can know in avance the number of packages making it
possible to have a real gauge.

The cache of the index is now a file that can be sourced, meaning it
is not anymore an index like file, but a post process one, simplifying
the code.

Each menu is now built calling directly pkg rquery with just the
informations required to build the menu instead of parsing an indexfile

install all the awk index processing into a separate file to ease
reading and debuggung

show more ...

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

usr.sbin/bsdconfig: minor spelling fixes on comments.

No functional change.

MfH @r295202

Expect to see panics in routing code at least now.

Merge ^/head r294777 through r294960.

MFH

Sponsored by: The FreeBSD Foundation

Use dpv(1) in `bsdconfig packages'

Merge from head@274682

IFC @r271694

IFC @r269962

Submitted by: Anish Gupta (akgupt3@gmail.com)

Sync to HEAD@r270409.

Add `-A' flag to pkg-install(8) invocation when installing dependencies.

MFC after: 3 days
X-MFC-to: stable/10, stable/9
Reported by: gjb
Discussed with: jelischer, gjb, bdrewery

Merge head from 7/28

Sync to HEAD@r269943.

Add example script `add_some_packages.sh', demonstrating how to install
a list of packages using the bsdconfig(8) API, providing dialog(8) based
user experience (versus plain console output were you

Add example script `add_some_packages.sh', demonstrating how to install
a list of packages using the bsdconfig(8) API, providing dialog(8) based
user experience (versus plain console output were you to use pkg(8)
directly to install the same list of packages).

Remove example script `browse_packages_ftp.sh', made obsolete because
the digests.txz and packagesite.txz databases for pkg(8) are not
available via FTP (HTTP only to pkg.freebsd.org SRV hosts).

Update example script `browse_packages_http.sh', made to work with new
pkg(8) demonstrating how to generate a local package repository.

Fix a bug in `bsdconfig packages' where packages were listed twice. This
fix requires pkg(8) version 1.2.7_4 or higher.

NB: It is the introduction of pkg(8) 1.2.7_4 wherein I am also able to
drastically reduce the generation time of package dependencies prior to
the dialog display (by utilizing the new `-I' flag to pkg-rquery(8)).

While here, fix a positional argument nit for f_index_initialize() of
`packages/index.subr' include (the one and only argument is positional
argument 1 to state the by-ref handle, indicating the variable to set
in the caller's namespace; the nit I'm fixing here is that we were
querying positional argument 2 for this information incorrectly; caused
by a missing hunk back around SVN r257795).

Fix a bug in sysrc(8) described by PR bin/187458 "sysrc(8) silently and
unexpectedly bootstraps pkg". This was caused by an explicit entry in
`/usr/share/bsdconfig/common.subr' (used by sysrc(8)) that called pkg(8)
to populate the $PKG_ABI global (called in a way that allows implicit
bootstrap of pkg(8)). The solution to which was to find every place in
bsdconfig(8) that requires the $PKG_ABI global and add a layer of
protection by way of introducing the new API call f_musthavepkg_init()
(provided by new include, `/usr/share/bsdconfig/packages/musthavepkg.subr'
intented to mirror `/usr/share/bsdconfig/mustberoot.subr'). When the
$PKG_ABI global is required, you can use `f_musthavepkg_init' to cause
graceful premature termination in the event that pkg(8) has not yet been
bootstrapped, and thus cannot be used to populate $PKG_ABI.

NB: If running interactively ($nonInteractive is NULL or unset), the
f_musthavepkg_init() API call will attempt to bootstrap pkg(8), but only
if the user chooses "Yes" to a Yes/No dialog confirming it is ok to
bootstrap.

While here, simplify an if-conditional in `media/http.subr' include to
use sh(1) inline assignment-with-break.

Also, fix a bug in `media/http.subr' and `media/httpproxy.subr' wherein
the error messages for various HTTP failures were not finding their way
to the console (needed to pass stdout to pass-thru descriptor).

While here, remove the executable bit from `packages/categories.subr',
`packages/index.subr', and `packages/packages.subr' includes.

Fix issues where pkg(8) complains about PACKAGESITE being defined.
Previously, we would set $PACKAGESITE and export it. Now we only set
$PACKAGESITE for invocations of "pkg update" -- getting rid of all the
spurious warnings about PACKAGESITE being deprecated (it's still used
in the case of "pkg update" for simplicity versus having to configure
a config-file).

Remove the leading argument from invocations of f_index_initialize() in
`packages/packages.subr' include. The leading argument no longer means
what it used to, pre SVN r257995).

PR: bin/187458
Reviewed by: nwhitehorn
MFC after: 1 week
X-MFC-to: stable/10, stable/9

show more ...

MFC @ r266724

An SVM update will follow this.

IFC @266473

Merge head

Implement GEOM based media device classification. You'll notice a few
different things from this commit:
+ More devices. Devices that were previously ignored are now present.
+ Faster device scanning

Implement GEOM based media device classification. You'll notice a few
different things from this commit:
+ More devices. Devices that were previously ignored are now present.
+ Faster device scanning. "There is no try, only Do" -- f_device_try()
is no longer the basis of device scanning as GEOM provides [nearly]
all devices (doesn't provide network devices).
+ More information available as non-root. Usually you have to be root
to do things like taste filesystems, and that limits the amount of
information available to non-root users; with GEOM, we see all even
running unprivileged as the brunt of information (except for so-
called ``dangerously dedicated'' file systems) is represented by the
`kern.geom.confxml' sysctl(8) MIB.
NB: Only really useful for external scripts that use the API and run as
non-root; where this code is used in bsdconfig(8) and bsdinstall(8)
you are running as root so can detect even ``dangerously dedicated''
file systems that are not present in GEOM; e.g., no PART class for
a DOS filesystem written directly to disk without partition table).
+ No more use of legacy tools such as diskinfo(8) to get disk capacity
or fdisk(8) to see partitions.

MFC after: 1 week

show more ...

IFC @264767

Merge head up to r262222 (last merge was incomplete).

MFC @ r259205 in preparation for some SVM updates. (for real this time)

Performance and debugging enhancements:
+ Remove UNAME_P=$(...) from startup/misc -- already supplied by common.subr
+ Use f_getvar instead of $(eval echo \$$var) -- f_getvar is sub-shell free
+ Add

Performance and debugging enhancements:
+ Remove UNAME_P=$(...) from startup/misc -- already supplied by common.subr
+ Use f_getvar instead of $(eval echo \$$var) -- f_getvar is sub-shell free
+ Add `-e' and `-k var' options to f_eval_catch -- increasing use-cases
+ Use f_eval_catch to display errors on failure -- reducing duplicated code
+ Use f_eval_catch when we need output from a command -- improving debugging
+ Optimize f_isinter of strings.subr for performance -- now sub-shell free
+ Improve error checking on pidfiles -- using f_eval_catch and f_isinteger
+ Use $var_to_set arg of f_ifconfig_{inet,netmask} -- eliminate sub-shells
+ Use f_sprintf instead of $(printf ...) -- consolidate sub-shells
+ Use $var_to_set arg of f_route_get_default -- eliminate sub-shells
+ Add f_count to replace $(set -- ...;echo $#) -- eliminate sub-shells
+ Add f_count_ifs to replace $(IFS=x;set -- ...;echo $#) -- no sub-shells
+ Replace var="$var${var:+ }..." in loops with var="$var ..." with a follow-
up var="${var# }" to trim leading whitespace -- optimize loops
+ Use $var_to_set arg of f_resolv_conf_nameservers -- eliminate sub-shells
+ Comments for the f_eval_catch function
+ Remove a duplicate `local ... desc ...' in f_device_get_all of device.subr
+ Use $var_to_set arg of f_device_capacity -- eliminate sub-shells
+ Whitespace fixes in f_dialog_init of dialog.subr
+ Optimize f_inet_atoi of media/tcpip.subr for performance -- sub-shell free
+ In several cases, send stderr to /dev/null -- clean up runtime execution
+ Change f_err of common.subr to go to program stderr not terminal stderr,
allowing redirection of output from functions that use f_err
+ Disable debugging when using f_getvar to get variable argument to
f_startup_rcconf_map_expand of startup/rcconf.subr
+ Use f_replace_all instead of $(echo ... | tr | sed) -- performance
+ Add a $var_to_set option to f_index_{file,menusel_{command,keyword}} of
common.subr -- centralize sub-shells

show more ...

Merge head up to r258343.

Fix package installation from physical media such as DVD.

Discussed with: re (gjb)
MFC after: 3 days