grep: Fix various bugs in recursive tree handling

The -OpS options were effectively ignored due to a collection of
bugs in the use of fts(3):

- fts_open(3) requires one of FTS_PHYSICAL or FTS_LOGIC

grep: Fix various bugs in recursive tree handling

The -OpS options were effectively ignored due to a collection of
bugs in the use of fts(3):

- fts_open(3) requires one of FTS_PHYSICAL or FTS_LOGICAL to be
specified, but in the -O case, only FTS_COMFOLLOW was given. Fix
this to use FTS_COMFOLLOW | FTS_PHYSICAL.

- The switch on the entry type returned by fts_read() did not check
for symbolic links, so symbolic links fell into the default case and
were always passed to procfile() even when -p was given. Fix this
by adding cases in the switch statement to explicitly ignore FTS_SL.

- FTS_NOSTAT was passed to fts_open(), so fts_open() couldn't detect
symbolic links when FTS_PHYSICAL was passed, instead both regular
files and symbolic links were returned as FTS_NSOK entries. Fix
by only using FTS_NOSTAT with FTS_LOGICAL.

While here, fix a few other nits:

- Treat FTS_NS as an error like FTS_DNR and FTS_ERR.

- Just ignore FTS_DP. The logic to skip descending into skipped
directories is only relevant when a directory is first visited, not
after the directory has been visited.

- Use warnc instead of warnx + strerror.

PR: 280676
Reviewed by: kevans
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D46255

show more ...

grep: don't rely on implementation-defined malloc(0) behavior

The very few places that rely on malloc/calloc of a zero-size region
won't attempt to dereference it, so just return NULL rather than ro

grep: don't rely on implementation-defined malloc(0) behavior

The very few places that rely on malloc/calloc of a zero-size region
won't attempt to dereference it, so just return NULL rather than rolling
the dice with the underlying malloc implementation.

Reported by: brooks, Shawn Webb

show more ...

Remove my middle name.

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Remove $FreeBSD$: one-line .c comment pattern

Remove /^/[*/]\s*\$FreeBSD\$.*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

grep: fix combination of quite and count flag

When the quite (-q) flag is provided, we don't expect any output.
Currently, the behavior is broken:
$ grep -cq flag util.c
1

$ grep -cs flag util.c
55

grep: fix combination of quite and count flag

When the quite (-q) flag is provided, we don't expect any output.
Currently, the behavior is broken:
$ grep -cq flag util.c
1

$ grep -cs flag util.c
55

First of all, we print a number to stdout. Secondly, it just returns
0 or 1 (which is unexpected). GNU grep with c and q flags doesn't
print anything.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D31108

show more ...

grep: fix -A handling in conjunction with -m match limitation

The basic issue here is that grep, when given -m 1, would stop all
line processing once it hit the match count and exit immediately. Th

grep: fix -A handling in conjunction with -m match limitation

The basic issue here is that grep, when given -m 1, would stop all
line processing once it hit the match count and exit immediately. The
problem with exiting immediately is that -A processing only happens when
subsequent lines are processed and do not match.

The fix here is relatively easy; when bsdgrep matches a line, it resets
the 'tail' of the matching context to the value supplied to -A and
dumps anything that's been queued up for -B. After the current line has
been printed and tail is reset, we check our mcount and do what's
needed. Therefore, at the time that we decide we're doing nothing, we
know that 'tail' of the context is correct and we can simply continue
on if there's still more to pick up.

With this change, we still bail out immediately if there's been no -A
flag. If -A was supplied, we signal that we should continue on. However,
subsequent lines will not even bothere to try and process the line. We
have reached the match count, so even if the next line would match then
we must process it if it hadn't. Thus, the loop in procfile() can
short-circuit and just process the line as a non-match until
procmatches() indicates that it's safe to stop.

A test has been added to reflect both that we should be picking up the
next line and that the next line should be considered a non-match even
if it should have been.

PR: 253350
MFC-after: 3 days

show more ...

grep: fix null pattern and empty pattern file behavior

The null pattern semantics were terrible because I tried to match gnugrep,
but I got it wrong. Let's unwind that:

- The null pattern should m

grep: fix null pattern and empty pattern file behavior

The null pattern semantics were terrible because I tried to match gnugrep,
but I got it wrong. Let's unwind that:

- The null pattern should match every line if neither -w nor -x.
- The null pattern should match empty lines if -x.
- The null pattern should not match any lines if -w.

The first two will stop processing (shortcut) even if additional patterns
are specified. In any other case, we will continue processing other
patterns. If no other patterns are specified beside a null pattern, then
we match if neither -w nor -x or set and do not match if either of those
are specified.

The justification for -w is that it should match on a whole word, but the
null pattern deos not have a whole word to match on.

Empty pattern files should never match anything, and more importantly, -v
should cause everything to be written.

PR: 253209
MFC-after: 4 days

show more ...

fts_read: Handle error from a NULL return better.

This is addressing cases such as fts_read(3) encountering an [EIO]
from fchdir(2) when FTS_NOCHDIR is not set. That would otherwise be
seen as a su

fts_read: Handle error from a NULL return better.

This is addressing cases such as fts_read(3) encountering an [EIO]
from fchdir(2) when FTS_NOCHDIR is not set. That would otherwise be
seen as a successful traversal in some of these cases while silently
discarding expected work.

As noted in r264201, fts_read() does not set errno to 0 on a successful
EOF so it needs to be set before calling it. Otherwise we might see
a random error from one of the iterations.

gzip is ignoring most errors and could be improved separately.

Reviewed by: vangyzen
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D27184

show more ...

Merge ^/head r352587 through r352763.

bsdgrep(1): various fixes of empty pattern/exit code/-c behavior

When an empty pattern is encountered in the pattern list, I had previously
broken bsdgrep to count that as a "match all" and ignore a

bsdgrep(1): various fixes of empty pattern/exit code/-c behavior

When an empty pattern is encountered in the pattern list, I had previously
broken bsdgrep to count that as a "match all" and ignore any other patterns
in the list. This commit rectifies that mistake, among others:

- The -v flag semantics were not quite right; lines matched should have been
counted differently based on whether the -v flag was set or not. procline
now definitively returns whether it's matched or not, and interpreting
that result has been kicked up a level.
- Empty patterns with the -x flag was broken similarly to empty patterns
with the -w flag. The former is a whole-line match and should be more
strict, only matching blank lines. No -x and no -w will will match the
empty string at the beginning of each line.
- The exit code with -L was broken, w.r.t. modern grep. Modern grap will
exit(0) if any file that didn't match was output, so our interpretation
was simply backwards. The new interpretation makes sense to me.

Tests updated and added to try and catch some of this.

This misbehavior was found by autoconf while fixing ports found in PR 229925
expecting either a more sane or a more GNU-like sed.

MFC after: 1 week

show more ...

bsdgrep(1): Remove redundant initialization; unconditionally assigned later

bsdgrep(1): Some more int -> bool conversions and name changes

Again motivated by upcoming work to rewrite a bunch of this- single-letter
variable names and slightly misleading variable names ("last

bsdgrep(1): Some more int -> bool conversions and name changes

Again motivated by upcoming work to rewrite a bunch of this- single-letter
variable names and slightly misleading variable names ("lastmatches" to
indicate that the last matched) are not helpful.

show more ...

bsdgrep(1): Slooowly peel away the chunky onion

(or peel off the band-aid, whatever floats your boat)

This addresses two separate issues:

1.) Nothing within bsdgrep actually knew whether it cared

bsdgrep(1): Slooowly peel away the chunky onion

(or peel off the band-aid, whatever floats your boat)

This addresses two separate issues:

1.) Nothing within bsdgrep actually knew whether it cared about line numbers
or not.

2.) The file layer knew nothing about the context in which it was being
called.

#1 is only important when we're *not* processing line-by-line. #2 is
debatably a good idea; the parsing context is only handy because that's
where we store current offset information and, as of this commit, whether or
not it needs to be line-aware.

show more ...

bsdgrep(1): Don't initialize fts_flags twice

Admittedly, this is a clang-scan complaint... but it wasn't wrong. fts_flags
is initialized by all cases in the switch(), which should be fairly obvious.

bsdgrep(1): Don't initialize fts_flags twice

Admittedly, this is a clang-scan complaint... but it wasn't wrong. fts_flags
is initialized by all cases in the switch(), which should be fairly obvious.
Annotate this anyways.

show more ...

bsdgrep(1): whoops, garbage collect the now write-only variable

bsdgrep(1): Do some less dirty things with return types

Neither procfile nor grep_tree return anything meaningful to their callers.
None of the callers actually care about how many lines were matche

bsdgrep(1): Do some less dirty things with return types

Neither procfile nor grep_tree return anything meaningful to their callers.
None of the callers actually care about how many lines were matched in all
of the files they processed; it's all about "did anything match?"

This is generally just a light refactoring to remind me of what actually
matters as I'm rewriting these bits to care less about 'stuff'.

show more ...

Remove NLS support from BSD grep

GNU grep as in actually in base does not have any translations support
compiled in, so no functionnality loss.

We do support 193 locales in base, we will never catc

Remove NLS support from BSD grep

GNU grep as in actually in base does not have any translations support
compiled in, so no functionnality loss.

We do support 193 locales in base, we will never catch up on that number of
translation with bsd grep.

Removing NLS support make bsd grep consistent with the other binaries in base
which are not translated, and also reduce a little bit the code.

Reviewed by: kevans
Approved by: kevans
Discussed with: kevans @BSDCan
Differential Revision: https://reviews.freebsd.org/D15682

show more ...

bsdgrep: annihilate our in-tree TRE, previously disabled by default

It was an old TRE that had plenty of bugs and no performance gain over
regex(3). I disabled it by default in r323615, and there wa

bsdgrep: annihilate our in-tree TRE, previously disabled by default

It was an old TRE that had plenty of bugs and no performance gain over
regex(3). I disabled it by default in r323615, and there was some confusion
about what the knob does- likely due to poor naming on my part- to the tune
of "well, it sounds like it should speed things up" (mentioned by multiple
people).

To compound this, I have no intention of maintaining a second regex
implementation. If someone would like to step up and volunteer to maintain a
lean-and-mean implementation for grep, this is OK, but we have very few
volunteers to maintain even our primary regex implementation.

show more ...

bsdgrep: Adjust a missed NLS reference that was invalidated by recent work

Submitted by: Dan McGregor <dan.mcgregor@usask.ca>

bsdgrep: Use grep_strdup instead of grep_malloc+strcpy

bsdgrep: Fix --include/--exclude ordering issues

Prior to r332851:
* --exclude always win out over --include
* --exclude-dir always wins out over --include-dir

r332851 broke that behavior, resultin

bsdgrep: Fix --include/--exclude ordering issues

Prior to r332851:
* --exclude always win out over --include
* --exclude-dir always wins out over --include-dir

r332851 broke that behavior, resulting in:
* First of --exclude, --include wins
* First of --exclude-dir, --include-dir wins

As it turns out, both behaviors are wrong by modern grep standards- the
latest rule wins. e.g.:

`grep --exclude foo --include foo 'thing' foo`
foo is included

`grep --include foo --exclude foo 'thing' foo`
foo is excluded

As tested with GNU grep 3.1.

This commit makes bsdgrep follow this behavior.

Reported by: se

show more ...

bsdgrep: More trivial cleanup/style cleanup

We can avoid branching for these easily reduced patterns

bsdgrep: Some light cleanup

There's no point checking for a bunch of file modes if we're not a
practicing believer of DIR_SKIP or DEV_SKIP.

This also reduces some style violations that were particu

bsdgrep: Some light cleanup

There's no point checking for a bunch of file modes if we're not a
practicing believer of DIR_SKIP or DEV_SKIP.

This also reduces some style violations that were particularly ugly looking
when browsing through.

show more ...