Bash Extglob Negate Not Working as I Expect

BASH extglob negate not working as I expect

Let's start with these files:

$ ls
a.b  b.c  testdir2  t.tar

This matches all files that have a period but do not have tar following the period:

$ echo *.!(tar)
a.b b.c

This matches all files that do not end with tar:

$ echo !(*tar)
a.b b.c testdir2

This is the extglob that surprises but shouldn't:

$ echo *!(tar)
a.b b.c testdir2 t.tar

The * itself can match anything, such as t.tar. Since it is not necessary to add tar after t.tar, this is a match.

How does negative matching work in extglob in parameter expansion

Parameter expansion of the form ${parameter/pattern/string} (where pattern doesn't start with a /) works by finding the leftmost longest substring in the value of the variable parameter that matches the pattern pattern and replacing it with string. In other words, $parameter is decomposed into three parts prefix,match, and suffix such that

1. $parameter == "${prefix}${match}${suffix}"
2. $prefix is the shortest possible string enabling the other requirements to be fulfilled (i.e. the match, if at all possible, occurs in the leftmost position)
3. $match matches pattern and is as long as possible
4. any of $prefix, $match and/or $suffix can be empty

and the result of ${parameter/pattern/string} is "${prefix}string${suffix}".

For the global replacement form (${parameter//pattern/string}) of this type of parameter expansion, the same process is recursively performed for the suffix part, however a zero-length match is handled as a special case (in order to prevent infinite recursion):

• if "${prefix}${match}" != ""

  "${parameter//pattern/string}" = "${prefix}string${suffix//pattern/string}"
  

  else suffix=${parameter:1} and

  "${parameter//pattern/string}" = "string${parameter:0:1}${suffix}//pattern/string}"
  

Now let's analyze the cases individually:

• "${a/!([0-9])/}" --> prefix='' match='1 2 3 4 5 6 7 8 9 10' suffix=''. Indeed, '1 2 3 4 5 6 7 8 9 10' is not a string consisting of a single digit, and therefore it matches the pattern !([0-9]). Hence the empty result of expansion.

• "${a/!(2)/}" --> prefix='' match='1 2 3 4 5 6 7 8 9 10' suffix=''. Similar to the above, '1 2 3 4 5 6 7 8 9 10' is not a string consisting of the single character '2', and therefore it matches the pattern !(2). Hence the empty result of expansion.

• "${a/!(*2*)/}" --> prefix='' match='1 ' suffix='2 3 4 5 6 7 8 9 10'. The substring '1 ' doesn't match the pattern *2*, and therefore it matches the pattern !(*2*).

• "${a/!(*2*)/,}". There were no surprises here, so no need to elaborate.

• "${a//!(*2*)/}". There were no surprises here, so no need to elaborate.

• "${a//!(*2*)/,}" --> prefix='' match='1 ' suffix='2 3 4 5 6 7 8 9 10'. Then ${suffix//!(*2*)/,} expands to ",2," as follows. The empty string in the beginning of suffix matches the pattern !(*2*), producing an extra comma in the result. Since the zero-length match special case (described above) was triggered, the first character of suffix is forcibly consumed, leaving us with ' 3 4 5 6 7 8 9 10', which matches the !(*2*) pattern in its entirety and is replaced with the last comma that we see in the final result of the expansion.

shopt -s extglob not working

The correct syntax for extended globs is

!(pattern-list)

The exclamation mark should go outside the parentheses.

Why doesn't this Bash globbing pattern work?

Because the * matches IMG_1236-renamed, !(-renamed) matches an empty string, and .jpg matches itself.

Extended glob does not expand as expected

The problem is that $* is in double quotes, which means that its contents will not be treated as a pattern, just like echo "*" does not expand the asterisk. Combining the outer pattern with the inner quoted portion automatically escapes the latter, so !("b|c") is treated like !(b\|c). Negation of the nonexistent b|c file naturally expands to all files in the directory.

An additional problem is that extended globbing is messed up by IFS being set to |, so you must reset it before expanding the pattern. Therefore, you must do the expansion in two steps: first, calculate the pattern, then reset IFS and expand it:

#!/usr/bin/env bash

# enable extended globbing
shopt -s extglob

# temporarily set IFS to | so that $* expands to part an extended pattern
old_ifs=$IFS
IFS='|'
pattern="!($*)"
IFS=$old_ifs

printf '%s' $pattern

Negate in bash extended globs does not work

* is matching everything, so *!(8)* is always going to match everything - first !(8) will not match anything (match empty), then * will match everything.

atop_20210 @(3|4) * [0-4]  *  !(8)  *
atop_20210    4       2             8

Why all the *? Remove them. You want to just match what you want to match, not to match anything in between. Just:

atop_20210@(3|4)[0-4]!(8)

How does extglob work with shell parameter expansion?

If you use @ instead of ? then it works as expected:

$> echo "${V#@(35|88)}"
xAB

$> echo "${V%@(xAB|Bzh)}"
35

Similarly behavior of + instead of *:

$> echo "${V#*(35|88)}"
35xAB

$>echo "${V#+(35|88)}"
xAB

It is because:

• ?(pattern-list) # Matches zero or one occurrence of the given patterns
• @(pattern-list) # Matches one of the given patterns

And:

• *(pattern-list) # Matches zero or more occurrences of the given patterns
• +(pattern-list) # Matches one or more occurrences of the given patterns

Bash negative wildcard using sub shell with `bash -c`

That's because !(..) is a extended glob pattern that is turned on by default in your interactive bash shell, but in an explicit sub-shell launched with -c, the option is turned off. You can see that

$ shopt | grep extglob
extglob         on
$ bash -c 'shopt | grep extglob'
extglob         off

One way to turn on the option explicitly in command line would be to use the -O flag followed by the option to be turned on

$ bash -O extglob -c 'shopt | grep extglob'
extglob         on  

See extglob on Greg's Wiki for the list of extended glob patterns supported and The Shopt Builtin for a list of the extended shell options and which ones are enabled by default.

Unable to exclude a single file from getting deleted after enabling bash extglob

Assuming save.tar is inside the test folder, you don't need to use a separate path to exclude the file from getting deleted. The negate operator already globs all the files except the one you provide inside !().

You can just do

rm -rvf "$outpath"/!(save.tar)

Or if you multiple .tar files which need to be excluded, you can move the glob inside the negation as !(*.tar) which means delete everything except the files ending with .tar

A simple reproducible example of your problem is below. Here all the files except file6 are commanded to be deleted and works as expected.

$ mkdir -p /tmp/foo/bar/test
$ touch /tmp/foo/bar/test/file{1..7}
ls /tmp/foo/bar/test/
file1  file2  file3  file4  file5  file6  file7
$ outpath=/tmp/foo/bar/test
$ rm -rvf "$outpath"/!(file6)
removed ‘/tmp/foo/bar/test/file1’
removed ‘/tmp/foo/bar/test/file2’
removed ‘/tmp/foo/bar/test/file3’
removed ‘/tmp/foo/bar/test/file4’
removed ‘/tmp/foo/bar/test/file5’
removed ‘/tmp/foo/bar/test/file7’

See more on bash extended globbing.

Can I do a negated wildcard on the command-line?

It depends on your shell.

In ksh, you can use this:

 !(*tmp*)

In bash, the same thing works if you first enable the feature with shopt -s extglob.

In zsh, you can enable the same syntax with setopt ksh_glob, but there's a conflict with another zsh feature that you have to disable with setopt no_bare_glob_qual before the above will actually work. Alternatively, you can just use zsh's native version via setopt extended_glob; the equivalent of the above expression then looks like this:

   ^*tmp*

---

Related Topics