Grep Fails Inside Bash Script But Works on Command Line

grep fails inside bash script but works on command line

Your script contains a CR at the end of the grep line. Use dos2unix to remove it.

BASH: grep doesn't work in shell script but echo shows correct command and it works on command line

The simple and obvious workaround is to remove all that complexity and simply use the features of the commands you are running anyway.

find "$FILE_DIR" -type f -name '*.json' \
  -exec grep -H -o -f "$SEARCHTEXT_FILE" {} + > "$RESULT_DIR/check_result_$TIMESTAMP.log"

Notice also the quoting fixes; see When to wrap quotes around a shell variable; to avoid mishaps, you should switch to lower case for your private variables (see Correct Bash and shell script variable capitalization).

shopt -s expand_aliases
and source ~/.bashrc merely look superfluous, but could contribute to whatever problem you are trying to troubleshoot; they should basically never be part of a script you plan to use in production.

Can be run on the command line, but not in a shell script?

Instead of trying to build a string holding a complex command, you're better off using grep's -f option and bash process substitution to pass the list of patterns to search for:

head -500 "$search_file_path" | grep -Faf <(printf "%s\n" "${patterns[@]}") > junk.log

It's shorter, simpler and less error prone.

(I added -F to the grep options because none of your example patterns have any regular expression metacharacters; so fixed string searching will likely be faster)

---

The biggest problem with what you're doing is the | is treated as just another argument to head when $cmd is word split. It's not treated as a pipeline delimiter like it is when a literal one is present.

Why grep fails within the script and not in the command line

Do

string="S1 s5 S0 s3 s2 S11 s13"
string1=$(sed 'S/[ ]*s[0-9]*[ ]*//g' <<<"$string") #removing big s
echo "$string1"
string2=$(sed 's/[ ]*S[0-9]*[ ]*//g' <<<"$string") #removing small s
echo "$string2"

Using grep inside shell script gives file not found error

You need to use command substitution:

#!/usr/bin/env bash

test=$(grep 'foo' "$1")
echo "$test"

Command substitution allows the output of a command to replace the command itself. Command substitution occurs when a command is enclosed like this:

$(command)

or like this using backticks:

`command`

Bash performs the expansion by executing COMMAND and replacing the command substitution with the standard output of the command, with any trailing newlines deleted. Embedded newlines are not deleted, but they may be removed during word splitting.

The $() version is usually preferred because it allows nesting:

$(command $(command))

For more information read the command substitution section in man bash.

---

Related Topics