Using Command Substitution Inside a Sed Script, with Arguments

Command substitution within sed expression

You can create a sed script from the error message catalog, then apply that sed script to the log file.

Basically, something along these lines:

sed 's/\(.*\), 0x\([0-9A-F]*\)$/s%ERRORID:0x\2%ERROR:\1%g/' errors.txt |
sed -f - logfile.txt

The output from the first sed script should be something like this:

s%ERRORID:0x00000001%ERROR:Out of memory%
s%ERRORID:0x00000002%ERROR:Stack overflow%
s%ERRORID:0x00000031%ERROR:values of beta may cause dom%

That is, a new sed script which specifies a substitution for each error code in the catalog.

There are different dialects of sed so this may require minor tweaking. The sed on Linux I believe should use backslash before grouping parentheses in regular expressions, and gladly tolerate standard input as the argument to the -f option. This is not portable to other Unices, though (but you could substitute Perl for sed if you need portability).

*Edit: If the error messages are fairly static, and/or you want to read the log from standard input, save the generated script in a file;

# Do this once
sed 's/\(.*\), 0x\([0-9A-F]*\)$/s%ERRORID:0x\2%ERROR:\1%g/' errors.txt >errors.sed
# Use it many times
sed -f errors.sed logfile.txt

You could also add #!/usr/bin/sed -f at the top of errors.sed and chmod +x it to make it into a self-contained command script.

Command substitution in sed

You should be able to do this with one sed command:

read -rn 6 date < "$1"
sed -E 's#^([a-zA-Z]+)#'"$date"' \1#g' "$1" > newfile

The capture group ensures there's at least one character on the line before inserting the date.

EDIT: Based on the revision to your question:

newfile="output.txt"
lineone=$(head -1 "$1");

read -rn 6 date <<< "$lineone"
sed -E 's#^([a-zA-Z]+)#'"$date"' \1#g; 1s#^.*$#'"$lineone"'#' "$1" > "$newfile" 

Since you aren't doing an in-place edit, you can do the $date insertions, and then go back and swap out the first line since it would end up with two dates. There might be "better" ways to do this such as using Perl, or losing the second sed command, although this should at least give you a basic idea though on how it works...

Result (newfile):

04/17 Walmart .toys $ 70 .cash $ -70

04/17 Caltex .gas 20 $ .cheque $ -20

04/17 McDonalds .burger 1 $ .cash $ -1

NOTE: In some versions of sed the option for extended regex can
either be -r or -E.

How to store sed arguments in variable for parameter substitution in BASH

Don't use a string, use an array:

myArgs=("-e" "s/foo/bar/" "-e" "s/foo2/bar2/")
sed "${myArgs[@]}" <<< "foo"

Use a variable as replacement in bash sed command

TL;DR:

Try:

sed -i '$ s@$@ '"$1"'@' "$DIR./result/doc.md"

Long version:

Let's start with your original code:

sed -i '$ s/$/ /replacement/' "$DIR./result/doc.md"

And let's compare it to the code you referenced:

sed -i '$ s/$/abc/' file.txt

We can see that they don't exactly match up. I see that you've correctly made this substitution:

file.txt --> "$DIR./result/doc.md"

That looks fine (although I do have my doubts about the . after $DIR ). However, the other substitution doesn't look great:

abc -->  /replacement

You actually introduced another delimeter /. However, if we replace the delimiters with '@' we get this:

sed -i '$ s@$@ /replacement@' "$DIR./result/doc.md"

I think that the above is perfectly valid in sed/bash. The $@ will not be replaced by the shell because it is single quoted. The $DIR variable will be interpolated by the shell because it is double quoted.

Looking at one of your attempts:

sed -i "$ s@$@ $1@" "$DIR./result/doc.md"

You will have problems due to the shell interpolation of $@ in the double quotes. Let's correct that by replacing with single quotes (but leaving $1 unquoted):

sed -i '$ s@$@ '"$1"'@' "$DIR./result/doc.md"

Notice the '"$1"'. I had to surround $1 with '' to basically unescape the surrounding single quotes. But then I surrounded the $1 with double quotes so we could protect the string from white spaces.

sed find and replace from line number

Using sed

$ sed "3c $(sed -n '3p' a.py)" b.py
import b

print('file a')

Use command substitution to allow another sed command to be run as an arguments and c to replace the contents of line 3.

Is it possible to pass a backreference to a function from inside sed?

How to properly escape a backreference in sed to pass it to a function?

The presented code properly handles the backreference. The backreference inside command substitution is not and will not be parsed by sed, but by shell, before running sed. The arguments to a program have to be expanded before running the program.

You may potentially use a GNU extension to sed - the e flag to s command that executes the replacement pattern via /bin/sh interpret. Using this flag is highly discouraged and is very hard to use, as figuring the correct quoting and escaping is very hard - it "works" in very simple cases. Because the input string has ; < > and also " special shell characters I doubt it's possible.

I suggest to pick a full fledged programming language, like python, perl or others, to solve your task. sed is not an utility for dynamically executing actions depending on contents of the file, it's a simple stream replacement utility.

In sed, it is possible to build a static list of strings to replace, like so:

sed -r '
     s/(<CHARSET c="T">)02C8/\1'"$(echo -e "\u02C8")"/
     s/(<CHARSET c="T">)0252/\1'"$(echo -e "\u0252")"/
     .... one s/// command for each character to replace ...
'

Sed substitution on capture group

As mentioned in the comments, sed probably isn't needed.

Here's a pure bash version:

my_func(){
    local cmd arg1 rest flags

    if [[ $2 =~ [^a-zA-Z0-9] ]]; then
        echo 1>&2 "bad character in arg2"
        exit 1
    fi

    read -r cmd arg1 rest <<< $1

    if [[ $arg1 =~ ^[^-] ]]; then
        # no flags
        echo "$cmd -$2 $arg1${rest:+ $rest}"
    else
        # have flags

        # strip unwanted/duplicates and append new
        eval 'flags=${arg1//[cd'$2']/}$2'

        # append new flags and return
        echo "$cmd $flags $rest"
    fi
}

eval is dangerous if passed untrusted data (as is your sed using $2 unchecked) - always do a sanity check - I've assumed here that flags must be alphanumerics.

---

If you want to use sed, this should work (same caveat about sanity-checking $2:

my_func(){
    sed <<< $1 '
        /^\([^ ][^ ]*\) \([^-]\)/ { # no flags
            s//\1 -'$2' \2/;
            q;
         }
        /^[^ ][^ ]* -\([^ ]*\) .*/ { # has flags
            h;
            s//\1/;
            s/[cd'$2']//g;
            G;
            s/^\([^\n]*\)\n\([^ ][^ ]* -\)[^ ]*\( .*\)/\2\1'$2'\3/;
            q;
        }
        {
            s/.*/#ERROR: malformed input/;
            q;
        }
    '
}

---

Related Topics