Command Substitution Within Sed Expression

Command substitution within sed expression

You can create a sed script from the error message catalog, then apply that sed script to the log file.

Basically, something along these lines:

sed 's/\(.*\), 0x\([0-9A-F]*\)$/s%ERRORID:0x\2%ERROR:\1%g/' errors.txt |
sed -f - logfile.txt

The output from the first sed script should be something like this:

s%ERRORID:0x00000001%ERROR:Out of memory%
s%ERRORID:0x00000002%ERROR:Stack overflow%
s%ERRORID:0x00000031%ERROR:values of beta may cause dom%

That is, a new sed script which specifies a substitution for each error code in the catalog.

There are different dialects of sed so this may require minor tweaking. The sed on Linux I believe should use backslash before grouping parentheses in regular expressions, and gladly tolerate standard input as the argument to the -f option. This is not portable to other Unices, though (but you could substitute Perl for sed if you need portability).

*Edit: If the error messages are fairly static, and/or you want to read the log from standard input, save the generated script in a file;

# Do this once
sed 's/\(.*\), 0x\([0-9A-F]*\)$/s%ERRORID:0x\2%ERROR:\1%g/' errors.txt >errors.sed
# Use it many times
sed -f errors.sed logfile.txt

You could also add #!/usr/bin/sed -f at the top of errors.sed and chmod +x it to make it into a self-contained command script.

Command substitution in sed

You should be able to do this with one sed command:

read -rn 6 date < "$1"
sed -E 's#^([a-zA-Z]+)#'"$date"' \1#g' "$1" > newfile

The capture group ensures there's at least one character on the line before inserting the date.

EDIT: Based on the revision to your question:

newfile="output.txt"
lineone=$(head -1 "$1");

read -rn 6 date <<< "$lineone"
sed -E 's#^([a-zA-Z]+)#'"$date"' \1#g; 1s#^.*$#'"$lineone"'#' "$1" > "$newfile" 

Since you aren't doing an in-place edit, you can do the $date insertions, and then go back and swap out the first line since it would end up with two dates. There might be "better" ways to do this such as using Perl, or losing the second sed command, although this should at least give you a basic idea though on how it works...

Result (newfile):

04/17 Walmart .toys $ 70 .cash $ -70

04/17 Caltex .gas 20 $ .cheque $ -20

04/17 McDonalds .burger 1 $ .cash $ -1

NOTE: In some versions of sed the option for extended regex can
either be -r or -E.

sed substitution with Bash variables

Variables inside ' don't get substituted in Bash. To get string substitution (or interpolation, if you're familiar with Perl) you would need to change it to use double quotes " instead of the single quotes:

# Enclose the entire expression in double quotes
$ sed "s/draw($prev_number;n_)/draw($number;n_)/g" file.txt > tmp

# Or, concatenate strings with only variables inside double quotes
# This would restrict expansion to the relevant portion
# and prevent accidental expansion for !, backticks, etc.
$ sed 's/draw('"$prev_number"';n_)/draw('"$number"';n_)/g' file.txt > tmp

# A variable cannot contain arbitrary characters
# See link in the further reading section for details
$ a='foo
bar'
$ echo 'baz' | sed 's/baz/'"$a"'/g'
sed: -e expression #1, char 9: unterminated `s' command

Further Reading:

• Difference between single and double quotes in Bash
• Is it possible to escape regex metacharacters reliably with sed
• Using different delimiters for sed substitute command
• Unless you need it in a different file you can use the -i flag to change the file in place

Environment variable substitution in sed

Your two examples look identical, which makes problems hard to diagnose. Potential problems:

1. You may need double quotes, as in sed 's/xxx/'"$PWD"'/'

2. $PWD may contain a slash, in which case you need to find a character not contained in $PWD to use as a delimiter.

To nail both issues at once, perhaps

sed 's@xxx@'"$PWD"'@'

Sed string substitution with Shell Variable

Based on the sample data provided the proposed sed can be simplified a bit:

$ sed -r "s|/[0-9]+ |/${REPOVER} |" testing.list
$ sed -r "s|/[[:digit:]]+ |/${REPOVER} |" testing.list

Find and replace the pattern /<at_least_one_number><space> with /${REPOVER}<space>

Both of which produce:

deb [trusted=yes] http://10.47.4.220/repos/test-repo/12  /

Use a variable as replacement in bash sed command

TL;DR:

Try:

sed -i '$ s@$@ '"$1"'@' "$DIR./result/doc.md"

Long version:

Let's start with your original code:

sed -i '$ s/$/ /replacement/' "$DIR./result/doc.md"

And let's compare it to the code you referenced:

sed -i '$ s/$/abc/' file.txt

We can see that they don't exactly match up. I see that you've correctly made this substitution:

file.txt --> "$DIR./result/doc.md"

That looks fine (although I do have my doubts about the . after $DIR ). However, the other substitution doesn't look great:

abc -->  /replacement

You actually introduced another delimeter /. However, if we replace the delimiters with '@' we get this:

sed -i '$ s@$@ /replacement@' "$DIR./result/doc.md"

I think that the above is perfectly valid in sed/bash. The $@ will not be replaced by the shell because it is single quoted. The $DIR variable will be interpolated by the shell because it is double quoted.

Looking at one of your attempts:

sed -i "$ s@$@ $1@" "$DIR./result/doc.md"

You will have problems due to the shell interpolation of $@ in the double quotes. Let's correct that by replacing with single quotes (but leaving $1 unquoted):

sed -i '$ s@$@ '"$1"'@' "$DIR./result/doc.md"

Notice the '"$1"'. I had to surround $1 with '' to basically unescape the surrounding single quotes. But then I surrounded the $1 with double quotes so we could protect the string from white spaces.

GNU sed long substitutions

Perl will let you do this - I recommend looking that up if it's an option -

but in sed you can still use a quoted line break, if perhaps not with the indentation.

sed -r -e 's/'\
'match/'\
'replacement/'\
'options'

In use:

$: echo this needs some match | sed -r -e 's/'\
> 'match/'\
> 'replacement/'\
> 'g'
this needs some replacement

Bash. Replace with sed. End of line problem

Using sed and bash

Since you are using bash, try:

sed -i "s~%text%~${text//$'\n'/\\n}~g" t.txt

${text//$'\n'/\\n} is an example of bash's pattern substitution.
In this case, it replaces all newline characters with with backslash followed by n which sed will interpret as a newline.

Example

Consider this text variable:

$ echo "$text"
line 2
 line 1
 line 3
 line 4

And this input file:

$ cat t.txt
start
%text%
end

Now, run our command:

$ sed "s~%text%~${text//$'\n'/\\n}~g" t.txt
start
line 2
 line 1
 line 3
 line 4
end

To change the file in-place, of course, add the -i option back.

Using awk

With the same text variable and t.txt file as above:

$ awk -v new="$text" '{gsub(/%text%/, new)} 1' t.txt
start
line 2
 line 1
 line 3
 line 4
end

sed: bad option in substitution expression : bad option in substitution expression

use a different sed separator:

sed -i "s#variable \"vpc\".*#variable \"vpc\" { default = \"$string\" }#g"  var.tf

---

Related Topics