How to make AWK use the variable created in Bash Script
You can use it like this.
for i in `find $1 -name \*.jar`
do
jar tvf $i| awk -F '/' '/class/{print "'${i}'" " " $NF }' >> $classFile
done
You should use
"'${i}'"
in AWK to use the
$i
created in Bash Script.
How do I use shell variables in an awk script?
#Getting shell variables into
awk
may be done in several ways. Some are better than others. This should cover most of them. If you have a comment, please leave below. v1.5
Using -v
(The best way, most portable)
Use the -v
option: (P.S. use a space after -v
or it will be less portable. E.g., awk -v var=
not awk -vvar=
)
variable="line one\nline two"
awk -v var="$variable" 'BEGIN {print var}'
line one
line two
This should be compatible with most awk
, and the variable is available in the BEGIN
block as well:
If you have multiple variables:
awk -v a="$var1" -v b="$var2" 'BEGIN {print a,b}'
Warning. As Ed Morton writes, escape sequences will be interpreted so \t
becomes a real tab
and not \t
if that is what you search for. Can be solved by using ENVIRON[]
or access it via ARGV[]
PS If you have vertical bar or other regexp meta characters as separator like |?(
etc, they must be double escaped. Example 3 vertical bars |||
becomes -F'\\|\\|\\|'
. You can also use -F"[|][|][|]"
.
Example on getting data from a program/function inn to
awk
(here date is used)
awk -v time="$(date +"%F %H:%M" -d '-1 minute')" 'BEGIN {print time}'
Example of testing the contents of a shell variable as a regexp:
awk -v var="$variable" '$0 ~ var{print "found it"}'
Variable after code block
Here we get the variable after the awk
code. This will work fine as long as you do not need the variable in the BEGIN
block:
variable="line one\nline two"
echo "input data" | awk '{print var}' var="${variable}"
or
awk '{print var}' var="${variable}" file
- Adding multiple variables:
awk '{print a,b,$0}' a="$var1" b="$var2" file
- In this way we can also set different Field Separator
FS
for each file.
awk 'some code' FS=',' file1.txt FS=';' file2.ext
- Variable after the code block will not work for the
BEGIN
block:
echo "input data" | awk 'BEGIN {print var}' var="${variable}"
Here-string
Variable can also be added to awk
using a here-string from shells that support them (including Bash):
awk '{print $0}' <<< "$variable"
test
This is the same as:
printf '%s' "$variable" | awk '{print $0}'
P.S. this treats the variable as a file input.
ENVIRON
input
As TrueY writes, you can use the ENVIRON
to print Environment Variables.
Setting a variable before running AWK, you can print it out like this:
X=MyVar
awk 'BEGIN{print ENVIRON["X"],ENVIRON["SHELL"]}'
MyVar /bin/bash
ARGV
input
As Steven Penny writes, you can use ARGV
to get the data into awk:
v="my data"
awk 'BEGIN {print ARGV[1]}' "$v"
my data
To get the data into the code itself, not just the BEGIN:
v="my data"
echo "test" | awk 'BEGIN{var=ARGV[1];ARGV[1]=""} {print var, $0}' "$v"
my data test
Variable within the code: USE WITH CAUTION
You can use a variable within the awk
code, but it's messy and hard to read, and as Charles Duffy
points out, this version may also be a victim of code injection. If someone adds bad stuff to the variable, it will be executed as part of the awk
code.
This works by extracting the variable within the code, so it becomes a part of it.
If you want to make an awk
that changes dynamically with use of variables, you can do it this way, but DO NOT use it for normal variables.
variable="line one\nline two"
awk 'BEGIN {print "'"$variable"'"}'
line one
line two
Here is an example of code injection:
variable='line one\nline two" ; for (i=1;i<=1000;++i) print i"'
awk 'BEGIN {print "'"$variable"'"}'
line one
line two
1
2
3
.
.
1000
You can add lots of commands to awk
this way. Even make it crash with non valid commands.
One valid use of this approach, though, is when you want to pass a symbol to awk to be applied to some input, e.g. a simple calculator:
$ calc() { awk -v x="$1" -v z="$3" 'BEGIN{ print x '"$2"' z }'; }
$ calc 2.7 '+' 3.4
6.1
$ calc 2.7 '*' 3.4
9.18
There is no way to do that using an awk variable populated with the value of a shell variable, you NEED the shell variable to expand to become part of the text of the awk script before awk interprets it.
Extra info:
Use of double quote
It's always good to double quote variable "$variable"
If not, multiple lines will be added as a long single line.
Example:
var="Line one
This is line two"
echo $var
Line one This is line two
echo "$var"
Line one
This is line two
Other errors you can get without double quote:
variable="line one\nline two"
awk -v var=$variable 'BEGIN {print var}'
awk: cmd. line:1: one\nline
awk: cmd. line:1: ^ backslash not last character on line
awk: cmd. line:1: one\nline
awk: cmd. line:1: ^ syntax error
And with single quote, it does not expand the value of the variable:
awk -v var='$variable' 'BEGIN {print var}'
$variable
More info about AWK and variables
Read this faq.
How to use awk build in variable in shell script
Something like that?
temp=$(awk '{print $NF}' aa)
Use variable updated in awk bash script
You can do the entire loop in AWK:
awk -v Q="$Q" -v k="$k" -v N="$N" -v col="$col_prob" '{ p_k=Q*k/N; k++ } $col+0<=p_k { print $0 }' $1 > $out_file
This solution is shorter and it will work faster than your bash loop.
I don't know what the values of your variables Q
, k
, N
, col_prob
are but this works:
Q=10e-10
k=1
N=10e+10
col_prob=7
awk -v Q="$Q" -v k="$k" -v N="$N" -v col="$col_prob" '{ p_k=Q*k/N; k++ } $col+0<=p_k { print $0 }' YOURFILE
Output:
XXX 405 0 307 98 G 3.9158562774e-33
P.S. Please, choose "best answer" if an answer works for you. I see from your history that you never did this before. You should.
using awk in Bash script
The crux of the problem is that you're trying to use a single-quoted string as if it were a double-quoted one.
What you meant to do when you used:
'\$2 == "$fruit"' # DOES NOT WORK - nothing is escaped or expanded
was:
"\$2 == \"$fruit\"" # keep literal $2, expand $fruit
Anything inside a single-quoted shell string is taken literally, so variable references aren't expanded, and nothing needs escaping - nor indeed can be escaped (the \
would become a literal part of the string too).
That said, it's better to keep the worlds of the shell and the Awk script separate, so as to prevent confusion, which means:
- Use a single-quoted string as the Awk script.
- Pass any shell-variable values to the script via Awk's
-v
option to create Awk variables.
If we put it all together:
a=$(awk -F'[: ]' -v fruit="$fruit" '$2 == fruit' data.txt)
Note that I've used modern command-substitution syntax $(...)
, which is preferable to legacy syntax `...`
.
awk variables to bash
Sounds like this question: setting multiple variables.
In your case, you can try something like:
awk '{ print($1 " " $2 " " $3 " " $4 " " $5); }' file.dat | { read a b c d e; echo $a $b $c $d $e; }
Or if you need to loop over lines from your file.dat:
awk '{ print( $1 " " $2 " " $3 " " $4 " " $5); }' file.dat | \
{ while read a b c d e; do echo $a $b $c $d $e; done; }
Awk use shell variable inside begin block
Your command is literally matching s
in every line.
You should be using ~
operator:
awk -v s="$site" 'BEGIN {IGNORECASE = 1} $0 ~ s' sites.txt
Note that IGNORECASE = 1
is a gnu awk feature and if you want a POSIX compliant way then use:
awk -v s="$site" 'tolower($0) ~ tolower(s)' sites.txt
Related Topics
Put Every N Rows of Input into a New Column
How to Make Ffmpeg Write Its Output to a Named Pipe
Codeigniter Url Rewriting .Htaccess Is Not Working on Centos
What Are the Possible List of Linux Bash Shell Injection Commands
How to Find the Inode of a Tcp Socket
Setting Up a Workspace Using Team Explorer Everywhere on Linux
How to Make a Bash Script Portable Between Linux and Freebsd
How to Apply Password to Sudo in One Line Command and Execute Su Root
How to Extract Text from a Psd File
Shell Variable Issue When Trying to Mkdir
How to Make Awk Use the Variable Created in Bash Script
How to Redirect Nohup Output to a Specified File
Elf Dynamic Loader Symbol Lookup Ordering
Curl Command for Https ( Ssl )
Parse CSV in Bash and Assign Variables