xargs sh -c skipping the first argument
First argument to sh -c
or bash -c
is the name of the script i.e. $0
which is not printed when you use $@
:
Examples:
echo a b c d e f| xargs -n 3 bash -c 'echo "$0 $@"'
a b c
d e f
To fix this you can pass _
as dummy name of the script and then it should work:
echo a b c d e f| xargs -n 3 bash -c 'echo "$@"' _
a b c
d e f
It will work fine even with your sleep
example:
echo a b c d e f| xargs -n 3 bash -c 'echo "$@"; sleep 1' _
a b c
d e f
xargs - place the argument in a different location in the command
The solution: -I
-I
lets you name your argument and put it anywhere you like. E.g.
ls | xargs -n 1 -I {} echo prefix_{}
(replace {}
with any string)
why xargs cannot receive argument
Redirections are handled by the shell before any commands are run. If you want xargs
to handle a redirection, you need to run a subshell.
ls *.sql | xargs -i sh -c 'mysql -uroot -p123456 foo < {}'
However, you should not be using ls
to drive scripts. You want
for f in *.sql; do
mysql -uroot -p123456 foo <"$f"
done
or quite possibly just
cat *.sql | mysql -uroot -p123456 foo
optimize xargs argument enumeration
Try without xargs
. For most situations xargs
is overkill.
Depending on what you really want you can choose a solution like
# Normally you want to avoid for and use while, but here you want the things splitted.
for i in $(echo {1 2 3} );do
echo line $i here;
done
# When you want 1 line turned into three, `tr` can help
echo {1..3} | tr " " "\n" | sed 's/.*/line & here/'
# printf will repeat itself when there are parameters left
printf "line %s here\n" $(echo {1..3})
# Using the printf feature you can avoid the echo
printf "line %s here\n" {1..3}
How can I pass all arguments with xargs in middle of command in linux
This is one way to do it
pdftk $(ls | sort -n) cat output combinewd2.pdf
or using backtick
pdftk `ls | sort -n` cat output combinewd2.pdf
For example, if the filenames are 100, 2, 9, 3.14, 10, 1 the command will be
pdftk 1 2 3.14 9 10 100 cat output combinewd2.pdf
To handle filenames with spaces or other special characters consider this fixed version of @joeytwiddle's excellent answer (which does not sort numerically, see discussion below):
#-- The following will handle special characters, and
# will sort filenames numerically
# e.g. filenames 100, 2, 9, 3.14, 10, 1 results in
# ./1 ./2 ./3.14 ./9 ./10 ./100
#
find . -maxdepth 1 -type f -print0 |
sort -k1.3n -z -t '\0' |
xargs -0 sh -c 'pdftk "$@" cat output combinewd2.pdf' "$0"
Alternatives to xargs (bash specific)
xargs
is an external command, in the previous example it invokes sh
which in turn invokes pdftk
.
An alternative is to use the builtin mapfile
if available, or use the positional parameters. The following examples use two functions, print0_files generates the NUL terminated filenames and create_pdf invokes pdftk
:
print0_files | create_pdf combinewd2.pdf
The functions are defined as follows
#-- Generate the NUL terminated filenames, numerically sorted
print0_files() {
find . -maxdepth 1 -type f -print0 |
sort -k1.3n -z -t '\0'
}
#-- Read NUL terminated filenames using mapfile
create_pdf() {
mapfile -d ''
pdftk "${MAPFILE[@]}" cat output "$1"
}
#-- Alternative using positional parameters
create_pdf() {
local -r pdf=$1
set --
while IFS= read -r -d '' f; do set -- "$@" "$f"; done
pdftk "$@" cat output "$pdf"
}
Discussion
As pointed out in the comments the simple initial answer does not work with filenames containing spaces or other special characters.
The answer by @joeytwiddle does handle special characters, although it does not sort numerically
#-- The following will not sort numerically due to ./ prefix,
# e.g. filenames 100, 2, 9, 3.14, 10, 1 results in
# ./1 ./10 ./100 ./2 ./3.14 ./9
#
find . -maxdepth 1 -type f -print0 |
sort -zn |
xargs -0 sh -c 'pdftk "$@" cat output combinewd2.pdf' "$0"
It does not sort numerically due to each filename being prefixed by ./
by the find
command. Some versions of the find
command support -printf '%P\0'
which would not include the ./
prefix. A simpler, portable fix is to add the -d, --dictionary-order
option to the sort
command so that it considers only blank spaces and alphanumeric characters in comparisons, but might still produce the wrong ordering
#-- The following will not sort numerically due to decimals
# e.g. filenames 100, 2, 9, 3.14, 10, 1 results in
# ./1 ./2 ./9 ./10 ./100 ./3.14
#
find . -maxdepth 1 -type f -print0 |
sort -dzn |
xargs -0 sh -c 'pdftk "$@" cat output combinewd2.pdf' "$0"
If filenames contain decimals this could lead to incorrect numeric sorting. The sort
command does allow an offset into a field when sorting, sort -k1.3n
, one must be careful though in defining the field separator if filenames are to be as general as possible, fortunately sort -t '\0'
specifies NUL as the field separator, and the find -print0
option indicates NUL is to be used as the delimiter between filenames, so sort -z -t '\0'
specifies NUL as both the record delimiter and field separator-- each filename is then a single field record. Given that, we can then offset into the single field and skip the ./
prefix by specifying the 3rd character of the 1st field as the starting position for the numeric sort, sort -k1.3n -z -t '\0'
.
How to skip the first argument in $@?
Use the offset parameter expansion
#!/bin/bash
for i in "${@:2}"; do
echo $i
done
Example
$ func(){ for i in "${@:2}"; do echo "$i"; done;}; func one two three
two
three
Print a file, skipping the first X lines, in Bash
You'll need tail. Some examples:
$ tail great-big-file.log
< Last 10 lines of great-big-file.log >
If you really need to SKIP a particular number of "first" lines, use
$ tail -n +<N+1> <filename>
< filename, excluding first N lines. >
That is, if you want to skip N lines, you start printing line N+1. Example:
$ tail -n +11 /tmp/myfile
< /tmp/myfile, starting at line 11, or skipping the first 10 lines. >
If you want to just see the last so many lines, omit the "+":
$ tail -n <N> <filename>
< last N lines of file. >
Reuse or repeat pipe input as arguments in command (example given)
You can use md5sum
, because I don't have md5
:)
ls -p | grep -v / | xargs md5sum | awk '{print $2,$1}'
and this may be a safer, more robust way:
find -maxdepth 1 -type f -exec md5sum {} \; | awk '{s=$2; $2=$1; $1=s;}1'
this should work with md5:
find -maxdepth 1 -type f -exec md5sum {} \; | sed 's/[^(]*(\([^)]*\)) =/\1/'
Related Topics
Set Multiline Environment Variable with Dockerfile
How to Break an Arbitrary Tcp/Ip Connection on Linux
How to Install Valgrind Properly
How to Acess the Physical Address from Linux Kernel Space
What Really Is the "Linger Time" That Can Be Set with So_Linger on Sockets
Npm Install Causes Errors Like Npm Err! Tar.Unpack Untar Error on Debian
Perl Escaping Argument for Bash Execution
Redirecting Console Output to a File in Unix
"Thread Apply All Bt Full" Gives Blank in Gdb
Is It Safe to Use "Ls" in for Loop in Bash
What Does the "Mov Rax, Qword Ptr Fs:0X28" Assembly Instruction Do
Remove All Text from Last Dot in Bash
Pkill Returns 255 in Combination with Another Command via Remote Ssh
Simulating a Spinner for Progress in Bash
Best Way to Make Linux Web Services
How to Extract One Column from Multiple Files, and Paste Those Columns into One File
No Error on Xcb_Grab_Key But Event Loop Not Catching (Global Hotkey)