How to Pass All Arguments with Xargs in Middle of Command in Linux

How can I pass all arguments with xargs in middle of command in linux

This is one way to do it

pdftk $(ls | sort -n) cat output combinewd2.pdf

or using backtick

pdftk `ls | sort -n` cat output combinewd2.pdf

For example, if the filenames are 100, 2, 9, 3.14, 10, 1 the command will be

pdftk 1 2 3.14 9 10 100 cat output combinewd2.pdf

To handle filenames with spaces or other special characters consider this fixed version of @joeytwiddle's excellent answer (which does not sort numerically, see discussion below):

#-- The following will handle special characters, and
#   will sort filenames numerically
#   e.g. filenames 100, 2, 9, 3.14, 10, 1 results in 
#      ./1 ./2 ./3.14 ./9 ./10 ./100
#
find . -maxdepth 1 -type f -print0 |
  sort -k1.3n -z -t '\0' |
  xargs -0 sh -c 'pdftk "$@" cat output combinewd2.pdf' "$0"

Alternatives to xargs (bash specific)

xargs is an external command, in the previous example it invokes sh which in turn invokes pdftk.

An alternative is to use the builtin mapfile if available, or use the positional parameters. The following examples use two functions, print0_files generates the NUL terminated filenames and create_pdf invokes pdftk:

print0_files | create_pdf combinewd2.pdf

The functions are defined as follows

#-- Generate the NUL terminated filenames, numerically sorted
print0_files() {
    find . -maxdepth 1 -type f -print0 |
        sort -k1.3n -z -t '\0'
}

#-- Read NUL terminated filenames using mapfile
create_pdf() {
    mapfile -d ''
    pdftk "${MAPFILE[@]}" cat output "$1"
}

#-- Alternative using positional parameters
create_pdf() {
    local -r pdf=$1
    set --
    while IFS= read -r -d '' f; do set -- "$@" "$f"; done
    pdftk "$@" cat output "$pdf"
}

Discussion

As pointed out in the comments the simple initial answer does not work with filenames containing spaces or other special characters.
The answer by @joeytwiddle does handle special characters, although it does not sort numerically

#-- The following will not sort numerically due to ./ prefix,
#   e.g. filenames 100, 2, 9, 3.14, 10, 1 results in 
#      ./1 ./10 ./100 ./2 ./3.14 ./9
#
find . -maxdepth 1 -type f -print0 |
  sort -zn |
  xargs -0 sh -c 'pdftk "$@" cat output combinewd2.pdf' "$0"

It does not sort numerically due to each filename being prefixed by ./ by the find command. Some versions of the find command support -printf '%P\0' which would not include the ./ prefix. A simpler, portable fix is to add the -d, --dictionary-order option to the sort command so that it considers only blank spaces and alphanumeric characters in comparisons, but might still produce the wrong ordering

#-- The following will not sort numerically due to decimals
#   e.g. filenames 100, 2, 9, 3.14, 10, 1 results in 
#      ./1 ./2 ./9 ./10 ./100 ./3.14
#
find . -maxdepth 1 -type f -print0 |
  sort -dzn |
  xargs -0 sh -c 'pdftk "$@" cat output combinewd2.pdf' "$0"

If filenames contain decimals this could lead to incorrect numeric sorting. The sort command does allow an offset into a field when sorting, sort -k1.3n, one must be careful though in defining the field separator if filenames are to be as general as possible, fortunately sort -t '\0' specifies NUL as the field separator, and the find -print0 option indicates NUL is to be used as the delimiter between filenames, so sort -z -t '\0' specifies NUL as both the record delimiter and field separator-- each filename is then a single field record. Given that, we can then offset into the single field and skip the ./ prefix by specifying the 3rd character of the 1st field as the starting position for the numeric sort, sort -k1.3n -z -t '\0'.

xargs - place the argument in a different location in the command

The solution: -I

-I lets you name your argument and put it anywhere you like. E.g.

ls | xargs -n 1 -I {} echo prefix_{}

(replace {} with any string)

xargs with multiple arguments

None of the solutions given so far deals correctly with file names containing space. Some even fail if the file names contain ' or ". If your input files are generated by users, you should be prepared for surprising file names.

GNU Parallel deals nicely with these file names and gives you (at least) 3 different solutions. If your program takes 3 and only 3 arguments then this will work:

(echo a1.txt; echo b1.txt; echo c1.txt;
 echo a2.txt; echo b2.txt; echo c2.txt;) |
parallel -N 3 my-program --file={1} --file={2} --file={3}

Or:

(echo a1.txt; echo b1.txt; echo c1.txt;
 echo a2.txt; echo b2.txt; echo c2.txt;) |
parallel -X -N 3 my-program --file={}

If, however, your program takes as many arguments as will fit on the command line:

(echo a1.txt; echo b1.txt; echo c1.txt;
 echo d1.txt; echo e1.txt; echo f1.txt;) |
parallel -X my-program --file={}

Watch the intro video to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ

How to pass command with parameters to xargs

The -c flag to sh only accepts one argument while xargs is splitting the arguments on whitespace - that's why the double quoting works (one level to make it a single word for the shell, one for xargs).

If you use the -0 or null argument to xargs your particular case will work:

echo ls -l -a / | xargs -0 sh -c

How to pass command output as multiple arguments to another command

You can use xargs:

grep 'pattern' input | xargs -I% cp "%" "%.bac"

Passing arrays as command line arguments with xargs

how to pass the array this_chunk using xargs

Note that xargs by default interprets ' " and \ sequences. To disable the interpretation, either preprocess the data, or better use GNU xargs with -d '\n' option. -d option is not part of POSIX xargs.

printf "%s\n" "${this_chunk[@]}" | xargs -d '\n' ./script2.sh

That said, with GNU xargs prefer zero terminated streams, to preserve newlines:

printf "%s\0" "${this_chunk[@]}" | xargs -0 ./script2.sh

Your script ./script2.sh ignores command line arguments, and your xargs spawns the process with standard input closed. Because the input is closed, read -r arr fails, so your scripts does not print anything, as expected. (Note that in POSIX xargs, when the spawned process tries to read from stdin, the result is unspecified.)

Make xargs execute the command once for each line of input

The following will only work if you do not have spaces in your input:

xargs -L 1
xargs --max-lines=1 # synonym for the -L option

from the man page:

-L max-lines
          Use at most max-lines nonblank input lines per command line.
          Trailing blanks cause an input line to be logically continued  on
          the next input line.  Implies -x.

Propagate all arguments in a Bash shell script

Use "$@" instead of plain $@ if you actually wish your parameters to be passed the same.

Observe:

$ cat no_quotes.sh
#!/bin/bash
echo_args.sh $@

$ cat quotes.sh
#!/bin/bash
echo_args.sh "$@"

$ cat echo_args.sh
#!/bin/bash
echo Received: $1
echo Received: $2
echo Received: $3
echo Received: $4

$ ./no_quotes.sh first second
Received: first
Received: second
Received:
Received:

$ ./no_quotes.sh "one quoted arg"
Received: one
Received: quoted
Received: arg
Received:

$ ./quotes.sh first second
Received: first
Received: second
Received:
Received:

$ ./quotes.sh "one quoted arg"
Received: one quoted arg
Received:
Received:
Received:

---

Related Topics