Maximum Number of Bash Arguments != Max Num Cp Arguments

Maximum number of Bash arguments != max num cp arguments?

As Ignacio said, ARG_MAX is the maximum length of the buffer of arguments passed to exec(), not the maximum number of files (this page has a very in-depth explanation). Specifically, it lists fs/exec.c as checking the following condition:

PAGE_SIZE*MAX_ARG_PAGES-sizeof(void *) / sizeof(void *)

And, it seems, you have some additional limitations:

On a 32-bit Linux, this is ARGMAX/4-1 (32767). This becomes relevant if the average length of arguments is smaller than 4.
Since Linux 2.6.23, this function tests if the number exceeds MAX_ARG_STRINGS in <linux/binfmts.h> (2^32-1 = 4294967296-1).
And as additional limit, one argument must not be longer than MAX_ARG_STRLEN (131072).

Too many arguments to Bash script?

Use ${10} instead of $10. You can use this for a seemingly arbitrary number of arguments. I've tested it up to ${100} successfully.

Argument list too long error for rm, cp, mv commands

The reason this occurs is because bash actually expands the asterisk to every matching file, producing a very long command line.

Try this:

find . -name "*.pdf" -print0 | xargs -0 rm

Warning: this is a recursive search and will find (and delete) files in subdirectories as well. Tack on -f to the rm command only if you are sure you don't want confirmation.

You can do the following to make the command non-recursive:

find . -maxdepth 1 -name "*.pdf" -print0 | xargs -0 rm

Another option is to use find's -delete flag:

find . -name "*.pdf" -delete

Argument list too long error when passing a find output as arguments

There is a limit to the argument list length. Since you have several millions files passed to wc, the command certainly crossed this line.

Better invoke find -exec COMMAND instead:

find /some/data/dir/with/text/files/ -type f -exec wc -l {} + > report.txt

Here, each found file find will be appended to the argument list of the command following -exec in place of {}. Before the argument length is reached, the command is run and the remaining found files will be processed in a new run of the command the same way, until the whole list is done.

See man page of find for more details.

---

Thanks to Charles Duffy for the improvements of this answer.

Using getopts to process long and short command line options

There are three implementations that may be considered:

• Bash builtin getopts. This does not support long option names with the double-dash prefix. It only supports single-character options.

• BSD UNIX implementation of standalone getopt command (which is what MacOS uses). This does not support long options either.

• GNU implementation of standalone getopt. GNU getopt(3) (used by the command-line getopt(1) on Linux) supports parsing long options.

---

Some other answers show a solution for using the bash builtin getopts to mimic long options. That solution actually makes a short option whose character is "-". So you get "--" as the flag. Then anything following that becomes OPTARG, and you test the OPTARG with a nested case.

This is clever, but it comes with caveats:

• getopts can't enforce the opt spec. It can't return errors if the user supplies an invalid option. You have to do your own error-checking as you parse OPTARG.
• OPTARG is used for the long option name, which complicates usage when your long option itself has an argument. You end up having to code that yourself as an additional case.

So while it is possible to write more code to work around the lack of support for long options, this is a lot more work and partially defeats the purpose of using a getopt parser to simplify your code.

concatenate ordered files using cat on Linux

Since * expands in a non-numeric-sorted way, you'd better create the sequence yourself with seq: this way, 10 will coome after 9, etc.

for id in $(seq $n)
do
   cat sim.o500.$id >> out.dat
done

Note I use seq so that you are able to use a variable to indicate the length of the sequence. If this value happens to be fixed and known beforehand, you can directly use range expansion writing the n value like: for id in {1..23}.

---

Related Topics