How to Increase the Maximum Number of Characters That Ksh Variable Accepts

What is the maximum number of characters that the ksh variable accepts?

The limit for any shell is the limit of the C command line maximum. Here's a little program that pulls the information out of /usr/include/limits.h for you:

cpp <<HERE | tail -1
#include <limits.h>
ARG_MAX
HERE

Mine gives me (256 * 1024) or 262144 characters.

Doesn't work if the C compiler isn't installed, but it's probably a similar limit.

Is it possible to increase the maximum number of characters that ksh variable accepts?

You could compile a Linux 3.7.x kernel, and edit its include/uapi/linux/limits.h file to increase the ARG_MAX argument (to some bigger power of two, e.g. 2097152). But you should rather have a lot of RAM (e.g. 8GBytes) if you want to increase it more.

The actual limit is related to execve(2). That man page has a paragraph on it.

But you could probably avoid having huge shell variables (in the Unix environment). Did you consider using some other tool (awk, python, perl ....) to read your file? Their variable environment is not the shell environment transmitted to forked programs, so they can have variables with very long values. Maybe ksh has some builtin (unexport) to avoid exporting some variable into the Unix environment.

Bash command line and input limit

The limit for the length of a command line is not imposed by the shell, but by the operating system. This limit is usually in the range of hundred kilobytes. POSIX denotes this limit ARG_MAX and on POSIX conformant systems you can query it with

$ getconf ARG_MAX    # Get argument limit in bytes

E.g. on Cygwin this is 32000, and on the different BSDs and Linux systems I use it is anywhere from 131072 to 2621440.

If you need to process a list of files exceeding this limit, you might want to look at the xargs utility, which calls a program repeatedly with a subset of arguments not exceeding ARG_MAX.

To answer your specific question, yes, it is possible to attempt to run a command with too long an argument list. The shell will error with a message along "argument list too long".

Note that the input to a program (as read on stdin or any other file descriptor) is not limited (only by available program resources). So if your shell script reads a string into a variable, you are not restricted by ARG_MAX. The restriction also does not apply to shell-builtins.

What is the maximum size of a Linux environment variable value?

I don't think there is a per-environment variable limit on Linux. The total size of all the environment variables put together is limited at execve() time. See "Limits on size of arguments and environment" here for more information.

A process may use setenv() or putenv() to grow the environment beyond the initial space allocated by exec.

Here's a quick and dirty program that creates a 256 MB environment variable.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

int main(void)
{
  size_t size = 1 << 28; /* 256 MB */
  char *var;

  var = malloc(size);
  if (var == NULL) {
  perror("malloc");
  return 1;
}

  memset(var, 'X', size);
  var[size - 1] = '\0';
  var[0] = 'A';
  var[1] = '=';

  if (putenv(var) != 0) {
  perror("putenv");
  return 1;
}

  /*  Demonstrate E2BIG failure explained by paxdiablo */
  execl("/bin/true", "true", (char *)NULL);
  perror("execl");

  printf("A=%s\n", getenv("A"));

  return 0;
}

ksh increment the while loop argument

Assuming there is the same number of lines in file1 and file2 AND that each line of file1 is either N or Y you could use something like that :

    #!/bin/ksh

    testArray=($(cat file1.txt))

    i=0

    while read line; do
        if [[ ${testArray[$i]} == "N" ]]; then
            print "waste"
        else
            print $line >> file3.txt
        fi
        ((i++))
    done < file2.txt

Or following comments

    #!/bin/ksh

    while read -r -u 3 line1 && read -r -u 4 line2; do
            if [[ $line1 == "N" ]]; then
                    print "waste"
            else
                    print $line2 >> file3.txt
            fi
    done 3<file1.txt 4<file2.txt

Works on my (very) small test files...

Print a character repeatedly in bash

sure, just use printf and a bit of bash string manipulation

$ s=$(printf "%-30s" "*")
$ echo "${s// /*}"
******************************

There should be a shorter way, but currently that's how i would do it. You can make this into a function which you can store in a library for future use

printf_new() {
 str=$1
 num=$2
 v=$(printf "%-${num}s" "$str")
 echo "${v// /*}"
}

Test run:

$ printf_new "*" 20
********************
$ printf_new "*" 10
**********
$ printf_new "%" 10
%%%%%%%%%%

---

Related Topics