Linux Terminal Input: Reading User Input from Terminal Truncating Lines at 4095 Character Limit

user input in terminal is truncated

Under Linux, the maximum number of characters that can be read in one go on a terminal is determined by N_TTY_BUF_SIZE in the kernel source. The value is 4096.

POSIX limits the number of character acceptable as user input to 4096, how to increase it?

As Mali said: stty -icanon in terminal before launching the program... solves the problem..

POSIX systems support two basic modes of input: canonical and noncanonical (or raw mode).

In canonical input processing mode, terminal input is processed in lines terminated by newline ('\n'), EOF, or EOL characters. The operating system provides input editing facilities: the ERASE and KILL characters are interpreted specially to perform editing operations within the current line of text.

In noncanonical input processing mode (raw mode), characters are not grouped into lines, and ERASE and KILL processing is not performed. The granularity with which bytes are read in raw mode is controlled by the MIN and TIME settings (see man termios the VMIN and VTIME value. VMIN specifies the minimum number of bytes before a read returns. VTIME specifies the number of tenths of a second to wait for data to arrive.

More bascally, in canonical mode, input are accumulated in a buffer (4096) until a \n occured. In raw mode, input is flushed when VMIN or VTIME occured

Thanks Mali..

String from input is limited?

4096 is 2^12

If you want larger input, please consider reading the message from a file instead.

with open('myfile.txt', 'r') as f:
    text = f.read()

Now, text will be a string which is all the text in the file. You can also do:

text = text.split('\n')

Now, text is a list of the lines in your text file

Making terminal input send after a certain number of characters

You've got two options, which solve the same problem in nearly the same way. The first is to use stdbuf when you run your program; the invocation is:

 stdbuf -i0 ./a.out

Using that prevents stdin from being line-buffered, and will let you use fread() or similar commands to retrieve input as it happens.

The other is to put the terminal in raw mode. It's well-described here. But the downside is that control characters are no longer dealt with. In your program, you

#include <termios.h> 

main(){
    struct termios trm;

    tcgetattr(STDIN_FILENO, &trm); /* get the current settings */
    trm.c_cc[VMIN] = 1;     /* return after 1 byte read; you might make this a 2*/
    trm.c_cc[VTIME] = 0;    /* block forever until 1 byte is read */
    tcsetattr(STDIN_FILENO, TCSANOW, &trm); 
}

How to input() value longer than 4095 characters?

Python has to follow the terminal rules. But you could use a system call from python to change terminal behaviour and change it back (Linux):

import subprocess,json

subprocess.check_call(["stty","-icanon"])
result = json.loads(input())
subprocess.check_call(["stty","icanon"])

Alternately, consider trying to get an indented json dump from your provider that you can read line by line, then decode.

data = "".join(sys.stdin.readlines())
result = json.loads(data)

---

Related Topics