Less Gets Keyboard Input from Stderr

Less gets keyboard input from stderr?

$ ls -l /dev/fd/
lrwx------ 1 me me 64 2009-09-17 16:52 0 -> /dev/pts/4
lrwx------ 1 me me 64 2009-09-17 16:52 1 -> /dev/pts/4
lrwx------ 1 me me 64 2009-09-17 16:52 2 -> /dev/pts/4

When logged in at an interative terminal, all three standard file descriptors point to the same thing: your TTY (or pseudo-TTY).

$ ls -fl /dev/std{in,out,err}
lrwxrwxrwx 1 root root 4 2009-09-13 01:57 /dev/stdin -> fd/0
lrwxrwxrwx 1 root root 4 2009-09-13 01:57 /dev/stdout -> fd/1
lrwxrwxrwx 1 root root 4 2009-09-13 01:57 /dev/stderr -> fd/2

By convention, we read from 0 and write to 1 and 2. However, nothing prevents us from doing otherwise.

When your shell runs ls -l * | less, it creates a pipe from ls's file descriptor 1 to less's file descriptor 0. Obviously, less can no longer read the user's keyboard input from file descriptor 0 – it tries to get the TTY back however it can.

If less has not been detached from the terminal, open("/dev/tty") will give it the TTY.

However, in case that fails... what can you do? less makes one last attempt at getting the TTY, assuming that file descriptor 2 is attached to the same thing that file descriptor 0 would be attached to, if it weren't redirected.

This is not failproof:

$ ls -l * | setsid less 2>/dev/null

Here, less is given its own session (so it is no longer a part of the terminal's active process group, causing open("/dev/tty") to fail), and its file descriptor 2 has been changed – now less exits immediately, because it is outputting to a TTY yet it fails to get any user input.

How does a program like less handle both piped stdin and keyboard input?

It reads keyboard input from /dev/tty instead of from stdin.

How does less command get stdin input?

less command can check both argc>1 and stdin be associated with a file(not a tty).

Confused about stdin, stdout and stderr?

Standard input - this is the file handle that your process reads to get information from you.

Standard output - your process writes conventional output to this file handle.

Standard error - your process writes diagnostic output to this file handle.

That's about as dumbed-down as I can make it :-)

Of course, that's mostly by convention. There's nothing stopping you from writing your diagnostic information to standard output if you wish. You can even close the three file handles totally and open your own files for I/O.

When your process starts, it should already have these handles open and it can just read from and/or write to them.

By default, they're probably connected to your terminal device (e.g., /dev/tty) but shells will allow you to set up connections between these handles and specific files and/or devices (or even pipelines to other processes) before your process starts (some of the manipulations possible are rather clever).

An example being:

my_prog <inputfile 2>errorfile | grep XYZ

which will:

• create a process for my_prog.
• open inputfile as your standard input (file handle 0).
• open errorfile as your standard error (file handle 2).
• create another process for grep.
• attach the standard output of my_prog to the standard input of grep.

---

Re your comment:

When I open these files in /dev folder, how come I never get to see the output of a process running?

It's because they're not normal files. While UNIX presents everything as a file in a file system somewhere, that doesn't make it so at the lowest levels. Most files in the /dev hierarchy are either character or block devices, effectively a device driver. They don't have a size but they do have a major and minor device number.

When you open them, you're connected to the device driver rather than a physical file, and the device driver is smart enough to know that separate processes should be handled separately.

The same is true for the Linux /proc filesystem. Those aren't real files, just tightly controlled gateways to kernel information.

What is “standard input”?

The C standard (e.g. C99 or C11) defines what should be expected from the standard <stdio.h> header (after having suitably #include-d it). See stdio(3) man page.
Then you have the stdin and stdout and stderr file handles (pointers to some FILE which is an abstract data type).

The fact that stdin is related to some device (e.g. a keyboard) is implementation specific.

You could (but that would be unethical and/or inefficient) implement the C standard with e.g. a room of human slaves (that is unethical, if you use paid workers that would be just inefficient), instead of using a computer. Often, computers gives your some implementation of the C standard thru the help of some operating system.

You may want to know, inside your C program, if stdin is a "keyboard" or redirected from some "file". Unfortunately, AFAIK, there is no C99-standard way to know that.

As you mention, stdin, stdout and stderr should be available in your program at startup (i.e. after entering main ....). Hence, unless you fclose the stdin stream, you can read it (with getchar, scanf, getline, fgets, fscanf ... and friends) without any prior care (so you don't need to fopen it yourself).

On Linux or most Posix systems, you might use as an approximation isatty(STDIN_FILENO) - see isatty(3) for more - to test if stdin "is" the "keyboard" (by testing if it is some tty). See also this & that.

How can I read stderr with python if I'm NOT using subprocess?

Under the hood, subprocess uses anonymous pipes in addition to dups to redirect subprocess output. To get a process to read its own stderr, you need to do this manually. It involves getting an anonymous pipe, redirecting the standard error to the pipe's input, running the stderr-writing action in question, reading the output from the other end of the pipe, and cleaning everything back up. It's all pretty fiddly, but I think I got it right in the code below.

The following wrapper for your cwiid.Wiimote call will return a tuple consisting of the result returned by the function call (None in case of RuntimeError) and stderr output generated, if any. See the tests function for example of how it's supposed to work under various conditions. I took a stab at adapting your example loop but don't quite understand what's supposed to happen when the cwiid.Wiimote call succeeds. In your example code, you just immediately re-loop.

Edit: Oops! Fixed a bug in example_loop() where Wiimote was called instead of passed as an argument.

import time

import os
import fcntl

def capture_runtime_stderr(action):
    """Handle runtime errors and capture stderr"""
    (r,w) = os.pipe()
    fcntl.fcntl(w, fcntl.F_SETFL, os.O_NONBLOCK)
    saved_stderr = os.dup(2)
    os.dup2(w, 2)
    try:
        result = action()
    except RuntimeError:
        result = None
    finally:
        os.close(w)
        os.dup2(saved_stderr, 2)
        with os.fdopen(r) as o:
            output = o.read()
    return (result, output)

## some tests

def return_value():
    return 5

def return_value_with_stderr():
    os.system("echo >&2 some output")
    return 10

def runtime_error():
    os.system("echo >&2 runtime error occurred")
    raise RuntimeError()

def tests():
    print(capture_runtime_stderr(return_value))
    print(capture_runtime_stderr(return_value_with_stderr))
    print(capture_runtime_stderr(runtime_error))
    os.system("echo >&2 never fear, stderr is back to normal")

## possible code for your loop

def example_loop():
    while True:
        (wm, output) = capture_runtime_stderr(cmiid.Wiimote)
        if wm == None:
            if "Socket connect error" in output:
                raise RuntimeError("library borked, time to reboot")
            time.sleep(1)
            continue
        ## do something with wm??

How detect assigned terminal device for interactive work

What less does in this situation is fall back to fd 2 (stderr). If stderr has been redirected away from the tty, it gives up on trying to get keyboard input, and just prints the whole input stream without paging.

The design of su doesn't allow for anything better. The new user is running a command on a tty owned by the original user, and that unpleasant fact can't be entirely hidden.

Here's a nice substitute for su that doesn't have this problem:

ssh -t localhost -l username sh -c 'command'

It has a little more overhead, of course.

---

Related Topics