what happens to background processes stdout and stderr when I log out?
This is the crucial loop where tee
sends output to stdout and opened files:
while (1)
{
bytes_read = read (0, buffer, sizeof buffer);
if (bytes_read < 0 && errno == EINTR)
continue;
if (bytes_read <= 0)
break;
/* Write to all NFILES + 1 descriptors.
Standard output is the first one. */
for (i = 0; i <= nfiles; i++)
if (descriptors[i]
&& fwrite (buffer, bytes_read, 1, descriptors[i]) != 1)
{
error (0, errno, "%s", files[i]);
descriptors[i] = NULL;
ok = false;
}
}
Pay closer attention on this part:
if (descriptors[i]
&& fwrite (buffer, bytes_read, 1, descriptors[i]) != 1)
{
error (0, errno, "%s", files[i]);
descriptors[i] = NULL;
ok = false;
}
It shows that when an error occurs, tee
would not close itself but just unset the file descriptor descriptors[i] = NULL
and continue to keep reading data until EOF
or error on input occurs besides EINTR
.
The date
command or anything that sends output to the pipe connected to tee
would not terminated since tee
still reads their data. Only that the data doesn't go anywhere besides the file foo
. And even if a file argument was not provided, tee
would still read their data.
This is what /proc/**/fd
looks like on tee
when disconnected from a terminal:
0 -> pipe:[431978]
1 -> /dev/pts/2 (deleted)
2 -> /dev/pts/2 (deleted)
And this one's from the process that connects to its pipe:
0 -> /dev/pts/2 (deleted)
1 -> pipe:[431978]
2 -> /dev/pts/2 (deleted)
You can see that tee
's stdout
and stderr
is already EOL but it's still running.
Redirect STDERR / STDOUT of a process AFTER it's been started, using command line?
Short of closing and reopening your tty (i.e. logging off and back on, which may also terminate some of your background processes in the process) you only have one choice left:
- attach to the process in question using gdb, and run:
- p dup2(open("/dev/null", 0), 1)
- p dup2(open("/dev/null", 0), 2)
- detach
- quit
e.g.:
$ tail -f /var/log/lastlog &
[1] 5636
$ ls -l /proc/5636/fd
total 0
lrwx------ 1 myuser myuser 64 Feb 27 07:36 0 -> /dev/pts/0
lrwx------ 1 myuser myuser 64 Feb 27 07:36 1 -> /dev/pts/0
lrwx------ 1 myuser myuser 64 Feb 27 07:36 2 -> /dev/pts/0
lr-x------ 1 myuser myuser 64 Feb 27 07:36 3 -> /var/log/lastlog
$ gdb -p 5636
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Attaching to process 5636
Reading symbols from /usr/bin/tail...(no debugging symbols found)...done.
Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
[New Thread 0x7f3c8f5a66e0 (LWP 5636)]
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /lib/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
(no debugging symbols found)
0x00007f3c8eec7b50 in nanosleep () from /lib/libc.so.6
(gdb) p dup2(open("/dev/null",0),1)
[Switching to Thread 0x7f3c8f5a66e0 (LWP 5636)]
$1 = 1
(gdb) p dup2(open("/dev/null",0),2)
$2 = 2
(gdb) detach
Detaching from program: /usr/bin/tail, process 5636
(gdb) quit
$ ls -l /proc/5636/fd
total 0
lrwx------ 1 myuser myuser 64 Feb 27 07:36 0 -> /dev/pts/0
lrwx------ 1 myuser myuser 64 Feb 27 07:36 1 -> /dev/null
lrwx------ 1 myuser myuser 64 Feb 27 07:36 2 -> /dev/null
lr-x------ 1 myuser myuser 64 Feb 27 07:36 3 -> /var/log/lastlog
lr-x------ 1 myuser myuser 64 Feb 27 07:36 4 -> /dev/null
lr-x------ 1 myuser myuser 64 Feb 27 07:36 5 -> /dev/null
You may also consider:
- using
screen
; screen provides several virtual TTYs you can switch between without having to open new SSH/telnet/etc, sessions - using
nohup
; this allows you to close and reopen your session without losing any background processes in the... process.
How to redirect output of an already running process
See Redirecting Output from a Running Process.
Firstly I run the command
cat > foo1
in one session and test that data from stdin is copied to the file. Then in another session I redirect the output.Firstly find the PID of the process:
$ ps aux | grep cat
rjc 6760 0.0 0.0 1580 376 pts/5 S+ 15:31 0:00 cat
Now check the file handles it has open:
$ ls -l /proc/6760/fd
total 3
lrwx—— 1 rjc rjc 64 Feb 27 15:32 0 -> /dev/pts/5
l-wx—— 1 rjc rjc 64 Feb 27 15:32 1 -> /tmp/foo1
lrwx—— 1 rjc rjc 64 Feb 27 15:32 2 -> /dev/pts/5
Now run GDB:
$ gdb -p 6760 /bin/cat
GNU gdb 6.4.90-debian
[license stuff snipped]
Attaching to program: /bin/cat, process 6760
[snip other stuff that's not interesting now]
(gdb) p close(1)
$1 = 0
(gdb) p creat("/tmp/foo3", 0600)
$2 = 1
(gdb) q
The program is running. Quit anyway (and detach it)? (y or n) y
Detaching from program: /bin/cat, process 6760
The
p
command in GDB will print the value of an expression, an expression can be a function to call, it can be a system call… So I execute aclose()
system call and pass file handle 1, then I execute acreat()
system call to open a new file. The result of thecreat()
was 1 which means that it replaced the previous file handle. If I wanted to use the same file for stdout and stderr or if I wanted to replace a file handle with some other number then I would need to call thedup2()
system call to achieve that result.For this example I chose to use
creat()
instead ofopen()
because there are fewer parameter. The C macros for the flags are not usable from GDB (it doesn’t use C headers) so I would have to read header files to discover this – it’s not that hard to do so but would take more time. Note that 0600 is the octal permission for the owner having read/write access and the group and others having no access. It would also work to use 0 for that parameter and run chmod on the file later on.After that I verify the result:
ls -l /proc/6760/fd/
total 3
lrwx—— 1 rjc rjc 64 2008-02-27 15:32 0 -> /dev/pts/5
l-wx—— 1 rjc rjc 64 2008-02-27 15:32 1 -> /tmp/foo3 <====
lrwx—— 1 rjc rjc 64 2008-02-27 15:32 2 -> /dev/pts/5
Typing more data in to
cat
results in the file/tmp/foo3
being appended to.If you want to close the original session you need to close all file handles for it, open a new device that can be the controlling tty, and then call
setsid()
.
How to make a program continue to run after log out from ssh?
Assuming that you have a program running in the foreground, press ctrl-Z, then:
[1]+ Stopped myprogram
$ disown -h %1
$ bg 1
[1]+ myprogram &
$ logout
If there is only one job, then you don't need to specify the job number. Just use disown -h
and bg
.
Explanation of the above steps:
You press ctrl-Z. The system suspends the running program, displays a job number and a "Stopped" message and returns you to a bash prompt.
You type the disown -h %1
command (here, I've used a 1
, but you'd use the job number that was displayed in the Stopped
message) which marks the job so it ignores the SIGHUP
signal (it will not be stopped by logging out).
Next, type the bg
command using the same job number; this resumes the running of the program in the background and a message is displayed confirming that.
You can now log out and it will continue running..
How do I use the nohup command without getting nohup.out?
The nohup
command only writes to nohup.out
if the output would otherwise go to the terminal. If you have redirected the output of the command somewhere else - including /dev/null
- that's where it goes instead.
nohup command >/dev/null 2>&1 # doesn't create nohup.out
Note that the >/dev/null 2>&1
sequence can be abbreviated to just >&/dev/null
in most (but not all) shells.
If you're using nohup
, that probably means you want to run the command in the background by putting another &
on the end of the whole thing:
nohup command >/dev/null 2>&1 & # runs in background, still doesn't create nohup.out
On Linux, running a job with nohup
automatically closes its input as well. On other systems, notably BSD and macOS, that is not the case, so when running in the background, you might want to close input manually. While closing input has no effect on the creation or not of nohup.out
, it avoids another problem: if a background process tries to read anything from standard input, it will pause, waiting for you to bring it back to the foreground and type something. So the extra-safe version looks like this:
nohup command </dev/null >/dev/null 2>&1 & # completely detached from terminal
Note, however, that this does not prevent the command from accessing the terminal directly, nor does it remove it from your shell's process group. If you want to do the latter, and you are running bash, ksh, or zsh, you can do so by running disown
with no argument as the next command. That will mean the background process is no longer associated with a shell "job" and will not have any signals forwarded to it from the shell. (A disown
ed process gets no signals forwarded to it automatically by its parent shell - but without nohup
, it will still receive a HUP
signal sent via other means, such as a manual kill
command. A nohup
'ed process ignores any and all HUP
signals, no matter how they are sent.)
Explanation:
In Unixy systems, every source of input or target of output has a number associated with it called a "file descriptor", or "fd" for short. Every running program ("process") has its own set of these, and when a new process starts up it has three of them already open: "standard input", which is fd 0, is open for the process to read from, while "standard output" (fd 1) and "standard error" (fd 2) are open for it to write to. If you just run a command in a terminal window, then by default, anything you type goes to its standard input, while both its standard output and standard error get sent to that window.
But you can ask the shell to change where any or all of those file descriptors point before launching the command; that's what the redirection (<
, <<
, >
, >>
) and pipe (|
) operators do.
The pipe is the simplest of these... command1 | command2
arranges for the standard output of command1
to feed directly into the standard input of command2
. This is a very handy arrangement that has led to a particular design pattern in UNIX tools (and explains the existence of standard error, which allows a program to send messages to the user even though its output is going into the next program in the pipeline). But you can only pipe standard output to standard input; you can't send any other file descriptors to a pipe without some juggling.
The redirection operators are friendlier in that they let you specify which file descriptor to redirect. So 0<infile
reads standard input from the file named infile
, while 2>>logfile
appends standard error to the end of the file named logfile
. If you don't specify a number, then input redirection defaults to fd 0 (<
is the same as 0<
), while output redirection defaults to fd 1 (>
is the same as 1>
).
Also, you can combine file descriptors together: 2>&1
means "send standard error wherever standard output is going". That means that you get a single stream of output that includes both standard out and standard error intermixed with no way to separate them anymore, but it also means that you can include standard error in a pipe.
So the sequence >/dev/null 2>&1
means "send standard output to /dev/null
" (which is a special device that just throws away whatever you write to it) "and then send standard error to wherever standard output is going" (which we just made sure was /dev/null
). Basically, "throw away whatever this command writes to either file descriptor".
When nohup
detects that neither its standard error nor output is attached to a terminal, it doesn't bother to create nohup.out
, but assumes that the output is already redirected where the user wants it to go.
The /dev/null
device works for input, too; if you run a command with </dev/null
, then any attempt by that command to read from standard input will instantly encounter end-of-file. Note that the merge syntax won't have the same effect here; it only works to point a file descriptor to another one that's open in the same direction (input or output). The shell will let you do >/dev/null <&1
, but that winds up creating a process with an input file descriptor open on an output stream, so instead of just hitting end-of-file, any read attempt will trigger a fatal "invalid file descriptor" error.
Running a script and get it alive after a SSH disconnection (like a broken pipe) without forcing it in background
As you've explained it in comments, your real goal is to let your script still run in the foreground by default, but make sure it survives SSH disconnecting.
You can do this without needing disown
in any way, though the code needs to be put at the top, not the bottom:
#!/usr/bin/env bash
# redirect all three of stdin, stdout and stderr (not just stdout!)
exec >"txt$$.txt" 2>"err$$.txt" </dev/null
# ignore any SIGHUP signals received
trap : SIGHUP
# ...put the content of your script here
How to prevent a background process from being stopped after closing SSH client in Linux
Check out the "nohup" program.
Related Topics
The Behavior When a Gnu Make Phony Target Happens to Be The Same as a Directory Name
Make Uses "Cc" Instead of "Arm-None-Eabi-As"
How to Use 'Catdoc' to Display Dock File Encoded in Utf-8
Can't Increment a 0-Padded Number Past 8 in Busybox Sh
Format and Filter File to CSV Table
Host Multiple ASP.NET Core Web Application Under a Single Linux Server
Difference Between The Commands "Gcloud Compute Ssh" and "Ssh"
Most Efficient Way to Concatenate Thousands of Files in Perl
A Wrong Size of "Len" Calculated by $ - Symbol with Fasm Equ
./Configure-With-Boost No Such File or Directory
Why Does Automating Sftp with Expect Hang After Sending The Password
How to Write Content to File on Linux Sftp Server Using Sshclient
Shell Must Parse Ls -Al Output and Get Last Field (File or Directory Name) Any Solution
How to Run a Bash Function() in a Remote Host? in Ubuntu
Linux - Bash Redirect a String to a File
How to Rename Files in Zip Archive Without Extracting and Recompressing Them