Interprocess Communication via Pipes

Interprocess communication via Pipes

The sending process can write until the pipe buffer is full (64k on Linux since 2.6.11). After that, write(2) will block.

The receiving process will block until data is available to read(2).

For a more detailed look into pipe buffering, look at https://unix.stackexchange.com/a/11954.

For example, this program

#include <sys/types.h>
#include <sys/wait.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

int
main(int argc, char *argv[])
{
    int pipefd[2];
    pid_t cpid;
    char wbuf[32768];
    char buf[16384];

    /* Initialize writer buffer with 012...89 sequence */
    for (int i = 0; i < sizeof(wbuf); i++)
      wbuf[i] = '0' + i % 10;

    if (pipe(pipefd) == -1) {
        perror("pipe");
        exit(EXIT_FAILURE);
    }

    cpid = fork();
    if (cpid == -1) {
        perror("fork");
        exit(EXIT_FAILURE);
    }

    if (cpid == 0) {    /* Child reads from pipe */
        close(pipefd[1]);          /* Close unused write end */
        while (read(pipefd[0], &buf, sizeof(buf)) > 0);
        close(pipefd[0]);
        _exit(EXIT_SUCCESS);

    } else {            /* Parent writes sequence to pipe */
        close(pipefd[0]);          /* Close unused read end */
        for (int i = 0; i < 5; i++)
          write(pipefd[1], wbuf, sizeof(wbuf));
        close(pipefd[1]);          /* Reader will see EOF */
        wait(NULL);                /* Wait for child */
        exit(EXIT_SUCCESS);
    }
}

will produce the following sequence when run with gcc pipes.c && strace -e trace=open,close,read,write,pipe,clone -f ./a.out:

open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
close(3)                                = 0
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\3\2\0\0\0\0\0"..., 832) = 832
close(3)                                = 0
pipe([3, 4])                            = 0
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f32117489d0) = 21114
close(3)                                = 0
write(4, "01234567890123456789012345678901"..., 32768) = 32768
write(4, "01234567890123456789012345678901"..., 32768) = 32768
write(4, "01234567890123456789012345678901"..., 32768strace: Process 21114 attached
 <unfinished ...>
[pid 21114] close(4)                    = 0
[pid 21114] read(3, "01234567890123456789012345678901"..., 16384) = 16384
[pid 21114] read(3,  <unfinished ...>
[pid 21113] <... write resumed> )       = 32768
[pid 21114] <... read resumed> "45678901234567890123456789012345"..., 16384) = 16384
[pid 21113] write(4, "01234567890123456789012345678901"..., 32768 <unfinished ...>
[pid 21114] read(3, "01234567890123456789012345678901"..., 16384) = 16384
[pid 21114] read(3,  <unfinished ...>
[pid 21113] <... write resumed> )       = 32768
[pid 21114] <... read resumed> "45678901234567890123456789012345"..., 16384) = 16384
[pid 21113] write(4, "01234567890123456789012345678901"..., 32768 <unfinished ...>
[pid 21114] read(3, "01234567890123456789012345678901"..., 16384) = 16384
[pid 21114] read(3,  <unfinished ...>
[pid 21113] <... write resumed> )       = 32768
[pid 21114] <... read resumed> "45678901234567890123456789012345"..., 16384) = 16384
[pid 21113] close(4)                    = 0
[pid 21114] read(3, "01234567890123456789012345678901"..., 16384) = 16384
[pid 21114] read(3, "45678901234567890123456789012345"..., 16384) = 16384
[pid 21114] read(3, "01234567890123456789012345678901"..., 16384) = 16384
[pid 21114] read(3, "45678901234567890123456789012345"..., 16384) = 16384
[pid 21114] read(3, "", 16384)          = 0
[pid 21114] close(3)                    = 0
[pid 21114] +++ exited with 0 +++
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=21114, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
+++ exited with 0 +++

You'll notice that the reads and writes are interleaved and that the writing and reading processes will block a few times as either the pipe is full or not enough data is available for reading.

Creating pipes in C for inter process communication between multiple processes

Here you have a code i did, and i will use to explain to you:

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>

int main() {
    char buff[1024];
    int aux, i, count;
    int fds[2], fdss[2];
    pipe(fds); //Here we initialize the file descriptors
    pipe(fdss);
    mode_t fd_mode = S_IRWXU;

    for (i = 0; i < 3; i++) {
        aux = fork();
        if (aux == 0)
            break;
    }

    switch (i) {

        case 0:
            printf("Write something:\n");
            scanf("%s[^\n]", buff);
            i = 0;
            count = 0;
            while(buff[i] != '\0') {

                count++;
                i++;
            }
            dup2(fds[1], 1);
            close(fds[1]);
            close(fds[0]);
            close(fdss[0]);
            close(fdss[1]);
            write (1, buff, sizeof(buff));
            break;

        case 1:

            dup2(fds[0], 0);
            dup2(fdss[1], 1);
            close(fds[0]);
            close(fds[1]);
            close(fdss[0]);
            close(fdss[1]);
            // 
            if (execl("/bin/grep", "grep", "example", NULL) == -1) {
                printf("Error\n");
                exit (1);
            }

            break;

        case 2:
            aux = open("result.txt", O_RDWR | O_CREAT , S_IRWXU);

            dup2(fdss[0], 0);
            dup2(aux, 1);
            close(fds[0]);
            close(fds[1]);
            close(fdss[0]);
            close(fdss[1]);
            close(aux);
            if (execl("/usr/bin/wc", "wc", "-l", NULL) == -1) {
                printf("Error \n");
                exit (1);
            }

    }
            close(fds[0]);
            close(fds[1]);
            close(fdss[0]);
            close(fdss[1]);
    for (i = 0; i < 3; i++) wait(NULL);

    return 0;
}

Ok, let's start:
We create and initialize pipes with pipe()
Then we write our code and before execl() we change the file descriptors, in order to pass the text we will write in the console, through processes and finally write in a file called result.txt the result of the "grep example" command applied to the text we have written.
The function dup2(new_descriptor, old_descriptor) is copying the new descriptor into the old descriptor and closes the old descriptor. For example:
Before dup2(fds[1], 1) we have:
0 STDIN
1 STDOUT
2 STDERR
After dup2(fds[1], 1) we have:
0 STDIN
1 fds[1]
2 STDERR
NOTE: If you don't want to use 1, yo can simply write STDOUT_FILENO
So now we are able to write through processes and in my example to a file too

InterProcess Communication using Pipes and Files

Could "IPC using files" be just one process writing a file and another process reading it. Examples of this would be writing files in /tmp or in /var. In the /var directory there are logs, locks and running PIDs. You can also use the /proc file system to talk to the kernel or /sys to talk to device drivers. These are all "IPC using files".

c - continously communicate between two child processes using pipes

Right now both the parent and child creates their own pair of pipes, that the other process have no knowledge about.

The pipe should be created in the parent process before the fork.

Also, you close the reading/writing ends of the pipe in the loops, when you should close them after the loop, when all the communication has been done.

---

And a small unrelated issue...

In the reader you should really loop while read doesn't return 0 (then the write-end of the pipe is closed) or -1 (if there's an error).

Piping as interprocess communication

Wow, that's a lot of questions. Let's see if I can cover everything...

It seems like the receiving end will
block waiting for input, which I would
expect

You expect correctly an actual 'read' call will block until something is there. However, I believe there are some C functions that will allow you to 'peek' at what (and how much) is waiting in the pipe. Unfortunately, I don't remember if this blocks as well.

will the sending end block sometimes
waiting for someone to read from the
stream

No, sending should never block. Think of the ramifications if this were a pipe across the network to another computer. Would you want to wait (through possibly high latency) for the other computer to respond that it received it? Now this is a different case if the reader handle of the destination has been closed. In this case, you should have some error checking to handle that.

If I write an eof to the stream can I
keep continue writing to that stream
until I close it

I would think this depends on what language you're using and its implementation of pipes. In C, I'd say no. In a linux shell, I'd say yes. Someone else with more experience would have to answer that.

Are there differences in the behaviour
named and unnamed pipes?
As far as I know, yes. However, I don't have much experience with named vs unnamed. I believe the difference is:

• Single direction vs Bidirectional communication
• Reading AND writing to the "in" and "out" streams of a thread

Does it matter which end of the pipe I
open first with named pipes?

Generally no, but you could run into problems on initialization trying to create and link the threads with each other. You'd need to have one main thread that creates all the sub-threads and syncs their respective pipes with each other.

Is the behaviour of pipes consistent
between different linux systems?

Again, this depends on what language, but generally yes. Ever heard of POSIX? That's the standard (at least for linux, Windows does it's own thing).

Does the behaviour of the pipes depend
on the shell I'm using or the way I've
configured it?

This is getting into a little more of a gray area. The answer should be no since the shell should essentially be making system calls. However, everything up until that point is up for grabs.

Are there any other questions I should
be asking

The questions you've asked shows that you have a decent understanding of the system. Keep researching and focus on what level you're going to be working on (shell, C, so on). You'll learn a lot more by just trying it though.

---

Related Topics