Reliable Bidirectional Communication to a Linux Process

Reliable bidirectional communication to a Linux process?

Unix domain sockets are your friend.

A Unix domain socket or IPC socket
(inter-process communication socket)
is a data communications endpoint that
is similar to an Internet socket, but
does not use a network protocol for
communication. It is used in POSIX
operating systems for inter-process
communication.

You reserve a name for your communications channel, such as /myapp/ipc, and then both processes open that address using a UNIX socket:

struct sockaddr_un local;
int len;

s = socket(AF_UNIX, SOCK_STREAM, 0);
local.sun_family = AF_UNIX;
strcpy(local.sun_path, "/myapp/ipc");
len = strlen(local.sun_path) + sizeof(local.sun_family);
bind(s, (struct sockaddr *)&local, len);

Now you can use listen or connect or whatever else in the socket family. It's a little bit of work, but is the best way to achieve IPC on Linux.

Since Erlang is just a nice language for specifying little servers (processes) that communicate over named pipes (processes), this model should feel comfortable to you.

Linux inter-processes communication

Pipes as such don't require you to provide a regular file, but they can have a unique, globally visible name, which is provided by a (unused) filename you have to specify. The contents of the file, if any, is handled by the library.

There are (simple) pipes for communication among related processes (such as a child and a parent process in the same process hierarchy) where the pipe handle can easily be passed to other processes.

The other flavor is called 'named pipes' for processes with any relation, where one can lookup the pipe handle using the global name (as explained in the answer of the question I linked). You can think of a pipe as of a directly connected speaking tube, allowing two processes to chitchat about whatever they like, using read and write functions. On Linux, a pipe is a simplex (at a time, one talks, the other one listens). One would nee two pipes for bidirectional async IO in this case (https://unix.stackexchange.com/questions/53641/how-to-make-bidirectional-pipe-between-two-programs). The immediate buffer for input and output is abstracted. Its just like with network sockets.

I'd suggest to compile this nice example in the accepted answer to play around with: https://stackoverflow.com/a/2789967/1175253

Edit

Example code with error handling. Treat pipe.h & pipe.c as a library (link NamedPipeReader and NamedPipeWriter against it).
This code would need further testing, however, the code is able to (re)open named pipes in any order.

---

pipe.h

#ifndef PIPE_H_
#define PIPE_H_

//C headers
#include <errno.h>
#include <assert.h>

//Linux headers
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>

#ifdef __cplusplus
extern "C"
{
#endif

int open_named_pipe(const char* const name, const int permissions, const int mode, int* pipe_created);

#ifdef __cplusplus
}
#endif

#endif /* PIPE_H_ */

---

pipe.c

#include "pipe.h"

#include <stdio.h>

int open_named_pipe(const char* const name, const int permissions, const int mode, int* pipe_created)
{
    int fd;

    assert(name);
    assert(permissions);
    assert(pipe_created);

    //Create or use an existing pipe special file
    if (0 == mkfifo(name, permissions))
    {
        *pipe_created = 1;
        printf("Successfully created named pipe '%s'\n", name);
    }
    else
    {
        switch (errno)
        {
        case EEXIST:
            //this is OK, as the other process might already has created the special file
            printf("Opened existing named pipe '%s'\n", name);
            break;
        default:
            fprintf(stderr, "Failed to create or access named pipe '%s'\n", name);
            perror("    ");
            return -1;
        };
    }

    fd = open(name, mode);
    if (fd < 0)
    {
        perror("Could not open pipe for writing");
        if (*pipe_created)
        {
            if (0 == unlink(name))
            {
                *pipe_created = 0;
            }
            else
            {
                perror("Failed to unlink named pipe");
            }
        }
    }

    return fd;
}

---

NamedPipeReader.c

#include <stdlib.h>
#include <stdio.h>
#include <signal.h>

#include "pipe.h"

//Globals
const char* const pipe_name = "/tmp/myfifo";
const int pipe_permissions = 0600;
const size_t read_buffer_size = 1024; //[bytes]
const size_t read_retry_delay = 25000; //[us]

int fd = -1;
int pipe_created = 0;
char* read_buffer = NULL;

//Handles EPIPE signal
void signal_handler(int signal)
{
    fprintf(stderr, "cought signal %d\n", signal);
}

//Handles cleanup on exit
void exit_handler(void)
{
    if (read_buffer)
        free(read_buffer);

    if (fd >= 0)
        close(fd);

    //if this process created the FIFO, we unlink it
    if (pipe_created == 0)
        unlink(pipe_name);
}

int main()
{
    //Locals
    int run = 1;
    int received = 0;

    //Install the exit handler
    atexit(&exit_handler);
    signal(EPIPE, signal_handler);
    signal(EACCES, signal_handler);

    //Allocate the buffer
    read_buffer = (char*) malloc(read_buffer_size);
    if (!read_buffer)
    {
        perror("Failed to allocate buffer");
        return EXIT_FAILURE;
    }

    restart: ;

    //Close if already open
    if(fd >= 0)
        close(fd);
    //Create or use an existing pipe special file
    fd = open_named_pipe(pipe_name, pipe_permissions, O_RDONLY, &pipe_created);
    if (fd < 0)
    {
        return EXIT_FAILURE;
    }

    while (run)
    {
        assert(fd >= 0);
        assert(read_buffer_size > 1);

        received = read(fd, read_buffer, read_buffer_size - 1);

        if (received > 0)
        {
            //add a NUL char for string termination
            read_buffer[received] = '0';
            printf("local process %llu received: %s\n", (unsigned long long) getpid(), read_buffer);
        }
        else if (received == 0)
        {
            //EOF reached, this happens in case the writer has closed its handle.
            //Perform a delayed restart and recreate the named pipe
            usleep(read_retry_delay);
            printf("Restarting...\n");
            goto restart;
        }
        else
        {
            switch (errno)
            {
            case EAGAIN:
                //Wait, if the pipe is empty,
                //happens when opened with the O_NONBLOCK flag
                usleep(read_retry_delay);
                break;
            case EPIPE:
            case EBADF:
            case EBADFD:
                perror("Pipe error");
                printf("Restarting...\n");
                goto restart;
            default:
                perror("Pipe error");
                return EXIT_FAILURE;
            };
        }
    }

    return EXIT_SUCCESS;
}

---

NamedPipeWriter.c

#include <stdlib.h>
#include <stdio.h>
#include <signal.h>

#include "pipe.h"

//Globals
const char* const pipe_name = "/tmp/myfifo";
const int pipe_permissions = 0600;
const size_t write_buffer_size = 1024; //[bytes]
const size_t write_retry_delay = 25000; //[us]
const size_t write_interval = 1000000;

int fd = -1;
int pipe_created = 0;
char* write_buffer = NULL;

//Handles EPIPE signal
void signal_handler(int signal)
{
    fprintf(stderr, "cought signal %d\n", signal);
}

//Handles cleanup on exit
void exit_handler(void)
{
    if (write_buffer)
        free(write_buffer);

    if (fd >= 0)
        close(fd);

    //if this process created the FIFO, we unlink it
    if (pipe_created == 0)
        unlink(pipe_name);
}

//Main Function
int main()
{
    //Locals
    int run = 1;
    int sent = 0;
    int msg_len = 0;

    //Install the exit handler
    atexit(&exit_handler);
    signal(EPIPE, signal_handler);
    signal(EACCES, signal_handler);

    //Allocate the buffer
    write_buffer = (char*) malloc(write_buffer_size);
    if (!write_buffer)
    {
        perror("Failed to allocate buffer");
        return EXIT_FAILURE;
    }

    restart: ;

    //Close if already open
    if(fd >= 0)
        close(fd);
    //Create or use an existing pipe special file
    fd = open_named_pipe(pipe_name, pipe_permissions, O_WRONLY, &pipe_created);
    if (fd < 0)
    {
        return EXIT_FAILURE;
    }

    while (run)
    {
        //Print message into the buffer
        msg_len = snprintf(write_buffer, write_buffer_size, "Greetings from process %llu\n", (unsigned long long) getpid());

        {
            char* msg_ptr = write_buffer;
            char* msg_end = write_buffer + msg_len;
            while (msg_ptr != msg_end)
            {
                assert(fd >= 0);
                assert(msg_ptr < msg_end);
                sent = write(fd, msg_ptr, msg_end - msg_ptr);
                if (sent > 0)
                {
                    msg_ptr += sent;
                }
                else if (sent == 0)
                {
                    //retry delay for nonblocking writes
                    usleep(write_retry_delay);
                }
                else
                {
                    switch (errno)
                    {
                    case EAGAIN:
                        //Wait, if the pipe is full,
                        //happens when opened with the O_NONBLOCK flag
                        usleep(write_retry_delay);
                        break;
                    case EPIPE:
                    case EBADF:
                    case EBADFD:
                        perror("Pipe error");
                        printf("Restarting...\n");
                        goto restart;
                    default:
                        perror("Pipe error");
                        return EXIT_FAILURE;
                    };
                }
            }

            printf("Written: %s\n", write_buffer);
            usleep(write_interval);
        }
    }

    return EXIT_SUCCESS;
}

Linux best way in two-way IPC in C

There are a number of different ways to implement IPC. For a good comparison, see Stevens' books. The classic is 'Advanced Programming in the UNIX environment', but there is also 'UNIX Network Programming, Volume 2, Second Edition: Interprocess Communications'. I know it is sometimes not considered good form to point to references elsewhere, but whether this is an academic or commercial problem, most UNIX programmers would recognise Stevens as an invaluable resource.

That said, here are your main options for IPC:

1. Use a pipe() between the processes. The format will always be stream-based; if you are sending datastructures, this can be a pain as you need to worry not only about serialization, but also about buffering and translating the 'packets' back into messages. Pipes are unidirectional, so you will need two for bidirectional communication.

2. Use a named pipe or fifo. This allows many-to-one communication and for the fifo to persist after one end has quit. Otherwise as per (1).

3. Use a socketpair between the processes - specifically a unix domain socket. Sockets are bidirectional. You can either use streaming sockets (SOL_STREAM), datagrams (unreliable, ordering not guaranteed - SOCK_DGRAM) or perhaps better sequenced reliable bidirectional packet communication (SOCK_SEQPACKET). Packet based communication means that you can (e.g.) put one datastructure in each packet.

4. Use signals. Effectively you get to send one integer at a time. Signals do not mix well with threading, handling interrupted system calls is hard, and various race conditions make them unreliable unless you know what you are doing and are not too worried about portability. Best avoided in most cases.

5. Use system V semaphores (semget etc.) or POSIX semaphores (sem_open etc.). Useful for sending signals between processes to achieve synchronization but not much more.

6. Use shared memory (shmget etc.) - the same page(s) are made visible to multiple processes. You will need to combine with some method of synchronisation.

7. System V message queues (msgget etc.) - maintain a queue of packets (messages) between two processes.

8. Some combination of the above.

I've omitted some things only in forks of the kernel (e.g. Binder) or under development (e.g. KDBus).

Examples of and tutorials for nearly all the above can be found here.

Now, most of these could be used for the application you mention. It looks like you want to send variable size messages, so if you use a stream based protocol, the normal hack is to send the length of the packet as the first 1, 2 or 4 bytes (depending on the maximum length). Packet based protocols are easier here (obviously) but each have their own maximum packet size. Do you care about reliability? Do you care about portability? Do you care about speed? All of these are valid concerns when choosing between them.

Finally, an advantage to the FD-based methods (e.g. pipes, socketpairs) is that you can add them to a normal select() loop; if you have other things going on in your program, this may be helpful.

You asked in comments for some examples of socketpair code. I reiterate the comment at the top re getting hold of Stephens. In the absence of that:

• Socketpair() in C/Unix shows a good example of setting up a socketpair for IPC using fork().
• The tutorial mentioned above has a good section on socketpair() here.

C programming bi directional communication

Pipes are uni-directional and using sockets seems painful for a little ipc thing.

I recommend you to use socketpair(). You can consider them as bi-directional pipes.

Here is an example where the parent sends a character to its child.Then child makes it uppercase and sends it back to the parent.(It is from beej's tutorial)

#include <stdio.h>
#include <stdlib.h>
#include <ctype.h>
#include <errno.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>

int main(void)
{
    int sv[2]; /* the pair of socket descriptors */
    char buf; /* for data exchange between processes */

    if (socketpair(AF_UNIX, SOCK_STREAM, 0, sv) == -1) {
        perror("socketpair");
        exit(1);
    }

    if (!fork()) {  /* child */
        read(sv[1], &buf, 1);
        printf("child: read '%c'\n", buf);
        buf = toupper(buf);  /* make it uppercase */
        write(sv[1], &buf, 1);
        printf("child: sent '%c'\n", buf);

    } else { /* parent */
        write(sv[0], "b", 1);
        printf("parent: sent 'b'\n");
        read(sv[0], &buf, 1);
        printf("parent: read '%c'\n", buf);
        wait(NULL); /* wait for child to die */
    }

    return 0;
}

Also you can easily use two pipes for your two directions.

What is a good Inter-Process Communication method between C and PHP in Linux

Possibly the simplest solution you can find is to use pipes. The processes would have an open pipe for reading "calls" and answering them in the same fashion.

One possible way of setting this up is to have a pair of named pipes (mkfifo) in a specific or variable location. Such pipes are known to both this process and PHP. The process would block reading a loop for requests/commands in some textual "protocol" and write back to PHP through the other pipe. In this way both PHP and the external processes could be stopped/killed and restarted and the communications path would still be stable.

You might need to do something else in order to verify whether the process is actually running, if this is needed, but a simple "ping" command over this "protocol" would be enough.

This assumes:

• you have the possibility of making improvements to the processes that communicates to the hardware (otherwise, your are bound to whatever it already offers)
• you don't have high performance requirements (pipes are relatively slow)
• there's no parallelism problem in case of concurrent accesses from the PHP script to process (unless you do some locking, 2 concurrent requests would be written mixed in the pipe)

There are off course other ways of achieving this, but I find hard to consider other way that is as simple as this. Queueing (d-bus, others), as suggested in some comments, are just building on top of this idea but adding more complexity IMHO, therefore, if no functionality provided by these other services is needed, pipes should be enough.

Bi-directional inter-process communication using two pipes

You're not closing wpipe in the child process. So you're actually passing 7 file descriptors to gnu chess when you start it - the dup'ed stdin, stdout and stderr; the 2 descriptors in wpipe, and the 2 descriptors in rpipe.

If you're on linux, find out the process id of gnu chess while your program is running, and do a ls /proc//fd to see all its file descriptors.

Once you add

close(wpipe[0]);
close(wpipe[1]);
close(rpipe[0]);
close(rpipe[1]);

between the dup2()s and the execl(), you should be ok.

(This still omits error handling, like the case that one of the dup2()s fails, or, even worse, your program has closed one of the fds [0, 1, 2] before calling pipe() so one of the standard descriptors gets replaced with a pipe accidentially. But i guess that's not the point here.)

Best way to port C++ code that uses a bidirectional popen() to POSIX

socketpair(2) creates a pair of sockets that are endpoints of a bidirectional pipe. This will directly replace popen(). I do not see why CMD being or not being an interactive program has anything to do with it.

It is possible that the program might need to have an actual terminal device (unlikely, since popen() doesn't really emulate one). If so, the pty(7) man page provides additional documentation for creating a bi-directional pipe that drives a pseudo-tty device.

---

Related Topics