Which Linux IPC technique to use?
I would go for Unix Domain Sockets: less overhead than IP sockets (i.e. no inter-machine comms) but same convenience otherwise.
Linux IPC selection?
Selection of IPC technique depends on application which you are trying to implement. Below is a good comparison base on performance:
IPC name Latency Throughput Description
-----------------------------------------------------------------------------------------
Signal Low n/a Can be used only for notification, traditionally-
to push process to change its state
Socket Moderate Moderate Only one mechanism which works for remote nodes,
not very fast but universal
D-Bus High High A high-level protocol that increases latency and
reduces throughput compared to when using base
sockets, gains in increased ease of use
Shared n/a High Data saved in-between process runs(due to swapping
memory access time) can have non-constant access latency
Mapped files n/a High Data can be saved in-between device boots
Message Low Moderate Data saved in-between process runs. The message
queue size is limited but there is less overhead
to handle messages
Here is one more nice comparison
Comparing Unix/Linux IPC
IPC speed and compare
Been facing a similar question myself.
I've found the following pages helpful - IPC performance: Named Pipe vs Socket (in particular) and Sockets vs named pipes for local IPC on Windows?.
It sounds like the concensus is that shared memory is the way to go if you're really concerned about performance, but if the current system you have is a message queue it might be a rather... different structure. A socket and/or named pipe might be easier to implement, and if either meets your specs then you're done there.
Which kind of inter process communication (ipc) mechanism should I use at which moment?
Long story short:
Use lock files, mutexes, semaphores and barriers when processes compete for a scarce resource. They operate in a similar manner: several process try to acquire a synchronisation primitive, some of them acquire it, others are put in sleeping state until the primitive is available again. Use semaphores to limit the amount of processes working with a resource. Use a mutex to limit the amount to 1.
You can partially avoid using synchronisation primitives by using non-blocking thread-safe data structures.
Use signals, queues, pipes, events, messages, unix sockets when processes need to exchange data. Signals and events are usually used for notifying a process of something (for instance,
ctrl+c
in unix terminal sends aSIGINT
signal to a process). Pipes, shared memory and unix sockets are for transmitting data.Use sockets for networking (or, speaking formally, for exchanging data between processes located on different machines).
Long story long: take a look at Modern Operating Systems book by Tanenbaum & Bos, namely IPC chapter. The topic is vast and can't be completely covered within a list or a paper.
UNIX/Linux IPC : Reading from a pipe. How to know length of data at runtime?
Since it seems that you intend to make a single read of all the data from the pipe, I think the following will serve you better than delimiter+encoding or miniheader techniques suggested in other answers:
From the pipe (7) manpage:
If all file descriptors referring to
the write end of a pipe have been
closed, then an attempt to read(2)
from the pipe will see end-of-file
(read(2) will return 0).
The following example was taken from the pipe (2) manpage and reversed so that the child does the writing, the parent the reading (just to be sure). I also added a variable size buffer. The child will sleep for 5 seconds. The delay will ensure that the exit() of the child can have nothing to do with pipeio (the parent will print a complete line before the child exits).
#include <sys/wait.h>
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
char *
slurpfd(int fd)
{
const int bytes_at_a_time = 2;
char *read_buffer = NULL;
int buffer_size = 0;
int buffer_offset = 0;
int chars_io;
while (1) {
if (buffer_offset + bytes_at_a_time > buffer_size) {
buffer_size = bytes_at_a_time + buffer_size * 2;
read_buffer = realloc(read_buffer, buffer_size);
if (!read_buffer) {
perror("memory");
exit(EXIT_FAILURE);
}
}
chars_io = read(fd,
read_buffer + buffer_offset,
bytes_at_a_time);
if (chars_io <= 0) break;
buffer_offset += chars_io;
}
if (chars_io < 0) {
perror("read");
exit(EXIT_FAILURE);
}
return read_buffer; /* caller gets to free it */
}
int
main(int argc, char *argv[])
{
int pipefd[2];
pid_t cpid;
assert(argc == 2);
if (pipe(pipefd) == -1) {
perror("pipe");
exit(EXIT_FAILURE);
}
cpid = fork();
if (cpid == -1) {
perror("fork");
exit(EXIT_FAILURE);
}
if (cpid == 0) { /* Child writes argv[1] to pipe */
close(pipefd[0]); /* Close unused read end */
write(pipefd[1], argv[1], strlen(argv[1]) + 1);
close(pipefd[1]); /* Reader will see EOF */
/* sleep before exit to make sure that there
will be a delay after the parent prints it's
output */
sleep(5);
exit(EXIT_SUCCESS);
} else { /* Parent reads from pipe */
close(pipefd[1]); /* Close unused write end */
puts(slurpfd(pipefd[0]));
close(pipefd[0]);
wait(NULL); /* Wait for child */
_exit(EXIT_SUCCESS);
}
}
From your comment I see now that you may want to read the data as it becomes available, to update the UI or whatever, to reflect your system's status. To do that open the pipe in non-blocking (O_NONBLOCK) mode. Read repeatedly whatever is available until -1 is returnd and errno == EAGAIN and do your parsing. Repeat unil read returns 0, which indicates that the child has closed the pipe.
To use an in-memory buffer for File* functions you can use fmemopen() in the GNU C library.
Related Topics
Appending a Line to a File Only If It Does Not Already Exist
Sorting Multiple Keys With Unix Sort
Linux Default Behavior of Executable .Data Section Changed Between 5.4 and 5.9
How to Ensure Only One Instance of a Bash Script Is Running
How to Compile a 32-Bit Binary on a 64-Bit Linux Machine With Gcc/Cmake
How to Remove ^[, and All of the Escape Sequences in a File Using Linux Shell Scripting
Bash Script Process Substitution Syntax Error: "(" Unexpected
The Difference Between Fork(), Vfork(), Exec() and Clone()
Curl: (6) Could Not Resolve Host: Google.Com; Name or Service Not Known
Sort a List With Lowercase First
How to Remove a Close_Wait Socket Connection
How to Set Environment Variables That Crontab Will Use
How to Find All Files Containing Specific Text on Linux