Toy Shell Not Piping Correctly

Toy shell not piping correctly

I'm virtually certain this is what you're trying to do. Apologies in advance for the sloppy coding. its somewhat late here and I really should be sleeping right now:

#include <iostream>
#include <string>
#include <cstdlib>
#include <cstdio>
#include <unistd.h>

#define READ 0
#define WRITE 1

// ps -A | grep argv[1] | wc -l

int main( int argc, char** argv )
{
    // start of input block
    if ( argc != 2 )
    {
        std::cout << "Usage: ./a.out arg1" << std::endl;
        return 0;
    }
    
    // make local copy of argument
    std::string in = argv[1];
    int fd1[2], fd2[2], pid;
    
    // allocate two pipe sets
    if (pipe(fd1) < 0 || pipe(fd2) < 0)
    {
        perror("Failed to create pipe.");
        return EXIT_FAILURE;
    }
    
    // launch first child process.
    if ((pid = fork()) < 0)
    {
        perror("Failed to fork child(1)");
        return EXIT_FAILURE;
    }
    
    if (pid == 0)
    {
        // wc -l process. 
        //  stdin  = fd2(read)
        close(fd1[READ]);
        close(fd1[WRITE]);
        close(fd2[WRITE]);
        dup2(fd2[READ],STDIN_FILENO);
        execlp("wc","wc","-l",NULL);
    }
    
    // fork again. this time for grep
    if ((pid = fork()) < 0)
    {
        perror("Failed to fork child(2)");
        return EXIT_FAILURE;
    }
    
    if (pid == 0)
    {
        // grep argv[1] process.
        //  stdin  = fd1(read)
        //  stdout = fd2(write)            
        close(fd1[WRITE]);
        close(fd2[READ]);
        dup2(fd2[WRITE], STDOUT_FILENO);
        dup2(fd1[READ], STDIN_FILENO);
        execlp("grep", "grep", in.c_str(), NULL);
    }
    
    //  fork once more. this time for ps -A
    if ((pid = fork()) < 0)
    {
        perror("Failed to fork child(3)");
        return EXIT_FAILURE;
    }
    
    if (pid == 0)
    {
        // ps -A process.
        //  stdout = fd1(write)
        close(fd2[WRITE]);
        close(fd2[READ]);
        close(fd1[READ]);
        dup2(fd1[WRITE], STDOUT_FILENO);
        execlp("ps", "ps", "-A", NULL);
    }
    
    int stat=0;
    wait(&stat);

    return EXIT_SUCCESS;
}

On my system, ps -A reports 141 lines, of those 41 have the word System somewhere within, verified by simply running ps -A | grep System | wc -l. The above code generates precisely the same output.

C - Single shell piping implementation keeps getting hanged in terminal

The parent process should close both ends of the pipe after it has launched the two children. If it doesn't close the writing end of the pipe, the child that is reading from the pipe will never get EOF, so it will never terminate. If it doesn't close the reading end of the pipe but the child that is reading terminates before it has read all the input, the child that is writing will never get an error and will block, waiting for the parent to read when the parent has no intention of ever reading.

So, the parent simply needs:

close(fd[0]);
close(fd[1]);

Pipe and Process management

Yes, the shell must fork to exec each subprocess. Remember that when you call one of the execve() family of functions, it replaces the current process image with the exec'ed one. Your shell cannot continue to process further commands if it directly execs a subprocess, because thereafter it no longer exists (except as the subprocess).

To fix it, simply fork() again in the pid == 0 branch, and exec the ls command in that child. Remember to wait() for both (all) child processes if you don't mean the pipeline to be executed asynchronously.

Problems with pipes in c

Note that the comments in Andrew Henle's answer apply in the long term — you need to run all the programs in a pipeline concurrently to ensure that there aren't any deadlocks because one process is trying to write to a full pipe and another is waiting for the one process to complete before executing the code (or program) that reads from the pipe. The comments made directly under the question also apply.

However, this code, derived from yours with minimal band-aiding (I did comment out the unused variables std_in and std_out, though), compiles cleanly and runs and produces plausible output.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/wait.h>

struct cmd
{
    char **argv;
};

int main(void)
{
    int fd_pipe[2];
    int pid;
    //int std_in;
    //int std_out;
    int status;
    char *cmd1[] = { "ls", 0 };
    char *cmd2[] = { "grep", "test", 0 };
    struct cmd comline[2] = { { cmd1 }, { cmd2 } };

    if(pipe(fd_pipe) < 0)
        perror("pipe error...");

    if ((pid = fork()) < 0) /* fork a child process */
    {     
        perror("ERROR: forking child process failed\n");
        exit(1);
    }
    else if (pid == 0) /* the child process, do the first command */
    {  
        printf("In first child...\n");
        //dupPipe(fd_pipe, 1, STDOUT_FILENO);   /* send to write end of pipe */

        fflush(stdout);
        //std_out = dup(1); // for later restore...
        close(fd_pipe[0]);
        dup2(fd_pipe[1], 1);

        printf("exec %s in first child...\n", comline[0].argv[0]);         
        if (execvp(comline[0].argv[0], comline[0].argv) < 0) /* execute the command  */
        {
            fprintf(stderr,"-mish: Exe of %s failed...",comline[0].argv[0]);
            fflush(stderr);
            perror("");
        }
        exit(EXIT_FAILURE);

    }
    else    /* parent process*/
    {
        while (wait(&status) != pid)    //wait for child to completion
            ;

        fflush(stdin);
        //std_in = dup(0); // for later restore...
        close(fd_pipe[1]);
        dup2(fd_pipe[0], 0);

        if (execvp(comline[1].argv[0], comline[1].argv) < 0) /* execute the command  */
        {
            fprintf(stderr,"-mish: Exe of %s failed...",comline[0].argv[0]);
            fflush(stderr);
            perror("");
        }
        exit(EXIT_FAILURE);
    }
    printf("Exiting from %d\n", (int)getpid());
    return 0;
}

In a directory with 130-odd files (giving about 1.5 KiB of output from ls), I get:

$ ./p97
In first child...
chk-utf8-test.sh
chk-wchar-test.sh
pbb-test.data
test-fstatat.c
test-rename.c
utf8-test.c
wchar-test.c
$

This is what I get from the command line running ls | grep test apart (of course) from the diagnostic message.

My suspicion, though, is that you've not shown us all the code. The std_in and std_out variables make me suspicious that there is actually some other code around, and maybe the pipe() operation is occurring in a parent process which then forks, and the child then runs the rest of the code shown in the question. If there is a scenario like that, then there could be problems with commands not completing because the pipe is not properly closed.

GNU C Multi-process handling with pipes

You're write logic is questionable, and the layout of your pipes is overtly complicated. Below is your code tailored down to as simple a case as I can muster. Comments are included to help yo along. I find it easiest when dealing with chained pipes (which is for all intents exactly what this is) to lay out a single descriptor array indexed by macros that exhibit the chaining:

// some hand macros for access the correct pipes
#define P_READ  0
#define C_WRITE 1
#define C_READ  2
#define P_WRITE 3
#define N_PIPES 4

the above will be in the final source list. The monikers should be self-evident, but in case they aren't, P_XXX notes the pipes the parent process uses, C_XXX notes the pipes the child process uses. Keep that in mind when see the code:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <assert.h>
#include <unistd.h>
#include <sys/types.h>

// some hand macros for access the correct pipes
#define P_READ  0
#define C_WRITE 1
#define C_READ  2
#define P_WRITE 3
#define N_PIPES 4

// reads a buffer up-to len size.
ssize_t readFromPipe(int pipe, char *buff, size_t len)
{
    buff[0] = 0;
    ssize_t res = read(pipe, buff, len);
    assert(res >= 0 && "Failed to read from pipe");
    return res;
}

ssize_t writeToPipe(int pipe, const char *input)
{
    size_t len = strlen(input)+1;
    ssize_t res = write(pipe, input, len);
    assert(res == len && "Failed to write to pipe");
    return res;
}

int main(void)
{
    int pipes[N_PIPES];
    char msg[128] = {0};
    pid_t pid;

    if(pipe(pipes) < 0 || pipe(pipes+2) < 0)
    {
        printf("[ERROR] create pipes\n");
        return EXIT_FAILURE;
    }

    if((pid = fork()) < 0)
    {
        printf("[ERROR] Fork error\n");
        return EXIT_FAILURE;
    }

    // parent code
    if(pid > 0)
    {
        // Parent code. close down the child pipes; don't need them
        printf("parent(%d) create: child(%d)\n", getpid(), pid);
        close(pipes[C_WRITE]);
        close(pipes[C_READ]);

        do
        {
            if (readFromPipe(pipes[P_READ], msg, sizeof(msg)) > 0)
            {
                printf("parent(%d) read : %s\n", getpid(), msg);
                writeToPipe(pipes[P_WRITE], "-1");
            }
            else break;
        }
        while(atoi(msg) != -1);

        // close remaining pipes. no longer needed
        close(pipes[P_READ]);
        close(pipes[P_WRITE]);
    }

    else if(pid == 0)
    {
        // Child code. don't need parent write or child-read lines
        close(pipes[P_READ]);
        close(pipes[P_WRITE]);

        // write message
        writeToPipe(pipes[C_WRITE],"test message");

        // read test message
        if (readFromPipe(pipes[C_READ], msg, sizeof(msg)) > 0)
            printf("child(%d) read : %s\n", getpid(), msg);

        // write another message
        writeToPipe(pipes[C_WRITE], "-1");

        // close remaining pipes. no longer needed
        close(pipes[C_READ]);
        close(pipes[C_WRITE]);
    }

    return EXIT_SUCCESS;
}

The pipe descriptor array not withstanding, the biggest change in this is the simplified writeToPipe logic, which simply writes a terminator C string to the pipe, including the terminating null char.

ssize_t writeToPipe(int pipe, const char *input)
{
    size_t len = strlen(input)+1;
    ssize_t res = write(pipe, input, len);
    assert(res == len && "Failed to write to pipe");
    return res;
}

The caller checks the result to ensure it wrote all the requested data, and an embedded assert() macro will trip your debugger on failure. Similar logic exists for the read functionality.

Output (varies by process id)

parent(2067) create: child(2068)
parent(2067) read : test message
child(2068) read : -1
parent(2067) read : -1

I hope this helps. When dealing with pipes, and in particular with redirection of stdio which you will likely encounter in the not-too-distant future (see spoiler here), it helps tremendously to have meaningful mnemonics for your code, such as the macros I used above for indexing the pipes array.

Best of luck.

Background Jobs in C (implementing & in a toy shell)

The call to wait made for the third job returns immediately because the second job has finished and is waiting to be handled (also called "zombie"). You could check the return value of wait(&status), which is the PID of the process that has exited, and make sure it is the process you were waiting for. If it's not, just call wait again.

Alternatively use waitpid, which waits for a specific process:

/* Wait for child. was: wait(&status) */
waitpid(fork_return, &status, 0); 

If you do this you should implement a signal handler for SIGCHLD to handle finished background jobs to prevent the accumulation of "zombie" child processes.

In addition to that, in the background job case, the branch where fork() returns 0 you are already in the new process, so the call to addJobToTable happens in the wrong process. Also, you should check the return values of all the calls; otherwise something may be failing and you don't know it. So the code for running a job in the background should look more like this:

    if (fork_return == 0) {
        setpgid(0, 0);
        if (execve(executableCommands[0], executableCommands, NULL) == -1) {
            perror("execve");
            exit(1);
        }
    } else if (fork_return != -1) {
        addJobToTable(strippedCommand, fork_return);
        return;
    } else {
        perror("fork"); /* fork failed */
        return;
    }

Communicating across child processes with a pipe

If a process writes to a pipe, but there is no immediate second process to read that data, does the data get lost?

No.

By default, writing to a pipe is a blocking action. That is, writing to a pipe will block execution of the calling process until there is enough room in the pipe to write the requested data.

The responsibility is on the reading side to drain the pipe to make room, or close their side of the pipe to signal they no longer wish to receive data.

With 2 children communicating over a pipe, does the main process need anything with the pipe, such as close(fd[0])?

Each process involved will have its own copy of the file descriptors.

As such, the parent process should close both ends of the pipe (after both forks), since it has no reason to hold onto those file descriptors. Failing to do so could result in the parent process running out of file descriptors (ulimit -n).

---

Your understanding of dup2 appears to be correct.

close(fd[1]);
dup2(fd[0], STDIN_FILENO);
close(fd[0]);

Both ends of the pipe are closed because after dup2 the file descriptor usually associated with stdin now refers to the same file description that the file descriptor for the read end of the pipe does.

stdin is of course closed closed when the replacement process image (exec*) exits.

---

Your second example of forking two processes, where they run concurrently, is the correct understanding.

In your typical shell, piped commands run concurrently. Otherwise, as stated earlier, the writer may fill the pipe and block before completing its task.

Generally, the parent waits for both processes to finish.

---

Here's a toy example. Run as ./program FILE STRING to emulate cat FILE | grep STRING.

#include <stdio.h>
#include <sys/wait.h>
#include <unistd.h>

int main(int argc, char **argv) {
    int fds[2];

    pipe(fds);

    int left = fork();

    if (0 == left) {
        close(fds[0]);
        dup2(fds[1], fileno(stdout));
        close(fds[1]);
        execlp("cat", "cat", argv[1], NULL);
        return 1;
    }

    int right = fork();

    if (0 == right) {
        close(fds[1]);
        dup2(fds[0], fileno(stdin));
        close(fds[0]);
        execlp("grep", "grep", argv[2], NULL);
        return 1;
    }

    close(fds[0]);
    close(fds[1]);

    waitpid(left, NULL, 0);
    waitpid(right, NULL, 0);
}

---

Related Topics