IPC - pipes in Unix

Apr 20, 2022

Introduction

IPC stands for Interprocess Communication. It is a way to allow communication between processes. There are many IPC mechanisms:

Shared files
Shared memory (with semaphores)
Pipes (named and unnamed)
Message queues
Sockets
Signals

A pipe connects the standard output of a process to the standard input of another. It is a unidirectional communication channel between:

producer: writer process
consumer: reader process

It synchronizes producer and consumer processes.
A process that’s reading from a pipe freezes if the pipe is empty and keeps waiting for data.
A process that’s writing to a pipe freezes if the pipe is full, waiting for free space on which to write.

Processes work on pipes as if they were files: they use the same primitives shown in Manage Files in Unix article.

Communication

From now on, fd variable will be assumed to be a two-element array storing the pipe’s file descriptors:

void main() {
    int fd[2];
}

The array above holds:

reader process: fd[0]
writer process: fd[1]

A process usually opens a pipe, then create a child process which has the same file descriptors of the parent and send data to (or wait data from) the child process.

Since both parent and child processes could read and write, each process close a pipe’s end: child close fd[0] and parent close fd[1] or vice-versa.

A huge limit of pipes is that allow communication only between processes that have at least a common relative.

Please Note: another common use is when a (parent) process opens a pipe to make two children processes communicate to each other.

Operations over a pipe

Create a pipe

If the data written by the writer process have been read, the writing on the pipe restart from the beginning: the memory is managed like a circular buffer.

#include <stdio.h>
#include <unistd.h>

void main() {
    int fd[2];

    if(pipe(fd) == 0)
        printf("Pipe created successfully\n");
}

Close a pipe

close() system call closes one end of the pipe. After fork() is called, the parent and child processes must both close the pipe’s end that they will not intend to use.

Closing a pipe’s end does not trigger any actions until there’s at least another process that has the same pipe’s end opened.

If a process closes the last reference of a pipe’s end, there could be the following scenarios:

reading from a closed pipe: read() returns 0
writing on a closed pipe: write() triggers writer process termination (sends SIGPIPE signal)

#include <stdio.h>
#include <unistd.h>

void main() {
    int fd[2];
    if(pipe(fd) == 0)
        printf("Pipe created successfully\n");

    // close reader file descriptor
    if(close(fd[0]) == -1) {
        perror("reader file descriptor close");
        exit(EXIT_FAILURE);
    }

    // close writer file descriptor
    if(close(pipe_p1p2[1]) == -1) {
        perror("writer file descriptor close");
        exit(EXIT_FAILURE);
    }
}

Read/Write on a pipe

Because of synchronization between producer and consumer processes:

read() blocks the reader if the pipe is empty
write() blocks the writer if there’s no space left in the pipe

Moreover, there’s no I/O pointer: read and write are FIFO only (First In First Out).

Pipes VS Files

Differences between pipes and files are measured in terms of:

Lifetime
- pipe has process lifetime
- file has file system lifetime
Data Management
Dimension

A pipe uses a FIFO management of data, while a file can be accessed at each position.

The pipe’s dimension is fixed at 4 KiB (4096 bytes).

Redirect stdout and stdin

Redirecting standard output is useful when a process needs to communicate its output to another process through a pipe.

On the other side, redirecting standard input is useful when a process needs to get data from another process through a pipe.

To implement redirection you’ll need to use dup() system call. It creates a copy of the file descriptor passed as parameter using the lowest-numbered unused file descriptor for the new descriptor.

Given the following scenario:

Processes P1 and P2 (both child of P0) communicate through a pipe
P1 is the writer process
P2 is the reader process

Given the pipe:

void main() {
    int pipe_fd[2];
    if (pipe(pipe_fd) != 0) exit(1);
}

If P1 wants to redirect its stdout to the pipe connected to P2, it will have to:

Close standard output:
close(1) or close(STDOUT_FILENO)
Duplicate writer file descriptor:
the lowest-numbered unused file descriptor is now stdout
dup(pipe_fd[1])
Close writer file descriptor:
close(pipe_fd[1])

While if P2 wants to redirect its stdin to the pipe connected to P1, it will have to:

Close standard input:
close(0) or close(STDIN_FILENO)
Duplicate reader file descriptor:
the lowest-numbered unused file descriptor is now stdin
dup(pipe_fd[0])
close reader file descriptor:
close(pipe_fd[0])

In Two Commands Piping example you can see it in action: stdout and stdin redirection is used to implement piping between processes which have a relative in common.

FIFO

FIFO are named pipes. They have the same semantic of pipes:

read() blocks the reader if the FIFO is empty
write() blocks the writer if there’s no space left in the FIFO
reading from a closed pipe returns 0
writing on a closed pipe triggers SIGPIPE signal (which terminate the writer process by default)

Moreover, open() blocks the process waiting for the opening of both sides (write and read).

The biggest difference between pipes and FIFO is that since a FIFO have a name on the file system, they can be used by processes that don’t have relatives in common. Both FIFO and pipes require however that processes run on the same hardware machine.

FIFO Operations

To create a FIFO you need to use mkfifo() system call, passing the name in the file system and the permission bits. Once created, you need to open() the FIFO before using it. To close a FIFO you need to use close() passing the file descriptor as parameter. To destroy a FIFO you need to use unlink passing its file system’s name.

#include <sys/types.h>
#include <sys/stat.h>

// ...

int mode_t mode;
const char *pathname;

int return_value = mkfifo(pathname, mode);

int fd = open(pathname, O_RDONLY);
// int fd = open(pathname, O_WRONLY);

// ...

close(fd);
unlink(pathname);

Examples

Pipe Examples

Note that stdlib.h defines:

EXIT_FAILURE = 1: Failing exit status.
EXIT_SUCCESS = 0: Successful exit status.

While unistd.h library defines standard file descriptors’ numbers:

STDIN_FILENO = 0: Standard input
STDOUT_FILENO = 1: Standard output
STDERR_FILENO = 2: Standard error output

Print argument on stdout

The snippet below prints on stdout the passed argument.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/wait.h>

enum {BUF_SIZE = 4096};

/**
 * Print on the standard output the string passed as first argument
 *
 * Parent writes the string retrieved from the command line argument and then waits for the child to terminate.
 * The child process reads from the pipe and prints the read bytes to the console.
 * Both child and parent close the one end of the pipe because the child process inherits the parent's file descriptors.
 * After the sent bytes are read, the read end of the pipe is closed by the child, terminating with an exit call.
 */
int main(int argc, char *argv[]) {
    int pipe_fd[2];     // store pipe's descriptors
    char buf[BUF_SIZE]; // allocate a buffer
    ssize_t num_read;   // number of characters to read

    // if not string is passed, terminate the program
    if (argc != 2) {
        fprintf(stderr, "Usage: %s string\n", argv[0]);
        exit(EXIT_FAILURE);
    }

    // create the pipe, terminate the program if an error occurs
    if (pipe(pipe_fd) == -1) {
        perror("pipe");
        exit(EXIT_FAILURE);
    }

    // fork process into child and parent processes
    switch (fork()) {
        case -1: {
            perror("fork"); // error while forking
            exit(EXIT_FAILURE);
        }
        case 0: {
            // child process

            // child closes writer file descriptor
            if (close(pipe_fd[1]) == -1) {
                perror("close - parent");
                exit(EXIT_FAILURE);
            }

            // exhausts the data in the pipe
            while(1) {
                // read from the pipe
                num_read = read(pipe_fd[0], buf, BUF_SIZE);

                if(num_read == -1) {
                    perror("read");
                    exit(EXIT_FAILURE);
                }

                if (num_read == 0)
                    break;

                // write on the standard output
                if(write(STDOUT_FILENO, buf, num_read) != num_read) {
                    perror("write - partial/failed write");
                    exit(EXIT_FAILURE);
                }
            }

            // write the new line character on the standard output
            write(STDOUT_FILENO, "\n", 1);

            // child closes reader file descriptor
            if (close(pipe_fd[0]) == -1) {
                perror("close");
                exit(EXIT_FAILURE);
            }

            // terminate program execution with the low-order 8 bits of STATUS.
            _exit(EXIT_SUCCESS);
        }
        default: {
            // parent process

            // parent closes reader file descriptor
            if (close(pipe_fd[0]) == -1) {
                perror("close - parent");
                exit(EXIT_FAILURE);
            }

            // write on the pipe the first argument passed
            if (write(pipe_fd[1], argv[1], strlen(argv[1])) != strlen(argv[1])) {
                perror("write - partial/failed write");
                exit(EXIT_FAILURE);
            }

            // parent closes writer file descriptor
            if (close(pipe_fd[1]) == -1) {
                perror("close");
                exit(EXIT_FAILURE);
            }

            wait(NULL);
            exit(EXIT_SUCCESS);
        }
    }
}

Two Commands Piping

Create piping.c as follows:

#include <stdio.h>    // ISO C99 Standard: 7.19 Input/output
#include <stdlib.h>   // ISO C99 Standard: 7.20 General utilities
#include <string.h>   // ISO C99 Standard: 7.21 String handling
#include <unistd.h>   // POSIX Standard: 2.10 Symbolic Constants
#include <sys/wait.h> // POSIX Standard: 3.2.1 Wait for Process Termination

#define SEPARATOR "@" // must be different from | (pipe sign)

void join(char *cmd1[], char *cmd2[]) {
    int status, p1_pid, p2_pid, pipe_p1p2[2];

    // CREATE PIPE
    if (pipe(pipe_p1p2) != 0) {
        fprintf(stderr, "error during pipe creation\n");
        exit(EXIT_FAILURE);
    }

    // FORK 1st CHILD
    p1_pid = fork();

    if(p1_pid < 0) {
        perror("1st child fork");
        exit(EXIT_FAILURE);
    } else if (p1_pid == 0) {
        // close reader file descriptor
        if (close(pipe_p1p2[0]) == -1) {
            perror("reader file descriptor close");
            exit(EXIT_FAILURE);
        }

        // REDIRECT STDOUT

        // close stdout // close(1);
        if (close(STDOUT_FILENO) == -1) {
            perror("stdout file descriptor close");
            exit(EXIT_FAILURE);
        }

        // first available file descriptor number is 1
        // dup() creates a copy of the file descriptor
        // passed as parameter using the lowest-numbered
        // unused file descriptor for the new descriptor, which is stdout
        // output on the pipe
        if(dup(pipe_p1p2[1]) == -1) {
            perror("duplicate stdout");
            exit(EXIT_FAILURE);
        }

        // close writer file descriptor
        if (close(pipe_p1p2[1]) == -1) {
            perror("write file descriptor close");
            exit(EXIT_FAILURE);
        }

        // EXECUTE THE 1st COMMAND
        execvp(cmd1[0], cmd1);

        // if child process arrives here
        // it means that an error occurred
        perror("1st command error");
        exit(EXIT_FAILURE);
    }

    // FORK 2nd CHILD
    p2_pid = fork();

    if(p2_pid < 0) {
        perror("2nd child fork");
        exit(EXIT_FAILURE);
    } else if (p2_pid == 0) {
        // close writer file descriptor
        if (close(pipe_p1p2[1]) == -1) {
            perror("writer file descriptor close");
            exit(EXIT_FAILURE);
        }

        // REDIRECT STDIN

        // close stdin // close(0);
        if (close(STDIN_FILENO) == -1) {
            perror("stdin file descriptor close");
            exit(EXIT_FAILURE);
        }

        // first available file descriptor number is 0
        // dup() creates a copy of the file descriptor
        // passed as parameter using the lowest-numbered
        // unused file descriptor for the new descriptor, which is stdin
        // input from pipe
        if(dup(pipe_p1p2[0]) == -1) {
            perror("duplicate stdin");
            exit(EXIT_FAILURE);
        }

        // close reader file descriptor
        if (close(pipe_p1p2[0]) == -1) {
            perror("reader file descriptor close");
            exit(EXIT_FAILURE);
        }

        // EXECUTE THE 2nd COMMAND
        execvp(cmd2[0], cmd2);

        // if child process arrives here
        // it means that an error occurred
        perror("2nd command error");
        exit(EXIT_FAILURE);
    }

    // PARENT PROCESS
    // close the non-necessary file descriptors
    close(pipe_p1p2[0]);
    close(pipe_p1p2[1]);

    // wait for children to terminate
    wait(&status);
    wait(&status);
}

/**
 * Provide two different commands separated by SEPARATOR character.
 * It's not possible to use | (pipe sign) in place of SEPARATOR
 * because it will be interpreted by the shell as a pipe.
 */
int main(int argc, char **argv) {
    int i, j;
    char *cmd1[10], *cmd2[10];

    if (argc <= 3) {
        fprintf(stderr, "usage: <cmd_1> SEPARATOR <cmd_2>\n");
        exit(EXIT_FAILURE);
    }

    // 1st COMMAND
    for (i = 1; i < argc && strcmp(argv[i], SEPARATOR); i++)
        cmd1[i - 1] = argv[i];

    // argv terminated by 0
    cmd1[i - 1] = (char *) 0;
    i++;

    // 2nd COMMAND
    for (j = 1; i < argc; i++, j++)
        cmd2[j - 1] = argv[i];

    // TERMINATOR
    cmd2[j - 1] = (char *) 0;

    join(cmd1, cmd2);

    return 0;
}

Compile into piping file:

gcc -o piping piping.c

Pipe the result of ls -lha to wc and get the result on standard output:

./piping ls -lha @ wc
      9      74     427

FIFO Example

Let’s simulate the communication between a client and a server with a couple of separated programs. They both include messages-exchange.h header file to avoid duplicating some common code. Note that the FIFO name must be passed as first parameter.

Client code also handles Ctrl + C signal in order to stop the server in the right way.

// messages-exchange.h

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>
#include <signal.h>
#include <sys/stat.h>

#define MSG_SIZE 50
#define PERM 0744
#define MSG_END "end"

client.c code is:

#include "messages-exchange.h"

int fd;

void client_close();
void handler(int);

int main(int argc, char **argv) {
    if (argc != 2) {
        printf("Error: FIFO name not specified.\n");
        exit(1);
    }

    // when Ctrl + C signal is triggered, send END
    signal(SIGINT, handler);


    char msg[MSG_SIZE];
    const char *pathname = argv[1];

    if ((mkfifo(pathname, PERM)) < 0) {
        perror("fifo already exists, proceed");
    }

    if ((fd = open(pathname, O_WRONLY)) < 0) {
        perror("fifo error");
        exit(1);
    }

    printf("Client: insert messages for the server (prompt \"%s\" to exit)\n", MSG_END);

    while (1) {
        scanf("%s", msg);
        if ((write(fd, msg, strlen(msg) + 1)) < 0) {
            perror("writing error");
            exit(2);
        }
        if (strcmp(msg, MSG_END) == 0) {
            client_close();
        }
    }
}

void handler(int signal) {
    printf("\tReceived signal: %d\n", signal);

    if ((write(fd, MSG_END, strlen(MSG_END) + 1)) < 0) {
        perror("MSG_END writing error");
        exit(2);
    }

    client_close();
}

void client_close() {
    printf("Client ends\n");
    close(fd);
    exit(0);
}

// gcc -o client-main client.c && ./client-main test-fifo

server.c code is:

#include <stdio.h>

int main(int argc, char **argv) {
    if (argc != 2) {
        printf("FIFO name not specified");
        exit(1);
    }

    int fd;
    char buffer[MSG_SIZE];
    const char *pathname = argv[1];

    if ((mkfifo(pathname, PERM)) < 0) {
        perror("fifo already exists, proceed");
    }

    if ((fd = open(pathname, O_RDONLY)) < 0) {
        perror("error opening fifo");
        exit(1);
    }

    printf("I am the server.\n");

    while (1) {
        if ((read(fd, buffer, MSG_SIZE)) < 0) {
            perror("writing error");
            exit(2);
        }

        printf("> server: %s\n", buffer);

        if (strcmp(buffer, MSG_END) == 0) {
            printf("server ends\n");
            close(fd);
            unlink(pathname);
            exit(0);
        }
    }

    return 0;
}

// gcc -o server-main server.c && ./server-main test-fifo

Start client and server programs in different shells:

# Terminal 1
gcc -o client-main client.c
./client-main test-fifo
Client: insert messages for the server (prompt "end" to exit)

Insert messages into FIFO:

Hello
Cat
Pietro

# Terminal 2
gcc -o server-main server.c
./server-main test-fifo
I am the server.

Start reading from FIFO looking for messages:

> server: Hello
> server: Cat
> server: Pietro

Conclusion

Useful links: