Chapter 7 Interprocess Communication
This chapter is for programmers who develop multiprocess applications.
The Solaris 8 and compatible operating environments has a large variety of mechanisms for concurrent processes to exchange data and synchronize execution. These mechanisms include:
-
Pipes: anonymous data queues
-
Named pipes: data queues with file names
-
System V message queues, semaphores, and shared memory
-
POSIX message queues, semaphores, and shared memory
-
Signals: software generated interrupts
-
Sockets
-
Mapped memory and files (see "Memory Management Interfaces")
All of these mechanisms except mapped memory are introduced in this chapter.
Pipes
A pipe between two processes is a pair of files that is created in a parent process. It connects the resulting processes when the parent process forks. A pipe has no existence in any file name space, so it is said to be anonymous. It is most common for a pipe to connect only two processes, although any number of child processes can be connected to each other and their parent by a single pipe.
A pipe is created in the process that becomes the parent by a call to pipe(2). The call returns two file descriptor in the array passed to it. After forking, both processes use p[0] to read from and p[1] to write to. The processes actually read from and write to a circular buffer that is managed for them.
Since on a fork(2) the per-process open file table is duplicated, each process has two readers and two writers. The extra readers and writers must be closed if the pipe is to function properly. For example, no end-of-file indication would ever be returned if the other end of a reader is also open for writing by the same process. The following code shows pipe creation, a fork, and clearing the duplicate pipe ends.
#include <stdio.h>
#include <unistd.h>
...
int p[2];
...
if (pipe(p) == -1) exit(1);
switch( fork() )
{
case 0: /* in child */
close( p[0] );
dup2( p[1], 1);
close P[1] );
exec( ... );
exit(1);
default: /* in parent */
close( p[1] );
dup2( P[0], 0 );
close( p[0] );
break;
}
...
Table 7-1 shows the results of reads from and writes to a pipe under certain conditions.
Table 7-1 Read/write results in a pipe|
Attempt |
Conditions |
Result |
|---|---|---|
|
read |
Empty pipe, writer attached |
Read blocked |
|
write |
Full pipe, reader attached |
Write blocked |
|
read |
Empty pipe, no writer attached |
EOF returned |
|
write |
No reader |
SIGPIPE |
Blocking can be prevented by calling fcntl(2) on the descriptor to set FNDELAY. This causes an error return (-1) from the I/O call with errno set to EWOULDBLOCK.
Named Pipes
Named pipes function much like pipes, but are created as named entities in a file system. This allows the pipe to be opened by any processes with no requirement that they be related by forking. A named pipe is created by a call to mknod(2). Then any process with appropriate permission can read from or write to a named pipe.
In the open(2) call, the process opening the pipe blocks until another process also opens the pipe.
To open a named pipe without blocking, the O_NDELAY mask (found in <sys/fcntl.h>) can be or-ed with the selected file mode mask on the call to open(2). If no other process is connected to the pipe when open(2) is called, -1 is returned with errno set to EWOULDBLOCK.
Sockets
Sockets provide point-to-point, two-way communication between two processes. Sockets are very versatile and are a basic component of interprocess and intersystem communication. A socket is an endpoint of communication to which a name can be bound. It has a type and one or more associated processes.
Sockets exist in communication domains. A socket domain is an abstraction that provides an addressing structure and a set of protocols. Sockets connect only with sockets in the same domain. Twenty-three socket domains are identified (see <sys/socket.h>), of which only the UNIX and Internet domains are normally used in SunOS 5.8 and compatible operating environments.
Sockets can be used to communicate between processes on a single system, like other forms of IPC. The UNIX domain (AF_UNIX) provides a socket address space on a single system. UNIX domain sockets are named with UNIX paths. UNIX domain sockets are further described in "UNIX Domain Sockets" in Network Interface Guide. Sockets can also be used to communicate between processes on different systems. The socket address space between connected systems is called the Internet domain (AF_INET). Internet domain communication uses the TCP/IP internet protocol suite. Internet domain sockets are described in "Socket Interfaces" in Network Interface Guide.
POSIX IPC
POSIX interprocess communication is a variation of System V interprocess communication. It was introduced in Solaris 7. Like System V objects, POSIX IPC objects have read and write (but not execute) permissions for the owner, the owner's group, and for others. There is no way for the owner of a POSIX IPC object to assign a different owner.
Unlike the System V IPC interfaces, the POSIX IPC interfaces are all multithread safe.
POSIX Messages
The POSIX message queue interfaces are:
|
Connects to, and optionally creates, a named message queue |
|
|
Ends the connection to an open message queue |
|
|
Ends the connection to an open message queue and causes the queue to be removed when the last process closes it |
|
|
Places a message in the queue |
|
|
Receives (removes) the oldest, highest priority message from the queue |
|
|
Notifies a process or thread that a message is available in the queue |
|
|
Set or get message queue attributes |
POSIX Semaphores
POSIX semaphores are much lighter weight than are System V semaphores. A POSIX semaphore structure defines a single semaphore, not an array of up to twenty-five semaphores.
The POSIX semaphore interfaces are
|
Connects to, and optionally creates, a named semaphore |
|
|
Initializes a semaphore structure (internal to the calling program, so not a named semaphore) |
|
|
Ends the connection to an open semaphore |
|
|
Ends the connection to an open semaphore and causes the semaphore to be removed when the last process closes it |
|
|
Initializes a semaphore structure (internal to the calling program, so not a named semaphore). |
|
|
Copies the value of the semaphore into the specified integer |
|
|
Blocks while the semaphore is held by other processes or returns an error if the semaphore is held by another process |
|
|
Increments the count of the semaphore |
POSIX Shared Memory
POSIX shared memory is actually a variation of mapped memory (see "Creating and Using Mappings"). The major differences are to use shm_open(3RT) to open the shared memory object (instead of calling open(2)) and use shm_unlink(3RT) to close and delete the object (instead of calling close(2) which does not remove the object). The options in shm_open(3RT) are substantially fewer than the number of options provided in open(2).
System V IPC
The Solaris 8 and compatible operating environments provides an InterProcess Communication (IPC) package that supports three types of interprocess communication that are more versatile than pipes and named pipes.
-
Messages allow processes to send formatted data streams to arbitrary processes.
-
Semaphores allow processes to synchronize execution.
-
Shared memory allows processes to share parts of their virtual address space.
See the ipcrm(1), ipcs(1), Intro(2), msgctl(2), msgget(2), msgrcv(2), msgsnd(2), semget(2), semctl(2), semop(2), shmget(2), shmctl(2), shmop(2), and ftok(3C) manual pages for more information about System V IPC.
Permissions
Messages, semaphores, and shared memory have read and write permissions (but no execute permission) for the owner, group, and others the same as ordinary files. Like files, the creating process identifies the default owner. Unlike files, the creator can assign ownership of the facility to another user; it can also revoke an ownership assignment.
IPC Functions, Key Arguments, and Creation Flags
Processes requesting access to an IPC facility must be able to identify it. To do this, functions that initialize or provide access to an IPC facility use a key_t key argument. The key is an arbitrary value or one that can be derived from a common seed at runtime. One way is with ftok(3C), which converts a file name to a key value that is unique within the system.
Functions that initialize or get access to messages, semaphores, or shared memory return an ID number of type int. IPC functions that perform read, write, and control operations use this ID.
If the key argument is specified as IPC_PRIVATE, the call initializes a new instance of an IPC facility that is private to the creating process.
When the IPC_CREAT flag is supplied in the flags argument appropriate to the call, the function tries to create the facility, if it does not exist already.
When called with both the IPC_CREAT and IPC_EXCL flags, the function fails if the facility already exists. This can be useful when more than one process might attempt to initialize the facility. One such case might involve several server processes having access to the same facility. If they all attempt to create the facility with IPC_EXCL in effect, only the first attempt succeeds.
If neither of these flags is given and the facility already exists, the functions to get access return the ID of the facility. If IPC_CREAT is omitted and the facility is not already initialized, the calls fail.
These control flags are combined, using logical (bitwise) OR, with the octal permission modes to form the flags argument. For example, the statement below initializes a new message queue if the queue does not exist:
msqid = msgget(ftok("/tmp", 'A'), (IPC_CREAT | IPC_EXCL | 0400));
The first argument evaluates to a key ('A') based on the string ("/tmp"). The second argument evaluates to the combined permissions and control flags.
System V Messages
Before a process can send or receive a message, the queue must be initialized through the msgget(2) function. The owner or creator of a queue can change its ownership or permissions using msgctl(2). Also, any process with permission to do so can use msgctl(2) for control operations.
IPC messaging lets processes send and receive messages, and queue messages for processing in an arbitrary order. Unlike the file byte-stream data flow of pipes, each IPC message has an explicit length.
Messages can be assigned a specific type. Because of this, a server process can direct message traffic between clients on its queue by using the client process PID as the message type. For single-message transactions, multiple server processes can work in parallel on transactions sent to a shared message queue.
Operations to send and receive messages are performed by the msgsnd(2) and msgrcv(2) functions, respectively. When a message is sent, its text is copied to the message queue. The msgsnd(2) and msgrcv(2) functions can be performed as either blocking or non-blocking operations. A blocked message operation remains suspended until one of the following three conditions occurs:
-
The call succeeds.
-
The process receives a signal.
-
The queue is removed.
Initializing a Message Queue
The msgget(2) function initializes a new message queue. It can also return the message queue ID (msqid) of the queue corresponding to the key argument. The value passed as the msgflg argument must be an octal integer with settings for the queue's permissions and control flags.
The MSGMNI kernel configuration option determines the maximum number of unique message queues that the kernel will support. msgget(2) fails when this limit is exceeded. The following code illustrates msgget(2):
#include <sys/ipc.h>
#include <sys/msg.h>
...
key_t key; /* key to be passed to msgget() */
int msgflg, /* msgflg to be passed to msgget() */
msqid; /* return value from msgget() */
...
key = ...
msgflg = ...
if ((msqid = msgget(key, msgflg)) == -1)
{
perror("msgget: msgget failed");
exit(1);
} else
(void) fprintf(stderr, "msgget succeeded");
...
Controlling Message Queues
The msgctl(2) function alters the permissions and other characteristics of a message queue. The msqid argument must be the ID of an existing message queue. The cmd argument is one of the following:
The following code illustrates msgctl(2) with all its various flags:
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
...
if (msgctl(msqid, IPC_STAT, &buf) == -1) {
perror("msgctl: msgctl failed");
exit(1);
}
...
if (msgctl(msqid, IPC_SET, &buf) == -1) {
perror("msgctl: msgctl failed");
exit(1);
}
...
Sending and Receiving Messages
The msgsnd(2) and msgrcv(2) functions send and receive messages, respectively. The msqid argument must be the ID of an existing message queue. The msgp argument is a pointer to a structure that contains the type of the message and its text. The msgsz argument specifies the length of the message in bytes. Various control flags can be passed in the msgflg argument.
The following code illustrates msgsnd(2) and msgrcv(2):
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
...
int msgflg; /* message flags for the operation */
struct msgbuf *msgp; /* pointer to the message buffer */
size_t msgsz; /* message size */
size_t maxmsgsize;
long msgtyp; /* desired message type */
int msqid /* message queue ID to be used */
...
msgp = malloc(sizeof(struct msgbuf) - sizeof (msgp->mtext)
+ maxmsgsz);
if (msgp == NULL) {
(void) fprintf(stderr, "msgop: %s %ld byte messages.\n",
"could not allocate message buffer for", maxmsgsz);
exit(1);
...
msgsz = ...
msgflg = ...
if (msgsnd(msqid, msgp, msgsz, msgflg) == -1)
perror("msgop: msgsnd failed");
...
msgsz = ...
msgtyp = first_on_queue;
msgflg = ...
if (rtrn = msgrcv(msqid, msgp, msgsz, msgtyp, msgflg) == -1)
perror("msgop: msgrcv failed");
...
System V Semaphores
Semaphores let processes query or alter status information. They are often used to monitor and control the availability of system resources such as shared memory segments. Semaphores can be operated on as individual units or as elements in a set.
Because System V IPC semaphores can be in a large array, they are extremely heavy weight. Much lighter weight semaphores are available in the threads library (see semaphore(3THR)). Also, POSIX semaphores are the most current implementation of System V semaphores (see "POSIX Semaphores"). Threads library semaphores must be used with mapped memory (see "Memory Management Interfaces").
A semaphore set consists of a control structure and an array of individual semaphores. A set of semaphores can contain up to 25 elements. The semaphore set must be initialized using semget(2). The semaphore creator can change its ownership or permissions using semctl(2). Any process with permission can use semctl(2) to do control operations.
Semaphore operations are performed by the semop(2) function. This function takes a pointer to an array of semaphore operation structures. Each structure in the array contains data about an operation to perform on a semaphore. Any process with read permission can test whether a semaphore has a zero value. Operations to increment or decrement a semaphore require write permission.
When an operation fails, none of the semaphores is altered. The process blocks (unless the IPC_NOWAIT flag is set), and remains blocked until:
-
The semaphore operations can all finish, so the call succeeds
-
The process receives a signal
-
The semaphore set is removed
Only one process at a time can update a semaphore. Simultaneous requests by different processes are performed in an arbitrary order. When an array of operations is given by a semop(2) call, no updates are done until all operations on the array can finish successfully.
If a process with exclusive use of a semaphore terminates abnormally and fails to undo the operation or free the semaphore, the semaphore stays locked in memory in the state the process left it. To prevent this, the SEM_UNDO control flag makes semop(2) allocate an undo structure for each semaphore operation, which contains the operation that returns the semaphore to its previous state. If the process dies, the system applies the operations in the undo structures. This prevents an aborted process from leaving a semaphore set in an inconsistent state.
If processes share access to a resource controlled by a semaphore, operations on the semaphore should not be made with SEM_UNDO in effect. If the process that currently has control of the resource terminates abnormally, the resource is presumed to be inconsistent. Another process must be able to recognize this to restore the resource to a consistent state.
When performing a semaphore operation with SEM_UNDO in effect, you must also have it in effect for the call that will perform the reversing operation. When the process runs normally, the reversing operation updates the undo structure with a complementary value. This ensures that, unless the process is aborted, the values applied to the undo structure are cancel to zero. When the undo structure reaches zero, it is removed.
Using SEM_UNDO inconsistently can lead to excessive resource consumption because allocated undo structures might not be freed until the system is rebooted.
Initializing a Semaphore Set
semget(2) initializes or gains access to a semaphore. When the call succeeds, it returns the semaphore ID (semid). The key argument is a value associated with the semaphore ID. The nsems argument specifies the number of elements in a semaphore array. The call fails when nsems is greater than the number of elements in an existing array; when the correct count is not known, supplying 0 for this argument ensures that it will succeed. The semflg argument specifies the initial access permissions and creation control flags.
The SEMMNI system configuration option determines the maximum number of semaphore arrays allowed. The SEMMNS option determines the maximum possible number of individual semaphores across all semaphore sets. Because of fragmentation between semaphore sets, it might not be possible to allocate all available semaphores.
The following code illustrates semget(2).
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>
...
key_t key; /* key to pass to semget() */
int semflg; /* semflg to pass to semget() */
int nsems; /* nsems to pass to semget() */
int semid; /* return value from semget() */
...
key = ...
nsems = ...
semflg = ...
...
if ((semid = semget(key, nsems, semflg)) == -1) {
perror("semget: semget failed");
exit(1);
} else
exit(0);
...
Controlling Semaphores
semctl(2) changes permissions and other characteristics of a semaphore set. It must be called with a valid semaphore ID. The semnum value selects a semaphore within an array by its index. The cmd argument is one of the following control flags.
|
GETVAL |
Return the value of a single semaphore. |
|
SETVAL |
Set the value of a single semaphore. In this case, arg is taken as arg.val, an int. |
|
GETPID |
Return the PID of the process that performed the last operation on the semaphore or array. |
|
GETNCNT |
Return the number of processes waiting for the value of a semaphore to increase. |
|
GETZCNT |
Return the number of processes waiting for the value of a particular semaphore to reach zero. |
|
GETALL |
Return the values for all semaphores in a set. In this case, arg is taken as arg.array, a pointer to an array of unsigned shorts. |
|
SETALL |
Set values for all semaphores in a set. In this case, arg is taken as arg.array, a pointer to an array of unsigned shorts. |
|
IPC_STAT |
Return the status information from the control structure for the semaphore set and place it in the data structure pointed to by arg.buf, a pointer to a buffer of type semid_ds. |
|
IPC_SET |
Set the effective user and group identification and permissions. In this case, arg is taken as arg.buf. |
|
IPC_RMID |
Remove the specified semaphore set. |
A process must have an effective user identification of owner, creator, or superuser to perform an IPC_SET or IPC_RMID command. Read and write permission is required as for the other control commands.
The following code illustrates semctl(2):
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>
...
register int i;
...
i = semctl(semid, semnum, cmd, arg);
if (i == -1) {
perror("semctl: semctl failed");
exit(1);
...
Semaphore Operations
semop(2) performs operations on a semaphore set. The semid argument is the semaphore ID returned by a previous semget(2) call. The sops argument is a pointer to an array of structures, each containing the following information about a semaphore operation:
-
The semaphore number
-
The operation to be performed
-
Control flags, if any
The sembuf structure specifies a semaphore operation, as defined in <sys/sem.h>. The nsops argument specifies the length of the array, the maximum size of which is determined by the SEMOPM configuration option; this is the maximum number of operations allowed by a single semop(2) call, and is set to 10 by default.
The operation to be performed is determined as follows:
-
Positive integer increments the semaphore value by that amount.
-
Negative integer decrements the semaphore value by that amount. An attempt to set a semaphore to a value less than zero fails or blocks, depending on whether IPC_NOWAIT is in effect.
-
Value of zero means to wait for the semaphore value to reach zero.
The two control flags that can be used with semop(2) are shown below:
|
IPC_NOWAIT |
Can be set for any operations in the array. Makes the function return without changing any semaphore value if any operation for which IPC_NOWAIT is set cannot be performed. The function fails if it tries to decrement a semaphore more than its current value, or tests a nonzero semaphore to be equal to zero. |
|
SEM_UNDO |
Allows individual operations in the array to be undone when the process exits. |
The following code illustrates the semop(2) function:
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>
...
int i; /* work area */
int nsops; /* number of operations to do */
int semid; /* semid of semaphore set */
struct sembuf *sops; /* ptr to operations to perform */
...
if ((i = semop(semid, sops, nsops)) == -1) {
perror("semop: semop failed");
} else
(void) fprintf(stderr, "semop: returned %d\n", i);
...
System V Shared Memory
In the SunOS 5.8 operating system, the most efficient way to implement shared memory applications is to rely on the mmap(2) function and on the system's native virtual memory facility. See Chapter 6, Memory Management .
SunOS 5.8 also supports System V shared memory, which is a less efficient way to let multiple processes attach a segment of physical memory to their virtual address spaces. When write access is allowed for more than one process, an outside protocol or mechanism such as a semaphore can be used to prevent inconsistencies and collisions.
A process creates a shared memory segment using shmget(2). This call is also used to get the ID of an existing shared segment. The creating process sets the permissions and the size in bytes for the segment.
The original owner of a shared memory segment can assign ownership to another user with shmctl(2). It can also revoke this assignment. Other processes with proper permission can perform various control functions on the shared memory segment using shmctl(2).
Once created, a shared segment can be attached to a process address space using shmat(2). It can be detached using shmdt(2). The attaching process must have the appropriate permissions for shmat(2). Once attached, the process can read or write to the segment, as allowed by the permission requested in the attach operation. A shared segment can be attached multiple times by the same process.
A shared memory segment is described by a control structure with a unique ID that points to an area of physical memory. The identifier of the segment is called the shmid. The structure definition for the shared memory segment control structure can be found in <sys/shm.h>.
Accessing a Shared Memory Segment
shmget(2) is used to obtain access to a shared memory segment. When the call succeeds, it returns the shared memory segment ID (shmid). The following code illustrates shmget(2):
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
...
key_t key; /* key to be passed to shmget() */
int shmflg; /* shmflg to be passed to shmget() */
int shmid; /* return value from shmget() */
size_t size; /* size to be passed to shmget() */
...
key = ...
size = ...
shmflg) = ...
if ((shmid = shmget (key, size, shmflg)) == -1) {
perror("shmget: shmget failed");
exit(1);
} else {
(void) fprintf(stderr,
"shmget: shmget returned %d\n", shmid);
exit(0);
}
...
Controlling a Shared Memory Segment
shmctl(2) is used to alter the permissions and other characteristics of a shared memory segment. The cmd argument is one of following control commands:
|
SHM_LOCK |
Lock the specified shared memory segment in memory. The process must have the effective ID of superuser to perform this command. |
|
SHM_UNLOCK |
Unlock the shared memory segment. The process must have the effective ID of superuser to perform this command. |
|
IPC_STAT |
Return the status information contained in the control structure and place it in the buffer pointed to by buf. The process must have read permission on the segment to perform this command. |
|
IPC_SET |
Set the effective user and group identification and access permissions. The process must have an effective ID of owner, creator or superuser to perform this command. |
|
IPC_RMID |
Remove the shared memory segment. The process must have an effective ID of owner, creator or superuser to perform this command. |
The following code illustrates shmctl(2):
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
...
int cmd; /* command code for shmctl() */
int shmid; /* segment ID */
struct shmid_ds shmid_ds; /* shared memory data structure to
hold results */
...
shmid = ...
cmd = ...
if ((rtrn = shmctl(shmid, cmd, shmid_ds)) == -1) {
perror("shmctl: shmctl failed");
exit(1);
...
Attaching and Detaching a Shared Memory Segment
shmat() and shmdt() (see shmop(2)) are used to attach and detach shared memory segments. shmat(2) returns a pointer to the head of the shared segment. shmdt(2) detaches the shared memory segment located at the address indicated by shmaddr. The following code illustrates calls to shmat(2) and shmdt(2):
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
static struct state { /* Internal record of attached segments. */
int shmid; /* shmid of attached segment */
char *shmaddr; /* attach point */
int shmflg; /* flags used on attach */
} ap[MAXnap]; /* State of current attached segments. */
int nap; /* Number of currently attached segments. */
...
char *addr; /* address work variable */
register int i; /* work area */
register struct state *p; /* ptr to current state entry */
...
p = &ap[nap++];
p->shmid = ...
p->shmaddr = ...
p->shmflg = ...
p->shmaddr = shmat(p->shmid, p->shmaddr, p->shmflg);
if(p->shmaddr == (char *)-1) {
perror("shmat failed");
nap--;
} else
(void) fprintf(stderr, "shmop: shmat returned %p\n",
p->shmaddr);
...
i = shmdt(addr);
if(i == -1) {
perror("shmdt failed");
} else {
(void) fprintf(stderr, "shmop: shmdt returned %d\n", i);
for (p = ap, i = nap; i--; p++) {
if (p->shmaddr == addr) *p = ap[--nap];
}
}
...
- © 2010, Oracle Corporation and/or its affiliates