STREAMS Programming Guide

Chapter 1 Overview of STREAMS

This chapter provides a foundation for later chapters. Background and simple definitions are followed by an overview of the STREAMS mechanisms. Because the application developer is concerned with a different subset of STREAMS interfaces than the kernel-level developer, application and kernel levels are described separately.

What Is STREAMS?

STREAMS is a general, flexible programming model for UNIX system communication services. STREAMS defines standard interfaces for character input/output (I/O) within the kernel, and between the kernel and the rest of the UNIX system. The mechanism consists of a set of system calls, kernel resources, and kernel routines.

STREAMS enables you to create modules to provide standard data communications services and then manipulate the modules on a stream. From the application level, modules can be dynamically selected and interconnected. No kernel programming, compiling, and link editing are required to create the interconnection.

STREAMS provides an effective environment for kernel services and drivers requiring modularity. STREAMS parallels the layering model found in networking protocols. For example, STREAMS is suitable for:

Implementing network protocols
Developing character device drivers
Developing network controllers (for example, for an Ethernet card)
I/O terminal services

The fundamental STREAMS unit is the stream. A stream is a full-duplex bidirectional data-transfer path between a process in user space and a STREAMS driver in kernel space. A stream has three parts: a stream head, zero or more modules, and a driver.

Figure 1–1 Simple Stream

Diagram shows the basic components of a stream.

STREAMS Definitions

The capitalized word “STREAMS” refers to the STREAMS programming model and facilities. The word “stream” refers to an instance of a full-duplex path using the model and facilities between a user application and a driver.

Stream as a Data Path

A stream is a data path that passes data in both directions between a STREAMS driver in kernel space, and a process in user space. An application creates a stream by opening a STREAMS device (see Figure 1–1).

Stream Head

A stream head is the end of the stream nearest the user process. It is the interface between the stream and the user process. When a STREAMS device is first opened, the stream consists of only a stream head and a STREAMS driver.

STREAMS Module

A STREAMS module is a defined set of kernel-level routines and data structures. A module does “black-box” processing on data that passes through it. For example, a module converts lowercase characters to uppercase, or adds network routing information. A STREAMS module is dynamically pushed on the stream from the user level by an application. Full details on modules and their operation are covered in Chapter 10, STREAMS Modules.

STREAMS Device Driver

A STREAMS device driver is a character device driver that implements the STREAMS interface. A STREAMS device driver exists below the stream head and any modules. It can act on an external I/O device, or it can be an internal software driver, called a pseudo-device driver. The driver transfers data between the kernel and the device. The interfaces between the driver and kernel are known collectively as the Solaris operating environment Device Driver Interface/Driver Kernel Interface (Solaris operating environment DDI/DKI). The relationship between the driver and the rest of the UNIX kernel is explained in Writing Device Drivers. Details of device drivers are explained in Chapter 9, STREAMS Drivers.

STREAMS Data

Data on a stream is passed in the form of messages. Messages are the means by which all I/O is done under STREAMS. Each stream head, STREAMS module, and driver has a read sideand a write side. When messages go from one module's read side to the next module's read side, they are said to be traveling upstream. Messages passing from one module's write side to the next module's write side are said to be traveling downstream. Kernel-level operation of messages is discussed in Message Components.

Message Queues

Each stream head, driver, and module has its own pair of queues, one queue for the read side and one queue for the write side. Messages are ordered into queues, generally on a first-in, first-out basis (FIFO), according to priorities associated with them. Kernel-level details of queues are covered in Structure of a Message Queue.

Figure 1–2 Messages Passing Using Queues

Diagram shows downstream and upstream message passing
between components of a stream.

Communicating With a STREAMS Device

To communicate with a STREAMS device, an application's process uses read(2), write(2), getmsg(2), getpmsg(2), putmsg(2), putpmsg(2), and ioctl(2) to transmit or receive data on a stream.

From the command line, configure a stream with autopush(1M). From within an application, configure a stream with ioctl(2) as described in streamio(7I).

The ioctl(2) interface performs control operations on and through device drivers that cannot be done through the read(2) and write(2) interfaces. ioctl(2) operations include pushing and popping modules on and off the stream, flushing the stream, and manipulating signals and options. Certain ioctl(2) commands for STREAMS operate on the whole stream, not just the module or driver. The streamio(7I) manual page describes STREAMS ioctl(2) commands. Chapter 4, Application Access to the STREAMS Driver and Module Interfaces details interstream communications.

STREAMS Multiplexing

The modularity of STREAMS allows one or more upper streams to route data into one or more lower streams. This process is defined as multiplexing (mux). Example configurations of multiplexers are described in Configuring Multiplexed Streams.

STREAMS Polling

Polling within STREAMS enables a user process to detect events occurring at the stream head, specifying the event to look for and the amount of time to wait for it to happen. An application might need to interact with multiple streams. The poll(2) system call enables applications to detect events that occur at the head of one or more streams. Chapter 3, STREAMS Application-Level Mechanisms describes polling.

Message Transfer Flow Control

Flow control regulates the rate of message transfer between the user process, stream head, modules, and driver. With flow control, a module that cannot process data at the rate being sent can queue the data to avoid flooding modules upstream. Flow control is local to each module or driver, and is voluntary. Chapter 8, STREAMS Kernel-Level Mechanisms describes flow control.

When to Use STREAMS

The STREAMS framework is most useful when modularity and configurability are issues. For instance, network drivers, terminal drivers, and graphics I/O device drivers benefit from using STREAMS. Modules can be pushed (added) and popped (removed) to create desired program behavior.

STREAMS is general enough to provide modularity between a range of protocols. It is a major component in networking support utilities for UNIX System V because it facilitates communication between network protocols.

How STREAMS Works—Application Interface

An application opens a STREAMS device, which creates the stream head to access the device driver. The stream head packages the data from the user process into STREAMS messages, and passes it downstream into kernel space. One or more cooperating modules can be pushed on a stream between the stream head and driver to customize the stream and perform any of a range of tasks on the data before passing it on. On the other hand, a stream might consist solely of the stream head and driver, with no module at all.

Opening a Stream

To a user application, a STREAMS device resembles an ordinary character I/O device, as it has one or more nodes associated with it in the file system, and is opened by calling open(2).

The file system represents each device as a special file. There is an entry in the file for the major device number, identifying the actual device driver that will activate the device. There are corresponding separate minor device numbers for each instance of a particular device, for example, for a particular port on a serial card, or a specific pseudo-terminal such as those used by a windowing application.

Different minor devices of a driver cause a separate stream to be connected between a user process and the driver. The first open call creates the stream; subsequent open calls respond with a file descriptor referencing that stream. If the same minor device is opened more than once, only one stream is created.

However, drivers can support a user process getting a dedicated stream without the application distinguishing which minor device is used. In this case, the driver selects any unused minor device to be used by the application. This special use of a minor device is called cloning. Chapter 9, STREAMS Drivers describes properties and behavior of clone devices.

Once a device is opened, a user process can send data to the device by calling write(2), and receive data from the device by calling read(2). Access to STREAMS drivers using read and write is compatible with the traditional character I/O mechanism. STREAMS-specific applications also can call getmsg(2), getpmsg(2), putmsg(2), and putpmsg(2) to pass data to and from the stream.

Closing a Stream

The close(2) interface closes a device and dismantles the associated stream when the last open reference to the stream is closed. The exit(2) interface terminates the user process and closes all open files.

Controlling Data Flow

If the stream exerts flow control, the write(2) call blocks until flow control has been relieved, unless the file has been specifically advised not to. open(2) or fcntl(2) can be used to control this nonblocking behavior.

Simple Stream Example

Example 1–1 shows how an application might use a simple stream. Here, the user program interacts with a communications device that provides point-to-point data transfer between two computers. Data written to the device is transmitted over the communications line, and data arriving on the line is retrieved by reading from the device.

Example 1–1 Simple Stream

#include <sys/fcntl.h>
#include <stdio.h>

main()
{
		char buf[1024];
		int fd, count;
	
		if ((fd = open("/dev/ttya", O_RDWR)) < 0) {
			perror("open failed");
			exit(1);
		}
		while ((count = read(fd, buf, sizeof(buf))) > 0) {
			if (write(fd, buf, count) != count) {
				perror("write failed");
				break;
			}
		}
		exit(0);
}

In this example, /dev/ttya identifies an instance of a serial communications device driver. When this file is opened, the system recognizes the device as a STREAMS device and connects a stream to the driver. Figure 1–3 shows the state of the stream following the call to open(2).

Figure 1–3 Stream to Communications Driver

Diagram shows the state of the stream in the Simple Stream
example after a call to open.

This example illustrates a simple loop, with the application reading data from the communications device, then writing the input back to the same device, echoing all input back over the communications line. The program reads up to 1024 bytes at a time, and then writes the number of bytes just read.

read(2) returns the available data, which can contain fewer than 1024 bytes. If no data is currently available at the stream head, read(2) blocks until data arrives.

Note –

The application program must loop on read(2) until the desired number of bytes are read. The responsibility for the application getting all the bytes it needs is that of the application developer, not the STREAMS facilities.

Similarly, the write(2) call attempts to send the specified number of bytes to /dev/ttya. The driver can implement a flow-control mechanism that prevents a user from exhausting system resources by flooding a device driver with data.

How STREAMS Works at the Kernel Level

Developers implementing STREAMS device drivers and STREAMS modules use a set of STREAMS-specific functions and data structures. This section describes some basic kernel-level STREAMS concepts.

Creating the Stream Head

The stream head is created when a user process opens a STREAMS device. It translates the interface calls of the user process into STREAMS messages, which it sends to the stream. The stream head also translates messages originating from the stream into a form that the application can process. The stream head contains a pair of queues; one queue passes messages upstream from the driver, and the other passes messages to the driver. The queues are the pipelines of the stream, passing data between the stream head, modules, and driver.

Message Processing

A STREAMS module does processing operations on messages passing from a stream head to a driver or from a driver to a stream head. For example, a TCP module might add header information to the front of data passing downstream through it. Not every stream requires a module. There can be zero or more modules in a stream.

Modules are stacked (pushed) onto and unstacked (popped) from a stream. Each module must provide open(), close(), and put() entries and provides a service() entry if the module supports flow control.

Like the stream head, each module contains a pair of queue structures, although a module only queues data if it is implementing flow control. Figure 1–4 shows the queue structures Au/Ad associated with Module A (“u” for upstream “d” for downstream) and Bu/Bd associated with Module B.

The two queues operate completely independently. Messages and data can be shared between upstream and downstream queues only if the module functions are specifically programed to share data.

Within a module, one queue can refer to the messages and data of the opposing queue. A queue can directly refer to the queue of the successor module (adjacent in the direction of message flow). For example, in Figure 1–4, Au (the upstream queue from Module A) can reference Bu (the upstream queue from Module B). Similarly Queue Bd can reference Queue Ad.

Figure 1–4 Stream in More Detail

Diagram shows multiple message queues in a complex stream
example.

Both queues in a module contain messages, processing procedures, and private data.

Messages: Blocks of data that pass through, and can be operated on by, a module.
Processing procedures: Individual put and service routines on the read and write queues process messages. The put procedure passes messages from one queue to the next in a stream and is required for each queue. It can do additional message processing. The service procedure is optional and does deferred processing of messages. These procedures can send messages either upstream or downstream. Both procedures can also modify the private data in their module.
Private data: Data private to the module (for example, state information and translation tables).
Open and close: Entry points must be provided. The open routine is invoked when the module is pushed onto the stream or the stream is reopened. The close is invoked when the module is popped or the stream is closed.

A module is initialized by either an I_PUSH ioctl(2), or pushed automatically during an open if a stream has been configured by the autopush(1M) mechanism, or if that stream is reopened.

A module is disengaged by close or the I_POP ioctl(2).

Structure of a STREAMS Device Driver

STREAMS device drivers are structurally similar to STREAMS modules and character device drivers. The STREAMS interfaces to driver routines are identical to the interfaces used for modules. For instance they must both declare open, close, put, and service entry points.

There are some significant differences between modules and drivers.

A driver:

Must be able to handle interrupts from the device.
Is represented in file system by a character-special file.
Is initialized and disengaged using open(2) and close(2). open(2) is called when the device is first opened and for each reopen of the device. close(2) is only called when the last reference to the stream is closed.

Both drivers and modules can pass signals, error codes, and return values to processes using message types provided for that purpose.

Message Components

All kernel-level input and output under STREAMS is based on messages. STREAMS messages are built in sets of three:

a message header structure (msgb(9S)) that identifies the message instance.
a data block structure (datab(9S)) points to the data of the message.
the data itself

Each data block and data pair can be referenced by one or more message headers. The objects passed between STREAMS modules are pointers to messages. Messages are sent through a stream by successive calls to the put procedure of each module or driver in the stream. Messages can exist as independent units, or on a linked list of messages called a message queue. STREAMS utility routines enable developers to manipulate messages and message queues.

All STREAMS messages are assigned message types to indicate how they will be used by modules and drivers and how they will be handled by the stream head. Message types are assigned by the stream head, driver, or module when the message is created. The stream head converts the system calls read, write, putmsg, and putpmsg into specified message types, and sends them downstream. It responds to other calls by copying the contents of certain message types that were sent upstream.

Message Queueing Priority

Sometimes messages with urgent information, such as a break or alarm conditions, must pass through the stream quickly. To accommodate them, STREAMS uses message queuing priority, and high-priority message types. All messages have an associated priority field. Normal (ordinary) messages have a priority of zero, while priority messages have a priority band greater than zero. High-priority messages have a high priority by virtue of their message type, are not blocked by STREAMS flow control, and are processed ahead of all ordinary messages on the queue.

Nonpriority, ordinary messages are placed at the end of the queue following all other messages that can be waiting. Priority messages can be either high priority or priority band messages. High-priority messages are placed at the head of the queue but after any other high-priority messages already in the queue. Priority band messages enable support of urgent, expedited data. Priority band messages are placed in the queue in the following order:

after high-priority messages but before ordinary messages.
below all messages that have a priority greater than or equal to their own.
above any messages with a lesser priority.

Figure 1–5 shows the message queueing priorities.

Figure 1–5 Message Priorities

Diagram demonstrates message queue priorities.

High-priority message types cannot be changed into normal or priority band message types. Certain message types come in equivalent high-priority or ordinary pairs (for example, M_PCPROTO and M_PROTO), so that a module or device driver can choose between the two priorities when sending information.

Structure of a Message Queue

A queue is an interface between a STREAMS driver or module and the rest of the stream (see queue(9S)). The queue structure holds the messages, and points to the STREAMS processing routines that should be applied to a message as it travels through a module. STREAMS modules and drivers must explicitly place messages on a queue, for example, when flow control is used.

Each open driver or pushed module has a pair of queues allocated, one for the read side and one for the write side. Queues are always allocated in pairs. Kernel routines are available to access each queue's mate. The queue's put or service procedure can add a message to the current queue. If a module does not need to queue messages, its put procedure can call the neighboring queue's put procedure.

The queue's service procedure deals with messages on the queue, usually by removing successive messages from the queue, processing them, and calling the put procedure of the next module in the stream to pass the message to the next queue. Chapter 7, STREAMS Framework – Kernel Level discusses the service and put procedures in more detail.

Each queue also has a pointer to an open and close routine. The open routine of a driver is called when the driver is first opened and on every successive open of the stream. The open routine of a module is called when the module is first pushed on the stream and on every successive open of the stream. The close routine of the module is called when the module is popped (removed) off the stream, or at the time of the final close. The close routine of the driver is called when the last reference to the stream is closed and the stream is dismantled.

Configuring Multiplexed Streams

Previously, streams were described as stacks of modules, with each module (except the head) connected to one upstream module and one downstream module. While this can be suitable for many applications, others need the ability to multiplex streams in a variety of configurations. Typical examples are terminal window facilities, and internetworking protocols (that might route data over several subnetworks).

An example of a multiplexer is a module that multiplexes data from several upper streams to a single lower stream. An upper stream is one that is upstream from the multiplexer, and a lower stream is one that is downstream from the multiplexer. A terminal windowing facility might be implemented in this fashion, where each upper stream is associated with a separate window.

A second type of multiplexer might route data from a single upper stream to one of several lower streams. An internetworking protocol could take this form, where each lower stream links the protocol to a different physical network.

A third type of multiplexer might route data from one of many upper streams to one of many lower streams.

The STREAMS mechanism supports the multiplexing of streams through special pseudo-device drivers. A user can activate a linking facility mechanism within the STREAMS framework to dynamically build, maintain, and dismantle multiplexed stream configurations. Simple configurations like those shown previously can be combined to form complex, multilevel multiplexed stream configurations.

STREAMS multiplexing configurations are created in the kernel by interconnecting multiple streams. Conceptually, a multiplexer can be divided into two components—the upper multiplexer and the lower multiplexer. The lower multiplexer acts as a stream head for one or more lower streams. The upper multiplexer acts as a device for one or more upper streams. How data is passed between the upper and lower multiplexer is up to the implementation. Chapter 13, STREAMS Multiplex Drivers covers implementing multiplexers.

Multithreading the Kernel

The Solaris operating environment kernel is multithreaded to make effective use of symmetric shared-memory multiprocessor computers. All parts of the kernel, including STREAMS modules and drivers, must ensure data integrity in a multiprocessing environment. For the most part, developers must ensure that concurrently running kernel threads do not attempt to manipulate the same data at the same time. The STREAMS framework provides multithreaded (MT) STREAMS perimeters, which provides the developer with control over the level of concurrency allowed in a module. The DDI/DKI provides several advisory locks for protecting data. See Chapter 12, Multithreaded STREAMS for more information.

Service Interfaces

Using STREAMS, you can create modules that present a service interface to any neighboring module or device driver, or between the top module and a user application. A service interface is defined in the boundary between two neighbors.

In STREAMS, a service interface is a set of messages and the rules that allow these messages to pass across the boundary. A module using a service interface, for example, receives a message from a neighbor and responds with an appropriate action (perhaps sending back a request to retransmit) depending on the circumstances.

You can stack a module anywhere in a stream, but connecting sequences of modules with compatible protocol service interfaces is better. For example, a module that implements an X.25 protocol layer, as shown in Figure 1–6, presents a protocol service interface at its input and output sides. In this case, other modules should be connected to the input and output side if they have the compatible X.25 service interface only.

Manipulating Modules

With STREAMS, you can manipulate modules from the user application level, interchange modules with common service interfaces, and change the service interface to a STREAMS user process. These capabilities yield further benefits when working with networking services and protocols:

User-level programs can be independent of underlying protocols and physical communication media.
Network architectures and higher-level protocols can be independent of underlying protocols, drivers, and physical communication media.
Higher-level services can be created by selecting and connecting lower-level services and protocols.

The following examples show the benefits of STREAMS capabilities for creating service interfaces and manipulating modules. These examples are only illustrations and do not necessarily reflect real situations.

Protocol Portability

Figure 1–6 shows how an X.25 protocol module can work with different drivers on different machines by using compatible service interfaces. The X.25 protocol module interfaces are Connection Oriented Network Service (CONS) and Link Access Protocol – Balanced (LAPB).

Figure 1–6 Protocol Module Portability

Diagram demonstrates that the same protocol module can
be used by different drivers on different machines.

Protocol Substitution

You can alternate protocol modules and device drivers on a system if the alternates are implemented to an equivalent service interface.

Protocol Migration

Figure 1–7 shows how STREAMS can move functions between kernel software and front-end firmware. A common downstream service interface lets the transport protocol module be independent of the number or type of modules below it. The same transport module will connect without modification to either an X.25 module or X.25 driver that has the same service interface.

By shifting functions between software and firmware, you can produce cost-effective, functionally equivalent systems over a wide range of configurations. This means you can swiftly incorporate technological advances. The same transport protocol module can be used on a lower-capacity machine, where economics preclude the use of front-end hardware, and also on a larger scale system where a front-end is economically justified.

Figure 1–7 Protocol Migration

Diagram shows that the protocol layer can be included
in a driver rather than in a separate module.

Module Reusability

Figure 1–8 shows the same canonical module (for example, one that provides delete and kill processing on character strings) reused in two different streams. This module would typically be implemented as a filter, with no service interface. In both cases, a TTY interface is presented to the stream's user process because the module is nearest the stream head.

Figure 1–8 Module Reusability

Diagram shows reuse of the same canonical module in two
different streams.