test

STREAMS Programming Guide

MT STREAMS Perimeters

Solaris STREAMS uses a facility known as perimeters for handling thread concurrency in STREAMS modules. Perimeters allow the module writer to select conditions that will result in exclusive access to a queue, a pair of queues, or all the queues in a module. This makes multithreading issues easier to work with in STREAMS.

Perimeters work somewhat like reader/writer locks, where there can be many readers, but only one writer. A synchronous access to the perimeter is similar to holding the writer lock, in that only one thread can be in the perimeter at a time. Synchronous entry will hold a perimeter exclusively until the thread eventually unwinds out of the perimeter (usually when it returns from a put/putnext or a qwriter call that initially invoked the synchronous behavior). While a thread has synchronous access to the perimeter, all other access (synchronous or asynchronous) will be deferred.

An asynchronous access is similar to the reader lock, where many threads can be in the perimeter at a time, including the possible recursive entry of a thread previously entering the perimeter. The asynchronous "claim" is not released until the thread winds out of the put/putnext. Because asynchronous access is similar to the reader lock, any synchronous access will be deferred or blocked until all asynchronous claims are released.

Caution – Caution –

Hardening Information. The perimeter data is private to the STREAMS framework, and should not be modified by the module itself.

STREAMS enables the definition of two synchronous perimeters. One is an inner perimeter, and is used to define synchronous entry points on a queue or a queue pair (the read and write queue's for a specific module instance). It also identifies an outer perimeter, which is made up of all the inner perimeters for all the queues for a specific module.

There is also a special inner perimeter, PERMOD, that is similar to the outer perimeter, but does not have the overhead of the outer perimeter. PERMOD identifies a single synchronous entry point for all queues for this module. Because PERMOD is like a hybrid of an inner perimeter and outer perimeter, the PERMOD perimeter cannot have an outer perimeter.

Inner Perimeters

For the most part, the module writer does not need to specify an inner perimeter, as the STREAMS framework automatically creates it for the module. What needs to be specified is the type of perimeter, and the concurrency of the perimeter.

Inner perimeters come in two types:

D_PERQ

Enables synchronous entry to be different between the read queue and the write queue. Therefore, if a synchronous putnext is occurring on the read queue, a synchronous or asynchronous putnext can occur on the write queue (or other (a)synchronous access on the write queue).

D_QPAIR

Protects both the read queue and the write queue, so synchronous access to one queue will prevent synchronous or asynchronous access to the other queue.

Another perimeter, D_PERMOD is slightly different, and is discussed in PERMOD Perimeter.

An inner perimeter becomes exclusive (writer) whenever an inner synchronous entry point is encountered. By default all the entry points are considered to be synchronous until enabled as "shared" entry points. As previously stated, synchronous entry points remain exclusive until the thread returns to the caller of the synchronous entry point. If the synchronous function calls putnext, the perimeter remains exclusive across the putnext, up till the synchronous function can return to its caller, and subsequent entries into the perimeter will be deferred.

Inner perimeters can specify additional concurrency on the STREAMS entry points for open/close, put/putnext, service, and callbacks as shown in Table 12–4.

Outer Perimeters

The module writer can also specify an outer perimeter. An outer perimeter is the linked list of all inner perimeters for all queues associated with the specified module. Entering the outer perimeter is equivalent to entering each of the inner perimeters. As this can also be an expensive operation, the outer perimeter is only entered synchronously, and upon successful completion of a qwriter(PERIM_OUTER) makes the outer perimeter exclusive. This also has the effect of making each of the inner perimeters exclusive.

Use of outer perimeters is reserved for module data that has an effect on all queue instances of the module, such as module state that might allow messages to pass between other instances of the module, information that allows a driver to configure shared hardware, or at open/close time when information is needed for all open instances for a module.

Outer perimeters, at this time, have only one concurrency modifier. This is D_MTOCEXCL, and instructs the framework to enter the outer perimeter on each open and close of queues for the module.

PERMOD Perimeter

The PERMOD perimeter is a hybrid of the inner and outer perimeter. It is implemented primarily for modules that might have a large number of queue instances, and cannot afford the latency for entering the outer perimeter. Because it is a hybrid, PERMOD perimeters cannot have an outer perimeter, and modules that have D_MTPERMOD and D_MTOUTPERIM defined will fail at open. As PERMOD perimeters are implemented as inner perimeters, they share all the concurrency states as the inner perimeter, see Table 12–2.

Hot Perimeters

All STREAMS modules and drivers in the Solaris operating environment must be D_MTSAFE, and must account for multithreading. Specifying an inner and/or outer perimeter will handle concurrency issues that the module writer may encounter while developing the module or driver. Experienced STREAMS programmers might decide that the perimeter should not have any synchronous entry points, and should run fully hot. To define a fully-hot perimeter, the module writer need only specify the D_MTSAFE flag without an inner perimeter type (D_MTPERQ, D_MTQPAIR, D_MTPERMOD) and without an outer perimeter (D_MTOUTPERIM).

Caution – Caution –

Hardening Information. All STREAMS entry points run concurrently, and in a multiprocessor environment, there can be a put procedure running simultaneously with a service procedure or even a close procedure. So the writer must take precautions against kernel panics by making sure that other concurrent threads will not reference data the current thread is trying to change or remove.

Defining Perimeter Types

To configure a module with perimeter types and concurrency types, use the f_flag field in fmodsw(9s) with D_MTSAFE or'd with the appropriate perimeter type flags. See Table 12–1.

The easiest method is to initially implement your module and configure it to be per-module single threaded, and increase the level of concurrency as needed. Sample Multithreaded Device Driver Using a Per Module Inner Perimeter provides a complete example of using a per-module perimeter, and Sample Multithreaded Module With Outer Perimeter provides a complete example with a higher level of concurrency.

To configure a driver with perimeter and concurrency types, put MT_SAFE and the appropriate perimeter flags in the cb_flag field of the cb_ops structure for the driver.

Choosing a Perimeter Type

Table 12–1 summarizes examples of when to use an inner perimeter, or both an inner and outer perimeter for a STREAMS module.

Table 12–1 Choosing a Perimeter Type

Module Description 

Perimeter Type 

A module where the put procedure reads as well as modifies module global data. Use a per-module inner perimeter on the single-threaded module.

inner 

A module where all the module private data associated with a queue (or a read/write pair of queues) can be configured to be single-threaded. Use an inner perimeter for each corresponding queue (or queue pair) . 

inner 

A module where most of the module private data is associated with a queue (or a queue pair); but has some module global data that is mostly read. Use an inner perimeter for the queue (or queue pair) plus an outer perimeter for global data. Use qwriter to protect the sections where it modifies the module's global data (see qwriter).

inner and outer 

A module that requires higher concurrency for certain message types while not requiring message ordering. Use an inner perimeter for shared access to the put procedures. Use an outer perimeter for put procedures that require exclusive access. Use qwriter when messages are handled in the put procedures that require exclusive access (see qwriter).

inner and outer 

A hardware driver can use an appropriate set of inner and outer perimeters to restrict the concurrency in the open, close, put, and service procedures. With explicit synchronization primitives (mutex, condition variables, readers/writer, semaphore), these drivers restrict the concurrency when accessing the hardware registers in interrupt handlers. When designing such drivers, you need to be aware of the issues listed in MT SAFE Modules Using Explicit Locks.

inner and outer 

Several flags specify the inner and outer perimeters (see Table 12–2 and Table 12–3). These flags fall into three categories:

You configure the inner perimeter by choosing one of the mutually exclusive flags shown in Table 12–2.

Table 12–2 Inner Perimeter Flags

Flag 

Description 

D_MTPERMOD

The module has an inner perimeter that encloses all the module's queues 

D_MTAPAIR

The module has an inner perimeter around each read/write pair of queues 

D_MTPERQ

The module has an inner perimeter around each queue 

none 

The module has no inner perimeter 

Configure the outer perimeter using the flag shown in Table 12–3.

Table 12–3 Outer Perimeter Flag

Flag 

Description 

D_MTOUTEPERIM

In addition to an inner perimeter (or none), the module has an outer perimeter that encloses all the module's queues. This can be combined with all the inner perimeter options except D_MTPERMOD.

By default all synchronous entry points enter the inner perimeter exclusively and enter the outer perimeter shared. To modify this behavior use the flags shown in Table 12–4.

Table 12–4 Modify Exclusive/Shared Access Flags

Flag 

Description 

D_MTOCEXCL

open/close entry points are synchronous.

D_MTPUTSHARED

put/putnext entry points are asynchronous

_D_MTOCSHARED

open/close entry points are asynchronous (experimental)

_D_MTCBSHARED

callbacks (via qtimeout, qbufcall) are asynchronous (experimental)

_D_MTSVCSHARED

service procedures are asynchronous (experimental)

Caution – Caution –

Hardening Information. Concurrency flags designated with a preceding underbar "_" are experimental, and their behavior might change in the future and should not be relied upon.