Part I Application Programming Interface
2. STREAMS Application-Level Components
3. STREAMS Application-Level Mechanisms
4. Application Access to the STREAMS Driver and Module Interfaces
7. STREAMS Framework - Kernel Level
Overview of Streams in Kernel Space
Sending and Receiving Messages
Message Queues and Message Priority
Flow Control in Service Procedures
8. STREAMS Kernel-Level Mechanisms
11. Configuring STREAMS Drivers and Modules
14. Debugging STREAMS-based Applications
B. Kernel Utility Interface Summary
The q_qinfo component points to a qinit structure. This structure defines the module's entry point procedures for each queue, which include the following:
int (*qi_putp)(); /* put procedure */ int (*qi_srvp)(); /* service procedure */ int (*qi_qopen)(); /* open procedure */ int (*qi_qclose)(); /* close procedure */ struct module_info *qi_minfo; /* module information structure */
There is generally a unique q_init structure associated with the read queue and the write queue. qi_putp identifies the put procedure for the module. qi_srvp identifies the optional service procedure for the module.
The open and close entry points are required for the read-side queue. The put procedure is generally required on both queues and the service procedure is optional.
Note - Hardening Information. If the put procedure is not defined and a subsequent put is done to the module, a panic occurs. As a precaution, putnext should be declared as the module's put procedure.
If a module only requires a service procedure, putq(9F) can be used as the module's put procedure. If the service procedure is not defined, the module's put procedure must not queue data (putq).
The qi_qopen member of the read-side qinit structure identifies the open(9E) entry point of the module. The qi_qclose member of the read-side qinit structure identifies the close(9E) entry point of the module.
The qi_minfo member points to the module_info(9S) structure.
struct module_info { ushort mi_idnum; /* module id number */ char *mi_idname; /* module name */ ssize_t mi_minpsz; /* min packet size accepted */ ssize_t mi_maxpsz; /* max packet size accepted */ size_t mi_hiwat; /* hi-water mark */ size_t mi_lowat; /* lo-water mark */ };
mi_idnum is the module's unique identifier defined by the developer and used in strlog(9F). mi_idname is an ASCII string containing the name of the module. mi_minpsz is the initial minimum packet size of the queue. mi_maxpsz is the initial maximum packet size of the queue. mi_hiwat is the initial high–water mark of the queue. mi_lowat is the initial low–water mark of the queue.
The open(9E) routine of a device is called once for the initial open of the device, then is called again on subsequent reopens of the stream. Module open routines are called once for the initial push onto the stream and again on subsequent reopens of the stream.
Figure 7-8 Order of a Module's open Procedure
The stream is analogous to a stack. Initially the driver is opened and as modules are pushed onto the stream, their open routines are invoked. Once the stream is built, this order reverses if a reopen of the stream occurs. For example, while building the stream shown in Figure 7-8, device A's open routine is called, followed by B's and C's when they are pushed onto the stream. If the stream is reopened, Module C's open routine is called first, followed by B's, and finally by A's.
Usually the module or driver does not check this, but the issue is raised so that dependencies on the order of open routines are not introduced by the programmer. Note that although an open can happen more than once, close is only called once. See the next section on the close routine for more details. If a file is duplicated (dup(2)) the stream is not reopened.
The prototype for the open entry point is:
int prefix_open(queue_t *q, dev_t *devp, int oflag, int sflag, cred_t *cred_p)
Pointer to the read queue of this module.
Pointer to a device number that is always associated with the device at the end of the stream. Modules cannot modify this value, but drivers can, as described in Chapter 9, STREAMS Drivers.
For devices, oflag can contain the following bit mask values: FEXCL, FNDELAY, FREAD, and FWRITE. See Chapter 9, STREAMS Drivers for more information on drivers.
When the open is associated with a driver, sflag is set to 0 or CLONEOPEN, see Chapter 9, STREAMS Drivers, Cloning STREAMS Drivers for more details. If the open is associated with a module, sflag contains the value MODOPEN.
Pointer to the user credentials structure.
The open routines to devices are serialized . If more than one process attempts to open the device, only one proceeds and the others wait until the first finishes. Interrupts are not blocked during an open. The driver's interrupt routine must continue to handle interrupts when multiple processes are opening the same device. See Chapter 9, STREAMS Drivers for more information.
The open routines for both drivers and modules have user context. For example, they can do blocking operations, but the blocking routine should return in the event of a signal. In other words, q_wait_sig is allowed, but q_wait is not.
If the module or driver is to allocate a controlling terminal, it should send an M_SETOPTS message with SO_ISTTY set to the stream head.
The open routine usually initializes the q_ptr member of the queue. q_ptr is generally initialized to some private data structure that contains various state information private to the module or driver. The module's close routine is responsible for freeing resources allocated by the module including q_ptr. The following example shows a simple open routine.
Example 7-1 A Simple open Routine
/* example of a module open */ int xx_open(queue_t *q, dev_t *devp, int oflag, int sflag, cred_t *crp) { struct xxstr *xx_ptr; xx_ptr = kmemzalloc(sizeof(struct xxstr), KM_SLEEP); xx_ptr->xx_foo = 1; q->q_ptr = WR(q)->q_ptr = xx_ptr; qprocson(q); return (0); }
In a multithreaded environment, data can flow up the stream during the open. A module receiving this data before its open routine finishes initialization can panic. To eliminate this problem, modules and drivers are not linked into the stream until qprocson(9F) is called (messages flow around the module). The following figure illustrates this process. See Chapter 12, Multithreaded STREAMS for more information on the multithreaded environment and the use of perimeters.
Figure 7-9 Messages Flowing Around the Module Before qprocson
The module or driver instance is guaranteed to be single-threaded before qprocson(9F) is called, except for interrupts or callbacks that must be handled separately.
Note - Hardening Information. qprocson must be called before calling qbufcall(9F), qtimeout(9F), qwait(9F), or qwait_sig(9F).
Before a module calls qprocson, it must be ready to accept data via the module's put procedure so all data structures must be fully initialized (see put Procedure). The most common method for calling qprocson is to call this function just before returning from a successful open (see Example 7-1).
Note - For a multithreaded environment, verify you are using the correct perimeter before accessing data. See Chapter 12, Multithreaded STREAMS for more information on the multithreaded environment and the use of perimeters.
The close routine of devices is called only during the last close of the device. Module close routines are called during the last close or if the module is popped.
The prototype for the close entry point is:
int prefix_close (queue *q, int flag, cred_t * cred_p)
is a pointer to the read queue of the module.
is analogous to the oflag parameter of the open entry point. If FNBLOCK or FNDELAY is set, then the module should attempt to avoid blocking during the close.
is a pointer to the user credential structure.
Like open, the close entry point has user context and can block. The blocking routines should return in the event of a signal. Device drivers must take into consideration that interrupts are not blocked during close.
close might be called in a context where a thread cannot receive signals, such as calling close during exit closure of open file descriptors. The system does not reawaken the thread when a user-level process attempts to send a signal, including SIGKILL, to the process.
Use ddi_can_receive_sig to determine whether a thread can receive user-level signals. ddi_can_receive_sig returns B_TRUE if the current thread can receive user-level signals, and B_FALSE if the thread cannot. In this case, qwait_sig behaves exactly like qwait. Use qtimeout or other facilities to prevent close from blocking indefinitely when a thread cannot receive signals.
In particular, asynchronous serial drivers should use caution when draining output data after calling close. Under most conditions, the driver must attempt to wait as long as possible to drain all output data and to discard the data only when a signal is received. However, if ddi_can_receive_sig returns B_FALSE and output flow control is asserted indefinitely by the peer, the driver must abort the drain operation after a reasonable time period has elapsed. Otherwise, the device could remain unusable until the next system boot.
The close routine must cancel all pending and qbufcall callbacks, and process any remaining messages on its service queue. In Example 7-2, the open and close procedures are only used on the read side of the queue and can be set to NULL in the write-side qinit structure initialization.
Example 7-2 Example of a Module Close
/* example of a module close */ static int xx_close(queue_t *, *rq, int flag, cred_t *credp) { struct xxstr *xxp; /* * Disable xxput() and xxsrv() procedures on this queue. */ qprocsoff(rq); xxp = (struct xxstr *) rq->q_ptr; /* * Cancel any pending timeout. * This example assumes that the timeout was issued * against the write queue. */ if (xxp->xx_timeoutid != 0) { (void) quntimeout(WR(rq), xxp->xx_timeoutid); xxp->xx_timeoutid=0; } /* * Cancel any pending bufcalls. * This example assumes that the bufcall was issued * against the write queue. */ if (xxp->xx_bufcallid !=0) { (void) qunbufcall(WR(rq), xxp->xx_bufcallid); xxp->xx_bufcallid = 0; } rq->q_ptr = WR(rq)->q_ptr = NULL; /* * Free resources allocated during open */ kmem_free (xxp, sizeof (struct xxstr)); return (0); }
qprocsoff does the inverse operation shown in Figure 7-9. This supports the need for cancelling callbacks before a qprocsoff.
qprocsoff is typically called at the begining of the close routine. The module can no longer receive messages from adjoining modules. The queue, however, still has pointers to it's adjoining modules and can putnext. However, as the queue is no longer inserted into the stream, these messages may be out of order from other messages in the stream, so it is best to process these messages before qprocsoff.
qwait is used because a module needs to get some response from another module or driver in the STREAM (i.e. a DLPI disconnect message sent downstream). qwait and qwait_sig must also be called before qprocsoff because once the queue is removed from the stream, there will be no way for the reply message to reach the queue.
The put procedure is the mechanism that other modules use to pass messages into this module. This procedure is called via the putor putnext routines on behalf of other modules. The queue's put procedure is invoked by the preceding module to process a message immediately (see put(9F) and putnext(9F)). Most modules will have a put routine. The common exception is on the read-side of drivers because there will not typically be a module downstream to the driver.
Note - Hardening Information. putnext is used by adjoining modules to ensure that the next module's queue is intact. Use of put cannot guarantee that the queue being called is currently valid and inserted into a stream; you must ensure that the queue is valid when using put.
A driver's put procedure must do one of the following:
Process and free the message.
Process and route the message back upstream.
Queue the message to be processed by the driver's service procedure.
All M_IOCTL type messages must be acknowledged through M_IOACK or rejected through M_IOCNACK. M_IOCTL messages should not be freed. Drivers must free any unrecognized message.
A module's put procedure must do one of the following as shown in Figure 7-10:
Process and free the message.
Process the message and pass it to the next module or driver.
Queue the message to be processed later by the module's service procedure.
Unrecognized messages are passed to the next module or driver. The stream operates more efficiently when messages are processed in the put procedure. Processing a message with the service procedure imposes some latency on the message.
Figure 7-10 Flow of put Procedure
If the next module is flow controlled (see canput(9F) and canputnext(9F)), the put procedure can queue the message for processing by the next service procedure (see putq(9F)). The put routine is always called before the component's corresponding srv(9E) service routine, so always use put for immediate message processing.
Note - Hardening Information. canput and canputnext operate similar to put and putnext; that is the next functions verify the integrity of the next queue. Not using the next functions can cause panics as the queue being referenced might have already been closed.
The preferred naming convention for a put procedure reflects the direction of the message flow. The read put procedure is suffixed by r (rput), and the write procedure by w (wput). For example, the read-side put procedure for module xx is declared as int xxrput (queue_t *q, mblk_t *mp). The write-side put procedure is declared as int xxwput(queue_t *q, mblk_t *mp), where q points to the corresponding read or write queue and mp points to the message to be processed.
Although high-priority messages can be placed on the service queue, processing them immediately in the put procedure is better. (See the stub code in Example 7-3.) Place ordinary or priority-band messages on the service queue (putq(9F)) if:
The stream has been flow controlled; that is, canput fails.
There are already messages on the service queue, that is, q_first is not NULL.
Deferred processing is desired.
If other messages already exist on the queue and the put procedure does not queue new messages (provided they are not high-priority), messages are reordered. If the next module is flow controlled (see canput(9F) and canputnext(9F)), the put procedure can queue the message for processing by the service procedure (see putq(9F)).
Example 7-3 Example of a Module put Procedure
/*example of a module put procedure */ int xxrput(queue_t *,mblk_t, *mp) { /* * If the message is a high-priority message or * the next module is not flow controlled and we have not * already deferred processing, then: */ if (mp->b_datap->db_type >= QPCTL || (canputnext(q) && q->q_first == NULL)) { /* * Process message */ . . . putnext(q,mp); } else { /* * put message on service queue */ putq(q,mp); } return (0); }
Put procedures must never call putq, putbq, or qenable if the module does not have a service procedure.
Note - Hardening Information. Once a message is passed using a putq, put, putnext, as well as the perimeter function qwriter, it cannot be accessed again because the use of this message has been given to the new routine. If a reference needs to be retained by the module, it should copy it by using copyb, copymsg, dupb, or dupmsg.
A module need not process the message immediately, and can queue it for later processing by the service procedure (see putq(9F)).
The SunOS STREAMS framework is multithreaded. Unsafe (nonmultithreaded) modules are not supported. To make multithreading of modules easier, the SunOS STREAMS framework uses perimeters (see MT STREAMS Perimeters for more information).
Because of the asynchronous nature of STREAMS, do not assume that a module's put procedure has been called just because put(9F), putnext(9F), or qreply(9F) has returned.
A queue's service procedure is invoked to process messages on the queue. It removes successive messages from the queue, processes them, and calls the put procedure of the next module in the stream to give the processed message to the next queue.
The service procedure is optional. A module or driver can use a service procedure for the following reasons:
Streams flow control is implemented by service procedures. If the next component on the stream has been flow controlled, the put procedure can queue the message. (See Flow Control in Service Procedures in Chapter 7, STREAMS Framework - Kernel Level for more on flow control.)
Resource allocation recovery. If a put or service procedure cannot allocate a resource, such as memory, the message is usually queued to process later.
A device driver can queue a message and get out of interrupt context.
To combine multiple messages into larger messages.
The service procedure is invoked by the STREAMS scheduler. A STREAMS service procedure is scheduled to run if:
The queue is not disabled (noenable(9F)) and the message being queued is either the first message on the queue, or a priority band message.
The message being queued (putq(9F) or putbq(9F)) is a high-priority message,
The queue has been back-enabled because flow control has been relieved,
The queue has been explicitly enabled (qenable(9F)).
A service procedure usually processes all messages on its queue (getq(9F)) or takes appropriate action to ensure it is re-enabled (qenable(9F)) at a later time. Figure 7-11 shows the flow of a service procedure.
Caution - High-priority messages (db_type and MSG_HIPRI) must never be placed back on a service queue (putbq(9F)). Placing these messages in a service queue can cause an infinite loop. |
Put procedures must never call putq, putbq, or qenable if the module does not have a service procedure.
Figure 7-11 Flow of service Procedure
The following example shows the stub code for a module service procedure.
Example 7-4 Module service Procedure
/*example of a module service procedure */ int xxrsrv(queue_t *q) { mblk_t *mp; /* * While there are still messages on our service queue */ while ((mp = getq(q) != NULL) { /* * We check for high priority messages, but * none is ever seen since the put procedure * never queues them. * If the next module is not flow controlled, then */ if (mp->b_datap->db_type >= QPCTL || (canputnext (q)) { /* * process message */ . . . putnext (q, mp); } else { /* * put message back on service queue */ putbq(q,mp); break; } } return (0); }
The queue flow information for each band, other than band 0, is contained in a qband(9S) structure. This structure is not visible to other modules. For accessible information see strqget(9F) and strqset(9F). qband is defined as follows:
typedef struct qband { struct qband *qb_next; /* next band's info */ size_t qb_count; /* number of bytes in band */ struct msgb *qb_first; /* beginning of band's data */ struct msgb *qb_last; /* end of band's data */ size_t qb_hiwat; /* high-water mark for band */ size_t qb_lowat; /* low–water mark for band */ uint qb_flag; /* see below */ } qband_t;
The structure contains pointers to the linked list of messages in the queue. These pointers, qb_first and qb_last, denote the beginning and end of messages for the particular band. The qb_count field is analogous to the queue's q_count field. However, qb_count only applies to the messages in the queue in the band of data flow represented by the corresponding qband structure. In contrast, q_count only contains information regarding normal and high-priority messages.
Each band has a separate high and low watermark, qb_hiwat and qb_lowat. These are initially set to the queue's q_hiwat and q_lowat respectively. Modules and drivers can change these values through the strqset(9F) function. The QB_FULL flag for qb_flag denotes that the particular band is full.
The qband(9S) structures are not preallocated per queue. Rather, they are allocated when a message with a priority greater than zero is placed in the queue using putq(9F), putbq(9F), or insq(9F). Since band allocation can fail, these routines return 0 on failure and 1 on success. Once a qband(9S) structure is allocated, it remains associated with the queue until the queue is freed. strqset(9F) and strqget(9F) cause qband(9S) allocation. Sending a message to a band causes all bands up to and including that one to be created.
The STREAMS utility routines should be used when manipulating the fields in the queue and qband(9S) structures. strqget(9F) and strqset(9F) are used to access band information.
Drivers and modules can change the qb_hiwat and qb_lowat fields of the qband structure. Drivers and modules can only read the qb_count, qb_first, qb_last, and qb_flag fields of the qband structure. Only the fields listed previously can be referenced.
Caution - There are fields in the qband structure that are reserved and are not documented. These fields are subject to undocumented, unnotified change at any time. |
The following figure shows a queue with two extra bands of flow.
Figure 7-12 Data Structure Linkage
Several routines are provided to aid you in controlling each priority band of data flow. These routines are
flushband(9F) is discussed in Flushing Priority Band. bcanputnext(9F) is discussed in Flow Control in Service Procedures, and the other two routines are described in the following section. Appendix B, Kernel Utility Interface Summary also has a description of these routines.
Typically, put procedures are required in pushable modules, but service procedures are optional. If the put routine queues messages, a corresponding service routine must be present to handle the queued messages. If the put routine does not queue messages, the service routine is not required.
Figure 7-10 shows typical processing flow for a put procedure which works as follows:
A message is received by the put procedure associated with the queue, where some processing can be performed on the message.
The put procedure determines if the message can be sent to the next module by the use of canput(9F) or canputnext(9F).
If the next module is flow controlled, the put procedure queues the message using putq(9F).
putq(9F) places the message in the queue based on its priority.
Then, putq(9F) makes the queue ready for execution by the STREAMS scheduler, following all other queues currently scheduled.
If the next module is not flow controlled, the put procedure does any processing needed on the message and sends it to the next module using putnext(9F). Note that if the module does not have a service procedure it cannot queue the message, and must process and send the message to the next module.
Figure 7-11 shows typical processing flow for a service procedure that works as follows:
When the system goes from kernel mode to user mode, the STREAMS scheduler calls the service procedure.
The service procedure gets the first message (q_first) from the message queue using the getq(9F) utility.
The put procedure determines if the message can be sent to the next module using canput(9F) or canputnext(9F).
If the next module is flow controlled, the put procedure requeues the message with putbq(9F), and then returns.
If the next module is not flow controlled, the service procedure processes the message and passes it to the put procedure of the next queue with putnext(9F).
The service procedure gets the next message and processes it. This processing continues until the queue is empty or flow control blocks further processing. The service procedure returns to the caller.
Caution - A service or put procedure must never block since it has no user context. It must always return to its caller. |
If no processing is required in the put procedure, the procedure does not have to be explicitly declared. Rather, putq(9F) can be placed in the qinit(9S) structure declaration for the appropriate queue side to queue the message for the service procedure. For example:
static struct qinit winit = { putq, modwsrv, ...... };
More typically, put procedures process high-priority messages to avoid queueing them.
Device drivers associated with hardware are examples of STREAMS devices that might not have a put procedure. Since there are no queues below the hardware level, another module does not call the module's put procedure. Data comes into the stream from an interrupt routine, and is either processed or queued for the service procedure.
A STREAMS filter is an example of a module without a service procedure—messages passed to it are either passed or filtered. Flow control is described in Flow Control in Service Procedures.
The key attribute of a service procedure in the STREAMS architecture is delayed processing. When a service procedure is used in a module, the module developer is implying that there are other, more time-sensitive activities to be performed elsewhere in this stream, in other streams, or in the system in general.
Note - The presence of a service procedure is mandatory if the flow control mechanism is to be utilized by the queue. If you do not implement flow control, queues can overflow and hang the system.