This section discusses how to use task queues to postpone processing of some tasks and delegate their execution to another kernel thread.
A common operation in kernel programming is to schedule a task to be performed at a later time, by a different thread. The following examples give some reasons that you might want a different thread to perform a task at a later time:
Your current code path is time critical. The additional task you want to perform is not time critical.
The additional task might require grabbing a lock that another thread is currently holding.
You cannot block in your current context. The additional task might need to block, for example to wait for memory.
A condition is preventing your code path from completing, but your current code path cannot sleep or fail. You need to queue the current task to execute after the condition disappears.
You need to launch multiple tasks in parallel.
In each of these cases, a task is executed in a different context. A different context is usually a different kernel thread with a different set of locks held and possibly a different priority. Task queues provide a generic kernel API for scheduling asynchronous tasks.
A task queue is a list of tasks with one or more threads to service the list. If a task queue has a single service thread, all tasks are guaranteed to execute in the order in which they are added to the list. If a task queue has more than one service thread, the order in which the tasks will execute is not known.
The following DDI interfaces manage task queues. These interfaces are defined in the sys/sunddi.h header file. See the taskq(9F) man page for more information about these interfaces.
Opaque handle
System default priority
Can block for memory
Cannot block for memory
Create a task queue
Destroy a task queue
Add a task to a task queue
Wait for pending tasks to complete
Suspend a task queue
Check whether a task queue is suspended
Resume a suspended task queue
The typical usage in drivers is to create task queues at attach(9E). Most taskq_dispatch() invocations are from interrupt context.
This section describes two techniques that you can use to monitor the system resources that are consumed by a task queue. Task queues export statistics on the use of system time by task queue threads. Task queues also use DTrace SDT probes to determine when a task queue starts and finishes execution of a task.
Every task queue has an associated set of kstat counters. Examine the output of the following kstat(8) command:
$ kstat2 -g '/taskq/*/*/*'
kstat:/taskq/unix/FCOE_WORKER_TASKQ/0
crtime 14516369220291455 ns from boot
executed 0
maxtasks 4
nactive 4
nalloc 4
pid 0
priority 60
snaptime 15394887717683378 ns from boot
tasks 4
threads 4
totaltime 0
The kstat output includes the following information:
The name of the task queue and its instance number
The number of scheduled (tasks) and executed (executed) tasks
The number of kernel threads processing the task queue (threads) and their priority (priority)
The total time (in nanoseconds) spent processing all the tasks (totaltime)
The following example shows how you can use the kstat command to observe how a counter (number of scheduled tasks) increases over time:
$ kstat2 -p '/taskq/unix/callout_taskq/0;tasks' -i 1 -c 5 kstat:/taskq/unix/callout_taskq/0;tasks 12978081 kstat:/taskq/unix/callout_taskq/0;tasks 12978083 kstat:/taskq/unix/callout_taskq/0;tasks 12978101 kstat:/taskq/unix/callout_taskq/0;tasks 12978121 kstat:/taskq/unix/callout_taskq/0;tasks 12978138
Task queues provide several useful SDT probes. All the probes described in this section have the following two arguments:
The task queue pointer returned by ddi_taskq_create()
The pointer to the taskq_ent_t structure. Use this pointer in your D script to extract the function and the argument.
You can use these probes to collect precise timing information about individual task queues and individual tasks being executed through them. For example, the following script prints the functions that were scheduled through task queues for every 10 seconds:
# !/usr/sbin/dtrace -qs
sdt:genunix::taskq-enqueue
{
this->tq = (taskq_t *)arg0;
this->tqe = (taskq_ent_t *) arg1;
@[this->tq->tq_name,
this->tq->tq_instance,
this->tqe->tqent_func] = count();
}
tick-10s
{
printa ("%s(%d): %a called %@d times\n", @);
trunc(@);
}On a particular system, the D script produced the following output:
callout_taskq(1): genunix`callout_execute called 51 times callout_taskq(0): genunix`callout_execute called 701 times kmem_taskq(0): genunix`kmem_update_timeout called 1 times kmem_taskq(0): genunix`kmem_hash_rescale called 4 times callout_taskq(1): genunix`callout_execute called 40 times USB_hid_81_pipehndl_tq_1(14): usba`hcdi_cb_thread called 256 times callout_taskq(0): genunix`callout_execute called 702 times kmem_taskq(0): genunix`kmem_update_timeout called 1 times kmem_taskq(0): genunix`kmem_hash_rescale called 4 times callout_taskq(1): genunix`callout_execute called 28 times USB_hid_81_pipehndl_tq_1(14): usba`hcdi_cb_thread called 228 times callout_taskq(0): genunix`callout_execute called 706 times callout_taskq(1): genunix`callout_execute called 24 times USB_hid_81_pipehndl_tq_1(14): usba`hcdi_cb_thread called 141 times callout_taskq(0): genunix`callout_execute called 708 times