Chapter 12 Device Context Management
Some device drivers, such as those for graphics hardware, provide user processes with direct access to the device. These devices often require that only one process at a time accesses the device.
This chapter describes the set of interfaces that allow device drivers to manage access to such devices.
What Is a Device Context?
The context of a device is the current state of the device hardware. The device driver manages the device context for a process on behalf of the process. The device driver must maintain a separate device context for each process that accesses the device. It is the device driver's responsibility to restore the correct device context when a process accesses the device.
Context Management Model
An accelerated frame buffer is an example of a device that allows user processes (such as graphics applications) to directly manipulate the control registers of the device through memory-mapped access. Because these processes are not using the traditional I/O system calls (read(2), write(2), and ioctl(2)), the device driver is no longer called when a process accesses the device. However, it is important that the device driver be notified when a process is about to access a device so that it can restore the correct device context and provide any needed synchronization.
To resolve this problem, the device context management interfaces enable a device driver to be notified when a user process accesses memory-mapped regions of the device and to control accesses to the device's hardware. Synchronization and management of the various device contexts are responsibilities of the device driver. When a user process accesses a mapping, the device driver must restore the correct device context for that process.
A device driver will be notified whenever one of the following events occurs:
-
Access to a mapping by a user process
-
Duplication of a mapping by a user process
-
Freeing of a mapping by a user process
-
Creation of a mapping by a user process
Figure 12-1 shows multiple user processes that have memory mapped a device. The driver has granted process B access to the device, and process B no longer notifies the driver of accesses. However, the driver is still notified if either process A or process C accesses the device.
Figure 12-1 Device Context Management
At some point in the future, process A accesses the device. The device driver is notified of this and blocks future access to the device by process B. It then saves the device context for process B, restores the device context of process A, and grants access to process A, as illustrated in Figure 12-2. At this point, the device driver will be notified if either process B or process C accesses the device.
Figure 12-2 Device Context Switched to User Process A
Multiprocessor Considerations
On a multiprocessor machine, multiple processes could be attempting to access the device at the same time. This can cause thrashing. Some devices require a longer time to restore a device context. To prevent more CPU time from being used to restore a device context than to actually use that device context, the minimum time that a process needs to have access to the device can be set using devmap_set_ctx_timeout(9F).
The kernel guarantees that once a device driver has granted access to a process, no other process will be allowed to request access to the same device for the time interval specified by devmap_set_ctx_timeout(9F).
Context Management Additions to the State Structure
This chapter adds the following fields to the state structure. See "Software State Structure" for more information.
kmutex_t ctx_lock; /* lock for context switching */ void *ctx_shared; /* pointer to shared context */ struct xxctx *current_ctx; /* current context structure */
The structure xxctx is the driver private device context structure for the examples used in this section. It looks like this:
struct xxctx {
devmap_cookie_t handle;
offset_t off;
size_t len;
uint_t flags;
void *context;
struct xxstate *xsp;
};
#define XXCTX_SIZE 0x1000 /* size of the context */
The context field stores the actual device context. In this case, it is simply a pointer to a chunk of memory; in other cases, it may actually be a series of structure fields corresponding to device registers.
Context Management Operation
In general, the steps for performing device context management are:
-
Define a devmap_callback_ctl(9S) structure.
-
Allocate space to save device context if necessary.
-
Set up user mappings to the device and driver notifications with devmap_devmem_setup(9F).
-
Manage user access to the device with devmap_load(9F) and devmap_unload(9F).
-
Free the device context structure, if needed.
devmap_callback_ctl Structure
The device driver must allocate and initialize a devmap_callback_ctl(9S) structure to inform the system of its device context management entry point routines. This structure contains the following fields:
struct devmap_callback_ctl {
int devmap_rev;
int (*devmap_map)(devmap_cookie_t dhp, dev_t dev,
u_int flags, offset_t off, size_t len, void **pvtp);
int (*devmap_access)(devmap_cookie_t dhp, void *pvtp,
offset_t off, size_t len, u_int type, u_int rw);
int (*devmap_dup)(devmap_cookie_t dhp, void *pvtp,
devmap_cookie_t new_dhp, void **new_pvtp);
void (*devmap_unmap)(devmap_cookie_t dhp, void *pvtp,
offset_t off, size_t len, devmap_cookie_t new_dhp1,
void **new_pvtp1, devmap_cookie_t new_dhp2,
void **new_pvtp2);
};
devmap_rev is the version number of the devmap_callback_ctl(9S) structure. It must be set to DEVMAP_OPS_REV.
devmap_map must be set to the address of the driver's devmap_map(9E) entry point.
devmap_access must be set to the address of the driver's devmap_access(9E) entry point.
devmap_dup must be set to the address of the driver's devmap_dup(9E) entry point.
devmap_unmap must be set to the address of the driver's devmap_unmap(9E) entry point.
Associating User Mappings With Driver Notifications
When a user process requests a mapping to a device with mmap(2), the driver`s segmap(9E) entry point is called. The driver must use ddi_devmap_segmap(9F) or devmap_setup(9F) when setting up the memory mapping if it needs to manage device contexts. Both functions will call the driver's devmap(9E) entry point, which uses devmap_devmem_setup(9F) to associate the device memory with the user mapping. See Chapter 11, Mapping Device or Kernel Memory for details on how to map device memory.
For the driver to get notifications on accesses to the user mapping, it has to inform the system of the devmap_callback_ctl(9S) entry points. It does this by providing a pointer to a devmap_callback_ctl(9S) structure to devmap_devmem_setup(9F). A devmap_callback_ctl(9S) structure describes a set of context management entry points that are called by the system to notify a device driver to manage events on the device mappings.
The system associates each mapping with a mapping handle. This handle is passed to each of the context management entry points. The mapping handle can be used to invalidate and validate the mapping translations. If the driver invalidates the mapping translations, it will be notified of any future access to the mapping. If the driver validates the mapping translations, it will no longer be notified of accesses to the mapping. Mappings are always created with the mapping translations invalidated so that the driver will be notified on first access to the mapping.
Example 12-1 shows how to set up a mapping using the device context management interfaces.
Example 12-1 devmap(9E) Entry Point With Context Management Support
static struct devmap_callback_ctl xx_callback_ctl = {
DEVMAP_OPS_REV,
xxdevmap_map,
xxdevmap_access,
xxdevmap_dup,
xxdevmap_unmap
};
static int
xxdevmap(dev_t dev, devmap_cookie_t handle, offset_t off,
size_t len, size_t *maplen, uint_t model)
{
struct xxstate *xsp;
u_int rnumber;
int error;
/* Setup data access attribute structure */
struct ddi_device_acc_attr xx_acc_attr = {
DDI_DEVICE_ATTR_V0,
DDI_NEVERSWAP_ACC,
DDI_STRICTORDER_ACC
};
xsp = ddi_get_soft_state(statep, getminor(dev));
if (xsp == NULL)
return (ENXIO);
len = ptob(btopr(len));
rnumber = 0;
/* Set up the device mapping */
error = devmap_devmem_setup(handle, xsp->dip,
&xx_callback_ctl,
rnumber, off, len, PROT_ALL, 0, &xx_acc_attr);
*maplen = len;
return (error);
}
Managing Mapping Accesses
The device driver is notified when a user process accesses an address in the memory-mapped region that does not have valid mapping translations. When the access event occurs, the mapping translations of the process that currently has access to the device must be invalidated. The device context of the process requesting access to the device must be restored, and the translations of the mapping of the process requesting access must be validated.
The functions devmap_load(9F) and devmap_unload(9F) are used to validate and invalidate mapping translations.
devmap_load()
int devmap_load(devmap_cookie_t handle, offset_t offset, size_t len, uint_t type, uint_t rw);
devmap_load(9F) validates the mapping translations for the pages of the mapping specified by handle,offset, and len. By validating the mapping translations for these pages, the driver is telling the system not to intercept accesses to these pages of the mapping and to allow accesses to proceed without notifying the device driver.
devmap_load(9F) must be called with the offset and the handle of the mapping that generated the access event for the access to complete. If devmap_load(9F) is not called on this handle, the mapping translations will not be validated, and the process will receive a SIGBUS.
devmap_unload()
int devmap_unload(devmap_cookie_t handle, offset_t offset, size_t len);
devmap_unload(9F) invalidates the mapping translations for the pages of the mapping specified by handle, offset, and len. By invalidating the mapping translations for these pages, the device driver is telling the system to intercept accesses to these pages of the mapping and notify the device driver the next time these pages of the mapping are accessed by calling the devmap_access(9E) entry point.
For both functions, requests affect the entire page containing the offset and all pages up to and including the entire page containing the last byte, as indicated by offset + len. The device driver must ensure that for each page of device memory being mapped only one process has valid translations at any one time.
Both functions return zero if they are successful. If, however, there was an error in validating or invalidating the mapping translations, that error is returned to the device driver. It is the device driver's responsibility to return this error to the system.
Device Context Management Entry Points
The following device driver entry points are used to manage device context.
devmap_map()
int xxdevmap_map(devmap_cookie_t handle, dev_t dev, u_int flags, offset_t offset, size_t len, void **new_devprivate);
This entry point is called after the driver returns from its devmap(9E) entry point and the system has established the user mapping to the device memory. The devmap_map(9E) entry point enables a driver to perform additional processing or to allocate mapping specific private data. For example, in order to support context switching, the driver has to allocate a context structure and associate it with the mapping.
The system expects the driver to return a pointer to the allocated private data in *new_devprivate. The driver must store offset and len, which define the range of the mapping, in its private data. Later, when the system calls devmap_unmap(9E), the driver will use offset and len stored in new_devprivate to check if the entire mapping, or just a part of it, is being unmapped.
flags indicates whether the driver should allocate a private context for the mapping. For example, a driver may allocate a memory region to store the device context if flags is set to MAP_PRIVATE, or it might return a pointer to a shared region if MAP_SHARED is set.
Example 12-2 shows an example of a devmap_map(9E) entry point. The driver allocates a new context structure and saves relevant parameters passed in by the entry point. Then the mapping is assigned a new context by either allocating a new one or attaching it to an already existing shared context. The minimum time interval that the mapping should have access to the device is set to one millisecond.
Example 12-2 devmap_map(9E) Routine
static int
int xxdevmap_map(devmap_cookie_t handle, dev_t dev, u_int flags,
offset_t offset, size_t len, void **new_devprivate)
{
struct xxstate *xsp = ddi_get_soft_state(statep,
getminor(dev));
struct xxctx *newctx;
/* create a new context structure */
newctx = kmem_alloc(sizeof (struct xxctx), KM_SLEEP);
newctx->xsp = xsp;
newctx->handle = handle;
newctx->offset = offset;
newctx->flags = flags;
newctx->len = len;
mutex_enter(&xsp->ctx_lock);
if (flags & MAP_PRIVATE) {
/* allocate a private context and initialize it */
newctx->context = kmem_alloc(XXCTX_SIZE, KM_SLEEP);
xxctxinit(newctx);
} else {
/* set a pointer to the shared context */
newctx->context = xsp->ctx_shared;
}
mutex_exit(&xsp->ctx_lock);
/* give at least 1 ms access before context switching */
devmap_set_ctx_timeout(handle, drv_usectohz(1000));
/* return the context strcuture */
*new_devprivate = newctx;
return(0);
}
devmap_access()
int xxdevmap_access(devmap_cookie_t handle, void *devprivate, offset_t offset, size_t len, u_int type, u_int rw);
This entry point is called when an access is made to a mapping whose translations are invalid. Mapping translations are invalidated when the mapping is created with devmap_devmem_setup(9F) in response to mmap(2), duplicated by fork(2), or explicitly invalidated by a call to devmap_unload(9F).
handle is the mapping handle of the mapping that was accessed by a user process.
devprivate is a pointer to the driver private data associated with the mapping.
offset is the offset within the mapping that was accessed.
len is the length in bytes of the memory being accessed.
type is the type of access operation.
rw specifies the direction of access.
The system expects devmap_access(9E) to call either devmap_do_ctxmgt(9F) or devmap_default_access(9F) to load the memory address translations before it returns. For mappings that support context switching, the device driver should call devmap_do_ctxmgt(9F). This routine is passed all parameters from devmap_access(9E), and in addition a pointer to the driver entry point devmap_contextmgt(9E), which handles the context switching. For mappings that do not support context switching, the driver should call devmap_default_access(9F), whose only purpose is to call devmap_load(9F) to load the user translation.
Example 12-3 shows an example of a devmap_access(9E) entry point. The mapping is devided into two regions. The region starting at offset OFF_CTXMG with a length of CTXMGT_SIZE bytes supports context management. The rest of the mapping supports default access.
Example 12-3 devmap_access(9E) Routine
#define OFF_CTXMG 0
#define CTXMGT_SIZE 0x20000
static int
xxdevmap_access(devmap_cookie_t handle, void *devprivate,
offset_t off, size_t len, u_int type, u_int rw)
{
offset_t diff;
int error;
if ((diff = off - OFF_CTXMG) >= 0 && diff < CTXMGT_SIZE) {
error = devmap_do_ctxmgt(handle, devprivate, off, len, type,
rw, xxdevmap_contextmgt);
} else {
error = devmap_default_access(handle, devprivate, off,
len, type, rw);
}
return (error);
}
devmap_contextmgt()
int xxdevmap_contextmgt(devmap_cookie_t handle, void *devprivate, offset_t offset, size_t len, u_int type, u_int rw);
In general, devmap_contextmgt(9E) should call devmap_unload(9F), with the handle of the mapping that currently has access to the device, to invalidate the translations for that mapping. This ensures that a call to devmap_access(9E) occurs for the current mapping the next time it is accessed. To validate the mapping translations for the mapping that caused the access event to occur, the driver must restore the device context for the process requesting access and call devmap_load(9F) on the handle of the mapping that generated the call to this entry point.
Accesses to portions of mappings that have had their mapping translations validated by a call to devmap_load(9F) do not generate a call to devmap_access(9E). A subsequent call to devmap_unload(9F) invalidates the mapping translations and allows devmap_access(9E) to be called again.
If either devmap_load(9F) or devmap_unload(9F) returns an error, devmap_contextmgt(9E) should immediately return that error. If the device driver encounters a hardware failure while restoring a device context, a -1 should be returned. Otherwise, after successfully handling the access request, devmap_contextmgt(9E) should return zero. A return of other than zero from devmap_contextmgt(9E) will cause a SIGBUS or SIGSEGV to be sent to the process.
Example 12-4 shows how to manage a one-page device context.
Note -
xxctxsave() and xxctxrestore() are device-dependent context save and restore functions. xxctxsave() reads data from the registers using the Solaris 7 DDI/DKI data access routines and saves it in the soft state structure. xxctxrestore() takes data saved in the soft state structure and writes it to device registers using the Solaris 7 DDI/DKI data access routines.
Example 12-4 devmap_contextmgt(9E) Routine
static int
xxdevmap_contextmgt(devmap_cookie_t handle, void *devprivate,
offset_t off, size_t len, u_int type, u_int rw)
{
int error;
struct xxctx *ctxp = devprivate;
struct xxstate *xsp = ctxp->xsp;
mutex_enter(&xsp->ctx_lock);
/* unload mapping for current context */
if (xsp->current_ctx != NULL) {
if ((error = devmap_unload(xsp->current_ctx->handle,
off, len)) != 0) {
xsp->current_ctx = NULL;
mutex_exit(&xsp->ctx_lock);
return (error);
}
}
/* Switch device context - device dependent */
if (xxctxsave(xsp->current_ctx, off, len) < 0) {
xsp->current_ctx = NULL;
mutex_exit(&xsp->ctx_lock);
return (-1);
}
if (xxctxrestore(ctxp, off, len) < 0){
xsp->current_ctx = NULL;
mutex_exit(&xsp->ctx_lock);
return (-1);
}
xsp->current_ctx = ctxp;
/* establish mapping for new context and return */
error = devmap_load(handle, off, len, type, rw);
if (error)
xsp->current_ctx = NULL;
mutex_exit(&xsp->ctx_lock);
return (error);
}
devmap_dup()
int xxdevmap_dup(devmap_cookie_t handle, void *devprivate, devmap_cookie_t new_handle, void **new_devprivate);
This entry point is called when a device mapping is duplicated, for example, by a user process calling fork(2). The driver is expected to generate new driver private data for the new mapping.
handle is the mapping handle of the mapping being duplicated.
new_handle is the mapping handle of the mapping that was duplicated.
devprivate is a pointer to the driver private data associated with the mapping being duplicated.
*new_devprivate should be set to point to the new driver private data for the new mapping.
Mappings created with devmap_dup(9E) will, by default, have their mapping translations invalidated. This will force a call to the devmap_access(9E) entry point the first time the mapping is accessed.
Example 12-5 shows a devmap_dup(9E) routine.
Example 12-5 devmap_dup(9E) Routine
static int
xxdevmap_dup(devmap_cookie_t handle, void *devprivate,
devmap_cookie_t new_handle, void **new_devprivate)
{
struct xxctx *ctxp = devprivate;
struct xxstate *xsp = ctxp->xsp;
struct xxctx *newctx;
/* Create a new context for the duplicated mapping */
newctx = kmem_alloc(sizeof (struct xxctx), KM_SLEEP);
newctx->xsp = xsp;
newctx->handle = new_handle;
newctx->offset = ctxp->offset;
newctx->flags = ctxp->flags;
newctx->len = ctxp->len;
mutex_enter(&xsp->ctx_lock);
if (ctxp->flags & MAP_PRIVATE) {
newctx->context = kmem_alloc(XXCTX_SIZE, KM_SLEEP);
bcopy(ctxp->context, newctx->context, XXCTX_SIZE);
} else {
newctx->context = xsp->ctx_shared;
}
mutex_exit(&xsp->ctx_lock);
*new_devprivate = newctx;
return(0);
}
devmap_unmap()
void xxdevmap_unmap(devmap_cookie_t handle, void *devprivate, offset_t off, size_t len, devmap_cookie_t new_handle1, void **new_devprivate1, devmap_cookie_t new_handle2, void **new_devprivate2);
This entry point is called when a mapping is unmapped. This can be caused by a user process exiting or calling the munmap(2) system call.
handle is the mapping handle of the mapping being freed.
devprivate is a pointer to the driver private data associated with the mapping.
off is the offset within the logical device memory at which the unmapping begins.
len is the length in bytes of the memory being unmapped.
new_handle1 is the handle that the system uses to describe the new region that ends at off - 1. new_handle1 may be NULL.
new_devprivate1 is a pointer to be filled in by the driver with the driver -private mapping data for the new region that ends at off - 1. It is ignored if new_handle1 is NULL.
new_handle2 is the handle that the system uses to describe the new region that begins at off + len. new_handle2 may be NULL.
new_devprivate2 is a pointer to be filled in by the driver with the driver private mapping data for the new region that begins at off + len. It is ignored if new_handle2 is NULL.
The devmap_unmap(9E) routine is expected to free any driver private resources that were allocated when this mapping was created, either by devmap_map(9F) or by devmap_dup(9E). If only a part of the mapping is being unmapped, the driver must allocate a new private data for the remaining mapping before freeing the old private data. There is no need to call devmap_unload(9F) on the handle of the mapping being freed, even if it is the mapping with the valid translations. However, to prevent future problems in devmap_access(9E), the device driver should make sure that its representation of the current mapping is set to "no current mapping".
Example 12-6 shows an example of a devmap_unmap(9E) routine.
Example 12-6 devmap_unmap(9E) Routine
static void
xxdevmap_unmap(devmap_cookie_t handle, void *devprivate,
offset_t off, size_t len, devmap_cookie_t new_handle1,
void **new_devprivate1, devmap_cookie_t new_handle2,
void **new_devprivate2)
{
struct xxctx *ctxp = devprivate;
struct xxstate *xsp = ctxp->xsp;
mutex_enter(&xsp->ctx_lock);
/*
* If new_handle1 is not NULL, we are unmapping
* at the end of the mapping.
*/
if (new_handle1 != NULL) {
/* Create a new context structure for the mapping */
newctx = kmem_alloc(sizeof (struct xxctx), KM_SLEEP);
newctx->xsp = xsp;
if (ctxp->flags & MAP_PRIVATE) {
/* allocate memory for the private context and copy it */
newctx->context = kmem_alloc(XXCTX_SIZE, KM_SLEEP);
bcopy(ctxp->context, newctx->context, XXCTX_SIZE);
} else {
/* point to the shared context */
newctx->context = xsp->ctx_shared;
}
newctx->handle = new_handle1;
newctx->offset = ctxp->offset;
newctx->flags = ctxp->flags;
newctx->len = off - ctxp->offset;
*new_devprivate1 = newctx;
}
/*
* If new_handle2 is not NULL, we are unmapping
* at the beginning of the mapping.
*/
if (new_handle2 != NULL) {
/* Create a new context for the mapping */
newctx = kmem_alloc(sizeof (struct xxctx), KM_SLEEP);
newctx->xsp = xsp;
if (ctxp->flags & MAP_PRIVATE) {
newctx->context = kmem_alloc(XXCTX_SIZE, KM_SLEEP);
bcopy(ctxp->context, newctx->context, XXCTX_SIZE);
} else {
newctx->context = xsp->ctx_shared;
}
newctx->handle = new_handle2;
newctx->offset = off + len;
newctx->flags = ctxp->flags;
newctx->len = ctxp->len - (off + len - ctxp->off);
*new_devprivate2 = newctx;
}
if (xsp->current_ctx == ctxp)
xsp->current_ctx = NULL;
mutex_exit(&xsp->ctx_lock);
if (ctxp->flags & MAP_PRIVATE)
kmem_free(ctxp->context, XXCTX_SIZE);
kmem_free(ctxp, sizeof (struct xxctx));
}
- © 2010, Oracle Corporation and/or its affiliates