JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Oracle Solaris Dynamic Tracing Guide     Oracle Solaris 11 Information Library
search filter icon
search icon

Document Information

Preface

1.  About DTrace

2.  D Programming Language

3.  Aggregations

4.  Actions and Subroutines

5.  Buffers and Buffering

6.  Output Formatting

7.  Speculative Tracing

8.  dtrace(1M) Utility

9.  Scripting

10.  Options and Tunables

11.  Providers

dtrace Provider

BEGIN Probe

END Probe

ERROR Probe

Stability

lockstat Provider

Overview

Adaptive Lock Probes

Spin Lock Probes

Thread Locks

Readers/Writer Lock Probes

Stability

profile Provider

profile- n probes

tick- n probes

Arguments

Timer Resolution

Probe Creation

Stability

cpc Provider

Probes

Arguments

Probe Availability

Probe Creation

Co-existence With Existing Tools

Examples

user-insts.d

kern-cycles.d

brendan-l2miss.d

brendan-generic-l2miss.d

Stability

fbt Provider

Probes

Probe arguments

entry probes

return probes

Examples

Tail-call Optimization

Assembly Functions

Instruction Set Limitations

x86 Limitations

SPARC Limitations

Breakpoint Interaction

Module Loading

Stability

syscall Provider

Probes

System Call Anachronisms

Subcoded System Calls

New System Calls

Deleted System Calls

Large File System Calls

Private System Calls

Arguments

Stability

sdt Provider

Probes

Examples

Creating SDT Probes

Declaring Probes

Probe Arguments

Stability

mib Provider

Probes

Arguments

Stability

fpuinfo Provider

Probes

Arguments

Stability

pid Provider

Naming pid Probes

Function Boundary Probes

entry Probes

return Probes

Function Offset Probes

Stability

plockstat Provider

Overview

Mutex Probes

Reader/Writer Lock Probes

Stability

fasttrap Provider

Probes

Stability

sysinfo Provider

Probes

Arguments

Example

Stability

vminfo Provider

Probes

Arguments

Example

Stability

proc Provider

Probes

Arguments

lwpsinfo_t

psinfo_t

Examples

exec

start and exit

lwp-start and lwp-exit

signal-send

Stability

sched Provider

Probes

Arguments

cpuinfo_t

Examples

on-cpu and off-cpu

enqueue and dequeue

sleep and wakeup

preempt and remain-cpu

change-pri

tick

cpucaps-sleep and cpucaps-wakeup

Stability

io Provider

Probes

Arguments

bufinfo_t structure

devinfo_t

fileinfo_t

Examples

Stability

Protocols

ip Provider

Probes

Arguments

args[0] - pktinfo_t Structure

args[1] - csinfo_t Structure

args[2] - ipinfo_t Structure

args[3] - ifinfo_t Structure

args[4] - ipv4info_t Structure

args[5] - ipv6info_t Structure

Examples

Packets by host address

Sent size distribution

ipio.d

ipproto.d

Stability

iscsi Provider

Probes

Arguments

Types

Examples

One-liners

iscsiwho.d

iscsixfer.d

nfsv3 Provider

Arguments

Probes

Examples

nfsv3rwsnoop.d

nfsv3ops.d

nfsv3fileio.d

nfsv3rwtime.d

nfsv3io.d

nfsv4 Provider

Arguments

Probes

Examples

nfsv4rwsnoop.d

nfsv4ops.d

nfsv4fileio.d

nfsv4rwtime.d

nfsv4io.d

srp Provider

Probes

Probes Overview

Service up/down Event Probes

Remote Port Login/Logout Event Probes

SRP Command Event Probes

SCSI Command Event Probes

Data Transfer Probes

Types

scsicmd_t

conninfo_t

srp_portinfo_t

srp_logininfo_t

srp_taskinfo_t

xferinfo_t

Examples

service.d

srpwho.d

srpsnoop.d

tcp Provider

Probes

Arguments

pktinfo_t Structure

csinfo_t Structure

ipinfo_t Structure

tcpsinfo_t Structure

tcplsinfo_t Structure

tcpinfo_t Structure

Examples

Connections by Host Address

Connections by TCP Port

Who is Connecting to What

Who Isn't Connecting to What

Packets by Host Address

Packets by Local Port

Sent Size Distribution

tcpstate.d

tcpio.d

tcp Stability

udp Provider

Probes

Arguments

pktinfo_t Structure

csinfo_t Structure

ipinfo_t Structure

udpsinfo_t Structure

udpsinfo_t Structure

Examples

Packets by Host Address

Packets by Local Port

Sent Size Distribution

udp Stability

12.  User Process Tracing

13.  Statically Defined Tracing for User Applications

14.  Security

15.  Anonymous Tracing

16.  Postmortem Tracing

17.  Performance Considerations

18.  Stability

19.  Translators

20.  Versioning

sdt Provider

The Statically Defined Tracing (SDT) provider creates probes at sites that a software programmer has formally designated. The SDT mechanism allows programmers to consciously choose locations of interest to users of DTrace and to convey some semantic knowledge about each location through the probe name. The Oracle Solaris kernel has defined a handful of SDT probes, and will likely add more over time. DTrace also provides a mechanism for user application developers to define static probes, described in Chapter 13, Statically Defined Tracing for User Applications.

Probes

The SDT probes defined by the Oracle Solaris kernel are listed in the following table. The name stability and data stability of these probes are both Private because their description here thus reflects the kernel's implementation and should not be inferred to be an interface commitment. For more information about the DTrace stability mechanism, see Chapter 18, Stability.

Table 11-9 SDT Probes

Probe name
Description
arg0
callout-start
Probe that fires immediately before executing a callout (see <sys/callo.h>). Callouts are executed by periodic system clock, and represent the implementation for timeout(9F)
Pointer to the callout_t (see <sys/callo.h>) corresponding to the callout to be executed.
callout-end
Probe that fires immediately after executing a callout (see <sys/callo.h>).
Pointer to the callout_t (see <sys/callo.h>) corresponding to the callout just executed.
interrupt-start
Probe that fires immediately before calling into a device's interrupt handler.
Pointer to the dev_info structure (see <sys/ddi_impldefs.h>) corresponding to the interrupting device.
interrupt-complete
Probe that fires immediately after returning from a device's interrupt handler.
Pointer to dev_info structure (see <sys/ddi_impldefs.h>) corresponding to the interrupting device.

Examples

The following example is a script to observe callout behavior on a per-second basis:

#pragma D option quiet

sdt:::callout-start
{
        @callouts[((callout_t *)arg0)->c_func] = count();
}

tick-1sec
{
        printa("%40a %10@d\n", @callouts);
        clear(@callouts);
}

Running this example reveals the frequent users of timeout(9F) in the system, as shown in the following output:

# dtrace -s ./callout.d
                                    FUNC      COUNT
                            TS`ts_update          1
              uhci`uhci_cmd_timeout_hdlr          3
                          genunix`setrun          5
                     genunix`schedpaging          5
                         ata`ghd_timeout         10
 uhci`uhci_handle_root_hub_status_change        309

                                    FUNC      COUNT
              ip`tcp_time_wait_collector          1
                            TS`ts_update          1
              uhci`uhci_cmd_timeout_hdlr          3
                     genunix`schedpaging          4
                          genunix`setrun          8
                         ata`ghd_timeout         10
 uhci`uhci_handle_root_hub_status_change        300

                                    FUNC      COUNT
              ip`tcp_time_wait_collector          0
                        iprb`mii_portmon          1
                            TS`ts_update          1
              uhci`uhci_cmd_timeout_hdlr          3
                     genunix`schedpaging          4
                          genunix`setrun          7
                         ata`ghd_timeout         10
 uhci`uhci_handle_root_hub_status_change        300

The timeout(9F) interface only produces a single timer expiration. Consumers of timeout requiring interval timer functionality typically reinstall their timeout from their timeout handler. The following example shows this behavior:

#pragma D option quiet

sdt:::callout-start
{
        self->callout = ((callout_t *)arg0)->c_func;
}

fbt::timeout:entry
/self->callout && arg2 <= 100/
{
        /*
         * In this case, we are most interested in interval timeout(9F)s that
         * are short.  We therefore do a linear quantization from 0 ticks to
         * 100 ticks.  The system clock's frequency - set by the variable
         * "hz" - defaults to 100, so 100 system clock ticks is one second. 
         */
        @callout[self->callout] = lquantize(arg2, 0, 100);
}

sdt:::callout-end
{
        self->callout = NULL;
}

END
{
        printa("%a\n%@d\n\n", @callout);
}

Running this script and waiting several seconds before typing Control-C results in output similar to the following example:

# dtrace -s ./interval.d
^C
genunix`schedpaging

           value  ------------- Distribution ------------- count    
              24 |                                         0        
              25 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 20       
              26 |                                         0        

ata`ghd_timeout

           value  ------------- Distribution ------------- count    
               9 |                                         0        
              10 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 51       
              11 |                                         0        

uhci`uhci_handle_root_hub_status_change

           value  ------------- Distribution ------------- count    
               0 |                                         0        
               1 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1515     
               2 |                                         0

The output shows that uhci_handle_root_hub_status_change in the uhci(7D) driver represents the shortest interval timer on the system: it is called every system clock tick.

The interrupt-start probe can be used to understand interrupt activity. The following example shows how to quantize the time spent executing an interrupt handler by driver name:

interrupt-start
{
        self->ts = vtimestamp;
}

interrupt-complete
/self->ts/
{
        this->devi = (struct dev_info *)arg0;
        @[stringof(`devnamesp[this->devi->devi_major].dn_name),
            this->devi->devi_instance] = quantize(vtimestamp - self->ts);
}

Running this script results in output similar to the following example:

# dtrace -s ./intr.d
dtrace: script './intr.d' matched 2 probes
^C
 isp                                                       0
           value  ------------- Distribution ------------- count    
            8192 |                                         0        
           16384 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1        
           32768 |                                         0        

  pcf8584                                                   0
           value  ------------- Distribution ------------- count    
              64 |                                         0        
             128 |                                         2        
             256 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@         157      
             512 |@@@@@@                                   31       
            1024 |                                         3        
            2048 |                                         0        

  pcf8584                                                   1
           value  ------------- Distribution ------------- count    
            2048 |                                         0        
            4096 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@          154      
            8192 |@@@@@@@                                  37       
           16384 |                                         2        
           32768 |                                         0        

  qlc                                                       0
           value  ------------- Distribution ------------- count    
           16384 |                                         0        
           32768 |@@                                       9        
           65536 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@      126      
          131072 |@                                        5        
          262144 |                                         2        
          524288 |                                         0        

  hme                                                       0
           value  ------------- Distribution ------------- count    
            1024 |                                         0        
            2048 |                                         6        
            4096 |                                         2        
            8192 |@@@@                                     89       
           16384 |@@@@@@@@@@@@@                            262      
           32768 |@                                        37       
           65536 |@@@@@@@                                  139      
          131072 |@@@@@@@@                                 161      
          262144 |@@@                                      73       
          524288 |                                         4        
         1048576 |                                         0        
         2097152 |                                         1        
         4194304 |                                         0        

  ohci                                                      0
           value  ------------- Distribution ------------- count    
            8192 |                                         0        
           16384 |                                         3        
           32768 |                                         1        
           65536 |@@@                                      143      
          131072 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@     1368     
          262144 |                                         0 

Creating SDT Probes

If you are a device driver developer, you might be interested in creating your own SDT probes in your Oracle Solaris driver. The disabled probe effect of SDT is essentially the cost of several no-operation machine instructions. You are therefore encouraged to add SDT probes to your device drivers as needed. Unless these probes negatively affect performance, you can leave them in your shipping code.

Declaring Probes

SDT probes are declared using the DTRACE_PROBE, DTRACE_PROBE1, DTRACE_PROBE2, DTRACE_PROBE3 and DTRACE_PROBE4 macros from <sys/sdt.h>. The module name and function name of an SDT-based probe corresponds to the kernel module and function of the probe. The name of the probe depends on the name given in the DTRACE_PROBEn macro. If the name contains no two consecutive underbars (_), the name of the probe is as written in the macro. If the name contains any two consecutive underbars, the probe name converts the consecutive underbars to a single dash (-). For example, if a DTRACE_PROBE macro specifies transaction_start, the SDT probe will be named transaction-start. This substitution allows C code to provide macro names that are not valid C identifiers without specifying a string.

DTrace includes the kernel module name and function name as part of the tuple identifying a probe, so you do not need to include this information in the probe name to prevent name space collisions. You can use the command dtrace -l -P sdt -m module on your driver module to list the probes you have installed and the full names that will be seen by users of DTrace.

Probe Arguments

The arguments for each SDT probe are the arguments specified in the corresponding DTRACE_PROBEn macro reference. The number of arguments depends on which macro was used to create the probe: DTRACE_PROBE1 specifies one argument, DTRACE_PROBE2 specifies two arguments, and so on. When declaring your SDT probes, you can minimize their disabled probe effect by not dereferencing pointers and not loading from global variables in the probe arguments. Both pointer dereferencing and global variable loading may be done safely in D actions that enable probes, so DTrace users can request these actions only when they are needed.

Stability

The SDT provider uses DTrace's stability mechanism to describe its stabilities, as shown in the following table. For more information about the stability mechanism, see Chapter 18, Stability.

Element
Name stability
Data stability
Dependency class
Provider
Evolving
Evolving
ISA
Module
Private
Private
Unknown
Function
Private
Private
Unknown
Nam
Private
Private
ISA
Arguments
Private
Private
ISA