JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Oracle Solaris 11.1 Dynamic Tracing Guide     Oracle Solaris 11.1 Information Library
search filter icon
search icon

Document Information

Preface

1.  About DTrace

2.  D Programming Language

3.  Aggregations

4.  Actions and Subroutines

5.  Buffers and Buffering

6.  Output Formatting

7.  Speculative Tracing

8.  dtrace(1M) Utility

9.  Scripting

10.  Options and Tunables

11.  Providers

dtrace Provider

BEGIN Probe

END Probe

ERROR Probe

Stability

lockstat Provider

Overview

Adaptive Lock Probes

Spin Lock Probes

Thread Locks

Readers/Writer Lock Probes

Stability

profile Provider

profile- n probes

tick - n probes

Arguments

Timer Resolution

Probe Creation

Stability

cpc Provider

Probes

Arguments

Probe Availability

Probe Creation

Co-existence With Existing Tools

Examples

user-insts.d

kern-cycles.d

brendan-l2miss.d

brendan-generic-l2miss.d

off_core_event.d

l2miss.d

Stability

fbt Provider

Probes

Probe arguments

entry probes

return probes

Examples

Tail-call Optimization

Assembly Functions

Instruction Set Limitations

x86 Limitations

SPARC Limitations

Breakpoint Interaction

Module Loading

Stability

syscall Provider

Probes

System Call Anachronisms

Subcoded System Calls

New System Calls

Deleted System Calls

Large File System Calls

Private System Calls

Arguments

Stability

sdt Provider

Probes

Examples

Creating SDT Probes

Declaring Probes

Probe Arguments

Stability

mib Provider

Probes

Arguments

Stability

fpuinfo Provider

Probes

Arguments

Stability

pid Provider

Naming pid Probes

Function Boundary Probes

entry Probes

return Probes

Function Offset Probes

Stability

plockstat Provider

Overview

Mutex Probes

Reader/Writer Lock Probes

Stability

fasttrap Provider

Probes

Stability

sysinfo Provider

Probes

Arguments

Example

Stability

vminfo Provider

Probes

Arguments

Example

Stability

proc Provider

Probes

Arguments

lwpsinfo_t

psinfo_t

Examples

exec

start and exit

lwp-start and lwp-exit

signal-send

Stability

sched Provider

Probes

Arguments

cpuinfo_t

Examples

on-cpu and off-cpu

enqueue and dequeue

sleep and wakeup

preempt and remain-cpu

change-pri

tick

cpucaps-sleep and cpucaps-wakeup

Stability

io Provider

Probes

Arguments

bufinfo_t structure

devinfo_t

fileinfo_t

Examples

Stability

Protocols

ip Provider

Probes

Arguments

args[0] - pktinfo_t Structure

args[1] - csinfo_t Structure

args[2] - ipinfo_t Structure

args[3] - ifinfo_t Structure

args[4] - ipv4info_t Structure

args[5] - ipv6info_t Structure

Examples

Packets by host address

Sent size distribution

ipio.d

ipproto.d

Stability

iscsi Provider

Probes

Arguments

Types

Examples

One-liners

iscsiwho.d

iscsixfer.d

nfsv3 Provider

Arguments

Probes

Examples

nfsv3rwsnoop.d

nfsv3ops.d

nfsv3fileio.d

nfsv3rwtime.d

nfsv3io.d

nfsv4 Provider

Arguments

Probes

Examples

nfsv4rwsnoop.d

nfsv4ops.d

nfsv4fileio.d

nfsv4rwtime.d

nfsv4io.d

srp Provider

Probes

Probes Overview

Service up/down Event Probes

Remote Port Login/Logout Event Probes

SRP Command Event Probes

SCSI Command Event Probes

Data Transfer Probes

Types

scsicmd_t

conninfo_t

srp_portinfo_t

srp_logininfo_t

srp_taskinfo_t

xferinfo_t

Examples

service.d

srpwho.d

srpsnoop.d

tcp Provider

Probes

Arguments

pktinfo_t Structure

csinfo_t Structure

ipinfo_t Structure

tcpsinfo_t Structure

tcplsinfo_t Structure

tcpinfo_t Structure

Examples

Connections by Host Address

Connections by TCP Port

Who is Connecting to What

Who Isn't Connecting to What

Packets by Host Address

Packets by Local Port

Sent Size Distribution

tcpstate.d

tcpio.d

Stability

udp Provider

Probes

Arguments

pktinfo_t Structure

csinfo_t Structure

ipinfo_t Structure

udpsinfo_t Structure

udpsinfo_t Structure

Examples

Packets by Host Address

Packets by Local Port

Sent Size Distribution

Stability

12.  User Process Tracing

13.  Statically Defined Tracing for User Applications

14.  Security

15.  Anonymous Tracing

16.  Postmortem Tracing

17.  Performance Considerations

18.  Stability

19.  Translators

20.  Versioning

Index

vminfo Provider

The vminfo provider makes available probes that correspond to the vm kernel statistics. Because these statistics provide the input for system monitoring utilities like vmstat(1M), the vminfo provider enables quick exploration of observed aberrant behavior.

Probes

The vminfo provider makes available probes that correspond to the fields in the vm named kernel statistic: a probe provided by vminfo fires immediately before the corresponding vm value is incremented. To display both the names and the current values of the vm named kernel statistic, use the kstat(1M) command, as shown in the following example:

$ kstat -n vm
module: cpu                             instance: 0
name:   vm                              class:    misc
        anonfree                        13
        anonpgin                        2620
        anonpgout                       13
        as_fault                        12528831
        cow_fault                       2278711
        crtime                          202.10625712
        dfree                           1328740
        execfree                        0
        execpgin                        5541
        ...

The vminfo probes are described in Table 11-22.

Table 11-22 vminfo Probes

anonfree
Probe that fires whenever an unmodified anonymous page is freed as part of paging activity. Anonymous pages are those that are not associated with a file. Memory containing such pages includes heap memory, stack memory, or memory obtained by explicitly mapping zero(7D).
anonpgin
Probe that fires whenever an anonymous page is paged in from a swap device.
anonpgout
Probe that fires whenever a modified anonymous page is paged out to a swap device.
as_fault
Probe that fires whenever a fault is taken on a page and the fault is neither a protection fault nor a copy-on-write fault.
cow_fault
Probe that fires whenever a copy-on-write fault is taken on a page. arg0 contains the number of pages that are created as a result of the copy-on-write.
dfree
Probe that fires whenever a page is freed as a result of paging activity. Whenever dfree fires, exactly one of anonfree, execfree or fsfree will also subsequently fire.
execfree
Probe that fires whenever an unmodified executable page is freed as a result of paging activity.
execpgin
Probe that fires whenever an executable page is paged in from the backing store.
execpgout
Probe that fires whenever a modified executable page is paged out to the backing store. Most paging of executable pages occurs in terms of execfree. execpgout can only fire if an executable page is modified in memory, an uncommon occurrence in most systems.
fsfree
Probe that fires whenever an unmodified file system data page is freed as part of paging activity.
fspgin
Probe that fires whenever a file system page is paged in from the backing store.
fspgout
Probe that fires whenever a modified file system page is paged out to the backing store.
kernel_asflt
Probe that fires whenever a page fault is taken by the kernel on a page in its own address space. Whenever kernel_asflt fires, it will be immediately preceded by a firing of the as_fault probe.
maj_fault
Probe that fires whenever a page fault is taken that results in I/O from a backing store or swap device. Whenever maj_fault fires, it will be immediately preceded by a firing of the pgin probe.
pgfrec
Probe that fires whenever a page is reclaimed off of the free page list.
pgin
Probe that fires whenever a page is paged in from the backing store or from a swap device. This probe differs from maj_fault in that maj_fault only fires when a page is paged in as a result of a page fault. pgin fires every time a page is paged in, regardless of the reason.
pgout
Probe that fires whenever a page is paged out to the backing store or to a swap device.
pgpgin
Probe that fires whenever a page is paged in from the backing store or from a swap device. The only difference between pgpgin and pgin is that pgpgin contains the number of pages paged in as arg0. pgin always contains 1 in arg0.
pgpgout
Probe that fires whenever a page is paged out to the backing store or to a swap device. The only difference between pgpgout and pgout is that pgpgout contains the number of pages paged out as arg0. (pgout always contains 1 in arg0.)
pgrec
Probe that fires whenever a page is reclaimed.
pgrrun
Probe that fires whenever the pager is scheduled.
pgswapin
Probe that fires whenever pages from a swapped-out process are swapped in. The number of pages swapped in is contained in arg0.
pgswapout
Probe that fires whenever pages are swapped out as part of swapping out a process. The number of pages swapped out is contained in arg0.
prot_fault
Probe that fires whenever a page fault is taken due to a protection violation.
rev
Probe that fires whenever the page daemon begins a new revolution through all pages.
scan
Probe that fires whenever the page daemon examines a page.
softlock
Probe that fires whenever a page is faulted as a part of placing a software lock on the page.
swapin
Probe that fires whenever a swapped-out process is swapped back in.
swapout
Probe that fires whenever a process is swapped out.
zfod
Probe that fires whenever a zero-filled page is created on demand.

Arguments

arg0
The value by which the statistic is to be incremented. For most probes, this argument is always 1, but for some it may take other values; these probes are noted in Table 24–1.
arg1
A pointer to the current value of the statistic to be incremented. This value is a 64–bit quantity that will be incremented by the value in arg0. Dereferencing this pointer allows consumers to determine the current count of the statistic corresponding to the probe.

Example

Examine the following output from vmstat(1M):

kthr      memory            page            disk          faults      cpu
 r b w   swap  free  re  mf pi po fr de sr cd s0 - -   in   sy   cs us sy id
 0 1 0 1341844 836720 26 311 1644 0 0 0  0 216 0  0  0  797  817  697  9 10 81
 0 1 0 1341344 835300 238 934 1576 0 0 0 0 194 0  0  0  750 2795  791  7 14 79
 0 1 0 1340764 833668 24 165 1149 0 0 0  0 133 0  0  0  637  813  547  5  4 91
 0 1 0 1340420 833024 24 394 1002 0 0 0  0 130 0  0  0  621 2284  653 14  7 79
 0 1 0 1340068 831520 14 202 380 0 0  0  0 59  0  0  0  482 5688 1434 25  7 68

The pi column in the above output denotes the number of pages paged in. The vminfo provider enables you to learn more about the source of these page-ins, as shown in the following example:

# dtrace -n pgin'{@[execname] = count()}'
dtrace: description 'pgin' matched 1 probe
^C
  xterm                                                             1
  ksh                                                               1
  ls                                                                2
  lpstat                                                            7
  sh                                                               17
  soffice                                                          39
  javaldx                                                         103
  soffice.bin                                                    3065

The output shows that a process associated with the StarOffice software, soffice.bin, is responsible for most of the page-ins. To get a better picture of soffice.bin in terms of virtual memory behavior, you could enable all vminfo probes. The following example runs dtrace(1M) while launching the StarOffice software:

# dtrace -P vminfo'/execname == "soffice.bin"/{@[probename] = count()}'
dtrace: description 'vminfo' matched 42 probes
^C

  kernel_asflt                                                      1
  fspgin                                                           10
  pgout                                                            16
  execfree                                                         16
  execpgout                                                        16
  fsfree                                                           16
  fspgout                                                          16
  anonfree                                                         16
  anonpgout                                                        16
  pgpgout                                                          16
  dfree                                                            16
  execpgin                                                         80
  prot_fault                                                       85
  maj_fault                                                        88
  pgin                                                             90
  pgpgin                                                           90
  cow_fault                                                       859
  zfod                                                           1619
  pgfrec                                                         8811
  pgrec                                                          8827
  as_fault                                                       9495

The following example script provides more information about the virtual memory behavior of the StarOffice software during its startup:

vminfo:::maj_fault,
vminfo:::zfod,
vminfo:::as_fault
/execname == "soffice.bin" && start == 0/
{
        /*
         * This is the first time that a vminfo probe has been hit; record
         * our initial timestamp.
         */
        start = timestamp;
}

vminfo:::maj_fault,
vminfo:::zfod,
vminfo:::as_fault
/execname == "soffice.bin"/
{
        /*
         * Aggregate on the probename, and lquantize() the number of seconds
         * since our initial timestamp.  (There are 1,000,000,000 nanoseconds
         * in a second.)  We assume that the script will be terminated before
         * 60 seconds elapses.
         */
        @[probename] =
            lquantize((timestamp - start) / 1000000000, 0, 60);
}

Run the script while again starting the StarOffice software. Then, create a new drawing, create a new presentation, and then close all files and quit the application. Press Control-C in the shell running the D script. The results provide a view of some virtual memory behavior over time:

# dtrace -s ./soffice.d
dtrace: script './soffice.d' matched 10 probes
^C

 maj_fault                                         
           value  ------------- Distribution ------------- count    
               7 |                                         0        
               8 |@@@@@@@@@                                88       
               9 |@@@@@@@@@@@@@@@@@@@@                     194      
              10 |@                                        18       
              11 |                                         0        
              12 |                                         0        
              13 |                                         2        
              14 |                                         0        
              15 |                                         1        
              16 |@@@@@@@@                                 82       
              17 |                                         0        
              18 |                                         0        
              19 |                                         2        
              20 |                                         0        

  zfod                                              
           value  ------------- Distribution ------------- count    
             < 0 |                                         0        
               0 |@@@@@@@                                  525      
               1 |@@@@@@@@                                 605      
               2 |@@                                       208      
               3 |@@@                                      280      
               4 |                                         4        
               5 |                                         0        
               6 |                                         0        
               7 |                                         0        
               8 |                                         44       
               9 |@@                                       161      
              10 |                                         2        
              11 |                                         0        
              12 |                                         0        
              13 |                                         4        
              14 |                                         0        
              15 |                                         29       
              16 |@@@@@@@@@@@@@@                           1048     
              17 |                                         24       
              18 |                                         0        
              19 |                                         0        
              20 |                                         1        
              21 |                                         0        
              22 |                                         3        
              23 |                                         0        

  as_fault                                          
           value  ------------- Distribution ------------- count    
             < 0 |                                         0        
               0 |@@@@@@@@@@@@@                            4139     
               1 |@@@@@@@                                  2249     
               2 |@@@@@@@                                  2402     
               3 |@                                        594      
               4 |                                         56       
               5 |                                         0        
               6 |                                         0        
               7 |                                         0        
               8 |                                         189      
               9 |@@                                       929      
              10 |                                         39       
              11 |                                         0        
              12 |                                         0        
              13 |                                         6        
              14 |                                         0        
              15 |                                         297      
              16 |@@@@                                     1349     
              17 |                                         24       
              18 |                                         0        
              19 |                                         21       
              20 |                                         1        
              21 |                                         0        
              22 |                                         92       
              23 |                                         0

The output shows some StarOffice behavior with respect to the virtual memory system. For example, the maj_fault probe didn't fire until a new instance of the application was started. As you would hope, a “warm start” of StarOffice did not result in new major faults. The as_fault output shows an initial burst of activity, latency while the user located the menu to create a new drawing, another period of idleness, and a final burst of activity when the user clicked on a new presentation. The zfod output shows that creating the new presentation induced significant pressure for zero-filled pages, but only for a short period of time.

The next iteration of DTrace investigation in this example would depend on the direction you want to explore. If you want to understand the source of the demand for zero-filled pages, you could aggregate on ustack in a zfod enabling. You might want to establish a threshold for zero-filled pages and use the stop destructive action to stop the offending process when the threshold is exceeded. This approach would enable you to use more traditional debugging tools like truss(1) or mdb(1). The vminfo provider enables you to associate statistics seen in the output of conventional tools like vmstat(1M) with the applications that are inducing the systemic behavior.

Stability

The vminfo provider uses DTrace's stability mechanism to describe its stabilities, as shown in the following table. For more information about the stability mechanism, see Chapter 18, Stability.

Element
Name stability
Data stability
Dependency class
Provider
Evolving
Evolving
ISA
Module
Private
Private
Unknown
Function
Private
Private
Unknown
Name
Evolving
Evolving
ISA
Arguments
Private
Private
ISA