Go to main content

Oracle® Solaris 11.4 DTrace (Dynamic Tracing) Guide

Exit Print View

Updated: September 2020
 
 

proc Provider

The proc provider include probes pertaining to the following activities: process creation and termination, light weight process (LWP) creation and termination, executing new program images, and sending and handling signals.

proc Probes

The following table describes the proc probes.

Table 43  List of proc Probes
Probe
Description
create
Fires when a process is created using fork, forkall, fork1, or vfork. psinfo_t corresponding to the new child process is pointed to by args[0]. You can distinguish vfork from the other fork variants by checking for PR_VFORKP in the pr_flag member of the forking thread's lwpsinfo_t. You can distinguish fork1 from forkall by examining the pr_nlwp members of both the parent process's psinfo_t (curpsinfo) and the child process's psinfo_t (args[0]). Because the create probe only fires after the process has been successfully created, and because LWP creation is part of creating a process, lwp-create will fire for any LWPs created at process creation time before the create probe fires for the new process.
exec
Fires whenever a process loads a new process image with a variant of the exec system call: exec, execle, execlp, execv, execve, and execvp. The exec probe fires before the process image is loaded. Process variables like execname and curpsinfo therefore contain the process state before the image is loaded. Some time after the exec probe fires, either the exec-failure probe or the exec-success probe will subsequently fire in the same thread. The path of the new process image is pointed to by args[0].
exec-failure
Fires when an exec variant has failed. The exec-failure probe fires only after the exec probe has fired in the same thread. The errno value is provided in args[0].
exec-success
Fires when an exec variant has succeeded. Like the exec-failure probe, the exec-success probe fires only after the exec probe has fired in the same thread. By the time the exec-success probe fires, process variables like execname and curpsinfo contain the process state after the new process image has been loaded.
exit
Fires when the current process is exiting. The reason for exit, which is expressed as one of the SIGCHLD siginfo.h(3HEAD) codes, is contained in args[0].
fault
Fires when a thread experiences a machine fault. The fault code, as defined in proc, is in args[0]. The siginfo structure corresponding to the fault is pointed to by args[1]. Only those faults that induce a signal can trigger the fault probe.
lwp-create
Fires when an LWP is created, typically as a result of thr_create. The lwpsinfo_t corresponding to the new thread is pointed to by args[0]. The psinfo_t of the process containing the thread is pointed to by args[1].
lwp-exit
Fires when an LWP is exiting, due either to a signal or to an explicit call to thr_exit.
lwp-start
Fires within the context of a newly created LWP. The lwp-start probe will fire before any user-level instructions are executed. If the LWP is the first LWP in the process, the start probe will fire, followed by lwp-start.
signal-clear
Probes that fires when a pending signal is cleared because the target thread was waiting for the signal in sigwait, sigwaitinfo, or sigtimedwait. Under these conditions, the pending signal is cleared and the signal number is returned to the caller. The signal number is in args[0]. signal-clear fires in the context of the formerly waiting thread.
signal-discard
Fires when a signal is sent to a single-threaded process, and the signal is both unblocked and ignored by the process. Under these conditions, the signal is discarded on generation. The lwpsinfo_t and psinfo_t of the target process and thread are in args[0] and args[1], respectively. The signal number is in args[2].
signal-handle
Fires immediately before a thread handles a signal. The signal-handle probe fires in the context of the thread that will handle the signal. The signal number is in args[0]. A pointer to the siginfo_t structure that corresponds to the signal is in args[1]. The address of the signal handler in the process is in args[2].
signal-send
Fires when a signal is sent to a thread or process. The signal-send probe fires in the context of the sending process and thread. The lwpsinfo_t and psinfo_t of the receiving process and thread are in args[0] and args[1], respectively. The signal number is in args[2]. signal-send is always followed by signal-handle or signal-clear in the receiving process and thread.
start
Fires in the context of a newly created process. The start probe will fire before any user-level instructions are executed in the process.

proc Probe Arguments

The argument types for the proc probes are listed in Figure 44, Table 44, List of proc Probe Arguments. The arguments are described in Figure 43, Table 43, List of proc Probes.

Table 44  List of proc Probe Arguments
Name of Probe
args[0]
args[1]
args[2]
create
psinfo_t *
-
-
exec
char *
-
-
exec-failure
int
-
-
exec-success
-
-
-
exit
int
-
-
fault
int
siginfo_t *
-
lwp-create
lwpsinfo_t *
psinfo_t *
-
lwp-exit
-
-
-
lwp-start
-
-
-
signal-clear
int
-
-
signal-discard
lwpsinfo_t *
psinfo_t *
int
signal-handle
int
siginfo_t *
void (*)(void)
signal-send
lwpsinfo_t *
psinfo_t *
int
start
-
-
-

proc lwpsinfo_t Structure

Several proc probes have arguments of type lwpsinfo_t. The definition of the lwpsinfo_t structure as available to DTrace consumers is as follows:

typedef struct lwpsinfo {
        int pr_flag;              /* flags; see below */
        id_t pr_lwpid;            /* LWP id */
        uintptr_t pr_addr;        /* internal address of thread */
        uintptr_t pr_wchan;       /* wait addr for sleeping thread */
        char pr_stype;            /* synchronization event type */
        char pr_state;            /* numeric thread state */
        char pr_sname;            /* printable character for pr_state */
        char pr_nice;             /* nice for cpu usage */
        short pr_syscall;         /* system call number (if in syscall) */
        int pr_pri;               /* priority, high value = high priority */
        char pr_clname[PRCLSZ];   /* scheduling class name */
        processorid_t pr_onpro;   /* processor which last ran this thread */
        processorid_t pr_bindpro; /* processor to which thread is bound */
        psetid_t pr_bindpset;     /* processor set to which thread is bound */
} lwpsinfo_t;

The pr_flag field is a bit-mask holding flags describing the process. These flags and their meanings are described in Figure 45, Table 45, proc pr_flag Values.

Table 45  proc pr_flag Values
Flags
Descriptions
PR_ISSYS
The process is a system process.
PR_VFORKP
The process is the parent of a vfork child.
PR_FORK
The process has its inherit-on-fork mode set.
PR_RLC
The process has its run-on-last-close mode set.
PR_KLC
The process has its kill-on-last-close mode set.
PR_ASYNC
The process has its asynchronous-stop mode set.
PR_MSACCT
The process has microstate accounting enabled.
PR_MSFORK
The process microstate accounting is inherited on fork.
PR_BPTADJ
The process has its breakpoint adjustment mode set.
PR_PTRACE
The process has its ptrace compatibility mode set.
PR_STOPPED
The thread is an LWP that is stopped.
PR_ISTOP
The thread is an LWP stopped on an event of interest.
PR_DSTOP
The thread is an LWP that has a stop directive in effect.
PR_STEP
The thread is an LWP that has a single-step directive in effect.
PR_ASLEEP
The thread is an LWP in an interruptible sleep within a system call.
PR_DETACH
The thread is a detached LWP. See the pthread_create(3C) and pthread_join(3C) man pages.
PR_DAEMON
The thread is a daemon LWP. See the pthread_create() function.
PR_AGENT
The thread is the agent LWP for the process.
PR_IDLE
The thread is the idle thread for a CPU. Idle threads only run on a CPU when the run queues for the CPU are empty.

The pr_addr field is the address of a private, in-kernel data structure representing the thread. While the data structure is private, the pr_addr field may be used as a token unique to a thread for the thread's lifetime.

The pr_wchan field is set when the thread is sleeping on a synchronization object. The meaning of the pr_wchan field is private to the kernel implementation, but the field may be used as a token unique to the synchronization object.

The pr_stype field is set when the thread is sleeping on a synchronization object. The following table describes the possible values for the pr_stype field.

Table 46  proc pr_stype Values
Value
Description
SOBJ_MUTEX
Kernel mutex synchronization object. Used to serialize access to shared data regions in the kernel. For more information about kernel mutex synchronization objects, see lockstat Stability and the mutex_init(9F) man page..
SOBJ_RWLOCK
Kernel readers/writer synchronization object. Used to synchronize access to shared objects in the kernel that can allow multiple concurrent readers or a single writer. For more information about kernel readers/writer synchronization objects, see lockstat Stability and the rwlock(9F) man page..
SOBJ_CV
Condition variable synchronization object. A condition variable is designed to wait indefinitely until some condition becomes true. Condition variables are typically used to synchronize for reasons other than access to a shared data region, and are the mechanism generally used when a process performs a program-directed indefinite wait. For example, blocking in poll, pause, wait, and the like.
SOBJ_SEMA
Semaphore synchronization object. A general-purpose synchronization object that – like condition variable objects – does not track a notion of ownership. Because ownership is required to implement priority inheritance in the Oracle Solaris kernel, the lack of ownership inherent in semaphore objects inhibits their widespread use. See the semaphore(9F) man page for details.
SOBJ_USER
A user-level synchronization object. All blocking on user-level synchronization objects is handled with SOBJ_USER synchronization objects. User-level synchronization objects include those created with mutex_init(), sema_init(), rwlock_init(), cond_init() and their POSIX equivalents.
SOBJ_USER_PI
A user-level synchronization object that implements priority inheritance. Some user-level synchronization objects that track ownership additionally allow for priority inheritance. For example, mutex objects created with pthread_mutex_init() may be made to inherit priority using pthread_mutexattr_setprotocol().
SOBJ_SHUTTLE
A shuttle synchronization object. Shuttle objects are used to implement doors. See door_create(3DOOR)() for more information.

The pr_state field is set to one of the values in Figure 47, Table 47, proc pr_state Values. The pr_sname field is set to a corresponding character shown in parentheses in the same table.

Table 47  proc pr_state Values
Value
Description
SSLEEP (S)
The thread is sleeping. The sched:::sleep probe will fire immediately before a thread's state is transitioned to SSLEEP.
SRUN (R)
The thread is runnable, but is not currently running. The sched:::enqueue probe will fire immediately before a thread's state is transitioned to SRUN.
SZOMB (Z)
The thread is a zombie LWP.
SSTOP (T)
The thread is stopped, either due to an explicit proc directive or some other stopping mechanism.
SIDL (I)
The thread is an intermediate state during process creation.
SONPROC (O)
The thread is running on a CPU. The sched:::on-cpu probe will fire in the context of the SONPROC thread a short time after the thread's state is transitioned to SONPROC.
SWAIT (W)
The thread is waiting on wait queue. The sched:::cpucaps-sleep probe will fire immediately before a thread state is transitioned to SWAIT.

proc psinfo_t Structure

Several proc probes have an argument of type psinfo_t, a structure that is documented in proc. The definition of the psinfo_t structure as available to DTrace consumers is as follows:

typedef struct psinfo {
        int     pr_nlwp;            /* number of active lwps in the process */
        pid_t   pr_pid;             /* unique process id */
        pid_t   pr_ppid;            /* process id of parent */
        pid_t   pr_pgid;            /* pid of process group leader */
        pid_t   pr_sid;             /* session id */
        uid_t   pr_uid;             /* real user id */
        uid_t   pr_euid;            /* effective user id */
        gid_t   pr_gid;             /* real group id */
        gid_t   pr_egid;            /* effective group id */
        uintptr_t pr_addr;          /* address of process */
        dev_t   pr_ttydev;          /* controlling tty device (or PRNODEV) */
        timestruc_t pr_start;       /* process start time, from the epoch */
        char    pr_fname[PRFNSZ];   /* name of exec'd file */
        char    pr_psargs[PRARGSZ]; /* initial characters of arg list */
        int     pr_argc;            /* initial argument count */
        uintptr_t pr_argv;          /* address of initial argument vector */
        uintptr_t pr_envp;          /* address of initial environment vector */
        char    pr_dmodel;          /* data model of the process */
        taskid_t pr_taskid;         /* task id */
        projid_t pr_projid;         /* project id */
        poolid_t pr_poolid;         /* pool id */
        zoneid_t pr_zoneid;         /* zone id */
} psinfo_t;

The pr_dmodel field is set to either PR_MODEL_ILP32, denoting a 32-bit process, or PR_MODEL_LP64, denoting a 64-bit process.

Using the proc Provider

Using the proc exec Probe

You can use the exec probe to determine which programs are being executed, and by whom, as shown in the following example:

#pragma D option quiet

proc:::exec
{
        self->parent = execname;
}

proc:::exec-success
/self->parent != NULL/
{
        @[self->parent, execname] = count();
        self->parent = NULL;
}

proc:::exec-failure
/self->parent != NULL/
{
        self->parent = NULL;
}

END
{
        printf("%-20s %-20s %s\n", "WHO", "WHAT", "COUNT");
        printa("%-20s %-20s %@d\n", @);
}

Running the example script for a short period of time on a build physical machine results in output similar to the following example:

# dtrace -s ./whoexec.d
^C
WHO                  WHAT                 COUNT
make.bin             yacc                 1
tcsh                 make                 1
make.bin             spec2map             1
sh                   grep                 1
lint                 lint2                1
sh                   lint                 1
sh                   ln                   1
cc                   ld                   1
make.bin             cc                   1
lint                 lint1                1
sh                   lex                  1
make.bin             mv                   2
sh                   sh                   3
sh                   make                 3
sh                   sed                  4
sh                   tr                   4
make                 make.bin             4
sh                   install.bin          5
sh                   rm                   6
cc                   ir2hf                33
cc                   ube                  33
sh                   date                 34
sh                   mcs                  34
cc                   acomp                34
sh                   cc                   34
sh                   basename             34
basename             expr                 34
make.bin             sh                   87

Using proc start and proc exit

If you want to know how long programs are running from creation to termination, you can enable the start and exit probes, as shown in the following example:

proc:::start
{
        self->start = timestamp;
}

proc:::exit
/self->start/
{
        @[execname] = quantize(timestamp - self->start);
        self->start = 0;
}

Running the example script on the build server for several seconds results in output similar to the following example:

# dtrace -s ./progtime.d
dtrace: script './progtime.d' matched 2 probes
^C

  ir2hf                                             
           value  ------------- Distribution ------------- count
         4194304 |                                         0        
         8388608 |@                                        1        
        16777216 |@@@@@@@@@@@@@@@@                         14       
        33554432 |@@@@@@@@@@                               9        
        67108864 |@@@                                      3        
       134217728 |@                                        1        
       268435456 |@@@@                                     4        
       536870912 |@                                        1        
      1073741824 |                                         0        

  ube                                               
           value  ------------- Distribution ------------- count
        16777216 |                                         0        
        33554432 |@@@@@@@                                  6        
        67108864 |@@@                                      3        
       134217728 |@@                                       2        
       268435456 |@@@@                                     4        
       536870912 |@@@@@@@@@@@@                             10       
      1073741824 |@@@@@@@                                  6        
      2147483648 |@@                                       2        
      4294967296 |                                         0        

  acomp                                             
           value  ------------- Distribution ------------- count
         8388608 |                                         0        
        16777216 |@@                                       2        
        33554432 |                                         0        
        67108864 |@                                        1        
       134217728 |@@@                                      3        
       268435456 |                                         0        
       536870912 |@@@@@                                    5        
      1073741824 |@@@@@@@@@@@@@@@@@@@@@@@@@                22       
      2147483648 |@                                        1        
      4294967296 |                                         0        

  cc                                                
           value  ------------- Distribution ------------- count
        33554432 |                                         0        
        67108864 |@@@                                      3        
       134217728 |@                                        1        
       268435456 |                                         0        
       536870912 |@@@@                                     4        
      1073741824 |@@@@@@@@@@@@@@                           13       
      2147483648 |@@@@@@@@@@@@                             11       
      4294967296 |@@@                                      3        
      8589934592 |                                         0        

  sh                                                
           value  ------------- Distribution ------------- count
          262144 |                                         0        
          524288 |@                                        5        
         1048576 |@@@@@@@                                  29       
         2097152 |                                         0        
         4194304 |                                         0        
         8388608 |@@@                                      12       
        16777216 |@@                                       9        
        33554432 |@@                                       9        
        67108864 |@@                                       8        
       134217728 |@                                        7        
       268435456 |@@@@@                                    20       
       536870912 |@@@@@@                                   26       
      1073741824 |@@@                                      14       
      2147483648 |@@                                       11       
      4294967296 |                                         3        
      8589934592 |                                         1        
     17179869184 |                                         0        

  make.bin                                          
           value  ------------- Distribution ------------- count
        16777216 |                                         0        
        33554432 |@                                        1        
        67108864 |@                                        1        
       134217728 |@@                                       2        
       268435456 |                                         0        
       536870912 |@@                                       2        
      1073741824 |@@@@@@@@@                                9        
      2147483648 |@@@@@@@@@@@@@@@                          14       
      4294967296 |@@@@@@                                   6        
      8589934592 |@@                                       2        
     17179869184 |                                         0

Using proc lwp-start and proc lwp-exit

Instead of knowing the amount of time that a particular process takes to run, you might want to know how long individual threads take to run. The following example shows how to use the lwp-start and lwp-exit probes for this purpose:

proc:::lwp-start
/tid != 1/
{
        self->start = timestamp;
}

proc:::lwp-exit
/self->start/
{
        @[execname] = quantize(timestamp - self->start);
        self->start = 0;
}

Running the example script on an NFS and calendar server results in output similar to the following example:

# dtrace -s ./lwptime.d
dtrace: script './lwptime.d' matched 3 probes
^C

  nscd                                              
           value  ------------- Distribution ------------- count
          131072 |                                         0        
          262144 |@                                        18       
          524288 |@@                                       24       
         1048576 |@@@@@@@                                  75       
         2097152 |@@@@@@@@@@@@@@@@@@@@@@@                  245      
         4194304 |@@                                       22       
         8388608 |@@                                       24       
        16777216 |                                         6        
        33554432 |                                         3        
        67108864 |                                         1        
       134217728 |                                         1        
       268435456 |                                         0        

  mountd                                            
           value  ------------- Distribution ------------- count
          524288 |                                         0        
         1048576 |@                                        15       
         2097152 |@                                        24       
         4194304 |@@@                                      51       
         8388608 |@                                        17       
        16777216 |@                                        24       
        33554432 |@                                        15       
        67108864 |@@@@                                     57       
       134217728 |@                                        28       
       268435456 |@                                        26       
       536870912 |@@                                       39       
      1073741824 |@@@                                      45       
      2147483648 |@@@@@                                    72       
      4294967296 |@@@@@                                    77       
      8589934592 |@@@                                      55       
     17179869184 |                                         14       
     34359738368 |                                         2        
     68719476736 |                                         0        

  automountd                                        
           value  ------------- Distribution ------------- count
         1048576 |                                         0        
         2097152 |                                         3        
         4194304 |@@@@                                     146      
         8388608 |                                         6        
        16777216 |                                         6        
        33554432 |                                         9        
        67108864 |@@@@@                                    203      
       134217728 |@@                                       87       
       268435456 |@@@@@@@@@@@@@@@                          534      
       536870912 |@@@@@@                                   223      
      1073741824 |@                                        45       
      2147483648 |                                         20       
      4294967296 |                                         26       
      8589934592 |                                         20       
     17179869184 |                                         19       
     34359738368 |                                         7        
     68719476736 |                                         2        
    137438953472 |                                         0        

  iCald
           value  ------------- Distribution ------------- count
         8388608 |                                         0        
        16777216 |@@@@@@@                                  20       
        33554432 |@@@                                      9        
        67108864 |@@                                       8        
       134217728 |@@@@@                                    16       
       268435456 |@@@@                                     11       
       536870912 |@@@@                                     11       
      1073741824 |@                                        4        
      2147483648 |                                         2        
      4294967296 |                                         0        
      8589934592 |@@                                       8        
     17179869184 |@                                        5        
     34359738368 |@                                        4        
     68719476736 |@@                                       6        
    137438953472 |@                                        4        
    274877906944 |                                         2        
    549755813888 |                                         0

Using proc signal-send

You can use the signal-send probe to determine the sending and receiving process associated with any signal, as shown in the following example:

#pragma D option quiet

proc:::signal-send
{
        @[execname, stringof(args[1]->pr_fname), args[2]] = count();
}

END
{
        printf("%20s %20s %12s %s\n",
            "SENDER", "RECIPIENT", "SIG", "COUNT");
        printa("%20s %20s %12d %@d\n", @);
}

Running this script results in output similar to the following example:

# dtrace -s ./sig.d
^C
              SENDER            RECIPIENT          SIG COUNT
               xterm               dtrace            2 1
               xterm          soffice.bin            2 1
                  tr                 init           18 1
               sched                 test           18 1
               sched                fvwm2           18 1
                bash                 bash           20 1
                 sed                 init           18 2
               sched                  ksh           18 15
               sched                 Xsun           22 471

proc Stability

The proc provider uses stability mechanism of DTrace to describe its stabilities, as shown in the following table. For more information about the stability mechanism, see DTrace Stability Mechanisms.

Table 48  Stability Mechanism for the proc Provider
Element
Name Stability
Data Stability
Dependency Class
Provider
Evolving
Evolving
ISA
Module
Private
Private
Unknown
Function
Private
Private
Unknown
Name
Evolving
Evolving
ISA
Arguments
Evolving
Evolving
ISA