Solaris Dynamic Tracing Guide

Chapter 25 proc Provider

The proc provider makes available probes pertaining to the following activities: process creation and termination, LWP creation and termination, executing new program images, and sending and handling signals.

Probes

The proc probes are described in Table 25–1.

Table 25–1 proc Probes

Probe 

Description 

create

Probe that fires when a process is created using fork(2), forkall(2), fork1(2), or vfork(2). The psinfo_t corresponding to the new child process is pointed to by args[0]. You can distinguish vfork from the other fork variants by checking for PR_VFORKP in the pr_flag member of the forking thread's lwpsinfo_t. You can distinguish fork1 from forkall by examining the pr_nlwp members of both the parent process's psinfo_t (curpsinfo) and the child process's psinfo_t (args[0]). Because the create probe only fires after the process has been successfully created, and because LWP creation is part of creating a process, lwp-create will fire for any LWPs created at process creation time before the create probe fires for the new process.

exec

Probe that fires whenever a process loads a new process image with a variant of the exec(2) system call: exec(2), execle(2), execlp(2), execv(2), execve(2), execvp(2). The exec probe fires before the process image is loaded. Process variables like execname and curpsinfo therefore contain the process state before the image is loaded. Some time after the exec probe fires, either the exec-failure probe or the exec-success probe will subsequently fire in the same thread. The path of the new process image is pointed to by args[0].

exec-failure

Probe that fires when an exec(2) variant has failed. The exec-failure probe fires only after the exec probe has fired in the same thread. The errno(3C) value is provided in args[0].

exec-success

Probe that fires when an exec(2) variant has succeeded. Like the exec-failure probe, the exec-success probe fires only after the exec probe has fired in the same thread. By the time the exec-success probe fires, process variables like execname and curpsinfo contain the process state after the new process image has been loaded.

exit

Probe that fires when the current process is exiting. The reason for exit, which is expressed as one of the SIGCHLD siginfo.h(3HEAD) codes, is contained in args[0].

fault

Probe that fires when a thread experiences a machine fault. The fault code (as defined in proc(4)) is in args[0]. The siginfo structure corresponding to the fault is pointed to by args[1]. Only those faults that induce a signal can trigger the fault probe.

lwp-create

Probe that fires when an LWP is created, typically as a result of thr_create(3C). The lwpsinfo_t corresponding to the new thread is pointed to by args[0]. The psinfo_t of the process containing the thread is pointed to by args[1].

lwp-start

Probe that fires within the context of a newly created LWP. The lwp-start probe will fire before any user-level instructions are executed. If the LWP is the first LWP in the process, the start probe will fire, followed by lwp-start.

lwp-exit

Probe that fires when an LWP is exiting, due either to a signal or to an explicit call to thr_exit(3C).

signal-discard

Probe that fires when a signal is sent to a single-threaded process, and the signal is both unblocked and ignored by the process. Under these conditions, the signal is discarded on generation. The lwpsinfo_t and psinfo_t of the target process and thread are in args[0] and args[1], respectively. The signal number is in args[2].

signal-send

Probe that fires when a signal is sent to a thread or process. The signal-send probe fires in the context of the sending process and thread. The lwpsinfo_t and psinfo_t of the receiving process and thread are in args[0] and args[1], respectively. The signal number is in args[2]. signal-send is always followed by signal-handle or signal-clear in the receiving process and thread.

signal-handle

Probe that fires immediately before a thread handles a signal. The signal-handle probe fires in the context of the thread that will handle the signal. The signal number is in args[0]. A pointer to the siginfo_t structure that corresponds to the signal is in args[1]. The value of args[1] is NULL if there is no siginfo_t structure or if the signal handler does not have the SA_SIGINFO flag set. The address of the signal handler in the process is in args[2].

signal-clear

Probes that fires when a pending signal is cleared because the target thread was waiting for the signal in sigwait(2), sigwaitinfo(3RT), or sigtimedwait(3RT). Under these conditions, the pending signal is cleared and the signal number is returned to the caller. The signal number is in args[0]. signal-clear fires in the context of the formerly waiting thread.

start

Probe that fires in the context of a newly created process. The start probe will fire before any user-level instructions are executed in the process.

Arguments

The argument types for the proc probes are listed in Table 25–2. The arguments are described in Table 25–1.

Table 25–2 proc Probe Arguments

Probe 

args[0]

args[1]

args[2]

create

psinfo_t *

— 

— 

exec

char *

— 

— 

exec-failure

int

— 

— 

exit

int

— 

— 

fault

int

siginfo_t *

— 

lwp-create

lwpsinfo_t *

psinfo_t *

— 

lwp-start

— 

— 

— 

lwp-exit

— 

— 

— 

signal-discard

lwpsinfo_t *

psinfo_t *

int

signal-discard

lwpsinfo_t *

psinfo_t *

int

signal-send

lwpsinfo_t *

psinfo_t *

int

signal-handle

int

siginfo_t *

void (*)(void)

signal-clear

int

— 

— 

start

— 

— 

— 

lwpsinfo_t

Several proc probes have arguments of type lwpsinfo_t, a structure that is documented in proc(4). The definition of the lwpsinfo_t structure as available to DTrace consumers is as follows:

typedef struct lwpsinfo {
	int pr_flag;              /* flags; see below */
	id_t pr_lwpid;            /* LWP id */
	uintptr_t pr_addr;        /* internal address of thread */
	uintptr_t pr_wchan;       /* wait addr for sleeping thread */
	char pr_stype;            /* synchronization event type */
	char pr_state;            /* numeric thread state */
	char pr_sname;            /* printable character for pr_state */
	char pr_nice;             /* nice for cpu usage */
	short pr_syscall;         /* system call number (if in syscall) */
	int pr_pri;               /* priority, high value = high priority */
	char pr_clname[PRCLSZ];   /* scheduling class name */
	processorid_t pr_onpro;   /* processor which last ran this thread */
	processorid_t pr_bindpro; /* processor to which thread is bound */
	psetid_t pr_bindpset;     /* processor set to which thread is bound */
} lwpsinfo_t;

The pr_flag field is a bit-mask holding flags describing the process. These flags and their meanings are described in Table 25–3.

Table 25–3 pr_flag Values

PR_ISSYS

The process is a system process. 

PR_VFORKP

The process is the parent of a vfork(2)'d child.

PR_FORK

The process has its inherit-on-fork mode set. 

PR_RLC

The process has its run-on-last-close mode set. 

PR_KLC

The process has its kill-on-last-close mode set. 

PR_ASYNC

The process has its asynchronous-stop mode set. 

PR_MSACCT

The process has microstate accounting enabled. 

PR_MSFORK

The process microstate accounting is inherited on fork. 

PR_BPTADJ

The process has its breakpoint adjustment mode set. 

PR_PTRACE

The process has its ptrace(3C)-compatibility mode set.

PR_STOPPED

The thread is an LWP that is stopped. 

PR_ISTOP

The thread is an LWP stopped on an event of interest. 

PR_DSTOP

The thread is an LWP that has a stop directive in effect. 

PR_STEP

The thread is an LWP that has a single-step directive in effect. 

PR_ASLEEP

The thread is an LWP in an interruptible sleep within a system call. 

PR_DETACH

The thread is a detached LWP. See pthread_create(3C) and pthread_join(3C).

PR_DAEMON

The thread is a daemon LWP. See pthread_create(3C).

PR_AGENT

The thread is the agent LWP for the process. 

PR_IDLE

The thread is the idle thread for a CPU. Idle threads only run on a CPU when the run queues for the CPU are empty. 

The pr_addr field is the address of a private, in-kernel data structure representing the thread. While the data structure is private, the pr_addr field may be used as a token unique to a thread for the thread's lifetime.

The pr_wchan field is set when the thread is sleeping on a synchronization object. The meaning of the pr_wchan field is private to the kernel implementation, but the field may be used as a token unique to the synchronization object.

The pr_stype field is set when the thread is sleeping on a synchronization object. The possible values for the pr_stype field are in Table 25–4.

Table 25–4 pr_stype Values

SOBJ_MUTEX

Kernel mutex synchronization object. Used to serialize access to shared data regions in the kernel. See Chapter 18, lockstat Provider and mutex_init(9F) for details on kernel mutex synchronization objects.

SOBJ_RWLOCK

Kernel readers/writer synchronization object. Used to synchronize access to shared objects in the kernel that can allow multiple concurrent readers or a single writer. See Chapter 18, lockstat Provider and rwlock(9F) for details on kernel readers/writer synchronization objects.

SOBJ_CV

Condition variable synchronization object. A condition variable is designed to wait indefinitely until some condition becomes true. Condition variables are typically used to synchronize for reasons other than access to a shared data region, and are the mechanism generally used when a process performs a program-directed indefinite wait. For example, blocking in poll(2), pause(2), wait(3C), and the like.

SOBJ_SEMA

Semaphore synchronization object. A general-purpose synchronization object that – like condition variable objects – does not track a notion of ownership. Because ownership is required to implement priority inheritance in the Solaris kernel, the lack of ownership inherent in semaphore objects inhibits their widespread use. See semaphore(9F) for details.

SOBJ_USER

A user-level synchronization object. All blocking on user-level synchronization objects is handled with SOBJ_USER synchronization objects. User-level synchronization objects include those created with mutex_init(3C), sema_init(3C), rwlock_init(3C), cond_init(3C) and their POSIX equivalents.

SOBJ_USER_PI

A user-level synchronization object that implements priority inheritance. Some user-level synchronization objects that track ownership additionally allow for priority inheritance. For example, mutex objects created with pthread_mutex_init(3C) may be made to inherit priority using pthread_mutexattr_setprotocol(3C).

SOBJ_SHUTTLE

A shuttle synchronization object. Shuttle objects are used to implement doors. See door_create(3DOOR) for more information. 

The pr_state field is set to one of the values in Table 25–5. The pr_sname field is set to a corresponding character shown in parentheses in the same table.

Table 25–5 pr_state Values

SSLEEP (S)

The thread is sleeping. The sched:::sleep probe will fire immediately before a thread's state is transitioned to SSLEEP.

SRUN (R)

The thread is runnable, but is not currently running. The sched:::enqueue probe will fire immediately before a thread's state is transitioned to SRUN.

SZOMB (Z)

The thread is a zombie LWP. 

SSTOP (T)

The thread is stopped, either due to an explicit proc(4) directive or some other stopping mechanism.

SIDL (I)

The thread is an intermediate state during process creation. 

SONPROC (O)

The thread is running on a CPU. The sched:::on-cpu probe will fire in the context of the SONPROC thread a short time after the thread's state is transitioned to SONPROC.

psinfo_t

Several proc probes have an argument of type psinfo_t, a structure that is documented in proc(4). The definition of the psinfo_t structure as available to DTrace consumers is as follows:

typedef struct psinfo {
	int     pr_nlwp;            /* number of active lwps in the process */
	pid_t   pr_pid;             /* unique process id */
	pid_t   pr_ppid;            /* process id of parent */
	pid_t   pr_pgid;            /* pid of process group leader */
	pid_t   pr_sid;             /* session id */
	uid_t   pr_uid;             /* real user id */
	uid_t   pr_euid;            /* effective user id */
	gid_t   pr_gid;             /* real group id */
	gid_t   pr_egid;            /* effective group id */
	uintptr_t pr_addr;          /* address of process */
	dev_t   pr_ttydev;          /* controlling tty device (or PRNODEV) */
	timestruc_t pr_start;       /* process start time, from the epoch */
	char    pr_fname[PRFNSZ];   /* name of execed file */
	char    pr_psargs[PRARGSZ]; /* initial characters of arg list */
	int     pr_argc;            /* initial argument count */
	uintptr_t pr_argv;          /* address of initial argument vector */
	uintptr_t pr_envp;          /* address of initial environment vector */
	char    pr_dmodel;          /* data model of the process */
	taskid_t pr_taskid;         /* task id */
	projid_t pr_projid;         /* project id */
	poolid_t pr_poolid;         /* pool id */
	zoneid_t pr_zoneid;         /* zone id */
} psinfo_t;

The pr_dmodel field is set to either PR_MODEL_ILP32, denoting a 32–bit process, or PR_MODEL_LP64, denoting a 64–bit process.

Examples

exec

You can use the exec probe to easily determine which programs are being executed, and by whom, as shown in the following example:

#pragma D option quiet

proc:::exec
{
        self->parent = execname;
}

proc:::exec-success
/self->parent != NULL/
{
	@[self->parent, execname] = count();
	self->parent = NULL;
}

proc:::exec-failure
/self->parent != NULL/
{
	self->parent = NULL;
}

END
{
	printf("%-20s %-20s %s\n", "WHO", "WHAT", "COUNT");
	printa("%-20s %-20s %@d\n", @);
}

Running the example script for a short period of time on a build machine results in output similar to the following example:


# dtrace -s ./whoexec.d
^C
WHO                  WHAT                 COUNT
make.bin             yacc                 1
tcsh                 make                 1
make.bin             spec2map             1
sh                   grep                 1
lint                 lint2                1
sh                   lint                 1
sh                   ln                   1
cc                   ld                   1
make.bin             cc                   1
lint                 lint1                1
sh                   lex                  1
make.bin             mv                   2
sh                   sh                   3
sh                   make                 3
sh                   sed                  4
sh                   tr                   4
make                 make.bin             4
sh                   install.bin          5
sh                   rm                   6
cc                   ir2hf                33
cc                   ube                  33
sh                   date                 34
sh                   mcs                  34
cc                   acomp                34
sh                   cc                   34
sh                   basename             34
basename             expr                 34
make.bin             sh                   87

start and exit

If you want to know how long programs are running from creation to termination, you can enable the start and exit probes, as shown in the following example:

proc:::start
{
	self->start = timestamp;
}

proc:::exit
/self->start/
{
	@[execname] = quantize(timestamp - self->start);
	self->start = 0;
}

Running the example script on the build server for several seconds results in output similar to the following example:


# dtrace -s ./progtime.d
dtrace: script './progtime.d' matched 2 probes
^C

  ir2hf                                             
           value  ------------- Distribution ------------- count
         4194304 |                                         0        
         8388608 |@                                        1        
        16777216 |@@@@@@@@@@@@@@@@                         14       
        33554432 |@@@@@@@@@@                               9        
        67108864 |@@@                                      3        
       134217728 |@                                        1        
       268435456 |@@@@                                     4        
       536870912 |@                                        1        
      1073741824 |                                         0        

  ube                                               
           value  ------------- Distribution ------------- count
        16777216 |                                         0        
        33554432 |@@@@@@@                                  6        
        67108864 |@@@                                      3        
       134217728 |@@                                       2        
       268435456 |@@@@                                     4        
       536870912 |@@@@@@@@@@@@                             10       
      1073741824 |@@@@@@@                                  6        
      2147483648 |@@                                       2        
      4294967296 |                                         0        

  acomp                                             
           value  ------------- Distribution ------------- count
         8388608 |                                         0        
        16777216 |@@                                       2        
        33554432 |                                         0        
        67108864 |@                                        1        
       134217728 |@@@                                      3        
       268435456 |                                         0        
       536870912 |@@@@@                                    5        
      1073741824 |@@@@@@@@@@@@@@@@@@@@@@@@@                22       
      2147483648 |@                                        1        
      4294967296 |                                         0        

  cc                                                
           value  ------------- Distribution ------------- count
        33554432 |                                         0        
        67108864 |@@@                                      3        
       134217728 |@                                        1        
       268435456 |                                         0        
       536870912 |@@@@                                     4        
      1073741824 |@@@@@@@@@@@@@@                           13       
      2147483648 |@@@@@@@@@@@@                             11       
      4294967296 |@@@                                      3        
      8589934592 |                                         0        

  sh                                                
           value  ------------- Distribution ------------- count
          262144 |                                         0        
          524288 |@                                        5        
         1048576 |@@@@@@@                                  29       
         2097152 |                                         0        
         4194304 |                                         0        
         8388608 |@@@                                      12       
        16777216 |@@                                       9        
        33554432 |@@                                       9        
        67108864 |@@                                       8        
       134217728 |@                                        7        
       268435456 |@@@@@                                    20       
       536870912 |@@@@@@                                   26       
      1073741824 |@@@                                      14       
      2147483648 |@@                                       11       
      4294967296 |                                         3        
      8589934592 |                                         1        
     17179869184 |                                         0        

  make.bin                                          
           value  ------------- Distribution ------------- count
        16777216 |                                         0        
        33554432 |@                                        1        
        67108864 |@                                        1        
       134217728 |@@                                       2        
       268435456 |                                         0        
       536870912 |@@                                       2        
      1073741824 |@@@@@@@@@                                9        
      2147483648 |@@@@@@@@@@@@@@@                          14       
      4294967296 |@@@@@@                                   6        
      8589934592 |@@                                       2        
     17179869184 |                                         0

lwp-start and lwp-exit

Instead of knowing the amount of time that a particular process takes to run, you might want to know how long individual threads take to run. The following example shows how to use the lwp-start and lwp-exit probes for this purpose:

proc:::lwp-start
/tid != 1/
{
	self->start = timestamp;
}

proc:::lwp-exit
/self->start/
{
	@[execname] = quantize(timestamp - self->start);
	self->start = 0;
}

Running the example script on an NFS and calendar server results in output similar to the following example:


# dtrace -s ./lwptime.d
dtrace: script './lwptime.d' matched 3 probes
^C

  nscd                                              
           value  ------------- Distribution ------------- count
          131072 |                                         0        
          262144 |@                                        18       
          524288 |@@                                       24       
         1048576 |@@@@@@@                                  75       
         2097152 |@@@@@@@@@@@@@@@@@@@@@@@                  245      
         4194304 |@@                                       22       
         8388608 |@@                                       24       
        16777216 |                                         6        
        33554432 |                                         3        
        67108864 |                                         1        
       134217728 |                                         1        
       268435456 |                                         0        

  mountd                                            
           value  ------------- Distribution ------------- count
          524288 |                                         0        
         1048576 |@                                        15       
         2097152 |@                                        24       
         4194304 |@@@                                      51       
         8388608 |@                                        17       
        16777216 |@                                        24       
        33554432 |@                                        15       
        67108864 |@@@@                                     57       
       134217728 |@                                        28       
       268435456 |@                                        26       
       536870912 |@@                                       39       
      1073741824 |@@@                                      45       
      2147483648 |@@@@@                                    72       
      4294967296 |@@@@@                                    77       
      8589934592 |@@@                                      55       
     17179869184 |                                         14       
     34359738368 |                                         2        
     68719476736 |                                         0        

  automountd                                        
           value  ------------- Distribution ------------- count
         1048576 |                                         0        
         2097152 |                                         3        
         4194304 |@@@@                                     146      
         8388608 |                                         6        
        16777216 |                                         6        
        33554432 |                                         9        
        67108864 |@@@@@                                    203      
       134217728 |@@                                       87       
       268435456 |@@@@@@@@@@@@@@@                          534      
       536870912 |@@@@@@                                   223      
      1073741824 |@                                        45       
      2147483648 |                                         20       
      4294967296 |                                         26       
      8589934592 |                                         20       
     17179869184 |                                         19       
     34359738368 |                                         7        
     68719476736 |                                         2        
    137438953472 |                                         0        

  iCald
           value  ------------- Distribution ------------- count
         8388608 |                                         0        
        16777216 |@@@@@@@                                  20       
        33554432 |@@@                                      9        
        67108864 |@@                                       8        
       134217728 |@@@@@                                    16       
       268435456 |@@@@                                     11       
       536870912 |@@@@                                     11       
      1073741824 |@                                        4        
      2147483648 |                                         2        
      4294967296 |                                         0        
      8589934592 |@@                                       8        
     17179869184 |@                                        5        
     34359738368 |@                                        4        
     68719476736 |@@                                       6        
    137438953472 |@                                        4        
    274877906944 |                                         2        
    549755813888 |                                         0

signal-send

You can use the signal-send probe to determine the sending and receiving process associated with any signal, as shown in the following example:

#pragma D option quiet

proc:::signal-send
{
	@[execname, stringof(args[1]->pr_fname), args[2]] = count();
}

END
{
	printf("%20s %20s %12s %s\n",
	    "SENDER", "RECIPIENT", "SIG", "COUNT");
	printa("%20s %20s %12d %@d\n", @);
}

Running this script results in output similar to the following example:


# dtrace -s ./sig.d
^C
              SENDER            RECIPIENT          SIG COUNT
               xterm               dtrace            2 1
               xterm          soffice.bin            2 1
                  tr                 init           18 1
               sched                 test           18 1
               sched                fvwm2           18 1
                bash                 bash           20 1
                 sed                 init           18 2
               sched                  ksh           18 15
               sched                 Xsun           22 471

Stability

The proc provider uses DTrace's stability mechanism to describe its stabilities, as shown in the following table. For more information about the stability mechanism, see Chapter 39, Stability.

Element 

Name stability 

Data stability 

Dependency class 

Provider 

Evolving 

Evolving 

ISA

Module 

Private 

Private 

Unknown 

Function 

Private 

Private 

Unknown 

Name 

Evolving 

Evolving 

ISA

Arguments 

Evolving 

Evolving 

ISA