You can use the pid provider to trace any instruction in any user function. Upon demand, the pid provider creates a probe for every instruction in a function. The name of each probe is the offset of its corresponding instruction in the function and is expressed as a hexadecimal integer. For example, to enable a probe that is associated with the instruction at offset 0x1c in the function foo of the module bar.so in the process with PID 123, you would use the following command:

# dtrace -n pid123:bar.so:foo:1c

To enable all of the probes in the function foo, including the probe for each instruction, you would use the following command:

# dtrace -n pid123:bar.so:foo:

Using the previous command demonstrates an extremely powerful technique for debugging and analyzing user applications. Infrequent errors can be difficult to debug because they can be difficult to reproduce. Often, you can identify a problem after the failure has occurred, which is too late to reconstruct the code path.

The following example demonstrates how to combine the pid provider with speculative tracing to solve this problem by tracing every instruction in a function. See Chapter 7, Speculative Tracing for a description.

Type the following source code and save it in a file named errorpath.d:

pid$1::$2:entry
{
  self->spec = speculation();
  speculate(self->spec);
  printf("%x %x %x %x %x", arg0, arg1, arg2, arg3, arg4);
}

pid$1::$2:
/self->spec/
{
  speculate(self->spec);
}

pid$1::$2:return
/self->spec && arg1 == 0/
{
  discard(self->spec);
  self->spec = 0;
}

pid$1::$2:return
/self->spec && arg1 != 0/
{
  commit(self->spec);
  self->spec = 0;
}

Executing the errorpath.d script results in output similar to the following:

# ./errorpath.d 123 _chdir
dtrace: script ’./errorpath.d’ matched 19 probes
CPU    ID                     FUNCTION:NAME
  0 25253                      _chdir:entry 81e08 6d140 ffbfcb20 656c73 0
  0 25253                      _chdir:entry
  0 25269                      _chdir:0
  0 25270                      _chdir:4
  0 25271                      _chdir:8
  0 25272                      _chdir:c
  0 25273                      _chdir:10
  0 25274                      _chdir:14
  0 25275                      _chdir:18
  0 25276                      _chdir:1c
  0 25277                      _chdir:20
  0 25278                      _chdir:24
  0 25279                      _chdir:28
  0 25280                      _chdir:2c
  0 25268                      _chdir:return