1 About DTrace
DTrace provides dynamic tracing, which is the ability to instrument a running operating system kernel.
DTrace enables you to associate actions, such as collecting or printing stack traces, function arguments, timestamps, and statistical aggregates, with probes, which can be runtime events or source-code locations. The D language is powerful, yet simple. DTrace is dynamic, has low overhead, and is safe to use on production systems. It enables you to examine the behavior of user programs and the operating system, to understand how your system works, to track down performance problems, and to locate the causes of aberrant behavior.
DTrace is a kernel framework that dynamically traces data into buffers that are read by consumers. On Oracle Linux, you will probably only use one consumer, the dtrace command-line utility, which contains the D language that grants you full access to the framework's power.
This guide is largely a reference manual. For information about how to use DTrace and step-by-step examples, see Oracle Linux: DTrace Tutorial.
Getting Started With DTrace
Note:
Most uses of DTrace require root privileges.
Prior to installing the dtrace_utils package,
ensure that you are subscribed to the ULN channel that corresponds
to the UEK kernel that you are running. For example, if you are
running Oracle Linux 7 with UEK R5, the dtrace_utils
package is available in the ol7_UEKR5 channel.
For more information about subscribing to channels on ULN, see
Oracle Linux: Unbreakable Linux Network User's Guide for Oracle Linux 6 and Oracle Linux 7.
For information about updating your Oracle Linux or UEK release, see the documentation at https://docs.oracle.com/en/operating-systems/linux.html.
Install the dtrace-utils package:
# yum install dtrace-utils
If you want to implement a libdtrace consumer:
# yum install dtrace-utils-devel
If you want to develop a DTrace provider:
# yum install dtrace-modules-provider-headers
To confirm that dtrace is properly installed on
your system and that you have all of the required privileges, use
the dtrace -l command. Running this command
should load any of the required kernel modules and the output
should indicate any available probes.
Note:
The dtrace-utils package installs
dtrace in
/usr/sbin/dtrace. Make sure your path detects
this path instead of the similarly named utility that is located
in /usr/bin/dtrace, which is installed by the
systemtap-sdt-devel package.
A provider is a set of probes with a particular kind of instrumentation.
Note:
To use a provider's probes, the kernel module that supports that
provider must be loaded. Typically, dtrace
automatically handles this for you. Upon first use, it will load
the dtrace module and all of the modules that
are listed in /etc/dtrace-modules, which the
system administrator can edit.
In some cases, the kernel module that supports the desired provider must be loaded manually, for example:
# more /etc/dtrace-modules sdt systrace profile fasttrap # modprobe sdt # modprobe systrace # modprobe profile # modprobe fasttrap
These required modules are different from the modules, if any,
that are instrumented by the provider's probes and are found in
the dtrace -l output. For example, while the
module that is required to support proc
probes is sdt, the module that these probes
instrument is vmlinux, as shown in the
following output:
# dtrace -l -P proc ID PROVIDER MODULE FUNCTION NAME 197 proc vmlinux _do_fork lwp-create 198 proc vmlinux _do_fork create 225 proc vmlinux do_exit lwp-exit 226 proc vmlinux do_exit exit 275 proc vmlinux do_sigtimedwait signal-clear ...
You dynamically assign actions to be taken at probes, which can be
runtime events or source-code locations. Every probe in DTrace has
two names: a unique integer ID, which is assigned as the probes
are loaded, and a human-readable string name. You can start
learning about DTrace by building some very simple requests that
use the probe named BEGIN. The
BEGIN probe fires once each time you start a
new tracing request.
Use the dtrace command with the -n option to enable a probe by specifying its name:
# dtrace -n BEGIN dtrace: description 'BEGIN' matched 1 probe CPU ID FUNCTION:NAME 0 1 :BEGIN ^C #
The default output of the previous example displays the following
information: the probes that were matched, column headers, and
then one row each time a probe fires. The default per row is the
CPU where the probe fired and information about which probe fired.
DTrace remains paused, waiting for other probes to fire. To exit,
press Ctrl-C.
You can construct DTrace requests by using arbitrary numbers of
probes and actions. For example, create a simple request using two
probes by adding the END probe to the command
shown in the previous example. The END probe
fires once when tracing is completed.
Type the following command, and then press
Ctrl-C in your shell again, after you see the
line of output for the BEGIN probe:
# dtrace -n BEGIN -n END dtrace: description 'BEGIN' matched 1 probe dtrace: description 'END' matched 1 probe CPU ID FUNCTION:NAME 0 1 :BEGIN ^C 1 2 :END
Pressing Ctrl-C to exit
dtrace triggers the END
probe. The dtrace command reports this probe
firing before exiting.
In addition to constructing DTrace experiments on the command line, you can also write DTrace experiments in text files by using the D programming language.
In a text editor, create a new file named
hello.d and type your first D program:
BEGIN
{
trace("hello, world");
exit(0);
}After you save the program, you can run it by using the dtrace -s command, as shown in the following example:
# dtrace -s hello.d dtrace: script 'hello.d' matched 1 probe CPU ID FUNCTION:NAME 0 1 :BEGIN hello, world #
The dtrace command printed the same output as
the previous example, followed by the text, ”hello, world”.
However, unlike the previous example, you did not have to wait and
then press Ctrl-C. These changes were the
result of the actions that you specified for the
BEGIN probe in hello.d.
To understand what happened, let us explore the structure of your D program in more detail.
-
Each D program consists of a series of clauses, and each clause describes one or more probes to enable, as well as an optional set of actions to perform when the probes fires.
-
The actions are listed as a series of statements that are enclosed in braces (
{}) that follow the probe name. Each statement ends with a semicolon (;). -
The first statement uses the
trace()function to indicate that DTrace should record the specified argument, the string, ”hello, world”, when theBEGINprobe fires and then print it out. -
The second statement uses the
exit()function to indicate that DTrace should cease tracing and exit the dtrace command.
DTrace provides a set of useful functions such as
trace() and exit() for you
to call in your D programs.
To call a function, you specify its name, followed by a parenthesized list of arguments. See Actions and Subroutines for the complete set of D functions.
If you are familiar with the C programming language, you probably have noticed that DTrace's D programming language is very similar to C. Indeed, D is derived from a large subset of C, combined with a special set of functions and variables to help make tracing easy. These features are described in more detail in subsequent chapters. If you have written a C program previously, you should be able to immediately transfer most of your knowledge to building tracing programs in D. If you have never written a C program, learning D is still relatively easy. By the end of this chapter, you will understand all of the syntax. First, let us take a step back from language rules and learn more about how DTrace works. Then, later in this guide, you will learn how to build more interesting D programs.
Providers and Probes
In the preceding examples, you learned how to use two simple
probes named BEGIN and END.
DTrace probes come in sets that are called
providers, each of which performs a
particular kind of instrumentation to create probes. When you use
DTrace, each provider is given an opportunity to publish the
probes that it can provide to the DTrace framework. You can then
enable and bind your tracing actions to any of the probes that
have been published.
You can list all of the available probes on your system by typing the following command:
# dtrace -l
ID PROVIDER MODULE FUNCTION NAME
1 dtrace BEGIN
2 dtrace END
3 dtrace ERROR
4 syscall vmlinux read entry
5 syscall vmlinux read return
6 syscall vmlinux write entry
7 syscall vmlinux write return
...Note that it might take some time for all of the output to be displayed.
To count all of the probes, type the following command:
# dtrace -l | wc -l 4097
Note that you might observe a different total on your system, as the number of probes can vary, depending on the following: your operating platform, the software you have installed, and the provider modules you have loaded. Note also that this output is not the complete list. As will be described later, some providers offer the ability to create new probes on-the-fly, based on your tracing requests, which makes the actual number of DTrace probes virtually unlimited. Notice that each probe has the two names previously mentioned: an integer ID and a human-readable name. The human-readable name is composed of four parts that are displayed as separate columns in the dtrace output and are as follows:
- provider
-
A name of the DTrace provider that is publishing this probe.
- module
-
If this probe corresponds to a specific program location, the name of the kernel module, library, or user-space program in which the probe is located.
- function
-
If this probe corresponds to a specific program location, the name of the program function in which the probe is located.
- name
-
A name that provides some idea of the probe's semantic meaning, such as
BEGINorEND.
When writing the full human-readable name of a probe, write all four parts of the name separated by colons like this:
provider:module:function:name
Notice that some of the probes in the list do not have a module
and function, such as the BEGIN and
END probes that were used previously. Some
probes leave these two fields blank because these probes do not
correspond to any specific instrumented program function or
location. Instead, these probes refer to a more abstract concept,
such as the idea of the end of your tracing request.
By convention, if you do not specify all of the fields of a probe
name, DTrace matches your request to all of the probes with
matching values in the parts of the name that you do specify. In
other words, when you used the probe name BEGIN
in the previous exercise, you were actually directing DTrace to
match any probe with the name field BEGIN,
regardless of the value of the provider, module, and function
fields. Because there is only one probe matching that description,
the result is the same. You now know that the true name of the
BEGIN probe is
dtrace:::BEGIN, which indicates that this probe
is provided by the DTrace framework itself and is not specific to
any function. Therefore, the hello.d program
could be written as follows and would produce the same result:
dtrace:::BEGIN
{
trace("hello, world");
exit(0);
}