test

System Administration Guide

Chapter 62 System Performance (Overview)

Getting good performance from a computer or network is an important part of system administration. This chapter is an overview of some of the factors that contribute to maintaining and managing the performance of the computer systems in your care.

This is a list of the overview information in this chapter.

What's New in System Performance

This section describes new system performance features in the Solaris 2.6 release.

Scheduler Activation

Scheduler activations provide kernel scheduling support for applications with particular scheduling needs, such as database and multithreaded applications. Multithreaded support changes for scheduler activation are implemented as a private interface between the kernel and the libthread library, without changing the libthread interface. Additionally, applications may give scheduling hints to the kernel to improve performance. See libsched(3X) for more information.

The psrset Command

Another new feature allows a group of processors to be allocated for the exclusive use of one or more applications. The /usr/sbin/psrset command gives a system administrator control over the creation and management of processor sets. See psrset(1M) for more information.

UFS Direct Input/Output (I/O)

Direct I/O is intended to boost bulk I/O operations. Bulk I/O operations use large buffer sizes to transfer large files (files larger than physical memory).

An example of a bulk I/O operation is downloading satellite data, which writes large amounts of data to a file. Direct I/O data is read or written into memory without using the overhead of the operating system's page caching mechanism.

There is a potential penalty on direct I/O startup. If a file requested for I/O is already mapped by another application, the pages will have to be flushed out of memory before the direct I/O operation can begin.

See directio(3C) for more information.

Direct I/O can also be enabled on a file system by using the forcedirectio option to the mount command. Enabling direct I/O is a performance benefit only when a file system is transferring large amounts of sequential data.

When a file system is mounted with this option, data is transferred directly between a user's address space and the disk. When forced direct I/O is not enabled for a file system, data transferred between a user's address space and the disk is first buffered in the kernel address space.

The default behavior is no forced direct I/O on a UFS file system. See mount_ufs for more information.

How to Enable Forced Direct I/O on a UFS File System

  1. Become superuser.

  2. Mount a file system with the forcedirectio mount option.


    # mount -F ufs -o forcedirectio /dev/dsk/c0t3d0s7 /datab

  3. Verify the mounted file system has forced direct I/O enabled.


    # mount
                    .
                    .
                    .
/export/home on /dev/dsk/c0t3d0s7 forcedirectio/setuid/read/write/largefiles
on Mon May 12 13:47:55 1997

Enhancements to the /proc File System and Watchpoints

The previous flat /proc file system has been restructured into a directory hierarchy that contains additional sub-directories for state information and control functions.

It also provides a watchpoint facility that is used to remap read/write permissions on the individual pages of a process's address space. This facility has no restrictions and is MT-safe.

The new /proc file structure provides complete binary compatibility with the old /proc interface except that the new watchpoint facility cannot be used with the old interface.

Debugging tools have been modified to use /proc's new watchpoint facility, which means the entire watchpoint process is faster.

The following restrictions have been removed when setting watchpoints using the dbx debugging tool:

See proc(4), core(4), adb(4) for more information.

Where to Find System Performance Tasks

Use these references to find step-by-step instructions for monitoring system performance.

System Performance and System Resources

The performance of a computer system depends upon how the system uses and allocates its resources. It is important to monitor your system's performance on a regularly so that you know how it behaves under normal conditions. You should have a good idea of what to expect, and be able to recognize a problem when it occurs.

System resources that affect performance include:

Chapter 64, Monitoring Performance (Tasks) describes the tools that display statistics about the activity and the performance of the computer system.

Other Sources of Information

Performance is a broad subject that can't be adequately covered in these chapters. There are several books available that cover various aspects of improving performance and tuning your system or network. Three useful books are:

Processes and System Performance

Terms related to processes are described in Table 62-1.

Table 62-1 Process Terminology

Term 

Description 

Process  

An instance of program in execution.  

Lightweight process (LWP)  

Is a virtual CPU or execution resource. LWPs are scheduled by the kernel to use available CPU resources based on their scheduling class and priority. LWPs include a kernel thread, which contains information that has to be in memory all the time and an LWP, which contains information that is swappable. 

Application thread  

A series of instructions with a separate stack that can execute independently in a user's address space. They can be multiplexed on top of LWPs. 

A process can consist of multiple LWPs and multiple application threads. The kernel schedules a kernel-thread structure, which is the scheduling entity in the SunOS 5.x environment. Various process structures are described in Table 62-2.

Table 62-2 Process Structures

Structure 

Description 

proc

Contains information that pertains to the whole process and has to be in main memory all the time. 

kthread

Contains information that pertains to one LWP and has to be in main memory all the time. 

user

Contains the per process information that is swappable. 

klwp

Contains the per LWP process information that is swappable. 

Figure 62-1 illustrates the relationship of these structures.

Figure 62-1 Process Structures

Graphic

Most process resources are accessible to all the threads in the process. Almost all process virtual memory is shared. A change in shared data by one thread is available to the other threads in the process.

Commands for Managing Processes

Table 62-3 describes commands for managing processes.

Table 62-3 Commands for Managing Processes

Use This Command ... 

To ... 

ps

Check the status of active processes on a system, as well as display detailed information about the processes 

dispadmin

List default scheduling policies 

priocntl

Assign processes to a priority class and manage process priorities 

nice

Change the priority of a timesharing process 

See Chapter 63, Managing Processes (Tasks) for more information about commands for managing processes.

In addition, process tools are available in /usr/proc/bin that display highly detailed information about the processes listed in /proc, also known as the process file system (PROCFS). Images of active processes are stored here by their process ID number.

The process tools are similar to some options of the ps command, except that the output provided by the tools is more detailed. In general, the process tools:

The new /usr/proc/bin utilities are summarized in Table 62-4.

Table 62-4 Process Tools

Tools That Control Processes 

What the Tools Do 
 

/usr/proc/bin/pstop pid

Stops the process 

/usr/proc/bin/prun pid

Restarts the process 

/usr/proc/bin/ptime pid

Times the process using microstate accounting 

/usr/proc/bin/pwait [-v] pid

Waits for specified processes to terminate 
 

Tools That Display Process Details 

What the Tools Display 
 

/usr/proc/bin/pcred pid

Credentials 

/usr/proc/bin/pfiles pid

fstat and fcntl information for open files

/usr/proc/bin/pflags pid

/proc tracing flags, pending and held signals, and other status information for each lwp

/usr/proc/bin/pldd pid

Dynamic libraries linked into each process 

/usr/proc/bin/pmap pid

Address space map 

/usr/proc/bin/psig pid

Signal actions 

/usr/proc/bin/pstack pid

Hex+symbolic stack trace for each lwp

/usr/proc/bin/ptree pid

Process trees containing specified pids 

/usr/proc/bin/pwdx pid

Current working directory 

In these commands, pid is a process identification number. You can obtain this number by using the ps -ef command.

Chapter 63, Managing Processes (Tasks), describes how to use the process tool commands to perform selected system administration tasks, such as displaying details about processes, and starting and stopping them. A more detailed description of the process tools can be found in proc(1).

If a process becomes trapped in an endless loop, or if it takes too long to execute, you may want to stop (kill) the process. See Chapter 63, Managing Processes (Tasks), for more information about stopping processes using the kill command.

Process Scheduling Classes and Priority Levels

A process is allocated CPU time according to its scheduling class and its priority level. By default, the Solaris operating system has four process scheduling classes: real-time, system, timesharing and interactive.

The scheduling priority determines the order in which processes will be run.

Real-time processes have fixed priorities. If a real-time process is ready to run, no system process or timesharing process can run.

System processes have fixed priorities that are established by the kernel when they are started. The processes in the system class are controlled by the kernel, and cannot be changed.

Timesharing and interactive processes are controlled by the scheduler, which dynamically assigns their priorities. You can manipulate the priorities of the processes within this class.

Disk I/O and System Performance

The disk is used to store data and instructions used by your computer system. You can examine how efficiently the system is accessing data on the disk by looking at the disk access activity and terminal activity. See Chapter 64, Monitoring Performance (Tasks) for a discussion of the iostat and sar commands, which report statistics on disk activity. Managing and allocating disk space and dividing your disk into slices are discussed in Chapter 21, Disk Management (Overview).

If the CPU spends much of its time waiting for I/O completions, there is a problem with disk slowdown. Some ways to prevent disk slowdowns are:

Memory and System Performance

Performance suffers when the programs running on the system require more physical memory than is available. When this happens, the operating system begins paging and swapping, which is costly in both disk and CPU overhead.

Paging involves moving pages that have not been recently referenced to a free list of available memory pages. Most of the kernel resides in main memory and is not pageable.

Swapping occurs if the page daemon cannot keep up with the demand for memory. The swapper will attempt to swap out sleeping or stopped lightweight processes (LWPs). If there are no sleeping or stopped LWPs, the swapper will swap out a runnable process. The swapper will swap LWPs back in based on their priority. It will attempt to swap in processes that are runnable.

Swap Space

Swap areas are really file systems used for swapping. Swap areas should be sized based on the requirements of your applications. Check with your vendor to identify application requirements.

Table 62-5 describes the formula used to size default swap areas by the Solaris 2.x installation program. These default swap sizes are a good place to start if you are not sure how to size your swap areas.

Table 62-5 Default Swap Sizes

If Your Physical Memory Size Is ... 

Your Default Swap Size Is ... 

16-64 Mbytes 

32 Mbytes 

64-128 Mbytes 

64 Mbytes 

128-512 Mbytes 

128 Mbytes 

greater than 512 Mbytes 

256 Mbytes 

See Chapter 30, Configuring Additional Swap Space (Tasks) for information about managing swap space.

Buffer Resources

The buffer cache for read and write system calls uses a range of virtual addresses in the kernel address space. A page of data is mapped into the kernel address space and the amount of data requested by the process is then physically copied to the process' address space. The page is then unmapped in the kernel. The physical page will remain in memory until the page is freed up by the page daemon.

This means a few I/O-intensive processes can monopolize or force other processes out of main memory. To prevent monopolization of main memory, balance the running of I/O-intensive processes serially in a script or with the at command. Programmers can use mmap(2) and madvise(3) to ensure that their programs free memory when they are not using it.

Kernel Parameters and System Performance

Many basic parameters (or tables) within the kernel are calculated from the value of the maxusers parameter. Tables are allocated space dynamically. However, you can set maximums for these tables to ensure that applications won't take up large amounts of memory.

By default, maxusers is approximately set to the number of Mbytes of physical memory on the system. However, the system will never set maxusers higher than 1024. The maximum value of maxusers is 2048, which can be set by modifying the /etc/system file.

See Chapter 66, Tuning Kernel Parameters (Tasks), and system(3S) for details on kernel parameters.

In addition to maxusers, a number of kernel parameters are allocated dynamically based on the amount of physical memory on the system, as shown in Table 62-6 below.

Table 62-6 Kernel Parameters

Kernel Parameter 

Description 

ufs_ninode

The maximum size of the inode table 

ncsize

The size of the directory name lookup cache 

max_nprocs

The maximum size of the process 

ndquot

The number of disk quota structures 

maxuprc

The maximum number of user processes per user-ID 

Table 62-7 lists the default settings for kernel parameters affected by the value assigned to maxusers.

Table 62-7 Default Settings for Kernel Parameters

Kernel Table 

Variable 

Default Setting 

Inode  

ufs_ninode

max_nprocs + 16 + maxusers + 64

Name cache  

ncsize

max_nprocs + 16 + maxusers + 64

Process  

max_nprocs

10 + 16 * maxusers

Quota table  

ndquot

(maxusers * NMOUNT) / 4 + max_nprocs

User process  

maxuprc

max_nprocs - 5

See Chapter 66, Tuning Kernel Parameters (Tasks), for a description of the kernel parameters and how to change the default values.

About Monitoring Performance

While your computer is running, counters in the operating system are incremented to keep track of various system activities. System activities that are tracked are:

Monitoring Tools

The Solaris 2.x system software provides several tools to help you keep track of how your system is performing. These include:

Table 62-8 Performance Monitoring Tools

The ...  

Enable(s) You To ... 

For More Information, See ... 

sar and sadc utilities

Collect and report on system activity data 

Chapter 64, Monitoring Performance (Tasks)

ps command

Display information about active processes 

Chapter 63, Managing Processes (Tasks)

Performance meter 

Display graphical representation of the status of systems on the network 

Chapter 64, Monitoring Performance (Tasks)

vmstat and iostat commands

Summarize system activity data, such as virtual memory statistics, disk usage, and CPU activity 

Chapter 64, Monitoring Performance (Tasks)

swap command

Display information about available swap space on your system 

Chapter 30, Configuring Additional Swap Space (Tasks)

netstat and nfsstat commands

Display information about network performance 

Chapter 65, Monitoring Network Performance (Tasks)

Solstice System Monitor (symon) 

Collect system activity data on UltraTM EnterpriseTM3000, 4000, 5000, and 6000 systems

Solstice System Monitor User's Guide