Go to main content

Administering Resource Management in Oracle® Solaris 11.3

Exit Print View

Updated: March 2018
 
 

Producing Reports With rcapstat

Use rcapstat to report resource capping statistics. Monitoring Resource Utilization With rcapstat explains how to use the rcapstat command to generate reports. That section also describes the column headings in the report. The rcapstat(1) man page also contains this information.

The following subsections use examples to illustrate how to produce reports for specific purposes.

Reporting Cap and Project Information

In this example, caps are defined for two projects associated with two users. user1 has a cap of 50 megabytes, and user2 has a cap of 10 megabytes.

The following command produces five reports at 5-second sampling intervals.

user1system$ rcapstat 5 5
    id project  cappd  nproc     vm    rss   cap    at   avgat    pg  avgpg
112270   user1   Yes      24   123M    35M   50M   50M      0K 3312K     0K
 78194   user2   Yes       1  2368K    1856K 10M    0K      0K    0K     0K
    id project  cappd  nproc     vm    rss   cap    at   avgat    pg    avgpg
112270   user1    Yes     24   123M    35M   50M    0K      0K    0K       0K
 78194   user2    Yes      1  2368K  1856K   10M    0K      0K    0K       0K
    id project  cappd  nproc     vm    rss   cap    at    avgat   pg    avgpg
112270   user1    Yes     24   123M    35M   50M    0K      0K    0K       0K
 78194   user2    Yes      1  2368K  1928K   10M    0K      0K    0K       0K
    id project  cappd  nproc     vm    rss   cap    at    avgat   pg    avgpg
112270   user1    Yes     24   123M    35M   50M    0K      0K    0K       0K
 78194   user2    Yes      1  2368K  1928K   10M    0K      0K    0K       0K
    id project  cappd  nproc     vm    rss   cap    at    avgat   pg    avgpg
112270   user1    Yes     24   123M    35M   50M    0K      0K    0K       0K
 78194   user2    Yes      1  2368K  1928K   10M    0K    0K    0K         0K 

The first three lines of output constitute the first report, which contains the cap and project information for the two projects and paging statistics since rcapd was started. The at and pg columns are a number greater than zero for user1 and zero for user2, which indicates that at some time in the daemon's history, user1 exceeded its cap but user2 did not.

The subsequent reports show no significant activity.

Monitoring the RSS of a Project

The following example uses project user1, which has an RSS in excess of its RSS cap.

The following command produces five reports at 5-second sampling intervals.

user1system$ rcapstat 5 5
    id project   cappd  nproc       vm       rss       cap    at     avgat     pg  avgpg
376565   user1   Yes       3     6249M     6144M     6144M  690M      220M  5528K  2764K
376565   user1   Yes       3     6249M     6144M     6144M    0M      131M  4912K  1637K
376565   user1   Yes       3     6249M     6171M     6144M   27M      147M  6048K  2016K
376565   user1   Yes       3     6249M     6146M     6144M 4872M      174M  4368K  1456K
376565   user1   Yes       3     6249M     6156M     6144M   12M      161M  3376K  1125K

The user1 project has three processes that are actively using physical memory. The positive values in the pg column indicate that rcapd is consistently paging out memory as it attempts to meet the cap by lowering the physical memory utilization of the project's processes. However, rcapd does not succeed in keeping the RSS below the cap value. This is indicated by the varying rss values that do not show a corresponding decrease. As soon as memory is paged out, the workload uses it again and the RSS count goes back up. This means that all of the project's resident memory is being actively used and the working set size (WSS) is greater than the cap. Thus, rcapd is forced to page out some of the working set to meet the cap. Under this condition, the system will continue to experience high page fault rates, and associated I/O, until one of the following occurs:

  • The WSS becomes smaller.

  • The cap is raised.

  • The application changes its memory access pattern.

In this situation, shortening the sample interval might reduce the discrepancy between the RSS value and the cap value by causing rcapd to sample the workload and enforce caps more frequently.


Note -  A page fault occurs when either a new page must be created or the system must copy in a page from a swap device.

Determining the Working Set Size of a Project

The following example is a continuation of the previous example, and it uses the same project.

The previous example shows that the user1 project is using more physical memory than its cap allows. This example shows how much memory the project workload requires.

user1system$ rcapstat 5 5
    id project  cappd  nproc    vm   rss   cap    at  avgat    pg  avgpg
376565   user1    Yes      3 6249M 6144M 6144M  690M    0K   689M     0K
376565   user1    Yes      3 6249M 6144M 6144M    0K    0K     0K     0K
376565   user1    Yes      3 6249M 6171M 6144M   27M    0K    27M     0K
376565   user1    Yes      3 6249M 6146M 6144M 4872K    0K  4816K     0K
376565   user1    Yes      3 6249M 6156M 6144M   12M    0K    12M     0K
376565   user1    Yes      3 6249M 6150M 6144M 5848K    0K  5816K     0K
376565   user1    Yes      3 6249M 6155M 6144M   11M    0K    11M     0K
376565   user1    Yes      3 6249M 6150M   10G   32K    0K    32K     0K
376565   user1    Yes      3 6249M 6214M   10G    0K    0K     0K     0K
376565   user1    Yes      3 6249M 6247M   10G    0K    0K     0K     0K
376565   user1    Yes      3 6249M 6247M   10G    0K    0K     0K     0K
376565   user1    Yes      3 6249M 6247M   10G    0K    0K     0K     0K
376565   user1    Yes      3 6249M 6247M   10G    0K    0K     0K     0K
376565   user1    Yes      3 6249M 6247M   10G    0K    0K     0K     0K
376565   user1    Yes      3 6249M 6247M   10G    0K    0K     0K     0K

Halfway through the cycle, the cap on the user1 project was increased from 6 gigabytes to 10 gigabytes. This increase stops cap enforcement and allows the resident set size to grow, limited only by other processes and the amount of memory in the system. The rss column might stabilize to reflect the project working set size (WSS), 6247M in this example. This is the minimum cap value that allows the project's processes to operate without continuously incurring page faults.

While the cap on user1 is 6 gigabytes, in every 5-second sample interval the RSS decreases and I/O increases as rcapd pages out some of the workload's memory. Shortly after a page out completes, the workload, needing those pages, pages them back in as it continues running. This cycle repeats until the cap is raised to 10 gigabytes, approximately halfway through the example. The RSS then stabilizes at 6.1 gigabytes. Since the workload's RSS is now below the cap, no more paging occurs. The I/O associated with paging stops as well. Thus, the project required 6.1 gigabytes to perform the work it was doing at the time it was being observed.

Also see the vmstat(1M) and iostat(1M) man pages.