Go to main content

Using Oracle® Solaris 11.4 StatsStore and System Web Interface

Exit Print View

Updated: November 2020
 
 

Customizing System Web Interface

Customizations include preferences properties and customized sheets and visualizations. Administrators can set global system preferences and users can set their own individual user preferences.

To customize global preferences properties, see the following sections:

To customize sheets and visualizations or create new sheets, visualizations, and statistics for all users to see, see the following resources:

To customize or add sheets just for yourself, see Individual User Customizations.

Global Preferences Properties Files

The following file contains the default system preferences for the System Web Interface. These are mostly accessibility properties:

/usr/lib/webui/htdocs/solaris/preferences/global-prefs.json

Do not modify this file directly. Any modifications will be overwritten when the system is upgraded or as a result of running pkg fix.

To customize the preferences in global-prefs.json, deliver your customizations in one or more files in the following location:

/usr/lib/webui/htdocs/solaris/site/preferences

The files can have any names, but they must be in the same format as the global-prefs.json file and have the .json extension. See the webui-preferences-json(5) man page. Files in the htdocs/solaris/site/preferences directory are loaded after files in the htdocs/solaris/preferences directory. Files in the htdocs/solaris/site/preferences directory might be loaded in any order, so ensure that no property is set to different values in different files.

Create an Image Packaging System (IPS) package to deliver files to the htdocs/solaris/site/preferences directory. Installing a custom package is the easiest way to preserve your preferences across system upgrades. See Packaging and Delivering Software With the Image Packaging System in Oracle Solaris 11.4.

Global Preferences Service Properties

The webui/server service delivers properties that affect the System Web Interface for all users of this host, such as the default landing page or absolute timeout. The following command lists these properties:

$ svcprop -p conf -p coordinator webui/server:default

See the webui-service(7) man page for descriptions of these properties.

Allowing Alternative System Web Interface URLs

By default, users need to enter the following URL to get to the System Web Interface log-in page on host-name:

https://host-name:6787/solaris/

If the conf/redirect_root_url property of the webui/server:default service is set to true on host-name, then users can get to the System Web Interface log-in page by entering only the following URL:

https://host-name:6787

The value of the redirect_root_url property is true by default.

If the conf/redirect_from_https property of the webui/server:default service is set to true on host-name, then users can get to the System Web Interface log-in page by entering only host-name for the URL.

The default value of the redirect_from_https property is false. Use the following steps to set the redirect_from_https property to true on host-name:

$ svcprop -p conf/redirect_from_https webui/server:default
false
$ svccfg -s webui/server:default setprop conf/redirect_from_https=true
$ svccfg -s webui/server:default refresh
$ svcprop -p conf/redirect_from_https webui/server:default
true

Setting the Landing Page

By default, the landing page is the Solaris Dashboard. To change the landing page to a different page for all users on host-name, set the conf/default_landing_page property of the webui/server:default service. The following example changes the landing page to the Solaris Analytics page:

$ svcprop -p conf/default_landing_page webui/server:default
/solaris
$ svccfg -s webui/server:default setprop conf/default_landing_page=/solaris/apps/analytics
$ svccfg -s webui/server:default listprop conf/default_landing_page
conf/default_landing_page astring     /solaris/apps/analytics
$ svccfg -s webui/server:default refresh
$ svcprop -p conf/default_landing_page webui/server:default
/solaris/apps/analytics

Individual User Customizations

Whenever a user of the System Web Interface modifies the application behavior or adds new sheets, visualizations, or statistics that are stored persistently, these changes are stored as user preferences in the following file:

/var/user/user-name/webui/preferences/solaris.json

These user preferences are owned and readable only by that user or by administrators. Do not modify this file directly.

For example, when you use System Web Interface to change your home sheet in Solaris Analytics (right-click on the thumbnail of the sheet and select “Your Home Sheet”), an entry similar to the following is written to your preferences file:

$ cat /var/user/user-name/webui/preferences/solaris.json
{
    "_lastUpdated": 1603306041264,
    "apps": {
        "analytics": {
            "v1": {
                "homeSheet": "9af5e11f-6804-5a79-8918-39cbc85b57ac"
            }
        }
    }
}

Troubleshooting Retrieving Statistics

The following sections describe how to diagnose and fix certain issues.

Ensure Services Are Online

Verify that the system services associated with the statistics store and System Web Interface are online and not in maintenance.

If a service is not online, use the svcs -Lx command to investigate. See Troubleshooting Services Problems in Managing System Services in Oracle Solaris 11.4.

StatsStore Services

The Oracle Solaris 11.4 OS includes StatsStore; you do not need to install it. The statistics store, CLI, and APIs are delivered by the sstore package. The sstored daemon is invoked automatically during system startup and restarted if any failures occur.

Verify that both of the following services are online:

  • sstore – The sstored daemon captures statistics and events for the OS and applications.

  • sysstat – The sysstatd daemon provides common OS statistics to the statistics store.

$ svcs sstore sysstat
STATE          STIME    FMRI
online         13:55:28 svc:/system/sstore:default
online         13:55:28 svc:/system/sysstat:default

System Web Interface Service

The System Web Interface is delivered by the webui-server package. You might need to install the webui-server package:

$ pkg install webui-server

Make sure the svc:/system/webui/server:default service is online.

Resolving System Web Interface Problems

Check the following log files:

  • System Web Interface log files in svcs -L webui/server and /var/webui/logs

  • Apache web server log files in svcs -L http:apache24 and /var/apache2/2.4/logs

Ensure the Certificate and Key are Installed

A certificate, key, and certificate signing request (CSR) are generated at system start from properties of the svc:/system/identity:cert service.

The generated certificate is self signed and might fail validation. Users can follow the instructions from their browser to add an exception to accept this certificate.

To configure your own SSL certificate chain, see How to Deliver a Custom Certificate and Key.

The following command shows that the identity:cert service has a certificate property group, the certificate property group has ca and cert child property groups, and the cert property group has a private_key child property group.

$ svccfg -s identity:cert listprop certificate/*
certificate/generate                            boolean     false
certificate/ca                                  application
certificate/ca/pem_value                        astring     "pem_value"
certificate/ca/uri                              astring     /etc/certs/localhost/host-ca/hostca.crt
certificate/cert                                application
certificate/cert/hash                           astring     sha256
certificate/cert/keylen                         integer     2048
certificate/cert/keytype                        astring     rsa
certificate/cert/lifetime                       astring     10-year
certificate/cert/pem_value                      astring     "pem_value"
certificate/cert/subject                        astring
certificate/cert/uri                            astring     /etc/certs/localhost/host.crt
certificate/cert/private_key                    application
certificate/cert/private_key/pem_value          astring     "pem_value"
certificate/cert/private_key/read_authorization astring     solaris.smf.read.identity
certificate/cert/private_key/uri                astring     /etc/certs/localhost/host.key

When the value of the certificate/generate property is true, the properties in the certificate/cert property group are used to set the certificate and CSR metadata. The default value of certificate/cert/subject is CN=fqdn.

A host CA is generated (certificate/ca), and the host certificate is signed by that host CA, which ensures that the services can come online initially. A CSR also is generated, which you can use to get a certificate signed by your own CA service and replace the generated certificate. The /etc/certs/localhost/host-ca/hostca.crt host CA certificate is linked into /etc/certs/CA so that it is trusted by at least local TLS clients. The host and CA certificates get a randomly generated serial number.

The CA, certificate, and private key are stored in the pem_value properties and in the locations given by the uri properties. Do not modify the uri properties.

How to Deliver a Custom Certificate and Key

Use this procedure to replace the generated self-signed certificate configuration with your own properly signed certificate from a certificate authority (CA).

The certificate and key are single PEM-encoded X.509 strings. The specified certificate and key are installed during system installation or sysconfig when the value of the certificate/generate property is false.

Follow this procedure carefully, noting the following cautions:

  • Modifying any of the uri property values will cause the system to fail.

  • Setting only the certificate pem_value or only the key pem_value, but not both, will cause the system to fail.

  • Setting both the certificate pem_value and the key pem_value when the value of the certificate/generate property is true will cause the system to fail.

  1. Verify that the value of the certificate/generate property is false.
    $ svcprop -p certificate/generate identity:cert
    true

    If necessary, set the value to false:

    $ svccfg -s identity:cert
    svc:/system/identity:cert> setprop certificate/generate=false
    certificate/generate boolean     false
  2. Set the certificate PEM string.
    svc:/system/identity:cert> setprop certificate/cert/pem_value = astring: \
    "$(cat /path/to/signed/certificate.crt )"
  3. Set the key PEM string.
    svc:/system/identity:cert> setprop certificate/cert/private_key/pem_value = astring: \
    "$(cat /path/to/signed/certificate.key )"
  4. Set the CA PEM string.
    svc:/system/identity:cert> setprop certificate/ca/pem_value = astring: \
    "$(cat /path/to/issuer/certificate.crt )"
  5. Refresh the service.
    svc:/system/identity:cert> refresh
    svc:/system/identity:cert> exit
  6. Restart the service.
    $ svcadm restart -sr svc:/system/identity:cert

    Because of SMF dependencies, restarting the svc:/system/identity:cert service causes the svc:/system/webui/server service to be restarted also.

Degraded Performance

If your System Web Interface performance is degraded, try one of the following steps:

  • Close some instances of the System Web Interface if you have multiple tabs or browsers open.

  • Check the performance impact of data you are showing. Select the Properties option from the Visualization Actions menu, and note the value of “Performance impact” at the bottom of the panel. If you can, reduce the number of expensive statistics you are monitoring.

    You can also check for expensive statistics at the command line:

    $ sstore info //:class.net/ip//:stat.out-bytes
     Identifier: //:class.net/ip//:stat.out-bytes
       agg-name: bytes
    description: bytes sent
     partitions: link
     partitions: protocol
     partitions: local-address
     partitions: remote-address
     partitions: application
           ssid: //:class.net/ip//:stat.out-bytes
          units: bytes
      stability: unstable
      expensive: True
           type: counter
  • Flush the browser cache.

  • Restart the browser.

No Matches Found for Statistic

A “no matches found” error message is shown in the following cases:

  • The class, resource, or statistic does not exist.

  • The SSID is specified incorrectly.

  • You used a wildcard in an SSID that is classified as not browsable or not stable.

SSID Does Not Exist

In the following example, resource identifiers 0-3 exist, but the specified identifier, 4, does not exist:

$ sstore list //:class.cpu//:res.id/4
Warning (//:class.cpu//:res.id/4) - lookup error: no matches found
$ sstore list //:class.cpu//:res.id/*
IDENTIFIER
//:class.cpu//:res.id/0
//:class.cpu//:res.id/1
//:class.cpu//:res.id/2
//:class.cpu//:res.id/3

SSID is Incorrectly Specified

In the following example, a colon character was omitted from the delimiter in front of the operation:

$ sstore capture //:class.proc//:res.*//:stat.cpu-percentage//op.top
TIME                VALUE IDENTIFIER
Warning (//:class.proc//:res.*//:stat.cpu-percentage//op.top) - lookup error: no matches found
$ sstore capture //:class.proc//:res.*//:stat.cpu-percentage//:op.top
TIME                VALUE IDENTIFIER
2016-05-16T17:33:22 0.20751953125 //:class.proc//:res.sstored/1696/root//:stat.cpu-percentage//:op.top
...

SSID is Unbrowsable or Unstable

If an SSID is classified as unbrowsable or unstable, use one of the following methods to get information about that SSID:

  • Specify the full SSID explicitly

  • If you want to use a wildcard in the SSID to match multiple SSIDs, use the -a option with the sstore subcommand.

The following example shows that the kstat class is not browsable, though you can get information when you specify the SSID without the wildcard.

$ sstore list //:class.kstat*
Warning (//:class.kstat*) - lookup error: no matches found
$ sstore info //:class.kstat
 Identifier: //:class.kstat
description: kstat free-for-all namespace
  stability: stable
  browsable: False

All resources in the kstat class are classified as unstable.

$ sstore list //:class.kstat//:res.*
Warning (//:class.kstat//:res.*) - lookup error: no matches found

You can get information or record data if you fully specify the SSID.

$ sstore info //:class.kstat//:res.system/cpu/0/intr
 Identifier: //:class.kstat//:res.system/cpu/0/intr
  stability: unstable
       type: Untyped
description: CPU Interrupt statistics

If you need to use a wildcard, use the -a option to get information or record data.

$ sstore list -a //:class.kstat//:res.system/cpu/0/intr//:*
IDENTIFIER
//:class.kstat//:res.system/cpu/0/intr//:stat.crtime
//:class.kstat//:res.system/cpu/0/intr//:stat.level-count
//:class.kstat//:res.system/cpu/0/intr//:stat.level-time
//:class.kstat//:res.system/cpu/0/intr//:stat.snaptime
$ sstore info //:class.kstat//:res.system/cpu/0/intr//:stat.snaptime
           Identifier: //:class.kstat//:res.system/cpu/0/intr//:stat.snaptime
            stability: unstable
                 type: counter
kstat name/value type: Relative time
          description: kstat snapshot time
          kstat scale: 1000000000

Some, but not all, //:class.net/ip statistics are classified as unstable.

The following command records values for eight statistics:

$ sstore capture //:class.net/ip//:stat.*

The following command records values for 26 statistics:

$ sstore capture -a //:class.net/ip//:stat.*

If you name the statistic explicitly, you can get information or record data without using the -a option.

$ sstore info //:class.net/ip//:stat.out-bytes
 Identifier: //:class.net/ip//:stat.out-bytes
   agg-name: bytes
description: bytes sent
 partitions: link
 partitions: protocol
 partitions: local-address
 partitions: remote-address
 partitions: application
       ssid: //:class.net/ip//:stat.out-bytes
      units: bytes
  stability: unstable
  expensive: True
       type: counter
$ sstore capture //:class.net/ip//:stat.out-bytes
TIME                VALUE IDENTIFIER
2016-05-18T15:15:06 1201563.0 //:class.net/ip//:stat.out-bytes

The sstore export command does not require the -a option because that data is already available.

$ sstore export //:class.net/ip//:stat.*
TIME                VALUE IDENTIFIER
2016-05-18T14:10:41 34731 //:class.net/ip//:stat.in-delivers

Size of the StatsStore Repository

An important feature of the StatsStore repository is that the data you request (whether through the System Web Interface, the CLI, or APIs) is saved, enabling you to later compare current and historical values. To prevent the StatsStore repository from growing too large, a size limit is set, and older data is pruned from the repository to stay within the defined limit.

You can prune data, and you can change the size limit of the repository.

Pruning Data from the StatsStore Repository

The sstored daemon prunes data by saving fewer and fewer samples of older data. The same set of data might be pruned multiple times as necessary to keep the repository within the size limit as new data is added. For example, if you had recorded some data every second, sstored might prune older data to one value every five seconds, then one value every minute, then one value every hour and so forth as needed.

You can do some data pruning yourself by using the sstoreadm purge command.

The following command deletes all data for both ssid-1 and ssid-2. If an SSID does not explicitly name statistics (for example the SSID only names a resource), then all statistic data associated with that SSID is deleted:

$ sstoreadm purge ssid-1 ssid-2

The following command deletes all data for ssid-1 that was collected between start-time and end-time. Data collected before start-time or after end-time is retained:

$ sstoreadm purge -t start-time -e end-time ssid-1

Specify only an end time and not a start time to delete data up to the end time and retain data after the end time. Specify only a start time and not an end time to delete data from the start time to the current time and retain data before the start time.

The following command deletes all data for ssid-1 that was collected between start-time and end-time and that is in increments of 60 seconds or less. Data collected in increments of 61 seconds or larger is retained:

$ sstoreadm purge -t start-time -e end-time -i 60 ssid-1

You can also use –i with just and end time and no start time, with just a start time and no end time, and with no start or end time.

See also the sstoreadm(1) man page.

Changing the StatsStore Repository Size Limit

The default maximum size of the StatsStore repository is 2 gigabytes. You can specify a higher or lower maximum size by setting the config/max-repo-size property in the svc:/system/sstore service.

Use the zfs list command to check the current usage:

$ zfs list rpool/VARSHARE/sstore
NAME                    USED  AVAIL  REFER  MOUNTPOINT
rpool/VARSHARE/sstore  26.8M   245G  26.8M  /var/share/sstore/repo

Like the config/repo-path property, the config/max-repo-size property is a global property of the sstore service that will be inherited by the sstore:default service instance. The max-repo-size property is not defined in the sstore service manifest. The first time you set the max-repo-size property, you must include the type of the property. The type of the max-repo-size property is integer and the value is in megabytes.

The following command shows that the max-repo-size property has never been set:

$ svccfg -s sstore
svc:/system/sstore> listprop config
config/repo-path astring     /var/share/sstore/repo

The following command sets the max-repo-size property:

svc:/system/sstore> setprop config/max-repo-size = integer: 4096
svc:/system/sstore> listprop config
config/max-repo-size integer     4096
config/repo-path     astring     /var/share/sstore/repo

By default, the svcprop command shows the composed view of properties. The max-repo-size property will not show until you refresh the service instance:

$ svcprop -p config sstore:default
config/repo-path astring /var/share/sstore/repo
$ svcadm refresh sstore:default
$ svcprop -p config sstore:default
config/max-repo-size integer 4096
config/repo-path astring /var/share/sstore/repo

For the new max-repo-size value to take effect, you must restart the service instance:

$ svcadm restart sstore:default

Run in Debug Mode

Another way to get more information about the statistics store is to run the statistics store daemon with the --debug and --log options:

# /usr/lib/sstored --debug --log --log-path=/var/tmp/log --no-cache --events --no-auth