Go to main content

Managing System Services in Oracle® Solaris 11.4

Exit Print View

Updated: February 2022
 
 

SMF Concepts and Components

This section defines terms that are used in the remainder of this guide.

The following figure shows the primary components of the SMF framework. When you boot an image, SMF updates the service configuration repository if necessary, reads the repository data, and starts enabled service instances in correct dependency order. Independent services are started in parallel. When you shut down an image, services are shut down in reverse dependency order.

In the following figure, libscf is the library interface that the restarters use to interact with the service configuration repository. Interaction between the service configuration repository and libscf library interfaces is managed by the svc.configd daemon. The svcs, svcprop, svcadm, and svccfg commands are the interface that administrators use to interact with the service configuration repository.

Figure 1  Service Management Facility Framework

image:This figure shows the interaction of the CLI, SCF library interfaces, SMF daemon, repository, and restarters

SMF Service

An SMF service is a persistently running application that represents a system entity such as the following:

  • Application services such as a database or a Web server

  • Essential system services

  • The software state of a device

  • Kernel configuration information

  • Milestones that correspond to a level of system readiness

A service instance is a child of a service and provides capabilities and dependency relationships to applications and other service instances. Only instances have a state and can be started and stopped. If an instance fails for any reason, such as a hardware or software fault, SMF automatically detects the failure and restarts the instance and any dependent instances.

Instances of a service allow multiple configurations of a service to run simultaneously. Service instances inherit and customize common service configuration. For example, you can define a Web server service with one instance configured to listen on port 80 and another instance configured to listen on port 1008. Most services have a default instance. A few services do not have instances, such as some services that use SMF to store configurations but not to run programs. For example, the x11/x11-server service does not have any instances.

An SMF service is described in a file called a service manifest. The manifest describes service instances, dependencies, configuration properties, and methods. Service methods start, stop, and refresh service instances. A method can be a daemon, other binary executable, or an executable script. A service profile file enables you to customize an existing service, primarily by adding properties and adding and overriding property values. The new properties and values are layered over the values assigned in the manifest, as described in Repository Layers. See Service Bundles for more information about manifests and profiles. A profile is also an excellent tool for applying the same custom configuration to multiple systems, as described in Creating SMF Profiles.

Service information is stored in the service configuration repository, which is also called the SMF database. The service configuration repository stores the current state of each service instance on the system and the configuration data for each service and service instance. The data is stored in layers according to how values were modified, as described in Repository Layers.

SMF provides actions that you can invoke on a service instance, including enable, disable, refresh, and restart. Each service instance is managed by a restarter, which performs these administrative actions. In general, restarters perform actions by executing methods to move the service instance from one state to another state. For more information about restarters, see Service Restarters.

A milestone service is a special type of service that represents a level of system readiness such as a system init state. A milestone is a service that other service instances depend on to start. For example, run levels are represented by milestone services such as svc:/milestone/multi-user-server. Milestones also can be used to indicate the readiness of a group of services, such as svc:/milestone/devices, svc:/milestone/network, or svc:/milestone/name-services. The svc:/milestone/goals service defines the set of services that need to be running in order for the system to function as intended. See Chapter 7, Creating a Service that Notifies if Conditions are not Satisfied in Developing System Services in Oracle Solaris 11.4.

Service Models

SMF services are one of the following models:

Transient service

The service does some work and then exits without starting any long running processes.

Child or wait service

The service is restarted whenever its child process exits cleanly. A child process that exits cleanly is not treated as an error.

Contract or daemon service

The service starts a long running daemon or starts several related processes that are tied together as part of a service contract. The contract service manages processes that it starts and any dependent services and their start order. You only need to manage the high-level service.

Goal service

The service notifies administrators if any of its dependent services is not running. See Chapter 7, Creating a Service that Notifies if Conditions are not Satisfied in Developing System Services in Oracle Solaris 11.4.

Periodic services are a special case that do not fit any of these models. A periodic or scheduled service starts short-running processes at regular or scheduled intervals, staying online between runs when no associated contracted processes exist. See Chapter 3, Creating a Service to Run Periodically in Developing System Services in Oracle Solaris 11.4 and Chapter 4, Creating a Service to Run on a Specific Schedule in Developing System Services in Oracle Solaris 11.4 for more information.

Service Names

Each service and service instance is represented by a Fault Management Resource Identifier (FMRI). The full FMRI for a service instance has the following format:

svc:/service_name:instance_name

The service_name is a hierarchical name such as network/dns/client or application/pkg/server. Components of the service_name that precede the final forward slash character (/) are the category of the service. Categories such as application, device, milestone, network, and system help identify the purpose of the service.

The site category is reserved to help you avoid name conflicts when you create your own SMF services. For example, a site-specific service named svc:/site/tool will not conflict with an Oracle Solaris service named svc:/tool.

Service instance names are appended to the parent service name after a colon character. For example, svc:/system/identity:node and svc:/system/identity:domain are instances of the svc:/system/identity service.

In scripts, best practice is to use the full service instance name. Interactively, names can be shortened to the rightmost portions of the name that result in a unique name. For example, svc:/system/identity can be shortened to identity, and svc:/system/identity:domain can be shortened to identity:domain. Instance names must be preceded by some portion of the service name, followed by a colon character.

Service States

At any particular time, an SMF service instance is in one of the following states:

  • degraded – The instance is running or available to run, but is functioning at a limited capacity.

  • disabled – The instance is not enabled and is not running or available to run.

  • incomplete – The instance is not completely installed on the system and has not been evaluated by its restarter. The instance is mentioned solely in a profile (and not explicitly marked complete in the profile), or is mentioned only as a dependent of another service.

  • maintenance – The instance is enabled but not able to run. The instance might be transitioning through the maintenance state because an administrative action has not yet completed. Otherwise, administrative action is required to resolve the problem.

  • offline – The instance is enabled but not running or available to run. For example, if the dependencies of an enabled service are not satisfied, the service is kept in the offline state. Failure of the start method can leave the instance in the degraded or maintenance state. Clearing the maintenance state (svcadm clear) places the instance in the uninitialized state so that the restarter can re-evaluate the instance.

  • online – The instance is enabled and running or available to run. The online state is the expected operating state for a correctly configured service instance with all dependencies satisfied.

  • uninitialized – This state is the initial state for all service instances, including newly created instances (through executing svccfg add, importing a manifest, or applying a profile). When an instance is cleared from the maintenance state (svcadm clear), it is placed in the uninitialized state so that its restarter can re-evaluate its configuration. Instances are moved to maintenance, offline, or disabled state after evaluation by the appropriate restarter. Note that evaluation of an instance can only occur if its restarter service is online.

A service instance transitions between states depending on conditions such as administrative actions or the state of its dependent services. For example, when you enable an instance that was in the disabled state, the newly-enabled instance first transitions into the offline state, and transitions into the online state when all of its dependencies are satisfied.

In addition to the current state, administrators can show the auxiliary state. Restarters (see Service Restarters) use the auxiliary state to store information about the state. The master restarter uses the auxiliary state to store the reason the instance transitioned to its current state. For example, after a transition to the online state, the value of the auxiliary state is typically dependencies_satisfied. The periodic restarter uses the auxiliary state to store whether the periodic task is currently running, as described in Creating a Periodic Service in Developing System Services in Oracle Solaris 11.4.

See the smf(7) man page for more information about these service states and about how service instances transition through these states.

Service Dependencies

A service can have a dependency on a service, a service instance, or a file. Service dependencies define relationships between services.

Dependency relationships determine when a service starts and automatically stops. When dependencies of an enabled service are not satisfied, the service is in the offline state. When dependencies of an enabled service are satisfied, the service is started. If the service start is successful, the service transitions to the online state.

Service dependencies are reevaluated as services transition through states. Service dependencies that are satisfied can later become not satisfied. File dependencies are evaluated only one time.

Dependencies can be required or optional. Service dependencies can be required to be running or disabled. A dependent service can be configured to restart or not when one of its service dependencies is stopped or refreshed.

Dependency relationships allow the following capabilities:

  • Scalable and reproducible initialization processes

  • Faster system startup on systems that have parallel capabilities by starting independent services in parallel

  • Precise fault containment and fault recovery by restarting only services that are directly affected by a fault, and restarting those services in correct dependency order

Service Restarters

Each SMF service instance is managed by a restarter. The restarter retrieves instance configuration and provides an execution environment. See the smf_restarter(7) man page for information common to all restarters.

Master Restarter Daemon

The svc.startd daemon is the master restarter daemon for SMF and the default restarter for all service instances. The svc.startd daemon manages states for all service instances and their dependencies. As dependencies are satisfied when instances move to the online state, the master restarter invokes start methods of other instances or directs the delegated restarter to invoke the start method. The master restarter stops a service instance when the dependencies of the instance are no longer satisfied. The restarter attempts to restart an instance if the instance fails. Because an instance cannot be online until all of its dependencies are satisfied, the dependencies of an instance help determine the restart behavior of the instance. Properties set on each dependency declaration define whether that dependency is required and in what cases the instance will be restarted if the dependency is restarted.

Among other tasks, the svc.startd daemon starts the appropriate /etc/rc*.d scripts at the appropriate run levels, which is work that was previously done by init.

The following example shows that svc.startd is the restarter for the network/ipmp:default service instance. Other output has been omitted from this example.

$ svcs -l ipmp:default
restarter   svc:/system/svc/restarter:default

If the restarter property is empty or set to svc:/system/svc/restarter:default, the service instance is managed by svc.startd. For more information about the svc.startd daemon, see the svc.startd(8) man page.

Delegated Restarters

Some services have a set of common behaviors on startup. A delegated restarter can provide a specific execution environment and application-specific restarting behavior for these services. The delegated restarter specified by the restarter property is responsible for managing the service instance once that restarter is available.

Oracle Solaris includes the following delegated restarters:

inetd

The inetd delegated restarter can start Internet services on demand, rather than having the services always running. The inetd restarter provides its service instances with an environment composed of a network connection as input and output file descriptors. For more information about the inetd daemon, see the inetd(8) man page. The following example shows that inetd is the restarter for the cups/in-lpd:default service instance. Other output has been omitted from this example.

$ svcs -l cups/in-lpd:default
restarter   svc:/network/inetd:default
svc.periodicd

The periodic restarter daemon, svc.periodicd, is invoked automatically at system startup as part of the svc:/system/svc/periodic-restarter service and is automatically restarted if any failures occur. Services started by the periodic restarter remain online persistently but run their start method tasks only periodically or at scheduled times. Start method tasks of a periodic service are expected to run for a relatively brief period of time and then terminate. For more information about the periodic restarter, see the svc.periodicd(8) man page. For more information about periodic services, see Chapter 3, Creating a Service to Run Periodically in Developing System Services in Oracle Solaris 11.4.

svc.zones

Non-global zones are managed by the svc:/system/zones:default delegated restarter service. The following command shows that svc:/system/zones:default is the restarter for the z1 non-global zone:

$ svcs -R svc:/system/zones:default
STATE          STIME    FMRI
online         12:11:12 svc:/system/zones/zone:z1

For more information about the zones restarter, see the svc.zones(8) man page. For more information about zones services, see Creating and Using Oracle Solaris Zones.

Service Properties and Property Groups

Information about services, including dependencies, methods, state, and application data, is stored in the service configuration repository as a set of properties. Properties can be defined on either the service or an instance of the service. Properties that are set on the service are inherited by all instances of that service. Properties that are set on an instance are used only by that instance. Service instances can customize the values of inherited properties and can define additional properties that are not defined for the parent service.

Properties are organized into property groups. Some common property groups include:

  • general – Contains information such as whether the instance is enabled

  • restarter – Contains runtime information that is stored by the restarter for the service, including the current state of the instance

  • start, refresh, stop – Contains information such as which program to execute to start, refresh, or stop the service

  • config – Used by service developers to hold application data.

A property group can be parented by a service or instance, or by another property group. A property group that is the child of another property group is also called a nested property group. Nested property groups more fully express interrelationships among configuration data.

See the smf(7) man page for more information about properties and property groups.

Service Configuration Repository

Information about each service is stored in the service configuration repository, which is also called the SMF database. The service configuration repository stores information as services, instances, property groups, and properties. In addition to information defined by service developers, the service configuration repository stores information such as the start time and current state of each service instance on the system.

The repository stores persistent configuration information as well as SMF runtime data for services.

  • Persistent configuration information is stored in layers according to the source of the data. See Repository Layers.

  • Runtime data, or non-persistent configuration information, is not preserved across reboot, and the repository does not store layer information for non-persistent data. Non-persistent data generally hold an active program state.

The repository also stores service template data, such as types, value constraints, and descriptions of properties. Template data is defined in the service manifest. See the smf_template(7) man page for more information about template data.

The service configuration repository can only be manipulated or queried by using SMF interfaces. Use the svcs, svcprop, svcadm, and svccfg commands or the Service Configuration Facility library functions listed in the libscf(3LIB) man page. You can read and write property values and show property values in specified layers and snapshots. For information about layers, see Repository Layers. For information about snapshots, see Repository Snapshots. You can show only the properties of the selected service instance or parent service, or you can show a composed view of properties. In a composed view, both properties set on the parent service and properties set on the service instance are shown; values shown are the values set on the service instance.

Service Bundles

A service bundle is an XML file that contains the information that is stored in the service configuration repository for a service or service instance. Information provided in service bundles is stored in the service configuration repository and can be exported from the repository. Service bundles in standard locations are imported into the repository during system boot.

The two types of service bundles are manifests and profiles.

Manifests

Manifests contain the complete set of properties associated with a specific set of services or service instances.

Profiles

Profiles typically provide customization of a service or service instance that augments or overrides information provided in the manifest. Examples of customizations include additional properties and changed property values.

The standard location for manifests is /lib/svc/manifest. The standard location for profiles is /etc/svc/profile.

When the system is booted or the manifest import service is restarted, manifests are imported and profiles are applied if they are new or changed. An IPS package that delivers a service bundle can specify that the manifest import service should be restarted when the package is installed.

Local customizations can be provided in profile files with an .xml suffix in the /etc/svc/profile/site directory. If the same property in the same repository layer for the same service or instance is defined by multiple manifests or profiles, SMF cannot determine which value to use. When this type of conflict is detected, the instance is placed in the maintenance state. See Repository Layers for more information about layers.

In addition to delivering services into Oracle Solaris, service bundles can also deliver custom configuration across a variety of systems.

A system profile, /etc/svc/profile/generic.xml, is applied during installation. Do not change this profile. Any changes made to this system profile will be overwritten on upgrade. See the smf_bootstrap(7) man page for more information.

Repository Layers

The service configuration repository can store different values for a single property. The repository stores data in layers according to the source of the data. The source can be manifests, profiles, and customizations made by using SMF commands and library interfaces. You can view values in different layers to understand the source of the value that is in use in the running configuration: whether a value was assigned in the manifest, in a profile, or was changed by an administrator. See Showing the Layer Where a Value Is Set.

Configuration changes made by using SMF commands and library interfaces appear only in the admin layer. Configuration in other layers is defined in profile and manifest files in standard locations. When a property is added to the repository from a file or a property value is changed in a file, you can show the name of the file that provided that configuration by using the -f or -o file options of the svccfg listprop command. See Showing the File that Contributed the Configuration.

If a property has different values assigned at different layers, the value that is used by the service instance is the value in the highest layer of the layer hierarchy. The following table shows the order of layers in the hierarchy. For example, if a property has a value in the node-profile layer, that value overrides the value in the manifest layer or any other lower layer. If a property has a value in the admin layer, that value overrides all other values set in any other layer.

Layer
Content
admin
Any changes that are made by an administrator using the SMF commands or by an application using SMF library interfaces. The admin layer also includes any changes that are made by importing a manifest or applying a profile from a non-standard location. See Importing and Applying Manifests and Profiles for caution about the use of non-standard locations.
sysconfig-profile
Changes that are made by an administrator by using the sysconfig command interactively, and configuration that is specified in a profile that is installed by an Automated Installer (AI) client or by a zone. You are using the sysconfig command interactively when you use System Configuration Interactive (SCI) Tool. See Creating Configuration Profiles in Customizing Automated Installations With Manifests and Profiles for information about using SCI Tool.
AI client profiles (created by installadm create-profile or installadm update-profile), zone profiles (created by zoneadm install, zoneadm attach, or zoneadm clone), and profiles created by using the sysconfig command interactively are stored in the /etc/svc/profile/sysconfig directory unless otherwise specified. If you specify the -c dir option with the install, attach, or clone subcommand of the zoneadm command or with the sysconfig configure command, and dir has subdirectories enterprise, site, or node, configuration from profiles in those directories will be set in the appropriate enterprise-profile, site-profile, or node-profile layer.
When the sysconfig command is used interactively to configure the system, if property values are defined that are also defined in the admin layer, those property values are removed in the admin layer so that the new property values defined by using the sysconfig command are effective.
node-profile
Configuration that is specific to an Oracle Solaris instance. This configuration comes from SMF profiles in the /etc/svc/profile/node directory and from the legacy /etc/svc/profile/site.xml file. Do not add new configuration to the /etc/svc/profile/site.xml file.
site-profile
Configuration that is common to systems at the same location or site. This configuration comes from SMF profiles in the /etc/svc/profile/site directory. Note that /var/svc/profile is deprecated as a standard location. Do not put new profiles in the /var/svc/profile directory.
enterprise-profile
Configuration that applies to all of your Oracle Solaris systems. This configuration comes from SMF profiles in /etc/svc/profile/enterprise.
system-profile
Any values from the /etc/svc/profile/generic.xml and /etc/svc/profile/platform.xml system profiles as well as any profiles in the /etc/svc/profile/system directory.
manifest
Values from manifests in the /lib/svc/manifest and /var/svc/manifest directories. Do not put new service manifests in the /var/svc/manifest directory.

Configuration conflicts are not permitted within any layer. Configuration that is set by using SMF commands, the sysconfig command, or SMF library interfaces overwrites the previous setting. If conflicting configuration is delivered by multiple files in any single layer, and is not set at a higher layer, the manifest-import service log reports the conflict, and the service with the conflicting configuration is not started. See Conflicting Configuration for more information.

You can specify the layer of configuration data to view and therefore identify which data are administrative customizations and which data were delivered with the software. When a client does not specify the layer from which to retrieve configuration data, the topmost layer data is provided. The topmost layer is determined by the order shown in the above table, where the admin layer is the topmost layer and the manifest layer is the lowest priority layer. If a property has a value in the admin layer, that is the value that the repository delivers. In this way, local customizations are preferred over the values that were provided when the system was installed.

Repository Snapshots

The repository captures a read-only snapshot of each service each time the service is successfully started. These snapshots enable you to easily return to a previous working state if necessary. The following snapshots might be available for any given instance:

initial

Initial configuration when the service and its instances were imported for the first time. An initial snapshot is not created if a profile starts the service or instance before manifest import.

previous

Current configuration captured when a manifest import is performed for a service that has already been delivered. The service could have already been delivered by the manifest being imported or by another manifest.

running

The running configuration of the service instance. When you change configuration data, use the svcadm refresh or svccfg refresh command to promote the new values to the running snapshot.

start

Configuration captured during a successful transition to the online state.

Repository Backups

SMF automatically takes the following backups of the service configuration repository:

  • The boot backup is taken immediately before the first change to the repository is made during each system startup.

  • The manifest_import backups occur before svc:/system/early-manifest-import:default or svc:/system/manifest-import:default completes, if the service imported any new manifests or ran any upgrade scripts.

Four backups of each type are maintained by the system, with the oldest backups deleted as necessary.

You can restore the repository from one of these backups. See How to Restore a Repository From Backup.