smf - service management facility
The Oracle Solaris service management facility defines a programming model for providing persistently running applications called services. The facility also provides the infrastructure in which to run services. A service can represent a running application, the software state of a device, or a set of other services. Services are represented in the framework by service instance objects, which are children of service objects. Instance objects can inherit or override the configuration of the parent service object, which allows multiple service instances to share configuration information. All service and instance objects are contained in a scope that represents a collection of configuration information. The configuration of the local Oracle Solaris instance is called the “localhost” scope, and is the only currently supported scope.
Each service instance is named with a fault management resource identifier (FMRI) with the scheme svc:. For example, the syslogd(8) daemon started at system startup is the default service instance named:
svc://localhost/system/system-log:default svc:/system/system-log:default system/system-log:default
Many commands also allow FMRI abbreviations. See the svcs(1) man page for one such example.
In the above example, default is the name of the instance and system/system-log is the service name. Service names can comprise multiple components separated by slashes (/). All components, except the last, compose the category of the service. Site-specific services should be named with a category beginning with site.
A service instance is either enabled or disabled. All services can be enabled or disabled with the svcadm(8) command.
The list of managed service instances on a system can be displayed with the svcs(1) command.
When an administrator deletes an entity backed by a manifest or profile in a standard location, that entity is masked, and is not seen by normal queries to SMF. Masked entity can be explored using svccfg listcust, and removed using the delcust subcommand to svccfg. For more information, see the svccfg(8) man page.
An instance name must begin with an alphanumeric character and can contain alphanumeric characters, underscore (_), hyphen (-), and dot (.). One comma (,) is allowed between the first and last character for names. For example:
ORCL, rcapd com.oracle, rcapd
Service names allow multiple level of identifiers, each under the same restrictions as instance names, to be delimited by slash (/). For example, system/svc/restarter.
Only ASCII characters are allowed.
Service instances can have dependencies on a set of entities which can include services, instances, and files. Dependencies govern when the service is started and automatically stopped. When the dependencies of an enabled service are not satisfied, the service is kept in the offline state. When its dependencies are satisfied, the service is started. If the start is successful, the service is transitioned to the online state. Unlike services and instances, file dependencies are not evaluated dynamically as files are created or deleted. They are evaluated only one time.
Whether a dependency is satisfied is determined by its grouping:
Satisfied when all cited services are running (online or degraded), or when all indicated files are present.
Satisfied when one of the cited services is running (online or degraded), or when at least one of the indicated files is present.
Satisfied if the cited services are running (online or degraded) or do not run without administrative action (disabled, maintenance, not present, or offline waiting for dependencies which do not start without administrative action). Incomplete services also satisfy optional dependencies.
Defines a startup order if the cited services are present. Generally, it is unsatisfied if the cited services are present, enabled, and not yet running. It is satisfied if the cited services are in online, degraded, disabled, maintenance, or are absent or incomplete state. It is also satisfied if the cited services are offline due to an absent dependency or an unsatisfied dependency that is in the disabled or maintenance state. This dependency type is not recommended for file dependencies, but if used it will behave as a require_all dependency.
Satisfied when all of the cited services are disabled, in the maintenance state, or when cited services or files are not present.
Once running (online or degraded), if a service cited by a require_all, require_any, or optional_all dependency is stopped or refreshed, the SMF considers why the service was stopped and the restart_on attribute of the dependency to decide whether to stop the service.
| restart_on value event | none error restart refresh -------------------+------------------------------ stop due to error | no yes yes yes non-error stop | no no yes yes refresh | no no no yes
A service is considered to have stopped due to an error if the service has encountered a hardware error or a software error such as a core dump. For exclude_all dependencies, the service is stopped if the cited service is started and the restart_on attribute is not none.
The dependencies on a service can be listed with svcs(1) or svccfg(8), and modified with svccfg(8).
Each service is managed by a restarter. The master restarter, svc.startd(8) manages states for the entire set of service instances and their dependencies. The master restarter acts on behalf of its services and on delegated restarters that can provide specific execution environments for certain application classes. For instance, inetd(8) is a delegated restarter that provides its service instances with an initial environment composed of a network connection as input and output file descriptors. Each instance delegated to inetd(8) is in the online state. While the daemon of a particular instance might not be running, the instance is available to run.
As dependencies are satisfied when instances move to the online state, svc.startd(8) invokes start methods of other instances or directs the delegated restarter to do so. These operations might overlap.
The current set of services and associated restarters can be examined using svcs(1). A description of the common configuration used by all restarters is given in smf_restarter(7).
Each service or service instance must define a set of methods that start, stop, and, optionally, refresh the service. See smf_method(7) for a more complete description of the method conventions for svc.startd(8) and similar fork(2)-exec(2) restarters.
Administrative methods, such as for the capture of legacy configuration information into the repository, are discussed on the svccfg(8) manual page.
The methods for a service can be listed and modified using the svccfg(8) command.
Each service instance is always in a well-defined state based on its dependencies, the results of the execution of its methods, and its potential contracts events. The following states are defined:
This state is the initial state for all service instances, including newly created instances (through executing svccfg add, importing a manifest, or applying a profile). When an instance is cleared from the maintenance state (svcadm clear), it is placed in the uninitialized state so that its restarter can re-evaluate its configuration. Instances are moved to maintenance, offline, or disabled state after evaluation by the appropriate restarter. Note that evaluation of an instance can only occur if its restarter service is online.
The instance is not completely installed on the system and has not been evaluated by its restarter. The instance is mentioned solely in a profile (and not explicitly marked complete in the profile), or is mentioned only as a dependent of another service.
The instance is enabled, but not yet running or available to run. If restarter execution of the service start method or the equivalent method is successful, the instance moves to the online state. Failures might lead to a degraded or maintenance state. Administrative action can lead to the uninitialized state.
The instance is enabled and running or is available to run. The specific nature of the online state is application-model specific and is defined by the restarter responsible for the service instance. Online is the expected operating state for a properly configured service with all dependencies satisfied. Failures of the instance can lead to a degraded or maintenance state. Failures of services on which the instance depends can lead to offline or degraded states.
The instance is functioning at a limited capacity in comparison to normal operation. Failures of the instance can lead to the maintenance state. Failures of services on which the instance depends can lead to offline or degraded states. Failure of instance methods, can also lead to the degraded state. Restoration of capacity should result in a transition to the online state.
The instance was not able to start, stop, or continue running. Administrative action through svcadm clear, after corrective steps are taken is required to move the instance out of the maintenance state. When an instance is cleared, SMF will transition the instance to its configured state. If the instance is disabled, SMF will move it to the disabled state. If the instance is enabled, SMF will attempt to bring the instance online.
The instance is disabled. Enabling the service results in a transition to the offline state and eventually to the online state with all dependencies satisfied.
This state represents a legacy instance that is not managed by the service management facility. Instances in this state have been started at some point, but might or might not be running. Instances can only be observed using the facility and are not transferred into other states.
States can also have transitions that result in a return to the originating state.
SMF allows notification by using SNMP or SMTP of state transitions. It publishes Information Events for state transitions which are consumed by notification daemons like snmp-notify(8) and smtp-notify(8). SMF state transitions of disabled services do not generate notifications unless the final state for the transition is disabled and there exist notification parameters for that transition. Notification is not be generated for transitions that have the same initial and final state.
Notification parameters for FMA Events are stored in svc:/system/fm/notify-params:default except for Information Events generated by SMF state transitions. Those are stored in the service or in the instance of the transitioning service. Notification parameters for SMF state transition generated events can be set system wide in svc:/system/svc/global:default. The system wide notification parameters are used when a composed lookup, as in scf_instance_get_pg_composed(3SCF), in the transitioning instance cannot be found. Notification parameters can be manipulated using svccfg(8). Notification parameters can be configured in a service manifest or profile using the notification_parameters element described in the DTD. An example is provided below:
<notification_parameters>
<event value='from-online' />
<type name='smtp' active="false">
<parameter name='to'>
<value_node value='root@local' />
<value_node value='admin-alias@eng' />
</parameter>
</type>
<type name='snmp' />
</notification_parameters>
events is a comma separated list of SMF state transition sets or a comma separated list of FMA event classes. events cannot have a mix of SMF state transition sets and FMA event classes.
For convenience, the tags problem- {diagnosed,updated,repaired,resolved} describe the life cycle of a problem diagnosed by the FMA subsystem - from initial diagnosis to interim updates and finally problem closure. These tags are aliases for underlying FMA protocol event classes (all in the list.* hierarchy), but the latter should not be used in configuring notification preferences.
A new problem has been diagnosed by the FMA subsystem. The diagnosis includes a list of one or more suspects, which (where appropriate) might have been automatically isolated to prevent further errors occurring. The problem is identified by a UUID in the event payload, and further events describing the resolution life cycle of this problem quote a matching UUID.
One or more of the suspect resources in a problem diagnosis has been repaired, replaced or acquitted (or has been faulted again), but there remains at least one faulted resource in the list. A repair could be the result of an fmadm command line (fmadm repaired, fmadm acquit, fmadm replaced) or might have been detected automatically such as through detection of a part serial number change.
All of the suspect resources in a problem diagnosis have been repaired, resolved or acquitted. Some or all of the resources might still be isolated at this stage.
All of the suspect resources in a problem diagnosis have been repaired resolved or acquitted and are no longer isolated (for example, a CPU that was a suspect and in an offline state, is now back online again; this un-isolate action is usually automatic).
State Transition Sets are defined as:
Set of all transitions that have <state> as the final state of the transition.
Set of all transitions that have <state> as the initial state of the transition.
Set of all transitions that have <state> as the initial state of the transition.
Set of all transitions.
Valid values of state are maintenance, offline, disabled, online and degraded. An example of a transitions set definition: maintenance, from-online, to-degraded.
The dependencies, methods, delegated restarter, and instance state mentioned above are represented as properties or property groups of the service or service instance. A service or service instance has an arbitrary number of property groups in which to store application data, and those property groups may be nested within other property group elements allowing for the representation of complex hierarchical application configuration data. Using property groups in this way allows the configuration of the application to derive the attributes that the repository provides for all data in the facility. The application can also use the appropriate subset of the service_bundle(5) DTD to represent its configuration data within the framework.
Property lookups are composed. If a property group-property combination is not found on the service instance, most commands and the high-level interfaces of libscf(3LIB) search for the same property group-property combination on the service that contains that instance. This allows common configuration among service instances to be shared. Composition can be viewed as an inheritance relationship between the service instance and its parent service.
Properties are protected from modification by unauthorized processes. See smf_security(7).
The general property group applies to all service instances. It includes the following properties:
Specifies whether the instance is enabled. If this property is not present on an instance, SMF does not tell the instance's restarter about the existence of the instance.
The restarter for this service. See the Restarters section for more information. If this property is unset, the default system restarter is used.
Whether this service is complete or is a partial definition that should not be started. This property is automatically set on manifest import. Alternatively, an instance without this property that successfully validates against the template definitions (see scf_tmpl_validate_fmri(3SCF)) will have this property created by svcadm(8) on enable.
When set to true, activates the behavior of goal services. For more information, see the Goal Services section for details.
The repository is assembled from a combination of administrative customization, current state, and default values from files in standard locations. Services, instances, property groups, and properties defined by manifests in SMF-managed filesystem locations are always accurately represented in the repository. Customizations made during runtime by administrators or other programs are captured and stored in the repository.
A property can have different values in the repository which reflect different settings from manifests, profiles, and administrative customizations. Which one is presented to the user and service by default is arbitrated by a simple priority scheme called layers.
Seven layers are tracked by SMF. In decreasing priority order, they are:
Any change made by interactive use of SMF commands or libraries. This layer has the highest priority.
Any values from the files in the /etc/svc/profile/sysconfig directory. This directory is exclusively used by the Oracle Solaris installer and sysconfig. Certain sysconfig(8) operations may remove all profiles from this directory.
Any values from the files in the /etc/svc/profile/node directory or the legacy /etc/svc/profile/site.xml file. These files are intended to represent configuration that is unique to a particular instance of Oracle Solaris.
Any values from the files in the /etc/svc/profile/site directory. These files are intended to represent configuration that is common to many systems located in a common location.
Any values from the files in the /etc/svc/profile/enterprise directory. These files are intended to represent configuration that is common to most systems in a distributed population of the Oracle Solaris instances.
Any values from the /etc/svc/profile/system directory and the files /etc/svc/profile/generic.xml and /etc/svc/profile/platform.xml
Any values from the system manifest locations /lib/svc/manifest or /var/svc/manifest.
Property conflicts are not permitted within any individual layer. A conflicting property in the admin layer simply overwrites the previous property. If the same property is delivered by multiple files in any other layer, and is not set at a higher layer, the entire instance is tagged as in-conflict, and are not started by svc.startd(8) until the conflicting definition is removed or the property is set at a higher layer. Other libscf consumers requesting a single value, including svccfg and svcprop, see a random property setting from amongst all appropriate values. We do not guarantee which of the conflicting values is returned.
The enterprise-profile, site-profile, and node-profile layers are intended to allow generally useful settings to be applied across a large number of systems, while allowing easy overrides for groups of systems and individual systems. For example, a company might have a set of DNS services that is typically used by all servers, but require specialized settings for certain sites or individual machines. Profiles can be selectively delivered into the various layers using the zoneadm(8) install. For more information, see the solaris(7) and the solaris-kz(7) man pages.
Historical data about each instance in the repository is maintained by the service management facility. This data is made available as read-only snapshots for administrative inspection and rollback. The following set of snapshot types might be available:
Initial configuration of the instance created by the administrator or produced during package installation.
Current configuration captured when an administrative undo operation is performed.
The running configuration of the instance.
Configuration captured during a successful transition to the online state.
The svccfg(8) command can be used to interact with snapshots.
Some property groups are marked as non-persistent. These groups are not backed up in snapshots and their content is cleared during system boot. Such groups generally hold an active program state which does not need to survive system restart.
The current state of each service instance, as well as the properties associated with services and service instances, is stored in a system repository managed by svc.configd(8).
The repository for service management facility data is managed by svc.configd(8).
The information associated with a service or service instance that is stored in the configuration repository can be exported as XML-based files. Such XML files, known as service bundles, are portable and suitable for backup purposes. Service bundles are classified as one of the following types:
Files that contain the complete set of properties associated with a specific set of services or service instances.
Files that contain a set of service instances and values for the enabled property (type boolean in the general property group) on each instance.
Profiles can also contain configuration values for properties in services and instances. Template elements cannot be defined in a profile.
Profiles can use a relaxed set of elements from the DTD described in service_bundle(5). To use these, the DOCTYPE entry should have the following definitions added:
<!ENTITY % profile "INCLUDE"> <!ENTITY % manifest "IGNORE">
Service bundles can be imported or exported from a repository using the svccfg(8) command. See service_bundle(5) for a description of the service bundle file format with guidelines for authoring service bundles.
An smf milestone is a service that aggregates a multiple service dependencies. Usually, a milestone does nothing useful itself, but declares a specific state of system-readiness on which other services can depend. One example is the name-services milestone, which simply depends upon the currently enabled name services.
Existing milestones on Oracle Solaris include: none, config, devices, unconfig, network, single-user, name-services, self-assembly-complete, multi-user, and multi-user-server.
System administrators may use goal services to define the expected set of services that should be running when a system is up and functioning for its intended purpose.
Goal services are expected to have all their dependencies satisfiable. If administrative intervention is required to have a dependency satisfied, the goal service is placed in the maintenance state. Goal services that are in the maintenance state will automatically leave that state once their dependencies are satisfiable.
In general, the goal service setting should not be used for services that perform actual work.
It is not recommended to have goal services depending on dynamically enabled services. Dynamically enabled services will lead a goal service into the maintenance state until they are enabled by another service. The admin can use svcadm(8) subcommand goals to set the dependencies of goal services.
milestone/goalsThe milestone/goals is a goal service with the purpose of providing a clear, unambiguous, and well-defined point where a system can be considered up and running. The dependencies of milestone/goals must be configured to represent the mission critical services for the system. The default dependency of milestone/goals is:
svc:/milestone/multi-user-server:default
Startup programs in the /etc/rc?.d directories are executed as part of the corresponding run-level milestone:
milestone/single-user:default
milestone/multi-user:default
milestone/multi-user-server:default
Execution of each program is represented as a reduced-functionality service instance named by the program's path. These instances are held in a special legacy-run state.
These instances do not have an enabled property (type boolean in the general property group) and, generally, cannot be manipulated with the svcadm(8) command. No error diagnosis or restart is done for these programs.
svcs(1), exec(2), fork(2), strftime(3C), libscf(3LIB), scf_tmpl_validate_fmri(3SCF), contract(5), service_bundle(5), smf_bootstrap(7), smf_method(7), smf_restarter(7), smf_security(7), inetd(8), smtp-notify(8), snmp-notify(8), svc.configd(8), svc.startd(8), svcadm(8), svccfg(8), solaris(7), solaris-kz(7), zoneadm(8)