5 Understanding Server Life Cycle

Server life cycle is the series of states through which a WebLogic Server instance can transition. These states cause specific changes to the operational state of a server instance and help to identify the accurate status of the running server. Use the server life cycle commands to track the progress of a booting server at a granular level.

Diagram of the Server Life Cycle

Server life cycle consists of several components including startup parameters and state transitions.

Figure 5-1 illustrates the server life cycle and the relationships between states and life cycle commands.

Figure 5-1 State Transitions for Server Life Cycle Commands

Description of "Figure 5-1 State Transitions for Server Life Cycle Commands"

To understand each state and the relationships among states and server life cycle commands, see Understanding Server States in the Server Life Cycle and Using Server Life Cycle Commands.

Getting and Using Server State

Server state signifies the specific condition of a server in the life cycle management. System administrators use the server state information to plan the administration tasks related to the application services. You can get the server state using Administration Console or command prompt scripts.

WebLogic Server displays and stores information about the current state of a server instance, and state transitions that have occurred since the server instance started up. This information is useful to administrators who:

Monitor the availability of server instances and the applications they host
Perform day-to-day operations tasks, including startup and shutdown procedures
Diagnose problems with application services
Plan corrective actions, such as migration of services, when a server instance fails or crashes

Get server state as follows:

WebLogic Server Administration Console—Multiple pages display state information:
- On the Summary of Servers page (Environment > Servers), the Servers table displays the current state of each server instance in the current domain.
- The SERVER_NAME > Monitoring page displays the state of the currently running server instance, and the date and time it entered the state.
- Diagnostics > Log Files, includes timestamped messages for state transitions that have occurred since the server instance was last started.
Programmatically—Use the getState() method on the server's weblogic.management.runtime.ServerRuntimeMBean. For example, to monitor the progress of a long-running graceful shutdown process, issue a getstate inquiry on a separate thread. For more information, see ServerRuntimeMBean in MBean Reference for Oracle WebLogic Server.
WebLogic Scripting Tool—See Getting Run-Time Information in Understanding the WebLogic Scripting Tool.

Understanding Server States in the Server Life Cycle

A server instance works independently in each server state enabling the administrator to track the current status of the server. The states include SHUTDOWN, STARTING, STANDBY, ADMIN, and more.

These sections describe each state in the WebLogic Server life cycle.

SHUTDOWN State

In the SHUTDOWN state, a WebLogic Server instance is configured but inactive.

A server instance enters the SHUTDOWN state as result of a Shutdown or Force Shutdown command. In addition, a server instance can kill itself when it detects, as a result of self-health monitoring, that it has become unstable. Only a server instance with its Auto Kill If Failed attribute is true will kill itself when it detects that it is failed. See Automatic Restart for Failed Server Instances.

You can transition a server instance in the SHUTDOWN state to the STARTING state with the Start, Start in Admin, or Start in Standby commands.

STARTING State

During the STARTING state, a WebLogic Server instance transitions from SHUTDOWN to STANDBY, as a result of a Start, Start in Admin, or Start in Standby command.

In the STARTING state, a server instance cannot accept any client or administrative requests.

The server instance obtains its configuration data:

An Administration Server retrieves domain configuration data, including the domain security configuration, from its config directory.
A Managed Server contacts the Administration Server for its configuration and security data. If the Managed Server is configured for SSL communications, it uses its own certificate files, key files, and other SSL-related files and contacts the Administration Server for the remaining configuration and security data.

Note:

If the Managed Server cannot contact its Administration Server, by default, it starts up in Managed Server Independence mode, using its locally cached copy of the domain config directory. See Understanding Managed Server Independence Mode.

Do not add non-configuration files in the config directory or subdirectories. Non-configuration files include log (.log) and lock (.lck) files. Administration Server replicates the config directory in all Managed Server instances. Storing non-configuration files in the config directory can cause performance issues in the domain.

The server instance starts the services listed in Table 5-1, in the order listed.

Table 5-1 Services Started in STARTING State

Service	Function
`weblogic.management.provider.internal.BeanInfoAccessService`
`weblogic.management.PropertyService`
`weblogic.management.internal.DomainDirectoryService`
`weblogic.upgrade.domain.DomainUpgradeServerService`
`weblogic.management.upgrade.ConfigurationMigrationService`
`weblogic.deploy.service.internal.DeploymentService`
`weblogic.management.provider.internal.RuntimeAccessDeploymentReceiverService`
`weblogic.management.provider.internal.RuntimeAccessService`
`weblogic.diagnostics.lifecycle.DiagnosticInstrumentationService`
`weblogic.t3.srvr.BootService`	Includes basic services such as kernel, execute queues, and the server run time.
`weblogic.management.provider.internal.DomainAccessService`	The root for Administration Server-only services.
`weblogic.diagnostics.lifecycle.DiagnosticFoundationService`	The container service for logging and debugging.
`weblogic.nodemanager.NMService`	The Node Manager service, responsible for reporting changes to server status to Node Manager via the server output stream.
`weblogic.timers.internal.TimerService`
`weblogic.rjvm.RJVMService`	During shutdown, closes all RJVMs except the Administration Server connection.
`weblogic.protocol.ProtocolService`
`weblogic.t3.srvr.DomainLibService`	Registers configured protocols, making them available for outbound traffic and inbound configuration. Managed Servers require this service to be available early in the startup sequence, to allow them to provide correct addressing information to the Administration Server.
`weblogic.server.channels.ChannelService`	This service is dependent on consistent configuration, and protocols being registered. By this point in the startup sequence, all protocols should have been registered. After this service starts, addressing information, such as `ServerChannelManager.findDefaultLocalServer Channel()`, is available.
`weblogic.server.channels.AdminPortService`
`weblogic.t3.srvr.ListenerService`
`weblogic.transaction.internal.PrimordialTransactionService`	The transaction helper is initialized, providing utilities that associate transactions with threads, obtaining the Transaction Manager, obtain the `UserTransaction` object, and perform other tasks. The transaction service itself is not enabled at this point in the startup sequence.
`weblogic.rmi.internal.RMIServerService`	The RMI boot service that is used for initialization only.
`weblogic.jndi.internal.NamingService`
`weblogic.iiop.IIOPClientService`	Installs VM-wide delegates.
`weblogic.management.PrimordialManagementService`
`weblogic.ldap.EmbeddedLDAP`
`weblogic.security.SecurityService`
`weblogic.jndi.internal.RemoteNamingService`
`weblogic.security.acl.internal.RemoteSecurityService`
`weblogic.rmi.cluster.RemoteBinderFactoryService`
`weblogic.cluster.ClusterService`
`weblogic.iiop.IIOPService`
`weblogic.protocol.ProtocolHandlerService`
`weblogic.management.internal.AdminService`
`weblogic.xml.registry.XMLService`
`weblogic.messaging.interception.MessageInterceptionService`
`weblogic.cluster.migration.rmiservice.MigratableRMIService`
`weblogic.messaging.interception.configuration.Configurator`
`weblogic.drs.internal.DataReplicationService`
`weblogic.management.provider.internal.EditAccessService`	Start the Management Edit Service.
`weblogic.health.HealthMonitorService`
`weblogic.cluster.migration.MigrationService`
`weblogic.t3.srvr.T3InitializationService`	Initializes deprecated T3 server services such as `BootServicesImpl.`
`weblogic.server.channels.ChannelRuntimeService`	Addressing information, such as `ServerRuntime.getListenAddress()`, and dynamic updates are available after this point in the startup sequence.
`weblogic.store.admin.DefaultStoreService`
`weblogic.transaction.internal.TransactionService`
`weblogic.jdbc.common.internal.JDBCService`
`weblogic.connector.common.ConnectorService`
`weblogic.store.admin.StoreDeploymentService`
`weblogic.jms.JMSServiceServerLifeCycleImpl`
`weblogic.jms.BridgeService`
`weblogic.application.ApplicationShutdownService`	Checks pending application work during graceful shutdown. Applications are also shutdown here.
`weblogic.messaging.saf.internal.SAFServerService`
`weblogic.ejb20.deployer.EJB20Service`
`weblogic.io.common.internal.FileService`
`weblogic.time.server.TimerService`	Cancels application triggers during shutdown.
`weblogic.rmi.internal.HeartbeatHelperService`	Supports heartbeats in protocol-only clients.
`weblogic.servlet.internal.WebService`
`weblogic.webservice.conversation.internal.ConversationServiceImpl`
`weblogic.wtc.gwt.WTCServerLifeCycleImpl`
`com.beasys.CORBA.pool.weblogic.WLECService`
`weblogic.management.service.ManagedServerNotificationService`
`weblogic.webservice.WSServerService`
`weblogic.management.mbeanservers.runtime.internal.RuntimeServerService`	Run-time JMX services.
`weblogic.management.mbeanservers.edit.internal.EditServerService`
`weblogic.management.mbeanservers.compatability.internal.CompatabilityMBeanServerService`
`weblogic.management.snmp.SNMPService`
`weblogic.management.deploy.classdeployment.ClassDeploymentService`	Adds handling of startup and shutdown classes.
`weblogic.server.ServerLifeCycleService`	Handles creation of the server life cycle run-time MBeans to allow for control of the domain.
`weblogic.server.channels.EnableAdminListenersService`	Enables Admin port before server goes into `ADMIN` state.
`domainweblogic.diagnostics.lifecycle.DiagnosticSystemService`

STANDBY State

A server instance in STANDBY does not process any request—its regular listen port is closed. The administration port is open, and accepts life cycle commands that transition the server instance to either the RUNNING or the SHUTDOWN state. Other Administration requests are not accepted.

Starting a server instance in standby is a method of keeping it available as a "hot" backup, a useful capability in high-availability environments.

The only life cycle command that causes a server instance to enter the STANDBY state and remain in that state is the Start in Standby command. A server instance transitions through the STANDBY state when you issue a Start or a Start in Admin command.

ADMIN State

In the ADMIN state, WebLogic Server is up and running, but available only for administration operations, allowing you to perform server and application-level administration tasks. When a server instance is in the ADMIN state:

The Administration Console is available.
The server instance accepts requests from users with the admin role. Requests from non-admin users are refused.
Applications are activated in the application ADMIN state. They accept requests from users with the admin and AppTester roles. Users with these roles, accessing an application in the application ADMIN state, have access to all application functionality, not just administrative functions.
The JDBC, JMS, and JTA subsystems are active, and administrative operations can be performed upon them. However, you do not have to have administrator-level privileges to access these subsystems when the server is in the ADMIN state.
Deployments or re-deployments are allowed, and take effect when you transition the server instance from the ADMIN to the RUNNING state (using the Resume command).
ClusterService is active and listens for heartbeats and announcements from other cluster members. It can detect that other Managed Servers have joined the cluster, but is invisible to other cluster members.

You can transition a server instance to the ADMIN state using the Start in Admin, Suspend, or Force Suspend commands.

A server instance transitions through the ADMIN state as a result of Start, Shutdown, and Force Shutdown commands.

You can transition a server instance in the ADMIN state to RUNNING with the Resume command, or to SHUTDOWN, with the Shutdown or Force Shutdown command.

RESUMING State

During this transitional state, WebLogic Server performs the operations required to move itself from the STANDBY or ADMIN state to the RUNNING state.

A server instance transitions to the RESUMING state when you issue the Resume command. A server instance transitions through the RESUMING state when you issue the Start command.

RUNNING State

In the RUNNING state, WebLogic Server is fully functional, offers its services to clients, and can operate as a full member of a cluster.

A server instance transitions to the RUNNING state as a result of the Start command, or the Resume command from the ADMIN or STANDBY states.

You can transition a server instance in the RUNNING state to the SUSPENDING state or the FORCE_SUSPENDING state using graceful and force Suspend and Shutdown commands.

Note:

You can use ReadyApp to determine when applications on the server instance are ready to accept requests. See Using the ReadyApp Framework in Deploying Applications to Oracle WebLogic Server.

SUSPENDING State

During this transitional state, WebLogic Server performs the operations required to place itself in the ADMIN state, suspending a subset of WebLogic Server subsystems and services in an ordered fashion, and completing a predefined portion of the application work currently in process ("in-flight" work).

A server instance transitions to the SUSPENDING state when you issue the Suspend command. A server instance transitions through the SUSPENDING state when you issue a Shutdown command.

For information about in-flight work, see Processing In-Flight Work During Suspend and Shutdown.

Note:

While in the SUSPENDING state, Work Managers complete in-flight processing for pending work in application threads. However, while in this state, you cannot schedule new work to Work Managers. See Understanding Work Managers in Administering Server Environments for Oracle WebLogic Server.

FORCE_SUSPENDING State

During this transitional state, WebLogic Server performs the operations required to place itself in the ADMIN state, suspending a subset of WebLogic Server subsystems and services in an ordered fashion. During the FORCE_SUSPENDING state, WebLogic Server does not complete in-flight work gracefully; application work in progress is abandoned.

A server instance transitions through the FORCE_SUSPENDING state when you issue the Force Suspend or Force Shutdown command.

SHUTTING_DOWN State

During this transitional state, WebLogic Server completes the suspension of subsystems and services and does not accept application or administration requests.

A server instance transitions to the SHUTTING_DOWN state when you issue a Shutdown or Force Shutdown command.

FAILED State

A running server instance can fail as a result of out-of-memory exceptions or stuck application threads, or if one or more critical services become dysfunctional. A server instance monitors its health, and upon detecting that one or more critical subsystems are unstable, it declares itself FAILED.

When a server instance enters the FAILED state, it attempts to return to a non-failed state. If it failed prior to reaching the ADMIN state, the server instance shuts itself down with an exit code that is less than zero. See WebLogic Server Exit Codes and Restarting After Failure.

If the server instance fails after reaching the ADMIN state, but before reaching the RUNNING state, by default, it returns to the ADMIN state, if the administration port is enabled.

Note:

If desired, you can configure a server instance that fails after reaching the ADMIN state, to shut itself down, rather than return to the ADMIN state.

A server instance can enter the FAILED state from any other state. However, once a server instance has entered the FAILED state, it cannot return to a running state directly. The FAILED state is fatal and a server must go into the ADMIN or SHUTDOWN state before returning to the RUNNING state.

Note:

It is theoretically possible that a server could become available again once the FAILED state is entered, for example if hung threads causing a failed state become un-hung.

However, once the FAILED state is entered, a server must go into the ADMIN or SHUTDOWN states before returning to RUNNING.

You can use the OverlaodProtectionMBean to configure automatic actions, such as SHUTDOWN or move to the ADMIN state, for when a server instance enters the FAILED state.

FAILED_NOT_RESTARTABLE State

This state indicates that the Managed Server has failed and one of the following scenarios has occurred:

The Node Manager AutoRestart attribute is set to false, and Node Manager will not restart the Managed Server.
Node Manager has exceeded the configured RestartMax value, which specifies the number of times Node Manager will attempt to restart a failed server within the interval defined by RestartInterval. See Server Startup Properties in Administering Node Manager for Oracle WebLogic Server.

If a Managed Server is in the FAILED_NOT_RESTARTABLE state, the root failure should be investigated. However, you can perform the same lifecycle operations as if the server instance was in the SHUTDOWN state (such as Start, Start in Admin, or Start in Standby commands) without any additional actions.

Using Server Life Cycle Commands

Server life cycle commands help to transition a server instance from one state to another.

This section describes each life cycle command, how to issue it, and its effect on the state of a server instance. For more information on:

How to issue life cycle commands, see:
- Life Cycle Commands and Managing Servers and Server Life Cycle in Understanding the WebLogic Scripting Tool.
- Start and stop servers in the Oracle WebLogic Server Administration Console Online Help.
- Starting and Stopping Servers.
Node Manager processing related to key life cycle events in environments that use Node Manager, see How Node Manager Works in the WebLogic Server Environment in the Administering Node Manager for Oracle WebLogic Server.
The processing that occurs during each life cycle state, see Understanding Server States in the Server Life Cycle.

For an illustration of the relationship between server states and server life cycle, see Figure 5-1.

Start

The Start command transitions a server instance from the SHUTDOWN state to the RUNNING state. Depending on the initial state of a server instance, the Start command causes these state transitions:

SHUTDOWN > STARTING > STANDBY > ADMIN > RESUMING > RUNNING

The ServerMBean.StartupMode attribute lets you specify the state in which a server instance should be started. Its values are displayed and configurable in the WebLogic Server Administration Console, using WLST, or when specified as a java weblogic.Server startup option. If you do not specify a startup mode value (either on the command line, in the WebLogic Server Administration Console, or in config.xml), the default is to start in the RUNNING state.

See Specify a startup mode in the Oracle WebLogic Server Administration Console Online Help and Other Server Configuration Options in Command Reference for Oracle WebLogic Server.

Command Usage

See start, startServer, and nmStart in WLST Command Reference for WebLogic Server and Start Managed Servers from the Administration Console in the Oracle WebLogic Server Administration Console Online Help.

Start in Standby

The Start command, with Standby mode enabled, transitions a server instance from the SHUTDOWN state to the STANDBY state, with this sequence of state transitions:

SHUTDOWN > STARTING > STANDBY

Command Usage

See -Dweblogic.management.startupmode in Command Reference for Oracle WebLogic Server and Start Managed Servers in Standby mode in the Oracle WebLogic Server Administration Console Online Help.

Start in Admin

The Start command, with Admin mode enabled, transitions a server instance from the SHUTDOWN state to the ADMIN state, with this sequence of state transitions:

SHUTDOWN > STARTING > STANDBY > ADMIN

Command Usage

See -Dweblogic.management.startupmode in Command Reference for Oracle WebLogic Server and Start Managed Servers in Admin mode in the Oracle WebLogic Server Administration Console Online Help.

Resume

The Resume command transitions a server instance from the STANDBY or ADMIN state to the RUNNING state, with this sequence of state transitions:

STANDBY > ADMIN > RESUMING > RUNNING

Command Usage

See resume in WLST Command Reference for WebLogic Server and Resume a server in the Oracle WebLogic Server Administration Console Online Help.

Graceful Suspend

The Graceful Suspend command transitions a server instance from the RUNNING state to the ADMIN state, allowing work in process to be handled gracefully, with this sequence of state transitions:

RUNNING > SUSPENDING > ADMIN

Command Usage

See suspend in WLST Command Reference for WebLogic Server and Suspend a server in the Oracle WebLogic Server Administration Console Online Help.

Force Suspend

The Force Suspend command transitions a server instance from the RUNNING state to the ADMIN state, without handling work in process gracefully, with this sequence of state transitions:

RUNNING > FORCE_SUSPENDING > ADMIN

Command Usage

See Suspend a server in the Oracle WebLogic Server Administration Console Online Help.

Graceful Shutdown

The Graceful Shutdown command transitions a server instance from the RUNNING state to the SHUTDOWN state, allowing work in process to be handled gracefully, with this sequence of state transitions:

RUNNING > SUSPENDING > ADMIN > SHUTTING_DOWN > SHUTDOWN

Command Usage

See shutdown in WLST Command Reference for WebLogic Server and Shut down a server instance in the Oracle WebLogic Server Administration Console Online Help.

Controlling Graceful Shutdown

ServerMBean has two attributes for controlling the length of the graceful shutdown process. Their values are displayed and configurable on the SERVER_NAME > Control > Start/Stop page:

Ignore Sessions During Shutdown—If you enable this option WebLogic Server will drop all HTTP sessions immediately, rather than waiting for them to complete or timeout. Waiting for abandoned sessions to timeout can significantly lengthen the graceful shutdown process, because the default session timeout is one hour.
Graceful Shutdown Timeout—Specifies a time limit for a server instance to complete a graceful shutdown. If you supply a timeout value, and the server instance does not complete a graceful shutdown within that period, WebLogic Server performs a forced shutdown on the server instance.

See Control graceful shutdowns and Shut down servers in a cluster in the Oracle WebLogic Server Administration Console Online Help.

Shutdown Operations and Application Undeployment

During both graceful and forced shutdown, subsystems undeploy applications as appropriate. This processing can result in invocation of application code, such as Servlet destroy() or ejbRemove() during shutdown. During the shutdown sequence, JMS, JDBC, and transactions are shutdown after applications are shutdown, allowing application code to access JMS, JDBC, and transaction services.

Force Shutdown

The Force Shutdown command transitions a server instance from the any state to the SHUTDOWN state, without allowing work in process to be handled gracefully. When run for a server instance in the RUNNING state, the Force Shutdown command results in these state transitions:

RUNNING > FORCE_SUSPENDING > ADMIN > STANDBY > SHUTDOWN

Command Usage

See shutdown in WLST Command Reference for WebLogic Server and Shutdown a server instance in the Oracle WebLogic Server Administration Console Online Help.

A forced shutdown is immediate—WebLogic Server subsystems stop all application processing currently in progress. A forced shutdown can be performed on a server instance in any state.

If a fatal exception causes the forced shutdown to fail, the server will exit after the number of seconds specified by the ServerLifecycleTimeoutVal attribute in ServerMBean.

Note:

When you force shutdown a server instance in a cluster, a clustered service will fail over to another server instance in the cluster, if its state is replicated on another server instance. However:

If you issue a Forced Shutdown command on a server instance that hosts an HTTP session for which a secondary session has not yet been created, the session will be lost.
If you issue a Forced Shutdown command on a server instance that hosts the replicated state of a stateful session EJB, and the server instance that hosts the EJB fails (the primary), the EJB will not fail over, because its replicated state no longer exists.

For information about undeployment processes during a forced shutdown, and related programming considerations, see Shutdown Operations and Application Undeployment.

Processing In-Flight Work During Suspend and Shutdown

Subsystems are self-contained systems and responsible for performing each work item in WebLogic Server. The subsystem includes the web containers and the various services associated with the server. Each subsystem follows a specific approach to handle the work during Suspend and Shutdown operations.

The following sections describe how each subsystem handles work in process during SUSPENDING and SHUTTING_DOWN operations.

RMI Subsystem

The Remote Method Invocation (RMI) subsystem suspends in three steps. Each step in this process completes before the following step commences.

Non-transaction remote requests are rejected by the Non-Transaction RMI Service.
The Client Initiated Transaction Service waits for pending client transactions to complete.
The Remote RMI Service rejects all remote requests with or without transactions.

After these steps are completed, no remote client requests are allowed. Requests with administrative privileges and internal system calls are accepted.

When a clustered server instance is instructed to prepare to suspend, the RMI system refuses any in-memory replication calls, to allow other cluster members to choose new hosts for replicated sessions.

Web Container

After the Web Container subsystem is instructed to prepare to suspend, it rejects new sessions requests. Existing sessions are handled according to the persistence method:

No persistence—Pending sessions with no persistence are allowed to complete.
In-memory replication in a cluster—Sessions with secondary sessions are immediately suspended. If a primary session does not have a secondary session, the Web Container waits until a secondary session is created, or until the session times out, whichever occurs first.
JDBC persistence and file persistence—The Web Container immediately suspends sessions that are replicated in a database or file store.

The completion of pending sessions is optional. To drop all sessions immediately, use the Ignore Sessions During Shutdown option on the SERVER_NAME > Control > Start/Stop page in the WebLogic Server Administration Console, or the -ignoreSessions option with the WLST shutdown command.

In a cluster, when a primary session is dropped, the corresponding replicated sessions on another clustered instance will be also destroyed, in addition to the primary session on the server that is being gracefully shut down.

Timer Service

The Timer Service cancels all triggers running on application execute queues. Application execute queues include the default queue and queues configured through the ExecuteQueueMBean.

Application Service

The Application Service completes pending work in the application queues before suspending. Application execute queues include the default queue and queues configured through the ExecuteQueueMBean.

EJB Container

The EJB Container suspends Message Drive Beans (MDBs).

JMS Service

The Java Messaging Service (JMS) marks itself as suspending, which causes new requests to be rejected. The JMS system suspends gracefully in this fashion:

If the server instance being shut down has a JMS server:

Any send requests that are waiting because of message quotas are returned immediately.
All consumers on destinations belonging the JMS Server are closed.
The persistent store is closed.

If the server instance being shutdown has a JMS connection factory:

Client connections are closed.

Generally each step in the graceful suspend of the JMS subsystem occurs quickly—in less than a second. Potentially, completion of a client request could take longer, if the request requires higher than normal disk I/O, for example, a request for a persistent "send" of a 100-megabyte message.

You can monitor the number of connections to a JMS server, the number of consumers to a JMS connection factory, and related run-time information using JMS run-time MBeans, including JMSRuntimeMbean, JMSConnectionRuntimeMBean, JMSConsumerRuntimeMBean.

JDBC Service

The JDBC Service closes idle connections in the connection pools.

Note:

If connections are still in use, the shutdown of the JDBC service will fail, and the graceful shutdown will not complete. To shut down a server instance while applications still hold connections, use a forced shutdown command, described in Force Shutdown.

Transaction Service

The Transaction Service waits for the pending transaction count in the Transaction Manager to drop to zero before suspending. Completing all pending transactions can be a lengthy process, depending on the configured transaction timeout.

If a graceful shutdown takes too long because of pending transactions, you can halt it with a forced shutdown command. Force Shutdown suspends all pending work in all subsystems.