C H A P T E R  5

SMS Domain Configuration

A dynamic system domain (DSD) is an independent environment, a subset of a server, that is capable of running a unique version of firmware and a unique version of the Solaris OS. Each domain is insulated from the other domains. Continued operation of a domain is not affected by any software failures in other domains or by most hardware failures in any other domain.

The system controller (SC) supports commands that enable you to logically group system boards into dynamic system domains, or simply domains, which are able to run their own OS and handle their own workload. Domains can be created and deleted without interrupting the operation of other domains. You can use domains for many purposes. For example, you can test a new OS version or set up a development and testing environment in a domain. In this way, if problems occur, the rest of your system is not affected.

You can also configure several domains to support different departments, with one domain per department. You can temporarily reconfigure the system into one domain to run a large job over the weekend.

The Sun Fire 15K system allows up to 18 domains to be configured. The Sun Fire 12K system allows up to 9 domains to be configured.

Domain configuration establishes mappings between the domains and the server's hardware components. Also included in domain configuration is the establishment of various system management parameters and policies for each domain. This chapter discusses all aspects of domain configuration functionality that the Sun Fire high-end system provides.

This chapter contains the following sections:


Domain Configuration Units

A domain configuration unit (DCU) is a unit of hardware that can be assigned to a single domain. DCUs are the hardware components from which domains are constructed. DCUs that are not assigned to any domain are said to be in no-domain.

All DCUs are system boards and all system boards are DCUs. The Sun Fire high-end system DCUs are:

Sun Fire high-end system hardware requires the presence of at least one regular system board, plus at least one of the I/O board types in each configured domain. csb, exb boards, and the SC are not DCUs.



Note - MaxCPU boards do not contain memory. To set up a domain, at least one regular CPU board is required.




Domain Configuration Requirements

You can create a domain out of any group of system boards, provided the following conditions are met:


DCU Assignment

The assignment of DCUs to a domain is the result of one of three logical operations acting on a DCU (system board):

Static Versus Dynamic Domain Configuration

Although there are logically three DCU assignment operations, the underlying implementation is based upon four domain configuration operations:

The first two domain configuration operations apply to inactive domains; that is, to domains that are not running OS software. These operations are called static domain configuration operations. The latter two domain configuration operations apply to active domains, that is, those running OS software, and are called dynamic domain configuration operations.

Dynamic domain configuration requires interaction with the domain's Solaris software to introduce or remove the DCU-resident resources such as CPUs, memory, or I/O devices from Solaris OS control. Sun Fire high-end system dynamic reconfiguration (DR) provides a capability called remote DR for an external agent, such as the SC, to request dynamic configuration services from a domain's Solaris environment.

The SC command user interfaces utilize remote DR as necessary to accomplish the requested tasks. Local automatic DR allows applications running on the domain to be aware of impending DR operations and to take action, as appropriate, to adjust to resource changes. This improves the likelihood of success of DR operations, particularly those which require active resources to be removed from domain use. For more information on DR, refer to the System Management Services (SMS) 1.6 Dynamic Reconfiguration User Guide.

When a domain is configured for local automatic DR, remote DR operations initiated from the SC benefit from the automation of DR operations for that domain. With local automatic DR capabilities available in Sun Fire domains, simple scripts can be constructed and placed in a crontab(1) file, allowing simple platform reconfigurations to take place on a time schedule.

SMS allows you to add boards to or remove boards from an active (running) domain. Initiation of a remote DR operation on a domain requires administrative privilege for that domain. SMS grants the ability to initiate remote DR on a domain to individual administrators on a per-domain basis.

The remote DR interface is secure. Since invocation of DR operations on the domain itself requires superuser privilege, remote DR services are provided only to known, authenticated remote agents.

The user command interfaces that initiate DCU assignment operations are the same whether the affected domains have local automatic DR capabilities or not.

SMS provides for the addition or removal of a board from an active domain, such as static domain configuration using addboard, deleteboard, and moveboard. For more information, refer to the System Management Services (SMS) 1.6 Dynamic Reconfiguration User Guide.

Global Automatic Dynamic Reconfiguration

Remote DR and local automatic DR functions are building blocks for a feature called global automatic DR. Global automatic DR introduces a framework that can be used to automatically redistribute the system board resources on a Sun Fire system. This redistribution can be based upon factors such as production schedule, domain resource utilizations, domain functional priorities, and so on. Global automatic DR accepts input from customers describing their Sun Fire resource utilization policies and then uses those policies to automatically marshal the Sun Fire high-end system resources to produce the most effective utilization. For more information on DR, refer to the System Management Services (SMS) 1.6 Dynamic Reconfiguration User Guide.


Configuration for Platform Administrators

This section briefly describes the configuration services available to the platform administrator.

Available Component List

Each domain (A-R) defaults to having a 0-board list of boards that are available to an administrator or configurator to assign to their respective domains. Boards can be added to the available component list of a domain by a platform administrator using the setupplatform(1M) command. Updating an available component list requires pcd to perform the following tasks:

After pcd notifies dxs about any added boards, dxs in turn notifies the running domain of the arrival of an available board.


procedure icon  To Set Up the Available Component List

setupplatform sets up the available component list for domains. If a domain-id or domain-tag is specified, a list of boards must be specified. If no value is specified for a parameter, it will retain its current value.

1. In an SC window, log in as a platform administrator.

2. Type the following command:


sc0:sms-user:> setupplatform -d domain-indicator -a location

where:


-a

Adds the slot to the available component list for the specified domain.

-d domain-indicator

Specifies the domain using:

 

domain-id - ID for a domain. Valid domain-ids are A-R and are not case sensitive.

 

domain-tag - Name assigned to a domain using addtag(1M).

location

The board (DCU) location.


The following location forms are accepted:


Valid Form for Sun Fire 15K/E25K

Valid Form for Sun Fire 12K/E20K

SB(0...17)

IO(0...17)

SB(0...8)

IO(0...8)


The following is an example of making boards at SB0, IO1, and IO2 available to domain A:


sc0:sms-user:> setupplatform -d A -a SB0 IO1 IO2

The platform administrator can now assign the board to domain A using the addboard(1M) command or leave that up to the domain administrator.

A platform administrator has privileges for only the -c assign option of the addboard command. All other board configuration requires domain privileges. For more information, refer to the addboard man page.

Configuring Domains


procedure icon  To Name or Change Domain Names From the Command Line

You do not need to create domains on the Sun Fire high-end system. Eighteen domains have already been established (domains A-R, case insensitive). These domain designations are customizable. This section describes how to uniquely name domains.



Note - Before proceeding, see Domain Configuration Requirements. If the system configuration must be changed to meet any of these requirements, call your service provider.



1. Log in to the SC.

2. Type the following command:


sc0:sms-user:> addtag -d domain-indicator  new-tag

where:


-d domain-indicator

Specifies the domain using:

 

domain-id - ID for a domain. Valid domain-ids are A-R and are not case sensitive.

 

domain-tag - Name assigned to a domain using addtag(1M).

new-tag

The new name you want to give to the domain. It must be unique among all domains controlled by the SC.


Naming a domain is optional.

The following is an example of naming Domain A to dmnA:


sc0:sms-user:> addtag -d A dmnA


procedure icon  To Add Boards to a Domain From the Command Line

1. Log in to the SC.



Note - Platform administrators are restricted to using the -c assign option. The option can be used only for boards classified as available, not for boards classified as active.



The system board must be in the available state to the domain to which it is being added. Use the showboards (1M) command to determine a board's state.

2. Type the following command:


sc0:sms-user:> addboard -d domain-indicator -c assign location...

where:


-d domain-indicator

Specifies the domain using:

 

domain-id - ID for a domain. Valid domain-ids are A-R and are not case sensitive.

 

domain-tag - Name assigned to a domain using addtag(1M).

-c assign

Specifies the transition of the board from the current configuration state to the assigned state.

location

The board (DCU) location. Multiple locations are permitted.


The following location forms are accepted:


Valid form for Sun Fire 15K/E25K

Valid form for Sun Fire 12K/E20K

SB(0...17)

IO(0...17)

SB(0...8)

IO(0...8)


For example:


sc0:sms-user:> addboard -d C -c assign SB0 I01 SB1 I02

SB0, IO1, SB1, and IO2 have now changed from a state of being available to domain C to being assigned to that domain.

addboard performs tasks synchronously and does not return control to the user until the command is complete. If the command fails, the board does not return to its original state. A dxs or dca error is logged to the domain and pcd reports an error to the platform log file. If the error is recoverable, you can retry the command. If it is unrecoverable, you must reboot the domain in order to use that board.


procedure icon  To Delete Boards From a Domain From the Command Line



Note - Platform administrators are restricted to using the -c unassign option. The option can be used only for boards with a status of assigned, not boards with a status of active.



1. Log in to the SC.

The system board must be in the assigned state to the domain from which it is being deleted. Use the showboards (1M) command to determine a board's state.

2. Type the following command:


sc0:sms-user:> deleteboard -c unassign  location...

where:

-c unassign

Specifies the transition of the board from the current configuration state to a new unassigned state.

location

The board (DCU) location. Multiple locations are permitted.

 

The following location forms are accepted:


Valid Form for Sun Fire 15K/E25K

Valid Form for Sun Fire 12K/E20K

SB(0...17)

IO(0...17)

SB(0...8)

IO(0...8)


For example:


sc0:sms-user:> deleteboard -c unassign SB0

SB0 has now changed from being assigned to the domain to being available to it.

If deleteboard fails, the board does not return to its original state. A dxs or dca error is logged to the domain and pcd reports an error to the platform log file. If the error is recoverable, you can retry the command. If it is unrecoverable, you must reboot the domain in order to use that board.


procedure icon  To Move Boards Between Domains From the Command Line



Note - Platform administrators are restricted to the -c assign option. The option can only be used for boards with a status of assigned. It cannot be used for active boards.



1. Log in to the SC.

The system board must be in the assigned state to the domain from which it is being deleted. Use the showboards (1M) command to determine a board's state.

2. Type the following command:


sc0:sms-user:> moveboard -d domain-indicator  -c assign location

where:


-d domain-indicator

Specifies the domain using:

 

domain-id - ID for a domain. Valid domain-ids are A-R and are not case sensitive.

 

domain-tag - Name assigned to a domain using addtag(1M).

-c assign

Specifies the transition of the board from the current configuration state to an assigned state.

location

The board (DCU) location.


The following location forms are accepted:


Valid Form for Sun Fire 15K/E25K

Valid Form for Sun Fire 12K/E20K

SB(0...17)

IO(0...17)

SB(0...8)

IO(0...8)


moveboard performs tasks synchronously and does not return control to the user until the command is complete. You can only specify one location when using moveboard.

For example:


sc0:sms-user:> moveboard -d C -c assign SB0

SB0 has been moved from its previous domain and assigned to domain C.

If moveboard fails, the board does not return to its original state. A dxs or dca error is logged to the domain and pcd reports an error to the platform log file. If the error is recoverable, you can retry the command. If it is unrecoverable, you must reboot the domain the board was in when the error occurred, in order to use that board.


procedure icon  To Set Domain Defaults

The SMS setdefaults(1M) command removes all instances of a previously active domain.

1. Log in to the SC.

Platform administrators can set domain defaults for all domains, but only one domain at a time. The domain must not be active and setkeyswitch must be set to off.

setdefaults removes all pcd entries except network information and log files. This includes removing the NVRAM and boot parameter data.

By default, you are asked whether you want to remove the NVRAM and boot parameter data. If you respond no, the data is preserved. If you use the -p option you are not prompted and the data is automatically preserved.

2. Type the following command:


sc0:sms-user:> setdefaults -d domain-indicator [-p]

where:


-d domain-indicator

Specifies the domain using:

 

domain-id - ID for a domain. Valid domain-ids are A-R and are not case sensitive.

 

domain-tag - Name assigned to a domain using addtag(1M).

-p

Preserves the NVRAM and boot parameter data without a prompt.


For more information on setdefaults, refer to the setdefaults man page or the System Management Services (SMS) 1.6 Reference Manual.


procedure icon  To Obtain Board Status

1. Log in to the SC.

Platform administrators can obtain board status for all domains.

2. Type:


sc0:sms-user:> showboards [-d domain-id|-d  domain-tag]

The board status is displayed.

The following partial example for the Sun Fire 15K system shows the board information for a user with platform administrator privileges. All domains are visible. On a Sun Fire 12K system, nine domains would be shown.


sc0:sms-user:> showboards
 
Location  Pwr  Type         Board Status  Test Status   Domain
----      ---  ----         ------------  -----------   ------
SB0        On   CPU          Active        Passed        domainC
SB1        On   CPU          Active        Passed        A
SB2        On   CPU          Active        Passed        A
SB3        On   CPU          Active        Passed        engB
SB4        On   CPU          Active        Passed        engB
SB5        On   CPU          Active        Passed        engB
SB6        On   CPU          Active        Passed        A
SB7        On   CPU          Active        Passed        domainC
SB8        Off  CPU          Available     Unknown       Isolated
SB9        On   CPU          Active        Passed        dmnJ
SB10       Off  CPU          Available     Unknown       Isolated
SB11       Off  CPU          Available     Unknown       Isolated
SB12       Off  CPU          Assigned      Unknown       engB
SB13        -   Empty Slot   Available         -         Isolated
SB14       Off  CPU          Assigned      Failed        domainC
SB15       On   CPU          Active        Passed        P
SB16       On   CPU          Active        Passed        domainC
SB17        -   Empty Slot   Assigned          -         dmnR
IO0         -   Empty Slot   Available         -         Isolated
IO1        On   HPCI         Active        Passed        A
IO2        On   MCPU         Active        Passed        engB
IO3        On   MCPU         Active        Passed        domainC
IO4        On   HPCI+        Available     Degraded      domainC
IO5        Off  HPCI+        Assigned      Unknown       engB
IO6        On   HPCI         Active        Passed        A
IO7        On   HPCI         Active        Passed        dmnJ
IO8        On   WPCI         Active        Passed        Q
IO9        On   HPCI+        Assigned      iPOST         dmnJ
IO10       Off  HPCI         Assigned      Unknown       engB
IO11       Off  HPCI         Assigned      Failed        engB
IO12       Off  HPCI         Assigned      Unknown        engB
IO13        -   Empty Slot   Available         -         Isolated
IO14       Off  HPCI+        Available     Unknown       Isolated
IO15       On   HPCI         Active        Passed        P
IO16       On   HPCI         Active        Passed        Q
IO17        -   Empty Slot   Assigned          -         dmnR


procedure icon  To Obtain Domain Status

1. Log in to the SC.

Platform administrators can obtain domain status for all domains.

2. Type the following command:


sc0:sms-user:> showplatform -d domain-indicator

where:


-d domain-indicator

Specifies the domain using:

 

domain-id - ID for a domain. Valid domain-ids are A-R and are not case sensitive.

 

domain-tag - Name assigned to a domain using addtag(1M).


The status listing is displayed.

The following partial example for the Sun Fire 15K system shows the domain information for a user with platform administrator privileges. All domains are visible. On a Sun Fire 12K system, nine domains would be shown.


sc0:sms-user:> showplatform 
...
Domain configurations:
======================
Domain ID Domain Tag    Solaris Nodename   Domain Status
A         newA          sun15-b0           Powered Off
B         engB          sun15-b1           Keyswitch Standby
C         domainC       sun15-b2           Running OBP
D         eng1          sun15-b3           Loading Solaris
E         -             sun15-b4           Running Solaris
F         domainF       sun15-b5           Running Solaris
G         dmnG          sun15-b6           Running Solaris
H         -             sun15-b7           Solaris Quiesced
I         -             sun15-b8           Powered Off
J         dmnJ          sun15-b9           Powered Off
K         -             sun15-b10          Booting Solaris
L         -             sun15-b11          Powered Off
M         -             sun15-b12          Powered Off
N         -             sun15-b13          Keyswitch Standby
O         -             sun15-b14          Powered Off
P         -             sun15-b15          Running Solaris
Q         -             sun15-b16          Running Solaris
R         dmnR          sun15-b17          Running Solaris

Virtual Time of Day

The Solaris environment uses the functions provided by a hardware time of day (TOD) chip to support Solaris system date and time. Typically, Solaris software reads the current system date and time at boot using a get TOD service. From that point forward, Solaris software either uses a high-resolution hardware timer to represent current date and time or, if configured, uses Network Time Protocol (NTP) to synchronize current system date and time to a (presumably more accurate) time source.

The SC is the only computer on the platform that has a real-time clock. The virtual TOD for domains is stored as an offset from that real-time clock value. Each domain can be configured to use NTP services instead of setdate (1M) to manage the running system date and time. For more information on NTP, see Configuring NTP or refer to the xntpd(1M) man page in the man Pages(1M): System Administration Commands section of the Solaris 9 Reference Manual Collection.



Note - NTP is a separate package that must be installed and configured on the domain in order to function as described. Use setdate on the domain prior to installing NTP.



However system date and time is managed while Solaris software is running, an attempt is made to keep the boot-time TOD value accurate by setting the TOD when variance is detected between the current TOD value and the current system date and time.

Since the Sun Fire high-end system hardware provides no physical TOD chip for Sun Fire domains, SMS provides the time-of-day services required by the Solaris environment for each domain. Each domain is supplied with a TOD service that is logically separate from that provided to any other domain. This difference allows system date and time management on a Sun Fire high-end system domain to be as flexible as that provided by standalone servers. In the unlikely event that a domain needs to be set up to run at a time other than real-world time, the Sun Fire high-end system TOD service allows that domain to be configured without affecting the TOD values supplied to other domains running real world time.

Time settings are implemented using setdate(1M). You must have platform administrator privileges to run setdate. See All Privileges for more information.

Setting the Date and Time

setdate (1M) allows the SC platform administrator to set the system controller date and time values. After setting the date and time, setdate(1M) displays the current date and time for the user.


procedure icon  To Set the Date on the SC

1. Log in to the SC.

2. Type the following command:


sc0:sms-user:> setdate 021210302000.00
System Controller: Tue Feb 12 10:30 2002 US/Pacific

Optionally, setdate(1M) can set a domain TOD. The domain's keyswitch must be in the off or standby position. You must have platform administrator privileges to run this command on the domain.


procedure icon  To Set the Date for Domain eng2

1. Log in to the SC.

2. Type the following command:


sc0:sms-user:> setdate -d eng2 021210302000.00
Domain eng2: Tue Feb 12 10:30 2002 US/Pacific

showdate(1M) displays the current SC date and time.


procedure icon  To Display the Date on the SC

1. Log in to the SC.

2. Type the following command:


sc0:sms-user:> showdate
System Controller: Tue Feb 12 10:30 2002 US/Pacific

Optionally, showdate(1M) can display the date and time for a specified domain. Superuser or any member of a platform or domain group can run showdate.


procedure icon  To Display the Date on Domain eng2

1. Log in to the SC.

2. Type the following command:


sc0:sms-user:> showdate -d eng2
Domain eng2: Tue Feb 12 10:30 2002 US/Pacific

Configuring NTP

The NTP daemon, called xntpd(1M) for the Solaris OS, provides a mechanism for keeping the time settings synchronized between the SC and the domains. The OpenBoot PROM obtains the time from the SC when the domain is booted, and NTP keeps the time synchronized on the domain from that point on.

NTP configuration is based on information provided by the system administrator.

The NTP packages are compiled with support for a local reference clock. This means that your system can poll itself for the time instead of polling another system or network clock. The poll is done through the network loopback interface. The numbers in the IP address are 127.127.1.0. This section describes how to set the time on the SC using setdate, and then to set up the SC to use its own internal time-of-day clock as the reference clock in the ntp.conf file.

NTP can also keep track of the drift (difference) between the SC clock and the domain clock. NTP corrects the domain clock if it loses contact with the SC clock, provided that you have a drift file declaration in the ntp.conf file. The drift file declaration specifies to the NTP daemon the name of the file that stores the error in the clock frequency computed by the daemon. See the following procedure for an example of the drift file declaration in an ntp.conf file.

If the ntp.conf file does not exist, create it as described in the following procedure. You must have an ntp.conf file on both the SC and the domains.


procedure icon  To Create the ntp.conf File

1. Log in to the main SC as superuser.

2. Change to the /etc/inet directory and copy the NTP server file to the NTP configuration file:


sc0:# cd /etc/inet
sc0:# cp ntp.server ntp.conf

3. Using a text editor, edit the /etc/inet/ntp.conf file created in the previous step.

The ntp.conf file for the Solaris 9 OS is located in /etc/inet.

The following is an example of server lines in the ntp.conf file on the main SC, to synchronize clocks.


server 127.127.1.0
fudge 127.127.1.0 stratum 13
driftfile /var/ntp/ntp.drift
statsdir /var/ntp/ntpstats/
filegen peerstats file peerstats type day enable
filegen loopstats file loopstats type day enable
filegen clockstats file clockstats type day enable

4. Save the file and exit.

5. Stop and restart the NTP daemon:


sc0:# /etc/init.d/xntpd stop
sc0:# /etc/init.d/xntpd start

6. Log in to the spare SC as superuser.

7. Change to the /etc/inet directory and copy the NTP server file to the NTP configuration file:


sc1:# cd /etc/inet
sc1:# cp ntp.server ntp.conf

8. Using a text editor, edit the /etc/inet/ntp.conf file created in the previous step.

The ntp.conf file for the Solaris 9 OS is located in /etc/inet.

The following is an example of server lines in the ntp.conf file on the spare SC, to synchronize clocks.


server 127.127.1.0
fudge 127.127.1.0 stratum 13
driftfile /var/ntp/ntp.drift
statsdir /var/ntp/ntpstats/
filegen peerstats file peerstats type day enable
filegen loopstats file loopstats type day enable
filegen clockstats file clockstats type day enable

9. Stop and restart the NTP daemon:


sc1:# /etc/init.d/xntpd stop
sc1:# /etc/init.d/xntpd start

10. Log in to each domain as superuser.

11. Change to the /etc/inet directory and copy the NTP client file to the NTP configuration file:


domain-id:# cd /etc/inet
domain-id:# cp ntp.client ntp.conf

12. Using a text editor, edit the /etc/inet/ntp.conf file created in the previous step.

The ntp.conf file for the Solaris 9 OS is located in /etc/inet.

For the Solaris 9 OS, you can add lines similar to the following to the /etc/inet/ntp.conf file on the domains:


server main-sc-hostname prefer
server spare-sc-hostname

13. Save the file and exit.

14. Change to the initialization directory, and stop and restart the NTP daemon on the domain:


domain-id:# /etc/init.d/xntpd stop
domain-id:# /etc/init.d/xntpd start

NTP is now installed and running on your domain. Repeat Step 10 through Step 14 for each domain.

For more information on the NTP daemon, refer to the xntpd(1M) man page in the man Pages(1M): System Administration Commands section of the Solaris 9 Reference Manual Collection.

Virtual ID PROM

Each configurable domain has a virtual ID PROM that contains identifying information about the domain, such as hostID and domain Ethernet address. The hostID is unique among all domains on the same platform. The Ethernet address is world unique.

Sun Fire high-end system management software provides a virtual ID PROM for each configurable domain containing identifying information that can be read, but not written, from the domain. The information provided meets the requirements of the Solaris environment.

The flashupdate Command

SMS provides the flashupdate(1M) command to update the Flash PROM in the system controller (SC), and the Flash PROMs in a domain's CPU and MaxCPU boards after SMS software upgrades or applicable patch installation. flashupdate displays both the current Flash PROM and the flash image file information prior to any updates.



Note - Once you have updated the SC FPROMs you must reset the SC using the reset-all command at the OpenBoot PROM (ok) prompt. No CLIs should be executed on a system board while flashupdate is running on that board. Wait until flashupdate completes before running any SMS commands involving that system board.





Note - After running a flashupdate command, new firmware is not active on system boards until the system power-on self test (POST) control application, hpost, is performed per board with a dynamic reconfiguration operation. For single boards, use the deleteboard(1M) or addboard(1M) commands to perform an hpost. For all boards in a domain, use the setkeyswitch(1M) command.



For more information and examples, refer to the flashupdate man page.


Configuration for Domain Administrators

This section briefly describes the configuration services available to the domain administrator.

Configuring Domains

The addboard, deleteboard, and moveboard commands offer more functionality to the domain administrator than to the platform administrator.


procedure icon  To Add Boards to a Domain From the Command Line

1. Log in to the SC as a domain administrator for that domain.



Note - For the domain administrator to add a board to a domain, that board must appear in the domain's available component list.



The system board must be in the available or assigned state to the domain to which it is being added. Use the showboards (1M) command to determine a board's state.

2. Type the following command:


sc0:sms-user:> addboard -d domain-indicator -c function location

where:


-d domain-indicator

Specifies the domain using:

 

domain-id - ID for a domain. Valid domain-ids are A-R and are not case sensitive.

 

domain-tag - Name assigned to a domain using addtag(1M).

-c function

Specifies the transition of the board from the current configuration state to a new configuration state.

location

The board (DCU) location.


Configuration states are as follows:


assign

Assigns the board to the logical domain. The board belongs to the domain but is not active.

connect

Transitions an assigned board to the connected/unconfigured state. This is an intermediate state and has no standalone implementation.

configure

Transitions an assigned board to the connected/configured state. The hardware resources on the board can be used by Solaris software.


If the -c function option is not specified, the default expected configuration state is configure. For more detailed information on the configuration states, refer to the addboard(1M) man page.

Multiple locations are accepted.

The following location forms are accepted:


Valid Form for Sun Fire 15K/E25K

Valid Form for Sun Fire 12K/E20K

SB(0...17)

IO(0...17)

SB(0...8)

IO(0...8)


For example:


sc0:sms-user:> addboard -d C -c assign SB0 I01 SB1 I02

In this example, SB0, IO1, SB1, and IO2 have changed from being available to domain C to being assigned to it.

addboard performs tasks synchronously and does not return control to the user until the command is complete. If the board is not powered on or tested, specify the -c connect|configure option, then the command will power on the board and test it.

If addboard fails, the board does not return to its original state. A dxs or dca error is logged to the domain and pcd reports an error to the platform log file. If the error is recoverable, you can retry the command. If it is unrecoverable, you must reboot the domain in order to use that board.


procedure icon  To Delete Boards From a Domain From the Command Line

1. Log in to the SC as a domain administrator for that domain.

The system board must be in the assigned or active state to the domain from which it is being deleted. Use the showboards (1M) command to determine a board's state.

2. Type the following command:


sc0:sms-user:> deleteboard -c function  location

where:

-c function

Specifies the transition of the board from the current configuration state to a new configuration state.

location

The board (DCU) location.

 

Configuration states are:


unconfigure

Transitions an assigned board to the connected or unconfigured state. The hardware resources on the board can no longer be used by Solaris software.

disconnect

Transitions an assigned board to the disconnected or unconfigured state.

unassign

Unassigns the board from the logical domain. The board no longer belongs to the domain, and its state is changed to available.


If the -c function option is not specified, the default expected configuration state is unassign. For more detailed information on the configuration states, refer to the deleteboard(1M) man page.

Multiple locations are accepted.

The following location forms are accepted:


Valid form for Sun Fire 15K/E25K

Valid form for Sun Fire 12K/E20K

SB(0...17)

IO(0...17)

SB(0...8)

IO(0...8)


For example:


sc0:sms-user:> deleteboard -c unassign SB0

In this example, SB0 has changed from being assigned to the domain to being available to it.



Note - A domain administrator can unconfigure and disconnect a board but is not allowed to delete a board from a domain unless the deleteboard [location] field appears in the domain's available component list.



If deleteboard fails, the board does not return to its original state. A dxs or dca error is logged to the domain and pcd reports an error to the platform log file. If the error is recoverable, you can retry the command. If it is unrecoverable, you must reboot the domain in order to use that board.


procedure icon  To Move Boards Between Domains From the Command Line



Note - You must have domain administrator privileges for both domains involved.



1. Log in to the SC as a domain administrator for that domain.

The system board must be in the assigned or active state to the domain from which it is being deleted. Use the showboards (1M) command to determine a board's state.

2. Type the following command:


sc0:sms-user:> moveboard -d domain-indicator  -c function location

where:


-d domain-indicator

This is the domain to which the board is being moved. Specifies the domain using:

 

domain-id - ID for a domain. Valid domain-ids are A-R and are not case sensitive.

 

domain-tag - Name assigned to a domain using addtag(1M).

-c function

Specifies the transition of the board from the current configuration state to an new configuration state.

location

The board (DCU) location.


Configuration states are:


assign

Unconfigures the board from the current logical domain. Moves the board out of the logical domain by changing its state to available. Assigns the board to the new logical domain. The board belongs to the new domain but is not active.

connect

Transitions an assigned board to the connected or unconfigured state. This is an intermediate state and has no standalone implementation.

configure

Transitions an assigned board to the connected or configured state. The hardware resources on the board can be used by Solaris software.


If the -c option is not specified, the default expected configuration state is configure. For more detailed information on the configuration states, refer to the moveboard(1M) man page.

The following location forms are accepted:


Valid Form for Sun Fire 15K/E25K

Valid Form for Sun Fire 12K/E20K

SB(0...17)

IO(0...17)

SB(0...8)

IO(0...8)


moveboard performs tasks synchronously and does not return control to the user until the command is complete. If the board is not powered on or tested, specify -c connect|configure; then, the command will power on the board and test it. You can only specify one location when using moveboard.

If moveboard fails, the board does not return to its original state. A dxs or dca error is logged to the domain and pcd reports an error to the platform log file. If the error is recoverable, you can retry the command. If it is unrecoverable, you must reboot the domain the board was in when the error occurred, in order to use that board.


procedure icon  To Set Domain Defaults

The SMS setdefaults(1M) command removes all instances of a previously active domain.

1. Log in to the SC.

Domain administrators can set domain defaults for all domains, but only one domain at a time. The domain must not be active and setkeyswitch must be set to off. setdefaults removes all pcd entries except network information, log files and, optionally, NVRAM and boot parameter data.

2. Type the following command:


sc0:sms-user:> setdefaults -d domain-indicator

where:


-d domain-indicator

Specifies the domain using:

 

domain-id - ID for a domain. Valid domain-ids are A-R and are not case sensitive.

 

domain-tag - Name assigned to a domain using addtag(1M).


For more information on setdefaults, refer to the setdefaults man page or the System Management Services (SMS) 1.6 Reference Manual.


procedure icon  To Obtain Board Status

1. Log in to the SC.

Domain administrators can obtain board status only for those domains for which they have privileges.

2. Type the following command:


sc0:sms-user:> showboards [-d domain-id|domain-tag]

The board status is displayed.

The following partial example shows the board information for a user with domain administrator privileges for domain A.


sc0:sms-user:> showboards -d A
 
Location    Pwr    Type  Board Status  Test Status   Domain  
-------    -----   ----  ------------  -----------   ------  
SB1         On     CPU    Active        Passed        A       
SB2         On     CPU    Active        Passed        A       
IO1         On     HPCI   Active        Passed        A       


procedure icon  To Obtain Domain Status

1. Log in to the SC.

Domain administrators can obtain domain status only for those domains for which they have privileges.

2. Type the following command:


sc0:sms-user:> showplatform -d domain-indicator

where:


-d domain-indicator

Specifies the domain using:

 

domain-id - ID for a domain. Valid domain-ids are A-R and are not case sensitive.

 

domain-tag - Name assigned to a domain using addtag(1M).


The status listing is displayed.

The following partial example shows the domain information for a user with domain administrator privileges for domains newA, engB, and domainC.


sc0:sms-user:> showplatform 
...
Domain configurations:
======================
Domain ID Domain Tag    Solaris Nodename   Domain Status
A         newA          sun15-b0           Powered Off
B         engB          sun15-b1           Keyswitch Standby
C         domainC       sun15-b2           Running OBP


procedure icon  To Obtain Device Status

1. Log in to the SC.

Domain administrators can obtain board status only for those domains for which they have privileges.

2. Type the following command:


sc0:sms-user:> showdevices [-d domain-id|domain-tag]

The device status is displayed.

The following partial example shows the device information for a user with domain administrator privileges for domain A.


sc0:sms-user:> showdevices IO1
 
IO Devices
----------
domain  location device  resource           usage
A       IO1      sd3     /dev/dsk/c0t3d0s0  mounted filesystem "/"
A       IO1      sd3     /dev/dsk/c0t3s0s1  dump device (swap)
A       IO1      sd3     /dev/dsk/c0t3s0s1  swap area
A       IO1      sd3     /dev/dsk/c0t3d0s3  mounted filesystem "/var"
A       IO1      sd3     /var/run           mounted filesystem "/var/run"

Virtual Keyswitch

Each Sun Fire high-end system domain has a virtual keyswitch. Like the Sun Enterprise server's physical keyswitch, the Sun Fire high-end system domain virtual keyswitch controls whether the domain is powered on or off, whether increased diagnostics are run at boot, and whether certain operations (for example, flash PROM updates and domain reset commands) are permitted.

Only domains configured with their virtual keyswitch powered on are booted, monitored, and subject to automatic recovery actions, should they fail.

Virtual keyswitch settings are implemented using setkeyswitch(1M). You must have domain administrator privileges for the specified domain in order to run setkeyswitch. See All Privileges for more information.

The setkeyswitch Command

setkeyswitch (1M) changes the position of the virtual key switch to the specified value. pcd (1M) maintains the state of each virtual key switch between power cycles of the SC or physical power cycling of the power supplies.

setkeyswitch(1M) is responsible for loading the bootbus SRAM of all the configured processors. All the processors are started, with one processor designated as the boot processor. setkeyswitch(1M) loads OpenBoot PROM into the memory of the Sun Fire high-end system domain and starts OpenBoot PROM on the boot processor.

The primary task of OpenBoot PROM is to boot and configure the OS from either a mass storage device or from a network. OpenBoot PROM also provides extensive features for testing hardware and software interactively.

The setkeyswitch(1M) command syntax follows:


sc0:sms-user:> setkeyswitch -d domain-indicator [-q -y|-n] 
on|standby|off|diag|secure -l level

where:


-d domain-indicator

Specifies the domain using:

 

domain-id - ID for a domain. Valid domain-ids are A-R and are not case sensitive.

 

domain-tag - Name assigned to a domain using addtag(1M).

-q

Quiet. Suppresses all messages to stdout including prompts. When used alone -q defaults to the -n option for all prompts. When used with either the -y or the -n option, -q suppresses all user prompts, and automatically answers with either Y or N based on the option chosen.

-n

Automatically answers no to all prompts. Prompts are displayed unless used with -q option.

-y

Automatically answers yes to all prompts. Prompts are displayed unless used with -q option.

-l level

Specifies the hpost level to be used at system startup.


The following operands are supported:

For more information on ASR, see Automatic System Recovery (ASR).


procedure icon  To Set the Virtual Keyswitch On in Domain A

1. Log in to the SC.

Domain administrators can set the virtual keyswitch only for those domains for which they have privileges.

2. Type the following command:


sc0:sms-user:> setkeyswitch -d A on

showkeyswitch (1M) displays the position of the virtual keyswitch of the specified domain. The state of each virtual keyswitch is maintained between power cycles of the SC or physical power cycling of the power supplies by the pcd (1M). Superuser or any member of a platform or domain group can run showkeyswitch.


procedure icon  To Display the Virtual Keyswitch Setting in Domain A

1. Log in to the SC.

Domain administrators can obtain keyswitch status only for those domains for which they have privileges.

2. Type the following command:


sc0:sms-user:> showkeyswitch -d A
Virtual keyswitch position: ON

Virtual NVRAM

Each domain has a virtual NVRAM containing OpenBoot PROM data, such as the OpenBoot PROM variables. OpenBoot PROM is a binary image stored on the SC in /opt/SUNWSMS/hostobjs which setkeyswitch downloads into domain memory at boot time. There is only one version of OpenBoot PROM for all domains.

SMS software provides a virtual NVRAM for each domain and allows OpenBoot PROM full read/write access to this data.

The only interface available to read from or write to most NVRAM variables is OpenBoot PROM. The exceptions are those OpenBoot PROM variables which must be altered in order to bring OpenBoot PROM up in a known working state, or to diagnose problems that hinder OpenBoot PROM from coming up. These variables are not a replacement for the OpenBoot PROM interface.

These limited number of OpenBoot PROM variable values in the domain NVRAM are readable and writable from SMS using setobpparams(1M). You must have domain administrator privileges to run set/showobpparams. If you change variables for a running domain, you must reboot the domain in order for the changes to take effect.



Note - Only experienced system administrators who are familiar with OpenBoot PROM commands and their dependencies should attempt to use setobpparams in any manner other than that described.



Setting the OpenBoot PROM Variables

setobpparams(1M) sets and gets a subset of a domain's virtual NVRAM variables and REBOOTINFO data using the following syntax.


sc0: sms-user:> setobpparams -d domain-indicator param=value... 

where:


-d domain-indicator

Specifies the domain using:

 

domain-id - ID for a domain. Valid domain-ids are A-R and are not case sensitive.

 

domain-tag - Name assigned to a domain using addtag(1M).


param=value is one of the following variables and its corresponding value:


Variables

=

Default Value

Comment

diag-switch?

=

false

When set to false, the default boot device is specified by boot-device and the default boot file by boot-file. If set to true, OpenBoot PROM runs in diagnostic mode, and you must set either diag-device or diag-file to specify the correct default boot device or file. These default boot device and file settings cannot be set using setobpparams. Use setenv(1) in OpenBoot PROM.

auto-boot?

=

false

When set to true, the domain boots automatically after poweron or reset-all. The boot device and boot file used are based on the settings for diag-switch (see above). Neither boot-device nor boot file can be set using setobpparams. If the ok prompt is unavailable during such as a repeated panic, use setobpparams to set auto-boot? to false. When the auto-boot? variable is set to false using setobpparams, the reboot variables are invalidated. In addition, the system will not boot automatically and will stop at OpenBoot PROM. At that point, you can set new NVRAM variables. See To Recover From a Repeated Domain Panic.

security-mode

=

none

Firmware security level. Valid variable values for security-mode are:

  • none - No password required (default).
  • command - All commands except for boot(1M) and go require the password.
  • full - All commands except for go require the password.

use-nvramrc?

=

false

When set to true, this variable executes commands in NVRAMRC during system startup.

fcode-debug?

=

false

When set to true, this variable includes name fields for plug-in device FCodes.


The following is an example of how setobpparams can be useful.


procedure icon  To Recover From a Repeated Domain Panic

In the following example, domain A encounters repeated panics caused by a corrupted default boot disk.

1. Log in to the SC with domain administrator privileges.

2. Stop automatic reboot:


sc0:sms-user:> setkeyswitch -d A standby
sc0:sms-user:> setobpparams -d A 'auto-boot?=false'



Note - Most, but not all, shells require using single quotes around the variable values to prevent the question mark from being treated as a special character.



3. Repost the domain:


sc0:sms-user:> setkeyswitch -d A off
sc0:sms-user:> setkeyswitch -d A on

4. Once the domain has come up to the OK prompt, set NVRAM variables to a new uncorrupted boot-device.


ok setenv boot-device bootdisk-alias 

where:

bootdisk-alias

A user-defined alias you created. The boot device must correspond to the bootable disk on which you have installed the OS.

 

5. Now that you have set up a new alias for your boot device, boot the disk by typing:


ok boot

For more information on OpenBoot variables, refer to the OpenBoot 4.x Command Reference Manual.


procedure icon  To Set the OpenBoot PROM Security Mode Variable in Domain A

1. Log in to the SC.

Domain administrators can set the OpenBoot PROM variables only for those domains for which they have privileges.

2. Type the following command:


sc0:sms-user:> setobpparams -d A security-mode=full 

security-mode has been set to full. All commands except go require a password on domain A. You must reboot a running domain in order for the change to take effect.


procedure icon  To See the OpenBoot PROM Variables

1. Log in to the SC.

Domain administrators can set the OpenBoot PROM variables only for those domains for which they have privileges.

2. Type the following command:


sc0:sms-user:> showobpparams -d domain-indicator 

where:


-d domain-indicator

Specifies the domain using:

 

domain-id - ID for a domain. Valid domain-ids are A-R and are not case sensitive.

 

domain-tag - Name assigned to a domain using addtag(1M).


SMS NVRAM updates are supplied to OpenBoot PROM at OpenBoot PROM initiation (or domain reboot time). For more information refer to the OpenBoot PROM 4.x Command Reference Manual.


Degraded Configuration Preferences

In most situations, hardware failures that cause a domain crash are detected and eliminated from the domain configuration either by POST or OpenBoot PROM during the subsequent automatic recovery boot of the domain. However, there can be situations where failures are intermittent or the boot-time tests are inadequate to detect failures that cause repeated domain failures and reboots. In those situations, Sun Fire high-end system management software uses configurations or configuration policies supplied by the domain administrator to eliminate hardware from the domain configuration in an attempt to get a stable domain environment running.

The following commands can be run by either platform or domain administrators. Domain administrators are restricted to the domains for which they have privileges.

The setbus Command

setbus(1M) dynamically reconfigures bus traffic on active expanders in a domain to use either one centerplane support board (CSB) or both. Using both CSBs is considered normal mode. Using one CSB is considered degraded mode.

setbus resets any boards that are powered on but not active. Any attach-ready state is lost. For more information on attach-ready states, refer to the System Management Services (SMS) 1.6 Dynamic Reconfiguration User Guide.

You must have platform administrator privileges or domain privileges for the specified domain in order to run setbus.

This feature allows you to swap out a CSB without having to power off the system. Valid buses are:


procedure icon  To Set All Buses on All Active Domains to Use Both CSBs

1. Log in to the SC.

Domain administrators can set the bus only for those domains for which they have privileges.

2. Type the following command:


sc0:sms-user:> setbus -c CS0,CS1

For more information on reconfiguring bus traffic, refer to the setbus(1M) man page.

The showbus Command

showbus(1M) displays the bus configuration of expanders in active domains. This information defaults to displaying configuration by slot order. Any member of a platform or domain group can run showbus.


procedure icon  To Show All Buses on All Active Domains

1. Log in to the SC.

2. Type the following command:


sc0:sms-user:> showbus 

For more information on reconfiguring bus traffic, refer to the showbus(1M) man page.