Chapter 11 Performing Other Administration Tasks

This chapter contains information and tasks about using the Logical Domains software that are not described in the preceding chapters.

This chapter covers the following topics:

• Entering Names in the CLI

• Connecting to a Guest Console Over a Network

• Using Console Groups

• Stopping a Heavily-Loaded Domain Can Time Out

• Operating the Solaris OS With Logical Domains

• Using LDoms With the Service Processor

• Configuring Domain Dependencies

• Determining Where Errors Occur by Mapping CPU and Memory Addresses

Entering Names in the CLI

The following sections describe the restrictions on entering names in the Logical Domains Manager CLI.

File Names (file) and Variable Names (var-name)

• First character must be a letter, a number, or a forward slash (/).

• Subsequent letters must be letters, numbers, or punctuation.

Virtual Disk Server backend and Virtual Switch Device Names

The names must contain letters, numbers, or punctuation.

Configuration Name (config-name)

The logical domain configuration name (config-name) that you assign to a configuration stored on the service processor (SP) must have no more than 64 characters.

All Other Names

The remainder of the names, such as the logical domain name (ldom), service names (vswitch-name, service-name, vdpcs-service-name, and vcc-name), virtual network name (if-name), and virtual disk name (disk-name), must be in the following format:

• First character must be a letter or number.

• Subsequent characters must be letters, numbers, or any of the following characters -_+#.:;~().

Connecting to a Guest Console Over a Network

You can connect to a guest console over a network if the listen_addr property is set to the IP address of the control domain in the vntsd(1M) SMF manifest. For example:

$ telnet host-name 5001

Enabling network access to a console has security implications. Any user can connect to a console and for this reason it is disabled by default.

A Service Management Facility manifest is an XML file that describes a service. For more information about creating an SMF manifest, refer to the Solaris 10 System Administrator Collection.

To access a non-English OS in a guest domain through the console, the terminal for the console must be in the locale required by the OS.

Using Console Groups

The virtual network terminal server daemon, vntsd(1M), enables you to provide access for multiple domain consoles using a single TCP port. At the time of domain creation, the Logical Domains Manager assigns a unique TCP port to each console by creating a new default group for that domain's console. The TCP port is then assigned to the console group as opposed to the console itself. The console can be bound to an existing group using the set-vcons subcommand.

Combine Multiple Consoles Into One Group

1. Bind the consoles for the domains into one group.

   The following example shows binding the console for three different domains (ldg1, ldg2, and ldg3) to the same console group (group1).

   primary# ldm set-vcons group=group1 service=primary-vcc0 ldg1 primary# ldm set-vcons group=group1 service=primary-vcc0 ldg2 primary# ldm set-vcons group=group1 service=primary-vcc0 ldg3

2. Connect to the associated TCP port (localhost at port 5000 in this example).

   # telnet localhost 5000 primary-vnts-group1: h, l, c{id}, n{name}, q:

   You are prompted to select one of the domain consoles.

3. List the domains within the group by selecting l (list).

   primary-vnts-group1: h, l, c{id}, n{name}, q: l DOMAIN ID           DOMAIN NAME                   DOMAIN STATE 0                   ldg1                          online 1                   ldg2                          online 2                   ldg3                          online

   ---

   Note –

   To re-assign the console to a different group or vcc instance, the domain must be unbound; that is, it has to be in the inactive state. Refer to the Solaris 10 OS vntsd(1M) man page for more information on configuring and using SMF to manage vntsd and using console groups.

   ---

Stopping a Heavily-Loaded Domain Can Time Out

An ldm stop-domain command can time out before the domain completes shutting down. When this happens, an error similar to the following is returned by the Logical Domains Manager.

LDom ldg8 stop notification failed

However, the domain could still be processing the shutdown request. Use the ldm list-domain command to verify the status of the domain. For example:

# ldm list-domain ldg8
NAME         STATE   FLAGS  CONS   VCPU MEMORY  UTIL UPTIME
ldg8         active  s----  5000   22   3328M   0.3% 1d 14h 31m

The preceding list shows the domain as active, but the s flag indicates that the domain is in the process of stopping. This should be a transitory state.

The following example shows the domain has now stopped.

# ldm list-domain ldg8
NAME         STATE   FLAGS  CONS   VCPU MEMORY  UTIL UPTIME
ldg8         bound   -----  5000   22   3328M

Operating the Solaris OS With Logical Domains

This section describes the changes in behavior in using the Solaris OS that occur once a configuration created by the Logical Domains Manager is instantiated.

OpenBoot Firmware Not Available After Solaris OS Has Started

The OpenBoot firmware is not available after the Solaris OS has started because it is removed from memory.

To reach the ok prompt from the Solaris OS, you must halt the domain. You can use the Solaris OS halt command to halt the domain.

Powercycling a Server

Whenever performing any maintenance on a system running LDoms software that requires powercycling the server, you must save your current logical domain configurations to the SP first.

Save Your Current Logical Domain Configurations to the SP

1. Use the following command.

   # ldm add-config config-name

Do Not Use the psradm(1M) Command on Active CPUs in a Power-Managed Domain

Do not attempt to change an active CPU's operational status in a power-managed domain by using the psradm(1M) command.

Result of Solaris OS Breaks

The behavior described in this section is seen when you do the following:

1. Press the L1-A key sequence when the input device is set to keyboard.

2. Enter the send break command when the virtual console is at the telnet prompt.

After these types of breaks, you receive the following prompt:

c)ontinue, s)ync, r)eset, h)alt?

Type the letter that represents what you want the system to do after these types of breaks.

Results From Halting or Rebooting the Control Domain

The following table shows the expected behavior of halting or rebooting the control (primary) domain.

Using LDoms With the Service Processor

The section describes information to be aware of in using the Integrated Lights Out Manager (ILOM) service processor (SP) with the Logical Domains Manager. For more information about using the ILOM software, see the documents for your specific platform, such as Sun SPARC Enterprise T5120 and T5220 Servers Topic Set for the Sun SPARC Enterprise T5120 and T5220 servers.

An additional option is available to the existing ILOM command.

-> set /HOST/bootmode config=config-name

The config=config-name option enables you to set the configuration on the next power on to another configuration, including the factory-default shipping configuration.

You can invoke the command whether the host is powered on or off. It takes effect on the next host reset or power on.

Reset the Logical Domain Configuration to the Default or Another Configuration

1. Reset the logical domain configuration on the next power on to the default shipping configuration by executing this command:

   -> set /HOST/bootmode config=factory-default

   You also can select other configurations that have been created with the Logical Domains Manager using the ldm add-config command and stored on the service processor (SP). The name you specify in the Logical Domains Manager ldm add-config command can be used to select that configuration with the ILOM bootmode command. For example, assume you stored the configuration with the name ldm-config1.

   -> set /HOST/bootmode config=ldm-config1

   Now, you must powercycle the system to load the new configuration.

   See the ldm(1M) man page for more information about the ldm add-config command.

Configuring Domain Dependencies

You can use the Logical Domains Manager to establish dependency relationships between domains. A domain that has one or more domains that depend on it is called a master domain. A domain that depends on another domain is called a slave domain.

Each slave domain can specify up to four master domains by setting the master property. For example, the pine slave domain specifies its four master domains in the following comma-separated list:

# ldm add-domain master=apple,lemon,orange,peach pine

Each master domain can specify what happens to its slave domains in the event that the master domain fails. For instance, if a master domain fails, it might require its slave domains to panic. If a slave domain has more than one master domain, the first master domain to fail triggers its defined failure policy on all of its slave domains.

If more than one master domain fails simultaneously, only one of the specified failure policies will be enforced on all the affected slave domains. For example, if the failed master domains have failure policies of stop and panic, all slave domains will be either stopped or panicked.

The master domain's failure policy is controlled by setting one of the following values to the failure-policy property:

• ignore ignores any slave domains when the master domain fails.

• panic panics any slave domains when the master domain fails.

• reset resets any slave domains when the master domain fails.

• stop stops any slave domains when the master domain fails.

In this example, the master domains specify their failure policy as follows:

# ldm set-domain failure-policy=ignore apple
# ldm set-domain failure-policy=panic lemon
# ldm set-domain failure-policy=reset orange
# ldm set-domain failure-policy=stop peach

You can use this mechanism to create explicit dependencies between domains. For example, a guest domain implicitly depends on the service domain to provide its virtual devices. A guest domain's I/O is blocked when the service domain on which it depends is not up and running. By defining a guest domain as a slave of its service domain, you can specify the behavior of the guest domain when its service domain goes down. When no such dependency is established, a guest domain just waits for its service domain to return to service.

The Logical Domains Manager does not permit you to create domain relationships that create a dependency cycle. For more information, see Dependency Cycles.

For domain dependency XML examples, see Example 12–6.

Domain Dependency Examples

The following examples show how to configure domain dependencies.

• The first command creates a master domain called twizzle. This command uses failure-policy=reset to specify that slave domains reset if the twizzle domain fails. The second command modifies a master domain called primary. This command uses failure-policy=panic to specify that slave domains panic if the primary domain fails. The third command creates a slave domain called chocktaw that depends on two master domains, twizzle and primary. The slave domain uses master=twizzle,primary to specify its master domains. In the event either the twizzle or primary domain fails, the chocktaw domain will reset or panic. The first master domain to fail is the one responsible for determining the behavior of the slave domains.

  # ldm add-domain failure-policy=reset twizzle # ldm set-domain failure-policy=panic primary # ldm add-domain master=twizzle,primary chocktaw

• This example shows how to use the ldm set-domain command to modify the orange domain to assign primary as the master domain. The second command uses the ldm set-domain command to assign orange and primary as master domains for the tangerine domain. The third command lists information about all of these domains.

  # ldm set-domain master=primary orange # ldm set-domain master=orange,primary tangerine # ldm list -o domain NAME STATE FLAGS UTIL primary active -n-cv- 0.2% SOFTSTATE Solaris running HOSTID 0x83d8b31c CONTROL failure-policy=ignore DEPENDENCY master= ------------------------------------------------------------------------------ NAME STATE FLAGS UTIL orange bound ------ HOSTID 0x84fb28ef CONTROL failure-policy=stop DEPENDENCY master=primary VARIABLES test_var=Aloha ------------------------------------------------------------------------------ NAME STATE FLAGS UTIL tangerine bound ------ HOSTID 0x84f948e9 CONTROL failure-policy=ignore DEPENDENCY master=orange,primary VARIABLES test_var=A hui hou

• The following shows an example listing with parseable output:

  # ldm list -o domain -p

Dependency Cycles

The Logical Domains Manager does not permit you to create domain relationships that create a dependency cycle. A dependency cycle is a relationship between two or more domains that lead to a situation where a slave domain depends on itself, or a master domain depends on one of its slave domains.

The Logical Domains Manager determines whether a dependency cycle exists before adding a dependency. The Logical Domains Manager starts at the slave domain and searches along all paths that are specified by the master array until the end of the path is reached. Any dependency cycles found along the way are reported as errors.

The following example shows how a dependency cycle might be created. The first command creates a slave domain called mohawk that specifies its master domain as primary. So, mohawk depends on primary in the following dependency chain:

Figure 11–1 Single Domain Dependency

The second command creates a slave domain called primary that specifies its master domain as counter. So, mohawk depends on primary, which depends on counter in the following dependency chain:

Figure 11–2 Multiple Domain Dependency

The third command attempts to create a dependency between the counter and mohawk domains, which would produce the following dependency cycle:

Figure 11–3 Domain Dependency Cycle

The ldm set-domain command will fail with the following error message:

# ldm add-domain master=primary mohawk
# ldm set-domain master=counter primary
# ldm set-domain master=mohawk counter
Dependency cycle detected: LDom "counter" indicates "primary" as its master

Determining Where Errors Occur by Mapping CPU and Memory Addresses

This section describes how you can correlate the information that is reported by the Solaris Fault Management Architecture (FMA) with the logical domain resources that are marked as being faulty.

FMA reports CPU errors in terms of physical CPU numbers and memory errors in terms of physical memory addresses.

If you want to determine within which logical domain an error occurred and the corresponding virtual CPU number or real memory address within the domain, then you must perform a mapping.

CPU Mapping

The domain and the virtual CPU number within the domain, which correspond to a given physical CPU number, can be determined with the following procedures.

Determine the CPU Number

1. Generate a long parseable list for all domains.

   primary# ldm list -l -p

2. Look for the entry in the list's VCPU sections that has a pid field equal to the physical CPU number.

   • If you find such an entry, the CPU is in the domain the entry is listed under, and the virtual CPU number within the domain is given by the entry's vid field.

   • If you do not find such an entry, the CPU is not in any domain.

Memory Mapping

The domain and the real memory address within the domain, which correspond to a given physical memory address (PA), can be determined as follows.

Determine the Real Memory Address

1. Generate a long parseable list for all domains.

   primary# ldm list -l -p

2. Look for the line in the list's MEMORY sections where the PA falls within the inclusive range pa to (pa + size - 1); that is, pa <= PA < (pa + size - 1).

   Here pa and size refer to the values in the corresponding fields of the line.

   • If you find such an entry, the PA is in the domain the entry is listed under and the corresponding real address within the domain is given by ra + (PA - pa).

   • If you do not find such an entry, the PA is not in any domain.

Examples of CPU and Memory Mapping

Suppose you have a logical domain configuration as shown in Example 11–1, and you want to determine the domain and the virtual CPU corresponding to physical CPU number 5, and the domain and the real address corresponding to physical address 0x7e816000.

Looking through the VCPU entries in the list for the one with the pid field equal to 5, you can find the following entry under logical domain ldg1.

|vid=1|pid=5|util=29|strand=100

Hence, the physical CPU number 5 is in domain ldg1 and within the domain it has virtual CPU number 1.

Looking through the MEMORY entries in the list, you can find the following entry under domain ldg2.

ra=0x8000000|pa=0x78000000|size=1073741824

Where 0x78000000 <= 0x7e816000 <= (0x78000000 + 1073741824 - 1); that is, pa <= PA <= (pa + size - 1). Hence, the PA is in domain ldg2 and the corresponding real address is 0x8000000 + (0x7e816000 - 0x78000000) = 0xe816000.

Example 11–1 Long Parseable List of Logical Domains Configurations

primary# ldm list -l -p
VERSION 1.0
DOMAIN|name=primary|state=active|flags=normal,control,vio-service|cons=SP|ncpu=4|mem=1073741824|util=0.6|
uptime=64801|softstate=Solaris running
VCPU
|vid=0|pid=0|util=0.9|strand=100
|vid=1|pid=1|util=0.5|strand=100
|vid=2|pid=2|util=0.6|strand=100
|vid=3|pid=3|util=0.6|strand=100
MEMORY
|ra=0x8000000|pa=0x8000000|size=1073741824
IO
|dev=pci@780|alias=bus_a
|dev=pci@7c0|alias=bus_b
...
DOMAIN|name=ldg1|state=active|flags=normal|cons=5000|ncpu=2|mem=805306368|util=29|uptime=903|
softstate=Solaris running
VCPU
|vid=0|pid=4|util=29|strand=100
|vid=1|pid=5|util=29|strand=100
MEMORY
|ra=0x8000000|pa=0x48000000|size=805306368
...
DOMAIN|name=ldg2|state=active|flags=normal|cons=5001|ncpu=3|mem=1073741824|util=35|uptime=775|
softstate=Solaris running
VCPU
|vid=0|pid=6|util=35|strand=100
|vid=1|pid=7|util=34|strand=100
|vid=2|pid=8|util=35|strand=100
MEMORY
|ra=0x8000000|pa=0x78000000|size=1073741824
...