These release notes contain changes for this release, a list of supported platforms, a matrix of required software and patches, and other pertinent information, including bugs that affect Logical Domains (LDoms) 1.2 software.
The Logical Domains 1.2 software is supported on the OpenSolarisTM OS starting with the OpenSolaris 2009.06 release. The Logical Domains 1.2 documentation focuses on the usage of Logical Domains on the Solaris 10 OS. The same Logical Domains features are available for both the Solaris 10 OS and the OpenSolaris OS. However, you might encounter some slight differences when using Logical Domains with the OpenSolaris OS. For more information about the OpenSolaris OS, see the OpenSolaris Information Center.
The major changes for this release of LDoms 1.2 software are as follows:
Support for CPU power management. See Using CPU Power Management Software in Logical Domains 1.2 Administration Guide.
Support for jumbo frames. See Configuring Jumbo Frames in Logical Domains 1.2 Administration Guide.
Restriction of delayed reconfiguration operations to the control domain. See Delayed Reconfiguration in Logical Domains 1.2 Administration Guide.
Support for configuring domain dependencies. See Configuring Domain Dependencies in Logical Domains 1.2 Administration Guide.
Support for autorecovery of configurations. See Managing Logical Domains Configurations in Logical Domains 1.2 Administration Guide.
Support for export of same backend multiple times. See Export a Virtual Disk Backend Multiple Times in Logical Domains 1.2 Administration Guide.
API to support LDMD discovery. See Appendix B, Logical Domains Manager Discovery, in Logical Domains 1.2 Administration Guide.
Support for physical-to-virtual migration tool. See Appendix C, Logical Domains Physical-to-Virtual Migration Tool, in Logical Domains 1.2 Administration Guide.
Support for configuration assistant tools. See Appendix D, Logical Domains Configuration Assistant, in Logical Domains 1.2 Administration Guide.
Bug Fixes
For information about features added to previous versions of the Logical Domains software, see the What's New in Logical Domains Software wiki.
This section contains system requirements for running LDoms software.
Logical Domains Manager 1.2 software is supported on the following platforms:
UltraSPARC® T2 Plus based Servers
Sun SPARC Enterprise® T5140 and T5240 Servers, refer to the Sun SPARC Enterprise T5140 and T5240 Servers Administration Guide
Sun SPARC Enterprise T5440 Server, refer to the Sun SPARC Enterprise T5440 Server Administration Guide
Sun BladeTM T6340 Server Module, refer to the Sun Blade T6340 Server Module Product Notes
NetraTM T5440 Server, refer to the Sun Netra T5440 Server Product Notes
UltraSPARC T2 based Servers
Sun SPARC Enterprise T5120 and T5220 Servers, refer to the Sun SPARC Enterprise T5120 and T5220 Servers Administration Guide
Sun Blade T6320 Server Module, refer to the Sun Blade T6320 Server Module Product Notes
Netra CP3260 Blade, refer to the Netra CP3260 Board Product Notes
Netra T5220 Server, refer to the Sun Netra T5220 Server Product Notes
UltraSPARC T1 based Servers
Sun FireTM or Sun SPARC Enterprise T1000 Server, refer to the Sun Fire Server Administration Guide or SPARC Enterprise T1000 Server Administration Guide
Sun Fire or Sun SPARC Enterprise T2000 Server, refer to the Sun Fire Server Administration Guide or SPARC Enterprise T2000 Server Administration Guide
Netra T2000 Server, refer to the Netra T2000 Server Administration Guide
Netra CP3060 Blade, refer to the Netra CP3060 Board Product Notes
Sun Blade T6300 Server Module, refer to the Sun Blade T6300 Server Module Administration Guide
This section lists the required software and patches for use with LDoms 1.2 software.
To use all the features of LDoms 1.2 software, the operating system on all domains should be at least equivalent to the Solaris 10 5/09 OS. This can be either a fresh or upgrade installation of the following:
OpenSolaris 2009.06 OS
Solaris 10 05/09 OS
Solaris 10 10/08 OS with Patch ID 139555-08
Solaris 10 5/08 OS with Patch ID 139555-08
Solaris 10 8/07 OS with Patch ID 139555-08
Solaris 10 11/06 OS with Patch ID 139555-08
The Logical Domains 1.2 software is supported on the OpenSolaris OS starting with the OpenSolaris 2009.06 release. The Logical Domains 1.2 documentation focuses on the use of Logical Domains on the Solaris 10 OS. The same Logical Domains features are available for both the Solaris 10 OS and the OpenSolaris OS. However, you might encounter slight differences when using Logical Domains with the OpenSolaris OS. For more information about the OpenSolaris OS, see the OpenSolaris Information Center.
Following are the required Solaris 10 5/09 patches for use with LDoms 1.2 software. An X marks whether a patch must be installed on that specific type of domain, but the patches can be applied to all domains.
This patch list includes the minimum required patch revisions. You can install later revisions of the same patch.
Patch ID |
Control Domain |
Service-I/O Domain |
Guest Domain |
---|---|---|---|
141778-02 (Console vntsd) |
X |
|
|
139983-04 (Domain Services) |
X |
|
|
Following is a matrix of required software to enable all the LDoms 1.2 features.
Table 1–2 Required Software to Enable LDoms 1.2 Features
Supported Server |
System Firmware |
Solaris OS |
---|---|---|
UltraSPARC T2 Plus based servers |
7.2.2 |
One of the configurations in Required and Recommended Solaris OS |
UltraSPARC T2 based servers |
7.2.2 |
One of the configurations Required and Recommended Solaris OS |
UltraSPARC T1 based servers |
6.7.4 |
One of the configurations Required and Recommended Solaris OS |
It is possible to run the LDoms 1.2 software along with previous revisions of the other software components. For example, you could have differing versions of the Solaris OS on the various domains in a machine. To take advantage of all features of Logical Domains 1.2, ensure that your logical domains run Solaris 10 5/09 plus the patches listed in Table 1–1. Note that you can run different OS releases in each logical domain on the same system. For those domains that run earlier versions of the OS, you might not have all the features that are available in Logical Domains 1.2. However, an alternate upgrade strategy could be to upgrade the control and service domains to Solaris 10 5/09 plus the patches listed in Table 1–1 and to continue running the guest domains at the existing patch level.
Following is a matrix of the minimum versions of software required. The LDoms 1.2 package, SUNWldm, can be applied to a system running at least the following versions of software. The minimum software versions are platform specific and depend on the requirements of the CPU in the machine. The minimum Solaris OS version for a given CPU type applies to all domain types (control, service, I/O, and guest).
Table 1–3 Minimum Versions of Software
Supported Server |
System Firmware |
Solaris OS |
---|---|---|
UltraSPARC T2 Plus based servers |
7.1.x |
Solaris 10 8/07 plus patch ID 127111-08 at a minimum |
UltraSPARC T2 based servers |
7.1.x |
Solaris 10 8/07 |
UltraSPARC T1 based servers |
6.6.x |
Solaris 10 11/06 plus patch IDs 124921-02, 125043-01, and KU 118833-36 at a minimum |
To take advantage of all features of Logical Domains 1.2, ensure that your server runs at least these revisions of the following system firmware patches:
Sun Fire and Sun SPARC Enterprise T2000 Servers
Sun Fire and Sun SPARC Enterprise T1000 Servers
Netra T2000 Server
Netra CP3060 Blade
Sun Blade T6300 Server Module
Sun SPARC Enterprise T5120 and T5220 Servers
Sun Blade T6320 Server Module
Netra T5220 Server
Sun SPARC Enterprise T5140 and T5240 Servers
Netra T5440 Server
Sun SPARC Enterprise T5440 Server
Sun Blade T6340 Server Module
You can find the LDoms 1.2 software to download at http://www.sun.com/ldoms.
The LDoms_Manager-1_2.zip file that you download contains the following:
Logical Domains Manager 1.2 software (SUNWldm.v)
ldm(1M) man page in the SUNWldm.v package that gets installed when the package is installed
Installation script for Logical Domains Manager 1.2 software and the Solaris Security Toolkit (install-ldm)
Solaris Security Toolkit (SUNWjass)
Logical Domains Management Information Base (SUNWldmib.v)
Physical to Virtual Migration Tool (SUNWldmp2v)
Configuration Assistant GUI (Configurator.jar)
The directory structure of the zip file is similar to the following:
LDoms_Manager-1_2/ Install/ install-ldm Legal/ 819-0764-10_SLA_Multi.pdf LDoms_1.2_DistributionREADME.txt LDoms_1.2_SLA&Entitlement(11June2009).txt Ldoms_MIB_1.0.1_Entitlement.txt Ldoms_MIB_1.0.1_SLA_Entitlement.txt Ldoms_MIB_1.0.1_TranslatedSLA.pdf Product/ Configurator/ Configurator.jar README.GUI SUNWjass SUNWldm.v SUNWldmib.v SUNWldmp2v README |
You can find the required Solaris OS and system firmware patches at the SunSolveSM site:
The Logical Domains 1.2 Administration Guide and these Logical Domains 1.2 Release Notes can be obtained from:
http://docs.sun.com/app/docs/prod/ldoms
The Sun Logical Domains Wiki contains Best Practices, Guidelines, and Recommendations for deploying LDoms software.
http://wikis.sun.com/display/SolarisLogicalDomains/Home
The Beginners Guide to LDoms: Understanding and Deploying Logical Domains can be used to get a general overview of Logical Domains software. However, the details of the guide specifically apply to the LDoms 1.0 software release and are now out of date for LDoms 1.2 software. The guide can be found at the Sun BluePrintsTM site.
http://www.sun.com/blueprints/0207/820-0832.html
This section describes software that is related to LDoms software.
Solaris Security Toolkit 4.2 software can help you secure the Solaris OS in the control domain and other domains. Refer to the Solaris Security Toolkit 4.2 Administration Guide and the Solaris Security Toolkit 4.2 Reference Manual for more information.
The Logical Domains 1.2 software comes bundled with Version 4.2 of the Solaris Security Toolkit (SST) to provide security services. Beginning with the next release of Logical Domains, Sun plans to remove SST from the Logical Domains software bundle.
You can still download and use SST to harden your Logical Domains systems by using the new version of SST. SST 5.0 is available for the Solaris 10 OS as well as for the OpenSolaris OS. You can get information about the project and access the source code from the http://opensolaris.org/os/project/sst/ project page.
Logical Domains Management Information Base (MIB) software can help you enable third-party applications to perform remote monitoring and a few control operations. Refer to the Logical Domains (LDoms) MIB 1.0.1 Administration Guide and the Logical Domains (LDoms) MIB 1.0.1 Release Notes for more information.
LDoms MIB software for LDoms software works with LDoms 1.0.1 software at a minimum.
This section details the software that is compatible with and can be used with the Logical Domains software. Be sure to check the software documentation or your platform documentation to find the version number of the software that is available for your version of the LDoms software and your platform.
SunVTS functionality is available in the control domain and guest domains on certain LDoms software releases and certain platforms. SunVTSTM is Sun's Validation Test Suite, which provides a comprehensive diagnostic tool that tests and validates Sun hardware by verifying the connectivity and proper functioning of most hardware controllers and devices on Sun servers. For more information about SunVTS, refer to the SunVTS 5.0 User’s Guide.
Sun Management Center 4.0 Add-On Software can be used only on the control domain with the Logical Domains Manager software enabled. Sun Management Center is an open, extensible system monitoring and management solution. It uses the JavaTM runtime environment and a variant of the Simple Network Management Protocol (SNMP) to provide integrated and comprehensive enterprise-wide management of Sun products and their subsystems, components, and peripheral devices. Support for hardware monitoring within the Sun Management Center environment is achieved through the use of appropriate hardware server module add-on software, which presents hardware configuration and fault reporting information to the Sun Management Center server and console. Refer to the Sun Management Center 4.0 Release Notes for more information about using Sun Management Center 4.0 on the supported servers.
Sun Explorer Data Collector can be used with the Logical Domains Manager software enabled on the control domain. Sun Explorer is a diagnostic data collection tool. The tool comprises shell scripts and a few binary executables. Refer to the Sun Explorer User’s Guide for more information about using the Sun Explorer Data Collector.
Solaris Cluster software can be used only on an I/O domain in Logical Domains software releases up through LDoms 1.0.2. In LDoms 1.0.3, 1.1, and 1.2 software, Solaris Cluster software can be used in a guest domain with some restrictions. Refer to Solaris Cluster documentation for more information about any restrictions and about the Solaris Cluster software in general.
The following system controller (SC) software interacts with the LDoms 1.2 software:
Sun Integrated Lights Out Manager (ILOM) 3.0 firmware is the system management firmware that you can use to monitor, manage, and configure UltraSPARC T2 based server platforms. ILOM is preinstalled on these platforms and can be used on the control domain on LDoms-supported servers with the Logical Domains Manager 1.2 software enabled. Refer to the Sun Integrated Lights Out Manager 3.0 User's Guide for features and tasks that are common to Sun rackmounted servers or blade servers that support ILOM. Other user documents present ILOM features and tasks that are specific to the server platform that you are using. You can find the ILOM platform-specific information within the documentation set that accompanies your system.
Advanced Lights Out Manager (ALOM) Chip Multithreading (CMT) Version 1.3 software can be used on the control domain on UltraSPARC T1 based servers with the Logical Domains Manager 1.0.1 software enabled. Refer to Using LDoms With ALOM CMT in Logical Domains 1.2 Administration Guide. The ALOM system controller enables you to remotely manage and administer your supported CMT servers. ALOM enables you to monitor and control your server either over a network or by using a dedicated serial port for connection to a terminal or terminal server. ALOM provides a command-line interface that you can use to remotely administer geographically distributed or physically inaccessible machines. For more information about using ALOM CMT Version 1.3 software, refer to the Advanced Lights Out Management (ALOM) CMT v1.3 Guide.
Netra Data Plane Software Suite is a complete board software package solution. The software provides an optimized rapid development and runtime environment on top of multistrand partitioning firmware for Sun CMT platforms. The Logical Domains Manager contains some ldm subcommands (add-vdpcs, rm-vdpcs, add-vdpcc, and rm-vdpcc) for use with this software. Refer to the Netra Data Plane Software Suite 2.0 User’s Guide for more information about this software.
This section contains general issues and specific bugs concerning the LDoms 1.2 software.
This section describes general known issues about this release of LDoms software that are broader than a specific bug number. Workarounds are provided where available.
For discussions in Logical Domains documentation, the terms service processor (SP) and system controller (SC) are interchangeable.
The following cards are not supported for this LDoms 1.2 software release:
Sun Dual Port 4x IB Host Channel Adapter PCI-X Card
Dual Port 4x PCI EXPRESS® Infiniband Host Channel Adapter - Low Profile
If these unsupported configurations are used with LDoms 1.2, stop and unbind all logical domains before the control domain is rebooted. Failure to do so can result in a system crash causing the loss of all the logical domains that are active in the system.
If a service domain is running a version of Solaris 10 OS prior to Solaris 10 5/09 and is exporting a physical disk slice as a virtual disk to a guest domain, then this virtual disk will appear in the guest domain with an inappropriate device ID. If that service domain is then upgraded to Solaris 10 5/09, the physical disk slice exported as a virtual disk will appear in the guest domain with no device ID.
This removal of the device ID of the virtual disk can cause problems to applications attempting to reference the device ID of virtual disks. In particular, this can cause the Solaris Volume Manager (SVM) to be unable to find its configuration or to access its metadevices.
Workaround: After upgrading a service domain to Solaris 10 5/09, if a guest domain is unable to find its SVM configuration or its metadevices, execute the following procedure.
Boot the guest domain.
Disable the devid feature of SVM by adding the following lines to the /kernel/dr/md.conf file:
md_devid_destroy=1; md_keep_repl_state=1; |
Reboot the guest domain.
After the domain has booted, the SVM configuration and metadevices should be available.
Check the SVM configuration and ensure that it is correct.
Re-enable the SVM devid feature by removing from the /kernel/drv/md.conf file the two lines that you added in Step 2.
Reboot the guest domain.
During the reboot, you will see messages similar to this:
NOTICE: mddb: unable to get devid for 'vdc', 0x10 |
These messages are normal and do not report any problems.
There is a limit to the number of LDCs available in any logical domain. For UltraSPARC T1 based platforms, that limit is 256. For all other platforms, the limit is 512. This only becomes an issue on the control domain because the control domain has at least part, if not all, of the I/O subsystem allocated to it. This might also be an issue because of the potentially large number of LDCs that are created for both virtual I/O data communications and the Logical Domains Manager control of the other logical domains.
The examples in this section are what happens on UltraSPARC T1 based platforms. However, the behavior is the same if you go over the limit on other supported platforms.
If you try to add a service, or bind a domain, so that the number of LDC channels exceeds the limit on the control domain, the operation fails with an error message similar to the following:
13 additional LDCs are required on guest primary to meet this request, but only 9 LDCs are available |
The following guidelines can help prevent creating a configuration that could overflow the LDC capabilities of the control domain:
The control domain allocates 12 LDCs for various communication purposes with the hypervisor, Fault Management Architecture (FMA), and the system controller (SC), independent of the number of other logical domains configured.
The control domain allocates 1 LDC to every logical domain, including itself, for control traffic.
Each virtual I/O service on the control domain consumes 1 LDC for every connected client of that service.
For example, consider a control domain and 8 additional logical domains. Each logical domain needs the following at a minimum:
Virtual network
Virtual disk
Virtual console
Applying the above guidelines yields the following results (numbers in parentheses correspond to the preceding guideline number from which the value was derived):
12(1) + 9(2) + 8 x 3(3)=45 LDCs in total.
Now consider the case where there are 32 domains instead of 8, and each domain includes 3 virtual disks, 3 virtual networks, and a virtual console. Now the equation becomes:
12 + 33 + 32 x 7=269 LDCs in total.
Depending upon the number of supported LDCs of your platform, the Logical Domains Manager will either accept or reject the configurations.
Logical Domains software does not impose a memory size limitation when creating a domain. The memory size requirement is a characteristic of the guest operating system. Some Logical Domains functionality might not work if the amount of memory present is less than the recommended size. For recommended and minimum size memory requirements, refer to the installation guide for the operating system you are using. Refer to System Requirements and Recommendations in Solaris 10 5/09 Installation Guide: Planning for Installation and Upgrade.
The OpenBootTM PROM has a minimum size restriction for a domain. Currently, that restriction is 12 Mbytes. If you have a domain less than that size, the Logical Domains Manager will automatically boost the size of the domain to 12 Mbytes. Refer to the release notes for your system firmware for information about memory size requirements.
You can the boot following number of domains depending on your platform:
Up to 128 on UltraSPARC T2 Plus based servers
Up to 64 on UltraSPARC T2 based servers
Up to 32 on UltraSPARC T1 based servers
If unallocated virtual CPUs are available, assign them to the service domain to help process the virtual I/O requests. Allocate 4 to 8 virtual CPUs to the service domain when creating more than 32 domains. In cases where maximum domain configurations have only a single CPU in the service domain, do not put unnecessary stress on the single CPU when configuring and using the domain. The virtual switch (vsw) services should be spread over all the network adapters available in the machine. For example, if booting 128 domains on a Sun SPARC Enterprise T5240 server, create 4 vsw services, each serving 32 virtual net (vnet) instances. Do not have more than 32 vnet instances per vsw service because having more than that tied to a single vsw could cause hard hangs in the service domain.
To run the maximum configurations, a machine needs the following amount of memory, depending on your platform, so that the guest domains contain an adequate amount of memory:
128 Gbytes of memory for UltraSPARC T2 Plus based servers
64 Gbytes of memory for UltraSPARC T2 based servers
32 Gbytes of memory for UltraSPARC T1 based servers
Memory and swap space usage increases in a guest domain when the vsw services used by the domain provides services to many virtual networks (in multiple domains). This is due to the peer-to-peer links between all the vnet connected to the vsw. The service domain benefits from having extra memory. Four Gbytes is the recommended minimum when running more than 64 domains. Start domains in groups of 10 or less and wait for them to boot before starting the next batch. The same advice applies to installing operating systems on domains.
If you have made any configuration changes since last saving a configuration to the SC, before you attempt to power off or power cycle a Logical Domains system, make sure that you save the latest configuration that you want to keep.
Shut down and unbind all the non-I/O domains.
Shut down and unbind any active I/O domains.
Halt the primary
domain.
Because no other domains are bound, the firmware automatically powers off the system.
Shut down and unbind all the non-I/O domains.
Shut down and unbind any active I/O domains.
Reboot the primary
domain.
Because no other domains are bound, the firmware automatically power cycles the system before rebooting it. When the system restarts, it boots into the Logical Domains configuration last saved or explicitly set.
Under certain circumstances, the Logical Domains Manager rounds up the requested memory allocation to either the next largest 8-Kbyte or 4-Mbyte multiple. This can be seen in the following example output of the ldm list-domain -l command, where the constraint value is smaller than the actual allocated size:
Memory: Constraints: 1965 M raddr paddr5 size 0x1000000 0x291000000 1968M |
With domaining enabled, variable updates persist across a reboot, but not across a powercycle, unless the variable updates are either initiated from OpenBoot firmware on the control domain, or followed by saving the configuration to the SC.
In this context, it is important to note that a reboot of the control domain could initiate a powercycle of the system:
When the control domain reboots, if there are no bound guest domains, and no delayed reconfiguration in progress, the SC powercycles the system.
When the control domain reboots, if there are guest domains bound or active (or the control domain is in the middle of a delayed reconfiguration), the SC does not powercycle the system.
LDom variables for a domain can be specified using any of the following methods:
At the OpenBoot prompt
Using the Solaris OS eeprom(1M) command
Using the Logical Domains Manager CLI (ldm
)
Modifying, in a limited fashion, from the system controller (SC) using the bootmode command, that is, only certain variables, and only when in the factory-default configuration
The goal is that, variable updates that are made by using any of these methods always persist across reboots of the domain. The variable updates also always reflect in any subsequent logical domain configurations that were saved to the SC.
In LDoms 1.2 software, there are a few cases where variable updates do not persist as expected:
With domaining enabled, all methods of updating a variable persist across reboots of that domain. However, they do not persist across a powercycle of the system, unless a subsequent logical domain configuration is saved to the SC. The methods of updating a variable include by OpenBoot firmware and by the eeprom and ldm command. Domaining is enabled by default except for the UltraSPARC T1000 and T2000 systems that run in factory-default configuration. In addition, in the control domain, updates made using OpenBoot firmware persist across a powercycle of the system, that is, even without subsequently saving a new logical domain configuration to the SC.
When domaining is not enabled, variable updates specified through the eeprom(1M) command persist across a reboot of the primary domain into the same factory-default configuration, but do not persist into a configuration saved to the SC. Conversely, in this scenario, variable updates specified using the Logical Domains Manager do not persist across reboots, but are reflected in a configuration saved to the SC.
So, when domaining is not enabled, if you want a variable update to persist across a reboot into the same factory-default configuration, use the eeprom command. If you want it saved as part of a new logical domains configuration saved to the SC, use the appropriate Logical Domains Manager command.
In all cases, when reverting to the factory-default configuration from a configuration generated by the Logical Domains Manager, all LDoms variables start with their default values.
The following Bug IDs have been filed to resolve these issues: 6520041, 6540368, 6540937, and 6590259.
Sun Simple Management Network Protocol (SNMP) Management Agent does not support multiple domains. Only a single global domain is supported.
The sysfwdownload utility takes significantly longer to run from within a Logical Domains environment on systems based on UltraSPARC T1 processors. This happens if you use the sysfwdownload utility while the LDoms software is enabled.
Workaround: Boot to the factory-default configuration with the LDoms software disabled before using the utility.
Using CPU dynamic reconfiguration (DR) to power down virtual CPUs does not work with processor sets, resource pools, or the zone's dedicated CPU feature.
When using CPU power management in elastic mode, the Solaris OS guest sees only the CPUs that are allocated to the domains that are powered on. That means that output from the psrinfo(1M) command dynamically changes depending on the number of CPUs currently power-managed. This causes an issue with processor sets and pools, which require actual CPU IDs to be static to allow allocation to their sets. This can also impact the zone's dedicated CPU feature.
Workaround: Set the performance mode for the power management policy.
A domain is in transition when booting, shutting down, at the ok prompt, or in the kernel debugger. Use the ldm list command to determine whether a guest domain is in the transition state. The command output shows a t flag for any domain that is in the transition state. To enable CPU Power Management for the other domains, boot the guest domain that is in the transition state, or use the ldm stop command to stop that guest domain.
There are several issues associated with FMA and power-managing CPUs. If a CPU faults when running in elastic mode, switch to performance mode until the faulted CPU recovers. If all faulted CPUs recover, then elastic mode can be used again.
For more information about faulted resources, see the OpenSolaris Fault Management web page.
When a primary domain is in a delayed reconfiguration state, CPUs are power managed only after the primary domain reboots. This means that CPU power management will not bring additional CPUs online when the domain is experiencing high-load usage until the primary domain reboots, clearing the delayed reconfiguration state.
Domain migrations are not supported for a source or target machine in elastic mode. If a migration is underway while in performance mode and the power management policy is set to elastic mode, the policy switch is deferred until the migration completes. The migration command returns an error if either the source or target machine is in elastic mode and a domain migration is attempted.
The power management feature requires dynamic CPU DR to function. So, do not use the power management feature in Integrated Lights-Out Management (ILOM) if your domains are to have cryptographic units bound. Currently, the Solaris OS support for cryptographic DR does not support CPU DR without a guest reboot.
This section summarizes the bugs that you might encounter when using this version of the software. The bug descriptions are in numerical order by bug ID. If a workaround and a recovery procedure are available, they are specified.
Bug ID 6447740: The Logical Domains Manager does not validate disk paths and network devices.
If a disk device listed in a guest domain's configuration is either non-existent or otherwise unusable, the disk cannot be used by the virtual disk server (vds). However, the Logical Domains Manager does not emit any warning or error when the domain is bound or started.
When the guest tries to boot, messages similar to the following are printed on the guest's console:
WARNING: /virtual-devices@100/channel-devices@200/disk@0: Timeout connecting to virtual disk server... retrying |
In addition, if a network interface specified using the net-dev= property does not exist or is otherwise unusable, the virtual switch is unable to communicate outside the physical machine, but the Logical Domains Manager does not emit any warning or error when the domain is bound or started.
Issue the ldm set-vsw command with the corrected net-dev property value.
Reboot the domain hosting the virtual switch in question.
Stop the domain owning the virtual disk bound to the errant device or volume.
Issue the ldm rm-vdsdev command to remove the errant virtual disk service device.
Issue the ldm add-vdsdev command to correct the physical path to the volume.
Restart the domain owning the virtual disk.
If a disk device listed in a guest domain's configuration is being used by software other than the Logical Domains Manager (for example, if it is mounted in the service domain), the disk cannot be used by the virtual disk server (vds), but the Logical Domains Manager does not emit a warning that it is in use when the domain is bound or started.
When the guest domain tries to boot, a message similar to the following is printed on the guest's console:
WARNING: /virtual-devices@100/channel-devices@200/disk@0: Timeout connecting to virtual disk server... retrying |
Unbind the guest domain.
Unmount the disk device to make it available.
Bind the guest domain.
Boot the domain.
Bug ID 6497796: Under rare circumstances, when a Logical Domains variable, such as boot-device, is being updated from within a guest domain by using the eeprom(1M) command at the same time that the Logical Domains Manager is being used to add or remove virtual CPUs from the same domain, the guest OS can hang.
Workaround: Ensure that these two operations are not performed simultaneously.
Recovery: Use the ldm stop-domain and ldm start-domain commands to stop and start the guest OS.
Bug ID 6506494: There are some cases where the behavior of the ldm stop-domain command is confusing.
# ldm stop-domain -f ldom |
If the domain is at the kernel module debugger, kmdb(1), prompt, then the ldm stop-domain command fails with the following error message:
LDom <domain name> stop notification failed |
Bug ID 6510214: In a Logical Domains environment, there is no support for setting or deleting wide-area network (WAN) boot keys from within the Solaris OS by using the ickey(1M) command. All ickey
operations fail with the following error:
ickey: setkey: ioctl: I/O error |
In addition, WAN boot keys that are set using OpenBoot firmware in logical domains other than the control domain are not remembered across reboots of the domain. In these domains, the keys set from the OpenBoot firmware are only valid for a single use.
Bug ID 6590259: This issue is summarized in Logical Domain Variable Persistence.
Bug ID 6533696: On a system configured to use the Network Information Services (NIS) or NIS+ name service, if the Solaris Security Toolkit software is applied with the server-secure.driver, NIS or NIS+ fails to contact external servers. A symptom of this problem is that the ypwhich(1) command (which returns the name of the NIS or NIS+ server or map master) fails with a message similar to the following:
Domain atlas some.atlas.name.com not bound on nis-server-1.c |
The recommended Solaris Security Toolkit driver to use with the Logical Domains Manager is ldm_control-secure.driver, and NIS and NIS+ work with this recommended driver.
If you are using NIS as your name server, you cannot use the Solaris Security Toolkit profile server-secure.driver because you might encounter Solaris OS Bug ID 6557663, IP Filter causes panic when using ipnat.conf. However, the default Solaris Security Toolkit driver, ldm_control-secure.driver, is compatible with NIS.
Log in to the system console from the system controller, and if necessary, switch to the ALOM mode by typing:
# #. |
Power off the system by typing the following command in ALOM mode:
sc> poweroff |
Power on the system.
sc> poweron |
Switch to the console mode at the ok prompt:
sc> console |
Power on the system.
ok boot -s |
Edit the file /etc/shadow.
Change the root entry of the shadow file to the following:
root::6445:::::: |
Log in to the system and do one of the following:
Add file /etc/ipf/ipnat.conf.
Undo the Solaris Security Toolkit, and apply another driver.
# /opt/SUNWjass/bin/jass-execute -ui # /opt/SUNWjass/bin/jass-execute -a ldm_control-secure.driver |
Bug ID 6486234: The virtual networking infrastructure adds additional overhead to communications from a logical domain. All packets are sent through a virtual network device, which, in turn, passes the packets to the virtual switch. The virtual switch then sends the packets out through the physical device. The lower performance is seen due to the inherent overheads of the stack.
Workaround: Do one of the following depending on your server:
On UltraSPARC T1 based servers, such as the Sun Fire T1000 and T2000, and UltraSPARC T2+ based servers such as the Sun SPARC Enterprise T5140 and T5240, assign a physical network card to the logical domain using a split-PCI configuration. For more information, refer to I/O Domains and PCI EXPRESS Buses in Logical Domains 1.2 Administration Guide.
On Ultra SPARC T2 based servers, such as the Sun SPARC Enterprise T5120 and T5220 servers, assign a Network Interface Unit (NIU) to the logical domain.
Bug ID 6590259: If the time or date on a logical domain is modified, for example using the ntpdate command, the change persists across reboots of the domain but not across a power cycle of the host.
Workaround: For time changes to persist, save the configuration with the time change to the SC and boot from that configuration.
Bug ID 6540368: This issue is summarized in Logical Domain Variable Persistence and affects only the control domain.
Bug ID 6544004: The following message appears at the ok prompt if an attempt is made to boot a guest domain that contains an Emulex-based Fibre Channel host adapter (Sun Part Number 375-3397):
ok> FATAL:system is not bootable, boot command is disabled |
Workaround: Do not use this adapter in a split-PCI configuration on Sun Fire T1000 servers.
Bug ID 6549382: If SunVTS is started and stopped multiple times, it is possible that using the console SC command to switch from the SC console to the host console can result in either of the following messages being repeatedly emitted on the console:
Enter #. to return to ALOM. Warning: Console connection forced into read-only mode |
Recovery: Reset the SC using the resetsc command.
Bug ID 6589660: Virtual disk timeouts do not work if either the guest or control domain using the disk is halted, for example, if the domain is taken into the kernel debugger (kmdb) or taken into the OpenBoot PROM with the send break.
Workaround: None.
Bug ID 6591844: If a CPU or memory fault occurs, the affected domain might panic and reboot. If the Fault Management Architecture (FMA) attempts to retire the faulted component while the domain is rebooting, the Logical Domains Manager is not able to communicate with the domain, and the retire fails. In this case, the fmadm faulty command lists the resource as degraded.
Recovery: Wait for the domain to complete rebooting, and then force FMA to replay the fault event by restarting the fault manager daemon (fmd) on the control domain by using this command:
primary# svcadm restart fmd |
Bug ID 6603974: If you configure more than four virtual networks (vnets) in a guest domain on the same network using the Dynamic Host Protocol (DHCP), the guest domain can eventually become unresponsive while running network traffic.
Workaround: Set ip_ire_min_bucket_cnt and ip_ire_max_bucket_cnt to larger values, such as 32, if you have 8 interfaces.
Recovery: Issue an ldm stop-domain ldom command followed by an ldm start-domain ldom command on the guest domain (ldom) in question.
Bug ID 6604253: If you run the Solaris 10 11/06 OS and you harden drivers on the primary domain that is configured with only one strand, rebooting the primary domain or restarting the fault manager daemon (fmd) can result in an fmd core dump. The fmd dumps core while it cleans up its resources, and this does not affect the FMA diagnosis.
Workaround: Add a few more strands into the primary domain. For example,
# ldm add-vcpu 3 primary |
Bug ID 6629230: The scadm command on a control domain running at least the Solaris 10 11/06 OS can hang following an SC reset. The system is unable to properly reestablish a connection following an SC reset.
Workaround: Reboot the host to reestablish connection with the SC.
Recovery: Reboot the host to reestablish connection with the SC.
Bug ID 6656033: Simultaneous net installation of multiple guest domains fails on Sun SPARC Enterprise T5140 and Sun SPARC Enterprise T5240 systems that have a common console group.
Workaround: Only net-install on guest domains that each have their own console group. This failure is seen only on domains with a common console group shared among multiple net-installing domains.
Bug ID 6694939: In certain cases, the prtdiag(1M) command does not list all the CPUs.
Workaround: For an accurate count of CPUs, use the psrinfo(1M) command.
Bug ID 6687634: If the Sun Volume Manager (SVM) volume is built on top of a disk slice that contains block 0 of the disk, then SVM prevents writing to block 0 of the volume to avoid overwriting the label of the disk.
If an SVM volume built on top of a disk slice that contains block 0 of the disk is exported as a full virtual disk, then a guest domain is unable to write a disk label for that virtual disk, and this prevents the Solaris OS from being installed on such a disk.
Workaround: SVM volumes exported as a virtual disk should not be built on top of a disk slice that contains block 0 of the disk.
A more generic guideline is that slices that start on the first block (block 0) of a physical disk should not be exported (either directly or indirectly) as a virtual disk. Refer to Directly or Indirectly Exporting a Disk Slice in Logical Domains 1.2 Administration Guide.
Bug ID 6705823: Attempting a net boot of the Solaris 10 8/07 OS on any guest domain serviced by a service domain running the Solaris 10 5/08 OS can result in a hang on the guest domain during the installation.
Workaround: Patch the miniroot of the Solaris 10 8/07 OS net install image with Patch ID 127111-05.
Bug ID 6713547: Cryptographic dynamic reconfiguration (DR) changes are incompatible with firmware that is prior to LDoms software releases. This problem prevents UltraSPARC T1 based systems running old firmware from using cryptographic hardware.
Bug ID 6742805: A domain shutdown or memory scrub can take over 15 minutes with a single CPU and a very large memory configuration. During a shutdown, the CPUs in a domain are used to scrub all the memory owned by the domain. The time taken to complete the scrub can be quite long if a configuration is imbalanced, for example, a single CPU domain with 512 Gbytes of memory. This prolonged scrub time extends the amount of time it takes to shut down a domain.
Workaround: Ensure that large memory configurations (>100 Gbytes) have at least one core. This results in a much faster shutdown time.
Bug ID 6753219: After adding virtual switches to the primary domain and rebooting, the primary domain hangs when installed with an Elara Copper card.
Workaround: Add this line to the /etc/system file on the service domain and reboot:
set vsw:vsw_setup_switching_boot_delay=300000000 |
Bug ID 6753683: Sometimes, executing the uadmin 1 0 command from the command line of an LDoms system does not leave the system at the OK prompt after the subsequent reset. This incorrect behavior is seen only when the LDoms variable auto-reboot? is set to true. If auto-reboot? is set to false, the expected behavior occurs.
Workaround: Use this command instead:
uadmin 2 0 |
Or, always run with auto-reboot? set to false.
Bug ID 6760933: On occasion, an active logical domain appears to be in the transition state instead of the normal state long after it is booted or following the completion of a domain migration. This glitch is harmless, and the domain is fully operational. To see what flag is set, check the flags field in the ldm list -l -p command output, or check the FLAGS field in the ldm list command, which shows -n---- for normal or -t---- for transition.
Recovery: The logical domain should display the correct state upon the next reboot.
Bug ID 6764613: If you do not have a network configured on your machine and have a Network Information Services (NIS) client running, the Logical Domains Manager will not start on your system.
Workaround: Disable the NIS client on your non-networked machine:
# svcadm disable nis/client |
Bug ID 6765355: Under rare conditions, when a new virtual network (vnet) is added to a logical domain, it fails to establish a connection with the virtual switch. This results in loss of network connectivity to and from the logical domain. If you encounter this error, you can see that the /dev/vnetN symbolic link for the virtual network instance is missing. If present, and not in error, the link points to a corresponding /devices entry as follows:
/dev/vnetN -> ../devices/virtual-devices@100/channel-devices@200/network@N:vnetN |
Workaround: Do one of the following:
Perform a reconfiguration boot of the logical domain, whenever a vnet is added to the logical domain.
If the logical domain is already booted, run the devfsadm(1M) command before plumbing the vnet.
Bug ID 6766202: If a guest domain with only one CPU is at the kernel module debugger, kmdb(1), prompt, and if that domain is migrated to another system, then the guest domain panics when it is resumed on the target system.
Workaround: Before migrating a guest domain, exit the kmdb shell, and resume the execution of the OS by typing ::cont. Then migrate the guest domain. After the migration is completed, re-enter kmdb with the command mdb -K.
Bug ID 6769808: If a service domain running up to the Solaris 10 5/08 OS is exporting a ZFS volume as a single-slice disk to a guest domain running the Solaris 10 10/08 OS, then this guest domain is unable to use that virtual disk. Any access to the virtual disk fails with an I/O error.
Workaround: Upgrade the service domain to Solaris 10 5/09.
Bug ID 6772089: In certain situations, a migration fails and ldmd reports that it was not possible to bind the memory needed for the source domain. This can occur even if the total amount of available memory on the target machine is greater than the amount of memory being used by the source domain.
This failure occurs because migrating the specific memory ranges in use by the source domain requires that compatible memory ranges are available on the target, as well. When no such compatible memory range is found for any memory range in the source, the migration cannot proceed.
Recovery: If this condition is encountered, you might be able to migrate the domain if you modify the memory usage on the target machine. To do this, unbind any bound or active logical domain on the target.
Use the ldm list-devices -a mem command to see what memory is available and how it is used. You might also need to reduce the amount of memory that is assigned to another domain.
Bug ID 6772120: If the virtual disk on the target machine does not point to the same disk backend that is used on the source machine, the migrated domain cannot access the virtual disk using that disk backend. A hang can result when accessing the virtual disk on the domain.
Currently, the Logical Domains Manager checks only that the virtual disk volume names match on the source and target machines. In this scenario, no error message is displayed if the disk backends do not match.
Workaround: Ensure that when you are configuring the target domain to receive a migrated domain that the disk volume (vdsdev) matches the disk backend used on the source domain.
Recovery: Do one of the following if you discover that the virtual disk device on the target machine points to the incorrect disk backend:
Do the following:
Migrate the domain back to the source machine.
Fix the vdsdev on the target to point to the correct disk backend.
Migrate the domain to the target machine again.
Stop and unbind the domain on the target, and fix the vdsdev. If the OS supports virtual I/O dynamic reconfiguration, and the incorrect virtual disk in not in use on the domain (that is, it is not the boot disk and is unmounted), do the following:
Use the ldm rm-vdisk command to remove the disk.
Fix the vdsdev.
Use the ldm add-vdisk command to add the virtual disk again.
Bug ID 6773569: After switching from one configuration to another (using the ldm set-config command followed by a powercycle), domains defined in the previous configuration might still be present in the current configuration, in the inactive state.
This is a result of the Logical Domains Manager's constraint database not being kept in sync with the change in configuration. These inactive domains do not affect the running configuration and can be safely destroyed.
Bug ID 6781589: During a migration, any explicitly assigned console group and port are ignored, and a console with default properties is created for the target domain. This console is created using the target domain name as the console group and using any available port on the first virtual console concentrator (vcc) device in the control domain. If there is a conflict with the default group name, the migration fails.
Recovery: To restore the explicit console properties following a migration, unbind the target domain, and manually set the desired properties using the ldm set-vcons command.
Bug ID 6784945: On a Sun SPARC Enterprise T5440 system, the pseudonyms (shortcut names) for the PCI buses are not correct.
Workaround: To configure PCI buses on a Sun SPARC Enterprise T5440 system, you must use the pci@xxxx form of the bus name, as listed under the DEVICE column of any of the following list commands:
ldm list -l ldom
ldm list -o physio ldom
ldm list-devices
Bug ID 6787057: On a guest domain with two or more virtual network devices (vnets) using multiple virtual switches (vsws), if an in-progress migration is cancelled, the domain being migrated might reboot instead of resuming operation on the source machine with the OS running. This issue does not occur if all the vnets are connected to a single vsw.
Workaround: If you are migrating a domain with two or more virtual networks using multiple virtual switches, do not cancel the domain migration (either by using Ctrl-C or the ldm cancel-operation command) after the operation starts. If a domain is inadvertently migrated, it can be migrated back to the source machine after the original migration is completed.
Bug ID 6703127: Virtual input/output (VIO) dynamic reconfiguration (DR) operations ignore the -f (force) option in CLI commands.
Bug ID 6736962: Power Management sometimes fails to retrieve policy from the service processor on LDoms startup after the control domain boots. If CPU power management could not retrieve the power management policy from the service processor, it allows LDoms to start up as expected, but logs the error Unable to get the initial PM Policy - timeout to the LDoms log and remains in performance mode.
Add forceload: drv/ds_snmp to /etc/system, then reboot the control domain.
Bug ID 6703958: Under rare circumstances, running CPU dynamic reconfiguration (DR) operations in parallel with network interface-related operations, such as plumb or unplumb, can result in a deadlock.
Workaround: Minimize the risk by avoiding network interface-related operations. If this deadlock occurs while booting a domain, set the domain to 2 CPUs and then reboot the domain.
Bug ID 6759853: The following error message might be written intermittently to the LDoms log when a domain is at the ok prompt:
fma_cpu_svc_get_p_status: Can't find fma_cpu_get_p_status routine error |
Workaround: Boot the domain.
Bug ID 6848114: ldmconfig can run on a system that does not have file systems of sufficient capacity to contain the virtual disks for the created domains. In this situation, an error message is issued. However, ldmconfig permits you to continue to use the disks that are in /ldoms/disks to deploy the configuration. This situation could cause the root file system of the control domain to become full and halt the system.
Workaround: Do the following:
Exit the Configuration Assistant by typing q or by typing Ctrl-C.
Add more file systems of adequate capacity.
Rerun the ldmconfig command.
Bug ID 6839787: Sometimes, a guest domain that runs at least the Solaris 10 10/08 OS does not make a proper Domain Services connection to a control domain that runs the Solaris 10 5/09 OS.
Domain Services connections enable features such as dynamic reconfiguration (DR), FMA, and power management (PM). Such a failure occurs when the guest domain is booted, so rebooting the domain usually clears the problem.
Workaround: Reboot the guest domain.
Bug ID 6815015: You can ignore these messages.
Bug ID 6840800: An otherwise usable corrupted or damaged autosave configuration cannot be downloaded.
Workaround: Use another, undamaged autosave configuration or SP configuration.
Bug ID 6839685: When you cancel a pending delayed reconfiguration operation to discard any changes that you made to a configuration, the changes are persisted in the current autosave configuration.
Workaround: Before starting a delayed reconfiguration operation on a configuration, save the existing autosave data for the current configuration, config-name:
# cd / # tar -cvf autosave.config-name.tar var/opt/SUNWldm/autosave-config-name |
After cancelling the delayed reconfiguration operation, restore the autosave data for the configuration:
# cd / # rm -rf var/opt/SUNWldm/autosave-config-name # tar -xvf autosave.config-name.tar |
Bug ID 6846468: Currently, the oldcfg autosave configuration is deleted, and newcfg is set to be the next poweron configuration. If oldcfg was marked as current or next poweron, subsequent configuration modifications will create or update the autosave configuration for oldcfg. The expected behavior is that the autosave configuration for oldcfg is left intact, and an autosave configuration for newcfg is created. If oldcfg is the current or next poweron configuration, it will remain so after using this command.
Bug ID 6841421: Under certain memory configurations, creating a guest domain might fail with this error message:
Unable to bind memory; limit of 31 segments reached |
Multiple memory segments are a normal occurrence that happens whenever there is a different amount of memory on the various CMP processors. However, the current versions of Logical Domains Manager can only support up to 31 memory segments for each guest domain.
Workaround: This situation might occur in the following situations:
Case 1 – The system firmware has determined that one or more FB-DIMMs have failed.
If the system firmware takes a FB-DIMM offline, you must replace it immediately.
Type the following command from the ALOM compatibility shell to list which FB-DIMM modules have been disabled:
sc> showcomponent |
The disabled FB-DIMMs are listed at the end of the output.
Case 2 – You have one or more CMP processors that have significantly more memory than the other processors.
Reallocate the FB-DIMMs across the CMP processors to keep the total number and types of FB-DIMMs as close as possible on each processor.
Case 3 – Your system is a T5440, and you have 3 CMP modules, or 2 modules in positions other than CMP0 and CMP1.
Consider upgrading to 4 CMP modules, or repositioning your 2 CMP modules so that they are in slots CMP0 and CMP1. After upgrading or repositioning the CMP modules, follow the instructions in “Node Reconfiguration” in Sun SPARC Enterprise T5440 Server Product Notes, which are in the Sun SPARC Enterprise T5440 Server Documentation collection on docs.sun.com.
Bug ID 6697096: Under certain circumstances, when a ldm rm-io operation is followed by multiple ldm set-vcpu operations, ldmd might abort and be restarted by SMF.
Workaround: After executing an ldm rm-io operation on a domain, take care when attempting an ldm set-vcpu operation. A single ldm set-vcpu operation will succeed, but a second ldm set-vcpu operation might cause the ldmd daemon to dump core under certain circumstances. Instead, reboot the domain before attempting the second set-vcpu operation.
Bug ID 6775847: For a period of time during a domain migration, a system can hang or end up with just one VCPU if another domain on the target system is rebooted.
ldm start and ldmm stop operations are prevented from running at this time. However, the issuing of a reboot or init command in the Solaris OS instance that runs on a guest domain cannot be prevented.
Workaround: Avoid rebooting domains on the target system while a migration is in progress.
Recovery: If the symptoms of this problem are detected, stop and restart the migrated domain on the target system.
Bug ID 6779482: If a migrating domain has a virtual network device with a MAC address that matches a MAC address on the target, the migration appropriately fails. However, the migration leaves a residual inactive domain of the same name and configuration on the target.
Workaround: On the target, use ldm destroy to manually remove the inactive domain. Then, fix the MAC address so that it is unique, and retry the migration.
Bug ID 6783450: The Domain Migration dry-run check (-n) does not ensure that the target system has enough free memory to bind the specified domain. If all other criteria are met, the command returns without an error. However, the command correctly returns an error when the migration is actually attempted.
Workaround: Run ldm list-devices mem on the target machine to verify that there is enough memory available for the domain to be migrated.
Bug ID 6836587: Sometimes ifconfig indicates that the device does not exist after you add a virtual network or virtual disk device to a domain. This situation might occur as the result of the /devices entry not being created.
Although this should not occur during normal operation, the error was seen when the instance number of a virtual network device did not match the instance number listed in /etc/path_to_inst file.
For example:
# ifconfig vnet0 plumb ifconfig: plumb: vnet0: no such interface |
The instance number of a virtual device is shown under the DEVICE column in the ldm list output:
# ldm list -o network primary NAME primary MAC 00:14:4f:86:6a:64 VSW NAME MAC NET-DEV DEVICE DEFAULT-VLAN-ID PVID VID MTU MODE primary-vsw0 00:14:4f:f9:86:f3 nxge0 switch@0 1 1 1500 NETWORK NAME SERVICE DEVICE MAC MODE PVID VID MTU vnet1 primary-vsw0@primary network@0 00:14:4f:f8:76:6d 1 1500 |
The instance number (0 for both the vnet and vsw shown previously) can be compared with the instance number in the path_to_inst file to ensure that they match.
# egrep '(vnet|vsw)' /etc/path_to_inst "/virtual-devices@100/channel-devices@200/virtual-network-switch@0" 0 "vsw" "/virtual-devices@100/channel-devices@200/network@0" 0 "vnet" |
Workaround: In the case of mismatching instance numbers, remove the virtual network or virtual switch device. Then, add them again by explicitly specifying the instance number required by setting the id property.
You can also manually edit the /etc/path_to_inst file. See the path_to_inst(4) man page.
Be aware of the warning contained in the man page that states “changes should not be made to /etc/path_to_inst without careful consideration.”
Bug ID 6845614: For most instances of a corrupted autosave configuration, the following misleading warning message is logged in the Logical Domains Manager log file:
warning: Autosave config 'config-name' missing HV MD |
The actual reason for this message could be when a guest domain or control domain has a corrupted MD, or has no valid MD.
Bug ID 6833994: This problem prevents the creation of more than 60 guest domains. This restriction is expected to be lifted with the release of the next Solaris 10 OS.
Bug ID 6757486: Occasionally, after a domain has been migrated, it is not possible to connect to the console for that domain.
Workaround: Restart the vntsd SMF service to enable connections to the console:
# svcadm restart vntsd |
This command will disconnect all active console connections.
Bug ID 6808832: You can configure a maximum of two domains with dedicated PCI-E root complexes on systems such as the Sun Fire T5240. These systems have two UltraSPARC T2+ CPUs and two I/O root complexes.
pci@500 and pci@400 are the two root complexes in the system. The primary domain will always contain at least one root complex. A second domain can be configured with an unassigned or unbound root complex.
The pci@400 fabric (or leaf) contains the onboard e1000g network card. The following circumstances could lead to a domain panic:
If the system is configured with a primary domain that contains pci@500 and a second domain that contains pci@400
For some blades, the primary domain (system disk) is on the pci@400 bus by default.
The e1000g device on the pci@400 fabric is used to boot the second domain
Avoid the following network devices if they are configured in a non-primary domain:
/pci@400/pci@0/pci@c/network@0,1 /pci@400/pci@0/pci@c/network@0 |
When these conditions are true, the domain will panic with a PCI-E Fatal error.
Avoid such a configuration, or if the configuration is used, do not boot from the listed devices.
Bug ID 6839284: For logical domains that have at least 120 Gbytes of memory, the ldom stop or ldom stop -f command might indicate that the operation timed out. Even though the stop operation has timed out, the process continues to shut down the logical domain in the background.
Workaround: You can ignore the timeout indications because the logical domain will continue with the shutdown process.
Bug ID 6852685: Starting with the Logical Domains 1.2 release, delayed reconfiguration operations are only supported on the control domain. However, the Logical Domains Manager does not properly enforce this restriction for the set-vdisk and set-vnet operations. If you issue either of these operations on a guest domain, that domain will enter delayed reconfiguration mode.
Workaround: If a guest domain enters delayed configuration mode as the result of a set-vdisk or set-vnet operation, do the following:
Use the ldm cancel-operation reconf command to cancel the pending delayed reconfiguration.
Stop the guest domain.
Re-issue the ldm set-vdisk or ldm set-vnet command.
Start the guest domain.
If the domain has already been stopped or was rebooted while in delayed reconfiguration mode, the pending configuration will be committed. For information about any issues or restrictions regarding the use of delayed reconfiguration operations, see the Logical Domains (LDoms) 1.1 Release Notes.
Bug ID 6853273: While a system is in power management elastic mode, rebooting a guest domain might produce the following warning messages and fail to boot successfully:
WARNING: /virtual-devices@100/channel-devices@200/disk@0: Sending packet to LDC, status: -1 WARNING: /virtual-devices@100/channel-devices@200/disk@0: Can't send vdisk read request! WARNING: /virtual-devices@100/channel-devices@200/disk@0: Timeout receiving packet from LDC ... retrying |
Workaround: If you see these warnings, perform one of the workarounds in the following order:
If the guest domain shows an ok> prompt and accepts input, type reset-all
From the control domain, issue an ldm stop domain-name command, then issue an ldm start domain-name command
Change the Power Management mode to performance mode, stop and start the affected guest domain, and then return to elastic mode
Bug ID 6852379: Under rare circumstances, when Power Management mode is set to elastic, a domain that is booting might panic very early in its boot sequence with messages similar to one of the following:
Boot device: /virtual-devices@100/channel-devices@200/disk@0 File and args: SunOS Release 5.10 Version Generic_139555-08 64-bit Copyright 1983-2009 Sun Microsystems, Inc. All rights reserved. Use is subject to license terms. panic[cpu0]/thread=180e000: cpu1 failed to start (2) |
Or:
Boot device: /virtual-devices@100/channel-devices@200/disk@0 File and args: SunOS Release 5.10 Version Generic_139555-08 64-bit Copyright 1983-2009 Sun Microsystems, Inc. All rights reserved. Use is subject to license terms. panic[cpu0]/thread=180e000: XC SPL ENTER already entered (0x0) |
Impact: Because the panic occurs very early in the boot sequence, it has no impact on applications or file systems because they have not yet been started. The domain should automatically reboot. Note that the domain might not reboot because of CR 6853590, see Reboot Stops at OpenBoot Prompt When Services Cannot Be Initialized.
Workaround: If the domain fails to boot, do one of the following:
Stop and start the impacted domain while the system is in elastic mode
Configure the Power Management mode from elastic to performance mode, boot the domain, and then return the system to elastic mode
Bug ID 6816969: A domain can sometimes be marked as being in transition mode even though it is booted. Transition mode is the mode in which a Solaris domain is booting or shutting down. CPU Power Management does not occur on a system when any domain is in transition mode. If a domain remains in transition mode, CPU Power Management will not occur when load is added to any domain.
Workaround: Switch from elastic mode to performance mode. If you want to return to elastic mode, reboot the domain that is stuck in transition mode.
Bug ID 6854189: If adding a virtual disk to a running guest domain fails, the guest domain might show messages like the following after the operation completes:
vdc: NOTICE: [5] Error initialising ports |
Workaround: When the guest domain is in this state, adding another virtual disk to the running guest domain might not be immediately visible to the system. In this case, run the devfsadm command to force the system to configure the available devices and make the newly added virtual disk visible.
Bug ID 6853590: Occasionally, a logical domain reboot operation stops at the OpenBoot prompt after one or more of the following messages are shown on the console:
NOTICE: Unable to complete Domain Service protocol version handshake WARNING: Unable to connect to Domain Service providers WARNING: Unable to get LDOM Variable Updates WARNING: Unable to update LDOM Variable |
Workaround: Boot the domain manually from the OpenBoot prompt.
Bug ID 6855079: An ldm command might be slow to respond when several domains are booting. If you issue an ldm command at this stage, the command might appear to hang. Note that the ldm command will return after performing the expected task. After the command returns, the system should respond normally to ldm commands.
Workaround: Avoid booting many domains simultaneously. However, if you must boot several domains at once, refrain from issuing further ldm commands until the system returns to normal. For instance, wait for about two minutes on Sun SPARC Enterprise T5140 and T5240 Servers and for about four minutes on the Sun SPARC Enterprise T5440 Server or Netra T5440 Server.
Bug ID 6846889: When rebooting the control domain or a guest domain, the following warning message might be logged on the control domain and on the guest domain that is rebooting:
WARNING: ds@0: ds_ldc_cb: LDC READ event while port not up |
Workaround: You can ignore this message.
Bug ID 6855534: When upgrading the control domain OS image from a previous release of Logical Domains, ensure that you preserve the constraints database file on the control domain. See Saving and Restoring the Logical Domains Constraints Database File in Logical Domains 1.2 Administration Guide.
If you were unable to preserve the constraints database, do not populate the control domain with a constraints database that does not match the running configuration. Such a mismatch could result in the Logical Domains Manager aborting when the ldm list -l command is issued, as follows:
primary# ldm list -l ldg0 Invalid response primary# |
Workaround: To recover, remove any existing constraints database files on the upgraded control domain. Then, use the svcadm restart ldmd command to restart the Logical Domains Manager and to resume normal operations.
Bug ID 6837313: Under rare circumstances on UltraSPARC T2 and UltraSPARC T2 Plus based systems, adding new CPUs to a domain might cause that domain to panic. This panic is more likely to occur when CPUs are added after PCI buses have been added or removed.
Impact: The domain panics with a stack trace that might contain references to the n2rng driver.
Workaround: This problem is triggered when the n2rng driver initializes structures for storing statistics. The problem can be prevented by disabling the generation of statistics for the n2rng driver, as follows:
Edit the /platform/sun4v/kernel/drv/n2rng.conf file to add the following line:
nostats=1; |
Update the n2rng driver by running the following command:
# update_drv n2rng |
This section contains documentation errors that have been found too late to resolve for the LDoms 1.2 release.
Bug ID 6843196: “Input/Output Bus Table (IOBusTable)” on page 31 of the Logical Domains (LDoms) MIB 1.0.1 Administration Guide shows incorrect parameter names.
IOBusDevName should be IOBusName, and IOBusDevPath should be IOBusPath.
This section lists bugs that have been fixed since the previous LDoms software release.
The following LDoms requests for enhancements (RFEs) and bugs were fixed for the Solaris 10 5/09 release.
You can track RFEs and bugs that are related to the Solaris, OpenSolaris, and LDoms releases at the OpenSolaris Bug Database.
vnet and vsw should support jumbo frames
vds should not serialize disk image IO
I/Os stuck in VDC/VDS layer causing hang
VIO drivers must use direct mapped shared memory d-ring descriptors for improved performance
File backed virtual I/O should be synchronous
LDoms Domain Services extensions for user program API
Dynamic virtual disk size management
uscsicmd on vdisk can overflow sense buffer
Race condition in ldc_mem_bind_handle when allocating map table
Panic in vgen_mdeg_cb if debugging enabled
Disk images on volumes should be exported using the ldi interface
eeprom command fails with concurrent ipmitool
vds accesses imported d-ring outside of LDC d-ring acquire/release calls
ldc_mem_unbind_handle passes invalid page size code to page_get_shift()
mdb -K panics when started on LDoms (sun4v) system
vsw memory leak in vsw_dispatch_ctrl_task
Bad EFI signature messages output during logical domain creation
vds can ACK a request twice
Primary domain panics reproducibly on a bad mutex
Cluster node hangs on panic fault injection when it is primary for resource group
vnet over hybridIO may not tag/untag packets when pvid is set
NIU HybridIO needs stats to reflect the assignment and traffic
Bakota DVD drive is not exported as a DVD
Netinstall fails with file-based backend
niumx does not re-distribute all registered interrupts
No ereports sent to root domains after rebooting control and panicking both root domains
libds corrupts memory in fmd etm module
vlds driver doesn't throttle misbehaving clients
libds entry missing from usr/src/Makefile.lint
Panic: Unrecoverable hardware error seen on guest w/2 hybrid vnets when service domain is stopped
Primary domain panic due to vsw_set_addrs set a mac address with a null pointer
Disks are not correctly exported with Hitachi HDLM multipathing
Possible deadlock between vnet and dds code
Domain Services loopback does not work
Domain Services client service registration request should not force reregister
Virtual Domain Services should implement FWARC/2008/563 and FWARC/2008/696
CPU Solaris stress test hangs with thread stuck in vdc_recv
NIU hio kstats setup can free the vres which is still on the vres_list of vnet
The following LDoms 1.2 RFEs and bugs were fixed for the LDoms 1.2 software release:
Add P2V tool and man page
Add option to stop guest domain when the control domain fails
PRI available outside the control domain
ldm: Must accept page retire and CPU offline requests from GM
Remove hardware topology for CPU->RNG associations in HV MD
LDoms Domain Services extensions for user program API
ldm: Add support for jumbo frames
Re-enable fma-io-domain-service
Add LDoms configuration auto-recovery
Add an LDoms discovery protocol
ldm: Enforce maximum sizes on domain, VIO, and variable names
Add support for exporting virtual disk server devices to multiple guests
ldmd on supernova should treat MAU commands correctly
Ethernet port name issue while configuring virtual switch in split PCI bus configuration
Guest domain fails virtual disk DR operations with LDC channel errors following a drd disable/enable
set-vdsdev does not work from default no options specified to any options like ro, excl, slice
ldm: Add CTF (Compact ANSI-C Type Format) support
ldmd core dumps on a delayed reconfiguration operation on a bound guest domain with vdpcc
ldm takes a fatal when starting on a cfg missing node0
Add XML action command to cancel a migration or reconfiguration operation
Virtual networks with duplicate MAC addresses are created on VIODR operations on low-mem guests
ldm still returns status 0 on a connection-refused failure
spconfig tagging implementation must incorporate del-reconf/reboot operations
Multidomain XML files still unusable for >10 domains: generates “Unexpected response message type”
HV MD version checking must include the major version number
HV MD version info incorrectly reported
Enable ldmd service by default
Remove changes to support configuration mode
Restrict delayed reconfig operations to the control domain
ldmd core dumps while performing set-vcc port-range under dly_reconf mode
Add LDoms XML Events schema properties
Removing a virtual disk can change remaining disk paths in guest domain
XML add and set virtual network broken for additional properties
No attempt should be made to DR in CPUs during start-domain in the PM Engine
XML must include virtual switch device information
XML device property needed for bound virtual network and virtual disk
XMPP server does not parse multiple LDM_interface from same document properly
Required resource properties must not be empty tags
Virtual switch VID gets truncated to 3 digits
Create server.{key | cert} in /var/opt/SUNWldm instead of /opt/SUNWldm/bin
Move ldm and ldm.1m to standard paths
Fix packaging so that package can be integrated in OpenSolaris dock
pri_svc segv fault while doing reboot.py test on snv_113
Same-path virtual disk server device error messages must be properly formatted
Cannot convert a READ-ONLY same-path virtual disk server device volume to READ-WRITE access
FMA error returned from PM during add-vcpu can lead to incomplete cleanup
ldmconfig: should allow zero LDoms be created - for users who just want to setup 'primary'
ldm rm-domain causes ldmd to dump core when run with libumem debugging enabled
Guest domain panics seen due to MD corruptions
ldm ls hung on multi-domain system (and single domain as well)
In elastic mode, rebooting guest domains hang due to a watchdog
OBP assumes a particular domain service port order in the MD
Port stuck high fix from LDoms 1.1 patch gate to LDoms 1.2 gate
ldm commands take a long time to report output after domains have started and are booting up