Logical Domains 1.2 Administration Guide

Chapter 1 Overview of the Logical Domains Software

This chapter provides an overview of the Logical Domains software.

The Sun Logical Domains software depends on particular Solaris OS versions, required software patches, and particular versions of system firmware. For more information, see Required and Recommended Solaris OS in Logical Domains 1.2 Release Notes.

This chapter covers the following topics:

Note –

The Logical Domains 1.2 software is supported on the OpenSolaris OS starting with the OpenSolaris 2009.06 release. The Logical Domains 1.2 documentation focuses on the usage of Logical Domains on the Solaris 10 OS. The same Logical Domains features are available for both the Solaris 10 OS and the OpenSolaris OS. However, you might encounter some slight differences when using Logical Domains with the OpenSolaris OS. For more information about the OpenSolaris OS, see the OpenSolaris Information Center.

Hypervisor and Logical Domains

This section provides an overview of the SPARC® hypervisor that supports Logical Domains.

The SPARC hypervisor is a small firmware layer that provides a stable virtualized machine architecture to which an operating system can be written. Sun servers that use the hypervisor provide hardware features to support the hypervisor's control over a logical operating system's activities.

A logical domain is a virtual machine comprised of a discrete logical grouping of resources. A logical domain has its own operating system and identity within a single computer system. Each logical domain can be created, destroyed, reconfigured, and rebooted independently, without requiring you to powercycle the server. You can run a variety of applications software in different logical domains and keep them independent for performance and security purposes.

Each logical domain is only permitted to observe and interact with those server resources that are made available to it by the hypervisor. The Logical Domains Manager enables you to specify what the hypervisor should do through the control domain. Thus, the hypervisor enforces the partitioning of the server's resources and provides limited subsets to multiple operating system environments. This partitioning and provisioning is the fundamental mechanism for creating logical domains. The following diagram shows the hypervisor supporting two logical domains. It also shows the following layers that make up the Logical Domains functionality:

Applications, or user/services
Kernel, or operating systems
Firmware, or hypervisor
Hardware, including CPU, memory, and I/O

Figure 1–1 Hypervisor Supporting Two Logical Domains

Graphic shows the layers that make up the Logical Domains functionality.

The number and capabilities of each logical domain that a specific SPARC hypervisor supports are server-dependent features. The hypervisor can allocate subsets of the overall CPU, memory, and I/O resources of a server to a given logical domain. This enables support of multiple operating systems simultaneously, each within its own logical domain. Resources can be rearranged between separate logical domains with an arbitrary granularity. For example, memory is assignable to a logical domain with an 8-Kbyte granularity.

Each logical domain can be managed as an entirely independent machine with its own resources, such as:

Kernel, patches, and tuning parameters
User accounts and administrators
Disks
Network interfaces, MAC addresses, and IP addresses

Each logical domain can be stopped, started, and rebooted independently of each other without requiring a powercycle of the server.

The hypervisor software is responsible for maintaining the separation between logical domains. The hypervisor software also provides logical domain channels (LDCs) that enable logical domains to communicate with each other. LDCs enable domains to provide services to each other, such as networking or disk services.

The service processor (SP), also known as the system controller (SC), monitors and runs the physical machine, but it does not manage the logical domains. The Logical Domains Manager manages the logical domains.

Logical Domains Manager

The Logical Domains Manager is used to create and manage logical domains, as well as map logical domains to physical resources. Only one Logical Domains Manager can run on a server.

Roles for Logical Domains

All logical domains are the same and can be distinguished from one another based on the roles that you specify for them. There following are the roles that logical domains can perform:

Control domain. The Logical Domains Manager runs in this domain, which enables you to create and manage other logical domains, and to allocate virtual resources to other domains. You can have only one control domain per server. The control domain is the first domain created when you install the Logical Domains software. The control domain is named primary.
Service domain. A service domain provides virtual device services to other domains, such as a virtual switch, a virtual console concentrator, and a virtual disk server. Any domain can be configured as a service domain.
I/O domain. An I/O domain has direct ownership of and direct access to physical I/O devices, such as a network card in a PCI EXPRESS® controller. An I/O domain is often used as a service domain to share physical devices with other domains in the form of virtual devices. The control domain is an I/O domain, and it can also be used as a service domain. The number of I/O domains that you can have depends on your platform. For example, if you are using a Sun SPARC Enterprise® Server T5440, you can have up to four I/O domains.
Guest domain. A guest domain is a non-I/O domain that consumes virtual device services that are provided by one or more service domains. A guest domain does not have any physical I/O devices, but only has virtual I/O devices, such as virtual disks and virtual network interfaces.

You can install the Logical Domains Manager on an existing system that is not already configured with Logical Domains. In this case, the current instance of the OS becomes the control domain. Also, the system is configured as a Logical Domains system that has only one domain, the control domain. After configuring the control domain, you can balance the load of applications across other domains to make the most efficient use of the entire system. You do this by adding domains and moving those applications from the control domain to the new domains.

Command-Line Interface

The Logical Domains Manager uses a command-line interface (CLI) to create and configure logical domains. The CLI is a single command, ldm, that has multiple subcommands. See the ldm(1M) man page.

The Logical Domains Manager daemon, ldmd, must be running to use the Logical Domains Manager CLI .

Virtual Input/Output

In a Logical Domains environment, you can provision up to 128 domains on an UltraSPARC® T2 Plus processor system. These systems have a limited number of I/O buses and physical I/O slots. As a result, you cannot provide exclusive access to a physical disk and network devices to all domains on these systems. You can assign a PCI bus to a domain to provide it with access to a physical device. Note that this solution is insufficient to provide all domains with exclusive device access. See I/O Domains and PCI EXPRESS Buses. This lack of direct physical I/O device access is addressed by implementing a virtualized I/O model.

Any logical domains that have no physical I/O access are configured with virtual I/O devices that communicate with a service domain . The service domain runs a virtual device service to provide access to a physical device or to its functions. In this client-server model, virtual I/O devices either communicate with each other or with a service counterpart through interdomain communication channels called logical domain channels (LDCs). The virtualized I/O functionality includes support for virtual networking, storage, and consoles.

Virtual Network

Logical Domains uses the virtual network device and virtual network switch device to implement virtual networking. The virtual network (vnet) device emulates an Ethernet device and communicates with other vnet devices in the system by using a point-to-point channel. The virtual switch (vsw) device primarily functions as a multiplexor of all the virtual network's incoming and outgoing packets. The vsw device interfaces directly with a physical network adapter on a service domain, and sends and receives packets on behalf of a virtual network. The vsw device also functions as a simple layer-2 switch and switches packets between the vnet devices connected to it within the system.

Virtual Storage

The virtual storage infrastructure uses a client-server model to enable logical domains to access block-level storage that is not directly assigned to them. The model uses the following components:

Virtual disk client (vdc) that exports a block device interface
Virtual disk service (vds) that processes disk requests on behalf of the virtual disk client and submits them to the backend storage that resides on the service domain

Although the virtual disks appear as regular disks on the client domain, most disk operations are forwarded to the virtual disk service and processed on the service domain.

Virtual Console

In a Logical Domains environment, console I/O from the primary domain is directed to the service processor. The console I/O from all other domains is redirected to the service domain that is running the virtual control concentrator (vcc). The domain that runs the vcc is typically the primary domain. The virtual console concentrator service functions as a concentrator for all domains' console traffic, and interfaces with the virtual network terminal server daemon (vntsd) to provide access to each console through a UNIX socket.

Dynamic Reconfiguration

Dynamic reconfiguration (DR) is the ability to add or remove resources while the operating system is running. The ability to perform dynamic reconfiguration of a particular resource type is dependent on having support in the OS running in the logical domain.

Dynamic reconfiguration is supported for the following resources:

Virtual CPUs – Supported in all versions of the Solaris 10 OS
Virtual I/O devices – Supported in at least the Solaris 10 10/08 OS
Memory – Not supported
Physical I/O devices – Not supported

To use the dynamic reconfiguration capability, the Logical Domains dynamic reconfiguration daemon, drd, must be running in the domain that you want to change. See the drd(1M) man page.

Delayed Reconfiguration

In contrast to dynamic reconfiguration operations that take place immediately, delayed reconfiguration operations take effect in the following circumstances:

After the next reboot of the OS
After a stop and start of the logical domain

The Logical Domains Manager 1.2 software restricts delayed reconfiguration operations to the control domain. For all other domains, you must stop the domain to modify the configuration unless the resource can be dynamically reconfigured.

If you are using a Sun UltraSPARC T1 processor , when the Logical Domains Manager is first installed and enabled (or when the configuration is restored to factory-default), ldmd runs in configuration mode. In this mode, reconfiguration requests are accepted and queued up, but are not acted upon. This allows a new configuration to be generated and stored on the SP without affecting the state of the running machine. Thus, configuration mode is not encumbered by any restrictions such as delayed reconfiguration and reboot of I/O domains.

When a delayed reconfiguration is in progress on the control domain, other reconfiguration requests for the control domain are deferred until it is rebooted, or stopped and started. Also, when a delayed reconfiguration is outstanding for the control domain, reconfiguration requests for other logical domains are severely restricted and will fail with an appropriate error message.

The Logical Domains Manager ldm cancel-operation reconf command cancels delayed reconfiguration operations on the control domain. You can list delayed reconfiguration operations by using the ldm list-domain command. For more information about how to use the delayed reconfiguration feature, see the ldm(1M) man page.

Note –

You cannot use the ldm cancel-operation reconf command if any other ldm remove-* commands have already performed a delayed reconfiguration operation on virtual I/O devices. The ldm cancel-operation reconf command fails in these circumstances.

Persistent Configurations

You can use the ldm command to store the current configuration of a logical domain on the service processor. You can add a configuration, specify a configuration to be used, remove a configuration, and list the configurations. See the ldm(1M) man page.

You can also use an ALOM CMT Version 1.3 command to select a configuration to boot, see Using LDoms With ALOM CMT.

For information about managing configurations, see Managing Logical Domains Configurations.

Logical Domains Physical-to-Virtual Migration Tool

The Logical Domains Physical-to-Virtual (P2V) Migration Tool automatically converts an existing physical system to a virtual system that runs in a logical domain on a chip multithreading (CMT) system. The source system can be any of the following:

Any sun4u SPARC system that runs at least the Solaris 8 Operating System
Any sun4v system that runs the Solaris 10 OS, but does not run the Logical Domains software

For information about the tool and about installing it, see Appendix C, Logical Domains Physical-to-Virtual Migration Tool. For information about the ldmp2v command, see the ldmp2v(1M) man page.

Logical Domains Configuration Assistant

The Logical Domains Configuration Assistant leads you through the configuration of a logical domain by setting basic properties. It runs on CMT-based systems that are known as Sun Coolthreads Servers. It can be used to configure any system where the Logical Domains software is installed but not already configured.

After gathering the configuration data, the Configuration Assistant creates a configuration that is suitable for booting as a logical domain. You can also use the default values selected by the Configuration Assistant to create a usable system configuration.

The Configuration Assistant is available as both a graphical user interface (GUI) and terminal-based tool.

For more information, see Appendix D, Logical Domains Configuration Assistant and the ldmconfig(1M) man page.