System Administration Guide: Virtualization Using the Solaris Operating System

Virtual Devices

Networks

The physical network consists of both an external physical LAN and the extension of the LAN within the host to a guest's network. Paravirtualized domains use the xnb backend driver in dom0 to communicate with a physical network interface.

The virtual network operates through the underlying physical network infrastructure.

You can create one physical network, also referred to as a switch, for each NIC on the system. If you have more than one physical NIC configured, you might want to configure the default-nic property of the xvm/xend SMF service, as described in the xend(1M).

To view the IP address assigned through DHCP, use the ifconfig command.

See the document New Network Options, Including Limiting Bandwidth and Setting a VLAN ID, for Virtual Network Interfaces Attached to a Guest to learn about conventions for network options used in the virt-install utility.

Virtual NICs

A single physical NIC can be carved into multiple VNICs, which can be assigned to different Solaris xVM instances running on the same system. VNICs are managed using the dladm command line utility described in the dladm(1M) man page. You can use virsh dumpxml output to correlate the domain's network interface with the assigned VNIC.

The dladm show-vnic command can be used to display VNIC information. In the following output, 1 is the domain ID, and 0 is the NIC number for that guest.

# dladm show-vnic
LINK         OVER         SPEED  MACADDRESS           MACADDRTYPE VID
xvm1_0       e1000g0      1000   0:16:3e:64:99:4d     fixed 0

For more information, see OpenSolaris Project: Crossbow: Network Virtualization and Resource Control. Crossbow provides network virtualization and resource control by virtualizing the stack and NIC around any service, protocol, or virtual machine.

Virtual FibreChannel HBAs

Certain FibreChannel HBAs that support N Port ID virtualization (NPIV) can be carved into multiple virtual HBAs, in a manner similar to VNICs. Each virtual HBA has a unique identity, or WWN, on the SAN, which can be used for LUN masking or zoning. Block devices present on these virtual HBAs can be automatically mounted to specific OpenSolaris xVM guest operating systems. When these guest operating systems migrate, the HBA's identity also migrates. NPIV is administered using the fcinfo(1M) and xm(1M) commands.

Virtual FibreChannel HBAs

Virtual block devices can be stored on FibreChannel disk subject to the following limitations:

Back end block device names (for example, c0t0d0s2) must be the same for migration to succeed. If device names are different, migration will fail.
For FibreChannel devices, it is likely that the cX number will be different if scsi_vhci, described in scsi_vhci(7D), is not used. If scsi_vhci is used, device names are likely to be identical. If scsi_vhci is enabled and has never been disabled, the controller should be at cX on x86/x64 machines.

NPIV and FibreChannel

NPIV is fully supported for xVM. The xm command can associate all block devices from one NPIV port to a guest domain. NPIV identity, specifically the port WWN and node WWN, will migrate between devices on the SAN. NPIV allows zoning and LUN masking to be used with Xen. Zoning and LUN masking are useful tools for access control on a SAN. Soft zoning rather than hard zoning (grouping by switch port) should be used on the switch. Soft zoning groups devices by the HBA's Port WWN. If there is more than one physical SAN, or if the system uses hard zoning, the administrator must ensure that the physical HBA is connected to the correct SAN. Switch administrative tools can be used for this purpose.

How to Configure NPIV for Hypervisor Use

This procedure uses the fcinfo command described in fcinfo(1M).

Identify the FibreChannel HBAs that will be used. If migration will be supported, the HBAs must be identified. The fcinfo command can be used to list the Port WWN.
Create NPIV ports in the Dom0 control domain.
View devices by usingfcinfo in dom0. Verify that they are visible in the respective guest domains.

Using `vdiskadm` to Create Virtual Disks

You can use the virt-install command to create disks.

The vdiskadm command described in the vdiskadm(1M) man page creates and manages virtual disks. vdiskadm is implemented as a set of subcommands, some of which have their own options and operands. All operations on the vdisk need to be performed using vdiskadm.

The types of virtual disks are:

-t vmdk:fixed
-t vmdk:sparse
-t raw:fixed
-t vdi:fixed
-t vdi:sparse
-t vhd:fixed
-t vhd:sparse

vmdk is the native VMware format, vdi is the native Sun VirtualBox format, vhd is the native Hyper-V format, and raw describes a file that looks like a raw disk. A raw disk is always in fixed format so that option can be explicitly set or implicitly understood. If the type is not specified, the default value is vmdk. If the option is not specified, the default value is fixed for type raw and sparse for types vmdk and vdi.

Create a new virtual disk of the specified size and at the location specified by vdname. If vdname includes a path to the virtual disk, the directories from that path will be created during creation of the virtual disk. -t type[:opt],[opt]] specifies the type of virtual disk to be created. Type as one line.

# vdiskadm create -s size [-t type[:opt],[opt]] [-c comment] vdname

You can import a disk image from a block device or file to a vdisk, convert it to a different type of vdisk, and export from a vdisk to a block device or file. This includes the full vhd support (sparse and fixed) and the ability to import a vmdk 1.1 optimized stream file. An optimized stream file is read-only and must be imported to another type (vmdk:sparse by default) in order to be used as a vdisk.

Examples

Creating a Default `vmdk:sparse` File

A directory of the vdisk name is created and populated with two files. vdisk.vmdk is the file with the disk data. vdisk.xml is the file containing information about the disk, such as creation time, type:option of disk, and snapshot information. Note that vdisk.vmdk has a suffix of the vdisk type.

# vdiskadm create -s 8g root_disk
# ls -l root_disk
total 82
-rw-------   1 root     root     1114112 May  8 16:15 vdisk.vmdk
-rw-r--r--   1 root     root         584 May  8 16:15 vdisk.xml

Creating a `vdisk` File of Type `vhd`

The suffix specified is now vhd. Since the option isn't specified with a type, the option has the default of sparse. Note that the disk file, vdisk_vhd, isn't fully populated to 8G.

# vdiskadm create -s 8g -t vhd root_disk_vhd
# ls -l root_disk_vhd
total 44
-rw-------   1 root     root       21504 May  8 16:15 vdisk.vhd
-rw-r--r--   1 root     root         590 May  8 16:15 vdisk.xml

Creating a `vmdk:fixed` File

Creating a vmdk type vdisk with option fixed takes a minute or more to create since it is creating and initializing 8G of data. The creation time is dependent upon the size of the vdisk.

# vdiskadm create -s 8g -t vmdk:fixed root_disk_fix
# ls -l root_disk_fix
total 16785428
-rw-------   1 root     root     8589934592 May  8 16:18 vdisk-flat.vmdk
-rw-------   1 root     root         638 May  8 16:18 vdisk.vmdk
-rw-r--r--   1 root     root         593 May  8 16:18 vdisk.xml

The contents of the xml file for root_disk_fix are:

# cat root_disk_fix/vdisk.xml
<?xml version="1.0"?>
<!DOCTYPE vdisk PUBLIC "-//Sun Microsystems Inc//DTD xVM Management All//EN" "file:///usr/share/lib/xml/dtd/vdisk.dtd">
<vdisk readonly="false" removable="false" cdrom="false" creation-time-epoch="1241821124" vtype="vmdk" sparse="false" rwcnt="0" rocnt="0">
  <name>root_disk_fix</name>
  <version>1.0</version>
  <parent>none</parent>
  <diskprop>
    <filename>root_disk_fix</filename>
    <vdfile>vdisk.vmdk</vdfile>
    <owner>root</owner>
    <max-size>8589934592</max-size>
    <sectors>16777216</sectors>
    <description>none</description>
  </diskprop>
</vdisk>

This same information can be retrieved from the vdiskadm command by using the subcommand prop-get:

# vdiskadm prop-get -p all root_disk_fix
readonly: false
removable: false
cdrom: false
creation-time-epoch: 1241821124
vtype: vmdk
sparse: false
rwcnt: 0
rocnt: 0
name: root_disk_fix
version: 1.0
parent: none
filename: root_disk_fix
vdfile: vdisk.vmdk
owner: root
max-size: 8589934592
sectors: 16777216
description: none
effective-size: 8589935230
creation-time: Fri May  8 16:18:44 2009
modification-time: Fri May  8 16:18:44 2009
modification-time-epoch: 1241821124

The modification times and effective size are all derived “on the fly,” and are not stored in the xml file. The creation and modification times are shown both in epoch format and in human readable format, for use by both software applications (such as Sun Ops Center) and system administrators.

The rwcnt and rocnt fields shown in the xml file are the reader/writer locks on the vdisk. There can be only one writer at a time, but multiple readers can be using the vdisk. These fields are used to set or reset the reader/writer lock associated with the virtual disk. These fields should not be set or reset by hand; they can only be modified by using vdiskadm ref-inc [-r] vdame or vdiskadm ref-dec vdname. These fields are used by blktap for shared or exclusive use of the virtual disk.

Snapshots

A snapshot is a read-only copy of a virtual disk. Snapshots can be created quickly and initially consume little space. As data within the active virtual disk changes, the snapshot consumes more data than would otherwise be shared with the active virtual disk.

vdisk supports snapshots in a manner similar to ZFS, except that the vdisk cannot be in use during a snapshot. The user can take a snapshot of the vdisk and later rollback to that snapshot, if needed. The user can also take a snapshot and then clone that snapshot into another vdisk.

To see the images are associated with a vdisk, type:

# vdiskadm list vhd_sp
vhd_sp

Take a snapshot of the virtual disk immediately after installing it:

# vdiskadm snapshot /export/home/vdisks/vhd_sp@install

List all images associated with the virtual disk:

# vdiskadm list /export/home/vdisks/vhd_sp
vhd_sp@install
vhd_sp

The original file, vdisk.vhd, has been moved to vdisk@install.vhd. A new file that contains the differences has been created. It is named vdisk.vhd.

# ls -l vhd_sp
total 2717732
-rw-------   1 root     root       17408 May 11 16:41 vdisk.vhd
-rw-r--r--   1 xvm      root         717 May 11 16:41 vdisk.xml
-rw-------   1 root     root     1390768640 May 11 16:41 vdisk@install.vhd

The vdisk.xml file shows the added snapshot element. When additional snapshots are created, new snapshot elements will be added to the xml description. The snapshot order in the list (and shown with vdiskadm list) shows the order in which the snapshots are loaded.

# cat vhd_sp/vdisk.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE vdisk PUBLIC "-//Sun Microsystems Inc//DTD xVM Management All//EN" "file:///usr/share/lib/xml/dtd/vdisk.dtd">
<vdisk readonly="false" removable="false" cdrom="false" creation-time-epoch="1241643718" vtype="vhd" sparse="true" rwcnt="0" rocnt="0">
  <name>vhd_sp</name>
  <version>1.0</version>
  <parent>none</parent>
  <diskprop>
    <filename>vhd_sp</filename>
    <vdfile>vdisk.vhd</vdfile>
    <owner>xvm</owner>
    <max-size>6442450944</max-size>
    <sectors>12582912</sectors>
    <description>none</description>
  </diskprop>
  <snapshot creation-time-epoch="1242081709">
    <name>@install</name>
    <vdfile>vdisk@install.vhd</vdfile>
  </snapshot>
</vdisk>

Now, take another snapshot after a bfu and list the contents:

# vdiskadm snapshot /export/home/vdisks/vhd_sp@bfu

# vdiskadm list /export/home/vdisks/vhd_sp
vhd_sp@install
vhd_sp@bfu
vhd_sp

To roll back the disk to a point right after the install:

# vdiskadm rollback -r /export/home/vdisks/vhd_sp@install
# vdiskadm list /export/home/vdisks/vhd_sp
vhd_sp@install
vhd_sp

The rollback operation removes vdisk.vhd and any intervening snapshot images after vdisk@install.vhd, and creates a new differences file named vdisk.vhd.

# ls -l vhd_sp
total 2717732
-rw-------   1 root     root       17408 May 11 16:47 vdisk.vhd
-rw-r--r--   1 xvm      root         717 May 11 16:47 vdisk.xml
-rw-------   1 root     root     1390768640 May 11 16:47 vdisk@install.vhd

Clones

A clone is a writable copy of a virtual disk. The default type of clone is a merged (that is, coalesced) copy of the original virtual disk. An example of a merged clone occurs when a virtual disk is comprised of several snapshots; a subsequent clone operation results in a new virtual disk containing no snapshots. A clone will be of the same type as the original virtual disk (that is, vmdk:fixed). When a merged clone is created there is no linkage back to the original virtual disk or to any of its snapshots. This lack of linkage allows the merged clone to be moved to another physical machine.

Create a clone of the specified snapshot or virtual disk. The clone is created with the type and option and the size of the virtual disk being cloned. If clone_vdname includes a path, the subdirectories from that path will be created during creation of the cloned virtual disk. By default, a merged clone image is created:

# vdiskadm clone [-c comment] vdname|snapshot clone_vdname

Virtual Devices

Networks

Virtual NICs

Virtual FibreChannel HBAs

Virtual FibreChannel HBAs

NPIV and FibreChannel

How to Configure NPIV for Hypervisor Use

Using vdiskadm to Create Virtual Disks

Examples

Creating a Default vmdk:sparse File

Creating a vdisk File of Type vhd

Creating a vmdk:fixed File

Snapshots

Clones

Using `vdiskadm` to Create Virtual Disks

Creating a Default `vmdk:sparse` File

Creating a `vdisk` File of Type `vhd`

Creating a `vmdk:fixed` File