Chapter 9 Introducing Network Virtualization and Resource Control (Overview)

This chapter explains the basic concepts involved in network virtualization and resource control. The following topics are covered:

• Network virtualization

• Types of virtual networks

• Virtual machines and zones

• Resource control, including flow management

• Enhanced network observability

These features help you to manage flow control, improve system performance, and configure the network utilization needed to achieve OS virtualization, utility computing, and server consolidation.

For specific tasks, refer to the following chapters:

• Chapter 11, Configuring Virtual Networks (Tasks)

• Chapter 13, Configuring Resource Management on an Interface

• Chapter 12, Administering Virtual Networks and Resource Controls (Tasks)

Network Virtualization and Virtual Networks

Network virtualization is the process of combining hardware network resources and software network resources into a single administrative unit. The goal of network virtualization is to provide systems and users with efficient, controlled, and secure sharing of the networking resources.

The end product of network virtualization is the virtual network. Virtual networks are classified into two broad types, external and internal. External virtual networks consist of several local networks that are administered by software as a single entity. The building blocks of classic external virtual networks are switch hardware and VLAN software technology. Examples of external virtual networks include large corporate networks and data centers.

An internal virtual network consists of one system using virtual machines or zones that are configured over at least one pseudo-network interface. These containers can communicate with each other as though on the same local network, providing a virtual network on a single host. The building blocks of the virtual network are virtual network interface cards or virtual NICs (VNICs) and virtual switches. Solaris network virtualization provides the internal virtual network solution.

You can combine networking resources to configure both internal and external virtual networks. For example, you can configure individual systems with internal virtual networks onto LANs that are part of a large, external virtual network. The network configurations that are described in this part include examples of combined internal and external virtual networks.

Types of Containers for Network Virtualization on the Solaris OS

You can use several different types of virtual containers in a Solaris OS-based virtual network. These containers include machines and zones. A virtual machine is a container with its own kernel and IP protocol stack. A zone is a container that provides an isolated environment for running applications.

Sun xVM Virtual Machines

Sun^TM xVM is virtual machine technology that enables you to create multiple instances of an operating system on the interfaces of a single x86–based system. The Sun xVM hypervisor controls the allocation and operation of the domains. For more information on xVM, refer to Introduction to the Sun xVM Hypervisor. xVM is based on the Open Source XEN hypervisor, which is described on the xen.org website.

Non-Global Zones and Exclusive IP Zones

Though not true virtual machines, zones are light weight application environments that share a host's kernel and IP stack. You can configure exclusive IP instances for a non-global zone, which provides that zone with its own, exclusive TCP/IP protocol stack. Both standard non-global zones and exclusive IP zones can be configured on a Solaris-based virtual network. For basic information about zones, refer to Chapter 16, Introduction to Solaris Zones, in System Administration Guide: Virtualization Using the Solaris Operating System.

LDOMs Virtual Machines

The Libvert for LDOMs (Logical Domains) software provides a hypervisor and set of commands that enable you to set up and administer logical domains on a Solaris OS-based virtual network. Each logical domain can run an instance of an operating system to enable multiple operating systems on the same computer. For information on LDOMs, refer to the Logical Domains (LDoms) 1.0.1 Administration Guide.

Parts of the Internal Virtual Network

An internal virtual network built on the Solaris OS contains the following parts:

• At least one network interface card, or NIC.

• A virtual NIC, or VNIC, which is configured on top of the network interface

• A virtual switch, which is configured at the same time as the first VNIC on the interface.

• A container, such as a zone or virtual machine , which is configured on top of the VNIC.

The next figure shows these parts and how they fit together on a single system.

Figure 9–1 VNIC Configuration for a Single Interface

The figure shows a single system with one NIC. The NIC is configured with three VNICs. Each VNIC supports a single zone. Therefore, Zone 1, Zone 2, and Zone 3 are configured over VNIC 1, VNIC 2, and VNIC 3, respectfully. The three VNICs are virtually connected to one virtual switch. This switch provides the connection between the VNICs and the physical NIC upon which the VNICs are built. The physical interface provides the system with its external network connection.

Alternatively, you can create a virtual network based on the etherstub. Etherstubs are purely software and do not require a network interface as the basis for the virtual network.

A VNIC is a virtual network device with the same data-link interface as a physical interface. You configure VNICs on top of a physical interface. For the current list of physical interfaces that support VNICs, refer to the Network Virtualization and Resource Control FAQ. You can configure up to 900 VNICs on a single physical interface. When VNICs are configured, they behave like physical NICs. In addition, the system's resources treat VNICs as if they were physical NICs.

Each VNIC is implicitly connected to a virtual switch that corresponds to the physical interface. The virtual switch provides the same connectivity between VNICs on a virtual network that switch hardware provides for the systems connected to a switch's ports.

In accordance with Ethernet design, if a switch port receives an outgoing packet from the host connected to that port, that packet cannot go to a destination on the same port. This design is a drawback for systems that are configured with zones or virtual machines. Without network virtualization, outgoing packets from a virtual machine or a zone with an exclusive stack cannot be passed to another virtual machine or zone on the same system. The outgoing packets go through a switch port out onto the external network. The incoming packets cannot reach their destination zone or virtual machine because the packets cannot return through the same port as they were sent. Therefore, when virtual machines and zones on the same system need to communicate, a data path between the containers must open on the local machine. Virtual switches provide these containers with the method to pass packets.

How Data Travels Through a Virtual Network

Figure 9–1 illustrates a simple VNIC configuration for a virtual network on a single system.

When the virtual network is configured, a zone sends traffic to an external host in the same fashion as a system without a virtual network. Traffic flows from the zone, through the VNIC to the virtual switch, and then to the physical interface, which sends the data out onto the network.

But what happens if one zone on a virtual network wants to send packets to another zone on the virtual network, given the previously mentioned Ethernet restrictions? As shown in Figure 9–1, suppose Zone 1 needs to send traffic to Zone 3? In this case packets pass from Zone 1 through its dedicated VNIC 1. The traffic then flows through the virtual switch to VNIC 3. VNIC 3 then passes the traffic to Zone 3. The traffic never leaves the system, and therefore never violates the Ethernet restrictions.

Who Should Implement Virtual Networks?

If you need to consolidate resources on Sun servers, consider implementing VNICs and virtual networks. Consolidators at ISPs, telecommunications companies, and large financial institutions can use the following network virtualization features to improve the performance of their servers and networks.

• NIC hardware, including the powerful new interfaces that support hardware rings

• Multiple MAC addresses for the VNICs

• The large amount of bandwidth provided by newer interfaces

You can replace many systems with a single system that implements running multiple zones or virtual machines, without significantly losing separation, security, and flexibility.

What Is Resource Control?

Resource control is the process of allocating a system's resources in a controlled fashion. The Solaris OS resource control features enable bandwidth to be shared among the VNICs on a system's virtual network. You can also use resource control features to allocate and manage bandwidth on a physical interface without VNICs and virtual machines. This section introduces the major features of resource control and briefly explains how these features work.

How Bandwidth Management and Flow Control Works

Searchnetworking.com defines bandwidth as “the amount of data that can be carried from one point to another in a given time period (usually a second).” Bandwidth management enables you to assign a portion of the available bandwidth of a physical NIC to a consumer, such as an application or customer. You can control bandwidth on a per- application, per-port, per-protocol, and per-address basis. Bandwidth management assures efficient use of the large amount of bandwidth available from the new GLDv3 network interfaces.

Resource control features enable you implement a series of controls on an interface's available bandwidth. For example, you can set a guarantee of an interface's bandwidth to a particular consumer. That guarantee is the minimum amount of assured bandwidth allocated to the application or enterprise The allocated portion of bandwidth is known as a share. By setting up guarantees, you can allocate enough bandwidth for applications that cannot function properly without a certain amount of bandwidth. For example, streaming media and Voice over IP consume a great deal of bandwidth. You can use the resource control features to guarantee that these two applications have enough bandwidth to successfully run.

You can also set a limit on the share. The limit is the maximum allocation of bandwidth the share can consume. Using limits, you can contain non-critical services from taking away bandwidth from critical services.

Finally, you can prioritize among the various shares allotted to consumers. You can give highest priority to critical traffic, such as heartbeat packets for a cluster, and lower priority for less critical applications.

For example, application service providers (ASPs) can offer customers fee-based levels of service that are based on the bandwidth share that the customer purchases. As part of the service level agreement (SLA), each share is then guaranteed an amount of bandwidth, to not exceed the purchased limit. (For more information on service level agreements, see Implementing Service-Level Agreements in System Administration Guide: IP Services. Priority controls might be based on different tiers of the SLA, or different prices paid by the SLA customer.

Bandwidth usage is controlled through management of flows. A flow is a stream of packets that all have certain characteristics, such as the port number or destination address. These flows are managed by transport, service, or virtual machine, including zones. Flows cannot exceed the amount of bandwidth that is guaranteed to the application or to the customer's purchased share.

When a VNIC or flow is assigned a guarantee, the VNIC is assured its designated bandwidth even if other flows or VNICs also use the interface. However, assigned guarantees are workable only if they do not exceed the maximum bandwidth of the physical interface.

Allocating Resource Control and Bandwidth Management on a Network

The following figure shows a corporate network topology that uses resource control to manage various applications.

Network With Resource Controls in Place

Figure 9–2 Network With Resource Controls in Place

This figure shows a typical network topology that uses resource controls to improve network efficiency and performance. The network does not implement VNICs and containers, such as exclusive zones and virtual machines. However, VNICs and containers could be used on this network for consolidation and other purposes.

The network is divided into four tiers:

• Tier 0 is the demilitarized zone (DMZ). This is a small local network that controls access to and from the outside world. Resource control is not used on the systems of the DMZ.

• Tier 1 is the web tier and includes two systems. The first system is a proxy server that does filtering. This server has two interfaces, bge0 and bge1. The bge0 link connects the proxy server to the DMZ on Tier 0. The bge0 link also connects the proxy server to the second system, the web server. The http and https services share the bandwidth of the web server with other standard applications. Due to the size and critical nature of web servers, shares of http and https require guarantees and prioritization.

• Tier 2 is the applications tier and also includes two systems. The second interface of the proxy server, bge1, provides the connection between the web tier and the applications tier. Through a switch, an applications server connects to bge1 on the proxy server. The applications server requires resource control to manage the shares of bandwidth given to the various applications that are run. Critical applications that need a lot of bandwidth must be given higher guarantees and priorities than smaller, or less critical applications.

• Tier 3 is the database tier. The two systems on this tier connect through a switch to the proxy server's bge1 interface. The first system, a database server, needs to issue guarantees and to prioritize the various processes involved in database lookups. The second system is a backup server for the network. This system must consume a great deal of bandwidth during backups. However, backup activities are typically carried out overnight. Using resource controls, you can control when the backup processes have the highest bandwidth guarantees and highest priorities.

Who Should Implement Resource Control Features

Any system administrator who wants to improve a system's efficiency and performance should consider implementing the resource control features. Consolidators can delegate bandwidth shares in combination with VNICs to help balance the load of large servers. Server administrators can use share allocation features to implement SLA's, such as those offered by ASPs. Traditional system administrators can use the bandwidth management features to isolate and prioritize certain applications. Finally, share allocation makes it easy for you to observe bandwidth usage by individual consumers.

Observability Features for Network Virtualization and Resource Control

Network virtualization and resource control includes observability features to help you view resource usage before setting up controls such as VNICs and flows. In tandem with Solaris extended accounting, the resource control observability features allow you to accumulate systems statistics into logs. The observability features of network virtualization and resource control include:

• Ability to monitor a running system.

• Ability to log and report statistics.

• Extended accounting features to log historical data

The new flowadm command and extensions to the dladm and netstat commands implement the network virtualization observability features. You can use these commands to monitor current system usage and to gather statistical data into logs.

By analyzing the historical logs, you can determine the following:

• Where network resources can be consolidated from many systems to a single system, possibly with greater bandwidth through the new generation of network interfaces. Do this prior to setting up VNICs and virtual machines or exclusive zones.

• Which applications consume the most bandwidth. This information can help you to set up bandwidth management, so that critical applications are guaranteed the most bandwidth within a particular time slot. For example, you can guarantee a video stream the greatest amount of an interface's bandwidth for 20 hours a day. For a designated four hours a day, you can give highest priority to the system's backup program. Do this as part of bandwidth management implementation.

• How to much bill customers for bandwidth used. Application service providers and other businesses that rent out system space can use the Resource control observability features to determine usage by paying customers. Some businesses offer customers Service Level Agreements, wherein the customer buys a guaranteed percentage of bandwidth from the provider. The observability features let you view how much bandwidth each customer uses and bill for possible overages. Other businesses offer customers bandwidth on a per use basis. Here the observability features directly help in billing. Do this after you have implemented resource control and, possibly, VNICs and virtual machines on a system.

The next chapter, Chapter 10, Planning for Network Virtualization and Resource Control, contains scenarios that show where the observability features are used for planning consolidation and resource control.