Part III Network Virtualization and Resource Management

Chapter 9 Introducing Network Virtualization and Resource Control (Overview)

This chapter explains the basic concepts involved in network virtualization and resource control. The following topics are covered:

• Network virtualization

• Types of virtual networks

• Virtual machines and zones

• Resource control, including flow management

• Enhanced network observability

These features help you to manage flow control, improve system performance, and configure the network utilization needed to achieve OS virtualization, utility computing, and server consolidation.

For specific tasks, refer to the following chapters:

• Chapter 11, Configuring Virtual Networks (Tasks)

• Chapter 13, Configuring Resource Management on an Interface

• Chapter 12, Administering Virtual Networks and Resource Controls (Tasks)

Network Virtualization and Virtual Networks

Network virtualization is the process of combining hardware network resources and software network resources into a single administrative unit. The goal of network virtualization is to provide systems and users with efficient, controlled, and secure sharing of the networking resources.

The end product of network virtualization is the virtual network. Virtual networks are classified into two broad types, external and internal. External virtual networks consist of several local networks that are administered by software as a single entity. The building blocks of classic external virtual networks are switch hardware and VLAN software technology. Examples of external virtual networks include large corporate networks and data centers.

An internal virtual network consists of one system using virtual machines or zones that are configured over at least one pseudo-network interface. These containers can communicate with each other as though on the same local network, providing a virtual network on a single host. The building blocks of the virtual network are virtual network interface cards or virtual NICs (VNICs) and virtual switches. Solaris network virtualization provides the internal virtual network solution.

You can combine networking resources to configure both internal and external virtual networks. For example, you can configure individual systems with internal virtual networks onto LANs that are part of a large, external virtual network. The network configurations that are described in this part include examples of combined internal and external virtual networks.

Types of Containers for Network Virtualization on the Solaris OS

You can use several different types of virtual containers in a Solaris OS-based virtual network. These containers include machines and zones. A virtual machine is a container with its own kernel and IP protocol stack. A zone is a container that provides an isolated environment for running applications.

Sun xVM Virtual Machines

Sun^TM xVM is virtual machine technology that enables you to create multiple instances of an operating system on the interfaces of a single x86–based system. The Sun xVM hypervisor controls the allocation and operation of the domains. For more information on xVM, refer to Introduction to the Sun xVM Hypervisor. xVM is based on the Open Source XEN hypervisor, which is described on the xen.org website.

Non-Global Zones and Exclusive IP Zones

Though not true virtual machines, zones are light weight application environments that share a host's kernel and IP stack. You can configure exclusive IP instances for a non-global zone, which provides that zone with its own, exclusive TCP/IP protocol stack. Both standard non-global zones and exclusive IP zones can be configured on a Solaris-based virtual network. For basic information about zones, refer to Chapter 16, Introduction to Solaris Zones, in System Administration Guide: Virtualization Using the Solaris Operating System.

LDOMs Virtual Machines

The Libvert for LDOMs (Logical Domains) software provides a hypervisor and set of commands that enable you to set up and administer logical domains on a Solaris OS-based virtual network. Each logical domain can run an instance of an operating system to enable multiple operating systems on the same computer. For information on LDOMs, refer to the Logical Domains (LDoms) 1.0.1 Administration Guide.

Parts of the Internal Virtual Network

An internal virtual network built on the Solaris OS contains the following parts:

• At least one network interface card, or NIC.

• A virtual NIC, or VNIC, which is configured on top of the network interface

• A virtual switch, which is configured at the same time as the first VNIC on the interface.

• A container, such as a zone or virtual machine , which is configured on top of the VNIC.

The next figure shows these parts and how they fit together on a single system.

Figure 9–1 VNIC Configuration for a Single Interface

The figure shows a single system with one NIC. The NIC is configured with three VNICs. Each VNIC supports a single zone. Therefore, Zone 1, Zone 2, and Zone 3 are configured over VNIC 1, VNIC 2, and VNIC 3, respectfully. The three VNICs are virtually connected to one virtual switch. This switch provides the connection between the VNICs and the physical NIC upon which the VNICs are built. The physical interface provides the system with its external network connection.

Alternatively, you can create a virtual network based on the etherstub. Etherstubs are purely software and do not require a network interface as the basis for the virtual network.

A VNIC is a virtual network device with the same data-link interface as a physical interface. You configure VNICs on top of a physical interface. For the current list of physical interfaces that support VNICs, refer to the Network Virtualization and Resource Control FAQ. You can configure up to 900 VNICs on a single physical interface. When VNICs are configured, they behave like physical NICs. In addition, the system's resources treat VNICs as if they were physical NICs.

Each VNIC is implicitly connected to a virtual switch that corresponds to the physical interface. The virtual switch provides the same connectivity between VNICs on a virtual network that switch hardware provides for the systems connected to a switch's ports.

In accordance with Ethernet design, if a switch port receives an outgoing packet from the host connected to that port, that packet cannot go to a destination on the same port. This design is a drawback for systems that are configured with zones or virtual machines. Without network virtualization, outgoing packets from a virtual machine or a zone with an exclusive stack cannot be passed to another virtual machine or zone on the same system. The outgoing packets go through a switch port out onto the external network. The incoming packets cannot reach their destination zone or virtual machine because the packets cannot return through the same port as they were sent. Therefore, when virtual machines and zones on the same system need to communicate, a data path between the containers must open on the local machine. Virtual switches provide these containers with the method to pass packets.

How Data Travels Through a Virtual Network

Figure 9–1 illustrates a simple VNIC configuration for a virtual network on a single system.

When the virtual network is configured, a zone sends traffic to an external host in the same fashion as a system without a virtual network. Traffic flows from the zone, through the VNIC to the virtual switch, and then to the physical interface, which sends the data out onto the network.

But what happens if one zone on a virtual network wants to send packets to another zone on the virtual network, given the previously mentioned Ethernet restrictions? As shown in Figure 9–1, suppose Zone 1 needs to send traffic to Zone 3? In this case packets pass from Zone 1 through its dedicated VNIC 1. The traffic then flows through the virtual switch to VNIC 3. VNIC 3 then passes the traffic to Zone 3. The traffic never leaves the system, and therefore never violates the Ethernet restrictions.

Who Should Implement Virtual Networks?

If you need to consolidate resources on Sun servers, consider implementing VNICs and virtual networks. Consolidators at ISPs, telecommunications companies, and large financial institutions can use the following network virtualization features to improve the performance of their servers and networks.

• NIC hardware, including the powerful new interfaces that support hardware rings

• Multiple MAC addresses for the VNICs

• The large amount of bandwidth provided by newer interfaces

You can replace many systems with a single system that implements running multiple zones or virtual machines, without significantly losing separation, security, and flexibility.

What Is Resource Control?

Resource control is the process of allocating a system's resources in a controlled fashion. The Solaris OS resource control features enable bandwidth to be shared among the VNICs on a system's virtual network. You can also use resource control features to allocate and manage bandwidth on a physical interface without VNICs and virtual machines. This section introduces the major features of resource control and briefly explains how these features work.

How Bandwidth Management and Flow Control Works

Searchnetworking.com defines bandwidth as “the amount of data that can be carried from one point to another in a given time period (usually a second).” Bandwidth management enables you to assign a portion of the available bandwidth of a physical NIC to a consumer, such as an application or customer. You can control bandwidth on a per- application, per-port, per-protocol, and per-address basis. Bandwidth management assures efficient use of the large amount of bandwidth available from the new GLDv3 network interfaces.

Resource control features enable you implement a series of controls on an interface's available bandwidth. For example, you can set a guarantee of an interface's bandwidth to a particular consumer. That guarantee is the minimum amount of assured bandwidth allocated to the application or enterprise The allocated portion of bandwidth is known as a share. By setting up guarantees, you can allocate enough bandwidth for applications that cannot function properly without a certain amount of bandwidth. For example, streaming media and Voice over IP consume a great deal of bandwidth. You can use the resource control features to guarantee that these two applications have enough bandwidth to successfully run.

You can also set a limit on the share. The limit is the maximum allocation of bandwidth the share can consume. Using limits, you can contain non-critical services from taking away bandwidth from critical services.

Finally, you can prioritize among the various shares allotted to consumers. You can give highest priority to critical traffic, such as heartbeat packets for a cluster, and lower priority for less critical applications.

For example, application service providers (ASPs) can offer customers fee-based levels of service that are based on the bandwidth share that the customer purchases. As part of the service level agreement (SLA), each share is then guaranteed an amount of bandwidth, to not exceed the purchased limit. (For more information on service level agreements, see Implementing Service-Level Agreements in System Administration Guide: IP Services. Priority controls might be based on different tiers of the SLA, or different prices paid by the SLA customer.

Bandwidth usage is controlled through management of flows. A flow is a stream of packets that all have certain characteristics, such as the port number or destination address. These flows are managed by transport, service, or virtual machine, including zones. Flows cannot exceed the amount of bandwidth that is guaranteed to the application or to the customer's purchased share.

When a VNIC or flow is assigned a guarantee, the VNIC is assured its designated bandwidth even if other flows or VNICs also use the interface. However, assigned guarantees are workable only if they do not exceed the maximum bandwidth of the physical interface.

Allocating Resource Control and Bandwidth Management on a Network

The following figure shows a corporate network topology that uses resource control to manage various applications.

Network With Resource Controls in Place

Figure 9–2 Network With Resource Controls in Place

This figure shows a typical network topology that uses resource controls to improve network efficiency and performance. The network does not implement VNICs and containers, such as exclusive zones and virtual machines. However, VNICs and containers could be used on this network for consolidation and other purposes.

The network is divided into four tiers:

• Tier 0 is the demilitarized zone (DMZ). This is a small local network that controls access to and from the outside world. Resource control is not used on the systems of the DMZ.

• Tier 1 is the web tier and includes two systems. The first system is a proxy server that does filtering. This server has two interfaces, bge0 and bge1. The bge0 link connects the proxy server to the DMZ on Tier 0. The bge0 link also connects the proxy server to the second system, the web server. The http and https services share the bandwidth of the web server with other standard applications. Due to the size and critical nature of web servers, shares of http and https require guarantees and prioritization.

• Tier 2 is the applications tier and also includes two systems. The second interface of the proxy server, bge1, provides the connection between the web tier and the applications tier. Through a switch, an applications server connects to bge1 on the proxy server. The applications server requires resource control to manage the shares of bandwidth given to the various applications that are run. Critical applications that need a lot of bandwidth must be given higher guarantees and priorities than smaller, or less critical applications.

• Tier 3 is the database tier. The two systems on this tier connect through a switch to the proxy server's bge1 interface. The first system, a database server, needs to issue guarantees and to prioritize the various processes involved in database lookups. The second system is a backup server for the network. This system must consume a great deal of bandwidth during backups. However, backup activities are typically carried out overnight. Using resource controls, you can control when the backup processes have the highest bandwidth guarantees and highest priorities.

Who Should Implement Resource Control Features

Any system administrator who wants to improve a system's efficiency and performance should consider implementing the resource control features. Consolidators can delegate bandwidth shares in combination with VNICs to help balance the load of large servers. Server administrators can use share allocation features to implement SLA's, such as those offered by ASPs. Traditional system administrators can use the bandwidth management features to isolate and prioritize certain applications. Finally, share allocation makes it easy for you to observe bandwidth usage by individual consumers.

Observability Features for Network Virtualization and Resource Control

Network virtualization and resource control includes observability features to help you view resource usage before setting up controls such as VNICs and flows. In tandem with Solaris extended accounting, the resource control observability features allow you to accumulate systems statistics into logs. The observability features of network virtualization and resource control include:

• Ability to monitor a running system.

• Ability to log and report statistics.

• Extended accounting features to log historical data

The new flowadm command and extensions to the dladm and netstat commands implement the network virtualization observability features. You can use these commands to monitor current system usage and to gather statistical data into logs.

By analyzing the historical logs, you can determine the following:

• Where network resources can be consolidated from many systems to a single system, possibly with greater bandwidth through the new generation of network interfaces. Do this prior to setting up VNICs and virtual machines or exclusive zones.

• Which applications consume the most bandwidth. This information can help you to set up bandwidth management, so that critical applications are guaranteed the most bandwidth within a particular time slot. For example, you can guarantee a video stream the greatest amount of an interface's bandwidth for 20 hours a day. For a designated four hours a day, you can give highest priority to the system's backup program. Do this as part of bandwidth management implementation.

• How to much bill customers for bandwidth used. Application service providers and other businesses that rent out system space can use the Resource control observability features to determine usage by paying customers. Some businesses offer customers Service Level Agreements, wherein the customer buys a guaranteed percentage of bandwidth from the provider. The observability features let you view how much bandwidth each customer uses and bill for possible overages. Other businesses offer customers bandwidth on a per use basis. Here the observability features directly help in billing. Do this after you have implemented resource control and, possibly, VNICs and virtual machines on a system.

The next chapter, Chapter 10, Planning for Network Virtualization and Resource Control, contains scenarios that show where the observability features are used for planning consolidation and resource control.

Chapter 10 Planning for Network Virtualization and Resource Control

This chapter contains information and example scenarios to help you evaluate and then design network virtualization and resource control solutions for your site. The chapter discusses the following scenarios:

• Basic Virtual Network on a Single System

• Private Virtual Network on a Single System

• Interface-based Resource Control for a Traditional Network

Each scenario contains “best usage” suggestions that explain the types of networks that best benefit from the particular scenario.

Network Virtualization and Resource Control Task Map

Planning and Designing a Virtual Network

This section describes two different scenarios for configuring a virtual network. Look over the scenarios to help determine which most closely fits the needs of your site. Then use that scenario as the basis for designing your specific virtualization solution. The scenarios include:

• Basic virtual network of two zones, especially useful for consolidating network services from the local network onto a single host.

• Private virtual network, useful for a development environment where you isolate applications and services from the public network.

Basic Virtual Network on a Single System

Figure 10–1 shows the basic virtual network, or “network in a box” that is used in examples throughout the section Configuring a Basic Virtual Network.

Figure 10–1 Virtual Network on a Single Host

This virtual network consists of the following:

• A single GLDv3 network interface e1000g0. This interface connects to the public network 192.168.3.0/24. Interface e1000g0 has the IP address 192.168.3.70.

• A virtual switch, which is automatically configured when you create the first VNIC.

• Two VNICs. vnic1 has the IP address 192.168.3.20, and vnic2 has the IP address 192.168.3.22.

• Two exclusive IP zones. zone1 is configured over vnic1, and zone2 is configured over vnic2.

  ---

  Note –

  Though the example uses exclusive IP zones, you can also use other types of virtual machines such as LDOMS in this scenario.

  ---

The VNICs and zones in this configuration allow access to the public. Therefore, the zones can pass traffic beyond the e1000g0 interface. Likewise, users on external networks can reach applications and services offered by the zones.

Best Uses for the Basic Virtual Network

The network in a box scenario enables you to isolate processes and applications into individual virtual machines or zones on a single host. Furthermore, this scenario is expandable to include many containers, each of which could run a completely isolated set of applications. The scenario improves a system's efficiency and, by extension, the efficiency of the local network. Therefore, this scenario is ideal for the following users:

• Network consolidators and others who want to consolidate the services of a LAN onto a single system.

• Any site that rents out services to customers. You can rent out individual zones or virtual machines, observe traffic, and take statistics for performance measuring or for billing purposes on each zone in the virtual network.

• Any administrator who wants to isolate processes and applications to separate containers to improve system efficiency .

For More Information

• For tasks that implement the basic virtual network, go to Configuring a Basic Virtual Network.

• For examples that show how to configure the virtual network shown in this section, go to Configuring a Basic Virtual Network.

• For conceptual information about VNICs and virtual networks, go to Network Virtualization and Virtual Networks.

• For conceptual information about zones, go to Chapter 16, Introduction to Solaris Zones, in System Administration Guide: Virtualization Using the Solaris Operating System.

Private Virtual Network on a Single System

Figure 10–2 shows a single system with a private network behind packet filtering software that performs network address translation (NAT). This figure illustrates the scenario that is built in Example 11–7.

Figure 10–2 Private Virtual Network on a Single Host

The topology features a single system with a public network, including a firewall, and a private network built on an etherstub pseudo-interface. The public network runs in the global zone and consists of the following elements:

• GLDv3 network interface e1000g0 with the IP address 192.168.3.70.

• A firewall implemented in the IP Filter software. For an introduction to IP Filter, refer to Introduction to Solaris IP Filter in System Administration Guide: IP Services.

• etherstub0, a pseudo-interface upon which the virtual network topology is built. Etherstubs provide the ability to create a virtual network on a host. That network is totally isolated from the external network.

The private network consists of the following elements:

• A virtual switch which provides packet forwarding among the VNICs of the private network.

• vnic0, which is the VNIC for the global zone, and has the IP address 192.168.0.250.

• vnic1 with the IP address 192.168.0.200 and vnic2 with the IP address 192.168.0.220. All three VNICs are configured over etherstub0.

• zone1, which is configured over vnic1, and zone2, which is configured over vnic2.

Best Uses for a Private Virtual Network

Consider creating a private virtual network for a host that is used in a development environment. Using the etherstub framework, you can totally isolate software or features under development to the containers of the private network. Moreover, you can use firewalling software for network address translation of outgoing packets that originate in the containers of the private network. The private network is a smaller version of the eventual deployment environment.

For More Information

• For tasks for implementing the private virtual network, go to Configuring a Private Virtual Network

• For the example that shows how to configure the private virtual network shown in this section, go toExample 11–7.

• For conceptual information about VNICs and virtual networks, go to Network Virtualization and Virtual Networks.

• For conceptual information about zones, go to Chapter 16, Introduction to Solaris Zones, in System Administration Guide: Virtualization Using the Solaris Operating System.

• For information about Solaris IP Filter, go to Introduction to Solaris IP Filter in System Administration Guide: IP Services.

Network Interface-Based Flow Control

This section contains scenarios for using interface-based flow control. Flow control incorporates resource control and bandwidth management. This process involves organizing packet traffic into flows of related applications, and then assigning portions of an interface's overall bandwidth to the individual flows. You can also assign a specific application's traffic to a flow to isolate the traffic without assigning the flow an amount of bandwidth. Isolating particular types of traffic to flows makes the traffic more easily observed, evaluated, and possibly billed. The following scenarios are described:

• Interface-based Resource Control for a Traditional Network

• Flow Control for the Virtual Network

Look over these scenarios to see which is most applicable for your site. For the tasks that implementing these scenarios, refer to Chapter 13, Configuring Resource Management on an Interface.

Interface-based Resource Control for a Traditional Network

Figure 10–3 shows the network topology for a small business that needs to manage the bandwidth on its proxy server. The proxy server offers a public web site as well as a proxy for internal clients that require services from various servers on the site's internal network.

This scenario does not show how to configure flow control for a virtual network, and consequentially does not include VNICs. For flow control on a virtual network, refer to Flow Control for a Virtual Network.

Figure 10–3 Resource Control for a Proxy Server on a Traditional Network

The figure shows that the company has a public network, 10.10.6.0/8, that also serves as a demilitarized zone (DMZ). A system on the DMZ provides name-to-address translation (NAT) through an IP Filter firewall. The company has a large system that functions as the proxy server. The system has two wired interfaces and 16 processor sets with IDs 0–16. This system is connected to the public network through the interface nge0, with IP address l0 10.6.5. The link name for the interface is DMZ0. Through DMZ0, the proxy server offers HTTP and HTTPS service through the company's public web site.

The figure also illustrates the company's internal network, 10.10.12.0/24. The proxy server connects to the internal 10.10.12.0/8 network through interface nge1, with the IP address 10.10.12.42. The link name for this interface is internal0. Through the internal0 data link, the proxy server operates on behalf of internal clients that request the services of an application server, 10.10.12.45, database server, 10.10.12.46, and backup server, 10.10.12.47.

Best Use of Interface–based Resource Control on a Traditional Network

Consider establishing flow control for heavily used systems, especially those with newer GLD.v3 interfaces with large amounts of available bandwidth. Interface-based flow control improves the efficiency of the interface, the system, and potentially the network. You can apply flow control to any system on any type of network. Furthermore, if your goal is to improve network efficiency, you can separate various services into individual flows. This action assigns separate hardware and software resources to the individual flows, thus isolating them from other services on a particular system. After you establish flows, you can observe traffic for each flow and gather statistics. Thereafter, you can assign bandwidth amount and priorities to control usage on the interfaces.

For More Information

• For tasks for implementing flow control, refer to Chapter 13, Configuring Resource Management on an Interface

• For conceptual information about bandwidth management and resource control, refer to What Is Resource Control?

• For detailed technical information, refer to the dladm(1M) and flowadm(1M)man pages.

Flow Control for the Virtual Network

This scenario shows how flow control is used within a virtual network, such as the basic virtual network that is introduced in Basic Virtual Network on a Single System.

Figure 10–4 Basic Virtual Network With Flow Controls

The topology is described in Basic Virtual Network on a Single System. Here a host has one network interface, e1000g0, with two VNICs, vnic1 and vnic2. zone1 is configured over vnic1, and zone2 is configured over vnic2. Resource management for the virtual network involves creating flows on a per-VNIC basis. These flows define and isolate packets with similar characteristics, such as port number or IP address of the sending host. You assign bandwidth based on the usage policy for the system.

Another very common usage for flow controls on VNIC traffic is by companies that rent out zones. You create different service level agreements for customers, and rent out zones with a guaranteed amount of bandwidth. When you create flows on a per-zone basis, you can isolate and observe each customer's traffic and monitor bandwidth usage. If your service level agreement is based strictly on usage, you can use Solaris statistics and accounting features to bill customers.

Flow controls are effective for any network that requires bandwidth management for traffic over zones. Larger organizations, such as application service providers (ASPs) or Internet service providers (ISP), can take advantage of resource control for VNICs for data centers and for multiprocessor systems. The individual zones can be rented out to customers for different levels of service. Therefore, you could rent out zone1 at the standard price and offer a standard bandwidth. Then, you could rent out zone2 at a premium price and give that customer a high level of bandwidth.

How to Create a Usage Policy for Applications on a Virtual Network

1. List the applications that you want to run on the host.

2. Determine which applications have historically used the most bandwidth or require the most bandwidth.

   For example, the telnet application might not consume huge amounts of bandwidth on your system, but it could be heavily used. Conversely, database applications consume a huge amount of bandwidth, but might only be used on a sporadic basis. Consider monitoring traffic for these applications prior to assigning them to zones. You can use the statistical option of the dladm show-link command to gather statistics, as described in Gathering Usage Statistics for VNICs and Flows.

3. Assign these applications to separate zones.

4. Create flows for any application running in zone1 whose traffic you want to isolate and control.

5. Assign bandwidth to flows based on usage policies in place for your site.

How to Create a Service Level Agreement for the Virtual Network

1. Design a policy that offers different levels of services at different prices.

   For example, you might create a basic, superior, and high levels of service, and price each level accordingly.

2. Decide whether you want to charge customers on a monthly, per service level basis, or charge customers on an actual bandwidth consumed basis.

   If you choose the latter pricing structure, you need to gather statistics on each customer's usage.

3. Create a virtual network on a host, with containers for each customer.

   A very common implementation is to give each customer their own zone running over a VNIC.

4. Create flows that isolate traffic for each zone.

   To isolate all traffic for the zone, you use the IP address that is assigned to the zone's VNIC.

5. Assign bandwidth to each VNIC based on the service level purchased by the customer assigned to that VNIC's zone.

Chapter 11 Configuring Virtual Networks (Tasks)

This chapter contains tasks for configuring internal virtual networks, or “networks in a box.” The topics that are covered include:

• Configuring a Basic Virtual Network

• Configuring a Private Virtual Network

Virtual Networks Task Map

This table lists the tasks for configuring a virtual network, including links to the specific tasks. Note that not all tasks will apply to your virtual network scenario.

Configuring a Basic Virtual Network

This section contains tasks for configuring a basic virtual network. For a topology diagram of a virtual network, see Figure 10–1. Use the following tasks to build the virtual network.

• How to Create a Virtual Network Interface

• How to Create an Exclusive IP Zone Over a VNIC

• How to Install the Exclusive IP Zone on a VNIC

• How to Configure an Exclusive IP Zone Over a VNIC Through the Zone Console

• How to Manually Configure the VNIC and Exclusive IP Zone

• How to Verify the Exclusive IP Zone Over VNIC Configuration

The steps in all tasks in this chapter use the vi text editor in a terminal window. Alternatively, you can use the text editor of your choice.

How to Create a Virtual Network Interface

This procedure shows how to create a virtual network interface card (VNIC). VNICs are pseudo-interfaces upon which to build the containers of the virtual network. The resulting VNIC has an automatically generated MAC address. Depending on the network interface in use, you can instead explicitly assign a MAC address to a VNIC, as described in the dladm(1M).

When you first log in to a system, you are automatically in its global zone, which is where you configure VNICs. You can use VNICs in the global zone or as the building blocks for a particular type of non-global zone, the exclusive IP zone. For an introduction to zones, refer to Zones Overview in System Administration Guide: Virtualization Using the Solaris Operating System.

1. Become superuser or assume the equivalent root role.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

2. View information about the system's available physical interfaces.

   # dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE e1000g2 Ethernet unknown 0 half e1000g2 e1000g0 Ethernet up 1000 full e1000g0

   Currently the system has two installed interfaces, e1000g0 and e1000g2.

3. Check the status of the data links on the system.

   # dladm show-link LINK CLASS MTU STATE OVER e1000g2 phys 1500 unknown -- e1000g0 phys 1500 up --

   Only the e1000g0 data link is running over that interface and is configured “UP”.

   Unless you create customized names for your data links, the data link has the same name as the network interface device name that is displayed by dladm show-phys. For example, network interface e1000g0 has the data link name e1000g0 until you customize it. For more information on customized data link names, refer to Data Link and IP Interface Configuration (Tasks).

4. Check the status of any interfaces on the IP layer.

   # ifconfig -a lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 e1000g0: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2 inet 192.168.3.70 netmask ffffff00 broadcast 192.168.3.255 ether 0:14:4f:94:d0:40

   The output indicates that interface e1000g0 has the IP address 192.168.3.70. Therefore, the system is connected to the 192.168.3.0/24 network. e1000g0 has the MAC address 0:14:4f:94:d0:40.

5. Create a VNIC in the system's global zone.

   # dladm create-vnic -l data-link vnic-name

   • data-link is the name of the interface where the VNIC is to be configured.

   • vnic-name is the name that you want to give the VNIC.

   For example, to create a VNIC named vnic0 on interface e1000g0, you would type the following:

   # dladm create-vnic -l e1000g0 vnic0

   Repeat this step for all planned VNICs in the virtual network.

6. Plumb the VNIC and assign it an IP address.

   All VNICs must be configured and plumbed on the IP level. VNICs that are used in conjunction with an exclusive IP zone can be plumbed as part of the initial zone configuration or manually, using the steps in How to Manually Configure the VNIC and Exclusive IP Zone.

   For VNICs to be configured in the global zone, do the following:

   1. Use the ifconfig command as shown to configure the interface.

      # ifconfig vnic-name plumb # ifconfig vnic-name IP-address # ifconfig vnic-name up

      For example, you would configure and plumb vnic0 over interface e1000g0as follows:

      # ifconfig vnic0 plumb # ifconfig vnic0 192.168.3.250 # ifconfig vnic0 up

   2. Verify that the VNIC is configured and plumbed.

      # ifconfig -a

      Your output should resemble the following:

      lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 e1000g0:flags=201100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4,CoS> mtu 1500 index 2 inet 192.168.3.70 netmask ffffff00 broadcast 192.168.3.255 ether 0:14:4f:94:d0:40 vnic0: flags=201100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4,CoS> mtu 9000 index 5 inet 192.168.3.250 netmask ffffff00 broadcast 192.168.0.255 ether 2:8:20:c2:39:38

      Look for the VNIC that you just configured in the ifconfig output. For example, vnic0 is in the previous output. The IP address that you specified and the ifconfig “UP” flag in the output must also be present. These items indicate that the VNIC is correctly configured and plumbed.

7. Ensure that the VNIC configuration persists across reboots

   Create the file /etc/hostname.vnic-name.

   • In the global zone, do the following:

     # cd /etc # vi hostname.vnic-name IP address of vnic-name

     For example, you type the following:

     # cd /etc # vi hostname.vnic0 192.168.3.250

   • Update the /etc/inet/hosts file with entries for all the VNICs you have created.

     The entries in the file should have the following format:

     vnic-IP-address zoneID-vnic-IP-address

     For example, you might create the following entries:

     192.168.3.250 zone0-192-168-3-250

     ---

     Note –

     When creating the zone alias entry, be sure to put a dash after the zoneID. Additionally, substitute dashes for the dot delimeters in the IP address, as shown previously.

     ---

   • For exclusive IP zones, refer to the instructions in How to Verify the Exclusive IP Zone Over VNIC Configuration

8. Verify that the new VNIC is created.

   # dladm show-vnic LINK SPEED MACADDRESS MACADDRTYPE vnic0 0 Mbps 2:8:20:c2:39:38 random

Example 11–1 Creating Virtual Network Interfaces (VNIC)

This example contains the commands to use to create and verify three VNICs. One VNIC is used in the global zone. Two other VNICs are used with the exclusive IP zones in the upcoming tasks. This example illustrates the steps in Configuring a Basic Virtual Network to accomplish the following:

• Create three VNICs, vnic0, vnic1, and vnic2 on the data link e1000g0.

• Verify that the VNICs were created

  .

• Configure and plumb vnic0 in the global zone.

• Make vnic0 persist across reboots.

  ---

  Note –

  You must log in to the system as superuser or equivalent role to run the next commands.

  ---

# dladm show-phys
LINK         MEDIA                STATE      SPEED DUPLEX   DEVICE
e1000g2      n                    unknown    0    half      e1000g2
e1000g0      Ethernet             up         1000 full      e1000g0
# dladm show-link
LINK        CLASS    MTU    STATE    OVER
e1000g2     phys     1500   unknown  --
e1000g0     phys     1500   up       --
# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
         inet 127.0.0.1 netmask ff000000
e1000g0: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2
        inet 192.168.3.70 netmask ffffff00 broadcast 192.168.3.255
        ether 0:14:4f:94:d0:40

# dladm create-vnic -l e1000g0 vnic0
# dladm create-vnic -l e1000g0 vnic1
# dladm create-vnic -l e1000g0 vnic2
# dladm show-vnic

LINK        OVER             SPEED  MACADDRESS         MACADDRTYPE
vnic0       e1000g0      1000 Mbps  2:8:20:c2:39:38    random
vnic1       e1000g0      1000 Mbps  2:8:20:5f:84:ff    random
vnic2       e1000g0      1000 Mbps  2:8:20:54:f4:74    random

# ifconfig vnic0 plumb
# ifconfig vnic0 192.168.3.250
# ifconfig vnic0 up

# ifconfig -a

lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g0:flags=201100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4,CoS>mtu 1500 index 2
        inet 192.168.3.70 netmask ffffff00 broadcast 192.168.3.255
        ether 0:14:4f:94:d0:40
vnic0: flags=201100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4,CoS> mtu 9000 index 5
        inet 192.168.3.250 netmask ffffff00 broadcast 192.168.0.255
        ether 2:8:20:c2:39:38

# vi /etc/hostname.vnic0
192.168.3.250
# vi /etc/inet/hosts
# Internet host table
#
::1     localhost
127.0.0.1       localhost
192.168.3.70    myhost     loghost
192.168.3.250      zone0-192-168-3-250

Next Steps

• To configure a zone over the VNIC, see How to Create an Exclusive IP Zone Over a VNIC.

• To configure an xVM domain over VNICs, information is to come.

• For an example of the configuration of a basic virtual network, see Example 11–6.

How to Create an Exclusive IP Zone Over a VNIC

The following task explains how to create two exclusive IP zones for a virtual network. If you want to use zones as the containers for the virtual network, always use exclusive IP zones. You cannot create non–global shared IP zones over VNICs in a virtual network scenario.

As an alternative, you can useSun xVM domains as the containers in the virtual network. For information about configuring Sun xVM Server and its domains, refer to theSun xVM Server Information Wiki.

Before You Begin

This procedure assumes that you have already configured at least two VNICs over a data link, as shown in Example 11–1. The VNICs are named vnic0, vnic1, and vnic2.

1. On the system where you create the virtual network, become superuser or assume the equivalent root role.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

2. View the state of the VNICs on the system.

   # dladm show-vnic LINK OVER SPEED MACADDRESS MACADDRTYPE vnic1 e1000g0 1000 Mbps 2:8:20:5f:84:ff random vnic2 e1000g0 1000 Mbps 2:8:20:54:f4:74 random

   The output indicates that vnic1 and vnic2 are currently configured over interface e1000g0.

3. Begin the creation process for the exclusive IP zone by running the zonecfg interactive utility.

   ---

   Tip –

   Alternatively, you can run zonecfg as a command with appropriate subcommands and options to create the zone. For more information, refer to How to Configure the Zone in System Administration Guide: Virtualization Using the Solaris Operating System and the zonecfg(1M) man page.

   ---

   # zonecfg -z zoneID

   where ID represents the number to identify the zone. For example, the following command creates “zone1.”

   # zonecfg -z zone1

   The zonecfg program runs and prompts for information about the new zone.

   zonecfg:zone1>

4. Start zone creation through the zonecfg interactive utility.

   zonecfg:zone1> create

   The remaining steps show how to create the exclusive IP zone and set other parameters. For a detailed description of parameters available for the zone, see How to Configure the Zone in System Administration Guide: Virtualization Using the Solaris Operating System.

5. Create the zone path by setting a home directory for the zone, and then enable automatic booting.

   zonecfg:zone1> set zonepath=zone-home-directory zonecfg:zone1> set autoboot=true

   For example, zone-home-directory might be /export/home/zone1.

   The global zone will include home directories for all zones that you create through zonecfg. Thus, the /export/home directory in the global zone must contain an entry for zone1.

6. Create the zone as exclusive IP.

   zonecfg:zone1> set ip-type=exclusive

7. Create the network interface for the zone.

   zonecfg:zone1> add net

   This response starts the network configuration subprogram of zonecfg.

8. Set the previously configured VNIC as the interface for the zone.

   zonecfg:zone1:net> set physical=vnic-data-link

   For example, you create vnic1 for zone1 as follows:

   zonecfg:zone1:net> set physical=vnic1

   ---

   Note –

   Although zonecfg has many options for describing a network interface, only use the set-physical parameter of add net for an IP exclusive zone.

   ---

9. Complete zone configuration and verify the results.

   zonecfg:zone1:net> end zonecfg:zone1> verify

   The verify command checks for any configuration errors. If you have received errors, fix the configuration. If verify does not respond, assume the configuration is correct and continue.

10. View information about the zone you just created.

    Use the info directive, as shown below:

    zonecfg:zone1> info zonename: zone1 zonepath: /export/home/zone1 brand: native autoboot: true . . net: address not specified physical: vnic1

    The message “address not specified” verifies that you have not specified an IP address for the zone. You create IP addresses for the zone's VNIC outside the zonecfg utility, as described in the upcoming procedure How to Configure an Exclusive IP Zone Over a VNIC Through the Zone Console.

    If info displays other incorrect information, you can modify the parameters, as explained in Using the zonecfg Command to Modify a Zone Configuration in System Administration Guide: Virtualization Using the Solaris Operating System. If the information is correct, continue to the next step.

11. Commit the zone and close zonecfg.

    zonecfg:zone1> commit zonecfg:zone1> exit

    Be sure to commit the zone before exiting zonecfg.

12. Create more zones, as needed, by following Steps 3 through 11.

Example 11–2 Creating an Exclusive IP Zone Over a VNIC

The following example contains the commands for creating a zone using the zonecfg utility. When the example is complete, the result is a zone called zone1 that is configured on vnic1. This example assumes that the VNIC is already created, as shown in Example 11–1. You can use this example for configuring as many exclusive IP zones over VNICs as you need for your virtual network. For an illustration of a basic virtual network, refer to Figure 10–1.

You must log in to the global zone of the system as superuser or equivalent role to run the next commands.

# dladm show-vnic

LINK        OVER             SPEED  MACADDRESS         MACADDRTYPE
vnic1       e1000g0      1000 Mbps  2:8:20:5f:84:ff    random
vnic2       e1000g0      1000 Mbps  2:8:20:54:f4:74    random

# zonecfg -z zone1

zonecfg:zone1> create
zonecfg:zone1> set zonepath=/export/home/zone1
zonecfg:zone1> set autoboot=true
zonecfg:zone1> set ip-type=exclusive
zonecfg:zone1> add net
zonecfg:zone1:net> set physical=vnic1
zonecfg:zone1:net> end
zonecfg:zone1> verify

zonecfg:zone1> info
zonename: zone1
zonepath: /export/home/zone1
brand: native
autoboot: true
.
.
net:
        address not specified
        physical: vnic1

zonecfg:zone1> commit
zonecfg:zone1> exit

Next Steps

• To continue with zone creation, go to How to Install the Exclusive IP Zone on a VNIC.

• For detailed information about the zonecfg command, refer to zonecfg(1M).

How to Install the Exclusive IP Zone on a VNIC

Before You Begin

This procedure assumes that you have completed VNIC creation, as described in How to Create a Virtual Network Interface. You also must have created and committed an exclusive IP zone, as described in How to Create an Exclusive IP Zone Over a VNIC.

In this procedure you install the newly created zone1 over vnic1.

1. On the system where you create the virtual network, become superuser or assume the equivalent root role.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

   ---

   Note –

   When you first log in to a system, you are automatically in its global zone. For an introduction to zones, refer to Zones Overview in System Administration Guide: Virtualization Using the Solaris Operating System.

   ---

2. Verify that the new zone exists.

   # zoneadm -z zoneID verify

   The zoneadm command displays output similar to the following for a zone that is not yet installed:

   WARNING: /export/home/zone1 does not exist, so it could not be verified. When 'zoneadm install' is run, 'install' will try to create /export/home/zone1, and 'verify' will be tried again, but the 'verify' may fail if: the parent directory of /export/home/zone1 is group- or other-writable or /export/home/zone1 overlaps with any other installed zones.

   This message indicates that zone is ready to be installed.

3. Install the new zone.

   Use the following syntax:

   # zoneadm -z zoneID install

   For example, you would type:

   # zoneadm -z zone1 install Preparing to install zone <zone1> Creating list of files to copy from the global zone. . . Zone <zone1> is initialized.

4. Verify that the zone is installed.

   zoneadm list -iv

   You receive output similar to the following:

   ID NAME STATUS PATH BRAND IP 0 global running / native shared - zone1 installed /export/home/zone1 native excl

   The output indicates that the exclusive IP zone is installed but not yet running.

5. Boot the zone and then observe its new status.

   # zoneadm -z zone1 boot # zoneadm list -v ID NAME STATUS PATH BRAND IP 0 global running / native shared 1 zone1 running /export/home/zone1 native excl

   Note that zone1 has changed its state to running.

6. Repeat this procedure for all exclusive IP zones in your virtual network.

Example 11–3 Installing and Booting an Exclusive IP Zone Over a VNIC

The following example contains the zoneadm and zlogin -C commands for installing the exclusive IP zone zone1 that is configured over vnic1. This example assumes that both the VNIC and zone are created, as shown in Example 11–2. You can use this example for installing every exclusive IP zone over a VNIC for your virtual network. For an illustration of a basic virtual network, refer to Figure 10–1.

You must log in to the global zone of the system as superuser or equivalent role to run the next commands.

# zoneadm -z zone1 verify
WARNING: /export/home/zone1 does not exist, so it could not be verified.
When 'zoneadm install' is run, 'install' will try to create
/export/home/zone1, and 'verify' will be tried again,
but the 'verify' may fail if:
the parent directory of /export/home/zone1 is group- or other-writable
or
/export/home/zone1 overlaps with any other installed zones.

# zoneadm -z zone1 install
Preparing to install zone <zone1>.
Creating list of files to copy from the global zone.
.
.
Zone <zone1> is initialized. 

zoneadm list -iv
  ID NAME             STATUS     PATH                           BRAND    IP
   0 global           running    /                              native   shared
   - zone1            installed  /export/home/zone1             native   excl
  

# zoneadm -z zone1 boot
# zoneadm list -v
  ID NAME             STATUS     PATH                           BRAND    IP
   0 global           running    /                              native   shared
   1 zone1            running    /export/home/zone1             native   excl
   

Next Steps

After booting the zone, you need to perform initial configuration steps for the exclusive IP zone over a VNIC. Use one of the following methods to complete zone configuration:

• Perform initial zone configuration on the newly-booted zone from the zone console, as explained in How to Configure an Exclusive IP Zone Over a VNIC Through the Zone Console.

• Manually perform basic zone and VNIC configuration, including plumbing the VNIC and updating the affected network files. For instructions, see How to Manually Configure the VNIC and Exclusive IP Zone.

• Configure the necessary parameters for zone configuration in the /etc/sysidcfg file, as explained in How to Use an /etc/sysidcfg File to Perform the Initial Zone Configuration in System Administration Guide: Virtualization Using the Solaris Operating System. If you configure the zone through /etc/sysidcfg, you might need to manually add IP addresses for the zone, as shown in How to Manually Configure the VNIC and Exclusive IP Zone.

How to Configure an Exclusive IP Zone Over a VNIC Through the Zone Console

After you have installed and booted all zones for the virtual network, your final step is to configure the zones.

Before You Begin

You must have created, installed, and booted exclusive IP zones over VNICs, as explained in the following procedures:

• How to Create an Exclusive IP Zone Over a VNIC

• How to Install the Exclusive IP Zone on a VNIC

1. On the system where you create the virtual network, become superuser or assume the equivalent root role.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

2. Log in to the console of a zone

   Begin initial zone configuration through the zone console.

   # zlogin -C zone-name

   where zone-name represents the name of the zone that you want to configure. For example, to log in to the console for zone1, type the following:

   # zlogin -C zone1

   Depending on your system, you might receive prompts from the console to set language preference and other parameters. Answer these prompts and continue.

3. Select a terminal type.

   The zone configuration program offers choices such as the following

   What type of terminal are you using? 1) ANSI Standard CRT 2) DEC VT52 . . 8) Sun Workstation 9) Televideo 910 10) Televideo 925 11) Wyse Model 50 12) X Terminal Emulator (xterms)

   Type the number for the console terminal type for your system, for example 12 for an X terminal window.

4. Confirm or change the information displayed by the zone configuration program.

   You receive a series of prompts for information about the new zone. Most of the responses are automatically generated. If the information is incorrect, you can press F4 and supply the correct information. Otherwise, press F2 to accept and continue to the next parameter.

   The information that you need to supply or verify includes:

   • IP address for the zone. Each exclusive IP zone and its corresponding VNIC must have a unique IP address. You can use a DHCP address or a static IP address.

   • Host name. Enter the host name for the zone, for example, zone1.

   • Whether the system with the virtual network is part of a subnet.

   • Netmask of the IP address.

   • Default route. You can use the IP address of the interface on which the virtual network is built.

   • IP address of a router on the system's network

   When you are finished configuring the zone, the system reboots. After the reboot, the zone is ready for use.

5. Repeat the initial configuration steps for all zones in the virtual network.

Example 11–4 Final Configuration of an Exclusive IP Zone Over a VNIC

This example shows a typical zone configuration session using the zone console configuration program.

# zlogin -C zone1
What type of terminal are you using?
.
.
.
8) Sun Workstation
9) Televideo 910
10) Televideo 925
11) Wyse Model 50
12) X Terminal Emulator (xterms)
13) CDE Terminal Emulator (dtterm)
14) Other
Type the number of your choice and press Return: 13
.
.
IP address for zone1: 192.168.3.20
.
Confirm the following information. If it is correct, press F2;
to change any information, press F4.

Hostname: zone1
IP address: 192.168.3.20
System part of a subnet: Yes
Netmask: 255.255.255.0
Enable IPv6: No
Default route: 192.168.3.70
Router IP address: 192.168.3.25

System reboots.

Next Steps

Verify that zone configuration is correct, as explained in How to Verify the Exclusive IP Zone Over VNIC Configuration.

How to Manually Configure the VNIC and Exclusive IP Zone

This procedure explains how to manually configure IP addresses for VNICs and their associated zones. If you configured zones through the zone console after the initial booting, these addresses are configured automatically. You need to follow the next steps only if one of the following conditions is true:

• You did not run the zone console configuration program after booting the zones and want to configure IP addresses manually. In this case, you should perform all the steps in the procedure.

• You performed the validation checks in How to Verify the Exclusive IP Zone Over VNIC Configurationand uncovered problems. Some typical problems include the VNIC was not plumbed, or problems with a relevant files, such as hostname.vnic-name. In this case, complete only the steps that relate to the specific problems that you found.

Before You Begin

The procedure assumes that both the VNIC and zone are created, installed, and booted in the global zone.

1. On the system where you create the virtual network, become superuser or assume the equivalent root role.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

2. Log in to the zone.

   For example, you would type:

   # zlogin zone1 # pwd /

3. Verify that the VNIC is configured.

   # ifconfig -a lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1 inet6 ::1/128

   In this output, only the IPv4 and IPv6 loopback addresses are plumbed and up. No entry exists for the VNIC.

4. Manually configure and plumb the VNIC from within the exclusive IP zone.

   You must plumb a VNIC in the following order for it to function properly in the virtual network.

   # ifconfig vnic-data-link plumb # ifconfig vnic-data-link IP-address # ifconfig vnic-data-link up

   For example, to add IP address 192.168.3.20 to vnic1, do the following:

   # ifconfig vnic1 plumb # ifconfig vnic1 192.168.3.20 # ifconfig vnic1 up

5. Verify that the VNIC is now configured and plumbed.

   # ifconfig -a lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 vnic1: flags=201000842<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2 inet 192.168.3.20 netmask ffffff00 broadcast 192.168.3.255 ether 2:8:20:54:f4:74

6. Exit the exclusive IP zone, and go to the zone's subdirectory tree in the global zone.

   # exit # cd /export/home/zone1

7. Create a hostname.vnic–name file for the VNIC.

   # cd root/etc # vi hostname.vnic1 zoneID-IP address

   For example, for zone1 you type:

   zone1-192.183.3.20

8. Add an entry for the zone in the root/etc/inet/hosts file.

   # cd inet # vi hosts # Internet host table # ::1 localhost 127.0.0.1 localhost 192.168.3.20 zone1 loghost

9. If the entry does not already exist, add the VNIC and its zone to the global zone's /etc/inet/hosts file.

   # cd /etc/inet # vi hosts # Internet host table # ::1 localhost 127.0.0.1 localhost 192.168.3.70 myhost loghost 192.168.3.20 zone1-192-168-3-20

Example 11–5 Manually Configuring a VNIC and Exclusive IP Zone

This example illustrates the following procedures:

• Plumbing vnic1 from within a zone and assigning an IP address to the VNIC.

• Adding the IP address for zone1 and vnic1 to the appropriate files, so that this IP address persists across reboots.

You must log in to the global zone of the system as superuser or equivalent role to run the next commands.

# zlogin zone1
/
# ifconfig vnic1 plumb
# ifconfig vnic1 192.168.3.20
# ifconfig vnic1 up
# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
	     inet 127.0.0.1 netmask ff000000
vnic1: flags=201000842<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2
           inet 192.168.3.20 netmask ffffff00 broadcast 192.168.3.255
           ether 2:8:20:54:f4:74
# exit
# cd /export/home
# cd zone1/root/etc
# vi hostname.vnic1
zone1-192.168.3.20

# vi inet/hosts
# Internet host table
#
::1                  localhost
127.0.0.1            localhost
192.168.3.20  zone1  loghost

# cd /etc/inet
# vi hosts
# Internet host table
#
::1     localhost
127.0.0.1         localhost
192.168.3.70      myhost     loghost
192.168.3.20      zone1-192-168-3-20

Next Steps

After you are finished, verify that your configuration is correct, as explained in How to Verify the Exclusive IP Zone Over VNIC Configuration.

How to Verify the Exclusive IP Zone Over VNIC Configuration

After you complete zone configuration, confirm that the zones and VNICs are now configured as you expected.

Before You Begin

The procedures in this task assume that you have installed and configured two or more exclusive IP zones over a VNIC. If you have not done this, perform the following procedures, in sequential order:

• How to Create an Exclusive IP Zone Over a VNIC

• How to Install the Exclusive IP Zone on a VNIC

• How to Configure an Exclusive IP Zone Over a VNIC Through the Zone Console or How to Manually Configure the VNIC and Exclusive IP Zone

1. On the system where you build the virtual network, become superuser or assume the equivalent root role in the global zone.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

2. Go to the parent directory for all zones that you created.

   You supply this directory to the zonecfg command as the first part of the zone path.

   # cd parent-zone-path

   For example, to access the parent directory for both zones created in the procedure How to Create an Exclusive IP Zone Over a VNIC, type:

   # cd /export/home

   If the parent directory for the zones does not exist, check your zone configuration.

3. Verify that the zone home directory trees exist in the correct parent directory in the global zone.

   # pwd /export/home # ls zone-name

   For example, to verify that the zone subdirectories have been created in the parent /export/home directory, in the global zone, type:

   # ls zone1 zone2

   The subdirectories for the two new zones have been created. If these subdirectories do not exist, check your zone configuration.

4. Verify that the hostname.vnic-name file exists and that its entry is correct.

   Each VNIC that you configure for a zone requires a hostname.vnic-name file to ensure that the IP address of the VNIC and zone persist after reboots. First, verify that a hostname.vnic-name file exists:

   cd /export/home/zone-name/root/etc # ls host* hostname.vnic1 hosts

   This output indicates that a hostname.vnic1 file exists. The file should contain one entry with the name of the zone, for example:

   cat hostname.vnic1 zone1

   If this file does not exist, create it as shown in How to Manually Configure the VNIC and Exclusive IP Zone.

5. Check the contents of the zone's hosts file.

   # pwd /export/home/zone-name/root/etc/ # cat hosts # Internet host table # ::1 localhost 127.0.0.1 localhost 192.168.3.20 zone1 loghost

   In this output, the entry 192.168.3.20 zone1 loghost shows the address that is assigned to the VNIC for zone1. Your output should have a similar entry for the zone and VNIC.

   If this file does not have an entry for the zone, refer to the appropriate step in How to Manually Configure the VNIC and Exclusive IP Zone.

6. Add the IP addresses of the VNICs and names of their associated zones to the /etc/inet/hosts file in the global zone.

   ---

   Note –

   Be sure that you are in the hosts file for the global zone, not the host file in a subdirectory tree for a zone.

   ---

   # cd /etc/inet # vi hosts # Internet host table # ::1 localhost 127.0.0.1 localhost 192.168.3.70 myhost loghost

   The only non-loopback IP address in this output is 192.168.3.70, the address associated with the system's network interface. Add entries for all VNICs associated with zones to this file, using the following format:

   VNIC-IP-address zone-name- IP address

   For example, you would type the following entry for vnic1 and zone1:

   192.168.3.20 zone1-192-168-3-20

7. Log in to the new zone and verify that you are in its home directory:

   For example, for zone1 you would type:

   # zlogin zone1 # pwd /

   You are now in the root directory of zone1. If you cannot log in to the zone, check your zone configuration.

8. Verify that the VNIC you previously defined for the zone is now configured as an IP interface.

   Your output should resemble the following:

   # ifconfig -a lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 vnic1: flags=201000842<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2 inet 192.168.3.20 netmask ffffff00 broadcast 192.168.3.255 ether 2:8:20:54:f4:74

   In the output, vnic1 is configured with the IP address that you specified during zone configuration. vnic1 also has an automatically generated unique MAC address ether 2:8:20:54:f4:74 . Note that there are no entries for the system's network interfaces or for VNICs that are configured for other zones.

   If you do not have an entry for the VNIC associated with the zone, you need to plumb the VNIC. In particular, you will have these results if you chose not to perform initial VNIC configuration from the zone console. For instructions for plumbing the VNIC, refer to the appropriate step in How to Manually Configure the VNIC and Exclusive IP Zone.

9. Exit the current zone.

   Return to the global zone, where you can repeat the previous steps to confirm that all VNICs and zones are properly configured.

Next Steps

You can use various tools to observe network traffic and take statistics on zone usage.

• To verify that your network is properly configured, refer to Verifying Virtual Network Connectivity

• To observe traffic over the virtual network, refer to How to Verify Virtual Network Connectivity by Using the snoop Command.

• To obtain statistics for accounting purposes, refer to Gathering Usage Statistics for VNICs and Flows.

If you need to disassemble the virtual network, refer to How to Remove the Virtual Network Without Removing the Zones.

Complete Example for Creating a Virtual Network

This section contains a complete set of commands for configuring a virtual network.

Example 11–6 Basic Virtual Network

This example shows how to implement the virtual network scenario shown in Figure 10–1. The example elaborates on the tasks presented in Configuring a Basic Virtual Network. The commands do the following:

• Configure two VNICs, vnic1 and vnic2 on the data link e1000g0.

• Configure two exclusive IP zones, zone1 and zone2.

  ---

  Note –

  The example shows only the steps to configure zone1. Repeat the same steps to create and configure zone2.

  ---

• Assign automatically configured MAC addresses to each VNIC.

• Set two static IP addresses for the zones and VNICs, 192.168.3.20 and 192.168.3.22.

  ---

  Note –

  You must log in to the system's global zone as superuser or equivalent role to run the next commands.

  ---

# dladm show-phys
# dladm show-link
# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
         inet 127.0.0.1 netmask ff000000
e1000g0: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2
        inet 192.168.3.70 netmask ffffff00 broadcast 192.168.3.255
        ether 0:14:4f:94:d0:40

# dladm create-vnic -l e1000g0 vnic1
# dladm create-vnic -l e1000g0 vnic2
# dladm show-vnic
LINK        OVER             SPEED  MACADDRESS         MACADDRTYPE
vnic1       e1000g0      1000 Mbps  2:8:20:5f:84:ff    random
vnic2       e1000g0      1000 Mbps  2:8:20:54:f4:74    random

# zonecfg -z zone1
zonecfg:zone1> create
zonecfg:zone1> set zonepath=/export/home/zone1
zonecfg:zone1> set autoboot=true
zonecfg:zone1> set ip-type=exclusive
zonecfg:zone1> add net
zonecfg:zone1:net> set physical=vnic1
zonecfg:zone1:net> end
zonecfg:zone1> verify

zonecfg:zone1> info
zonename: zone1
zonepath: /export/home/zone1
brand: native
autoboot: true
.
.
net:
        address not specified
        physical: vnic1

zonecfg:zone1> commit
zonecfg:zone1> exit

# zoneadm -z zone1 verify
WARNING: /export/home/zone1 does not exist, so it could not be verified.
When 'zoneadm install' is run, 'install' will try to create
/export/home/zone1, and 'verify' will be tried again,
but the 'verify' may fail if:
the parent directory of /export/home/zone1 is group- or other-writable
or
/export/home/zone1 overlaps with any other installed zones.

# zoneadm -z zone1 install
Preparing to install zone <zone1>.
Creating list of files to copy from the global zone.
.
.
Zone <zone1> is initialized. 


zoneadm list -iv
  ID NAME             STATUS     PATH                           BRAND    IP
   0 global           running    /                              native   shared
   - zone1            installed  /export/home/zone1             native   excl

# zoneadm -z zone1 boot

# zoneadm list -v
  ID NAME             STATUS     PATH                           BRAND    IP
   0 global           running    /                              native   shared
   1 zone1            running    /export/home/zone1             native   excl

# zlogin zone1
# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
 inet6 ::1/128

# ifconfig vnic1 plumb
# ifconfig vnic1 192.168.3.20
# ifconfig vnic1 up

# ifconfig -a
.
vnic1: flags=201000842<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2
        inet 192.168.3.20 netmask ffffff00 broadcast 192.168.3.255
        ether 2:8:20:54:f4:74

# pwd
vnic1/
# cd root/etc
# vi hostname.vnic1
zone1-192.183.3.20

# vi /etc/inet/hosts
# Internet host table
#
::1     localhost
127.0.0.1         localhost
192.168.3.70      myhost     loghost
192.168.3.20      zone1-192-168-3-20

After you repeat the same steps to create zone2 and to assign vnic2 to zone2, the following example shows you how to verify that the two zones are properly configured with their respective VNICs.

# zoneadm list -v
  ID NAME             STATUS     PATH                           BRAND    IP
   0 global           running    /                              native   shared
   1 zone1            running    /export/home/zone1             native   excl
   2 zone2            running    /export/home/zone2             native   excl

# vi /etc/inet/hosts
# Internet host table
#
::1     localhost
127.0.0.1         localhost
192.168.3.70      myhost     loghost
192.168.3.20      zone1-192-168-3-20
192.168.3.22      zone2-192-168-3-22

How to Remove the Virtual Network Without Removing the Zones

The following procedure shows how to take down a virtual network while leaving its zones intact. The instructions refer to the virtual network that is configured in Configuring a Basic Virtual Network.

Use this procedure if you must do any of the following:

• Use the existing zones in a different configuration. For example, you might need to configure the zones as part of a private network. See Configuring a Private Virtual Network.

• Migrate the zones to another network.

• Move the zones to a different zone path.

• Clone the zones, as explained in Cloning a Non-Global Zone on the Same System in System Administration Guide: Virtualization Using the Solaris Operating System.

Before You Begin

This task assumes that you have a running virtual network that consists of exclusive IP zones.

1. On the system with the virtual network, become superuser or assume the equivalent root role in the global zone.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

2. Verify the state of the currently configured zones.

   # zoneadm list -v

   For example, you receive output similar to the following:

   ID NAME STATUS PATH BRAND IP 0 global running / native shared 1 zone1 running /export/home/zone1 native excl 2 zone2 running /export/home/zone2 native excl

3. Halt the exclusive IP zones of the virtual network.

   Issue the following command separately for each zone to be halted.

   # zoneadm -z zone-name halt

   Replace zone-name with the name of each zone.

   When you halt the zone, you remove the zone's application environment and terminate a number of system activities, as explained in Halting a Zone in System Administration Guide: Virtualization Using the Solaris Operating System.

4. Verify that the zones have been halted.

   # zoneadm list -iv

   You receive output similar to the following:

   ID NAME STATUS PATH BRAND IP 0 global running / native shared - zone1 installed /export/home/zone1 native excl - zone2 installed /export/home/zone2 native excl

   Note that the zones are no longer running, although they remain installed. To reboot a halted zone, refer to How to Boot a Zone in System Administration Guide: Virtualization Using the Solaris Operating System.

5. Review the state of the VNICs that were configured for the halted zones.

   # dladm show-vnic

   You receive output similar to the following:

   LINK OVER SPEED MACADDRESS MACADDRTYPE vnic1 e1000g0 1000 Mbps 2:8:20:5f:84:ff random vnic2 e1000g0 1000 Mbps 2:8:20:54:f4:74 random

   The resulting output shows that the VNICs are still configured as data links in the global zone. These VNICs were only plumbed and up in their associated exclusive IP zones, which are now halted. These VNICs are not plumbed in the global zones.

6. Delete the VNICs.

   # dladm delete-vnic vnic-link-name

   For example, you would type the following to delete the VNICs in the zones in Figure 10–1.

   # dladm delete-vnic vnic1 # dladm delete-vnic vnic1

Next Steps

You can perform further operations on the existing zones, as required.

• To restart the zones without reconfiguring them in a virtual network, refer to How to Boot a Zone in System Administration Guide: Virtualization Using the Solaris Operating System.

• To fully delete the zones, refer to How to Remove a Non-Global Zone in System Administration Guide: Virtualization Using the Solaris Operating System.

• To reconfigure the zones as part of a virtual network, create new VNICs and modify the zones, as described in Configuring a Basic Virtual Network.

Configuring a Private Virtual Network

The tasks in this section explain how to configure a private virtual network on a single system. If you need to isolate a software development environment from the external network, consider creating a private virtual network on a single host.

Private virtual networks are quite different from private virtual networks (VPNs). VPN software creates a secure point-to-point link between two endpoint systems. The private network configured by the tasks in this section is a virtual network on a box that cannot be accessed by external systems.

Pseudo-network interfaces called etherstubs are the building blocks of private virtual networks, as shown in Private Virtual Network on a Single System. You create VNICs over the etherstub, and then configure the containers over the VNICs. A firewall or similar network address translation (NAT) device translates the VNIC's private IP addresses to the routable IP address of the network interface. This enables the containers of the private network to send packets beyond the host without exposing the VNICs' private IP addresses to the external network.

How to Create Etherstubs and VNICs for the Private Virtual Network

This procedure uses exclusive IP zones as the containers for the private virtual network. Solaris IP Filter software performs NAT for outgoing packets from the private network.

Before You Begin

For the VNICs in the private network configuration, be sure to create private IP addresses that cannot be forwarded by the default router of the external network. However, for the network interface, use an IP address that is routable on the host's external network.

1. On the system where you create the private virtual network, become superuser or assume the equivalent root role.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

2. Create the etherstub for the private virtual network.

   # dladm create-etherstub etherstub-link-name

   For example, to create an etherstub called etherstub0, you would type the following:

   # dladm create-etherstub etherstub0

3. Verify that the etherstub was created.

   # dladm show-etherstub

   You should receive output similar to the following:

   LINK etherstub0

4. Create VNICs over the etherstub.

   # dladm create-vnic -l etherstub-link-name vnic-link-name

   For example, you might type the following:

   # dladm create-vnic -l etherstub0 vnic0

   Reserve one VNIC for the global zone. The global zone consists of all applications and services of a system's working environment that have not been delegated to a zone or virtual machine.

   Then, create at least two more VNICs for the exclusive IP zones of the private network. The virtual switch is automatically created with the first VNIC.

5. Verify that the VNICs are correctly created over the etherstub.

   # dladm show-link

   You should receive output similar to the following:

   LINK CLASS MTU STATE OVER e1000g0 phys 1500 up -- vnic0 vnic 9000 up etherstub0

   The “OVER” column contains the entry etherstub0 in the vnic0 row, indicating that vnic0 is created over etherstub0.

6. Create the exclusive IP zones.

   • For instructions, refer to How to Create an Exclusive IP Zone Over a VNIC.

   • Be sure to type the associated VNIC data link name for the zone in the set-physical parameter of add-net.

7. Install the zones.

   Use Steps 1–4 in the procedure How to Install the Exclusive IP Zone on a VNIC

   ---

   Note –

   Do not boot the zones at this time. You boot them as part of the next procedure,How to Configure Routing and Network Address Translation for the Private Virtual Network.

   ---

How to Configure Routing and Network Address Translation for the Private Virtual Network

Before You Begin

This procedure assumes that you have created the etherstub, VNICs, and exclusive IP zones or virtual machines for the private network, as described in How to Create Etherstubs and VNICs for the Private Virtual Network.

1. On the system where you create the private virtual network, become superuser or assume the equivalent root role.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

2. Check the status of the host's network interface.

   # ifconfig -a

   You should receive output similar to the following:

   lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 e1000g0: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2 inet 192.168.3.70 netmask ffffff00 broadcast 192.168.3.255 ether 0:14:4f:94:d0:40

   The interface, e1000g0 in this case, must be configured and plumbed before you can use it as part of the virtual network.

3. Assign an IP address to the VNIC that you reserved for the global zone.

   Make sure that all IP addresses that you assign to the VNICs on this host are private, reserved for use on this host only. Do not use the IP address prefix of the public network to which the network interface is connected as the network portion of the VNIC's IP address.

   For example, the ifconfig -a command above shows the IP address 192.168.3.70 for interface e1000g0. The output indicates that the interface is on local network 192.168.3.0/24. Therefore, do not assign the IP address 192.168.3.x to the VNIC. A safer choice might be 192.168.0.250, assuming that there is no 192.168.0.0/24 network that is known to the default router.

   For specific instructions on assigning the IP address to the VNIC, refer to Steps 5 through 7 of How to Create a Virtual Network Interface.

4. Check the status of routing protocols on the system.

   # routeadm

   You should receive output similar to the following:

   Configuration Current Current Option Configuration System State --------------------------------------------------------------- IPv4 routing enabled enabled IPv6 routing disabled disabled IPv4 forwarding disabled disabled IPv6 forwarding disabled disabled Routing services "route:default ripng:default"

   Note that routing is enabled but packet forwarding is disabled. You need to enable IPv4 forwarding in the global zone before you set up NAT or other rules through the IP Filter firewall.

5. Enable IP forwarding.

   # routeadm -u -e ipv4-forwarding

6. Create the basic packet filtering file /etc/ipf/ipnat.conf to provide network address translation.

   The next steps use Solaris IP Filter to perform NAT for outgoing packets originated from inside the private network. For an introduction to IP Filter, refer to Chapter 24, Solaris IP Filter (Overview), in System Administration Guide: IP Services

   # cd /etc/ipf # vi ipnat.conf map e1000g0 192.168.0.0/24 -> 0/32 portmap tcp/udp auto map e1000g0 192.168.0.0/24 -> 0/32

   This rule set tells the IP Filter software how to translate the IP addresses of outgoing packets when they arrive at interface e1000g0. Any TCP and UDP packets that arrive from private network 192.168.0.0/24 have their IP addresses translated to the address of the global zone before exiting the system. The global zone has the same IP address as network interface e1000g0, 192.168.3.70. This interface is connected to external network 192.168.3.0/24, which is known to the network's default router.

   The rule set above implements a simple NAT scenario, but you can also add packet filtering rules to /etc/ipf/ipnat.conf, if required. For more information, see Configuring Solaris IP Filter in System Administration Guide: IP Services.

7. Start IP Filter and verify that the rules in /etc/ipf/ipnat.conf are active.

   # svcadm enable network/ipfilter # ipnat -l List of active MAP/Redirect filters: map e1000g0 192.168.0.0/24 -> 0.0.0.0/32 portmap tcp/udp auto map e1000g0 192.168.0.0/24 -> 0.0.0.0/32 List of active sessions:

8. Boot an already-installed exclusive IP zone.

   # zoneadm -z zone-name boot

   Repeat this step for all zones to be part of the private virtual network.

9. Log in to each exclusive IP zone and plumb its associated VNIC.

   # zlogin zone-name # ifconfig vnic-link-name plumb #ifconfig vnic-link-name vnic-IP-address # ifconfig vnic-link-name up

10. Exit the final zone that you configured and return to the global zone.

11. Add entries for all VNICs in the /etc/inet/hosts file, as shown in How to Manually Configure the VNIC and Exclusive IP Zone.

12. Edit the /etc/hostname/vnic-name files, as shown in How to Manually Configure the VNIC and Exclusive IP Zone.

Example 11–7 Private Virtual Network Configuration

The following example shows the commands to implement the private virtual network that is shown in Figure 10–2.

To use the commands, you must first log in to the system's global zone as superuser or equivalent role.

# dladm create-etherstub etherstub0
# dladm show-etherstub
LINK
etherstub0
# dladm create-vnic -l etherstub0 vnic0
# dladm create-vnic -l etherstub0 vnic1
# dladm create-vnic -l etherstub0 vnic2

# dladm show-vnic
LINK        OVER             SPEED  MACADDRESS         MACADDRTYPE
vnic0       etherstub0      0 Mbps  2:8:20:c2:39:38    random
vnic1       etherstub0      0 Mbps  2:8:20:45:8f:c9    random
vnic2       etherstub0      0 Mbps  2:8:20:6b:8:ab     random

# dladm show-link
LINK        CLASS    MTU    STATE    OVER
e1000g0     phys     1500   up       --
vnic0       vnic     9000   up       etherstub0
vnic1       vnic     9000   up       etherstub0
vnic2       vnic     9000   up       etherstub0

At this stage, you configure exclusive IP zones over VNICs, configure them, and assign IP addresses to them, as explained in Configuring a Basic Virtual Network.

# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g0:flags=201100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4,CoS>mtu 1500 index 2
        inet 192.168.3.70 netmask ffffff00 broadcast 192.168.3.255
        ether 0:14:4f:94:d0:40
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
        inet6 ::1/128

# ifconfig vnic0 plumb
# ifconfig vnic0 192.168.0.250
# ifconfig vnic0 up

# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g0:flags=201100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4,CoS>mtu 1500 index 2
        inet 192.168.3.70 netmask ffffff00 broadcast 192.168.3.255
        ether 0:14:4f:94:d0:40
vnic0: flags=201100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4,CoS> mtu 9000 index 5
        inet 192.168.0.250 netmask ffffff00 broadcast 192.168.0.255
        ether 2:8:20:c2:39:38
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
        inet6 ::1/128

# routeadm
              Configuration   Current              Current
                     Option   Configuration        System State
---------------------------------------------------------------
               IPv4 routing   enabled              enabled
               IPv6 routing   disabled             disabled
            IPv4 forwarding   disabled             disabled
            IPv6 forwarding   disabled             disabled

           Routing services   "route:default ripng:default"

# routeadm -u -e ipv4-forwarding

# cd /etc/ipf
# vi ipnat.conf
map e1000g0 192.168.0.0/24 -> 0/32  portmap tcp/udp auto
map e1000g0 192.168.0.0/24 -> 0/32
# svcadm enable network/ipfilter

# zoneadm -z zone1 boot
# zoneadm -z zone2 boot

Next Steps

• Test the connectivity of the private network by using the various observability tasks in Chapter 12, Administering Virtual Networks and Resource Controls (Tasks).

• Create a firewall that filters outgoing packets from the private network. For more information, see Configuring Solaris IP Filter in System Administration Guide: IP Services.

Chapter 12 Administering Virtual Networks and Resource Controls (Tasks)

This chapter contains monitoring and statistics-gathering tasks for a system that has configured interface-based flow control, or a virtual network, or both. The connectivity monitoring tasks use standard network tools to observe and evaluate activity on the virtual network. The tasks for gathering usage statistics on packet flows and, if applicable, on VNICs, use the dladm, flowadm, and acctadm commands. These last tasks are especially useful for provisioning, consolidation, and billing purposes. The following subjects are discussed:

• Monitoring and Statistics–Gathering Task Map

• Verifying Virtual Network Connectivity

• Observing Traffic on Virtual Networks

• Gathering Usage Statistics for VNICs and Flows

Monitoring and Statistics–Gathering Task Map

Verifying Virtual Network Connectivity

You can use standard network tools to verify your virtual network's connectivity. This section contains simple tasks to help you verify that the VNICs of your virtual network are correctly configured and have the expected network connectivity. Following is a list of the tools used in the tasks, along with links to their man pages.

• dladm(1M)

• ifconfig(1M)

• netstat(1M)

• ping(1M)

How to Verify the VNIC Configuration for the Global Zone

Before You Begin

The following task assumes that you have created a VNIC for the global zone of your system.

1. On the system with the virtual network, become superuser or assume the equivalent root role.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

2. Verify the state of the data links on the system.

   # dladm show-link

   Your output should resemble either of the following:

   • For a system that has a publicly accessible virtual network, such as the network that is configured in How to Create a Virtual Network Interface:

     # dladm show-link LINK CLASS MTU STATE OVER bge0 phys 1500 up bge0 vnic0 vnic 9000 up bge10

     In this output, both the physical network interface bge0 and the VNIC pseudo-interface vnic0 are configured as data links.

   • For a system with a private virtual network that cannot be accessed by external users, such as the network that is configured in How to Create Etherstubs and VNICs for the Private Virtual Network:

     # dladm show-link LINK CLASS MTU STATE OVER e1000g2 phys 1500 unknown -- e1000g0 phys 1500 up -- vnic0 vnic 9000 up etherstub0 vnic1 vnic 9000 up etherstub0

     The network interface e1000g0 is configured as a data link. The presence of etherstub0 indicates this is a private network. Two VNICs, vnic0 and vnic1, are successfully configured over the etherstub.

3. Verify that the VNIC is plumbed and running on the IP level of the TCP/IP protocol stack:

   # ifconfig -a

   You should receive output similar to the following:

   lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 bge0: flags=1000843 <UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 192.168.8.50 netmask ffffff00 broadcast 192.168.8.255 ether 8:0:20:c8:f4:1d vnic0: flags=201000842<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2 inet 192.168.8.10 netmask ffffff00 broadcast 192.168.8.255 ether 2:8:20:54:f4:74

   Both the network interface bge0 and the VNIC vnic0 are plumbed and up.

How to Verify Configuration of a Virtual Network of Exclusive IP Zones

Before You Begin

The procedure assumes that you have created at least two VNICs and corresponding exclusive IP zones to form a virtual network. You also have configured and plumbed these VNICs while logged into their respective zones. The next task verifies the configuration of the virtual network created in Basic Virtual Network on a Single System.

1. On the system where you create the virtual network, become superuser or assume the equivalent root role in the global zone.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

2. Ensure that the VNICs are configured as data links in the global zone.

   # dladm show-vnic

   You should receive output similar to the following:

   LINK OVER SPEED MACADDRESS MACADDRTYPE vnic1 e1000g0 1000 Mbps 2:8:20:5f:84:ff random vnic2 e1000g0 1000 Mbps 2:8:20:54:f4:74 random

   In this example, both VNICs of the virtual network are configured as data links over network interface e1000g0.

3. Verify that any interfaces known to the global zone are plumbed and up.

   # ifconfig -a lo0: flags=2001000849<UP,UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 e1000g0: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2 inet 192.168.3.70 netmask ffffff00 broadcast 192.168.83.255 ether 0:14:4f:94:d0:40

   Only the network interface e1000g0 is plumbed for the global zone. This interface has the IP address 192.168.3.70 and connects the system to the external 192.168.3.0/24 network. For the virtual network configuration, ifconfig -a in the global zone should not report any VNICs.

4. Check the state of the configured zones.

   # zoneadm list -v ID NAME STATUS PATH BRAND IP 0 global running / native shared 5 zone2 running /export/home/zone2 native excl 7 zone1 running /export/home/zone1 native excl

   The STATUS column indicates that the zones are up and running. If the status of the zones indicates a condition other than “running,” you need to reboot the zone. For instructions, refer to Chapter 20, Installing, Booting, Halting, Uninstalling, and Cloning Non-Global Zones (Tasks), in System Administration Guide: Virtualization Using the Solaris Operating System.

5. Check the global zone's known routes.

   # netstat -rn

   You should receive output similar to the following:

   Routing Table: IPv4 Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ---------- --------- default 192.168.3.1 UG 1 8 e1000g0 192.168.3.0 192.168.3.70 U 1 143 e1000g0 127.0.0.1 127.0.0.1 UH 1 13 lo0 Routing Table: IPv6 Destination/Mask Gateway Flags Ref Use If --------------------------- --------------------------- ----- --- ------- ----- ::1 ::1 UH 1 22 lo0

   The global zone's default route to external networks is through the gateway 192.168.3.1. This is the IP address of the default router for network 192.168.3.0/24. The global zone also reports that the route to the gateway is through 192.168.3.70, the IP address of the system's e1000g0 interface.

6. Log in to one of the zones of the virtual network, for example, zone1, and ensure that the zone's VNIC is plumbed and up.

   # zlogin zone1 # ifconfig -a vnic1 vnic1: flags=201000842<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2 inet 192.168.3.20 netmask ffffff00 broadcast 192.168.3.255 ether 2:8:20:54:f4:74

7. Check the known routes between the local zone and the external network.

   # netstat -rn

   You should receive output similar to the following:

   Routing Table: IPv4 Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ---------- --------- default 192.168.3.1 UG 1 0 vnic1 192.168.3.0 192.168.3.20 U 1 2 vnic1 127.0.0.1 127.0.0.1 UH 1 23 lo0

   The output verifies that the default route for zone1 is to the default router, 192.168.3.1. zone1 also knows to route packets through vnic1, 192.168.3.20. This traffic is then passed to the global zone, where the packets travel through the network interface e1000g0.

8. Verify the VNICs' connectivity.

   Perform these steps while logged into a local zone. The following steps assume that you are logged into zone1.

   1. Check the connectivity between the local zone's VNIC and the system's network interface.

      # ping network-interface-address

      For example, check that vnic1 can pass traffic to network interface e1000g0, IP address 192.168.3.70.

      # ping 192.168.3.70 192.168.3.70 is alive

   2. Check that the VNIC can pass traffic through the default router, IP address 192.168.3.1.

      # ping 192.168.3.1 192.168.3.1 is alive

   3. Check that the VNIC can pass traffic to another VNIC in the virtual network.

      # ping vnic-IP-address

      For example, to check that vnic1 can pass traffic to vnic2 (IP address192.168.3.22), run the following command.

      # ping 192.168.3.22 192.168.3.22 is alive

Next Steps

• If Steps 5–7 complete successfully in one exclusive IP zone, then repeat them for each exclusive IP zone in the virtual network.

• To observe packet flows and take statistics, go on to the procedure Observing Traffic on Virtual Networks.

Observing Traffic on Virtual Networks

Use the standard snoop command to observe and analyze the status of the virtual network. snoop gathers packets and displays their output, enabling you to observe and analyze their content. You can use snoop output to verify the connectivity by observing the “conversation” among the VNICs on a virtual network. For full details on snoop usage, refer to the snoop(1M) man page.

How to Verify Virtual Network Connectivity by Using the snoop Command

The following task observes traffic on the private network configured in Example 11–7. However, you can use snoop to observe traffic over a publicly-accessible virtual network, as well.

1. On the system where you create the private virtual network, become superuser or assume the equivalent root role in the global zone.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

2. Gather information about network traffic on the private virtual network.

   # snoop -d etherstub0

   By “snooping” on the private network's etherstub, you can obtain information about activities on all the VNICs configured over that etherstub. You can also snoop the individual VNICs.

3. Check the snoop output to verify connectivity among the VNICs of the etherstub.

   Using device etherstub0 (promiscuous mode) 192.168.0.250 -> 192.168.0.200 RIP R (10 destinations) 192.168.0.250 -> 192.168.0.200 RIP R (10 destinations) 192.168.0.250 -> 224.0.0.1 ICMP Router advertisement (Lifetime 1800s [1]: {192.168.0.250 0}) 192.168.0.250 -> 192.168.0.200 RIP R (10 destinations) 192.168.0.250 -> 192.168.0.200 RIP R (10 destinations) 192.168.0.220 -> (broadcast) ARP C Who is 192.168.0.250, 192.168.0.250 ? 192.168.0.200-> (broadcast) ARP C Who is 192.168.0.220, 192.168.0.220 ? 192.168.0.250 -> (broadcast) ARP C Who is 192.168.0.200, 192.168.0.200 ? 192.168.0.200 -> 192.168.0.220 ARP R 192.168.0.200, 192.168.0.200 is 2:8:20:45:8f:c9 192.168.0.250 -> 192.168.0.200 ICMP Echo request (ID: 20291 Sequence number: 0) 192.168.0.200 -> (broadcast) ARP C Who is 192.168.0.250, 192.168.0.250 ? 192.168.0.250 -> 192.168.0.200 ARP R 192.168.0.250, 192.168.0.250 is 2:8:20:c2:39:38 192.168.0.200 -> 192.168.0.250 ICMP Echo reply (ID: 20291 Sequence number: 0) 192.168.0.250 -> 192.168.0.250 RIP R (10 destinations)

   This output shows the contents of packets that are exchanged among the three VNICs in Figure 10–2 as they contact each other.

   • etherstub0, over vnic0 (IP address 192.168.0.250) sends out RIP routing protocol packets to vnic1 (192.168.0.200).

   • vnic2 (192.168.0.220) sends out an ARP broadcast message (“Who is”), to vnic0 (192.168.0.250). Then, vnic0 sends out an ARP request to vnic1:

     192.168.0.220 -> (broadcast) ARP C Who is 192.168.0.250, 192.168.0.250 ? 192.168.0.250 -> (broadcast) ARP C Who is 192.168.0.200, 192.168.0.200 ?

   • vnic1 (192.168.0.200) sends out an ARP broadcast message (“Who is”) to vnic2 (192.168.0.220).

   • Eventually vnic1 responds to the vnic2's “Who is” broadcast by sending its MAC address, as follows:

     192.168.0.200 -> 192.168.0.220 ARP R 192.168.0.200, 192.168.0.200 is 2:8:20:45:8f:c9

     This response proves that the two VNICs of the virtual network, vnic1 and vnic2, can send packet traffic to each other.

   • Then vnic0 responds to vnic1's ARP request with its MAC address:

     192.168.0.250 -> 192.168.0.200 ARP R 192.168.0.250, 192.168.0.250 is 2:8:20:c2:39:38

     This response proves that etherstub0 on vnic0 and vnic1 on the virtual network can send traffic to each other.

   The previous output is but a sample of the many ways to use snoop for diagnostic purposes. For more information, refer to the snoop(1M) man page.

Gathering Usage Statistics for VNICs and Flows

Network virtualization and resource control introduces tools for observing traffic on a per-interface or per-flow basis. The next tasks show how to use options of the dladm and flowadm commands to obtain statistics on packet traffic. For full details on each command, refer to the following man pages:

• dladm(1M)

• flowadm(1M)

How to Obtain Statistics on VNICs

VNIC statistics are useful for provisioning purposes, such as determining whether a system needs additional VNICs or possibly additional interfaces. You can also use VNIC traffic statistics to determine how well network consolidation onto a virtual network is working.

Before You Begin

The task assumes that you have a working virtual network, such as the network configured in Configuring a Basic Virtual Network.

1. On the system where you create the virtual network, become superuser or assume the equivalent root role in the global zone.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

2. Observe traffic flow over network interfaces

   Before checking the flow usage on individual VNICs, you might want to view overall usage of the VNIC's underlying interface.

   where link-name is the name of a currently plumbed interface, for example internal0.

   # dladm show-link -s -i 5 internal0 LINK IPACKETS RBYTES IERRORS OPACKETS OBYTES OERRORS internal0 5315127 400738164 0 169526 29260024 0 LINK IPACKETS RBYTES IERRORS OPACKETS OBYTES OERRORS internal0 1 60 0 2 340 0 LINK IPACKETS RBYTES IERRORS OPACKETS OBYTES OERRORS internal0 17 1020 0 4 456 0 ^C

   To halt the display, press Ctrl-C.

3. Observe traffic flow over configured VNICs.

   Use the following syntax for each VNIC in the virtual network. You must run this command in the global zone for all VNICs configured on the system.

   # dladm show-link -s -i 5 vnic-link-name

   where vnic-name is the name of the VNIC whose traffic you want to observe.

   You should receive output similar to the following:

   # dladm show-link -s -i 5 vnic0 ipackets rbytes ierrors opackets obytes oerrors vnic0 537001 48104701 0 5 210 0 ipackets rbytes ierrors opackets obytes oerrors vnic0 3 270 0 0 0 0 ^C

   The output indicates that vnic0 has had both incoming packet (ipackets) and outgoing packet (opackets) traffic.

   To halt the display, press Ctrl-C.

How to Obtain Statistics on Flows

Flow statistics help you evaluate packet traffic on your network before assigning bandwidth and priorities to already configured flows. Like VNIC statistical information, flow statistics are also useful for provisioning and, possibly, for billing purposes.

Before You Begin

This procedure assumes that you have configured flows, as described in Chapter 13, Configuring Resource Management on an Interface.

1. On the system where you configure flow control, become superuser or assume the equivalent root role in the global zone.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

2. Observe packet flow statistics for a system that is configured with interface-based flow control.

   The following example shows output for a system with two network interfaces that uses traditional bandwidth control. The system does not have a virtual network configured.

   # flowadm show-flow -s FLOW IPACKETS RBYTES IERRORS OPACKETS OBYTES OERRORS net20 0 0 0 1723 72366 0 httpsflow 0 0 0 32932 3225851 0 httpflow 29 3982 0 42 20799 0

   where

   net20

   is a flow for all UDP traffic across interface 10.10.3.20/24 on the internal network.

   httpsflow

   is a flow for all secure HTTP traffic over interface 192.168.3.25 , which is connected to the external network

   httpflow

   is a flow for all HTTP traffic over interface 192.168.3.25.

3. To halt the display, press Ctrl-C.

Setting Up Accounting for Packet Flows

You can use the extended accounting feature to capture these statistics into a file, which can then be printed for usage reports.

How to Configure Flow Accounting

Use the flowadm command and extended accounting feature of the Solaris OS to collect statistical data on flow usage. In this manner, you can maintain records of flow traffic for tracking, provisioning, or billing purposes. The following task shows how to gather statistics for a system on a traditional network, which has implemented flow control on the system's interfaces. For an example, refer to Figure 10–3. For the tasks that implement flow control on this network, see How to Set Up and Configure Flow Control for a System on a Traditional Network.

Before You Begin

You must have configured flows on the system's interfaces, as explained in Interface-Based Flow Control for Traditional Networks.

1. Assume the Primary Administrator role or become superuser on the system with the interfaces whose usage you want to track.

   The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration.

2. On the system with the interfaces whose usage you want to track, become superuser or assume the equivalent root role.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

3. View the system's currently configured flows.

   # flowadm show-flow NAME LINK ATTR VALUE app-flow internal0 ip_version 4 local_ip 10.10.12.45/ httpsflow nge0 transport tcp local_port 443 httpflow nge0 transport tcp local_port 80

   The output shows three flows. app-flow is a flow configured on interface internal0. This interface isolates traffic coming from the application server on the network, by using the IP address 10.10.12.45/ of the server as the key. The flows httpflow and httpsflow are configured on interface nge0. These flows isolate HTTP and secure HTTP packets by using their port numbers, 80 and 443, respectively.

4. View usage statistics for the system's three flows.

   # flowadm show-flow -s FLOW IPACKETS RBYTES IERRORS OPACKETS OBYTES OERRORS app-flow 65 0 0 1723 72366 0 httpsflow 0 0 0 32932 3225851 0 httpflow 29 3982 0 42 20799 0

   The -soption of flowadm gathers usage statistics at the point when the command is issued.

5. View the accounting options that are available from extended accounting.

   # acctadm Task accounting: inactive Task accounting file: none Tracked task resources: none Untracked task resources: extended Process accounting: inactive Process accounting file: none Tracked process resources: none Untracked process resources: extended,host Flow accounting: inactive Flow accounting file: none Tracked flow resources: none Untracked flow resources: extended Network accounting: inactive Network accounting file: none Tracked Network resources: none Untracked Network resources: extended

   The last four options implement extended accounting for flows. The previous output verifies that network accounting is not turned on and an accounting file is unknown.

6. Confirm that an extended accounting file does not already exist.

   # cd /var/adm/exacct ; ls

   If no accounting file is listed, this confirms that you can now create the file.

7. Create an extended accounting file for the flows on the system.

   Use the syntax:

   # acctadm -e extended -f /var/adm/exacct/file-name net

   The net argument turns on extended accounting for network flows.

   Suppose you wanted to track usage of the flows you have defined for a system. You would type the following:

   # acctadm -e extended -f /var/adm/exacct/httpflow net

8. Verify that extended accounting is now turned on.

   # acctadm Task accounting: inactive . . Network accounting: active Network accounting file: /var/adm/exacct/httpflow Tracked Network resources: extended Untracked Network resources: none

9. Show usage of all the system's flows.

   # flowadm show-usage -f /var/adm/exacct/httpflow Bytes Packets Errors Duration Bandwidth Link/Flow ___________________________________________________________________________ 68912 76 0 140 3.94 Kbps httpflow 3449 32 0 140 197.09 bps httpsflow 0 0 0 140 0.00 bps app-flow

10. Show the contents of the extended accounting file that have to do with a particular flow.

    Use the following syntax:

    # flowadm show-usage -f /var/adm/exacct/filename flow

    For example, use the following command to obtain data on HTTP usage on the system.

    # flowadm show-usage -f /var/adm/exacct/httpflow httpflow 14:35:51 14:36:11 0 0 0.00 bps httpflow 14:36:11 14:36:31 0 0 0.00 bps httpflow 14:36:31 14:36:51 3789 64895 27.47 Kbps httpflow 14:36:51 14:37:11 120 108 91.20 bps httpflow 14:37:11 14:37:31 0 0 0.00 bps httpflow

    This output gathers statistics on httpflow since the command was issued.

11. Obtain a count of all packet flows on the system.

    Gathering statistics on interface usage can indicate over time whether you should configure a virtual network on the interface or possibly add another interface.

    # dladm show-usage -f /var/adm/exacct/httpflow Bytes Packets Errors Duration Bandwidth Link/Flow ___________________________________________________________________________ 159080 1188 0 400 3.18 Kbps nge0 1600 17 0 400 32.00 bps internal0

    The output shows packet count for the nge0 interface, which has flows for HTTP and HTTPS traffic, as well as unfiltered traffic. The internal0 interface has a flow for traffic received from IP address 10.10.12.45, the address of the application server.

12. Obtain a count of all packet traffic for a specific interface.

    # dladm show-usage -f /var/adm/exacct/httpflow nge0 Start Time End Time In Bytes Out Bytes Bandwidth Device ___________________________________________________________________________ 14:34:51 14:35:11 2512 0 1.00 Kbps nge0 14:35:11 14:35:31 986 0 394.40 bps nge0 14:35:31 14:35:51 2108 0 843.20 bps nge0 14:35:51 14:36:11 2632 0 1.05 Kbps nge0 14:36:11 14:36:31 1653 91 697.60 bps nge0 14:36:31 14:36:51 5237 64937 28.07 Kbps nge0 14:36:51 14:37:11 8902 3424 4.93 Kbps nge0 14:37:11 14:37:31 9648 4958 5.84 Kbps nge0 14:37:31 14:37:51 1766 0 706.40 bps nge0 14:37:51 14:38:11 3334 0 1.33 Kbps nge0 14:38:11 14:38:31 1834 578 964.80 bps nge0

Chapter 13 Configuring Resource Management on an Interface

This chapter explains how to set up resource management on a per-interface basis. This capability improves system and network performance for many types of network configurations, from corporate intranetworks, to local area networks, to virtual networks on a single system.

The chapter covers the following topics:

• How Resource Management Works

• Interface-Based Flow Control for Traditional Networks

Resource Management Task Map

How Resource Management Works

With interface-based resource management, you can isolate, prioritize, track, and control data traffic on an individual system. These features also enable you to improve the efficiency and performance of a network. Isolating types of data traffic is especially helpful for network provisioning, establishing service level agreements, billing clients, and diagnosing security problems. The concepts in this section pertain to traffic on an internal virtual network as well to traffic on systems configured in traditional external networks.

Resource control helps to isolate processes to improve a system's efficiency and to make the processes easier to observe and track for accounting purposes. Configuring resource control involves organizing packet traffic on the interface into flows that have the same characteristics. These characteristics are derived from the information contained in the fields of an individual packet's header. Therefore, you can organize packet traffic into flows by one of the following characteristics:

• IP address

• Transport protocol name (UDP, TCP, or SCTP)

• Application port number, for example, port 21 for FTP

• DS field attribute, which is used for quality-of-service in IPv6 packets only. (For more information about the DS field, refer to DS Codepoint in System Administration Guide: IP Services.)

Note that a flow can be based only on one of the previously listed characteristics.

For example, you can create a flow for only FTP packets or only for all packets received from a particular source IP address. You cannot create a flow for packets from port number 21 (FTP) that come only from a specified IP address. Or you cannot create a flow for all traffic from IP address 192.168.1.10, and then create flows for transport layer traffic on 192.168.1.10.

Bandwidth management involves assigning a portion of the interface's bandwidth to each flow. Modern network interfaces, such as GLD.v3 interfaces e1000g, bge, nge, and others, have large amounts of bandwidth available for assignment to flows. When you create a flow, you can allocate bandwidth to it and then give the flow a relative priority among all flows on the interface. Furthermore, if the system has processor sets, you can assign a CPU processor set to a flow.

The resulting set of rules that define the characteristics of all flows on a system make up the system's flow control policy. You implement these rules by using the flowadm command and its set-flowprop subcommand. For complete technical information, refer to the flowadm(1M) man page.

This chapter uses the term flow control to generically refer to both resource control and bandwidth management, unless the text specifically states otherwise.

Interface-Based Flow Control for Traditional Networks

This section shows how to set up flow control for a heavily used system on a traditional network. A proxy server in a small network is used as an example. However, the same generic procedures apply for configuring flow control on the interface of any system on your network.

The steps use the scenario shown in Interface-based Resource Control for a Traditional Network. This topology is typical of the networks in use at colleges or small businesses that host their own services and do not outsource their applications. The scenario is illustrated in Figure 10–3.

During the next task, you create a flow control policy to control traffic over both of the proxy server's interfaces. The task shows how to improve the proxy server's efficiency on its public side by doing the following for traffic over DMZ0:

• Creating two flows to isolate web traffic, one flow for HTTP packets and a second flow for secure HTTPS packets.

• Assigning a specific amount of the interface's bandwidth to each flow

• Prioritizing the two flows in comparison to other types of packet traffic.

The task then shows how to improve proxy server performance on the internal network by doing the following for traffic over internal1:

• Creating separate flows for the application server, database server, and backup server.

• Using the IP address of each server as the identifier for packet traffic from the proxy to the application server.

• Not creating flows for traffic from the individual systems on subnet 10.10.12.0/24

You use the flowadm and dladm commands throughout the procedure. For detailed technical information about these commands, refer to the dladm(1M) and the flowadm(1M) man pages.

How to Set Up and Configure Flow Control for a System on a Traditional Network

Before You Begin

This procedure assumes that you are using a system with at least two interfaces. However, you can use the flowadm command as shown in the next steps to configure flow control for a single-interface host.

The procedure assumes that the names of the interface nge0 and nge1 have already been changed to DMZ0 andinternal0, respectively.

You do not have to change the interface device names to use the steps in this procedure. However, many sites might prefer to create their own link names to provide more clarity for their configurations. To change the device name of an interface, refer to How to Rename a Data Link.

1. On the system where you set up flow control, become superuser or assume the equivalent root role.

   To create and assign the root role, see How to Make root User Into a Role in System Administration Guide: Security Services.

2. Check the status of the links.

   # dladm show-link LINK CLASS MTU STATE OVER internal0 phys 1500 up -- e1000g0 phys 1500 unknown -- e1000g1 phys 1500 unknown -- DMZ0 phys 1500 up --

   Note that the nge interfaces are now listed by their new link names.

3. Verify that the interfaces are plumbed and up on the IP layer.

   # ifconfig -a lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 DMZ0: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2 inet 10.10.6.5 netmask ff000000 broadcast 10.255.255.255 ether 0:14:4f:94:d0:60 internal0: flags=201000849<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2 inet 10.10.12.42 netmask ffffff00 broadcast 10.10.10.255 ether 0:10:20:30:40:aa

   The output verifies that the interfaces are plumbed and listed by their link names.

4. Create a flow to isolate traffic by the port number of the application.

   Specify the port number and the associated transport layer protocol of the application to the flowadm command. Use the following syntax:

   # flowadm add-flow -l link-name -a transport=name,port-number flow-name

   The -l option is followed by the link name. The -a option is followed by the attributes of the flow that you want to configure. Use the following syntax to specify the attributes of a flow that is defined by the port number of the application:

   transport=name

   Name of the appropriate transport layer protocol that is used in conjunction with the application's port number. Possible values include TCP, UDP, SCTP, ICMP, and ICMPv6.

   local_port | remote_port=port-number

   Port number of the application whose packets you want to isolate into a flow. You also must indicate whether the packets are flowing through the system's local port or arriving from a remote system's port . To find out the port number of an application, consult the /etc/services file, as explained in the services(4) man page.

   flow-name

   Name that you create for the flow.

   For example, use the following commands to create flows for the DMZ0 link that is shown in Figure 10–3:

   # flowadm add-flow -l DMZ0 -a transport=tcp,local_port=80 httpflow # flowadm add-flow -l DMZ0 -a transport=tcp,local_port=443 httpsflow

   The flow httpflow isolates traffic for the HTTP application, which runs over the standard port 80 on the proxy server. The flow httpsflow isolates traffic for the HTTPS application, which runs over the standard port 443.

5. Verify the status of the flows on a link.

   Use the following syntax for the show-flow subcommand of flowadm:

   flowadm show-flow -l link-name

   The next example shows how to display the status of the DMZ0 link.

   # flowadm show-flow -l DMZ0 NAME LINK ATTR VALUE httpsflow DMZ0 ip_version 4 transport tcp local_port 443 httpflow DMZ0 ip_version 4 transport tcp local_port 80

6. Add bandwidth controls to a flow.

   Use the following syntax of the show-flowprop subcommand of flowadm:

   flowadm set-flowprop -p flow-properties flow-name

   You can specify any of the following for flow-properties:

   maxbw

   Maximum amount of bandwidth that packets in this flow can use on the link.

   priority

   Amount of priority to give to packets in this flow, in relation to other flows on the link. The possible values are high, normal, or low.

   cpus

   Allocate packets of the flow to a processor set, for systems that have multiple processor sets.

   For example, for the flows on the DMZ0 link, you might set the following bandwidths:

   # flowadm set-flowprop -p maxbw=100 httpflow # flowadm set-flowprop -p maxbw=100 httpsflow

   These flows set a bandwidth limit on both the open and secure HTTP services on link DMZ0 of the proxy server shown in Figure 10–3.

7. Create flows to isolate traffic from a host by IP address.

   Use the following syntax:

   flowadm add-flow -l link-name -a local_ip | remote_ip=IP address flow-name

   For example, in Figure 10–3, the proxy server acts as a proxy for three servers on internal network 10.10.12.0/24. This system also forwards packets for users on network 10.10.12.0/24. To create flows to separate the traffic from the three servers, you would do the following:

   flowadm add-flow -l internal0 -a local_ip=10.10.12.45 app-flow flowadm add-flow -l internal0 -a local_ip=10.10.12.46 db-flow flowadm add-flow -l internal0 -a local_ip=10.10.12.47 backup-flow

   You do not need to create a separate flow for user packet traffic.

8. Set priorities for a flow.

   When you set priorities, flows are assigned importance in comparison to each other. The three priority settings are high, normal, and low. You use the set-flowprop subcommand of flowadm, as shown in Step 7, to set flow priority.

   For example, you might set priorities for the flows from the three servers on network 10.10.12.0/24 in Figure 10–3 as follows:

   • Give the application server a medium amount of bandwidth and high priority.

   • Give the database server slightly less bandwidth and high priority.

   • Give the backup server a narrow bandwidth and low priority.

   On a system with processor sets, you can also assign cpus to flows, as shown in the syntax in Step 7. You can isolate a flow further by assigning to it a specified amount of cpus. For example, the proxy server in Figure 10–3 has 16 processor sets, which you can assign to particular flows.

   Here are the set-flowprop policies for the internal servers shown in Figure 10–3

   # flowadm set-flowprop -p maxbw=100,priority=high,cpus=2 app-flow # flowadm set-flowprop -p maxbw=100,priority=high,cpus=3 db-flow # flowadm set-flowprop -p maxbw=10,priority=low,cpus=4 backup-flow

9. Verify that the flows now exist on the system's interfaces.

   # flowadm show-flow NAME LINK ATTR VALUE app-flow internal0 ip_version 4 local_ip 10.10.12.45/ db-flow internal0 ip_version 4 local_ip 10.10.12.46/ backup-flow internal0 ip_version 4 local_ip 10.10.12.47/ httpsflow DMZ0 ip_version 4 transport tcp local_port 443 httpflow DMZ0 ip_version 4 transport tcp local_port 80

Example 13–1 Setting Up Traditional Flow Control on an Interface

This example contains the steps for setting flow control on the interfaces of the proxy server that is shown in Interface-based Resource Control for a Traditional Network. Five flows are created with the following characteristics:

httpflow

Created on link DMZ0 over interface nge0. This flow contains HTTP packets, which flow through the local port 80.

httpsflow

Created on link DMZ0 over interface nge0. This flow contains secure HTTP packets, which flow through the local port 443.

app-flow

Created on link internal0 over interface nge1. This flow contains all traffic from IP address 10.10.12.45, which is the address of an application server on the network.

db-flow

Created on link internal0 over interface nge1. This flow contains all traffic from IP address 10.10.12.46, which is the address of a database server on the network.

backup-flow

Created on link internal0 over interface nge1. This flow contains all traffic from IP address 10.10.12.47, which is the address of a backup server on the network.

# dladm show-phys
LINK         MEDIA                STATE      SPEED DUPLEX   DEVICE
nge1         Ethernet             up         1000 full      nge1
e1000g0      n                    unknown    0    half      e1000g0
e1000g1      n                    unknown    0    half      e1000g1
nge0         Ethernet             up         1000 full      nge0
# dladm show-link
LINK        CLASS    MTU    STATE    OVER
internal0   phys     1500   up       --
e1000g0     phys     1500   unknown  --
e1000g1     phys     1500   unknown  --
DMZ0        phys     1500   up       --
#ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
         inet 127.0.0.1 netmask ff000000
DMZ0: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2
        inet 10.10.6.5 netmask ff000000 broadcast 10.255.255.255
        ether 0:14:4f:94:d0:60
internal0: flags=201000849<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2
        inet 10.10.12.42 netmask ffffff00 broadcast 10.10.10.255
        ether 0:10:20:30:40:aa
# flowadm add-flow -l DMZ0 -a transport=tcp,local_port=80 httpflow
# flowadm add-flow -l DMZ0 -a transport=tcp,local_port=443 httpsflow
# flowadm set-flowprop -p maxbw=100 httpflow
# flowadm set-flowprop -p maxbw=100 httpsflow
# flowadm add-flow -l internal0 -a local_ip=10.10.12.45 app-flow 
# flowadm add-flow -l internal0 -a local_ip=10.10.12.46 db-flow
# flowadm add-flow -l internal0 -a local_ip=10.10.12.47 backup-flow
# flowadm set-flowprop -p maxbw=100,priority=high,cpus=2 app-flow
# flowadm set-flowprop -p maxbw=100,priority=high,cpus=3 db-flow
# flowadm set-flowprop -p maxbw=10,priority=low,cpus=4 backup-flow
# flowadm show-flow
NAME            LINK            ATTR            VALUE
app-flow        internal0
                                ip_version      4
                                local_ip        10.10.12.45/
db-flow         internal0
                                ip_version      4
                                local_ip        10.10.12.46/
backup-flow    internal0
                                ip_version      4
                                local_ip        10.10.12.47/

httpsflow       DMZ0
                                ip_version      4
                                transport       tcp
                                local_port      443
httpflow        DMZ0
                                ip_version      4
                                transport       tcp
                                local_port      80

See Also

• To observe flow traffic on a network, refer to How to Verify Virtual Network Connectivity by Using the snoop Command (flow control specific information to come)

• To gathered statistics for flows for accounting purposes, refer to Gathering Usage Statistics for VNICs and Flows.