| Skip Navigation Links | |
| Exit Print View | |
|
Oracle Solaris Administration: Network Interfaces and Network Virtualization Oracle Solaris 11 Express 11/10 |
| Skip Navigation Links | |
| Exit Print View | |
|
|
Oracle Solaris Administration: Network Interfaces and Network Virtualization Oracle Solaris 11 Express 11/10 |
2. NWAM Configuration and Administration (Overview)
3. NWAM Profile Configuration (Tasks)
4. NWAM Profile Administration (Tasks)
5. About the NWAM Graphical User Interface
Part II Administering Single Interfaces
6. Overview of the Networking Stack
7. Datalink Configuration and Administration
8. Configuring an IP Interface
9. Configuring Wireless Interface Communications on Oracle Solaris
Part III Administering Interface Groups
11. Administering Link Aggregations
Part IV Network Virtualization and Resource Management
15. Introducing Network Virtualization and Resource Control (Overview)
16. Planning for Network Virtualization and Resource Control
17. Configuring Virtual Networks (Tasks)
18. Using Link Protection in Virtualized Environments
19. Managing Network Resources
Overview of Network Resource Management
Datalink Properties for Resource Control
Network Resource Management by Using Flows
Commands for Network Resource Management
Network Resource Management (Task Map)
Managing Resources on Datalinks
MAC Clients and Ring Allocation
Properties for Ring Allocation
Preparations for Configuring Hardware-Based Clients
How to Configure a Hardware-Based Client
How to Create a Software-Based Client
How to Identify Ring Assignments in Static Ring Allocation
Configuring Flows on the Network
This section describes selected link properties that you can set to improve network performance for either a physical network or a virtual network.
On NICs, receive (Rx) rings and transmit (Tx) rings are hardware resources through which the system receives and sends network packets, respectively. The following sections provide an overview of rings followed by procedures that are used to allocate rings for networking processes. Examples are also provided to show the mechanism works when you issue commands to allocate rings.
MAC clients such as VNICs and other datalinks are configured over the NIC to enable communication between a system and other network nodes. After a client is configured, it uses both Rx and Tx rings to receive or transmit network packets respectively. A MAC client can either be hardware-based or software-based. A hardware-based client fulfills any one of the following conditions:
It has dedicated use of one or more Rx rings.
It has dedicated use of one or more Tx rings.
It has dedicated use of one or more Rx rings and one or more Tx rings.
Clients that do not fulfill any of these conditions are called software-based MAC clients.
Hardware-based clients can be assigned rings for exclusive use depending on the NIC. NICs such as nxge support dynamic ring allocation. On such NICs, you can configure not only hardware-based clients. You also have the flexibility to determine the number of rings to allocate to such clients, assuming that rings remain available for allocation. Use of rings is always optimized for the primary interface, for example, nxge0. The primary interface is also known as the primary client. Any available rings that have not been assigned for exclusive use by other clients are automatically assigned to the primary interface.
Other NICs such as ixge only support static ring allocation. On these NICs, you can only create hardware-based clients. The clients are automatically configured with a fixed set of rings per client. The fixed set is determined during the NIC driver's initial configuration. For more information about a driver's initial configuration for static ring allocation, refer to the Oracle Solaris Tunable Parameters Reference Manual.
With VLANs, the assignment of rings proceeds differently depending on how the VLAN is created. VLANs are created in one of two ways:
By using the dladm create-vlan subcommand:
# dladm create-vlan -l link -v VID vlan
By using the dladm create-vnic subcommand:
# dladm create-vnic -l link -v VID vnic
A VLAN that is created by the dladm create-vlan subcommand shares the same MAC address as the underlying interface. Consequently, that VLAN also shares the Rx and Tx rings of the underlying interface. A VLAN that is created as a VNIC by using the dladm create-vnic command has a different MAC address from its underlying interface. The allocation of rings for such a VLAN is independent of the allocation for the underlying link. Thus, that VLAN can be assigned its own dedicated rings, assuming that the NIC supports hardware-based clients.
To administer rings, two ring properties can be set by using the dladm command:
rxrings refers to the number of assigned Rx rings to a specified link.
txrings refers to the number of assigned Tx rings to a specified link.
You can set each property to one of three possible values:
sw indicates that you are configuring a software-based client. The client does not have exclusive use of rings. Rather, the client shares rings with any other existing clients that are similarly configured.
n > 0 (number greater than zero) applies to the configuration of a hardware-based client only. The number refers to the quantity of rings that you allocate to the client for its exclusive use. You can specify a number only if the underlying NIC supports dynamic ring allocation.
hw also applies to the configuration of a hardware-based client. However, for such a client, you cannot specify the actual number of dedicated rings. Rather, the fixed number of rings per client is already set according to the NIC driver's initial configuration. You set the *rings properties to hw if the underlying NIC supports static ring allocation only.
To provide information about current ring assignments and use, the following additional read-only ring properties are available:
rxrings-available and txrings-available indicate the number of Rx and Tx rings that are available for allocation.
rxhwclnt-available and txhwclnt-available indicate the number of Rx and Tx hardware-based clients that can be configured over a NIC.
Before you configure hardware-based clients, you must know the ring allocation capabilities of the NIC on your system. To obtain the required information, use the following command:
# dladm show-linkprop link
where link refers to the datalink of your specific NIC.
To display specific properties, use the following command:
# dladm show-linkprop -p property[,property,...] link
To properly configure hardware-based clients, you must determine the following:
Whether the NIC supports hardware-based clients
The rxrings and txrings properties in the command output indicate whether a NIC supports hardware-based clients. From the same data, you can also determine the type of ring allocation that is supported by the NIC.
The availability of rings to allocate to hardware-based clients
The rxrings-available and txrings-available properties in the command output indicate the available Rx rings and Tx rings that you can allocate to a hardware-based client.
The availability of hardware-based clients that you can configure on the link
Rings are allocated as sets. No one-to-one correspondence exists between the number of available rings and the number of clients that can use dedicated rings. Consequently, to allocate rings, you must check not only the availability of rings but also the number of additional hardware-based clients that you can still configure to use dedicated rings. You can allocate rings only if both rings and hardware-based clients are available.
The rxhwclnt-available and txhwclnt-available properties in the command output indicate how many hardware-based clients you can configure that can use dedicated Rx and Tx rings.
If the NIC supports ring allocation, and rings and hardware-based clients are available, then you can configure this type of client on the system, as explained in How to Configure a Hardware-Based Client. Alternatively, you can configure a software-based client instead, as explained in How to Create a Software-Based Client.
The following examples show different information that is displayed for ring-related link properties of an nxge NIC, an ixgbe NIC, and an e1000g NIC.
Example 19-1 nxge NIC Ring Information
The following example shows ring information for an nxge NIC.
# dladm show-linkprop nxge0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... nxge0 rxrings rw -- -- sw,<1-7> ... nxge0 txrings rw -- -- sw,<1-7> ... nxge0 rxrings-available r- 5 -- -- nxge0 txrings-available r- 5 -- -- nxge0 rxhwclnt-available r- 2 -- -- nxge0 txhwclnt-available r- 2 -- -- ...
The POSSIBLE field lists sw and <1-7> as acceptable values for the rxrings and txrings properties. These values indicate that nxge supports hardware-based clients as well as software-based clients. The range <1-7> indicates that the number of Rx rings or Tx rings you set must be within the specified range. You can also infer from the range that the NIC supports dynamic ring allocation for both the receive and transmit sides.
In addition, the *rings-available properties indicate that five Rx rings and five Tx rings are available to allocate to hardware-based clients.
However, based on the *clnt-available properties, you can configure only two clients that can have exclusive use of available Rx rings. Likewise, you can configure only two clients that can have exclusive use of available Tx rings.
Example 19-2 ixgbe NIC Ring Information
The following example shows ring information for an ixgbe NIC.
# dladm show-linkprop ixgbe0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... ixgbe0 rxrings rw -- -- sw,hw ... ixgbe0 txrings rw -- -- sw,hw,<1-7> ... ixgbe0 rxrings-available r- 0 -- -- ixgbe0 txrings-available r- 5 -- -- ixgbe0 rxhwclnt-available r- 0 -- -- ixgbe0 txhwclnt-available r- 7 -- -- ...
The POSSIBLE field for both the rxrings and txrings properties indicates that both hardware-based clients and software-based clients can be configured on ixgbe0. Only static ring allocation is supported for Rx rings, where the hardware assigns a fixed set of Rx rings to each hardware-based client. However, you can allocate Tx rings dynamically, meaning that you can determine the number of Tx rings to assign to a hardware-based client, in this example, up to seven rings.
In addition, the *rings-available properties indicate that five Tx rings are available to allocate to hardware-based clients, but no Rx rings can be assigned.
Finally, based on the *hwclnt-available properties, you can configure seven hardware-based Tx clients to use Tx rings exclusively. However, dynamic Rx ring allocation is not supported in ixgbe cards. Therefore, you cannot create a hardware-based client with a specified set of dedicated Rx rings.
A zero (0) under the VALUE field for either of the *rings-available properties can mean one of the following:
No more rings are available to allocate to clients.
Dynamic ring allocation is not supported.
You can verify the meaning of the zero by comparing the POSSIBLE field for rxrings and txrings and the VALUE field for rxrings-available and txrings-available.
For example, suppose that txrings-available is 0, as follows:
# dladm show-linkprop ixgbe0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... ixgbe0 rxrings rw -- -- sw,hw ixgbe0 txrings rw -- -- sw,hw,<1-7> ixgbe0 rxrings-available r- 0 -- -- ixgbe0 txrings-available r- 0 -- -- ...
In this output, the VALUE field for rxrings-available is 0 while the POSSIBLE field for rxrings is sw,hw. The combined information means that no Rx rings are available because the NIC does not support dynamic ring allocation. On the transmit side, the VALUE field for txrings-available is 0 while the POSSIBLE field for txrings is sw,hw,<1-7>. The combined information indicates that, no Tx rings are available because all the Tx rings have already been allocated. However, as the POSSIBLE field for txrings indicates, dynamic ring allocation is supported. Thus, you can allocate Tx rings as these rings become available.
Example 19-3 e1000g NIC Ring Information
The following example shows ring information for an e1000g NIC.
# dladm show-linkprop e1000g0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... e1000g0 rxrings rw -- -- -- ... e1000g0 txrings rw -- -- -- ... e1000g0 rxrings-available r- 0 -- -- e1000g0 txrings-available r- 0 -- -- e1000g0 rxhwclnt-available r- 0 -- -- e1000g0 txhwclnt-available r- 0 -- -- ...
The output indicates that neither rings nor hardware-based clients can be configured because ring allocation is not supported in e1000g NICs.
This procedure shows how to configure a hardware-based client either on a NIC that supports dynamic ring allocation or on a NIC that supports static ring allocation.
Make sure that you have obtained the following information about the NIC on your system:
Whether the NIC supports hardware-based clients
The type of ring allocation that the NIC supports
The availability of rings to allocate to hardware-based clients
The availability of hardware-based clients that you can configure on the link
To obtain the information, refer to Preparations for Configuring Hardware-Based Clients.
If the NIC supports dynamic ring allocation, use the following syntax:
# dladm create-vnic -p rxrings=number[,txrings=number] -l link vnic
Refers to the number of Rx rings and Tx rings that you allocate to the client. The number must be within the range of the number of available rings for allocation.
Note - Some NICs support dynamic allocation on either Rx rings or Tx rings, but not on both types. You specify number on the ring type for which dynamic ring allocation is supported.
Refers to the datalink over which you are creating the client.
Refers to the client that you are configuring.
If the NIC supports static ring allocation, use the following syntax:
# dladm create-vnic -p rxrings=hw[,txrings=hw] -l link vnic
Note - Some NICs support static allocation on either Rx rings or Tx rings, but not on both types. You specify hw on the ring type for which static ring allocation is supported.
# dladm show-linkprop vnic
A software-based client does not have exclusive use of rings. Rather, the client shares the use of rings with the primary client or interface with other existing software-based clients. The ring count for software-based clients depends on the number of existing hardware-based clients.
To create a new software-based client, type the following command:
# dladm create-vnic -p rxrings=sw[,txrings=sw] -l link vnic
Refers to the datalink over which you are creating the client.
Refers to the client that you are configuring.
To configure an existing client to share rings with other clients, type the following command:
# dladm set-linkprop -p rxrings=sw[,txrings=sw] vnic
Example 19-4 Configuring Hardware-Based Clients and Software-Based Clients
This example shows how to configure both hardware-based clients and software-based clients on a system with an ixgbe NIC. To show how ring allocation is implemented, the example is divided into parts. Ring-related information is displayed and explained at each step of the configuration process. The configuration proceeds as follows:
Display the links and ring usage on the system prior to the configuration of clients.
Configure the primary client.
Configure a software-based client.
Configure another client without any dedicated rings.
Statically allocate rings to the newly configured client.
Configure a third client with dedicated rings that are dynamically allocated.
First, display the links, ring usage, and ring-related properties.
# dladm show-link LINK CLASS MTU STATE BRIDGE OVER ixgbe0 phys 1500 down -- -- # dladm show-phys -H ixgbe0 LINK RINGTYPE RINGS CLIENTS ixgbe0 RX 0-1 <default,mcast> ixgbe0 TX 0-7 <default> ixgbe0 RX 2-3 -- ixgbe0 RX 4-5 -- ixgbe0 RX 6-7 -- # dladm show-linkprop ixgbe0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... ixgbe0 rxrings rw -- -- sw,hw ixgbe0 rxrings-effective r -- -- -- ixgbe0 txrings rw -- -- sw,hw,<1-7> ixgbe0 txrings-effective r -- -- -- ixgbe0 txrings-available r- 7 -- -- ixgbe0 rxrings-available r- 0 -- -- ixgbe0 rxhwclnt-available r- 3 -- -- ixgbe0 txhwclnt-available r- 7 -- -- ...
The command output shows a single link ixgbe0 on the system, but no existing clients. In addition, the following information is also gleaned from this output:
The NIC has eight Rx rings and eight Tx rings (rings 0 to 7).
For hardware-based clients, only static ring allocation is supported for Rx rings, while both static and dynamic ring allocations are supported for Tx rings.
Software-based clients can be configured for both Rx rings and Tx rings.
Seven Tx rings, 1 to 7, are available to be dynamically allocated to other clients (ring 0 is typically reserved for the primary client). No Rx rings are available because dynamic ring allocation is not supported for Rx rings.
Three hardware-based clients can be configured to use Rx rings, while seven hardware-based clients can be configured to use Tx rings.
For an explanation of the *rings-effective properties, see How to Identify Ring Assignments in Static Ring Allocation.
Next, configure the primary client.
# ipadm create-addr -T static -a 192.168.10.10/24 ixgbe0/v4 # dladm show-phys -H ixgbe0 LINK RINGTYPE RINGS CLIENTS ixgbe0 RX 0-1 <default,mcast> ixgbe0 TX 0-7 <default>ixgbe0 ixgbe0 RX 2-3 ixgbe0 ixgbe0 RX 4-5 -- ixgbe0 RX 6-7 -- # dladm show-linkprop ixgbe0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... ixgbe0 rxrings rw -- -- sw,hw ixgbe0 rxrings-effective r 2 -- -- ixgbe0 txrings rw -- -- sw,hw,<1-7> ixgbe0 txrings-effective r 8 -- -- ixgbe0 txrings-available r- 7 -- -- ixgbe0 rxrings-available r- 0 -- -- ixgbe0 rxhwclnt-available r- 3 -- -- ixgbe0 txhwclnt-available r- 7 -- -- ...
The output provides the following information:
ixgbe0, the primary client, automatically receives two Rx rings (rings 2 and 3) for dedicated use. However, ixgbe0 uses all Tx rings. By default, all unused rings are automatically assigned to the primary client.
The number of available Tx rings that can be allocated to other clients remains at seven.
The number of available hardware-based clients that can be configured with Rx rings remains at three. The number of available hardware-based clients that can be dynamically configured with Tx rings remains at seven.
Next, create a VNIC as a software-based client.
# dladm create-vnic -l ixgbe0 -p rxrings=sw,txrings=sw vnic0 # dladm show-phys -H ixgbe0 LINK RINGTYPE RINGS CLIENTS ixgbe0 RX 0-1 <default,mcast>,vnic0 ixgbe0 TX 0-7 <default>vnic0,ixgbe0 ixgbe0 RX 2-3 ixgbe0 ixgbe0 RX 4-5 -- ixgbe0 RX 6-7 -- # dladm show-linkprop vnic0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... vnic0 rxrings rw sw -- sw,hw ... vnic0 txrings rw sw -- sw,hw,<1-7> ... # dladm show-linkprop ixgbe0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... ixgbe0 rxrings rw -- -- -- ixgbe0 rxrings-effective r 2 -- -- ixgbe0 txrings rw -- -- sw,hw,<1-7> ixgbe0 txrings-effective r -- -- -- ixgbe0 txrings-available r- 7 -- -- ixgbe0 rxrings-available r- 0 -- -- ixgbe0 rxhwclnt-available r- 3 -- -- ixgbe0 txhwclnt-available r- 7 -- -- ...
The output provides the following information:
As a software based-client, vnic0 is automatically assigned to use Rx rings 0 and 1. Other software-based clients with Rx rings that are subsequently created will be assigned to use this pair by default. By default, vnic0 is also assigned the use of all eight Tx rings (rings 0 to 7). Other software-based clients with Tx rings that are subsequently created will be assigned to use this set of rings by default.
As a software-based client, vnic0's rxrings and txrings properties are accordingly set to sw.
No Tx rings are assigned. Therefore, the number of available Tx rings that can be allocated to other clients remains at seven.
The number of available hardware-based clients that can be configured with Rx rings remains at three. The number of available hardware-based clients that can be configured with Tx rings remains at seven.
Next, configure another client without any ring allocation.
# dladm create-vnic -l ixgbe0 vnic1 # dladm show-phys -H ixgbe0 LINK RINGTYPE RINGS CLIENTS ixgbe0 RX 0-1 <default,mcast>,vnic0 ixgbe0 TX 0,2-7 <default>vnic0,ixgbe0 ixgbe0 RX 2-3 ixgbe0 ixgbe0 RX 4-5 vnic1 ixgbe0 RX 6-7 -- ixgbe0 TX 1 vnic1 # dladm show-linkprop vnic1 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... vnic1 rxrings rw -- -- sw,hw vnic1 rxrings-effective r- 2 -- -- vnic1 txrings rw -- -- sw,hw,<1-7> vnic1 txrings-effective r- -- -- -- ... # dladm show-linkprop ixgbe0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... ixgbe0 rxrings rw -- -- sw,hw ixgbe0 rxrings-effective r- 2 -- -- ixgbe0 txrings rw -- -- sw,hw,<1-7> ixgbe0 txrings-effective r- -- -- -- ixgbe0 txrings-available r- 7 -- -- ixgbe0 rxrings-available r- 0 -- -- ixgbe0 rxhwclnt-available r- 3 -- -- ixgbe0 txhwclnt-available r- 7 -- -- ...
The output provides the following information:
When ring allocation is supported, a client that is configured is considered a hardware-based client, even though the rxrings and txrings properties are not set. Thus, vnic1 automatically receives two dedicated Rx rings (rings 4 and 5) for its use. Likewise, vnic1 also receives a dedicated Tx ring (ring 1).
Of the eight Tx rings, ixgbe0 and vnic0 now share seven rings (ring 0 and rings 2 through 7). Ring 1 has become a dedicated Tx ring for vnic1.
No Tx rings are assigned. Therefore, the number of available Tx rings that can be allocated to other clients remains at seven.
The number of available hardware-based clients that can be configured with Rx rings remains at three. The number of available hardware-based clients that can be configured with Tx rings remains at seven.
Next, statically allocate rings to the newly configured client, vnic1.
# dladm set-linkprop -p rxrings=hw,txrings=hw vnic1 # dladm show-phys -H ixgbe0 LINK RINGTYPE RINGS CLIENTS ixgbe0 RX 0-1 <default,mcast>,vnic0 ixgbe0 TX 0,2-7 <default>vnic0,ixgbe0 ixgbe0 RX 2-3 ixgbe0 ixgbe0 RX 4-5 vnic1 ixgbe0 RX 6-7 -- ixgbe0 TX 1 vnic1 # dladm show-linkprop vnic1 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... vnic1 rxrings rw hw -- sw,hw vnic1 rxrings-effective r- 2 -- -- vnic1 txrings rw hw -- sw,hw,<1-7> vnic1 txrings-effective r- -- -- -- ... # dladm show-linkprop ixgbe0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... ixgbe0 rxrings rw -- -- sw,hw ixgbe0 rxrings-effective r- 2 -- -- ixgbe0 txrings rw -- -- sw,hw,<1-7> ixgbe0 txrings-effective r- -- -- -- ixgbe0 txrings-available r- 6 -- -- ixgbe0 rxrings-available r- 0 -- -- ixgbe0 rxhwclnt-available r- 3 -- -- ixgbe0 txhwclnt-available r- 6 -- -- ...
The output provides the following information:
The distribution of Rx and Tx rings for vnic1 remains the same as when vnic1 was created without ring allocation.
Similarly, ring information remains the same as when vnic1 was created without ring allocation.
The rxrings and txrings properties of vnic1 have been explicitly set to hw. Consequently, the number of available Tx rings for dynamic allocation has been reduced to six. Likewise, the number of available hardware-based clients that can be configured has been reduced to six.
Next, configure a hardware-based client with Tx rings that are dynamically allocated.
# dladm create-vnic -l ixgbe0 -p txrings=2 vnic2 # dladm show-phys -H ixgbe0 LINK RINGTYPE RINGS CLIENTS ixgbe0 RX 0-1 <default,mcast>,vnic0 ixgbe0 TX 0,4-7 <default>vnic0,ixgbe0 ixgbe0 RX 2-3 ixgbe0 ixgbe0 RX 4-5 vnic1 ixgbe0 RX 6-7 vnic2 ixgbe0 TX 1 vnic1 ixgbe0 TX 2-3 vnic2 # dladm show-linkprop vnic2 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... vnic2 rxrings rw -- -- sw,hw vnic2 rxrings-effective r- 2 -- -- vnic2 txrings rw 2 -- sw,hw,<1-7> vnic2 txrings-effective r- 2 -- -- ... # dladm show-linkprop ixgbe0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... ixgbe0 rxrings rw -- -- sw,hw ixgbe0 rxrings-effective r- 2 -- -- ixgbe0 txrings rw -- -- sw,hw,<1-7> ixgbe0 txrings-effective r- -- -- -- ixgbe0 txrings-available r- 4 -- -- ixgbe0 rxrings-available r- 0 -- -- ixgbe0 rxhwclnt-available r- 3 -- -- ixgbe0 txhwclnt-available r- 5 -- -- ...
The output provides the following information:
The hardware automatically assigned a pair of Rx rings (rings 6 and 7) to vnic2 for exclusive use. However, vnic2's two dedicated Tx rings (rings 2 and 3) were assigned by the administrator.
With two Tx rings administratively assigned to vnic2, the number of available Tx rings that can be allocated to other clients has been reduced to four.
With vnic2 configured as a hardware-based client with two Tx rings, the number of available clients that can be configured has been reduced to five.
When you configure a hardware-based client with static ring allocation, the hardware determines the number of rings to assign. However, the rxrings and txrings properties are set to hw and do not indicate the number of rings that are actually allocated. Instead, the number can be obtained by checking the rxrings-effective and txrings-effective properties.
To create the client with static ring allocation, type the following command:
# dladm create-vnic -l link -p rxrings=hw[,txrings=hw] vnic
Refers to the datalink over which you are creating the client.
Refers to the client that you are configuring.
To statically allocate rings to an existing client, type the following command:
# dladm set-linkprop -p rxrings=hw[,txrings=hw] vnic
# dladm show-linkprop link
where link refers to the hardware-based client or VNIC.
For example, if you statically allocated Rx rings, check the rxrings-effective property. If you statically allocated Tx rings, check the txrings-effective property. The number indicates how many rings were allocated by the hardware.
# dladm show-phys -H link
where link refers to the primary client.
Example 19-5 Identifying Rings That Are Statically Allocated
This example shows how Rx rings have been statically allocated to a client that is configured over an ixgbe NIC. On such a NIC, only static allocation is supported for Rx rings. The example proceeds as follows:
Display the links on the system. In this example, the system has only one link, which is ixgbe0.
Create vnic1 as a hardware-based client with Rx rings that are statically allocated.
Display ring information to know the number of rings allocated by the hardware.
Display ring usage to identify which rings have been allocated.
# dladm show-link LINK CLASS MTU STATE BRIDGE OVER ixgbe0 phys 1500 down -- -- # dladm create-vnic -l ixgbe0 -p rxrings=hw vnic1 # dladm show-linkprop vnic1 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... vnic1 rxrings rw hw -- sw,hw vnic1 rxrings-effective r- 2 -- -- vnic1 txrings rw -- -- sw,hw,<1-7> vnic1 txrings-effective r- -- -- -- # dladm show-phys -H ixgbe0 LINK RINGTYPE RINGS CLIENTS ixgbe0 RX 0-1 <default,mcast> ixgbe0 TX 0,2-7 <default> ixgbe0 RX 2-3 vnic1 ixgbe0 RX 4-5 -- ixgbe0 RX 6-7 -- ixgbe0 TX 1 vnic1 ...
The output indicates that after vnic1 was configured with Rx rings, the hardware allocated two dedicated Rx rings, as reflected by the rxrings-effective property. Based on the output of the dladm show-phys -H command, Rx rings 2 and 3 were dedicated for vnic1's use.
As a result of being configured as a client, vnic1 also automatically received Tx ring 1 for its dedicated use. However, the txrings-effective property displays no value because the txrings property is not explicitly set.
The pool is a link property that enables you to bind network processing to a pool of CPUs. With this property, you can better integrate network resource management with CPU binding and administration in zones. In Oracle Solaris, zone administration includes the binding of non-networking processes to a pool of CPU resources by using the zonecfg or poolcfg command. To bind that same pool of resources to also manage network processes, you use the dladm set-linkprop command to configure a link's pool property. Then you assign that link to the zone.
By setting the pool property for a link and assigning the link as the zone's network interface, then that link becomes bound to a zone's pool as well. If that zone is set to become an exclusive zone, then CPU resources in the pool can no longer be used by other datalinks that are not assigned to that zone.
Note - A separate property, cpu, can be set to assign specific CPUs to a datalink. The two properties, cpu and pool, are mutually exclusive. You cannot set both properties for a given datalink. To assign CPU resources to a datalink by using the cpu property, see How to Allocate CPUs to Links.
For more information about pools within a zone, see Chapter 13, Creating and Administering Resource Pools (Tasks), in System Administration Guide: Oracle Solaris Zones, Oracle Solaris 10 Containers, and Resource Management. For more information about creating pools and assigning CPU sets to the pools, refer to the poolcfg(1M) man page.
The following figure show how pools work when the pool property is assigned to a datalink.
Figure 19-1 pool Property of a VNIC Assigned to a Zone
In the figure, the system has eight CPUs. When no pools are configured on the system, all the CPUs belong to the default pool and are used by the global zone. However, in this example, the pool99 pool has been created and consists of CPU 3 and CPU 4. This pool is associated with zone1, which is an exclusive zone. If pool99 is set as a property of vnic1, then pool99 becomes dedicated to also manage vnic1's networking processes. After vnic1 is assigned to be zone1's network interface , then the CPUs in pool99 become reserved to manage both networking and non–networking processes of zone1.
The pool property is dynamic in nature. Zone pools can be configured with a range of CPUs, and the kernel determines which CPUs are assigned to the pool's CPU set. Changes to the pool are automatically implemented for the datalink, which simplifies pool administration for that link. By contrast, assigning specific CPUs to the link by using the cpu property requires you to specify the CPU to be assigned. You have to set the cpu property every time you want to change the CPU components of the pool.
For example, suppose that in the system in Figure 19-1, CPU 4 is taken offline. Because the pool property is dynamic, the software automatically associates an additional CPU with the pool. Thus, the pool's original configuration of two CPUs is preserved. For vnic1, the change is transparent. The adjusted configuration is shown in the following figure.
Figure 19-2 Automatic Reconfiguration of the pool Property
Additional pool related properties display information about a datalink's use of CPUs or a pool of CPUs. These properties are read-only and cannot be set by the administrator.
pool-effective displays the pool that is being used for network processes.
cpus-effective displays the list of CPUs that are being used for network processes.
To manage CPU resources of a zone, setting a datalink's pool property is not normally performed as an initial step. More frequently, commands such as zonecfg and poolcfg are used to configure a zone to use a pool of resources. The cpu and pool link properties themselves are not set. In such cases, the pool-effective as well as the cpus-effective properties of these datalinks are set automatically according to those zone configurations when the zone is booted. The default pool is displayed under pool-effective, while the value of cpus-effective is selected by the system. Thus, if you use the dladm show-linkprop command, the pool and cpu properties will be empty, while the pool-effective and cpus-effective properties will contain values.
Directly setting the pool and cpu properties of a datalink is an alternative step that you can use to bind a zone's CPU pool for networking processes. After you configure these properties, their values are reflected in the pool-effective and cpus-effectiveproperties as well. Note, however, that this alternative step is less typically used to manage a zone's network resources.
As with other link properties, the pool property can be set for a datalink either at the moment when the link is created or later when the link requires further configuration. For example:
# dladm create-vnic -p pool=pool-name -l link vnic
sets the pool property while you create the VNIC. To set the pool property of an existing VNIC, you use the following syntax:
# dladm setlinkprop -p pool=pool-name vnic
The following procedure provides the steps to configure a CPU pool for a VNIC.
You must have completed the following:
Created a processor set with its assigned number of CPUs.
Created a pool with which the processor set will be associated.
Associated the pool with the processor set.
Note - For the instructions to complete these prerequisites, see How to Modify a Configuration in System Administration Guide: Oracle Solaris Zones, Oracle Solaris 10 Containers, and Resource Management
If the VNIC has not yet been created, use the following syntax:
# dladm create-vnic -l link -p pool=pool vnic
where pool refers to the name of the pool that was created for the zone.
If the VNIC exists, use the following syntax:
# dladm setlinkprop -p pool=pool vnic
zonecfg>zoneid:net> set physical=vnic
Note - For the complete steps that explain how to assign a networking interface to a zone, refer to the Configuring, Verifying, and Committing a Zone in System Administration Guide: Oracle Solaris Zones, Oracle Solaris 10 Containers, and Resource Management
Example 19-6 Assigning a Link's CPU Pool to a Zone With an Exclusive IP-Type
This example shows how a pool is assigned to a zone's datalink. The scenario is based on the configuration in Figure 19-1. The example assumes that a pool of CPUs named pool99 has already been configured for the zone. The pool is then assigned to a VNIC. Finally, the non–global zone zone1 is set to use the VNIC as the networking interface.
# dladm create-vnic -l e1000g0 -p pool99 vnic0 # zonecfg -c zone1 zonecfg:zone1> set ip-type=exclusive zonecfg:zone1> add net zonecfg:zone1>net> set physical=vnic0 zonecfg:zone1>net> end zonecfg:zone1> exit
The following procedure explains how to assign specific CPUs to process traffic traversing a datalink by configuring the cpu property.
# dladm show-linkprop -p cpus link
By default, no CPUs are assigned to any specific interface. Thus, the parameter VALUE in the command output will not contain any entry.
# echo ::interrupts | mdb -k
The output lists parameters for each link in the system, including the CPU number.
The CPUs can include those with which the link's interrupts are associated.
# dladm set-linkprop -p cpus=cpu1,cpu2,... link
where cpu1 is the CPU number to be assigned to the link. You can dedicate multiple CPUs to the link.
# echo ::interrupts | mdb -k
# dladm show-linkprop -p cpus link
Example 19-7 Allocating CPUs to the Interface
This example shows how to dedicate specific CPUs to the internal0 interface in Figure 16-3.
Note the following information in the output that is generated by the different commands. For clarity, the significant information is emphasized in the output.
By default internal0 has no dedicated CPU. Thus VALUE is --.
The interrupt of internal0 is associated with CPU 18.
After CPUs are allocated, internal0 displays a new CPU list under VALUE.
# dladm show-linkprop -p cpus internal0
LINK PROPERTY PERM VALUE DEFAULT POSSIBLE
internal0 cpus rw -- -- --
# echo ::interrupts | mdb -k
Device Shared Type MSG # State INO Mondo Pil CPU
external#0 no MSI 3 enbl 0x1b 0x1b 6 0
internal#0 no MSI 2 enbl 0x1a 0x1a 6 18
# dladm set-linkprop -p cpus=14,18,19,20 internal0
# dladm show-linkprop -p cpus internal0
LINK PROPERTY PERM VALUE DEFAULT POSSIBLE
internal0 cpus rw 14,18,19,20 -- --
All the supporting threads including the interrupt are now confined to the newly assigned set of CPUs.
|