Fibre Channel Over Ethernet (FCoE) is a T11 standard that transports encapsulated FC frames over enhanced Ethernet. FCoE enables network convergence and cost-effective SAN expansion in large deployments.
This section provides information about how to configure the software FCoE port hosted on a normal Ethernet interface, and hardware and software information about FCoE ports on converged network adapters (CNAs).
CNAs combine the functionality of FC-HBAs and Ethernet network interface card (NIC) to transfer Ethernet and FCoE traffic. CNAs convert the FCoE traffic into FC traffic, which is then sent to the connected SAN over the FC network.
Second-generation CNAs are generally used to set up the SAN. Using CNAs have the following advantages:
Using CNAs in storage network reduces the number of adapters used in a storage network, which helps reduce the number of switch ports, cables, and PCI Express slots.
CNAs can offload FCoE protocol processing tasks, which helps to reduce the consumption of server CPU resources.
The CNA connects to the server by using PCI Express (PCI-E) expansion interface.
CNAs can be used over FC networks, with FC switches and Fibre Channel Management utilities.
CNAs can carry or transfer Ethernet traffic and FCoE traffic.
CNAs can be used as a stand-alone 10 GE NIC, if FCoE and FC SAN are not immediately available.
For more information about how to perform hardware installation and configuration of CNA based FCoE ports, refer to the following guides:
For QLogic, see Sun Storage 16 Gb Fibre Channel PCIe Universal Host Bus Adapter, Qlogic
For Emulex, see Sun Storage 16 Gb Fibre Channel PCIe Universal Host Bus Adapter, Emulex
The Oracle Solaris FCoE initiator is a software implementation that works with normal Ethernet controllers.
Oracle Solaris FCoE implementations has the following limitations:
FCoE ports cannot be used for booting on QLogic and Emulex cards.
FCoE ports cannot be configured in Oracle VM Server for SPARC or Oracle VM Server 3.0 for x86 guest operating systems.
FCoE is supported in fabric and point-to-point topologies.
FCoE is not supported on all hardware. FCoE works with Ethernet controllers that support 802.3x (PAUSE) and jumbo frames.
You can configure FCoE ports by using the fcadm command. Use the fcinfo and fcadm commands to determine the status of FCoE ports. These commands also report the relationship between the Ethernet interface and the FCoE port hosted on that interface.
Other FC commands, such as cfgadm, report FCoE information, although no distinction is made between FCoE and native FC ports.
Before you create an FCoE port, you must perform the following tasks:
Enable the 802.3x (also called PAUSE) setting on the Ethernet interface. This setting ensures a lossless Ethernet transport.
Enable jumbo frames (greater than 2.5 KB) on the Ethernet interface. An FC data frame can be as large as 2136 bytes.
These settings can vary for different Ethernet hardware and drivers. In most cases, you must modify the driver.conf file of the Ethernet interface and then reboot. See the driver.conf file for your Ethernet interface for details about how to enable these features.
Each virtual port must have a port name and node name. The port name must be unique on the SAN. You can assign names manually or use the built-in world wide name (WWN) generator. If you attempt to register duplicate names, the switch will report an error status on the newly registered WWN, and the switch will not register the new WWN. For more information on acceptable name formats, refer to the T11 standard: Fibre Channel Framing and Signaling (FC-FS 2).
If you try to create an FCoE port on a network interface that does not support FCoE, an error occurs and the FCoE port is not created.
Ensure the following services are enabled:
# svcs svc:/system/fcoe_initiator:default STATE STIME FMRI online Dec_09 svc:/system/fcoe_initiator:default
As an administrator, you can use the following command to create an FCoE port.
# fcadm create-fcoe-port -i -p Port-WWN -n Node-WWN Ethernet-interface
If the selected Ethernet interface does not support Multiple Unicast Address, you are prompted to explicitly enable promiscuous mode on that interface.
# fcadm create-fcoe-port -i -f Ethernet-interface
For example:
# fcadm create-fcoe-port -i net0
As an administrator, you can use the following command to delete an FCoE port:
# fcadm delete-fcoe-port network-interface
For example:
# fcadm delete-fcoe-port net0
You can use the fcadm list-fcoe-ports command to display the Ethernet interfaces hosting the FCoE ports.
As an administrator, you can use the following command to display the status of currently configured FCoE ports:
# fcinfo hba-port -e
For example:
# fcinfo hba-port -e HBA Port WWN: 200000144fc1f5c8 Port Mode: Initiator Port ID: 9a0042 OS Device Name: /dev/cfg/c6 Manufacturer: Sun Microsystems, Inc. Model: FCoE Virtual FC HBA Firmware Version: N/A FCode/BIOS Version: N/A Serial Number: N/A Driver Name: SunFC FCoEI v20090422-1.00 Driver Version: v20090422-1.00 Type: N-port State: online Supported Speeds: 1Gb 10Gb Current Speed: 10 Gb Node WWN: 100000144fc1f5c8
You can use the following command to list FC specific information for all FCoE ports in the system:
# fcadm list-fcoe-ports
For example:
# fcadm list-fcoe-ports
HBA Port WWN: 2000001b2165a630
Port Mode: Initiator
Port ID: e00033
VLAN ID: 7
Link Name: net2
MTU Size: 2500
Primary MAC Address: 00:1b:21:65:a6:30
FCoE MAC Address: 0e:fc:00:e0:00:33
Promiscuous Mode: Off
State: Online
FIP Mode: On
FCoE Hardware Offload: Supported
HBA Port WWN: 2000001b2165a631
Port Mode: Target
Port ID: e00034
VLAN ID: 7
Link Name: net3
MTU Size: 2500
Primary MAC Address: 00:1b:21:65:a6:31
FCoE MAC Address: 0e:fc:00:e0:00:34
Promiscuous Mode: Off
State: Online
FIP Mode: On
FCoE Hardware Offload: Supported
If new devices are added to an FC SAN or if there is a device misbehaving on a SAN, you can force an FCoE port reinitialization. In many cases, this operation can resolve problems in an FC SAN.
When the command is issued on the target port side, the target port is reset. When the command is issued on the host port side, the host port is reset.
When you connect an FC switch or other FC ports in a SAN, the system generates a remote state change notification (RSCN). Other initiators rediscover the port after the RSCN is generated, and the FC login session is established or reused. The fcadm force-lip command is disruptive to I/Os, but I/Os continue to communicate. This command does not cause any data loss.
As an administrator, you can use the following command to force a link that is connected to a port to reinitialize:
# fcadm force-lip port-number
For example:
# fcadm force-lip 200000144fc2d508