| Netra Data Plane Software Suite 2.0 Update 2 User’s Guide |    
|
| C H A P T E R 10 |
|
Reference Applications |
This chapter describes Netra DPS reference applications. Topics include:
Each UltraSPARC T2 processor CPU core has a crypto unit. This unit leverages the Modular Arithmetic Unit and the Hash and Cipher Unit inside the SPU to accelerate crypto operations. There are a total of eight crypto units in an eight-core system.
If you are using NIU ports under LDoms environment, refer to How Do I Calculate the Base PA Address for NIU/LDoms to Use with the tnsmctl Command?.
The IP packet forwarding application (ipfwd) simulates an IPv4 (Internet Protocol Version 4) forwarding operation. When each packet is received, the program performs a forward table search (three memory searches), rewrites the MAC source and destination address, rewrites the TTL, and recalculates the IP header checksum. The packet is then sent out to a specified port. The application can either simulate the lookup or actually do a lookup.
All ipfwd source files are located in the following package directory: /opt/SUNWndps/src/apps/ipfwd. The contents include:
Copy the ipfwd reference application from the /opt/SUNWndps/src/apps/ipfwd directory to a desired directory location and do the build script in that location.
TABLE 10-1 shows the IP packet forwarding (ipfwd) application build script.
|
(See Build Script Arguments.) |
Build ipfwd application to run on e1000g Ethernet interface on the Netra ATCA CP3060 System. |
|
Build ipfwd application to run on Sun QGC (quad 1Gbps nxge Ethernet interface). |
|
|
Build ipfwd application to run on Sun multithreaded 10G (dual 10Gbps nxge Ethernet interface). |
|
|
Build ipfwd application to run on NIU (dual 10Gbps UltraSPARC T2 Ethernet interface) on a CMT2-based system. |
./build <cmt> <type> [ldoms [diffserv] [acl] [gdb] [arp] [no_freeq]] [profiler] [2port] -hash <POLICY_NAME>
This argument specifies whether to build the ipfwd application to run on the CMT1 (UltraSPARC T1) platform or CMT2 (UltraSPARC T2) platform.
cmt1 - Build for CMT1 (UltraSPARC T1)
cmt2 - Build for CMT2 (UltraSPARC T2)
Build the ipfwd application to run on e1000g Ethernet interface on a Netra ATCA CP3060 System.
Build ipfwd application to run on 10Gb Ethernet QGC (quad 1Gbps nxge Ethernet interface).
Build ipfwd application to run on 10Gb Ethernet (dual 10Gbps nxge Ethernet interface).
Build ipfwd application to run on NIU (dual 10Gbps UltraSPARC T2 Ethernet interface) on a CMT2-based system.
This is an optional argument specifying whether to build the ipfwd application to run on the LDoms environment. When this flag is specified, the IP forwarding LDoms reference application will be compiled. If this argument is not specified, then the non-LDoms (standalone) application will be compiled. Refer to How Do I Calculate the Base PA Address for NIU/LDoms to Use with the tnsmctl Command?.
This is an optional argument to enable the Access Control List (ACL) reference application.
This is an optional argument to enable gdb support in the LDoms environment.
This option disables the use of free queues. May be used with the diffserv option in an LDoms environment.
This is an optional argument that generate code with profiling enabled.
This is an optional argument to compile dual ports on the 10Gb Ethernet or the UltraSPARC T2 NIU.
This is an optional argument used to enable flow policies. For more information, see Other IPFWD Options.
In ...src/sys/lwrte/apps/ipfwd, pick the correct build script and run it. For example, to build for 10G Ethernet on a Netra or Sun Fire T2000 system, enter the following at your shell window.
In this example, the build script with the 10g option is used to build the IP forwarding application to run on the 10G Ethernet. The <cmt> argument is specified as cmt1 to build the application to run on UltraSPARC T1-based Netra or Sun Fire T2000 systems.
1. Copy the binary into the /tftpboot directory of the tftpboot server.
2. On the tftpboot server, type:
3. On the target machine, type the following at the ok prompt:
| Note - net is an OpenBoot PROM alias corresponding to the physical path of the network. |
This section shows the default configurations:
Main files that control the system configurations:
Main files that control the application configurations:
To enable the Radix Forwarding Algorithm for IP forwarding, uncomment the following line from Makefile.nxge for e1000g 1Gb PCIe ethernet adapter or Sun multithreaded 10Gb and 1Gb PCIe Ethernet adapter, respectively:
To bypass the ipfwd operation (that is, receive --> transmit without forwarding operation), uncomment the following line from Makefile.nxge e1000g 1Gb PCIe ethernet adapter or Sun multithreaded 10 Gb and 1Gb PCIe Ethernet adapter, respectively:
When the Multiple Forward Destinations option is enabled, the output destination port is determined by the output of the Forwarding Table Lookup. Otherwise, the output destination port is the same as the input port. To enable this option, uncomment the following line from Makefile.nxge for e1000g 1Gb PCIe ethernet adapter or Sun multithreaded 10GbE Ethernet, respectively:
The N2_1_MODE flag is enabled by default. This flag needs to be disabled when running Netra DPS on UltraSPARC T2 Version 2.2 and above.
When IP Forwarding is configured as “Cross Configuration”, the IPFWD_STATIC_CROSS_CONFIG flag needs to be enabled. The following is one example of cross configuration:
Port0 ---> Port1
Port1 ---> Port0
Specify a policy for spreading traffic into multiple DMA flows by hardware hashing. TABLE 10-2 describes each policy:
To enable one of the above policies, use the -hash option.
If none of the policies listed in TABLE 10-2 are specified, a default policy is given. The default policy is set to HASH_ALL. When you use the default policy, all L2/L3/L4 header fields are used for spreading traffic.
The ipfwd_config.c file assists you in mapping application tasks to CPU core and hardware strands. Normally, mapping is set in the ipfwd_map.c file in the config directory. This configuration file is a productivity tool. This file provides a way to facilitate mapping in a quick manner without any modification to the ipfwd_map.c file.
This configuration file is not a replacement of ipfwd_hwarch.c, ipfwd_swarch.c and ipfwd_map.c. This framework is to conduct performance analysis and measurement with different system configurations. The default (*_def) configurations specified assumes a minimum of 16 threads of the system allocated for Netra DPS in ipfwd_map.c and all memory pool resources required are declared in ipfwd_swarch.c. The user still needs to specify system resources declarations and mapping in ipfwd_hwarch.c, ipfwd_swarch.c, and ipfwd_map.c. The configuration is assigned to a pointer named ipfwd_thread_config.
Each application configuration is represented in an array of a six-element entry. Each entry (each row) represents a software task and its corresponding resources:
TROLE_ETH_NETIF_RX -- This task performs a receive function. TROLE_ETH_NETIF_TX -- This task performs a transmit function. TROLE_APP_IPFWD -- This task performs IP forwarding function.
See common.h for all definitions. If you do not want to run any software task on this hardware strand, the role field should be set to -1. By default, during initialization of the ipfwd application, the hardware strand that encounters a -1 software role is parked.
| Note - A parked strand is a strand that does not consume any pipeline cycles (an inactive strand). |
The Differentiated Services (DiffServ) reference application is integrated with the IP forwarding application. The DiffServ data path consists of classifier, meter, marker, and policing components. These components provide Quality of Services (QoS) features for traffic entering the node and avoids congestion in the network. These components can be arranged in the pipeline such that each component performs specific task and propagates the result (traffic class and policing information) to the next component. The following are major features of DiffServ:
FIGURE 10-1 shows the arrangement of the components in the data path. The scheduler and queue manager are executed in a separate thread, whereas the other components are located in the forwarding thread. The following sections describe the functions of the different parts.
FIGURE 10-1 IPv4 DiffServ Internal Data Path
This section describes the Diffserv classifiers.
The differentiated services code point (DSCP) classifier fast path component sets QoS variables (flow and color) based on the DSCP value extracted from the IPv4 packet header and directs packets to the proper next component (meter, marker, IPv4) for further processing. The DSCP classifier always remain enabled.
The 6-tuple classifier fast path component performs an exact-match lookup on the IPv4 header. The classifier maintains a hash table with exact-match rules. Thus, a table lookup can fail only if there is no static rule defined. An empty rule corresponds to best-effort traffic. As a result, on a lookup failure a packet is assigned to the best-effort service (default rule) and passed on for further processing. The classifier slow path component configures the hash table used by the classifier fast path component. 6-tuple classifier can be enabled or disabled at run time.
The three color (TC) meter implements two metering algorithms: single rate three color meter (SRTCM) and two rate three color meter (TRTCM).
The Single-Rate Three-Color Marker (SRTCM) meters an IP packet stream and marks its packets green, yellow, or red. Marking is based on a Committed Information Rate (CIR) and two-associated burst sizes, a Committed Burst Size (CBS) and an Excess Burst Size (EBS). A packet is marked green if it does not exceed the CIR; yellow if it does exceed the CBS, but not the EBS; and red, otherwise.
The Two-Rate Three-Color Marker (TRTCM) meters an IP packet stream and marks its packets green, yellow, or red. A packet is marked red if it exceeds the Peak Information Rate (PIR). Otherwise, it is marked either yellow or green depending on whether it exceeds or does not exceed the Committed Information Rate (CIR).
The DSPC marker updates the type of service (TOS) field in the IPv4 header and recomputes the IPv4 header checksum
The deficit round robin (DRR) scheduler schedules packets in a flexible queuing policy with priority concept. With this scheduler, the parameters are the number of sequential service slots that each queue can get during its service turn. The number of services for each queue depends on the value of its parameter called deficit factor. The deficit of queue is reduced as the scheduler schedules packets from that queue. The maximum deficit of each queue can be configured and is called weight of that queue. The DRR scheduler will schedule the packets by considering the packet size of the packet at the top of the queue. Queues are still served in round robin fashion (cyclically) in a preassigned order.
The Queue Manager performs enqueue and dequeue operations on the queues. The queue manager manages an array of queues, with each queue corresponding to a particular per hop behavior (PHB), for queuing packets per port. The Queue Manager receives enqueue requests from the IPv4/DiffServ pipeline. Upon receiving the enqueue request, the queue manager places the packet into the queue corresponding to the PHB indicated by the DSCP value in the packet.The Queue Manager maintains the state for each queue and uses the tail drop mechanism in case of congestion.
The Queue Manager receives the dequeue requests from the scheduler. The dequeue request consists of the PHB and the output port. Packets from the queue corresponding to this PHB and output port is dequeued and the dequeued packet is placed on the transmit queue for the output port.
To build the DiffServ application, specify the diffserv keyword on the build script command line. All files of the DiffServ data path implementation are located in the diffserv subdirectory of stc/app in the IP forwarding application. The DiffServ application requires an LDoms environment, as all configuration is through an application running on a Solaris control domain that communicates with the data plane application through IPC.
For example, to build the DiffServ application to make use of both NIU ports on an UltraSPARC T2-based system, use the following command:
The IPv4 Forwarding Information Base (FIB) table configuration (fibctl) command-line interface (CLI) has been extended to support configuration of DiffServ tables. This support behavior is the same as the FIB table configuration protocol over IPC between the control plane and data plane LDoms. Support is provided for configuring (choosing) the following DiffServ tables:
To build the extended control utility, issue the following command in the src/solaris subdirectory of the IP forwarding reference application.
This command is used to add the DSCP classifier entry in DSCP table.
diffserv dscp add DSCP_value port_number flow_id color_id class_id next_block
DSCP_value - DSCP value should be greater than 0 and less than 64.
port_number - Port number should be less than NUM_PORTS.
flow_id - ID used to identify the traffic flow to which the packet belongs.
color_id - ID should be green, yellow, or red.
class_id - ID used to identify the queue number within an output port.
next_block - Next block should be meter, marker, or fwder.
diffserv dscp add 1 0 1 green 1 meter
This command is used to delete DSCP classifier entry from DSCP table.
diffserv dscp delete DSCP_value port_number
DSCP_value - DSCP value should be greater than 0 and less than 64.
port_number - Port number should be less than NUM_PORTS.
This command is used to update the existing DSCP classifier entry in DSCP table.
diffserv dscp update DSCP_value port_number flow_id color_id class_id next_block
DSCP_value - DSCP value should be greater than 0 and less than 64.
port_number - Port number should be less than NUM_PORTS.
flow_id - ID used to identify the traffic flow to which the packet belongs.
color_id - ID should be green, yellow, or red.
class_id - ID used to identify the queue number within an output port.
next_block - Next block should be meter, marker, or fwder.
diffserv dscp update 1 0 1 yellow 1 fwder
This command is used to purge the DSCP table.
This command is used to display the DSCP table.
This command is used to add classifier 6-tuple entry in 6-tuple table.
diffserv class6tuple add SrcIp DestIp Proto Tos SrcPrt DestPrt IfNum flow_id color_id next_block class_id
SrcIp - Source IP address (for example, 192.168.1.5) in the IP header of packet.
DestIp - Destination IP address (for example, 192.168.1.5) in the IP header of packet.
Proto - IP protocol field in the IP header of packet.
Tos - Differentiated services code point (6 bits of TOS field).
SrcPrt - Source port number in the TCP/UDP header packet.
DestPrt - Destination port number in the TCP/UDP header packet.
IfNum - Input port starting form port 0, on which the packet is received.
flow_id - ID used to identify the traffic flow to which the packet belongs.
color_id - ID used to identify the packet drop precedence level (green, yellow, or red).
next_block - Used to identify the next packet processing block meter, marker, and fwder.
class_id - ID used to identify the queue number within an output port
(for example: ef, af0, af1, af2, af3, be).
diffserv class6tuple add 211.2.9.195 192.168.115.76 17 16 61897 2354 0 50 green meter 44
This command is used to delete 6-tuple classifier entry from 6-tuple table.
diffserv class6tuple delete SrcIp DestIp Proto Tos SrcPrt DestPrt IfNum
SrcIp - Source IP address (for example, 192.168.1.5) in the IP header of packet.
DestIp - Destination IP address (for example, 192.168.1.5) in the IP header of packet.
Proto - IP protocol field in the IP header of packet.
Tos - Differentiated services code point (6 bits of TOS field).
SrcPrt - Source port number in the TCP/UDP header packet.
DestPrt - Destination port number in the TCP/UDP header packet.
IfNum - Input port starting form port 0, on which the packet is received.
diffserv class6tuple delete 211.2.9.195 192.168.115.76 17 16 61897 2354 0
This command is used to update the existing 6-tuple classifier entry in 6-tuple table.
diffserv class6tuple update SrcIp DestIp Proto Tos SrcPrt DestPrt IfNum flow_id color_id next_block class_id
SrcIp - Source IP address (for example, 192.168.1.5) in the IP header of packet.
DestIp - Destination IP address (for example, 192.168.1.5) in the IP header of packet.
Proto - IP protocol field in the IP header of packet.
Tos - Differentiated services code point (6 bits of TOS field).
SrcPrt - Source port number in the TCP/UDP header packet.
DestPrt - Destination port number in the TCP/UDP header packet.
IfNum - Input port starting form port 0, on which the packet is received.
flow_id - ID used to identify the traffic flow to which the packet belongs.
color_id - ID used to identify the packet drop precedence level (green, yellow, or red).
next_block - Used to identify the next packet processing block meter, marker, and fwder.
class_id - ID used to identify the queue number within an output port
(for example: ef, af0, af1, af2, af3, be).
diffserv class6tuple update 211.2.9.195 192.168.115.76 17 16 61897 2354 0 50 red marker 44
This command is used to purge the 6-tuple table.
This command is used to display the 6-tuple table.
This command is used to enable/disable the 6-tuple table.
diffserv class6tuple enable/disable
This command is used to add a meter instance in TC meter table.
diffserv meter add flow_id CBS EBS CIR EIR green_dscp green_action yellow_dscp yellow_action red_dscp red_action meter_type stat_flag
flow_id - ID used to identify the traffic flow to which the packet belongs.
CBS - The value of the Committed Burst Size (CBS) is larger than 0, it is larger than or equal to the size of the largest possible IP packet in the stream. cbs is measured in bytes.
EBS - The value of the Excess Burst Size (EBS) is larger than 0, it is larger than or equal to the size of the largest possible IP packet in the stream. EBS is measured in bytes.
CIR - Committed Information Rate (CIR) at which a traffic source is signed up to send packets to the meter instance. It is measured in bytes/sec. The cir should be in M-bytes per seconds.
EIR - Excess Information Rate (EIR) at which a traffic source is signed up to send packets to the meter instance. It is measured in bytes/sec. This is used only when TRTCM is enabled. The eir should be in M-bytes per seconds.
green_dscp - DSCP packet mark value for green packets.
green_action - Select the next packet processing block for green packets (drop, fwder, and marker).
yellow_dscp - DSCP packet mark value for yellow packets.
yellow_action - Select the next packet processing block for yellow packets (drop, fwder, and marker).
red_dscp - DSCP packet mark value for red packets.
red_action - Select the next packet processing block for red packets (drop, fwder, and marker).
0 - Statistics Disable or 1 - Statistics Enable
diffserv meter add 1 1500 1500 1 1 12 marker 13 drop 14 drop 1 1
This command is used to delete a meter instance in TC meter table.
flow_id - ID used to identify the traffic flow to which the packet belongs.
This command is used to update a meter instance in TC meter table.
diffserv meter update flow_id CBS EBS CIR EIR green_dscp green_action yellow_dscp yellow_action red_dscp red_action meter_type stat_flag
flow_id - ID used to identify the traffic flow to which the packet belongs.
CBS - The value of the Committed Burst Size (CBS) is larger than 0, it is larger than or equal to the size of the largest possible IP packet in the stream. cbs is measured in bytes.
EBS - The value of the Excess Burst Size (EBS) is larger than 0, it is larger than or equal to the size of the largest possible IP packet in the stream. EBS is measured in bytes.
CIR - Committed Information Rate (CIR) at which a traffic source is signed up to send packets to the meter instance. It is measured in bytes/sec. The cir should be in M-bytes per seconds.
EIR - Excess Information Rate (EIR) at which a traffic source is signed up to send packets to the meter instance. It is measured in bytes/sec. This is used only when TRTCM is enabled. The eir should be in M-bytes per seconds.
green_dscp - DSCP packet mark value for green packets.
green_action - Select the next packet processing block for green packets (drop, fwder, and marker).
yellow_dscp - DSCP packet mark value for yellow packets.
yellow_action - Select the next packet processing block for yellow packets (drop, fwder, and marker).
red_dscp - DSCP packet mark value for red packets.
red_action - Select the next packet processing block for red packets (drop, fwder, and marker).
0 - Statistics Disable or 1 - Statistics Enable
diffserv meter update 1 1500 1500 1 1 12 marker 13 drop 14 drop 0 0
This command is used to purge meter table.
This command used to display the TC meter table.
This command used to display the TC meter statistics.
flow_id - ID used to identify the traffic flow to which the packet belongs.
This command is used to configure weight for all AF classes and maximum rate limit for EF class.
diffserv scheduler add output_port class_id weight
output_port - Port number should be less than NUM_PORTS.
class_id - ID used to identify the queue number within an output port
(for example: ef, af0, af1, af2, af3, be).
weight - Maximum number of bytes to be scheduled. If class is ef the weight will be bytes/seconds, otherwise, number of bytes.
diffserv scheduler add 1 af1 128
This command is used to update weight for all AF classes and maximum rate limit for EF class.
diffserv scheduler update output_port class_id weight
output_port - Port number should be less than NUM_PORTS.
class_id - ID used to identify the queue number within an output port
(for example: ef, af0, af1, af2, af3, be).
weight - Maximum number of bytes to be scheduled. If class is ef the weight will be bytes/seconds, otherwise, number of bytes.
diffserv scheduler update 1 af1 256
This command is used to display scheduler table entries.
diffserv scheduler display output_port
output_port - Port number should be less than NUM_PORTS.
TABLE 10-3 lists DiffServ references.
The Access Control List (ACL) reference application is integrated with the IP forwarding application. The ACL component classifies IPv4 packets using a set of rules. The classification can be done using the source and destination addresses and ports, as well as the protocol and the priority of the packet.
The algorithms (trie, bspl, and hicut) are used in the ACL library trade memory for speed. The rules are preprocessed to achieve a high lookup rate while using a lot of memory.
To build the IP packet forwarding (ipfwd) application with ACL, specify the acl keyword on the build script command line. For example:
The ipfwd application with ACL requires an LDoms environment, since all configurations are done through an application running on a Solaris control domain. The Netra DPS domain needs to be configured with at least 8 Gbytes of memory, which is a requirement for the ACL application.
1. Enable shared memory by adding the following line to the /etc/system file:
2. Enable the ACL communication channel between the Netra DPS domain and the Solaris control domain.
To do this, a special configuration channel must be set up between these domains. The channel is established as follows:
% ldm add-vdpcs shmem-server Netra DPS domain name % ldm add-vdpcc shmem-client shmem-server Solaris control domain name |
3. Add /opt/SUNWndpsd/lib to LD_LIBRARY_PATH.
The acltool is a command-line tool that sends commands to the ACL engine running in the Netra DPS domain. The interface is similar to iptables(8). The major difference is that it does not take a chain as a parameter. The acltool can be found in the /opt/SUNWndpsd/bin directory.
Following is a description of the various acltool commands and options.
--init algorithm Initializes ACL engine using algorithm for packet lookup.
--start Starts the packet classification.
--stop Stops the packet classification.
--status Prints the status of the ACL engine.
-c --config-file filename Reads rule commands from the configuration file.
-A --append rule Appends a rule.
-D --delete rule Removes the matching rule.
-F --flush Flushes (removes) all rules.
-p --protocol num Protocol (tcp, udp, icmp) or protocol number
-s --source ip[/mask] Source ip prefix
-d --destination ip[/mask] Destination ip prefix
-j --jump num Where to jump (action)
--sport num[:num] Source protocol port
--source-port num[:num] Source protocol port
--dport num[:num] Destination protocol port
--destination-port num[:num] Destination protocol port
The Radio Link Protocol (RLP) application (rlp) simulates Radio Link Protocol operation, which is one of the protocols in the CDMA-2000 High Rate Packet Data interfaces (HRPD-A). This application implements the forwarding direction fully, with packets flowing from PDSN --> AN --> AT (that is, Packet Data Serving Node to Access Network to Access Terminal). Reverse direction support is also implemented, but requires an AT side application which can generate NAKs (Negative Acknowledges). The application must be modified to process reverse traffic.
Copy the rlp reference application from the /opt/SUNWndps/src/apps/rlp directory to a desired directory location and do the build script in that location.
TABLE 10-4 shows the radio link protocol (rlp) application build script.
|
(See Argument Descriptions.) |
|
|
Build rlp application to run on e1000g Ethernet interface on the Netra ATCA CP3060 System. |
|
|
Build rlp application to run on Sun QGC (quad 1Gbps nxge Ethernet interface). |
|
|
Build rlp application to run on Sun multithreaded 10G (dual 10Gbps nxge Ethernet interface). |
|
|
Build rlp application to run on NIU (dual 10Gbps UltraSPARC T2 Ethernet interface) on a CMT2-based system. |
./build <cmt> <type> [ldoms] [arp] [profiler][-hash FLOW_POLICY]
This argument specifies whether to build the ipfwd application to run on the CMT1 (UltraSPARC T1) platform or CMT2 (UltraSPARC T2) platform.
cmt1 - Build for CMT1 (UltraSPARC T1)
cmt2 - Build for CMT2 (UltraSPARC T2)
Build the rlp application to run on e1000g Ethernet interface on a CMT1-based system.
Build rlp application to run on QGC (quad 1Gbps nxge Ethernet interface).
Build rlp application to run on 10Gb Ethernet (dual 10Gbps nxge Ethernet interface).
Build rlp application to run on NIU (dual 10Gbps UltraSPARC T2 Ethernet interface) on a CMT2-based system.
This is an optional argument specifying whether to build the rlp application to run on the LDoms environment. When this flag is specified, the rlp LDoms reference application will be compiled. If this argument is not specified, then the non-LDoms (standalone) application will be compiled. Refer to How Do I Calculate the Base PA Address for NIU/LDoms to Use with the tnsmctl Command?.
This is an optional argument to enable arp and can run only on the LDoms environment.
This is an optional argument that generate code with profiling enabled.
This is an optional argument used to enable flow policies. For more information, see Other RLP Options.
In ...src/apps/rlp, pick the correct build script and run it. For example, For example, to build for 10G Ethernet on a Netra or Sun Fire T2000 system, enter the following at your shell window:
In this example, the 10g option is used to build the RLP application to run on the Sun multithreaded 10Gb Ethernet. The <cmt> argument is specified as cmt1 to build the application to run on UltraSPARC T1-based Netra or Sun Fire T2000 systems.
|
1. Copy the binary into the /tftpboot directory of the tftpboot server, and perform.
2. On the tftpboot server, type:
3. On the target machine, type the following at the ok prompt:
| Note - net is an OpenBoot PROM alias corresponding to the physical path of the network. |
This section shows the default configurations.
Main files that control the system configurations are:
Main files that control the application configurations are:
To bypass the rlp operation (that is, receive --> transmit without rlp_process operation), uncomment the following line from Makefile.nxge for 1Gb PCIe ethernet adapter or Sun multithreaded 10 Gb and 1Gb PCIe Ethernet adapter, respectively:
| Note - This action disables the RLP processing operation only, the queues are still used. This is not the default option. |
By default, the RLP application uses a single global memory pool for all the DMA channels.
1. Enable the single memory pool by using the following flag:.
2. Update the rlp_swarch.c file to use individual memory pools.
The user can specify a policy for spreading traffic into multiple DMA flows by hardware hashing or by hardware TCAM lookup (classification). Refer to TABLE 10-2 for flow policy options.
The IPSec Gateway Reference Application implements the IP Encapsulating Security Payload (ESP) protocol using Tunnel Mode. This application allows two Gateways (or a host and a gateway) to securely send packets over an unsecure network with the original IP packet tunneled and encrypted (privacy service). This application also implements the optional integrity service allowing the ESP header and tunneled IP packet to be hashed on transmit and verified on receipt.
The design calls for six Netra DPS threads in a classic architecture where four threads are dedicated to packet reception and transmission (two receivers, two senders). In this architecture, a thread takes plain text packets and encapsulates and encrypts them, as well as a thread that de-encapsulates and decrypts. The architecture is shown in FIGURE 10-2.
FIGURE 10-2 IPSec Gateway Application Architecture
Refer to Request for Comments (RFC) documents for a description of IPSec and the ESP protocol:
The IPSec RFC refers to outbound and inbound packets. These design notes refer to these terms.
IPSec is a complex protocol. This application handles the following most common processing:
Contains the type of service to provide (privacy, integrity), crypto and hashing types and keys to be used for a session, among other housekeeping items. An item in the SADB is called a Security Association (SA). An SA can be unique to one connection, or shared among many.
A partial implementation that is used to contain “selectors” that designate what action should be taken on a packet based on the source and destination IP addresses, protocol, and port numbers.
A critical cache used to quickly look up the SA to use for packets coming from the plaintext side. The packet source and destination addresses and ports are hashed to find the action to take on the packet (discard, pass-through, or IPSec protect) and the SA.
A cache is used to quickly look up an SA for ESP packets entering the system from the ciphertext side. The Security Parameter Index is in the ESP header.
This IPSec implementation uses the ESP protocol (it does not currently handle AH, though ESP provides most of the AH functionality). Tunnel Mode is used to encapsulate (tunnel) IP packets between hosts and interface to a peer gateway machine.
The following describes functions of outbound and inbound packet processing:
The packet encapsulation and encryption code is straight-forward once you have a pointer to the SA. The SA contains the following information:
Packet encapsulation and de-encapsulation is just a matter of determining where the new IP header goes or where the original IP header is, building the new IP header, and invoking the crypto APIs on the correct packet location and length. For the IPSec implementation, you need to find the SA to use when a packet is received (either outbound on inbound). The user must use software hashing and hash table lookups for every packet. Note that when this is ported to Sun multithreaded 10G Ethernet on PCIe, the packet classification features speed-up this hashing.
The following sections describe how the SA is obtained for each packet.
The following sections describe how the SA is obtained for each packet.
The user must look at the packet selectors to determine what action to take, either DISCARD, PASS-THROUGH (as is), or PROTECT. The selectors are the source and destination IP addresses, the source and destination ports, and the protocol (TCP, UDP, and others).
The action to take is stored in the Security Policy Database (SPD). For this application, the complete SPD is not implemented. A static SPD exists that consists of rules that must be searched in order using the packet selectors.
For each selector (source IP, destination IP, source port, destination port, protocol), the rule states one of the following:
If all selectors match the rules, use the SP entry to determine what action to take. If it is PROTECTED (IPSec), the inbound and outbound Security Parameter Index (SPI) knows which SA to use. This implies the following:
The last rule in the SPD should be a catch-all that says DISCARD the packet.
The SPD structures and definitions can be found in spd.h.
The source code for the SPD can be found in spd.c.
The function used to lookup a rule is SPD_Search(), which is passed the selector values from the packet.
The above lookup is complex for every packet. Because of this, a cache called the SPD-Cache is maintained. The first time you lookup a particular connection, create a SPDC structure, hash the selectors, and place this SPDC in a hash table.
When packet that uses the exact combination of selectors comes in, it needs to be looked up in the SPDC hash table using the SPDC_HASH() function. If found, immediate access to the SA is made.
The definitions of this SPDC and the function can be found in sadb.h and sadb.c, respectively.
This application does not hash on the protocol type because UDP or TCP protocols types are assumed due to the presence of the source and destination ports in the packets.
The SPDC hash table is defined as:
The primary function used to lookup an SPDC entry is:
For this hash table, take the hash value, mask off the hash table size - 1, then index into this table to get an entry. The application then compares the entry for a match, and if there is not a match, the function will walk the chain until one is found.
Inbound IPSec packets contain an ESP header with an SPI. The application parses the SPI, hashes it using SPI_HASH_FROM_SPI(), looks it up in the SPI hash table, and accesses the SA pointer from there. The application cannot use the same hashing as done for outbound packets since the selectors (source/destination IP address and ports) have been encapsulated and encrypted. Decryption cannot be done until the SA is looked up.
The SPI hash table is defined as:
For the purposes of the application, statically define the test SPD and SAD in compile-time initialized C-code in the following C file: sa_init_static_data.c
This rule matches any source or destination IP address and protocol (TCP or UDP), and a source port of 6666 and a destination port of 7777. The load generator is set to send UDP packets with those ports. This needs to be changed if other ports are used.
These rules are added to the SPD at init-time (init_ipsec() calls sa_init_static_data()) through the following call: SPD_Add()
Two other functions are defined but not currently used: SPD_Delete() and SPD_Flush().
The SAD is also statically defined in sa_init_static_data.c. There are currently two SA entries: one for the outbound SA and one for the inbound SA. Only the outbound SA needs to be defined since the inbound SA is just a copy of the outbound SA, except for the SPI.
To perform various encryption and hashing scenarios, this SA entry is where the user needs to make changes, as shown below:
The first element to note is the service type. If the user wants to test privacy (encryption), leave INTEGRITY commented out. No hashing will be done. If the user wants hashing, comment in the #define for INTEGRITY.
The next fields you might change are the encryption parameters: encr alg, encr mode, encr key len, encr IV len, encr block len, and the encr key. The IV is only used for certain modes, such as CBC for AES.
It is important to ensure the proper key lengths and IV lengths in the table.
The user might then need to modify the hashing algorithms in a similar manner assuming you chose INTEGRITY.
Eventually, the SPD and SAD need to be integrated with a Control Plane (CP) such that the CP determines the static databases. There are two scenarios on how this takes place: download the tables and shared memory.
The CP uses the LDoms IPC mechanism to interface with Netra DPS to download (add) or modify the SPD and SA. Some functionality already exists to build these databases once the protocol is defined:
The CP sets up the tables in memory that is accessible from both the CP and Netra DPS and informs the Netra DPS application of updates through the LDoms IPC mechanism.
The main packet processing functions are called from the two processing threads which reside in ipsecgw.c.
The main plaintext packet processing thread is called PlaintextRcvProcessLoop() and it pulls a newly received packet out of a Netra DPS fast queue and calls:
IPSEC_Process_Plaintext_Pkt(mblk)
The main ciphertext packet processing thread is called CiphertextRcvProcessLoop() and it takes a packet off a fast queue and calls:
IPSEC_Process_Ciphertext_Pkt(mblk)
Find the IPSEC_Process_Plaintext_Pkt() and IPSEC_Process_Ciphertext_Pkt() functions in:
ipsec_proc.c
The following two functions perform the hashing and invoke the actual processing code:
The message block (mblk) contains pointers to the start and end of the incoming packets (b_rptr and b_wptr). Because plaintext packets must be prepended with a new outer IP header and ESP header, the user application should not shift the incoming packet data down which is a copy. Therefore, when the ethernet driver asks for a new receive buffer through teja_dma_alloc(), a buffer is grabbed from the receive buffer Netra DPS memory pool. The memory pool size is 2KB and the memory pool function returns an offset into that buffer which tells the driver where to place the packet data. This offset is set to 256 (MAX_IPSEC_HEADER), which is enough space to prepend the IPSec header information.
This section contains notes on how to calculate the location of the various parts of the ESP packet (outbound and inbound).
The IPSec Gateway uses the Netra DPS memory pools shown in table. The names and sizes are defined in ipsecgw_config.h:
The two main processing threads (PlaintextRcvProcessLoop and CiphertextRcvProcessLoop) are pipelined into two threads: one to perform most of the packet encapsulation and de-encapsulation, and the other to perform the encryption and decryption and optional hashing.
An extra fast queue is inserted in each path. For example, the pipeline for the eight threads configurations is shown as follows:
The two new threads (EncryptAndHash and HashAndDecrypt) reside in ipsec_processing.c rather than ipsecgw.c where the other threads reside.
The packet processing portion of this pipeline must pass the packet to the crypto part of the pipeline. Packets are normally passed on fast queues through the mblk pointer. Other vital information also needs to be passed, such as the SA pointer. Rather than allocation of a new structure to pass the data and the mblk (message block), this data is piggy-backed at the beginning of the receive buffer, which is not used. Refer to the cinfo structure defined in ipsec_processing.c.
The IPSec package comes with the following directories:
This directory consists of IPSec code that supports the Sun multithreaded 10G Ethernet on PCI-E or on-chip NIU in UltraSPARC T2.
This directory consists of Crypto API that interface to the Crypto hardware.
This directory consists of IPSec library functions.
./build <cmt> <type> [auth] [-hash FLOW_POLICY]
This is an argument specifying whether to build the IPSec Gateway application to run on the CMT1 platform or CMT2 platform.
cmt1 - Build for CMT1 (UltraSPARC T1)
cmt2 - Build for CMT2 (UltraSPARC T2)
This argument specifies the application type. Available application types are shown as follows.
Build the ipsecgw application to run Crypto on IP packets only (no IPSec protocol). This application configuration can be used to measure raw Crypto overheads.
Build the ipsecgw application to run on the Sun Multithreaded Quad Gigabit Ethernet.
Build the ipsecgw application to run one application instance on the UltraSPARC T2 10G Ethernet (NIU).
Build the ipsecgw application to run up to four application instances on the UltraSPARC T2 10G Ethernet (NIU).
Build the ipsecgw application to run up to eight application instances on the UltraSPARC T2 10G Ethernet (NIU).
Build the ipsecgw application to run up to eight application instances on the UltraSPARC T2 10G Ethernet (NIU).
This is an optional argument to apply Authentication (hashing protocol) to the packet stream along with Crypto. The hash algorithm is specified in the sa_init_static_data.c source file.
This is an optional argument used to enable flow policies. Refer to TABLE 10-2 for the Flow Policies for all flow policy options.
The file descriptions in the following tables are based on the files in the
ipsec-gw-nxge directory.
TABLE 10-6 lists the source files.
TABLE 10-7 lists the IPSec library files.
TABLE 10-8 lists the Crypto library files.
IPSec and Crypto has five reference application configurations:
This configuration can be used to evaluate the raw performance of the Crypto Engine. Two UltraSPARC T2 crypto engines are used. One for encryption, one for decryption.
This configuration implements one traffic flow on the PCIE Quad Gigabit Ethernet card.
This configuration runs one instance of IPSec Gateway Application on the NIU 10G Ethernet interface. Two UltraSPARC T2 Crypto engines are used. One for encrypt/hash, the other for hash/decrypt. This configuration is not yet supported on the Netra CP3260 platform.
This configuration implements multiple instances of IPSEC Gateway Application on the NIU interface through internal loopback. Eight UltraSPARC T2 Crypto engines are used. Four to perform encrypt/hash, four to perform decrypt/hash.
| Note - To build for running on Netra ATCA CP3260 systems, HASH_POLICY options are limited to the following policies: IP_ADDR, IP_DA, and IP_SA. |
- If <FLOW_POLICY> is TCAM_CLASSIFY
SA=69.235.4.0
DA=69.235.4.1 ~ 69.235.4.4 (increment by 1 and repeat every 4 counts)
| Note - This setting of the traffic generator applies to the Sun SPARC Enterprise T5120 and T5220 systems. For Netra ATCA CP3260 systems, see Flow Policy for Spreading Traffic to Multiple DMA Channels. |
| Note -
To build for Netra CP3260, in src/libs/ipsec/sa_init_static_data.c, the sa_outb1 remote_gw_mac must be set to the port address of the outgoing Ethernet port. |
| Note - In the application configuration file (for example, ipsecgw_niu_config.c), if port 0 is used, no action is required; if port 1 is used, add: ..., OPEN_OPEN, NXGE_10G_START_PORT+1, ... |
This configuration implements multiple instances of the IPSec Gateway Application on the NIU interfaces through back-to-back between two systems.
For Crypto only:
./build cmt2 niu_tunnel_in -hash <FLOW_POLICY>
For Crypto and Authentication:
./build cmt2 niu_tunnel_in auth -hash <FLOW_POLICY>
For Crypto only:
./build cmt2 niu_tunnel_out -hash TCAM_CLASSIFY
For Crypto and Authentication:
./build cmt2 niu_tunnel_out auth -hash TCAM_CLASSIFY
-If <FLOW_POLICY> is TCAM_CLASSIFY
SA=69.235.4.0
DA=69.235.4.1 ~ 69.235.4.8 (increment by 1 and repeat every 8 counts)
| Note - In the application configuration file (for example, ipsecgw_niu_config.c), if port 0 is used, no action is required; if port 1 is used, add: ..., OPEN_OPEN, NXGE_10G_START_PORT+1, ... |
The user can specify a policy for spreading traffic into multiple DMA flows by hardware hashing or by hardware TCAM lookup (classification). Refer to TABLE 10-2 for flow policy options.
Add the following into the gmake line:
whereas policy is one of the above specified policies.
For example, to enable hash on an IP destination and source address, run the build script with the following arguments:
| Note - If you specify FLOW_POLICY=HASH_ALL which is backward compatible with Sun SPARC Enterprise T5120/T5220 systems, all fields are used. |
If none of the policies in TABLE 10-2 are specified (do not specify the FLOW_POLICY in the above gmake line, for example, #FLOW_POLICY=HASH_IP_ADDR, a default policy will be given. When the default policy is used, all level (L2/L3/L4) header fields are used for spreading traffic.
This section explains how to compile Netra DPS traffic generator tool (ntgen), how to use the tool, and the options provided by this tool.
The traffic generator (ntgen) tool allows the generation of IPv4 UDP packets with options to modify various header fields in headers of all layers.
The traffic generator operates only with LDoms enabled. The user interface application runs in the Solaris LDoms and the ntgen tool runs in the Netra DPS domain.
The user interface application provides a template packet to ntgen with user-provided options for modifications. The traffic generator creates new packets using the template packet and applies the modifications as per the user options and transmits the packets.
The template packets are read by the user interface application from a snoop capture file (see the templates/ directory in the ntgen application directory.). This chapter explains the various options provided by the ntgen tool, methods on using ntgen, and compiling with the ntgen build script.
The ntgen control plane application is represented by the binary ntgen.
./ntgen [options ...] filename
See TABLE 10-9 for the list of options.
See Parameters Description for further descriptions and examples.
TABLE 10-9 lists the options for the ntgen control plane application. See -I for further descriptions and examples.
-D xx:xx:xx:xx:xx:xx
Change destination MAC address
Allows the user to change the destination MAC address of a packet. The destination MAC address is specified in the colon format.
ntgen -D aa:bb:cc:dd:ee:00 filename
-S xx:xx:xx:xx:xx:xx
Change source MAC address
Allows the user to change the source MAC address of a packet. The source MAC address is specified in the colon format.
ntgen -S 11:22:33:44:55:00 filename
-A xx.xx.xx.xx, yy.yy.yy.yy
Change IP addresses
Allows the user to change the source and destination IP addresses in the packet. The IP addresses are specified in the dotted decimal notation.
The first argument in the option is the source IP address. The second argument in the option is the destination IP address. An asterisk (*) can be specified for either the source IP address or the destination IP address to imply that no change needs to be done for that parameter.
In this example, the source IP address is changed to 192.168.1.1 and the destination IP address is changed to 192.168.2.1.
In this example, the source IP is changed to 192.168.1.10 and the destination IP is unchanged. The destination IP in this case is retained as it is in the template packet.
-p xx,yy
Change UDP port numbers
Allows the user to change the UDP source port and destination port numbers.
The first argument is the UDP source port number and the second argument is the UDP destination port number. An asterisk (*) can be specified for either the source port or the destination port to imply that no change needs to be done to that parameter. In that case, the value present in the template packet is retained.
In this example, the source port number is changed to 1111 and the destination port number is changed to 2222.
In this example, the source port number remains unchanged from its value in the template packet. The destination port number is changed to 2222 in the packets generated.
Allows the user to increase the UDP payload size. The value specified must be between 1 and 65536. The value denotes the number of bytes that needs to be added to the payload.
In this example, the UDP packet payload size gets incremented by 1024 bytes, that is, original size plus 1024 bytes is the new payload size.
| Note - Currently, the -P option is not supported for GRE encapsulated UDP datagrams. |
-V VLAN ID Start Value, VLAN ID End Value
Change VLAN IDs
Allows the user to create Ethernet frames with 802.1Q VLAN tags in the traffic packets. The Ethernet header of each packet that is generated is appended with a VLAN tag. The VLAN Identifier (VLAN ID) in the VLAN tags of the outgoing frames vary between VLAN ID Start Value and VLAN ID End Value. Two methods of VLAN ID variation are provided through the -iV option. When the -iV option is used with an argument of 1, the VLAN IDs are incremented. When the -iV option is used with an argument of 0, the VLAN IDs are decremented. See the explanation of the
-iV option for further details and examples.
In this example, Ethernet frames with VLAN tags are generated where the VLAN IDs in the VLAN tags of all frames are set to 100, that is, the VLAN ID start value. The VLAN IDs do not vary in this example since the -iV option is not used.
In this example, Ethernet frames with VLAN tags are generated where the VLAN IDs in the VLAN tags vary from 1 to 4094 in an incremental fashion.
In this example, Ethernet frames with VLAN tags are generated where the VLAN IDs in the VLAN tags vary from 1 to 4094 in a decremental fashion.
-k GRE Key Start Value, GRE Key End Value
Change GRE Keys
Allows the user to change the GRE key of GRE encapsulated packets in the range specified using this option. The GRE Key field in the generated packets will vary between the GRE Key Start value and the GRE Key End value. Two methods of the GRE Key variation are provided with the -ik option. When the -ik option is used with value 1, GRE Keys are incremented. When the -ik option is used with value 0, the GRE keys are decremented. See -ik 1/0 Increment or decrement GRE Key for further details.
In this example, GRE keys in the generated traffic start from 1 and increase to 1000.
In this example, GRE keys in the generated traffic start from 1000 and decrease to 1.
| Note - Only the file_gre_novlan template file can be used with this option. |
-iD xx:xx:xx:xx:xx:xx
Increment destination MAC address
Allows the user to specify the bytes in the destination MAC address that is specified using the -D option, to be incremented. The option is followed by the byte mask. An ff implies increment the byte and 0 implies no increment for that byte.
In this example, only byte 0 is incremented.
In this example, all bytes are incremented.
-iS xx:xx:xx:xx:xx:xx
Increment source MAC address
Allows the user to specify the bytes in the source MAC address that is specified using the -S option, to be incremented. The option is followed by the byte mask. An ff implies increment the byte and a 0 implies no increment for that byte.
In this example, only byte 0 is incremented.
In this example, all bytes are incremented.
-iA host/net/[pfx]/*, host/net/[pfx]/*
Increment the host or the network portion of an IP address
Allows the user to increment the source IP address and destination IP address (that were specified using the -A option) based on the IP address class or based on a prefix. The first argument corresponds to the source IP address of a packet. The second argument corresponds to the destination IP Address of a packet.
A class-based increment can be done by using the host or net arguments with the
-iA option. ntgen determines the class of IP address (class A, class B, class C or class D) that is specified with the -A option. From the class, it determines the length of the host part and the network part of the IP address. Then, based on the parameters passed through the -iA option, either the host part or the network part of the IP address is incremented. If an asterisk (*) is passed, then the IP address is not incremented.
The string net denotes that the network portion of the corresponding IP address needs to be incremented. The string host denotes that the host part of the IP address needs to be incremented.
Prefix-based increment can be done by providing the prefix length argument with the -iA option. The user provides a prefix length for each IP address (source and destination) as an arguments to the -iA option. This is used to calculate the portion of the IP address that must be incremented. If an asterisk (*) is passed, then the corresponding IP address is not incremented.
| Note - Currently, only 16 bits of an IP address can be incremented using either class-based or prefix-based methods. |
In this example, the network portion of the source IP address and the host portion of the destination IP address are incremented
In this example, the host portion of both the source and destination IP addresses are incremented.
In this example, the host portion of the source IP address is incremented. The destination IP address is not incremented.
In this example, the source IP address is incremented with a prefix length of 10 and the destination IP address is incremented with a prefix length of 12.
In this example, the source IP address is incremented with a prefix length of 10. The destination IP address is not incremented.
-ip 0/1, 0/1
Increment the UDP source and destination port
Allows the user to increment the UDP source port and destination port numbers. The first argument corresponds to the UDP source port and the second argument to the UDP destination port. 0 implies no increment and 1 implies that increment is needed.
In this example, the source port is not incremented, but the destination port is incremented.
In this example, both the source and destination ports are incremented.
-iV 1/0
Increment or decrement VLAN ID
Allows the user to increment or decrement VLAN IDs in the VLAN tag s of the generated Ethernet frames. Argument 1 denotes increment operation and a 0 denotes decrement operations.
The VLAN IDs are provided by the user using the -V option. For increment operation, the first VLAN ID is the VLAN ID Start Value that is provided in the -V option. Thereon, the VLAN ID is incremented for each subsequent frame until VLAN ID End Value provided with the -V option is reached. Then the VLAN ID wraps around to the VLAN ID Start Value and the sequence is repeated.
For the decrement operation, the first VLAN ID is the VLAN ID End Value that is provided with the -V option. Thereon, the VLAN ID is decremented for each subsequent frame until VLAN ID Start Value provided with the -V option is reached. Then the VLAN ID wraps around to the VLAN ID End Value and the sequence is repeated.
In this example, Ethernet frames are appended with a VLAN tag that contain VLAN ID in the range 100 to 200. The VLAN IDs are incremented for each frame starting at 100 until 200.
In this example, Ethernet frames are appended with a VLAN tag that contain VLAN ID in the range 200 to 100. The VLAN IDs are decremented for each frame starting at 200 until 100.
-ik 1/0
Increment or decrement GRE Key
Allows the user to increment or decrement GRE Keys in the GRE header of the generated GRE packets. An argument of 1 denotes increment operation and a 0 denotes decrement operation. The GRE Keys are provided using the -k option.
For the increment operation, the first GRE key is the GRE Key Start Value provided with the -k option. Thereon, the GRE Key is incremented for each subsequent packet until the GRE Key End value provided with the -k option is reached. The GRE Key then wraps around to the GRE Key Start Value and the sequence is repeated.
For the decrement operation, the first GRE key is the GRE Key End Value provided with the -k option. Thereon, the GRE Key is decremented for each subsequent packet until the GRE Key Start Value provided with the -k option is reached. The GRE Key then wraps around to the GRE Key End Value and the sequence is repeated.
In this example, GRE packets with key values in the range 1 to 100 are generated. The key value is incremented for each packet starting at 1 until 100.
In this example, GRE packets with key values in the range 100 to 1 are generated. The key value is decremented for each packet starting at 100 until 1.
-dD xx:xx:xx:xx:xx:xx
Decrement destination MAC address
Allows the user to specify the bytes in the destination MAC address, that is specified using the -D option, to be decremented. The option is followed by a byte mask. An ff implies decrement the byte and a 00 implies no decrement for that byte.
In this example, only byte 0 of the MAC address is decremented.
In this example, all bytes of the MAC address are decremented.
-dS xx:xx:xx:xx:xx:xx
Decrement source MAC address
Allows the user to specify the bytes in the source MAC address, that is specified using the -S option, to be decremented. The option is followed by a byte mask. An ff implies decrement the byte and a 00 implies no decrement for that byte.
In this example, only byte 0 of the MAC address is decremented.
In this example, all bytes of the MAC address are decremented.
-dA host/net/[pfx]/*, host/net/[pfx]/*
Decrement the host or the network portion of an IP address
Allows the user to decrement the source IP address and destination IP address (that were specified using the -A option) based on the IP address class or based on a prefix. The first argument corresponds to the source IP address of a packet. The second argument denotes the destination IP address of a packet.
A class-based decrement can be done by using the host or net arguments with the -dA option. ntgen determines the class of the IP address (class A, class B, class C or class D) that is specified using the -A option. From the class, it determines the length of the host part and the network part of the IP address. Then, based on the parameters passed through the -iA option, either the host part or the network part of the IP address is decremented. If an asterisk (*) is passed, then the IP address is not decremented.
The string net denotes that the network portion of the corresponding IP address needs to be decremented. The string host denotes that the host part of the corresponding IP address needs to be decremented.
Prefix-based decrement can be done by providing the prefix length argument with the -dA option. The user provides a prefix length for each IP address (source and destination) as an argument to the -dA option. This is used to calculate the portion of the IP address that needs to be decremented. If an asterisk (*) is passed, then the corresponding IP address is not decremented.
| Note - Currently, only 16 bits of an IP address can be decremented using either class-based or prefix-based methods. |
In this example, the network portion of the source IP address and the host portion of the destination IP address are decremented.
In this example, the host portion of both the source and destination IP addresses are decremented.
In this example, the host portion of the source IP address is decremented. The destination IP address is not decremented.
In this example, the source IP address is decremented using a prefix length of 10 and the destination IP address is decremented using a prefix length of 12.
In this example, the source IP address is decremented using a prefix length of 10. The destination IP address is not decremented.
-dp 0/1,0/1
Decrement the UDP source and destination port
Allows the user to decrement the UDP source port and destination port numbers. The first argument corresponds to the UDP source port and the second argument to the UDP destination port. A 0 implies no decrement and a 1 implies that decrement is needed.
In this example, the UDP source port is not decremented, but the destination port is decremented.
In this example, both the source and destination ports are decremented.
-c
Generate packets continuously
Allows the user to generate packets continuously.
In this example, the packets in the file are generated continuously without applying any modifications.
In this example, all the modifications pertaining to the options specified are applied and the packets are generated continuously.
-n <number of packets>
Generate a specified number of packets
Allows the user to specify the number of packets that need to be generated.
In this example, a million packets are generated.
-I
Ingress or Receive only mode
Allows the user to run the traffic generator in ingress mode. In this mode the traffic generator only receives packets, displays statistics about the ingress traffic and discards the received traffic. This option takes no arguments.
filename - The Snoop input file
Allows the user to specify a snoop file that contains the template packet to be used for creating the traffic packets. One of the files in the templates/ directory in the ntgen application directory can be used. These files contain packets whose fields can be modified with the ntgen tool options. These snoop files can be analyzed by the user using the snoop program in Solaris. The protocol header fields can be modified using the options provided by ntgen. A detailed explanation of the template snoop files is provided in Template Files.
| Note - Only one packet (the first) from a snoop is used by ntgen for generating traffic. |
The increment options (-iD, -iS, -iA and -ip) and the decrement options (-dD,
-dS, -dA and -dp) have effect only when the values that need to be incremented/decremented are also being modified.
For example, the following commands have no effect:
This command has no effect and the destination MAC address will not be incremented.
This command has no effect. The source and destination IP addresses will not be incremented.
This command has no effect. The port numbers will not be incremented.
The following commands will have effect:
This command increments the destination MAC address after changing it to aa:bb:cc:dd:ee:00. Since -D option is being used, the -iD option takes effect.
This command increments the source and destination IP addresses. Since the -A option is being used, the -iA option takes effect.
This command increments the source and destination UDP ports. Since the -p option is being used, the -ip option takes effect.
TABLE 10-10 shows an example of the traffic generator output.
TABLE 10-11 describes the traffic generator output.
|
Port is the port number and Chan is the channel number for which the statistics are displayed. In the example output shown in TABLE 10-10 for NxGE QGC, port varies from 0 to 3 and Chan is 0 for all ports. |
|
The following template files are provided with the application to be used with ntgen.
Snoop file that contains a single 64-byte Ethernet frame that has no VLAN tag. This file has a UDP/IPv4 payload.
Snoop file that contains a single 256 bytes Ethernet frame that has no VLAN tag. The file has a UDP/IPv4 payload.
Snoop file that contains a single 1514 bytes Ethernet frame that has no VLAN tag. This file has a UDP/IPv4 payload.
Snoop file that contains a GRE packet with an IPv4 as the delivery protocol and IPv4 as the payload protocol. The payload is a UDP datagram. The UDP datagram has a payload of 22 bytes. Both IPv4 headers have no IP options. GRE header consists of GRE Key and GRE checksum values.
This section describes configuring, starting, and stopping the ntgen tool.
TABLE 10-12 shows the domain role in the configuration.
TABLE 10-13 shows the LDC channels configured.
These LDC channels can be added with the following LDoms manager commands:
In the Solaris domains, the tnsm driver needs to be added. To do this,
1. Install the SUNWndpsd package.
2. Type the following to install the driver:
The primary-gc and tnsm-gc0 combination is the global configuration channel. LWRTE accepts configuration messages on this channel.
The config-tnsm-ldgx and config-tnsm0 combination is for setup messages between LWRTE and the control plane domain.
To find out what the LDC IDs are on both sides, use the following:
ldm list-bindings In ldg1: Vdpcs: config-tnsm-ldg2 [LDom ldg2, name: config-tnsm0] [LDC: 0x6] In ldg2: Vdpcc: config-tnsm0 service: config-tnsm-ldg2 @ ldg1 [LDC: 0x5] |
ldm list-bindings -e In ldg1: VDPCS NAME config-tnsm-ldg2 CLIENT LDC config-tnsm0@ldg2 6 In ldg2: VDPCC NAME SERVICE LDC config-tnsm0 config-tnsm-ldg2@ldg1 5 |
Pick a channel number to be used for the control IPC channel that uses this LDC channel, for example, 3. Bring up the control channel with the following command:
In the above tnsmctl command example,
Control channel 3 can now be used to set up general purpose IPC channels between LWRTE and Solaris.
For example, set up channel ID 4 for use by the ntgen to ndpstgen communication:
Look up the LDC IDs on both ends.
ldg1: Vdpcs: ldg2-vdpcs0 [LDom ldg2, name: vdpcc0] [LDC: 0x7] ldg2: Vdpcc: vdpcc0 service: ldg2-vdpcs0 @ ldg1 [LDC: 0x6] |
ldg1: VDPCS NAME ldg2-vdpcs0 CLIENT LDC vdpcc0@ldg2 7 ldg2: VDPCC NAME SERVICE LDC vdpcc0‘ ldg2-vdpcs0@ldg1 6 |
With this information, do the following in ldg3:
Note that the -C 4 parameter is the ID for the new channel, the -F 3 has the channel set up before.
Also, note that the global configuration channel between ldg3 and LWRTE comes up automatically as soon as the application is booted in LWRTE and the tnsm device driver is added in ldg3.
Build the ntgen utility in the Solaris subtree. Once the channel to be used is initialized using tnsmctl (must be channel ID 4 which is hard coded into the ndpstgen application), use ntgen to generate traffic (refer to the NTGEN User’s Manual).
2. Build the ntgen user interface application (in the src/solaris subdirectory)
1. Configure the primary domain.
2. Save the configuration (ldm add-spconfig) and reboot.
3. Configure the Netra DPS domain (including the vdpcs services).
4. Configure the Solaris domains (including vdpcc clients).
5. Bind the Netra DPS domain (ldg1).
6. Bind the Solaris domains (ldg2 and ldg3).
7. Start and boot all domains (can be in any order).
8. Install the SUNWndpsd package in the Solaris domains.
9. Load the tnsm driver in the Solaris domains (add_drv tnsm).
10. In the global configuration Solaris domain (ldg3), use /opt/SUNWndpsd/bin/tnsmctl to set up the control channel between the Netra DPS domain (ldg1) and the control domain (ldg2).
11. In the global configuration Solaris domain (ldg3), use /opt/SUNWndpsd/bin/tnsmctl to set up the ntgen control channel (channel ID 4).
12. In the control domain (ldg2), use the ntgen utility to start traffic generation.
Start the traffic generator tool by using the ntgen application binary.
The traffic generation can be stopped at any point of time by pressing Ctrl-C from the keyboard.
Copy the ntgen reference application from the /opt/SUNWndps/src/apps/ntgen directory to a desired directory location and do the build script in that location.
TABLE 10-14 shows the traffic generator (ntgen) application build script.
|
(See Argument Descriptions.) |
Build ntgen application to run on e1000g Ethernet interface. |
|
Build ntgen application to run on e1000g Ethernet interface on the Netra ATCA CP3060 System. |
|
|
Build ntgen application to run on Sun QGC (quad 1Gbps nxge Ethernet interface). |
|
|
Build ntgen application to run on Sun multithreaded 10G (dual 10Gbps nxge Ethernet interface). |
|
|
Build ntgen application to run on NIU (dual 10Gbps UltraSPARC T2 Ethernet interface) on a CMT2-based system. |
./build <cmt> <app> [profiler] [2port]
This is an argument specifying whether to build the traffic generator application to run on the CMT1 platform or CMT2 platform.
cmt1 - Build for CMT1 (UltraSPARC T1)
cmt2 - Build for CMT2 (UltraSPARC T2)
4g - Builds the traffic generator application to run on QGC (quad 1Gbps nxge Ethernet interface).
10g - Builds the traffic generator application to run on 10Gb Ethernet (dual 10Gbps nxge Ethernet interface).
10g_niu - Builds the traffic generator application to run on NIU (dual 10Gbps UltraSPARC T2 Ethernet interface) on a CMT2 based system.
This is an optional argument that generates code with profiling enabled.
This is an optional argument to compile dual ports on the 10Gb Ethernet card or the UltraSPARC T2 Network Interface Unit (NIU).
For example, to build for 10Gb Ethernet on the Netra T2000 system, type:
In this example, the build script is used to build the traffic generator application to run on the 10Gb Ethernet. <cmt> argument is specified as cmt1 to build the application to run on the Netra T2000 system which is an UltraSPARC T1-based system. The <app> argument is specified as 10g to run on 10Gb Ethernet.
At the target machine OK prompt, type:
The following is the default system configuration:
The main files that control the system configuration are:
The following is the default ntgen application configuration:
The main files that control the application configurations are:
The IPC reference application showcases the programming interfaces of the IPC framework (see Chapter 5, Interprocess Communication Software and the Netra Data Plane Software Suite 2.0 Update 2 Reference Manual).
The IPC reference application consists of the following three components:
The application is designed to run in an LDoms environment very similar to the one described in (see Example Environment for UltraSPARC T1 Based Servers and Example Environment for UltraSPARC T2 Based Servers).
The complete source code for the IPC reference application can be found in the SUNWndps package in the /opt/SUNWndps/src/apps/ipc_test directory. The source code files include the following:
build <cmt> [single_thread] | solaris
This argument specifies whether to build the ipc_test application to run on the CMT1 (UltraSPARC T1) platform or CMT2 (UltraSPARC T2) platform.
cmt1 - Build for CMT1 (UltraSPARC T1)
cmt2 - Build for CMT2 (UltraSPARC T2)
This argument is required to build the Netra DPS application.
With this option, two data IPC channels are polled by the same thread. In the default case, three channels are polled, each one on its own thread. The interfaces and usage for the Solaris side remain unchanged.
Build the Solaris user space application and the STREAMS module in their respective source directories.
The commands shown below build the Netra DPS application for single thread polling on a UltraSPARC T2 processor and the Solaris components, respectively.
In addition to the channels described in Example Environment for UltraSPARC T1 Based Servers, two IPC channels with IDs 5 and 6, respectively, need to be set up using the ldm and tnsmctl commands.
The Netra DPS application is booted from either a physical or a virtual network interface assigned to its domain. For example, if a tftp server has been set up in the subnet, and there is a vnet interface for the Netra DPS domain, the IPC test application can be booted with the following command at the OpenBoot Prom:
First, the ipc_test application must be booted in the Netra DPS domain, and the IPC channels to be used must be set up using the tnsmctl utility from the control domain. Copy the ipctest binary from the src/solaris/cmd directory to the Solaris domain (for example, ldg2 as shown in the Solaris user space application in src/solaris/cmd).
The ipctest utility drives a single IPC channel, which is selected by the connect command (see Commands). Multiple channels can be driven by separate instances of the utility, and the utility can be used at the same time as the STREAMS module (see Installing the lwmod STREAMS Module). In this case, however, the IPC channel with ID 5 is not available for this utility. For example, the utility can be used on channel 4 to read statistics of the traffic between the Netra DPS application and the Solaris module on channel 5.
The ipctest utility opens the tnsm driver and offers the following commands:
Connects to the channel with ID Channel_ID. The forwarding application is hard coded to use channel ID 4. The IPC type is hard coded on both sides. This command must be issued before any of the other commands.
Requests statistics from the ipc_test application and displays them.
Requests statistics from the ipc_test application for iterations times and displays the time used.
Send request to the Netra DPS to send num_messages messages with a data size of message_size and receive the messages.
Send num_messages messages with a data size of message_size to the Netra DPS domain.
Sends request to the Netra DPS to send num_messages messages with a data size of message_size and receive the messages. Also, spawns a thread that sends as many messages of the same size to the Netra DPS domain.
Contains program help information.
After copying the lwmod module from the rc/solaris/module/sparcv9 directory to the Solaris domain (for example, ldg2 as shown in the Solaris STREAMS module in src/solaris/module), the module needs to be loaded and inserted just above the driver for either a virtual or a physical networking device.
To use a physical device, modify the configuration such that the primary domain is connected through IPC channel 5, or, on an UltraSPARC T1-based system, assign the second PCI bus to ldg2.
| Note - Before inserting the module, the ipc_test application must have been booted in the Netra DPS domain, and the IPC channels must have been set up. |
Set up the module on a secondary vnet interface by entering the following sequence of commands:
After this sequence, the modules position can be displayed by entering the following command:
With the module installed, all packets sent to vnet1 will be diverted to the Netra DPS domain, where the application will reverse the MAC addresses and echo the packets back to the Solaris module. The module will transmit the packet on the same interface.
| Note - No packet will be delivered to the stack above the module. If networking to the domain is needed, the module should not be inserted in the primary interface. |
The module can be removed from the interface by entering the following command:
| Netra Data Plane Software Suite 2.0 Update 2 User’s Guide | 820-5211-11 |
|
Copyright © 2008 Sun Microsystems, Inc. All Rights Reserved.
To Run the ipfwd Application