Sun Oracle Logo


Netra Data Plane Software Suite 2.1 Update 1 User’s Guide

User’s Guide

E20948-01



Contents

Using This Documentation

1. Sun Netra Data Plane Software Suite Overview

Product Description

Supported Systems

Software Installation

Platform Firmware Prerequisites

procedure iconsmall spaceTo Check Your OpenBoot PROM Firmware Version

Package Dependencies

Package Installation Procedures

procedure iconsmall spaceTo Install the Software Into the Default Directory

procedure iconsmall spaceTo Install the Software in a Directory Other Than the Default

procedure iconsmall spaceTo Remove the Software

Building and Booting Reference Applications

.cshrc File and Required Compiler Path

Building Reference Application Instructions

procedure iconsmall spaceTo Boot an Application Image

Programming Methodology

Reusing Existing C Code

tejacc Compiler Basic Operation

tejacc Compiler Mechanics

tejacc Compiler Options

tejacc Compiler Configuration

tejacc Compiler and Sun Netra DPS Interaction

Architecture Elements

Hardware Architecture API Overview

Hardware Architecture Elements

Architecture Relationships

Utility Functions

Advanced Hardware Architecture Elements

Software Architecture and Late-Binding API Overview

Late-Binding Elements

Other Elements

Utility Functions

User API Overview

Late-Binding API Overview

Sun Netra DPS Runtime API Overview

Finite State Machine API Overview

Map API Overview

2. tejacc Basics

Command-Line Options

tejacc Command-Line Options

Optimization

Optimization Options

Context-Sensitive Generation

procedure iconsmall spaceTo Enable Optimization

Language

Include Files

Late-Binding Object Identifiers

3. Profiler

Profiler Introduction

How the Profiler Works

Groups and Events

Profiler Output

Profiler Examples

Profiler API

Profiler API Usage for the Sun UltraSPARC T1 Processor

Profiler API Usage for the Sun UltraSPARC T2 Processor

Profiler Configuration

Profiler Output Example

Profiling Application Performance

Sun UltraSPARC T1 Performance Counters

Sun UltraSPARC T2 Performance Counters

User-Defined Statistics

Profiling Metrics

Using the Profiler Script

Profiler Scripts

Usage

Raw Profile Data

Summarized Profile Data

Sun UltraSPARC T1 Processor Profiler Output

Sun UltraSPARC T2 Processor Profiler Output

Performance Parameters Calculations

Sun UltraSPARC T1 Processor

Instructions per Packet:

Instructions per Cycle (IPC):

Packet Rate:

SB_full per thousand instructions:

FP_instr_cnt per thousand instructions:

IC_miss per thousand instructions:

DC_miss per thousand instructions:

ITLB_miss per thousand instructions:

DTLB_miss per thousand instructions:

L2_imiss per thousand instructions:

L2_dmiss_LD per thousand instructions:

Sun UltraSPARC T2 Processor

Instruction per Packet:

Instructions per Cycle (IPC):

Store Instructions per Packet:

Load Instructions per Packet:

L2 Load misses per Packet:

Icache misses per 1000 Packets:

Dcache misses per Packet:

Packet Rate:

procedure iconsmall spaceTo Use a Spreadsheet for Performance Analysis

4. Debugger

Debugger Introduction

Native Debugger

Debugging Configuration Code

Entering the Debugger

Native Debugger Commands

Displaying Help

help or h

Example:

Managing Breakpoints

break address or b address

info break or i break

Example:

delete breakpoint ID or d breakpoint ID

Managing Program Execution

cont or c

step or s

Example:

Displaying and Setting Memory

x/nfu address

w/u address value

Example:

Managing Threads

info threads or i threads

thread ID

Example:

Displaying Registers

info reg or i reg

Displaying Stack Trace

bt frame-count

Resolving Symbols Using Options

-h

-f function-name

-g global-variable

-l file-name:line-number

GNU Project Debugger

Configuring Oracle VM Server for SPARC Software for GDB Support

procedure iconsmall spaceTo Configure the Oracle VM Server for SPARC Software Required to Run the Sun Netra DPS Application With GDB Support

procedure iconsmall spaceTo Configure the Oracle Solaris Domain for GDB

GDB Showcase Application

procedure iconsmall spaceTo Compile the GDB Showcase

procedure iconsmall spaceTo Load the GDB Showcase Binary in the Sun Netra DPS Domain

procedure iconsmall spaceTo Run the GDB Command

GDB Commands

procedure iconsmall spaceTo Run Sun Netra DPS Application With GDB Support

5. Interprocess Communication Software

IPC Introduction

Programming Interfaces Overview

Configuring the Environment for IPC

Memory Management

IPC in the Logical Domains Environment

Logical Domain Channel Setup

IPC Channel Setup

Example Environment for UltraSPARC T1 Based Servers

Domains

primary

ldg1 - LWRTE

ldg2 - Control Plane Application

ldg3 - Solaris Control Domain

Virtual Data Plane Channels

Global Control Channel

Client Control Channel

Data Channel

IPC Channels

Example Environment for UltraSPARC T2 Based Servers

IPC Reference Applications

Common Header

6. Remote Command-Line Interface

Remote Command-Line Interface Introduction

IPC Setup for Remote CLI

procedure iconsmall spaceTo Configure the Oracle Solaris Domain for Remote CLI

Accessing the Remote CLI

procedure iconsmall spaceTo Access the CLI Console

Debugging Remotely

procedure iconsmall spaceTo Access the Sun Netra DPS Debugger

Coredump Support

System Configuration

procedure iconsmall spaceTo Go to the sys Mode From the Remote CLI

Compiling the Remote CLI Application

Build Script

Usage

Build Script Arguments

Argument Descriptions

cmt

[profiler]

7. Eclipse Development Environment

ADE Introduction

Starting the Eclipse-Based ADE GUI

procedure iconsmall spaceTo Start the Eclipse-Based ADE GUI

Creating a Teja Project

procedure iconsmall spaceTo Create a Project in the Same Directory as an Existing Teja Application

procedure iconsmall spaceTo Add the Graphic Files to a Project

Files and Viewers

Hardware Architecture Viewer

Software Architecture Viewer

Mapping Viewer

Build

procedure iconsmall spaceTo Compile the Teja Application in the Eclipse-Based ADE

8. Receive Packet Classification

Receive Packet Classification Introduction

Supported Networking Interfaces

Sun Multithreaded 10GbE and NIU Receive Packet Classifier

Receive DMA Channel Selection

Hashing Based on Layer 2, Layer 3, and Layer 4 Header Classification

Hash Algorithm

Hash Key

Application

Hash Policy

Flow Match Based on Layer 2, Layer 3, and Layer 4 Header Classification

Layer 2 (L2) Classification

Layer 3 and Layer 4 (L3/L4) Classification

Applications

Classification Programming Interface

opcode

action

flow_spec

channel

ue or um

hd

flow_spec_ipv4_tab_s

flow_spec_ipv6_t

flow_spec_l2_t

Examples

procedure iconsmall spaceTo Use Hash Flow

procedure iconsmall spaceTo Use TCAM Classification

9. Auto-Configuration

Auto-Configuration Introduction

Installation

Prerequisites

User Interface

Configuring a Logical Domain Environment for Reference Applications

Custom Configuring a Primary Domain

Custom Configuring a Guest Domain

Configuring LDC and IPC

Saving Current Guest Domains Configuration

Configuring the Oracle VM Server for SPARC Software from a Saved Location

10. Transparent Interprocess Communication

Transparent Interprocess Communication Introduction

TIPC Components

Installing TIPC

procedure iconsmall spaceTo Install TIPC

Programming Interfaces Overview

Configuring Environment for TIPC

SUNWndpsd and SUNWndps-tipc Binaries

Configuring Sun Netra DPS TIPC Stack from an Oracle Solaris Guest Logical Domain

Configuring Sun Netra DPS TIPC Stack from a Linux Guest Logical Domain

procedure iconsmall spaceTo Set the TIPC Address

Enabling TIPC Ethernet Bearer

Enabling the TIPC IPC Bearer

Enabling TIPC vnet Bearer for a NDPS TIPC Node

11. Reference Applications

IP Packet Forwarding Reference Applications

Receive Thread

Forwarding Thread

Transmit Thread

Traffic Flows

Source Files

procedure iconsmall spaceTo Compile the ipfwd Application

Usage

Argument Descriptions

procedure iconsmall spaceTo Build the ipfwd Application

procedure iconsmall spaceTo Run the ipfwd Application

Default System Configuration

Default ipfwd Application Configuration

Other IP Forwarder Options

IP Forward Static Cross Configuration

Flow Policy for Spreading Traffic to Multiple DMA Channels

ipfwd Flow Configurations

ipfwd Configuration File Format

System Configuration

Standalone Environment

Logical Domain Environment

Forwarding Application

Data Plane Components

Control Plane Components and Utilities

IPv4 Packet Forwarding Application with Exception Handling

ARP Processing

ARP in lwIP

ARP in the Oracle Solaris OS

ARP in the Oracle Solaris OS or Linux OS Using vnet

IPv4 Protocol Exception Handling

Fragmentation

Reassembly and Local Delivery

Reassembly and Local Delivery Using vnet

FIB Management

FIB Management When Using vnet

Exception Path Framework Components

IPv4 Forwarder (ipfwd Thread)

Exception Application (excpd)

lwIP ARP Layer

ARP STREAMS Module (lwmodarp)

The IPv4 STREAMS Module (lwmodip4)

Fastpath Manager

Exceptions Path Framework Tools

ifctl

fibctl

insarp

procedure iconsmall spaceTo Compile the ipfwd Application for IPv4 Exception Handling

procedure iconsmall spaceTo Compile the IPv4 Forwarding Application With Exception Handling By Using Sun Netra DPS

Compiling the excpd Application

Usage

procedure iconsmall spaceTo Build the excpd Application When lwIP ARP Is Used With IPC

procedure iconsmall spaceTo Build the excpd Application When lwIP ARP Is Used With TIPC

procedure iconsmall spaceTo Build the excpd Application When the Oracle Solaris OS ARP Is Used With IPC

procedure iconsmall spaceTo Build the excpd Application When the Oracle Solaris OS ARP Is Used With TIPC

Compiling the lwmodip4 STREAMS Module

Usage

procedure iconsmall spaceTo Build the lwmodip4 STREAMS Module for IPv4 Exception Handling Using IPC

procedure iconsmall spaceTo Build the lwmodip4 Module for IPv4 Exception Handling Using TIPC

Compiling the lwmodarp STREAMS Module

Usage

procedure iconsmall spaceTo Build the lwmodarp Module for Oracle Solaris ARP Handling Using IPC

procedure iconsmall spaceTo Build the lwmodarp Module for Oracle Solaris ARP Handling Using TIPC

Compiling the insarp Tool

procedure iconsmall spaceTo Compile the insarp Tool

procedure iconsmall spaceTo Run the ipfwd Application with IPv4 Exception Handling in lwIP

procedure iconsmall spaceTo Run the ipfwd Application with IPv4 Exception Handling and ARP Handling in the Oracle Solaris Host

procedure iconsmall spaceTo Compile the ipfwd Application with IPv4 Exception Handling using vnet in Sun Netra DPS

procedure iconsmall spaceTo Run the ipfwd Application with IPv4 Exception Handling and ARP Handling in an Oracle Solaris OS Host Using vnet

procedure iconsmall spaceTo Compile the IPv4 Forwarding Application With Exception Handling Using vnet in Sun Netra DPS

procedure iconsmall spaceTo Run the ipfwd Application with IPv4 Exception Handling and ARP Handling in the Linux Host Using vnet

IPv6 Packet Forwarding Application with Exception Handling

Interface Management

IPv6 Protocol Exception Handling

IPv6 Protocol Exception Handling Using vnet

FIB Management

FIB Management Using vnet Exception Handling

IP-IP Tunneling

Data-Plane and Control-Plane Synchronization

Exception Path Components

IPv6 Forwarder (ipfwd Strand)

IPv6 STREAMS Module (lwmodip6)

Fastpath Manager

Exception Path Tools

ifctl

fibctl

fibctl.sh

ipfwd_sync.d

procedure iconsmall spaceTo Compile the Reference Application

procedure iconsmall spaceTo Compile the IPv6 Forwarding Application With Exception Handling Using Sun Netra DPS

Compiling the lwmodip6 STREAMS module

procedure iconsmall spaceTo Build the lwmodip6 Module for IPv6 Exception Handling Using IPC

procedure iconsmall spaceTo Build the lwmodip6 Module for IPv6 Exception Handling Using TIPC

procedure iconsmall spaceTo Run the ipfwd Application With IPv6 Exception Handling

procedure iconsmall spaceTo Compile the IPv6 Forwarding Application With Exceptional Handling Using vnet

procedure iconsmall spaceTo Run the ipfwd Application With IPv6 Exception Handling

procedure iconsmall spaceRun the ipfwd Application That Is Compiled With Exception Handling

procedure iconsmall spaceTo Compile the IPv6 Forwarding Application Using vnet Exceptional Handling in a Linux Guest Logical Domain

procedure iconsmall spaceTo Run the ipfwd Application Using IPv6 Exception Handling in a Linux Guest Logical Domain

procedure iconsmall spaceRun the ipfwd Application That Is Compiled With Exception Handling

Differentiated Services Reference Application

Classifiers

Differentiated Services Code Point Classifier

6-Tuple Classifier

Policing (Meter)

Single-Rate Three-Color Marker

Two-Rate Three-Color Marker

DSCP Marker

Shaping

Deficit Round Robin Scheduler

Queue Manager

Building the DiffServ Application

DiffServ Command-Line Interface Implementation

procedure iconsmall spaceTo Build the Extended Control Utility

Command-Line Interface for the IPv4-DiffServ Application

DSCP Classifier

add

delete

update

purge

display

6-Tuple Classifier

add

delete

update

purge

display

enable or disable

TC Meter

add

delete

update

purge

display

stats

Scheduler

add

update

display

DiffServ References

Generic Routing Encapsulation Reference Application

Generic Routing Encapsulation Introduction

References

Data Plane Architecture

IPv4 Forwarding Data Plane

GRE Over IPv4 Data Plane

GRE Over IPv4 Data Plane Internal Block Diagram

GRE Over IPv4 Application

IPv4 Forwarder

GRE Encapsulator

GRE Decapsulator

Key and Sequence Number Extensions to GRE

GRE Command-Line Interface Implementation

Directory Structure

procedure iconsmall spaceTo Compile the GRE Code

procedure iconsmall spaceTo Compile the IPv4 and GRE Application Using Sun Netra DPS

procedure iconsmall spaceTo Compile the Command-Line Interface Application

procedure iconsmall spaceTo Run the IPv4 and GRE Application

procedure iconsmall spaceTo Run the CLI Application

CLI for the IPv4-GRE Application

add

update

delete

purge

display

GRE Reference Application Example

procedure iconsmall spaceTo Build the GRE Reference Application

Traffic Generator Configuration

Access Control List Reference Application

procedure iconsmall spaceTo Build the ACL Application

procedure iconsmall spaceTo Run the ACL Application

procedure iconsmall spaceTo Configure the ACL Application Environment Using LDC

procedure iconsmall spaceTo Configure the ACL Application Environment Using TIPC

Command-Line Interface for the ACL Application

procedure iconsmall spaceTo Use acltool in a Linux OS Control Domain

Radio Link Protocol Reference Application

procedure iconsmall spaceTo Compile the RLP Application

Build Script

Usage

Argument Descriptions

procedure iconsmall spaceTo Build the RLP Application

procedure iconsmall spaceTo Run the Application

Default System Configuration

Default RLP Application Configuration

Other RLP Options

procedure iconsmall spaceTo Bypass the rlp Operation

procedure iconsmall spaceTo Use One Global Memory Pool

Flow Policy for Spreading Traffic to Multiple DMA Channels

IPSec Gateway Reference Application

IPSec Gateway Application Architecture

IPSec Gateway Application Capabilities

High-Level Packet Processing

Outbound Packets

Inbound Packets

Security Association Database and Security Policy Database

Outbound Packets and Inbound Packets

Static Security Policy Database and Security Association Database

SPD

SAD

Packet Encapsulation and De-encapsulation

Packet Encapsulation

Memory Pools

Pipelining

Source Code File Description

Build Script

Usage

Argument Descriptions

Reference Application Configurations

IP with Encryption and Decryption

IPSec Gateway on Quad GE

IPSec Gateway on NIU 10-Gbps Interface (One Instance)

IPSec Gateway on NIU 10-Gbps Interface (Up to Four Instances)

Multiple Instances (Up to Eight Instances) Back-to-Back Tunneling Configuration

Flow Policy for Spreading Traffic to Multiple DMA Channels

procedure iconsmall spaceTo Enable a Flow Policy

Traffic Generator Reference Application

Using the User Interface

procedure iconsmall spaceTo Start the ntgen User Interface

ntgen Option Descriptions

Option Descriptions

ntgen Parameter Description

Notes

Traffic Generator Output

Template Files

Using the Traffic Generator

Configuring Logical Domains for the Traffic Generator

procedure iconsmall spaceTo Add the tnsm Driver

procedure iconsmall spaceTo Prepare Building the ntgen Utility

procedure iconsmall spaceTo Set Up and Use Logical Domains for the Traffic Generator

procedure iconsmall spaceTo Start the Traffic Generation

procedure iconsmall spaceTo Stop Traffic Generation

procedure iconsmall spaceTo Compile the Traffic Generator

Build Script

Usage

Argument Descriptions

procedure iconsmall spaceTo Run ndpstgen

Default Configurations

Interprocess Communication Reference Application

IPC Reference Application Content

Building the IPC Reference Application

Usage

Argument Descriptions

Example

Running the IPC Application

procedure iconsmall spaceTo Use the ipctest Utility

ipctest Commands

procedure iconsmall spaceTo Install the lwmod STREAMS Module

procedure iconsmall spaceTo Remove the lwmod STREAMS Module

Transparent Interprocess Communication Reference Application

Source Files

Default Configurations

procedure iconsmall spaceTo Compile the TIPC Application

Build Script

Usage

Argument Descriptions

procedure iconsmall spaceTo Run the TIPC Application

IP Forward Reference Application Using TIPC

procedure iconsmall spaceTo Build the IP Packet Forward (ipfwd) Application

procedure iconsmall spaceTo Configure the Environment for TIPC

procedure iconsmall spaceTo Configure Oracle Solaris OS TIPC Stack in Oracle Solaris Domain (ldg2)

Command-Line Interface Application using TIPC

procedure iconsmall spaceTo Build the Extended Control Utility

FIB Table Configuration Command Line Interface (fibctl)

Interface Configuration Command Line Interface (ifctl)

IPv4 Exception Process (excpd)

vnet Reference Application

UltraSPARC T2 Platform

UltraSPARC T1 Platform

Supported Tests

testvnet Commands

Test Setup

Virtual Network Setup

vnet Reference Application Content

Building the Sun Netra DPS vnet Reference Application

Usage

Argument Descriptions

procedure iconsmall spaceTo Build the vnet Reference Application

procedure iconsmall spaceTo Run the vnet Sun Netra DPS Application, vnettest

procedure iconsmall spaceTo Build the vnet Guest Logical Domain Application for the Oracle Solaris OS

procedure iconsmall spaceBuilding the vnet Guest Logical Domain Application for the Linux OS

procedure iconsmall spaceTo Run the vnet Guest Logical Domain Application on a Oracle Solaris OS Guest Logical Domain

procedure iconsmall spaceTo Run the vnet Guest Logical Domain Application on a Linux OS Guest Logical Domain

12. Performance Tuning

Performance Tuning Introduction

UltraSPARC T1 Processor Overview

UltraSPARC T2 Processor Overview

Identifying Performance Issues

UltraSPARC T1 Performance

UltraSPARC T2 Performance

Optimization Techniques

Code Optimization

Pipelining

Parallelization

Mapping

Parking Idle Strands

Slowing Down Polling

Tuning Troubleshooting

What Is a Compute-Bound Versus a Memory-Bound Thread?

Cannot Reach Line Rate for Packets Smaller Than 300 Bytes

Cannot Scale Throughput to Multiple Ports

How Do I Achieve Line Rate for 64-byte Packets?

When Should I Consider Thread Placement?

Example RLP Exercise

Application Configuration

Configuration 1

Configuration 2

Using the Profiling API

Profiling Data

Metrics

Configuration 1 Results

Configuration 2 Results

Analysis

Other Uses for Profiling

A. Tutorial

Application Code

Configuration Code

Build Process

procedure iconsmall spaceTo Create the Binary Image

Executing the Binary Image

procedure iconsmall spaceTo Execute the Binary Image

B. Frequently Asked Questions

Summary

General Questions

What Is Teja 4.x and How Does It Differ From an Ordinary C Compiler?

Where Are the Tutorials?

Configuration Questions

What Purpose Are the Hardware Architecture, Software Architecture, and Mapping Dynamic Libraries?

How Can I Debug the Dynamic Libraries?

procedure iconsmall spaceTo Debug the Dynamic Libraries

What Should I Do When the tejacc Compiler Crashes?

What if the Hardware Architecture, Software Architecture, or Mapping Dynamic Libraries Crash?

Can I Build Hardware Architecture, Software Architecture, and Mapping in the Same Dynamic Library?

Can I Map Multiple Variables With One Function Call?

Building Questions

Where Is the Generated Code?

Where Is the Executable Image?

How Can I Compile Multiple Modules on the Same Command Line?

How Can I Pass Different CLI Options to Different Modules on the tejacc Command Line?

How Can I Change the Behavior of the Generated makefile Without Modifying it?

How Do I Compile the Reference Applications?

Late-Binding Questions

What Is the Late-Binding API?

What Is a Memory Pool?

What Is a Channel?

What Is the Difference Between OS Based and Non-OS Based Memory Pools and Channels?

How Do I Access a Late-Binding Object From Application Code?

Can I Define a Symbol in the Software Architecture and Use it in My Application Code?

Eclipse Questions

How Can I Change the Build Command?

How Can I Change the Compiler Invocation Command?

API and Application Questions

How Do I Synchronize a Critical Region?

How Do I Send Data From a Thread to Another Thread?

How Do I Allocate Memory?

When Should I Use Queues Instead of Channels?

Why Is it Not Necessary to Block Interface or Queue Reads?

Can Multiple Strands on the Same Queue Take Advantage of the Extra CPU Cycles if the Strands Are Not Being Used?

Why Does the Application Choose the Role for the Strand From the Code Instead of the Software Architecture API?

Is It Possible to Park a Strand Under Logical Domains as Done in a Non-Logical Domains Environment?

Can You Assign Partial Cores to a Sun Netra DPS domain?

What Is bss_mem?

What Is the Significance of bss_mem Placement in the Code Listing?

How Are app.cmt2board.heap_mem0 and Similar Heaps Affected?

Can You Clarify BSS, Code, Heap, and DRAM Memory Allocation?

Does the eth_* API Support Virtual Ethernet (VNET) Devices?

How Do I Calculate the Base PA Address for NIU or Logical Domains to Use with the tnsmctl Command?

How Do I Modify the IP Forwarding Application to Use a New Classifier Type Instead of the Default UDP Type?

How Do I Add a New Packet Type to ntgen?

Optimization Questions

How Do I Enable Optimization?

What Is Context-Sensitive Generation?

What Is Global Inlining?

Legacy Code Integration Questions

How Can I Reuse Legacy C Code in a Sun Netra DPS Application?

Linking Legacy Code to Sun Netra DPS Code

Changing Legacy Source Code

How Can I Reuse Legacy C++ Code in a Sun Netra DPS Application?

Mixing C and C++ Code

Translating C++ Code to C Code

Sun CMT Specific Questions

Is There a Maximum Allowed Size for Text and BSS in My Program?

How Is Memory Organized in the Sun CMT Hardware Architecture?

How Do I Increase the Size of the DRAM membank?

Address Resolution Protocol Questions

How Do I Enable ARP in the RLP Application?

procedure iconsmall spaceTo Enable ARP in RLP

How Do I Enable ARP Without Relying on a Control Domain?

How Do I Enable ARP Using a Control Domain?

Oracle Solaris Domain and Sun Netra DPS Domain Question

How Do I Access kstat Information From the Oracle Solaris Domain for Network Interfaces That Are in Use by the Sun Netra DPS domain?

Traffic Generation

How Do I Stop Traffic Generation?

Oracle Solaris TIPC Application

What Should I Do When the Oracle Solaris TIPC Application Is Not Able to Create a Socket and Does a Core Dump?

Glossary

Index