Appendix A Configuring IPC

This appendix describes how to configure the IPC feature within the ChorusOS operating system to provide either local IPC communication, remote IPC communication over Ethernet, or remote IPC communication over the VME bus.

• "Generic IPC Configuration" introduces the three different IPC configurations, and how to add them to the ChorusOS operating system.

• "Specific IPC Configuration" describes remote IPC configuration in more detail.

Generic IPC Configuration

IPC Feature Configuration

The ChorusOS IPC feature is an optional component of the ChorusOS kernel which can be added in three different configurations:

1. local IPC. This configuration provides only local IPC communications.

2. local IPC + remote IPC. This configuration enables local and remote IPC communications to take place over a network data-link such as Ethernet or ATM (Asynchronous Transfer Method). This data-link is also called an external data-link, which means that the data-link driver is implemented within an independent driver outside of the kernel. It is also unreliable.

3. local IPC + remote IPC over the VME bus. This configuration allows local and remote IPC communications to take place over the VME bus.

The comand-line configuration tool configurator(1CC) is used to set up each configuration.

To configure the local IPC feature:

% configurator -set IPC=true 

To configure the local IPC + remote IPC feature:

% configurator -set IPC_REMOTE=true
% configurator -set IPC_REMOTE_COMM=EXT

To configure the local IPC + remote IPC feature over the VME bus:

% configurator -set IPC_REMOTE=true
% configurator -set IPC_REMOTE_COMM=VME

Site Number Administration

The IPC feature has the concept of a site number, a 32-bit unsigned integer which uniquely identifies a target board. Applications exchange messages through IPC ports, which are designated by a global identifier which includes the site number of the target board where the port is located.

The site number of a target is sent to the kernel at boot time in one of two ways:

1. dynamically, by the boot program, which sets the siteNumber field of the bootConf structure before invoking the kernel start entry

2. statically, by setting the chorusSiteId kernel tunable in the ChorusOS system image built on the host:

   % configurator -set chorusSiteId=n

   n is the site number assigned to the target board. This number can be specified in hexadecimal, by prefixing the number with 0x, or decimal.

When the site number is set dynamically, it is the responsibility of the boot program to determine the site number of the target. The method by which the site number is found by the boot program is fully boot dependent, and specific to the target board. It may, for example, be stored in NVRAM, dynamically generated from a unique board identifier. When the target is booted with the standard ChorusOS network boot monitor, the whole IP address used by the boot monitor is provided as the site number of the target.

When the site number is set statically, the site number is fixed within the system image. This approach is less flexible than the dynamic method because the same system image cannot be booted on similar target boards. A system image with a unique site number must be built for each target. For this reason, it should only be used when there is no way for the boot program to determine the site number of the target board.

The value of the site number set with the chorusSiteId tunable takes precedence over the value of the site number provided by the boot program.

The site number is set to zero by default. If the IPC_REMOTE feature has been enabled, and the site number remains at zero, the following message is displayed on the system console:

WARNING - LOCAL SITE ID. NOT SET => REMOTE IPC disabled

Only local IPC communications are enabled if the site number has not been set.

Specific IPC Configuration

Remote IPC over Ethernet Data-link

To configure the remote IPC over Ethernet feature, set the IPC_REMOTE feature to true and the IPC_REMOTE_COMM feature to EXT (mentioned in "IPC Feature Configuration"). In addition, switch on the IOM_IPC feature:

% configurator -set IOM_IPC=true

This adds the Ethernet-specific module into the IOM component which acts as the IPC Ethernet data-link driver.

Once you have built and booted the ChorusOS system image on the target board tgtbd1, the IPC Ethernet data-link can be dynamically started. Use the built-in ethIpcStackAttach command of C_INIT (running on the target board tgtbd1) :

% rsh tgtbd1 ethIpcStackAttach ethernet-device-name

The ethernet-device-name argument is the name of your Ethernet device with which your remote IPC stack will communicate. This name is the full pathname of the Ethernet device in the target device tree, displayed on the target system console by the system at boot time. For example, a genesis2 board with a dec21140 Ethernet controller connected through a raven PCI bridge, has the pathname /raven/pci1011,9@e,0. This argument is only needed when the target board has more than one Ethernet controller.

See the ethIpcStackAttach(1M) man page for more details.

The following example describes how to build a ChorusOS system image for two similar PowerPC-based target boards, tgtbd1 and tgtbd2, each with an Ethernet controller. Site numbers must be unique and statically configured in each ChorusOS system image.

1. Configure Remote IPC over Ethernet, if not already configured:

   % configurator -set IPC=true % configurator -set IPC_REMOTE=true % configurator -set IPC_REMOTE_COMM=EXT % configurator -set IOM_IPC=true

2. Assign a site number to tgtbd1, then build and uniquely identify the ChorusOS system image:

   % configurator -set chorusSiteId=1 % make chorus % mv chorus.RAM chorus.RAM.tgtbd1

   Assign a site number to tgtbd2, then build and uniquely identify the ChorusOS system image:

   % configurator -set chorusSiteId=2 % make chorus % mv chorus.RAM chorus.RAM.tgtbd2

3. Once you have booted the chorus.RAM.tgtbd1 system image on tgtbd1 and the chorus.RAM.tgtbd2 system image on tgtbd2, run the ethIpcStackAttach command:

   % rsh tgtbd1 ethIpcStackAttach % rsh tgtbd2 ethIpcStackAttach

Applications which need to communicate through remote IPC can now be launched on the tgtbd1 and tgtbd2 targets.

Remote IPC over VME Bus

In current implementation of IPC over VME bus the following constraints must be satisfied:

• The kernel must be configured for remote IPC feature over the VME bus (see "IPC Feature Configuration").

• Each ChorusOS system image for every VME bus board must have the same logical and uniform view of all the devices present on the VME bus, through their respective device trees. As the device tree of each VME board will be different, a different ChorusOS archive should be built and booted on each target board. Typically, on genesis2 targets, the file src/bsp/powerpc/genesis2/ src/boot/deviceTree.c must be modified and recompiled for each different CPU board involved in the IPC protocol.

• IPC memory allocated to each CPU board must be in contiguous blocks of 64 Kilobytes on the VME bus.

• The ChorusOS system image must be booted first on the VME bus system controller, and then on other (non-system controller) boards, in any order.

The following example describes what the device sub-tree representing the VME bus on each board should be. It assumes that your target consists of three VME boards in cage slots 0,1, and 2, your VME bus system controller is located in slot 0, 64Kb of memory is used on each board for IPC, and VME memory dedicated to IPC is allocated as follows:

The device sub-tree representing the VME bus on each board is illustrated in Figure A-1.

Figure A-1 Device sub-tree representing the VME bus

The main differences between ChorusOS system images for each board are:

• the properties associated with each VME bus controller device node

• the presence of the BUSCOM_PROP_REMOTE property in each child node. This property selects which local or remote instance of the bus communication (BUSCOM) VME device driver will be started on each node.

Building and booting ChorusOS system images on a VME target system is similar to the one described in the previous example. Embedding only the genesis2 targets in a system image is achieved as follows:

• Edit the file src/bsp/powerpc/genesis2/src/boot/deviceTree.c.

  Change VME_MAX_BOARD to 3:

  #define VME_MAX_BOARD 3

• For each board:

  • Change LOCAL_CPU to 0 for board 0, 2 for board 2, or 3 for board 3, for example:

    #define LOCAL_CPU     0

  • Compile the BSP, and build a ChorusOS system image using make chorus.

  • Rename the ChorusOS system image to a file suffixed by the board number, for example chorus.RAM.0.

• Boot the system controller target first, using the system image suffixed with .0.

• Boot your other targets in any order, using the appropriate archive.