This appendix describes how to configure the IPC feature within the ChorusOS operating system to provide either local IPC communication, remote IPC communication over Ethernet, or remote IPC communication over the VME bus.
"Generic IPC Configuration" introduces the three different IPC configurations, and how to add them to the ChorusOS operating system.
"Specific IPC Configuration" describes remote IPC configuration in more detail.
The ChorusOS IPC feature is an optional component of the ChorusOS kernel which can be added in three different configurations:
local IPC. This configuration provides only local IPC communications.
local IPC + remote IPC. This configuration enables local and remote IPC communications to take place over a network data-link such as Ethernet or ATM (Asynchronous Transfer Method). This data-link is also called an external data-link, which means that the data-link driver is implemented within an independent driver outside of the kernel. It is also unreliable.
local IPC + remote IPC over the VME bus. This configuration allows local and remote IPC communications to take place over the VME bus.
The comand-line configuration tool configurator(1CC) is used to set up each configuration.
To configure the local IPC feature:
% configurator -set IPC=true |
To configure the local IPC + remote IPC feature:
% configurator -set IPC_REMOTE=true % configurator -set IPC_REMOTE_COMM=EXT |
To configure the local IPC + remote IPC feature over the VME bus:
% configurator -set IPC_REMOTE=true % configurator -set IPC_REMOTE_COMM=VME |
The IPC feature has the concept of a site number, a 32-bit unsigned integer which uniquely identifies a target board. Applications exchange messages through IPC ports, which are designated by a global identifier which includes the site number of the target board where the port is located.
The site number of a target is sent to the kernel at boot time in one of two ways:
dynamically, by the boot program, which sets the siteNumber field of the bootConf structure before invoking the kernel start entry
statically, by setting the chorusSiteId
kernel tunable in the ChorusOS system image built on the host:
% configurator -set chorusSiteId=n |
n is the site number assigned to the target board. This number can be specified in hexadecimal, by prefixing the number with 0x, or decimal.
When the site number is set dynamically, it is the responsibility of the boot program to determine the site number of the target. The method by which the site number is found by the boot program is fully boot dependent, and specific to the target board. It may, for example, be stored in NVRAM, dynamically generated from a unique board identifier. When the target is booted with the standard ChorusOS network boot monitor, the whole IP address used by the boot monitor is provided as the site number of the target.
When the site number is set statically, the site number is fixed within the system image. This approach is less flexible than the dynamic method because the same system image cannot be booted on similar target boards. A system image with a unique site number must be built for each target. For this reason, it should only be used when there is no way for the boot program to determine the site number of the target board.
The value of the site number set with the chorusSiteId
tunable takes precedence over the value of the site number provided by the
boot program.
The site number is set to zero by default. If the IPC_REMOTE
feature has been enabled, and the site number remains at zero,
the following message is displayed on the system console:
WARNING - LOCAL SITE ID. NOT SET => REMOTE IPC disabled
Only local IPC communications are enabled if the site number has not been set.
To configure the remote IPC over Ethernet feature, set the IPC_REMOTE
feature to true and the IPC_REMOTE_COMM
feature to EXT (mentioned
in "IPC Feature Configuration"). In addition, switch on the IOM_IPC
feature:
% configurator -set IOM_IPC=true |
This adds the Ethernet-specific module into the IOM component which acts as the IPC Ethernet data-link driver.
Once you have built and booted the ChorusOS system image on the target
board tgtbd1
, the IPC Ethernet
data-link can be dynamically started. Use the built-in ethIpcStackAttach command of C_INIT
(running on the target
board tgtbd1
) :
% rsh tgtbd1 ethIpcStackAttach ethernet-device-name |
The ethernet-device-name argument is the
name of your Ethernet device with which your remote IPC
stack will communicate. This name is the full pathname of the Ethernet device
in the target device tree, displayed on the target system console by the system
at boot time. For example, a genesis2
board with
a dec21140
Ethernet controller connected through
a raven
PCI bridge, has the pathname /raven/pci1011,9@e,0. This argument is only needed when the target
board has more than one Ethernet controller.
See the ethIpcStackAttach(1M) man page for more details.
The following example describes how to build a ChorusOS system image
for two similar PowerPC-based target boards, tgtbd1
and tgtbd2
, each with an Ethernet controller. Site
numbers must be unique and statically configured in each ChorusOS system
image.
Configure Remote IPC over Ethernet, if not already configured:
% configurator -set IPC=true % configurator -set IPC_REMOTE=true % configurator -set IPC_REMOTE_COMM=EXT % configurator -set IOM_IPC=true |
Assign a site number to tgtbd1
,
then build and uniquely identify the ChorusOS system image:
% configurator -set chorusSiteId=1 % make chorus % mv chorus.RAM chorus.RAM.tgtbd1 |
Assign a site number to tgtbd2
, then build
and uniquely identify the ChorusOS system image:
% configurator -set chorusSiteId=2 % make chorus % mv chorus.RAM chorus.RAM.tgtbd2 |
Once you have booted the chorus.RAM.tgtbd1 system image on tgtbd1
and the chorus.RAM.tgtbd2 system image on tgtbd2
,
run the ethIpcStackAttach command:
% rsh tgtbd1 ethIpcStackAttach % rsh tgtbd2 ethIpcStackAttach |
Applications which need to communicate through remote IPC
can now be launched on the tgtbd1
and tgtbd2
targets.
In current implementation of IPC over VME bus the following constraints must be satisfied:
The kernel must be configured for remote IPC feature over the VME bus (see "IPC Feature Configuration").
Each ChorusOS system image for every VME bus board must have the same logical and uniform view of all
the devices present on the VME bus, through their respective
device trees. As the device tree of each VME board will
be different, a different ChorusOS archive should be built and booted on
each target board. Typically, on genesis2
targets,
the file src/bsp/powerpc/genesis2/ src/boot/deviceTree.c
must be modified and recompiled for each different CPU
board involved in the IPC protocol.
IPC memory allocated to each CPU board must be in contiguous blocks of 64 Kilobytes on the VME bus.
The ChorusOS system image must be booted first on the VME bus system controller, and then on other (non-system controller) boards, in any order.
The following example describes what the device sub-tree representing the VME bus on each board should be. It assumes that your target consists of three VME boards in cage slots 0,1, and 2, your VME bus system controller is located in slot 0, 64Kb of memory is used on each board for IPC, and VME memory dedicated to IPC is allocated as follows:
Table A-1 VME memory dedicated to IPC
bridge I/O registers |
IPC memory |
|
---|---|---|
board 0: |
0x20000000-0x2000ffff |
0x20030000-0x2003ffff |
board 1: |
0x20010000-0x2001ffff |
0x20040000-0x2004ffff |
board 2: |
0x20020000-0x2002ffff |
0x20050000-0x2005ffff |
The device sub-tree representing the VME bus on each board is illustrated in Figure A-1.
The main differences between ChorusOS system images for each board are:
the properties associated with each VME bus controller device node
the presence of the BUSCOM_PROP_REMOTE property in each child node. This property selects which local
or remote instance of the bus communication (BUSCOM
) VME device driver will be started on each node.
Building and booting ChorusOS system images on a VME
target system is similar to the one described in the previous example. Embedding
only the genesis2
targets in a system image is achieved
as follows: