Applications and I/O Processes, Colocate or Run Separately?

If you plan to configure only a subset of the nodes on a cluster as PFS I/O servers, you will have the option of either colocating applications and I/O daemons on the same PFS I/O servers or segregating them onto separate nodes. If, however, you configure all the nodes in a cluster as PFS I/O servers, you will of necessity colocate applications and PFS I/O daemons.

Guidelines for making this choice are provided below.

Conditions That Favor Colocating

Each of the following conditions favors colocating applications with PFS I/O daemons.

• Large nodes (many CPUs per node).

• Fast disk-storage devices (storage arrays, for example) on each node.

• Lower-performance cluster interconnect, such as 10- or 100-BaseT Ethernet.

• Small number of applications competing for node resources.

When these conditions exist in combination, the network is more likely to be a performance-limiting resource than the relatively more powerful nodes. Therefore, it becomes advantageous to locate applications on the PFS I/O servers to decrease the amount of data that must be sent across the network.

Conditions That Favor Separating Applications and IODs

You should avoid running applications on I/O server nodes when some or all of the following conditions exist.

• Smaller nodes.

• Slow disk storage devices (single disks, for example) on each node.

• Relatively high-performance cluster interconnect, such as SCI or ATM.

• Large number of applications competing for node resources.

In this case, the competition for memory, bus bandwidth, and CPU cycles may offset any performance advantages local storage would provide.

Effect of Cluster Size

By itself, the size of a cluster (number of nodes) does not favor either colocating or not colocating applications and PFS I/O daemons. Larger clusters do, however, attenuate the benefits of colocating. This is because the amount by which colocating reduces network traffic can be expressed as

    T_c = T_s - T_s/N

where T_c is the level of network traffic using colocating, T_s is the level of network traffic without colocating, and N is the number of nodes in the cluster. In other words, colocating reduces network traffic by 1/number-of-nodes. The more nodes there are in the cluster, the smaller the effect of colocating.