This section contains guidelines for properly placing and connecting the disk shelves.
Cabinet Configuration
For safety reasons, mount the heaviest equipment, typically disk shelves, at the bottom of the cabinet. Refer to the appropriate Oracle Safety and Compliance Guide for rack-mounting guidelines.
To best prepare for cabling controllers to disk shelves, now and in the future, mount controllers in the middle of the cabinet. This also allows for optimal air circulation, whether in a floor- or ceiling-cooled environment.
Do not remove cabinet panels to run cables between cabinets.
Load Distribution
Balance the number of disk shelves across the disk chains in your system.
Balance the number of disk chains across the number of HBAs in your system. For example, attaching two chains to two separate HBAs will have better performance than attaching two chains to a single HBA.
Do not mix disks with different capacities or rotation speeds within a single disk shelf.
To maximize performance, use the maximum number of disk chains supported by the controller's SAS HBAs. For example, four SAS HBAs with eight chains and eight disk shelves will have better performance than two SAS HBAs with four chains and eight disk shelves.
To maximize performance, do not attach more than four DE2-24C, DE2-24P, or DE3-24C disk shelves to a single chain, and do not attach more than three DE3-24P disk shelves to a single chain. This only applies to systems that are performance critical. The maximum of six disk shelves per chain is available for cases where capacity is preferred over performance.
HBA Support for Disk Shelves
Disk shelves may require adding or replacing HBAs in your configuration. For HBA maintenance procedures for each controller, see Oracle ZFS Storage Appliance Customer Service Manual.
A SAS-2 HBA always operates at SAS-2 interface speeds, whether it is directly attached to DE2-24 or DE3-24 disk shelves, or whether the disk chain includes all DE2-24, all DE3-24, or an intermix of DE2-24 and DE3-24 disk shelves.
A SAS-3 HBA always operates at SAS-2 interface speeds if directly attached to a DE2-24 disk shelf.
A SAS-3 HBA always operates at SAS-3 interface speeds if directly attached to a DE3-24 disk shelf, or when the disk chain comprises all DE3-24 disk shelves.
Aggregate bandwidth should be based on SAS-2 interface speeds when mixing DE2-24 and DE3-24 disk shelves in a disk chain connected to a SAS-3 HBA.
Cable Lengths
The maximum cable length between DE3-24 disk shelves is three meters.
The maximum cable length is six meters between DE3-24 disk shelves in a single chain that spans two adjacent cabinets.
The maximum cable length between DE2-24 disk shelves is three meters.
The maximum cable length between Sun Disk Shelves is three meters.
The maximum cable length between a controller and disk shelves is six meters.
All-Flash Disk Shelves
Oracle ZFS Storage Appliance all-flash disk shelves contain only SSDs, and utilize all-flash pools (AFP). To maximize performance, observe the following guidelines:
Connect all-flash disk shelves to an exclusive SAS-3 HBA. Do not attach all-flash and non-all-flash disk shelves to the same SAS-3 HBA.
Connect a maximum of two all-flash disk shelves in a single chain.
Spread disk chains across as many SAS-3 HBAs as allowed by the system.
Segregate all-flash disk chains from non-all-flash disk chains. For example, do not mix all-flash DE3-24P disk shelves in a chain with DE3-24C disk shelves.
Only connect all-flash disk shelves to a SAS-3 HBA in the controller, and use SAS-3 qualified cabling (do not use SAS-2 cabling).
Disk Shelf Installation and Upgrade Considerations
Installing a new system - To maximize performance, avoid intermixing DE3-24 and DE2-24 disk shelves when installing a new system. Exclusively use a SAS-3 HBA, DE3-24 disk shelves, and associated SAS-3 cabling to take advantage of the high performance SAS-3 fabric.
Upgrading a system - When upgrading a system, choose the mixture of disk shelves according to your storage needs:
Upgrading an obsolete component - Replace the obsolete component and its associated components, which could be the SAS HBA, or disk shelf, and respective cabling. When possible for your system, add SAS-3 fabric components.
Segregating the upgrade for increased performance - If a controller has empty HBA slots, add SAS-3 HBAs (if supported by the controller model), DE3-24 disk shelves, and associated cabling.
Maximizing performance - Upgrade all components, possibly including controllers, to support an all DE3-24 disk shelf configuration with SAS-3 HBAs in the controller, and appropriate cabling.
Maximizing Performance with Intermixed DEx-24 Disk Shelves
To maximize performance when mixing DE3-24 and DE2-24 disk shelves, observe the following guidelines:
A disk chain should ideally contain only DE3-24 or DE2-24 disk shelves.
Attach SAS-3 HBAs to DE3-24 disk shelves, and attach SAS-2 HBAs to DE2-24 disk shelves.
Cabling for Intermixed DEx-24 Disk Shelves
Disk Chain Cables - The DE2-24 and DE3-24 disk shelves require different cable connectors. Select the correct cables according to the disk shelves being connected:
DE2-24 to DE2-24 disk shelf: SFF-8088 connectors on both cable ends
DE3-24 to DE3-24 disk shelf: SFF-8644 connectors on both cable ends
DE2-24 to DE3-24 disk shelf: SFF-8088 connector on DE2-24 cable end, SFF-8644 connector on DE3-24 cable end
HBA Cables - The different disk shelf types require different cables for connecting to a SAS-3 or SAS-2 HBA. Select the correct cables according to the disk shelves being connected:
SAS-3/SAS-2 HBA to DE3-24 disk shelf: SFF-8644 connectors on both cable ends
SAS-3/SAS-2 HBA to DE2-24 disk shelf: SFF-8644 connectors on HBA cable end, and SFF-8088 connector on disk shelf end
Intermixing DE2-24 and Sun Disk Shelves
For controllers that support using DE2-24 and Sun Disk Shelves together, the controller must use 4X4 port SAS-2 HBAs, which are supported as of software release 2013.1.0.
Do not use DE2-24 and Sun Disk Shelves in the same disk chain.