C H A P T E R 3 - Planning Your Storage Architecture

C H A P T E R 3

Planning Your Storage Architecture

There are two popular methods for connecting storage to servers.

Direct attached storage (DAS) uses a direct connection between a server and its storage system. The DAS solution of connecting each server to its own dedicated storage system is straightforward, and the absence of storage switches can minimize costs in some instances.

A storage area network (SAN) places a Fibre Channel (FC) storage switch between network servers and storage systems. A SAN solution shares a storage system among multiple servers using FC storage switches, and reduces the total number of storage systems required for a particular environment at the cost of additional element management (FC switches) and FC path complexity.

FIGURE 3-1 DAS and SAN Storage Architectures

Figure showing representations of Direct-Attached Storage (DAS) and Storage Area Network (SAN) configurations.

Selecting the best storage architecture for a particular environment can be a confusing exercise. In general, some environments are well-suited for DAS while others benefit greatly from SAN.

The challenge of selecting between DAS and SAN is often further complicated by the need to choose between different storage systems, one designed for DAS or another intended for SAN. Fortunately, Sun StorEdge 3510 FC arrays and Sun StorEdge 3511 SATA arrays inherently support both DAS Sun StorEdge 3510 FCnd SAN.

Direct-Attached Storage

One powerful feature of Sun StorEdge 3510 FC arrays and Sun StorEdge 3511 SATA arrays is their ability to support multiple direct-attached servers without requiring storage switches. They accomplish this by using intelligent internal Fibre Channel networks. Servers can be directly connected using built-in external Fibre Channel ports, if available, or add-in Fibre Channel host adapter cards.

Sun StorEdge 3510 FC arrays automatically configure ports to match the transfer speed and communication method of each connection.

Note - Some older 1-Gbyte FC HBAs do not correctly support current auto-negotiation. In such configurations, set the transfer speed to 1-Gbyte rather than Auto. Refer to the release notes for your array to see the HBAs supported for your host and any limitations. Refer to the Sun StorEdge 3000 Family RAID Firmware User's Guide for information about how to set transfer speed.

For Sun StorEdge 3511 SATA arrays, channels 0 and 1 automatically configure their ports to match the transfer speed and communication method of each connection. Channels 4 and 5 only support a 2-Gbyte transfer rate.

FIGURE 3-2 Two DAS Configurations

Figure showing standard DAS and high-availability DAS configurations.

The actual number of servers that can be connected varies according to the number of controllers in the array. It also depends on the number of Fibre Channel connections used for each server and the total number of small form-factor pluggable (SFP) interface modules installed. DAS configurations often include single or dual servers only, though a dual-controller array can support up to:

Four servers with redundant connections for Sun StorEdge 3510 FC arrays, or eight servers in non-redundant DAS configurations.

Six servers with redundant connections for Sun StorEdge 3511 SATA arrays, or 12 servers in non-redundant DAS configurations.

Note - Do not use a Sun StorEdge 3511 SATA array to store single instances of data. It is more suitable for use in configurations where the array has a backup or archival role.

Additional SFP modules are required to support more than two servers with redundant connections or four servers in non-redundant configurations. For information about obtaining and relocating SFP modules, refer to the Sun StorEdge 3000 Family Installation, Operation, and Service Manual for the Sun StorEdge 3510 FC Array and Sun StorEdge 3511 SATA Array.

Note - Except in some clustering configurations, in a DAS loop configuration when you connect two hosts to channel 0 (both FC 0 ports of either controller), or channel 1 (both FC 1 ports of either controller) on a Sun StorEdge 3511 SATA array, you must use host filtering if you want to control host access to storage. Refer to the user documentation for your clustering software to determine whether the clustering software can manage host access in this configuration.

Storage Area Networking

Combining storage switches with a Sun StorEdge 3000 series array configuration creates a SAN, increasing the number of servers that can be connected. Essentially, the maximum number of servers that can be connected to the SAN becomes equal to the number of available storage switch ports. Storage switches generally include the ability to manage and monitor the Fibre Channel networks they create, which can reduce storage management workloads in multiple server environments.

Sun StorEdge 3510 FC arrays and Sun StorEdge 3511 SATA arrays are designed to be deployed in SANs based on switched Fibre Channel fabrics. In a SAN scenario, the server HBAs are connected to one side of the fabric and storage is connected to the other. A SAN fabric automatically routes Fibre Channel packets between ports on one or many Fibre Channel switches.

SAN deployment enables Sun StorEdge 3510 FC arrays and Sun StorEdge 3511 SATA arrays to be used by a larger number of hosts. This storage strategy tends to utilize storage resources more effectively and is commonly referred to as storage consolidation.

The number of hosts that can effectively share one Sun StorEdge 3510 FC array or Sun StorEdge 3511 SATA array depends on several factors, such as the type of host application, bandwidth requirements, and the need for concurrent IOPs. Since most applications have moderate performance needs, it is quite feasible to have several hosts sharing the same Sun StorEdge 3510 FC or 3511 SATA array controller, with the following network characteristics:

Applications suitable for the Sun StorEdge 3511 SATA array might feature higher bandwidth and lower IOPS than those better suited to the Sun StorEdge 3510 FC array. Sharing a StorEdge 3511 SATA array effectively among several servers depends upon the division of performance among the applications installed on each server actively accessing the Sun StorEdge 3511 SATA array.

The Sun StorEdge 3510 FC Array is better suited for larger configurations with broader application requirements and higher IOPS.

A SAN can also support multiple Sun StorEdge 3510 FC arrays and Sun StorEdge 3511 SATA arrays. Increasing the number of arrays makes more performance and capacity available within a storage network for sharing among the servers connected to the SAN. A SAN also provides great flexibility in how storage capacity can be allocated among servers and eliminates cabling changes when reallocation of storage becomes necessary.

When a Sun StorEdge 3510 FC array or Sun StorEdge 3511 SATA array is deployed in a SAN, both point-to-point (full fabric) and arbitrated loop (public loop) modes are supported. Point-to-point mode enables superior full-duplex performance but limits the total number of addressable LUNs to 128, or to 64 when redundant pathing is used.

Scaling Capacity

Sun StorEdge 3510 FC arrays and Sun StorEdge 3511 SATA arrays are available in a number of configurations to address a broad range of storage capacities.

The Sun StorEdge 3510 FC array is available with 15,000 RPM FC disks to address high-performance requirements.

The Sun StorEdge 3511 SATA array is available in larger capacities to better address the needs of the bulk storage market. The additional storage capacity of the Sun StorEdge 3511 SATA array provides more storage for each server attached. Larger capacity disks do not effectively support large groups of servers or users, so spreading the extra capacity among a larger number of servers results in poor performance and, in extreme cases, possible loss of data.

Base systems include single or redundant controllers and a choice of five or twelve disks. Additional storage capacity can be dynamically created, starting with a system with five disks and then adding one or more disks. Expansion units can be dynamically added to base systems when more storage capacity is required than a single Sun array can provide.

Note - A Sun StorEdge 3510 FC array can be connected to as many as eight Sun StorEdge 3510 FC expansion units. Alternatively, Sun StorEdge 3510 FC arrays can be connected to as many as five Sun StorEdge 3511 SATA expansion units or combined 3510 and 3511 expansion units. Sun StorEdge 3511 SATA arrays, however, can only be connected to Sun StorEdge 3511 SATA expansion units. See Combining Sun StorEdge 3510 FC Arrays and Sun StorEdge 3511 SATA Expansion Units for more information.

Sun StorEdge 3510 FC arrays and Sun StorEdge 3511 SATA arrays remain single storage systems as expansion units are added, even though there are multiple interconnected physical units. Expansion units simply add bays to base units to increase the total number of disks that can be supported.

See Maximum Number of Disks and Maximum Usable Capacity per Logical Drive for maximum capacities per RAID configuration.

First Steps in Designing a Solution

There are two simple yet effective approaches for designing a Sun StorEdge 3510 FC array or Sun StorEdge 3511 SATA array solution into your environment. Both methods allow for the rapid estimation of an appropriate DAS or SAN solution. Regardless of which method is used, the storage needs of each application and server involved must be identified to establish the total amount of storage capacity required.

Designing a Storage Solution for an Existing Environment

The first method works well for existing environments. Start by identifying the number of servers that can immediately benefit from the storage a Sun StorEdge 3510 FC array or Sun StorEdge 3511 SATA array provides.

A Sun StorEdge 3510 FC array can support five or more servers. If there are four or fewer servers, a DAS solution is sufficient.

A Sun StorEdge 3511 SATA array can support seven or more servers. If there are six or fewer servers, a DAS solution is sufficient. If you connect two servers to channel 0 or to channel 1, use host filtering if you want to control host access to logical drives.

With either the Sun StorEdge 3510 FC array or the Sun StorEdge 3511 SATA array, a SAN solution can be a powerful option, even when the array is connected to a number of servers that can otherwise be supported in a DAS solution. Combining both the Sun StorEdge 3510 FC array and Sun StorEdge 3511 SATA array on the same SAN allows for a tiered storage strategy, using Sun StorEdge SAM-FS as the data mover among tiers. Determine how much storage is currently accessible to these servers and plan for that total capacity as the minimum amount of Sun StorEdge 3000 family disk capacity needed.

Designing a New Storage Solution

Another technique involves matching a particular environment to one of the best practices solutions described in this document. This approach works particularly well with new deployments, but it can be used for existing environments as well. Take note of special features, such as the number of connections between servers and storage. While these solutions do not match every environment exactly, use the closest one as a design blueprint that can be customized to suite your particular environment. For environments with different server configurations, choose the solution that best matches the servers whose applications are mission-critical or most important.

General Configuration Considerations

The entry-level configuration for a Sun StorEdge 3510 FC array or Sun StorEdge 3511 SATA array uses only one RAID controller. If this configuration is used, two single-controller arrays should use host-based mirroring to ensure high reliability, availability, and serviceability.

Note - Please refer to product documentation for VERITAS Volume Manager or an equivalent host mirroring application to set up the optimum configuration with single-controller arrays.

Use dual-controller arrays to avoid a single point of failure. A dual-controller FC array features a default active-to-active controller configuration. This configuration improved application availability because, in the unlikely event of a controller failure, the array automatically fails over to a second controller, resulting in no interruption of data flow. Single controller arrays are provided for small configurations requiring fast, scratch disk, as in EDA environments.

Sun StorEdge 3510 FC arrays and Sun StorEdge 3511 SATA arrays are extremely flexible, but when designing storage solutions remember to keep them as simple as possible. Keep the following suggestions in mind when designing the configuration of a Fibre Channel storage system:

To ensure power redundancy, connect the two power modules to two separate circuits, such as one commercial circuit and one UPS.

In a single-controller configuration, disable the Write-Back Cache feature to avoid the possibility of data corruption in the event of a controller failure. This will have a negative effect on performance. To avoid either issue, use dual controllers. You can also use the RAID firmware to create event triggers that temporarily disable the Write-Back Cache feature in the event of hardware failures or out-of-spec environmental conditions.

Prior to creating logical drives and mapping them to host channels, set the appropriate cache optimization, Fibre Channel protocol, and controller channel IDs. Reset the controller after these configuration parameters have been set.

For best performance and RAS, create logical drives across expansion units.

To avoid disruptions to other hosts sharing the same array, do not share a logical drive among multiple hosts.

Use either local or global spare drives when creating logical drives. Any free drive can be designated as a spare and more than one drive can be used as a spare.

Use dual pathing for each LUN and use Sun StorEdge Traffic Manager software to provide load balancing across controller ports for increased performance.

The maximum number of LUNs when using point-to-point protocol is 128 for single-path configurations and 64 for dual-path configurations.

Power-up the equipment in the following order:

a. Expansion units

b. RAID array

c. Host computers

Install Sun SAN Foundation software along with the latest patches and firmware. Note that the Solaris 10 operating system includes equivalent functionality, so no additional SAN Foundation software is necessary.

Install current Sun Storage Automated Diagnostic Environment (StorADE) software.

Connect the Ethernet management ports to a private Ethernet network (Sun Alert# 26464).

For security reasons, use the RAID firmware to assign a password to the RAID controller. Changing the firmware's Network Protocol Support settings can provide further security by disabling the ability to remotely connect to the array using individual protocols such as HTTP, HTTPS, telnet, FTP, and SSH.

Changing the firmware's Network Protocol Support settings can provide further security by disabling the ability to remotely connect to the array using individual protocols such as HTTP, HTTPS, telnet, FTP, and SSH.

After completing the configuration of a Sun StorEdge 3510 FC array or Sun StorEdge 3511 SATA array, the configuration should be saved using the firmware "Save nvram to disks" menu option or the Sun StorEdge Configuration Service Console's save configuration utility.

Partial or complete loss of data can be caused by the presence of media read error conditions on physical disk drives. The impact varies, depending on RAID level. To minimize availability issues caused by possible media read error conditions, enable Media Scan frequently (at least once every thirty days) on logical drives.