Best Practices for the Sun StorEdge 3320 SCSI Array and Sun StorEdge 3310 SCSI Array

This manual complements the Sun StorEdge 3000 Family Installation, Operation, and Service Manual for these products, collectively known as Sun StorEdge 3000 Family SCSI arrays. These arrays are ideal for small and large storage solutions for entry-level server environments such as file servers, application servers, email servers, database servers, and consolidation servers.

The Sun StorEdge 3320 SCSI array is a next-generation Ultra320 SCSI storage system designed to provide direct-attached storage (DAS) to entry-level servers, or a JBOD (an array with disks and no controller), designed to provide a high-performance, storage device that contains up to twelve disk drives with SCSI connectivity to the data host.

The Sun StorEdge 3310 SCSI array is an earlier Ultra160 version of the product that is no longer offered but is still supported.

Sun StorEdge 3000 Family SCSI arrays offer many performance and reliability, availability and serviceability (RAS) features using familiar SCSI technology, and can be used as-is or tailored to fit your needs.

Examples of customization opportunities include choosing RAID protection levels, selecting SCSI bus configurations, adding more disks and adding disk enclosures. A large number of potential storage solutions exist between the small and large recommendations. Choosing a configuration that fits within this range will provide the best results.

If you assign an IP address to an array in order to manage it out-of-band, for security reasons it is preferable that the IP address be on a private network, rather than a publicly routable network.

Since it is not possible to predict the side-effects of all current and future security software products, place your Sun StorEdge 3000 Family SCSI arrays on private subnets.

Sun StorEdge 3000 Family SCSI Array Features

On Sun StorEdge 3000 Family SCSI arrays, RAID controller channels 1 and 3 are designated host channels. Any of the host channels can be configured as a drive channel.

Sun StorEdge 3000 Family SCSI RAID controller channels 0 and 2 are dedicated drive channels that connect to expansion units. Each I/O board has two ports designated as disk drive loops. These ports connect to the internal dual-ported SCSI disk drives and are used to add expansion units to the configuration.

For more information about host and drive channels, refer to the Sun StorEdge 3000 Family RAID Firmware User's Guide.

Underlying Concepts and Practices

This section provides a brief overview of important concepts and practices that underlie the configurations you can use. These concepts and practices are described in more detail in other books in the Sun StorEdge 3000 family documentation set. Refer to the "Release Documentation" section of the Release Notes for your array for a list of those books.

Supported RAID Levels

Several RAID levels are available: RAID 0, 1, 3, 5, 1+0, 3+0, and 5+0. RAID levels 1, 3, and 5 are the most commonly used. Sun StorEdge 3000 family arrays support the use of both global and local spare drives in the unlikely event of disk failure. It is good practice to use spare drives when configuring RAID devices. Refer to the Sun StorEdge 3000 Family RAID Firmware User's Guide for detailed information about how RAID levels and spare drives are implemented.

Logical Drives

A logical drive (LD) is a group of physical drives configured with a RAID level. Each logical drive can be configured for a different RAID level.

Sun StorEdge 3000 Family SCSI arrays support a maximum of 16 logical drives. A logical drive can be managed by either the primary or secondary controller. The best practice for creating logical drives is to add them evenly across the primary and secondary controllers. The most efficient maximum configuration would have 8 logical drives assigned to each controller. With at least one logical drive assigned to each controller, both controllers are active. This configuration is known as an active-active controller configuration and allows maximum use of a dual-controller array's resources.

Supporting large storage capacities requires using maximum-sized logical drives. The largest supported logical drive configuration is determined by the size of your disk drives, cache optimization and the logical drive's RAID level. Maximizing existing logical drives before building a new logical drive is recommended. In the Sun StorEdge 3320 SCSI array, for example, this can result in a supported storage capacity of 4.68 Tbyte (16 logical drives x 300 Gbyte each = 4.68 Tbyte total capacity).

Each logical drive can be partitioned into up to 32 separate partitions or be used as a single partition. Partitions are presented to hosts as LUNs.

For information about maximum disk, logical drive, and array capacity, refer to the Sun StorEdge 3000 Family RAID Firmware User's Guide.

Once the logical drives have been created, assigned to a controller, and partitioned, the partitions must be mapped to host channels as LUNs in order for them to be seen by a host. It is usually desirable to map each partition to two host channels for redundant pathing.

A partition can only be mapped to a host channel where its controller has an assigned ID. For example, if LD 0 is assigned to the primary controller, all partitions on LD 0 will need to be mapped to a host channel ID on the primary controller (PID). Any logical drives assigned to the secondary controller will need to have all partitions mapped to a host channel ID on the secondary controller (SID).

Saving and Restoring Configuration Information

An important feature of these management tools is the ability to save and restore configuration information in a number of ways. Using the array's firmware application, the configuration information (NVRAM) can be saved to disk. This provides a backup of the controller-dependent configuration information such as channel settings, host IDs, and cache configuration. It does not save LUN mapping information. The NVRAM configuration file can restore all configuration settings but does not rebuild logical drives.

The Sun StorEdge Configuration Service software can be used to save and restore all configuration data, including LUN mapping information. It can also be used to rebuild all logical drives and therefore can be used to completely duplicate an array's configuration to another array.

Array Management Tools

Sun StorEdge 3000 family arrays use the same management interfaces and techniques. They can be configured and monitored through any of the following methods:

Using the out-of-band serial port connection (RAID only), a Solaris tip session or terminal emulation program for other supported operating systems can be used to access an array's internal firmware application. All procedures can be performed by using the firmware's terminal interface via the COM port.

Using the out-of-band Ethernet port connection, telnet can be used to access the firmware application. All procedures except the initial assignment of an IP address can be done through an Ethernet port connection. Refer to the Sun StorEdge 3000 Family Installation, Operation and Service Manual for your array for more information.

Using the out-of-band Ethernet port connection or in-band connection, Sun StorEdge Configuration Service or the Sun StorEdge CLI can configure and manage an array from a host system. Sun StorEdge Configuration Service provides a graphical user interface (GUI) that displays information about multiple aspects of the system at a glance. The main advantages of the CLI are that commands can be scripted and information can be passed to other programs.

Note - To set up and use Sun StorEdge Configuration Service, refer to the Sun StorEdge 3000 Family Software Installation Guide. The CLI is installed as part of the SUNWsccli package. Information about CLI functionality can be found in the Sun StorEdge 3000 Family CLI User's Guide, and in the sccli man page once the package is installed.

Note - Do not use both in-band and out-of-band connections at the same time to manage the array. Otherwise conflicts between multiple operations might occur.

Direct-Attached Storage

One powerful feature of Sun StorEdge 3000 Family SCSI arrays is their ability to support multiple direct-attached servers without requiring storage switches. Servers can be directly connected using built-in external SCSI ports, if available, or add-in SCSI host-bus adapter cards.

Sun StorEdge 3000 Family SCSI arrays automatically configures ports to match the transfer speed and communication method of each connection.

The actual number of servers that can be connected depends on the number of SCSI connections used for each server. Configurations can support:

Two servers with redundant connections for Sun StorEdge 3320 SCSI arrays

Three servers in non-redundant configurations

Dual-Controller Configurations

Using redundant connections between SCSI arrays and servers provides failover protection in the event a connection becomes unreliable or fails.

Single-Controller Configurations

Using single connections between SCSI arrays and servers creates single points of failure (SPOF) that can cause interruptions in the event a connection becomes unreliable or fails. This is not a recommended configuration unless host-based mirroring is utilized to protect against single points of failure. Similarly, using only a single controller creates a single point of failure, unless single controllers are used in pairs and mirrored. Using a dual-controller configuration is preferable to using a single controller or a pair of single controllers.

Tips and Techniques

A Sun StorEdge 3000 Family SCSI array with a single controller can be configured to support up to three host connections. These connections can be used in pairs, individually, or in any combination of both.

This single-controller configuration offers no redundancy. Consider mirroring single controllers or using dual controllers instead to achieve redundancy and increase reliability, availability, and serviceability. See General Configuration Considerations for more information.

Scaling Capacity

Sun StorEdge 3000 Family SCSI arrays are available in a number of configurations to address a broad range of storage capacities.

Base systems include single or redundant controllers and a choice of five or twelve disks. This results in storage capacities as small as:

0.175 Tbyte with five 36-Gbyte disks (3310 only).

Capacities can be as large as:

3.51 Tbyte with twelve 300-Gbyte disks.

Additional storage capacity can be dynamically created, starting with a system with five disks and then adding one or more disks. Sun StorEdge 3000 Family SCSI arrays remain single storage systems as expansion units are added, even though there are multiple interconnected physical units. Expansion units simply add bays to base units to increase the total number of disks that can be supported.

Sun StorEdge 3000 Family SCSI arrays can support two expansion units for a total of 36 300-Gbyte disks, providing a maximum of storage capacity of 10.54 Tbyte.

For information about maximum disk, logical drive, and array capacity, refer to the Sun StorEdge 3000 Family RAID Firmware User's Guide.

Caution - For optimum and successful use of each array, check that you are using the array with the correct applications.

First Steps in Designing a Solution

Two simple yet effective approaches exist for designing a Sun StorEdge 3000 Family SCSI array solution into your environment. Both methods allow for the rapid estimation of an appropriate solution. Regardless of which method is used, the storage needs of each application and server involved must be identified to establish the total amount of storage capacity required.

When Designing a Storage Solution for an Existing Environment

The first method works well for existing environments. Start by identifying the number of servers that can immediately benefit from the storage the array provides.

When Designing a New Storage Solution

Another technique involves matching a particular environment to one of the best practices solutions described in this document. This approach works particularly well with new deployments, but it can be used for existing environments as well. Take note of special features, such as the number of connections between servers and storage. While these solutions do not match every environment exactly, use the closest one as a design blueprint that can be customized to suite your particular environment. For environments with different server configurations, choose the solution that best matches the servers whose applications are mission-critical or most important.

General Configuration Considerations

The entry-level configuration for a SCSI array uses only one RAID controller. If this configuration is used, two single-controller arrays should use host-based mirroring to ensure high reliability, availability, and serviceability (RAS).

Note - Refer to VERITAS Volume Manager or an equivalent host mirroring application to set up the optimum configuration with single-controller arrays.

Use dual-controller arrays to avoid a single point of failure. A dual-controller SCSI array features a default active-to-active controller configuration. This configuration improved application availability because, in the unlikely event of a controller failure, the array automatically fails over to a second controller, resulting in no interruption of data flow. Single-controller arrays are provided for small configurations requiring fast, scratch disk, as in EDA environments.

Sun StorEdge 3000 Family SCSI arrays are extremely flexible, but when designing storage solutions keep them as simple as possible. Keep the following suggestions in mind when designing the configuration of a SCSI storage system:

To ensure power redundancy, connect the two power modules to two separate circuits, such as one commercial circuit and one UPS.

In a single-controller configuration, disable the Write-Back Cache feature to avoid the possibility of data corruption in the event of a controller failure. This will have a negative effect on performance. To avoid either issue, use dual controllers.

Using two single-controllers in a clustering environment with host-based mirroring provides some of the advantages of using a dual controller. However, you still need to disable the Write-Back Cache in case one of the single controllers fails and you want to avoid data corruption. For this reason, a dual-controller configuration is preferable.

Prior to creating logical drives and mapping them to host channels, set the appropriate cache optimization and controller channel IDs. Reset the controller after these configuration parameters have been set.

For best performance and RAS, create logical drives across expansion units.

To avoid disruptions to other hosts sharing the same array, do not share a logical drive among multiple hosts.

Use either local or global spare drives when creating logical drives. Any free drive can be designated as a spare and more than one drive can be used as a spare.

The maximum number of LUNs is 128.

Power-up the equipment in the following order:

a. Expansion units

b. RAID array

c. Host computers

Install Sun Storage Automated Diagnostic Environment (StorADE) software.

Connect the Ethernet management ports to a private Ethernet network.

After completing the configuration of the array, save the configuration using the firmware "Save nvram to disks" menu option and the Sun StorEdge Configuration Service Console's "save configuration" utility.

Partial or complete loss of data can be caused by the presence of media read error conditions on physical disk drives. The impact varies, depending on RAID level. To minimize availability issues caused by possible media read error conditions, enable Media Scan frequently (at least once every thirty days) on logical drives.

Single Processor Server Applications

Print, file and application services are essential network requirements and have become among the most popular uses of entry-level servers. The servers used to provide these functions are generally very inexpensive, highly compact units that are often installed in racks for convenience.

One example of such a server is the Sun Fire V120, an expandable single-processor server that occupies just 1 rack-unit (1U) of rack space. Network servers are often distributed throughout an enterprise, making Sun StorEdge 3000 Family SCSI arrays ideally suited for these applications due to their cost-effective direct-attached storage design.

TABLE 1 Storage Requirements for Single Processor Servers
	Print Server	File Server	Application Server
Availability	Medium	Medium to high	Medium to high
Storage Capacity	Low	Low to high	Low to medium
Special Needs	High data rates and low cost	High data and transaction rates	Low cost and high transaction rates
Access Pattern	Sequential	Sequential	Random

Multiple Processor Server Applications

Another popular use of entry-level servers is for providing mail, database and other services to workgroup, departmental and branch office users. The server and storage solutions used to provide these sophisticated services must be able to scale in application performance and storage capacity to keep pace with the needs of increasing number of network users.

The Sun Fire V480 Server and StorEdge 3310 SCSI array are excellent examples of highly scalable solutions and combine to form a powerful solution. Performance scales by adding server memory and processors while capacity scales by adding enclosures and disks.

TABLE 2 Storage Requirements for Multiple Processor Servers
	Mail Server	Database Server	Consolidated Server
Availability	High	High	High
Storage Capacity	Medium to high	Medium to high	Medium to high
Special Needs	High transaction and data rates	High transaction and data rates	High transaction and data rates
Access Pattern	Random	Random	Random

Best Practices: Print Servers

The following section outlines small and large storage solutions for print server environments. FIGURE 1 shows the scalability between the print server and Sun StorEdge 3000 Family SCSI array.

Print Server Architecture and Configuration

FIGURE 1 Optimized Architecture for Print Servers

Figure showing optimized architecture for print servers.

TABLE 3 describes possible print server configurations.

TABLE 3 Configuration Details for Print Servers (JBOD Only)
	Small Configuration	Large Configuration
RAID Enclosures	0	0
JBOD Enclosures	1	1
Number of Controllers	Not applicable	Not applicable
Number of Disks	5	12
Bus Configuration	Split Bus	Split Bus
Cache Optimization	Not applicable	Not applicable
RAID Levels Used	Host-based	Host-based
Drive Configuration	Two LUNs One spare	Two LUNs One spare

Print Server Tips and Techniques

Consider the following tips and techniques when configuring print servers.

Using a server's built-in SCSI port (if compatible) to minimize cost will result in satisfactory print server performance in most environments, even if the SCSI port does not operate at full Ultra320 or Ultra160 speeds.

Data protection can be provided via host-based software volume management software using the operating system's volume manager or a third-party volume manager. The recommended configurations will provide RAS similar to a single controller RAID array.

Each server must be connected to a different SCSI bus when using the recommended large configuration.

When adding a second server, one RAID 1 logical drive can be reassigned to the second server by moving its drives to the second SCSI bus if the server operating systems and volume managers are compatible.

Best Practices: File Servers

The following section outlines small and large storage solutions for file server environments. FIGURE 2 shows the scalability between the file server and Sun StorEdge 3000 Family SCSI array.

File Server Architecture and Configuration

FIGURE 2 Optimized Architecture for File Servers

Figure showing optimized architecture for file servers.

TABLE 4 describes typical file server configurations.

TABLE 4 Configuration Details for File Servers
	Small Configuration	Large Configuration
RAID Enclosures	1	1
JBOD Enclosures	0	2
Number of Controllers	1	2
Number of Disks	5	36
Bus Configuration	Single Bus	Single Bus
Cache Optimization	Sequential	Sequential
RAID Levels Used	RAID 3	RAID 5
Drive Configuration	One LUNs One spare drive	Two LUNs One spare drive

File Server Tips and Techniques

Consider the following tips and techniques when configuring file servers.

You can use an Ultra3 SCSI port to connect the RAID array whenever the server is connected to a LAN via gigabit Ethernet. Otherwise, a slower SCSI connection may create a performance bottleneck on the SCSI connection.

If the server has room for only one host adapter and you must choose between Ultra320 or Ultra160 SCSI or Gigabit Ethernet adapters, install the Gigabit Ethernet host adapter and use the server's built-in SCSI port to provide the most benefit to users. This, however, reduces the I/O to a slower SCSI speed.

A configuration with redundant RAID controllers and two logical drives provides very high transaction rates, even if there are a relatively small number of disk drives in use.

Adding disk drives to logical drives linearly improves transaction performance whenever cache misses occur, regardless of whether the additional storage capacity is utilized or not.

Rapidly growing network response times as users are added is an indication the file server is limiting performance. If this occurs, examine the utilization of server memory, processors, and network adapters and expand those with the highest usage.

Best Practices: Application Servers

The following section outlines small and large storage solutions for application server environments. FIGURE 3 shows the scalability between the application server and Sun StorEdge 3000 Family SCSI array.

Application Server Architecture and Configuration

FIGURE 3 Optimized Architecture for Application Servers

Figure showing optimized architecture for application servers.

TABLE 5 describes typical application server configurations.

TABLE 5 Configuration Details for Application Servers
	Small Configuration	Large Configuration
RAID Enclosures	1	1
JBOD Enclosures	0	0
Number of Controllers	1	2
Number of Disks	5	12
Bus Configuration	Split Bus	Split Bus
Cache Optimization	Random	Random
RAID Levels Used	RAID 5	RAID 5
Drive Configuration	One LUN One spare drive	Two LUNs One spare drive

Application Server Tips and Techniques

Consider the following tips and techniques when configuring application servers.

A single RAID array providing storage for two servers reduces storage costs with little to no effect on application performance.

Use the built-in SCSI ports of application servers to further minimize costs rather than adding a host adapter, particularly when not using Gigabit Ethernet to the LAN. Even a 40 MB/sec SCSI port is several times faster than a Fast Ethernet port.

A second RAID controller can be added to enhance RAS without adding disks and creating a second logical drive. Redundant RAID controllers will operate in an active-standby mode when only one logical drive is available.

Enhance application server availability by booting them from the RAID array rather than their internal drives. This also facilitates the rapid replacement of malfunctioning or failed servers.

If attaching the RAID array to multiple hosts, assign a separate LUN to each server and SCSI bus.

Best Practices: Mail Servers

The following section outlines small and large storage solutions for mail server environments. FIGURE 4 shows the scalability between the mail server and Sun StorEdge 3000 Family SCSI array.

Mail Server Architecture and Configuration

FIGURE 4 Optimized Architecture for Mail Servers

Figure showing optimized architecture for mail servers.

TABLE 6 describes typical mail server configurations.

TABLE 6 Configuration Details for Mail Servers
	Small Configuration	Large Configuration
RAID Enclosures	1	1
JBOD Enclosures	0	1
Number of Controllers	2	2
Number of Disks	12	24
Bus Configuration	Split Bus	Split Bus
Cache Optimization	Random	Random
RAID Levels Used	RAID 3 and 5	RAID 3 and 5
Drive Configuration	Two LUNs One spare drive	Two LUNs One spare drive

Mail Server Tips and Techniques

Consider the following tips and techniques when configuring mail servers.

Definitely use two Ultra320 or Ultra160 SCSI ports when connecting the RAID array to the server for best performance.

The recommended configuration for email servers consists of one RAID 3 and one RAID 5 logical drive. Use the RAID 3 logical drive to store attachments and other large files and the RAID 5 logical drive to store messages and small files.

There is no need for the two logical drives to include similar numbers of disks. Assign disks as needed to establish the desired amounts of RAID 3 and RAID 5 storage capacity. Assign at least one disk as a spare.

Scale storage capacity by adding disks to either the RAID 3 or RAID 5 logical drive, depending on need.

Balance performance by assigning the RAID 3 logical drive to one RAID controller and the RAID 5 logical drive to the other RAID controller.

When connecting a server to the array using two SCSI buses, map each LUN to one SCSI bus so that they can both be active and have a dedicated path.

Best Practices: Database Servers

The following section outlines small and large storage solutions for database server environments. FIGURE 5 shows the scalability between the database server and Sun StorEdge 3000 Family SCSI array.

Database Server Architecture and Configuration

FIGURE 5 Optimized Architecture for Database Servers

Figure showing optimized architecture for database servers.

TABLE 7 describes typical database server configurations.

TABLE 7 Configuration Details for Database Servers
	Small Configuration	Large Configuration
RAID Enclosures	1	1
JBOD Enclosures	0	1
Number of Controllers	2	2
Number of Disks	12	24
Bus Configuration	Split Bus	Split Bus
Cache Optimization	Random	Random
RAID Levels Used	RAID 1 and 5	RAID 1 and 5
Drive Configuration	Two LUNs One spare drive	Two LUNs One spare drive

Database Server Tips and Techniques

Consider the following tips and techniques when configuring database servers.

Database servers make heavy use of storage resources. Always use Ultra320 or Ultra160 SCSI ports for best performance. Further optimize performance by dedicating one RAID controller to each logical drive.

The recommended configuration for database servers consists of one RAID 1 and one RAID 5 logical drive. Use the RAID 5 logical drive to store data files and the RAID 1 (1+0) logical drive to store tables and other performance sensitive files.

Assign two drives as global spares, four drives to the RAID 1 logical drive (it automatically becomes RAID 1+0) and the remaining drives to the RAID 5 logical drive. Scale storage capacity by adding disks to the RAID 5 logical drive.

Adding disk drives to RAID 5 logical drives linearly improves transaction performance whenever cache misses occur, regardless of whether the additional storage capacity is utilized or not.

For environments with extreme performance needs, use two of the small configuration arrays rather than one of the large configuration arrays. Doing so doubles storage performance for less than twice the storage cost.

When connecting a server to the array using two SCSI buses, map each LUN to one SCSI bus so that they can both be active and have a dedicated path.

Best Practices: Consolidated Servers

The following section outlines small and large storage solutions for consolidated server environments. FIGURE 6 shows the scalability between consolidated servers and the Sun StorEdge 33x0 SCSI array.

Consolidated Server Architecture and Configuration

FIGURE 6 Optimized Architecture for Consolidated Servers

Figure showing optimized architecture for consolidated servers.

TABLE 8 describes typical consolidated server configurations.

TABLE 8 Configuration Details for Consolidated Servers
	Small Configuration	Large Configuration
RAID Enclosures	1	1
JBOD Enclosures	0	1
Number of Controllers	2	2
Number of Disks	12	24
Bus Configuration	Split Bus	Split Bus
Cache Optimization	Random	Random
RAID Levels Used	RAID 1, 3 and 5	RAID 1, 3 and 5
Drive Configuration	Three LUNs One spare drive	Three LUNs One spare drive

Consolidated Server Tips and Techniques

Consider the following tips and techniques when configuring consolidated servers.

Consolidated servers have very dynamic storage requirements. Use Ultra320 or Ultra160 SCSI ports to eliminate potential bandwidth bottlenecks between the server and RAID array.

If most of the server's resources are used for databases, create a RAID 1 logical drive using two disks, for log and transaction data. If more RAID 1 capacity is required in the future, create a new RAID 1 logical drive using two unassigned disks. Otherwise, use a portion of the RAID 3 logical drive for database storage as an alternative to a separate, dedicated RAID 1 logical drive.

To avoid reduced performance, add extra storage when a LUN reaches 80% full.

Balance workloads by assigning the RAID 5 logical drive to one RAID controller and the RAID 3 logical drive to the other controller. If there is a RAID 1 logical drive created, assign it to the same RAID controller as the RAID 3 logical drive.

When connecting a server to the array using two SCSI buses, map each LUN to one SCSI bus so that they can both be active and have a dedicated path.

Best Practices Summary

Entry-level servers are used for a wide range of applications with distinct storage requirements, so Sun StorEdge 3000 Family SCSI arrays feature a modular architecture with flexible configurations. For example, a storage solution can consist of a JBOD array, a RAID array, or a combination of both.

Configuration preferences include user-selectable RAID protection levels, controller optimization and more. Modularity and flexibility allow the storage solution to quickly and easily adapt to a particular environment.