Configuration Requirements

The following table identifies specific configuration rules for each segment of the COTC environment.

Segment	Configuration Rules
Sun QFS metadata server	`mm` devices must be on a separate physical LUN from data devices. `mr` devices must be on a separate physical LUN from metadata devices. `mm` devices must not have the Oracle Solaris Cluster `localonly` flag.
Sun QFS clients	`mm` devices must be identified as `nodev` in the `mcf` file. `mr` devices must follow the `/dev/dsk` syntax rather than the `/dev/did/dsk` syntax. The definition for the `mr` device on the client `/dev/sdk` must match the definition for the `/dev/did/dsk` on the metadata server.
Oracle Solaris Cluster devices	Set `localonly` flag on the `mr` devices.
Private metadata network	Oracle Solaris Cluster nodes that serves as metadata servers should use IPMP groups. Oracle Solaris Cluster Sun QFS resource groups should have a logical host name configured for metadata communication between the metadata server and the clients that are outside of the cluster. Oracle Solaris Cluster Sun QFS resource groups should be defined to ensure that Sun QFS file system resources depend on the logical host name resource.

How to Configure the Metadata Server

The following procedure includes embedded examples to better explain what you should expect to see in a COTC configuration.

Use the cluster show -t device command to identify the devices that will be used for the Sun QFS configuration and build the mcf file.

Note the use of the /dev/did syntax in the following sample mcf file for this configuration.

#
# File system Qfs1
#
Qfs1 2 ma Qfs1 on shared
/dev/did/dsk/d7s0 20 mm Qfs1 on
/dev/did/dsk/d8s0 21 mm Qfs1 on
/dev/did/dsk/d16s0 22 mr Qfs1 on
/dev/did/dsk/d10s0 23 mr Qfs1 on
/dev/did/dsk/d13s0 24 mr Qfs1 on
#
# File system Qfs2
#
Qfs2 5 ma Qfs2 on shared
/dev/did/dsk/d9s0 50 mm Qfs2 on
/dev/did/dsk/d11s0 51 mm Qfs2 on
/dev/did/dsk/d17s0 52 mr Qfs2 on
/dev/did/dsk/d12s0 53 mr Qfs2 on
/dev/did/dsk/d14s0 54 mr Qfs2 on
/dev/did/dsk/d15s0 55 mr Qfs2 on
/dev/did/dsk/d18s0 56 mr Qfs2 on

Example /etc/opt/SUNWsamfs/defaults.conf file:

trace
all = on
sam-fsd.size = 10M
sam-sharefsd.size = 10M
endtrace

Example /etc/opt/SUNWsamfs/samfs.cmd file:

fs = Qfs1
meta_timeo=0

fs = Qfs2
meta_timeo=0

Build the Sun QFS file system hosts tables.
The example configuration uses shared and local file system host files for this QFS shared file system setup.

Note - To communicate with metadata clients that are outside of the cluster, you must establish Sun QFS metadata traffic over the Sun QFS network. Because the metadata client is not a member of the Oracle Solaris Cluster configuration, the example uses a logical host for this traffic. In the example configuration, sc-qfs1 is the logical host name.

Build the shared host table on the metadata server.

Use the clnode show command and syntax to obtain the host order information to build the table.

For example:

ctelab30:root> clnode show

=== Cluster Nodes ===                          

Node Name:                                      ctelab30
  Node ID:                                         1
  Enabled:                                         yes
  privatehostname:                                 clusternode1-priv
  reboot_on_path_failure:                          disabled
  globalzoneshares:                                1
  defaultpsetmin:                                  1
  quorum_vote:                                     1
  quorum_defaultvote:                              1
  quorum_resv_key:                                 0x4AB78CC400000001
  Transport Adapter List:                          e1000g1, e1000g3

Node Name:                                      ctelab31
  Node ID:                                         2
  Enabled:                                         yes
  privatehostname:                                 clusternode2-priv
  reboot_on_path_failure:                          disabled
  globalzoneshares:                                1
  defaultpsetmin:                                  1
  quorum_vote:                                     1
  quorum_defaultvote:                              1
  quorum_resv_key:                                 0x4AB78CC400000002
  Transport Adapter List:                          e1000g1, e1000g3

Example of the /etc/opt/SUNWsamfs/hosts.Qfs1 file:

In the following example, each file system must have its own host table.

#
#MDS
# Shared MDS Host file for family set 'Qfs1'
#
#
ctelab30 clusternode1-priv,sc-qfs1 1 - server
ctelab31 clusternode2-priv,sc-qfs1 2 -
ctelab32 ctelab32,ctelab32-4 - -
ctelab33 ctelab33,ctelab33-4 - -
ctelab28 ctelab28,ctelab28-4 - -

Example of the /etc/opt/SUNWsamfs/hosts.Qfs1.local file:

In the following example, each file system must have its own host table.

#
#MDS
# Local MDS Host file for family set 'Qfs1'
#
#
ctelab30 clusternode1-priv
ctelab31 clusternode2-priv

Create the file system by running sammkfs on each file system.
```
# /opt/SUNWsamfs/sbin/sammkfs -S Qfs1
```
Add file system mounts to the /etc/vfstab file.
Edit the /etc/vfstab file on each cluster node and append the Sun QFS file system mounts.
```
###
# QFS Filesystems
###
Qfs1 - /cluster/qfs1 samfs - no shared
Qfs2 - /cluster/qfs2 samfs - no shared
```
Mount the file systems.
1. Make the mount points on each cluster node.
```
# mkdir -p /cluster/qfs1 /cluster/qfs2
```
2. Mount Qfs1 and Qfs2 on each cluster node.
Create the Oracle Solaris Cluster metadata server resource group.
When you have configured Oracle Solaris Cluster and Sun QFS, you need to create the MDS resource group under Oracle Solaris Cluster. Follow these steps:
1. Add the Sun QFS resource type.
```
# clresourcetype register SUNW.qfs
```
2. Create a resource group for the metadata server.
```
# clresourcegroup create -n ctelab30,ctelab31 sc-qfs-rg
```
3. Add the logical host name to the resource group that you just created.
```
# clreslogicalhostname create -g sc-qfs-rg \\ -N qfs_ipmp1@ctelab30,qfs_ipmp1@ctelab31 sc-qfs1
```
4. Add Sun QFS file system resources to the MDS resource group.
```
# clresource create -g sc-qfs-rg -t SUNW.qfs \\ -x QFSFileSystem=/cluster/qfs1,/cluster/qfs2 -y Resource_dependencies=sc-qfs1 \\ fs-qfs-rs 
```
5. Bring the resource group online.
```
# clresourcegroup online -emM sc-qfs-rg
```
6. Check resource group status.
```
# clresourcegroup status
```

Set local mode on the Sun QFS data devices.

For each data device (mr device) that is part of the file system, set local mode. In the following example, each data device in the Qfs1 file system must have local mode set.

# /usr/cluster/bin/scconf -r -D name=dsk/d16,nodelist=ctelab31
# /usr/cluster/bin/scconf -c -D name=dsk/d16,localonly=true
# /usr/cluster/bin/scconf -r -D name=dsk/d10,nodelist=ctelab31
# /usr/cluster/bin/scconf -c -D name=dsk/d10,localonly=true
# /usr/cluster/bin/scconf -r -D name=dsk/d13,nodelist=ctelab31
# /usr/cluster/bin/scconf -c -D name=dsk/d13,localonly=true

How to Configure the File System Clients

After the operating system and Oracle Solaris Cluster software have been installed on the clients, install and configure the Sun QFS software on each metadata client.

Install the metadata client.
Specific steps differ slightly, depending on the client architecture.
For Oracle Solaris systems:
- Go to the location of the Sun QFS software packages. For example:
```
# cd /net/eyelid/builds/src_ctl/R4_5/6/pkg/obj/SunOS_5.9_sparc
```
- Add the packages.
```
# pkgadd-d . SUNWqfsr SUNWqfsu
```
- If necessary, go to the Oracle eDelivery site and download any additional patches.
- If necessary, apply patches.
For Linux systems:
Follow similar steps to the Oracle Solaris process to install the base packages and apply any patches. Once the install completes, the install process will detect and build the mcf file configuration.
- Create mount directories and mount the packages as needed. For example:
```
# mkdir /mnt/pkgs /mnt/pkg
# mount eyelid-mn.central:/builds/src_ctl/R4_5/6/pkg-linux /mnt/pkgs
# mount -o loop,ro iso_open.iso /mnt/pkg
```
- Add the packages.
```
# /mnt/pkg/Install
```
- If necessary, go to My Oracle Support (MOS) and download any additional patches.
- If necessary, apply patches.
Verify MPXIO has been enabled and clients can discover QFS devices.

Identify devices.

Using the Oracle Solaris format command on the clients and the cluster show -t device command on the metadata server, match device IDs that identify the devices that will be used for the Sun QFS configuration and build the mcf file on the SPARC clients.

Example mcf file

#
# File system Qfs1
#
Qfs1 2 ma Qfs1 on shared
nodev 20 mm Qfs1 on
nodev 21 mm Qfs1 on
/dev/dsk/c6t600C0FF00000000000332B21D0B90000d0s0 22 mr Qfs1 on
/dev/dsk/c6t600C0FF0000000000876E9124FAF9C00d0s0 23 mr Qfs1 on
/dev/dsk/c6t600C0FF000000000004CAD7CC3CDE500d0s0 24 mr Qfs1 on
#
# File system Qfs2
#
Qfs2 5 ma Qfs2 on shared
nodev 50 mm Qfs2 on
nodev 51 mm Qfs2 on
/dev/dsk/c6t600C0FF00000000000332B057D2FF100d0s0 52 mr Qfs2 on
/dev/dsk/c6t600C0FF0000000000876E975EDA6A000d0s0 53 mr Qfs2 on
/dev/dsk/c6t600C0FF0000000000876E9780ECA8100d0s0 54 mr Qfs2 on
/dev/dsk/c6t600C0FF000000000004CAD139A855500d0s0 55 mr Qfs2 on
/dev/dsk/c6t600C0FF000000000004CAD4C40941C00d0s0 56 mr Qfs2 on

Configure the /etc/opt/SUNWsamfs/defaults.conf file on SPARC and Linux clients.
```
trace
all = on
sam-fsd.size = 10M
sam-sharefsd.size = 10M
endtrace
```
Configure the /etc/opt/SUNWsamfs/samfs.cmd file on SPARC and Linux clients.
```
fs = Qfs1
meta_timeo=0

fs = Qfs2
meta_timeo=0
```
Build the Sun QFS file system host tables.
Using information already configured on the metadata server, build host tables. Follow the examples below.

Note - Remember that for metadata communications between the metadata server and metadata clients, the clients that are not members of the cluster must communicate over the logical host.

Example of the /etc/opt/SUNWsamfs/hosts.Qfs1 file:
```
#
#MDC
# Shared Client Host file for family set 'Qfs1'
#
#
ctelab30 sc-qfs1 1 - server
ctelab31 sc-qfs1 2 -
ctelab32 ctelab32,ctelab32-4 - -
ctelab33 ctelab33,ctelab33-4 - -
ctelab28 ctelab28,ctelab28-4 - -
```
Bind each local host to the metadata server.
Edit the local hosts file on each client to bind the file system to the metadata server. The /etc/opt/SUNWsamfs/hosts.fsname.local information will be different for each client. In the following example, the client is configured to use the interface on ctelab32-4 to bind to host sc-qfs1 for metadata traffic.
```
#
#MDC
# Local Client Host file for family set 'Qfs1'
#
#
ctelab30 sc-qfs1@ctelab32-4
ctelab31 sc-qfs1@ctelab32-4
```

In the /etc/vfstab file on each client node, append the Sun QFS file system mounts.

SPARC: Example of the /etc/vfstab file

###
# QFS Filesystems
###
Qfs1 - /cluster/qfs1 samfs - no shared
Qfs2 - /cluster/qfs2 samfs - no shared

Linux: Example of the /etc/vfstab file

###
# QFS Fileystems
#
Qfs1 /cluster/qfs1 samfs noauto,rw,shared 0 0
Qfs2 /cluster/qfs2 samfs noauto,rw,shared 0 0

Mount file systems on each client.
1. Make the mount points on each client.
```
# mkdir -p /cluster/qfs1 /cluster/qfs2
```
2. Mount the file systems on each cluster node.
```
# mount Qfs1;mount Qfs2
```

Skip Navigation Links
Exit Print View
	Using Sun QFS and Sun Storage Archive Manager With Oracle Solaris Cluster Sun QFS and Sun Storage Archive Manager 5.3 Information Library