Log in to the installation server as superuser.

Create the Solaris JumpStart environment on the installation server by using the appropriate document for the Solaris release:

• Solaris 9 or Solaris 10 Installation Guide

You can access these documents on http://www.oracle.com/technetwork/indexes/documentation/index.html.

In the /etc/hosts file, add the names and IP addresses of the master-eligible nodes.

Share the Solaris-distribution-dir and Jumpstart-dir directories by adding these lines to the /etc/dfs/dfstab file:

share -F nfs -o ro,anon=0 Solaris-distribution-dirshare -F nfs -o ro,anon=0 Jumpstart-dir

• Solaris-distribution is the directory that contains the Solaris distribution.

• Jumpstart-dir is the directory that contains the Solaris JumpStart files.

Share the directories that are defined in the /etc/dfs/dfstab file:

# shareall

Change to the directory where the add_install_client command is located:

# cd Solaris-dir/Solaris_x/Tools

• Solaris-dir is the directory that contains the Solaris installation software. This directory could be on a CD-ROM or in an NFS-shared directory.

• x is 9 or 10 depending on the Solaris version installed.

Run the add_install_client command for each master-eligible node.

For information, see the add_install_client1M man page.

Connect to the console of each master-eligible node.

Boot each master-eligible node with the appropriate command using a network boot.

If you are unsure of the appropriate command, refer to the hardware documentation for your platform. The common command for SPARC systems is shown in the following example:

ok> boot net - install

If the installation server is connected to the second Ethernet interface, type:

ok> boot net2 - install

This command installs the Solaris Operating System on the master-eligible nodes.

For information about performing this task on the AMD64 platform, refer to the hardware documentation.

Log in to a master-eligible node as superuser.

Create /etc/notrouter file:

# touch /etc/notrouter

Modify the /etc/default/login file so that you can connect to a node from a remote system as superuser:

# mv /etc/default/login /etc/default/login.orig
# chmod 644 /etc/default/login.orig
# sed '1,$s/^CONSOLE/#CONSOLE/' /etc/default/login.orig > /etc/default/login
# chmod 444 /etc/default/login

Disable power management:

# touch /noautoshutdown

Modify the .rhosts file according to the security policy for your cluster:

# touch /.rhosts
# cp /.rhosts /.rhosts.orig
# echo "+ root" > /.rhosts
# chmod 444 /.rhosts

Set the boot parameters:

# /usr/sbin/eeprom local-mac-address?=true
# /usr/sbin/eeprom auto-boot?=true
# /usr/sbin/eeprom diag-switch?=false

The preceding example is for SPARC-based hardware. For the commands required on the x64 platform, refer to the hardware documentation.

If you are using the Network Time Protocol (NTP) to run an external clock, configure the master-eligible node as an NTP server.

This procedure is described in the Solaris documentation.

If your master-eligible node has an IDE disk, edit the /usr/kernel/drv/sdbc.conf file.

Change the value of the sdbc_max_fbas parameter from 1024 to 256.

Create the data/etc and data/var/dhcp directories in the /SUNWcgha/local/export/ file system on the master-eligible node:

# mkdir -p /SUNWcgha/local/export/data/etc
# mkdir -p /SUNWcgha/local/export/data/var/dhcp

• /SUNWcgha/local/export/data/etc directory is required for the Cluster Membership Manager (CMM).

• /SUNWcgha/local/export/data/var/dhcp directory is required for the Reliable Boot Service.

Repeat Step 1 through Step 9 on the second master-eligible node.

Log in to the installation server as superuser.

Check that the mountd and nfsd daemons are running on the installation server.

For example, use the ps command:

# ps -ef | grep mountd
root   184     1  0   Aug 03 ?        0:01 /usr/lib/autofs/automountd
root   290     1  0   Aug 03 ?        0:00 /usr/lib/nfs/mountd
root  2978  2974  0 17:40:34 pts/2    0:00 grep mountd
# ps -ef | grep nfsd
root   292     1  0   Aug 03 ?        0:00 /usr/lib/nfs/nfsd -a 16
root  2980  2974  0 17:40:50 pts/2    0:00 grep nfsd
# 

If a process ID is not returned for the mountd and nfsd daemons, start the NFS daemons as follows:

On the Solaris 9 OS, use the following command

# /etc/init.d/nfs.server start

On the Solaris 10 OS, use the following command

# svcadm enable svc:/network/nfs/server

Share the directory containing the distributions for the Netra HA Suite and the Solaris Operating System by adding the following lines to the /etc/dfs/dfstab file:

share -F nfs -o ro,anon=0 software-distribution-dir

where software-distribution-dir is the directory that contains the Netra HA Suite packages and Solaris patches.

Share the directories that are defined in the /etc/dfs/dfstab file:

# shareall

Log in to the a master-eligible node as superuser.

Create the mount point directories Solaris and NetraHASuite on the master-eligible node:

# mkdir /NetraHASuite
# mkdir /Solaris

Mount the Netra HA Suite and Solaris distribution directories on the installation server:

# mount -F nfs \installation-server-IP-address:/software-distribution-dir/Product/NetraHASuite_3.0/
FoundationServices/Solaris_x/sparc
# mount -F nfs installation-server-IP-address:/Solaris-distribution-dir/Solaris

• installation-server-IP-address is the IP address of the cluster network interface that is connected to the installation server.

• software-distribution-dir is the directory that contains the Netra HA Suite packages. (Note that in the preceding example, this line wraps, due to space constraints; however, when typing this path, it should contain no spaces or line breaks.)

• x is the Solaris OS version.

• Solaris-distribution-dir is the directory that contains the Solaris distribution.

Repeat Step 5 through Step 7 on the other master-eligible node.

Log in to each master-eligible node as superuser.

Install the necessary Solaris patches on each master-eligible node:

# patchadd -M /NetraHASuite/Patches/ patch-number

Log in to a master-eligible node as superuser.

Add the man page package:

# pkgadd -d /NetraHASuite/Packages/ SUNWnhas-manpages

The man pages are installed in the /opt/SUNWcgha/man directory. To access the man pages, see the Netra High Availability Suite 3.0 1/08 Foundation Services Reference Manual.

Repeat Step 1 and Step 2 on the other master-eligible node.

Before you install the CGTP packages, make sure that you have installed the Solaris patches for CGTP.

See To Install Solaris Patches.

As superuser, install the following CGTP packages on each master-eligible node:

# pkgadd -d /NetraHASuite/Packages/ SUNWnhas-cgtp SUNWnhas-cgtp-cluster

Become superuser.

Type the following command:

For information on configuring the EAM, see the Netra High Availability Suite 3.0 1/08 Foundation Services Overview.

As superuser, install the following Reliable NFS and SNDR packages on a master-eligible node in the following order:

# pkgadd -d /NetraHASuite/Packages/ SUNWscmr SUNWscmu SUNWspsvr \  SUNWspsvu SUNWrdcr SUNWrdcu SUNWnhas-rnfs-client \  SUNWnhas-rnfs-server

Repeat Step 1 on the second master-eligible node.

Install the SNDR patches on each master-eligible node.

See the Netra High Availability Suite 3.0 1/08 Foundation Services README for a list of SNDR patches.

Edit the /usr/kernel/drv/rdc.conf file on each master-eligible node to change the value of the rdc_bitmap_mode parameter.

To have changes to the bitmaps written on the disk at each update, change the value of the rdc_bitmap_mode parameter to 1.

To have changes to the bitmaps stored in memory at each update, change the value of the rdc_bitmap_mode parameter to 2. In this case, changes are written on the disk when the node is shut down. However, if both master-eligible nodes fail, both disks must be synchronized.

For example: rdc_bitmap_mode=2.

As superuser, install the following Reliable NFS packages on a master-eligible node in the following order:

# pkgadd -d /NetraHASuite/Packages/ SUNWnhas-rnfs-client \
#   SUNWnhas-rnfs-server

Repeat Step 1 on the second master-eligible node.

As superuser, install the following NMA package and Java DMK package on both master-eligible nodes:

# pkgadd -d /NetraHASuite/Packages/ SUNWnhas-nma-local SUNWjsnmp

On the first master-eligible node, install the following shared Java DMK package and NMA packages:

# pkgadd -d /NetraHASuite/Packages/ \-M -R /SUNWcgha/local/export/services/ha_3.0 \ SUNWjdrt SUNWnhas-nma-shared

The packages are installed with a predefined root path in the /SUNWcgha/local/export/services/ha_3.0 directory.

To configure the NMA, see the Netra High Availability Suite 3.0 1/08 Foundation Services NMA Programming Guide.

Log in to each master-eligible node as superuser.

In the /etc/hosts file on each master-eligible node, add the three IP addresses, followed by the name of each interface:

10.250.1.10     netraMEN1-nic0
10.250.2.10     netraMEN1-nic1
10.250.3.10     netraMEN1-cgtp
10.250.1.20     netraMEN2-nic0
10.250.2.20     netraMEN2-nic1
10.250.3.20     netraMEN2-cgtp
10.250.1.1      master-nic0
10.250.2.1      master-nic1
10.250.3.1      master-cgtp

In the rest of this book, the node netraMEN1 is the first master-eligible node. The first master-eligible node is the node that is booted first after you complete installing the Netra HA Suite. The node netraMEN2 is the second master-eligible node that is booted after the first master-eligible node has completed booting.

Create or update the file /etc/hostname.NIC0 for the NIC0 interface.

This file must contain the name of the master-eligible node on the first interface, for example, netraMEN1-nic0.

Create or update the file /etc/hostname.NIC1 for the NIC1 interface.

This file must contain the name of the master-eligible node on the second interface, for example, netraMEN1-nic1.

Create or update the file /etc/hostname.cgtp0 for the cgtp0 interface.

This file must contain the name of the master-eligible node on the cgtp0 interface, for example, netraMEN1-cgtp.

Update the /etc/nodename file with the IP address of the master-eligible node.

• If you have installed CGTP, add the name set on the CGTP interface, for example, netraMEN1-cgtp.

• If you have not installed CGTP, add the name set on the NIC0 interface, for example, netraMEN1-nic0.

Create a /etc/netmasks file with a netmask of 255.255.255.0 for all subnetworks in the cluster.

If required, add the hostname associated with the external floating address in /etc/host on each master-eligible node.

Add, if required, the associated netmask for the subnetwork in /etc/netmasks on each master-eligible node.

Create or update the file /etc/hostname.interface for the interface supporting the external floating address on each master-eligible node.

If the file does not exist, create the following lines (the file must contain at least two lines for the arguments to be taken into account):

If the file already exists, add the following line:

Configure the external floating address parameter in the nhfs.conf file on each master-eligible node.

For more information, see the nhfs.conf4 man page.

Add, if required, the hostname associated to test IP addresses and the external floating address in /etc/host on each master-eligible node.

IP addresses for testing must be different on each node.

Add, if required, the associated netmask for the subnetwork in /etc/netmasks on each master-eligible node.

Create or update the file /etc/hostname.interface for the first interface on each master-eligible node.

The file must contain the definition of the test IP address for this interface and the external floating address in this format:

test IP address #1 netmask + broadcast + -failover deprecated group name up addif floating address netmask + broadcast + failover down

For example:

Create or update the file /etc/hostname.interface for the second interface on each master-eligible node.

The file must contain the definition of the test IP address for this interface in this format:

test IP address #1 netmask + broadcast + -failover deprecated group name up

For instance:

Configure the external floating address parameters (floating address and IPMP group to be monitored) in the nhfs.conf file on each master-eligible node.

For more information, see the nhfs.conf4 man page.

On the first master-eligible node, change the node name with the name of the host associated to the CGTP interface:

Repeat for the second master-eligible node:

On the first master-eligible node, restart the rpcbind daemon to make it use the new node name:

Repeat Step 3 on the second master-eligible node.

Create the database replicas for the dedicated root disk slice on each master-eligible node:

Repeat Step 9 for the second master-eligible node:

(Optional) If you plan to use CGTP, configure a temporary network interface on the first private network and make it match the name and IP address of the CGTP interface on the first master-eligible node:

(Optional) If you plan to use CGTP, repeat Step 7 for the second master-eligible node:

On the first master-eligible node, verify that the /etc/nodename file matches the name of the CGTP interface (or the name of the private network interface, if CGTP is not used):

Create the Solaris VM diskset that manages the shared disks:

Remove any possible existing SCSI3-PGR keys from the shared disks.

In the following example, there was no key lying on the disks:

On x64 master-eligible nodes, repartition the shared disk as described in .

Add the names of the shared disks to the previously created diskset:

Verify that the SVM configuration is set up correctly:

On SPARC master-eligible nodes, repartition the shared disk as described in .

Create the metadevices for partition mapping and mirroring.

Create the metadevices on the primary disk:

Create the metadevices on the secondary disk:

Create the mirror sets:

Create your specific Solaris VM RAID configuration (refer to the Solaris Volume Manager Administration Guide for information on specific configurations).

In the following example, the two disks form a mirror called d0:

Create soft partitions to host the shared data.

These soft partitions are the file systems managed by Reliable NFS. In the following example, d1 and d2 are managed by Reliable NFS.

The devices managed by Reliable NFS are now accessible through /dev/md/nhas_diskset/dsk/d1 and /dev/md/nhas_diskset/dsk/d2.

Create the file systems on the soft partitions:

Create the following directories on both master-eligible nodes:

Mount the file systems on the metadevice on the first node:

Retrieve disk geometry information using the prtvtoc command.

A known problem in the diskless management tool, smosservice, prevents the creation of the diskless environment on a metadevice. To avoid this problem, mount the /export directory on a physical partition during the diskless environment creation.

To support access to the /export via a metadevice without preventing its access on a physical partition, the disk must be re-partitioned in a particular way after it has been inserted into a diskset. This re-partitioning preserves data already stored by SVM, since there is no formatting of created partitions.

The following table gives an example of the prtvtoc command output after inserting a disk into a diskset.

Create the data file using the fmthard command.

The fmthard command (see its man page for more information) is used to create physical partitions. It requires you to input a data file describing the partitions to be created. There is one entry per partition, using the following format:

starting sector and size in sectors values must be rounded to a cylinder boundary and must be computed as explained below.

• starting sector = starting sector of the previous slice + size in sectors of the previous slice

• size in sectors = the required partition size in bytes divided by bytes per sector, the result being rounded to sectors per cylinder (upper value)

  Three particular slices must be created:

  • Slice 7 containing the meta-database (also called metadb). This slice must be created the same size as that created by the Solaris VM to overlap the existing one (to preserve data).

  • A slice to support /export (diskless environment)

  • A slice to support /SUNWcgha/local (shared Netra HA Suite packages and files)

    Other slices can be added depending on your application requirements. The following table gives an example for partitioning:

    
    

    Slice Number	Usage	Size in MBytes
    0	/export	4096
    1	/SUNWcgha/local	2048
    7	metadb	Not Applicable

    
    

    The following slice constraints must be respected:

    • Slice 7 (metadb) is the first slice of the disk starting at sector # 0, with the size size of slice 7 with tag 4 (user partition) and flag 0x01: (unmountable)

    • Slice 2: On SPARC, Slice 2 maps the whole disk: size in bytes = accessible cylinders* sectors per cylinder * bytes per sector with tag 5 (backup) and flag 0x01 (unmountable). On x64, Slice 2 maps the whole disk except slice 7 with tag 5 (backup) and flag 0x01 (unmountable). This particular partitioning may provoke the display of warning message while formatting the disk.

    • Other slices use tag 0 (unassigned) and flag 0x00 (mountable in R/W)

      An example of computing for slice 0 (located after slice 7):

      • Starting sector = (0 + 8667) = 8667

      • Size in bytes = (4096 * 1024²) = 4294967296

      • Size in sectors = 4294967296 / 512 = 8388608

      • Size in sector rounded to cylinder boundaries (2889) = 8389656

        
        

        7 4 01 0 8667 0 0 00 8667 8389656 1 0 00 8398323 4194828 2 5 01 0 71127180

        
        
        

        7 4 01 0 8667 0 0 00 8667 8389656 1 0 00 8398323 4194828 2 5 01 8667 71118513

        
        

These values would display the following content in the data file (datafile.txt):

On x64, the content of the file appears as follows:

Note that this example leaves some unallocated spaces on the disk that can be used for user-specific partitions.

Re-partition the disk.

Execute the following commands for the primary and for the secondary disk:

Ensure that the following directories exist on the first master-eligible node:

# mkdir /SUNWcgha/local/export
# mkdir /SUNWcgha/local/export/data
# mkdir /SUNWcgha/local/export/services
# mkdir /SUNWcgha/local/export/services/NetraHASuite_version/opt

where NetraHASuite_version is the version of the Netra HA Suite you install, for example, ha_3.0.

These directories contain packages and data shared between the master-eligible nodes.

If you are using shared disks, install the shared Java DMK package and NMA packages onto the first master-eligible node as explained in Step 2 of To Install the Node Management Agent.

Create the following mount points on each master-eligible node:

# mkdir /SUNWcgha/services
# mkdir /SUNWcgha/remote
# mkdir /SUNWcgha/swdb

These directories are used as mount points for the directories that contain shared data.

Add the following lines to the /etc/vfstab file on each master-eligible node:

• If you have configured the CGTP, use the floating IP address for the cgtp0 interface that is assigned to the master role to define the mount points.

  
  

  master-cgtp:/SUNWcgha/local/export/data - \ /SUNWcgha/remote nfs - no rw,hard,fg,intr,noac master-cgtp:/SUNWcgha/local/export/services/ha_3.0/opt \ - /SUNWcgha/services nfs - no rw,hard,fg,intr,noac master-cgtp:/SUNWcgha/local/export/services/ha_3.0 - \ /SUNWcgha/swdb nfs - no rw,hard,fg,intr,noac

  
  

  where master-cgtp is the host name associated with the floating address of the cgtp0 interface of the master node. For more information, see To Create the Floating Address Triplet Assigned to the Master Role.

• If you have not configured the CGTP, use the floating IP address for the NIC0 interface that is assigned to the master role.

  
  

  master-nic0:/SUNWcgha/local/export/data - \ /SUNWcgha/remote nfs - no rw,hard,fg,intr,noac master-nic0:/SUNWcgha/local/export/services/ha_3.0/opt \ - /SUNWcgha/services nfs - no rw,hard,fg,intr,noac master-nic0:/SUNWcgha/local/export/services/ha_3.0 - \ /SUNWcgha/swdb nfs - no rw,hard,fg,intr,noac

  
  

  where master-nic0 is the host name associated with the floating address of the NIC0 interface of the master node. For more information, see To Create the Floating Address Triplet Assigned to the Master Role.

Check the following in the /etc/vfstab file:

• The mount at boot field is set to no for all RNFS-managed partitions.

  Example for SNDR:

  
  

  /dev/dsk/c0t0d0s1 /dev/rdsk/c0t0d0s1 /SUNWcgha/local ufs 2 no logging

  
  

  Example for Solaris VM:

  
  

  /dev/md/nhas_diskset/dsk/d1 /dev/md/nhas_diskset/rdsk/dl /SUNWcgha/local ufs 2 no logging

  
  

• The root file system (/) has the logging option.

  
  

  /dev/dsk/c0t0d0s0 /dev/rdsk/c0t0d0s0 / ufs 1 logging

  
  

(Only applicable to SNDR) Create the file systems on the replicated partitions:

Unmount the shared file system, /NetraHASuite, on each master-eligible node by using the umount command.

See the umount1M man page and To Mount an Installation Server Directory on the Master-Eligible Nodes.

Reboot the first master-eligible node, which becomes the master node:

After the first master-eligible node has completed rebooting, reboot the second master-eligible node:

This node becomes the vice-master node. To check the role of each node in the cluster, see the nhcmmrole1M man page.

Create the INST_RELEASE file to allow patching of shared packages:

# /opt/SUNWcgha/sbin/nhadm confshare

Log in to the master-eligible node as superuser.

Run the nhadm tool to validate the configuration:

# nhadm check starting

If all checks pass the validation, the installation of the Netra HA Suite was successful. See the nhadm1M man page.