Solaris Firewall Service Prevents Oracle Solaris Cluster 4.4 From Rebooting in to Cluster Mode (32987543)

Description: A cluster boot hangs indefinitely after configuring the Solaris Firewall service for a cluster that runs Oracle Solaris Cluster 4.4.

The following output shows that when you reboot the node01 node, it never joins the cluster:

root@node01:~ May 21 09:42:14 node01 cl_runtime: NOTICE: CMM: Cluster doesn't
have operational quorum yet; waiting for quorum.
May 21 09:42:14 node01 cl_runtime: NOTICE: clcomm: Path node01:net3 -
node02:net1 errors during initiation
May 21 09:42:14 node01 cl_runtime: NOTICE: clcomm: Path node01:net0 -
node02:net0 errors during initiation
May 21 09:42:14 node01 cl_runtime: NOTICE: clcomm: Path node01:net1 -
node02:net2 errors during initiation
May 21 09:42:14 node01 cl_runtime: NOTICE: clcomm: Path node01:net2 -
node02:net3 errors during initiation

For security purposes, you must configure Oracle Solaris Cluster to use the Solaris Firewall service.

Workaround: Unblock the Oracle Solaris Cluster interconnect traffic.

Use the appropriate Solaris Firewall service configuration file that is associated with your environment.

• Oracle Solaris global cluster. Update the following example Oracle Solaris global cluster fp.conf configuration file to replace example interconnect network objects with the objects in your environment.

  The example interconnect network objects are net0, net1, net2, net3, and clprivnet0.

  # cat /etc/firewall/pf.conf
  #!/usr/sbin/pfctl -f
  
  ext_if = "aggr0"
  client_out = "{22, 111, 8059, 8060, 8061, 8062, 6499, 11161, 11162, 11163,
  11164, 11165}"
  
  # do not filter local interface
  set skip on lo0
  # do not filter global Cluster interconnects
  set skip on net0
  set skip on net1
  set skip on net2
  set skip on net3
  set skip on clprivnet0
  # do not filter local ILOM interface
  set skip on sp-phys0
  
  block in log quick on egress proto tcp to port { 22 }
  block return log all
  
  pass in log proto tcp from any to any port 22 <> 23
  pass out log proto tcp from any to any
  pass in log proto udp from any to any
  pass out inet proto icmp all icmp-type echoreq keep state
  pass in log proto icmp from any to any
  
  table <accesslist> persist file "/etc/accesslist"
  pass in log on $ext_if proto tcp from <accesslist> to port $client_out flags
  S/SA keep state
  pass out on $ext_if proto udp all
  
  # Solaris Cluster: allow all connections initiated from this system
  pass out
  
  set debug info

• Oracle Solaris exclusive-IP zone cluster. Update the following example Oracle Solaris exclusive-IP zone cluster fp.conf configuration file to replace example interconnect network objects with the objects in your environment.

  The example interconnect network objects are net0_zcdb1_vnic0, net1_zcdb1_vnic0, net2_zcdb1_vnic0, net3_zcdb1_vnic0, and clprivnet1.

  # cat /etc/firewall/pf.conf
  set reassemble yes no-df
  
  ext_if = "zcdb1vnic1"
  client_out = "{22, 5201, 111, 8059, 8060, 8061, 8062, 6499, 11161, 11162,
  11163, 11164, 11165}"
  
  set skip on lo0
  
  set skip on net0_zcdb1_vnic0
  set skip on net1_zcdb1_vnic0
  set skip on net2_zcdb1_vnic0
  set skip on net3_zcdb1_vnic0
  set skip on clprivnet1
  
  block in log quick on egress proto tcp to port { 22 }
  block return log all
  
  pass in log proto tcp from any to any port 22 <> 23
  pass out log proto tcp from any to any
  pass in log proto udp from any to any
  pass out inet proto icmp all icmp-type echoreq keep state
  pass in log proto icmp from any to any
  
  table <accesslist> persist file "/etc/accesslist"
  pass in log on $ext_if proto tcp from <accesslist> to port $client_out flags
  S/SA keep state
  pass out on $ext_if proto udp all
  
  pass out
  
  set debug error

• Oracle Solaris shared-IP zone cluster. Unblock the clprivnet interconnect network object because a shared-IP zone cluster shares the private interconnects with the global zone.