The ovs-agent-secondary command can be used to automatically create and setup a secondary domain. In particular, the command indicates whether the server is suitable for creating a secondary service domain, and which PCIe buses are available for the secondary service domain.
To create a secondary service domain, run the following command on the control domain:
# ovs-agent-secondary create secondary
The ovs-agent-secondary command is a helper script that is provided as is. This command might not work with some servers or configurations. If the command does not work, create the secondary service domain manually, as described in Section 1.1.3.3, “Creating a Secondary Service Domain”.
If the Oracle VM Agent is enabled then a menu is displayed for disabling the Oracle VM Agent. The Oracle VM Agent must be disabled when creating a secondary service domain. Example output from the ovs-agent-secondary command, for this scenario, is displayed below:
This command can not be run while the ovs-agent is online.
Do you want to disable the ovs-agent service?
1) Yes, disable the ovs-agent service
2) No, exit the ovs-agent-secondary tool
Choice (1-2): 1The list of all PCIe buses present on the server is displayed, with information indicating whether or not they are available for creating a secondary service domain. Example output from the ovs-agent-secondary command, for this step, is displayed below:
Gathering information about the server...
The server has 4 PCIe buses.
----------------------------------------------------------------------
This is the list of PCIe buses present on the server, and whether
or not they are available for creating a secondary service domain
Bus Available Reason
--- --------- ------
pci_0 no Bus is assigned and used by the primary domain
pci_1 yes Bus is assigned to the primary domain but it is not used
pci_2 yes Bus is assigned to the primary domain but it is not used
pci_3 yes Bus is assigned to the primary domain but it is not used
Enter + or - to show or hide details about PCIe buses.
+) Show devices in use
Or select one of the following options.
0) Exit and do not create a secondary service domain
1) Continue and select PCIe buses to create a secondary service domain
Choice (0-1): 1Use this information to figure out which PCIe buses are available, and which buses you want to use for the secondary service domain. You can display more or less information about the PCIe buses by entering "+" or "-".
A PCIe bus is not available for creating a secondary service in the following cases:
The PCIe bus is assigned to a domain other than the primary domain.
If you want to use such a PCIe bus for the secondary service domain then you must first remove it from the domain it is currently assigned to.
The PCIe bus is assigned to the primary domain and devices on that bus are used by the primary domain.
If you want to use such a PCIe bus for the secondary service domain then you must reconfigure the primary domain so that it stops using devices from that bus.
When a PCIe bus is assigned to the primary domain, the tool may not always be able to figure out if devices from the bus are used by the primary domain. Furthermore, the tool only identifies common devices (such as network interfaces and disks) and the common usage of these devices (including link aggregation, IP configuration or ZFS pool). If you want to create a secondary domain with a PCIe bus that is currently assigned to the primary domain, make sure that this bus is effectively not used by the primary domain at all.
The next step provided by the ovs-agent-secondary command allows you to actually select the PCIe buses that are to be used for the secondary service domain. Typically, this step may appear as follows:
The following PCIe buses can be selected for creating a secondary
service domain.
Bus Selected Devices Count
--- -------- -------------
pci_1 no ETH(2) FC(2) IB(2)
pci_2 no ETH(2) FC(2) IB(2)
pci_3 no DSK(2) ETH(2)
Enter + or - to show or hide details about PCIe buses.
+) Show PCIe slots
Or enter the name of one or more buses that you want to add to the
selection of PCIe buses to create a secondary service domain.
Or select one of the following options.
0) Exit and do not create a secondary service domain
1) Add all PCIe buses to the selection
2) Remove all PCIe buses from the selection
Choice (0-2): pci_2 pci_3Note that in addition to the menu options, which allow you to add all available PCIe buses to the secondary service domain, you can also manually specify a space separated list of PCIe buses by bus name to individually add particular buses to the secondary service domain.
As soon as at least one PCIe bus is marked as selected, the menu options change to allow you to create the secondary service domain with the selected PCIe buses:
The following PCIe buses can be selected for creating a secondary
service domain.
Bus Selected Devices Count
--- -------- -------------
pci_1 no ETH(2) FC(2) IB(2)
pci_2 yes ETH(2) FC(2) IB(2)
pci_3 yes DSK(2) ETH(2)
Enter + or - to show or hide details about PCIe buses.
+) Show PCIe slots
Or enter the name of one or more buses that you want to add to the
selection of PCIe buses to create a secondary service domain.
Or select one of the following options.
0) Exit and do not create a secondary service domain
1) Add all PCIe buses to the selection
2) Remove all PCIe buses from the selection
3) Create a secondary services domain with the selected buses
Choice (0-3): 3A final confirmation screen displays the buses selected for the secondary service domain, before you can proceed to create the secondary service domain. This confirmation screen looks as follows:
You have selected the following buses and devices for the secondary
domain.
Bus Current Domain Slot Devices Count
--- -------------- ---- -------------
pci_2 primary
/SYS/MB/PCIE2 IB(2)
/SYS/MB/PCIE3 FC(2)
/SYS/MB/PCIE4 ETH(2)
pci_3 primary
/SYS/MB/PCIE8
/SYS/MB/SASHBA1 DSK(2)
/SYS/MB/NET2 ETH(2)
Verify that the selection is correct.
0) Exit and do not create a secondary service domain
1) The selection is correct, create a secondary domain with pci_2 pci_3
2) Go back to selection menu and change the selection
Choice (0-2): 1After the selection of PCIe buses for the secondary service domain has been confirmed, the secondary domain is created and instructions for configuring the secondary service domain are displayed. The output from the tool looks similar to the following:
ldm create secondary ldm set-core 1 secondary ldm set-memory 8G secondary ldm add-io pci_2 secondary ldm add-io pci_3 secondary ldm start-reconf primary ldm rm-io pci_2 primary ldm rm-io pci_3 primary ---------------------------------------------------------------------- The secondary service domain has been created. Next, you need to install Solaris on that domain. Then you can configure the Oracle VM Agent to run with the secondary domain. Once the secondary service domain is up and running with Solaris, run the following command to configure the Oracle VM Agent to run with the secondary domain: # ovs-agent-secondary configure
If a reboot is required to complete the creation of the secondary service domain then a corresponding menu is displayed, otherwise the tool terminates and the creation of secondary service domain is already finished. The following menu is displayed if a reboot is required:
You need to reboot the system in order to complete the
creation of the secondary domain.
Do you want to reboot the system now?
1) Yes, reboot the system now
2) No, I will reboot the system later
Choice (1-2): 1When you have finished creating the new service domain, you need to install it. Simply follow the instructions already provided in Section 1.1.3.4, “Installing the Secondary Service Domain”.
Once the secondary service domain is correctly installed, you must configure the Oracle VM Agent to use it by running the ovs-agent-secondary command on the control domain, as follows:
# ovs-agent-secondary configure
The ovs-agent-secondary command guides you through a number of configuration steps before the configuration changes are made to the Oracle VM Agent.
The first step in the configuration process requires you to confirm that the secondary domain is installed and running. This step is displayed as follows:
The secondary service domain exists and is active. It should be up
and running Solaris 11.1.
Confirm that the secondary service domain is up and running Solaris 11.1
1) Yes, the secondary service domain is up and running Solaris 11.1.
2) No, the secondary service domain is not running Solaris 11.1
Choice (1-2): 1Each network link in the primary domain should have a corresponding network link in the secondary domain connected to the same physical network. By default, a network link in the primary domain is associated with the network link with the same name in the secondary domain. If a network link in the primary domain should be associated with a network link in the secondary domain with a different name, then you need to define a network link mapping. This is achieved in the next step of the configuration process, which is displayed as follows:
Each network link in the primary domain should have a corresponding
network link in the secondary domain connected to the same physical
network. By default, a network link in the primary domain will be
associated with the network link with the same name in the secondary
domain.
Network links in the primary domain and corresponding link in the
secondary domain:
Primary Secondary
------- ---------
net0 net0
net1 net1
net4 net4
net5 net5
net6 net6
net7 net7
If a network link in the primary domain should be associated with
a network link in the secondary domain with a different name, then
you need to define a network link mapping.
Do you need to define a network link mapping?
1) Yes, I need to map a network link in the primary domain to
a network link in the secondary domain with a different name.
2) No, each network link in the primary domain has a corresponding
network link in the secondary domain with the same name.
Choice (1-2): 1
Ideally, you should be able to select option
2 here to continue. However, it is possible
that network link names may not correspond correctly. In
this case, you should select option 1 and
redefine the mapping as follows:
Enter the mapping for net0 [net0]: Enter the mapping for net1 [net1]: Enter the mapping for net4 [net4]:net2Enter the mapping for net5 [net5]:net3Enter the mapping for net6 [net6]: Enter the mapping for net7 [net7]: Network links in the primary domain and corresponding link in the secondary domain: Primary Secondary ------- --------- net0 net0 net1 net1 net4 net2 net5 net3 net6 net6 net7 net7 Is the mapping correct? 1) Yes, the mapping is correct. 2) No, the mapping is not correct, redo the mapping. Choice (1-2):1
Note that you are prompted for the mapping for each network link in the primary domain. If you enter a blank line, the existing default mapping is used. If you need to change a mapping, you must specify the network link name in the secondary domain that is connected to the same physical network as the network link listed in the primary domain.
When you have finished redefining the mappings, you should
select option 1 to continue to the next
step in the configuration process.
Each Fibre Channel (FC) disk accessible from the primary domain domain should also be accessible from the secondary domain. By default, a FC disk is accessed using the same device path in the primary domain and in the secondary domain. In particular, each disk is accessed using the same disk controller name. If a disk controller in the primary domain should be associated with a disk controller in the secondary domain with a different name, then you must define a disk controller mapping.
It is recommended that Solaris I/O multipathing is enabled in the primary and in the secondary domain on all multipath-capable controller ports, in particular on all FC ports. In this case, all FC disks appear under a single disk controller (usually c0), and disk controller mapping is usually not needed.
The following screen is displayed for this step in the configuration process:
Each Fibre Channel (FC) disk accessible from the primary domain
domain should also be accessible from the secondary domain. By
default, a FC disk will be access using the same device path in
the primary domain and in the secondary domain. In particular,
each disk will be accessed using the same disk controller name.
FC disk controllers in the primary domain and corresponding
controller in the secondary domain:
Primary Secondary
------- ---------
c0 c0
If a disk controller in the primary domain should be associated with
a disk controller in the secondary domain with a different name, then
you need to define a disk controller mapping.
Do you need to define a disk controller mapping?
1) Yes, I need to map a disk controller in the primary domain to
a disk controller in the secondary domain with a different name.
2) No, each disk controller in the primary domain has a corresponding
disk controller in the secondary domain with the same name.
Choice (1-2): 1
Ideally, you should be able to select option
2 here to continue. However, it is possible
that disk controller names may not correspond correctly. In
this case, you should select option 1 and
redefine the mapping as follows:
Enter the mapping for c0 [c0]: c1
FC disk controllers in the primary domain and corresponding
controller in the secondary domain:
Primary Secondary
------- ---------
c0 c1
Is the mapping correct?
1) Yes, the mapping is correct.
2) No, the mapping is not correct, redo the mapping.
Choice (1-2): 1Note that you are prompted for the mapping for each FC disk controller in the primary domain. If you enter a blank line, the existing default mapping is used. If you need to change a mapping, you must specify the FC disk controller name in the secondary domain that is connected to the same FC disk listed in the primary domain.
When you have finished redefining the mappings, you should
select option 1 to continue to the next
step in the configuration process.
The Oracle VM Agent uses a configuration file to be able to access and configure itself for resources in the secondary service domain. In this step of the configuration process, the configuration file is created and saved to disk within the primary control domain:
Creating configuration file Saving configuration ovm-shadow on the service processor The secondary service domain is configured. Continuing with the configuration of the Oracle VM Agent.
Finally, the Oracle VM Agent is automatically reconfigured to use the secondary service domain and the Oracle VM Agent is enabled:
Network Configuration Network Configuration OK Storage Configuration Storage Configuration OK OVS Agent Configuration OVS Agent Configuration OK Cluster Configuration Cluster Configuration OK LDoms Manager Configuration LDoms Manager Configuration OK Virtual I/O Services Configuration Virtual I/O Services Configuration OK LDoms Configuration *** ERROR: The vds service is not configured on the secondary domain. Configuring the vds service Saving configuration ovm-shadow on the service processor LDoms Configuration OK The LDoms configuration has been dynamically updated and stored on the service processor. Enabling Oracle VM Agent Services
The configuration values that are used for this process map onto the values that you entered for the configuration steps when you first configured Oracle VM Agent for your primary control domain, as described in Configuring Oracle VM Agent for SPARC in the Oracle VM Installation and Upgrade Guide.
When the process is complete, the Oracle VM Agent is enabled and your environment is configured to use both a primary and a secondary service domain.