CPU Allocation

When you run threads from the same core in separate domains, you might experience unpredictable and poor performance. The Oracle VM Server for SPARC software uses the CPU affinity feature to optimize CPU allocation during the logical domain binding process, which occurs before you can start the domain. This feature attempts to keep threads from the same core allocated to the same logical domain because this type of allocation improves cache sharing between the threads within the same core.

CPU affinity attempts to avoid the sharing of cores among domains unless there is no other recourse. When a domain has been allocated a partial core and requests more strands, the strands from the partial core are bound first, and then another free core is located to fulfill the request, if necessary.

The CPU allocation mechanism uses the following constraints for CPU resources:

• Whole-core constraint. This constraint specifies that CPU cores are allocated to a domain rather than virtual CPUs. As long as the domain does not have the max-cores constraint enabled, the whole-core constraint can be added or removed by using the ldm set-core or ldm set-vcpu command, respectively. The domain can be inactive, bound, or active. However, there must be enough cores available to satisfy the request to apply the constraint. As a worst-case example, if a domain that shares cores with another domain requests the whole-core constraint, cores from the free list would need to be available to satisfy the request. As a best-case example, all the virtual CPUs in the core are already on core boundaries, so the constraint is applied without changes to CPU resources.

• Maximum number of cores (max-cores) constraint. This constraint specifies the maximum number of cores that can be assigned to a bound or active domain.

How to Apply the Whole-Core Constraint

It is best to ensure that the domain has the whole-core constraint enabled prior to setting the max-cores constraint.

1. Apply the whole-core constraint on the domain.

   # ldm set-core 1 domain

2. Verify that the domain has the whole-core constraint enabled.

   # ldm ls -o resmgmt domain

   Notice that max-cores is set to unlimited. The domain cannot be used in conjunction with hard partitioning until the max-cores constraint is enabled.

Example 10-1 Applying the Whole-Core Constraint

This example shows how to apply the whole-core constraint on the ldg1 domain. The first command applies the constraint, while the second command verifies that it is enabled:

# ldm set-core 1 ldg1
# ldm ls -o resmgmt ldg1
NAME
ldg1

CONSTRAINT
    cpu=whole-core
    max-cores=unlimited
    threading=max-throughput

How to Apply the Max-Cores Constraint

It is best to ensure that the domain has the whole-core constraint enabled prior to setting the max-cores constraint.

You can only enable, modify, or disable the max-cores constraint on an inactive domain, not on a domain that is bound or active. Before you update the max-cores constraint on the control domain, you must first initiate a delayed reconfiguration.

1. Enable the max-cores constraint on the domain.

   # ldm set-domain max-cores=max-number-of-CPU-cores domain

   ---

   Note - The cryptographic units that are associated with those cores are unaffected by core additions. So, the system does not automatically add the associated cryptographic units to the domain. However, a cryptographic unit is automatically removed only when the last virtual CPU of the core is being removed. This action prevents a cryptographic unit from being “orphaned.”

   ---

2. Verify that the whole-core constraint is enabled.

   # ldm ls -o resmgmt domain

3. Bind and restart the domain.

   # ldm bind domain
   # ldm start domain

   Now, you can use the domain with hard partitioning.

Example 10-2 Applying the Max-Cores Constraint

This example shows how to constrain max-cores to three cores by setting the max-cores property, and verifying that the constraint is enabled:

# ldm set-domain max-cores=3 ldg1
# ldm ls -o resmgmt ldg1
NAME
ldg1

CONSTRAINT
    cpu=whole-core
    max-cores=3
    threading=max-throughput

Now, you can use the domain with hard partitioning.

The following example removes the max-cores constraint from the unbound and inactive ldg1 domain, but leaves the whole-core constraint as-is:

# ldm stop ldg1
# ldm unbind ldg1
# ldm set-domain max-cores=unlimited ldg1

Alternately, to remove both the max-cores constraint and the whole-core constraint from the ldg1 domain, assign virtual CPUs instead of cores, as follows:

# ldm set-vcpu 8 ldg1

In either case, bind and restart the domain.

# ldm bind ldg1
# ldm start ldg1

Interactions Between the Whole-Core Constraint and Other Domain Features

This section describes the interactions between the whole-core constraint and the following features:

• CPU Dynamic Reconfiguration

• Dynamic Resource Management

• Domain Migration

• Power Management

CPU Dynamic Reconfiguration

The whole-core constraint is fully compatible with CPU dynamic reconfiguration (DR). When a domain is defined with the whole-core constraint, you can use the ldm add-core, ldm set-core, or ldm remove-core command to change the number of cores on an active domain.

However, if a bound or active domain is not in delayed reconfiguration mode, its number of cores cannot exceed the maximum number of cores. This maximum is set with the maximum core constraint, which is automatically enabled when the whole-core constraint is enabled. Any CPU DR operation that does not satisfy the maximum core constraint fails.

Dynamic Resource Management

The whole-core constraint is not compatible with dynamic resource management (DRM). When a DRM policy is enabled on a domain that uses the whole-core constraint, the policy is automatically disabled. The whole-core constraint remains enabled.

Even though a DRM policy cannot be enabled when the whole-core constraint is in effect, you can still define a DRM policy for the domain. Note that when a policy is automatically disabled, it still remains active. The policy is automatically re-enabled if the domain is restarted without the whole-core constraint.

The following are the expected interactions between the whole-core constraint and DRM:

• If the whole-core constraint is set on a domain, a warning message is issued when you attempt to enable a DRM policy on that domain.

• If a DRM policy is in effect on an inactive domain, you are permitted to enable the whole-core constraint on the domain. When the domain becomes active and the policy is enabled, the system automatically disables the DRM policy for the domain.

• If a DRM policy is enabled on an active or bound domain, you are not permitted to enable the whole-core constraint.

Domain Migration

If you migrate a domain that is configured with whole cores, you must reconfigure the target domain to use hard partitioning after the migration completes. Also, you must ensure that your license agreement permits you to use the domain on both the source and target systems.

Power Management

The whole-core constraint is fully compatible with the power management (PM) performance and elastic policies. When the elastic policy is in effect, the PM subsystem can add or remove CPU cores to or from domains that are configured with the whole-core constraint. In that case, the whole-core constraint continues to be honored, and domains that use that constraint remain configured only with whole cores.